Providing access to different data stores with the Repository Pattern

I've been trying to learn how the service layer and repository pattern work. So far I've written a service layer and a simple repository pattern interface. However, I often see articles stating that the repository pattern allows for being able to swap in and out different data stores without the consuming code having to be changed.

In my case I want to be able to support reading and writing the application data to CSV and/or XML files. This is the part where I do not understand how to properly implement this with the repository pattern. Should I have a repository per data store?

ProductCsvRepository : IProductRepository
ProductXmlRepository : IProductRepository

However if I do this then the service layer would have to be aware of the underlying data store, which breaks the idea of being able to easily swap in and out different data stores.

Would I then have to have a service layer that looks like this?

private readonly IProductXmlRepository _productXmlRepository;
private readonly IProductCsvRepository _productCsvRepository;

public ProductService()
{
    _productXmlRepository = new IProductXmlRepository();
    _productCsvRepository = new IProductCsvRepository();
}

public ICollection<Product> GetAllXml()
{
    return _productXmlRepository.GetAllCsv();
}

public ICollection<Product> GetAll()
{
    return _productCsvRepository.GetAllXml();
}

This then raises two questions:

• Surely this then breaks the idea of the consuming code needing to know what the data store is?
• What about in the situations where the consuming code does need to know about the data store, such as for "File > Export As" type functionality? Should export functionality actually be a different service that utilises the appropriate CSV or XML service?

I think I am definetly not understanding how to correctly implement a repository patten and a service layer. How should I actually design repository patten and service layer?