Reusable model in repository pattern

I've recently come across the problem of having multiple database types which should be swappable. My solution for this would be the repository pattern. Having models like these.

class Book {
    public string Title { get; set; }
    public virtual Author Author { get; set; }
}
class Author {
    public string Name { get; set; }
    public virtual ICollection<Book> Books { get; set; }
}

Having these two classes as my model. And the following method in my repository.

class AuthorRepository {
    IEnumerable<Author> GetAll() {
        return Context.Set<Author>().ToList();
    }
}

Now I've got a few problems. First would be that using the repository like this.

using(var unitOfWork = new UnitOfWork(new MyContext())) {
    MyObservableCollection = new ObservableCollection<Author>(unitOfWork.Authors.GetAll());
}

If I were to try and access the books inside the author model I would get a ObjectDisposedException. Which is obvious since books can only be accessed inside of the DbContext so the property should really only be used inside of the repository and not outside.

Now my second issue is that when I want to change from entity framework to another persistence framework the virtual methods would not work since (again as far as I am aware) this is only used in entity framework.

The setup shown above is how I've seen the repository pattern implemented just about everywhere, but I don't see the use in the pattern when I need to change my model whenever I want to change the persistence framework.

My fix would be the following.

class Author {
    public string Name { get; set; }
}
class EntityFrameworkAuthor : Author {
    public virtual ICollection<Book> Books { get; set; }
}

The EF author would only be used in the repositories and the Author would be returned to the business layer.

Now to my questions.

1. Is the method shown above the right way to use the repository pattern if I want to be able to switch frameworks easily (which I assumed the repository pattern was for).
2. Is my fix a good way to improve my current model? Or does it break the pattern in some way?
3. If not how would I go about making my model reusable for different persistence frameworks.