Drawbacks of using singleton with models

So let's say I'm making a React Redux app for handling a library. I want to create an API for my backend where each model (book, author, etc) is displayed in the UI. Each model does not provide a public constructor, but a from static function which ensures that only one instance per id exists:

static from (id: string) {
  if (Books.books[id]) {
    return Books.books[id];
  }
  return Book.books[id] = new Book(id);
}

Each model provides an async fetch function which will fetch its props using the backend. The advantage is that there is no thousands instances, also I don't have to fetch twice (if two parts of my app needs the same model, fetch will actually be called only once). But I fail to find any drawbacks, except that there might be a discrepancy between a code that fetches its models and one that assumes they are still not fetched, but I still don't see when it would really be an issue