How to trade space for time, when there are too many subjects and a few observers?

http://ift.tt/1MJHMwb

says

The simplest way for a subject to keep track of the observers it should notify is to store references to them explicitly in the subject. However, such storage may be too expensive when there are many subjects and few observers. One solution is to trade space for time by using an associative look-up (e.g., a hash table) to maintain the subject-to-observer mapping. Thus a subject with no observers does not incur storage overhead. On the other hand, this approach increases the cost of accessing the observers

How does it save space?

The subject has still a pointer to the map.

And the map has references to all the observers of registered subjects.

The solution proposed in the book uses more space.

Am I wrong?