DESIGN PROBLEM: How to use composition the best

I'm working on a project and have some doubts about it's design. How can I design the following problem the best (in JAVA):

Class A with the following attributes:

• HashSet of Pixels where each pixel has x,y coordinates and value v between 0-1.
• instance of class B.

Class B with the following function:

• a function that gets a Pixel and returns its left neighbor.

When I'm in class A I want to use B.function on each pixel in A and add it to the HashSet only if it's not already there. The problem is that I don't want to send the HashSet to the function, how bad is it to return new instance of Pixel from the function if it might already exist (This function going to run on many pixels and will create many unused instances of Pixel).

What other options do I have?