How to forbid using a single Java class

I am currently using the factory pattern, which would call the JXErrorPane class. I want to make sure that no other class can call JXErrorPane directly. I tried the JDepend library, but it didn't work as I hoped as it only works for packages but there are other classes in the package which I don't want to move.

Is there a way to forbid other classes call JXErrorPane class except the factory class? Can this be done by configuring in IDE such as Eclipse or an explicit test is needed?

Thanks.