Best way of handling two endpoints that depend on each other

I have two endpoints. The first endpoint generates a png image with matplotlib. The second endpoint generates a list of points (x, y coordinates in pixels). The frontend will then display the image as well as the x, y points and the user will be able to hover over the x,y points and view data.

The setup is:

1. /endpoint1 -> returns svg/png
2. /endpoint2 -> returns something like

{
   name1: {
     coords: [203, 3204],
     other_field: true,
   },
   name2: {
     coords: [509, 320],
     other_field: false,
   }
}

The issue is that the x, y points shift depending on the requested parameters of the image. For example, if the user requests a different image layer of the same image, the image ends up being shifted a bit and the points change. To generate the points, I would need to replicate a lot of the work to generate the image. Therefore, it makes a lot of sense to just return the points and image together. However, this is a lot of work for one endpoint and I'm wondering if there is a better way to handle this or if this design is considered good/okay?