Consumer-producer: Pausing the consumer if memory usage goes beyond a particular threshold

I have a situation where my main thread (producer) allocates a huge chunk of memory on heap for a task, does some work on that buffer and then provides the buffer to worker threads (consumers) for further processing(which will first compress that data and then write it to disk). Once the worker thread is done with it's job, it releases memory that was acquired by the producer for the task.

However there can be a situation where my main thread allocates too much of memory and thus my system starts swapping out other programs to disk to accommodate the memory requirement. Since the disk becomes busy the worker threads find it difficult to write on disk (and eventually free any memory) and meanwhile the producers continues to allocate more memory for other tasks. This in the end kills my system's performance.

What can be a good design for this problem?

Additionally, if pausing the main thread by pre-computing the memory requirement, in advance, is an option how can I come to a reliable number?