mercredi 27 mai 2015

event driven pattern for writing chunks to a file - JS

I'm trying to transfer a file over WebRTC, and I'm struggling to figure out a good pattern for writing data as it's coming in. Since file chunks will be coming in at an unknown rate, I need to be able to write each chunk as it becomes available; this means two things:

  1. if data is coming in too fast, then we need to queue each chunk to be written later after the current write task is finished
  2. if data is coming in too slow, then we need to wait for a chunk to become available

Ideally, I'd like to avoid having to rely on setTimeout() for waiting on chunks to arrive. What I have so far is close, but not exactly what I'm looking for:

// container to hold chunks as they come in
var chunkCollection = {};

// callback function for RTCDataChannel messages
function onData(data) {

  chunkCollection[data.chunkIndex] = data.chunk;
  writeToFile(data.chunkIndex);
}

function writeToFile(chunkIndexToWrite) {

  // if we have the next chunk, write it to the file
  if (chunkCollection[chunkIndexToWrite]) {
    var chunk = chunkCollection[chunkIndexToWrite];
    delete chunkCollection[chunkIndexToWrite];
    fileWriter.seek(chunk.offset);
    fileWriter.write(chunk.blob);
  }
  else {
    // we don't have the next chunk, so we have to wait
    setTimeout(function() {
      writeToFile(chunkIndexToWrite);
    }, 100);
  }
}

The problem with this is that if chunks come in too fast, fileWriter will not be ready to write the next chunk and will throw an exception. However, if chunks come in too slowly, setting the correct timeout will be extremely tricky.

It seems like an event-driven approach will work best here. One event is the data message coming in from RTCDataChannel. Another event is that fileWriter has finished writing, and is ready to write the next chunk. But the piece that I'm getting stuck on is how to properly wait for a chunk to come in...

If the browser is not busy writing to the file, and is just sitting waiting for a chunk to come in, then the browser should start writing the chunk as soon as the chunk is available. But I can't figure out how to do this without an ugly setTimeout loop; I can't figure out how to publish an event that says we don't need to wait anymore, and can continue writing.

What's a good pattern for doing this?

Aucun commentaire:

Enregistrer un commentaire