vendredi 22 juillet 2022

Encrypt files in background while using the application (Design Problem)

I've to design a functionality which encrypts and decrypts files(around 30) based on user input.

During the encryption, I want the user to be still able to use the system just like before. (which means that files can be updated during the encryption process)

I'm looking to find the best approach to implement it. There are few different approaches (that are still incomplete) which I want to discuss.

Simple Approach:

  • Start encrypting 30 files in parallel.
  • If user updates one of the file, the change will be missed in the encrypted file as the change was not there when encryption was called.

Cons: Dirty read = data loss. (failed)

2nd Approach:

  • save all operations that can change the files (difficult to achieve).
  • wait for encryption to complete.
  • when encryption completes, perform all those operations manually.

Cons: User updates are not reflected during the process, as we're only saving operations and not performing them unless process is completed.

3rd Approach:

  • When encryption starts:
  1. use current files as "old".
  2. current system works fine as old (updates saved in old files if any change)
  3. start encrypting files and save them with different name.
  • when all files are encrypted, there are two cases:
  1. No file was updated during process (best case). -> replace encrypted files name and delete old files.
  2. Files changed. Find out which files change(easy to check) and only encrypts those again. (with the hope user doesn't update again, if he does keep repeating the process unless he stops)

Cons: Can end up in an infinite loop if user doesn't stop updating files.

Pros: User can use the system just like before and data won't be lost.

I'd love to read better approaches from you guys.

I'll be using AWS KMS to encrypt S3 files inside lambda function.

Aucun commentaire:

Enregistrer un commentaire