I have been using Syncthing to sync most of my home directory. It doesn’t have a rigorous change management system and sometimes I wish for a different result after a sync.
Suppose I set up an inotify daemon to record every change. Can I run a Pijul server as an automatic file sync service? Do you foresee any problems with this setup? For example about a million files, including some multi-gigabyte binaries.
The complexity is in the number of edits, so that would probably be ok.
One issue is that there is no binary diff for now (I haven’t had time to write one), so if you edit these giant files, you’ll end up storing every version of it.
Other than that, you should be good. I’d be interested in feedback.
Cool. If I use it now, and the binary diff feature is added later, my storage will be reduced then?
You could reduce it by not storing the old versions. But you (obviously?) won’t be able to get back to these old versions.
Another issue with the inotify daemon is that you could end up with lots of tiny changes, depending on how you do it. This will become a problem with the current Pijul, but it won’t be that big of a problem in a few weeks.
I just tried recording a 135 GiB directory tree with mixed file types. It took 96 seconds with Ryzen 9, PCIe4 and a hot cache. It put a single core up to 100%. I wonder if parallelism is possible on record.
Some parallelism could be possible indeed, although I’m not sure of where these 96 seconds are spent (it’s possible that a good share of it is in compressing the patch). There’s also a big improvement in performance coming up in a few days.