Owncloud Download Synchronizer

About

This is a little tool to synchronize files from a public link share to a local directory. Features:

• only download, cannot upload
• update files based on timestamp of local and upstream files
• include / exclude glob patterns to select files to download
• ignore glob patterns to select local files to keep
• logging to screen or file
• dry-run mode

Basic Usage

Synchronize a public share link to a local directory:

python oc_downsync.py https://owncloud.example.com/s/aSdFgHjKl local-dir/

With password:

python oc_downsync.py https://owncloud.example.com/s/aSdFgHjKl --password=123456 local-dir/

Only download *.md files (and delete other files, if synchronized to local-dir before):

python oc_downsync.py https://owncloud.example.com/s/aSdFgHjKl local-dir/ --include='*.md'

Download all files except *.md files, but do download /README.md (and delete other *.md files, if synchronized to local-dir before):

python oc_downsync.py https://owncloud.example.com/s/aSdFgHjKl local-dir/ --exclude='*.md' --include='/README.md'

Keep all files in local directory /bin and subdirectories although it is not in the public share (note: a path starting with / in the exclude, include and ignore patterns refers to the shared folder root, similar to a gitignore file):

python oc_downsync.py https://owncloud.example.com/s/aSdFgHjKl local-dir/ --ignore='/bin/*'

See actions before applying them:

python oc_downsync.py https://owncloud.example.com/s/aSdFgHjKl local-dir/ --log-to-screen --dry-run

Periodic Execution

To make a systemd user timer that synchronizes a link periodicly, follow these instructions.

1. Make the systemd user directory

   mkdir -p ~/.config/systemd/user/

2. Create a systemd user service file like ~/.config/systemd/user/oc-sync.service with e. g. the following contents. Here we assume you use a conda environment myenv with the packages installed listed in requirements.txt. If you use some other way of managing the environment, write a wrapper script to activate it and execute oc_downsync.py. The spcifier %h expands to the user's $HOME variable.

   [Unit]
   Description=OC Downsync
   
   [Service]
   Type=oneshot
   ExecStart=/opt/conda/bin/conda run -n myenv python \
       %h/oc-downsync/oc_downsync.py \
           --log-file=%h/sync.log \
           # source \
           https://owncloud.example.com/s/aSdFgHjKl \
           --include='*.csv' \
           # destination \
           %h/datasets/

3. Create a systemd user timer file like ~/.config/systemd/user/oc-sync.timer with e. g. the following contents (see Calendar Events, also try systemd-analyze calendar "*-*-* *:00/5:00"):

   [Unit]
   Description=Timer for Sync (every five minutes)
   
   [Timer]
   OnCalendar=*-*-* *:00/5:00
   AccuracySec=1s
   Unit=oc-sync.service
   
   [Install]
   WantedBy=default.target

4. Reload user files and enable service:

   # reload service and timer files (do this every time a file changes!)
   systemctl --user daemon-reload
   # enable the timer
   systemctl --user enable --now oc-sync.timer

   And check whether it works:

   # check that timer is activated
   systemctl --user list-timers
   # check status of the service
   systemctl --user status oc-sync.service
   # check logs of the service
   journalctl --user -u oc-sync.service