Rclone systemd timer transfer script

  • Updated

Purpose of this article is to setup an Rclone transfer process on a Linux compute node that transfers data between a Filespace and a cloud storage provider continuously to a predetermined scheduled interval.

Rclone supports many of the most common cloud storage providers. Utilizing this process you can configure a virtual or physical machine, on-premises or within the cloud to perform the transfer.

Please note that depending on where your Filespace is hosted, where you locate your compute node, along with which destination cloud provider selected to maintain the backup, you many incur egress and/or Internet service provider transfer or network fees. 

Employing `rclone sync` will allow you to purge the destination and keep your backup in lockstep with your Filespace. Your cloud provider might provide a versioning option (typically object storage) which enables you to maintain multiple versions of your data, in-line with your transfer schedule.

This process will complement our Filespace snapshots with granular data points in time within the backup, retaining data separately to the live Filespace content and snapshots. 

Follow the following steps to prepare your compute node and configure your systemd services, transfer script, and systemd timer to trigger the backup process.

Install Lucid and Rclone

apt-get update; wget https://www.lucidlink.com/download/latest/lin64/stable/ -O lucidinstaller.deb; apt-get install ./lucidinstaller.deb -y; rm ./lucidinstaller.deb
sudo -v ; curl https://rclone.org/install.sh | sudo bash

Configure Rclone and setup your storage provide as preferred

rclone config

Create lucidlink.service systemd

nano /etc/systemd/system/lucidlink.service

Description=LucidLink Filespace Daemon
ExecStart=/usr/bin/lucid daemon --fs <llfilespace> --user <lluser> --password <llpassword> --mount-point <llmount> --fuse-allow-other
ExecStop=/usr/bin/lucid exit

Edit fuse.conf and uncomment (remove #)

sudo nano /etc/fuse.conf

Reload systemd daemon

systemctl daemon-reload

Enable LucidLink daemon systemd service

systemctl enable lucidlink.service

Start LucidLink daemon systemd service

systemctl start lucidlink.service

Check LucidLink systemd service

systemctl status lucidlink.service

Rclone transfer script to `sync` (or `copy`) from source to destination.

nano /usr/local/bin/rclone-script.sh


if [[ "`pidof -x $(basename $0) -o %PPID`" ]]; then
echo "rclone-script.sh is already running"

rclone sync -v --fast-list --checkers 30 --transfers 30 <source> <destination>

Make script executable

chmod +x /usr/local/bin/rclone-script.sh

Create a systemd service

nano /etc/systemd/system/rclone.service

Description=Rclone Transfer Service


Create Rclone systemd timer https://www.freedesktop.org/software/systemd/man/systemd.timer.html#

nano /etc/systemd/system/rclone.timer

Description=Rclone Transfer Timer

# Run 15 minutes after boot, since the timer must run at least once
# before OnUnitInactiveSec will trigger
# Run 15 minutes after rclone.service last finished
# Run once when the timer is first started


Reload systemd daemon

systemctl daemon-reload

Enable Rclone timer

systemctl enable rclone.timer

Start Rclone timer

systemctl start rclone.timer

Manually trigger Rclone transfer

systemctl start rclone.service

Reach out to LucidLink support should you require any assistance. 


Kudos to https://codingnotions.com/fully-automated-backup-rclone/

Was this article helpful?

0 out of 0 found this helpful