Purpose of this article is to setup an Rclone transfer process on a Linux compute node that transfers data between a Filespace and a cloud storage provider continuously to a predetermined scheduled interval.

Rclone supports many of the most common cloud storage providers. Utilizing this process you can configure a virtual or physical machine, on-premises or within the cloud to perform the transfer.

Please note that depending on where your Filespace is hosted, where you locate your compute node, along with which destination cloud provider selected to maintain the backup, you many incur egress and/or Internet service provider transfer or network fees. 

Employing `rclone sync` will allow you to purge the destination and keep your backup in lockstep with your Filespace. Your cloud provider might provide a versioning option (typically object storage) which enables you to maintain multiple versions of your data, in-line with your transfer schedule.

This process will complement our Filespace snapshots with granular data points in time within the backup, retaining data separately to the live Filespace content and snapshots. 

Follow the following steps to prepare your compute node and configure your systemd services, transfer script, and systemd timer to trigger the backup process.

Install Lucid and Rclone

apt-get update; wget https://www.lucidlink.com/download/latest/lin64/stable/ -O lucidinstaller.deb; apt-get install ./lucidinstaller.deb -y; rm ./lucidinstaller.deb
sudo -v ; curl https://rclone.org/install.sh | sudo bash

Configure Rclone and setup your storage provide as preferred

rclone config

Create lucidlink.service systemd

nano /etc/systemd/system/lucidlink.service

[Unit]
Description=LucidLink Filespace Daemon 
After=network-online.target
[Service]
Type=simple
#User=<user>
#Group=<group>
#WorkingDirectory=/home/<user>
ExecStart=/usr/bin/lucid daemon --fs <llfilespace> --user <lluser> --password <llpassword> --mount-point <llmount> --fuse-allow-other
ExecStop=/usr/bin/lucid exit
Restart=on-abort
[Install]
WantedBy=multi-user.target

Edit fuse.conf and uncomment (remove #)

sudo nano /etc/fuse.conf
`user_allow_other`

Reload systemd daemon

systemctl daemon-reload

Enable LucidLink daemon systemd service

systemctl enable lucidlink.service

Start LucidLink daemon systemd service

systemctl start lucidlink.service

Check LucidLink systemd service

systemctl status lucidlink.service

Rclone transfer script to `sync` (or `copy`) from source to destination.

nano /usr/local/bin/rclone-script.sh

#!/bin/bash

if [[ "`pidof -x $(basename $0) -o %PPID`" ]]; then
echo "rclone-script.sh is already running"
exit;
fi

rclone sync -v --fast-list --checkers 30 --transfers 30 <source> <destination>

Make script executable

chmod +x /usr/local/bin/rclone-script.sh

Create a systemd service

nano /etc/systemd/system/rclone.service

[Unit]
Description=Rclone Transfer Service

[Service]
Type=simple
ExecStart=/usr/local/bin/rclone-script.sh

Create Rclone systemd timer https://www.freedesktop.org/software/systemd/man/systemd.timer.html#

nano /etc/systemd/system/rclone.timer

[Unit]
Description=Rclone Transfer Timer

[Timer]
Unit=rclone.service
# Run 15 minutes after boot, since the timer must run at least once
# before OnUnitInactiveSec will trigger
OnBootSec=15m
# Run 15 minutes after rclone.service last finished
OnUnitInactiveSec=15m
# Run once when the timer is first started
OnActiveSec=1s

[Install]
WantedBy=timers.target

Reload systemd daemon

systemctl daemon-reload

Enable Rclone timer

systemctl enable rclone.timer

Start Rclone timer

systemctl start rclone.timer

Manually trigger Rclone transfer

systemctl start rclone.service

Reach out to LucidLink support should you require any assistance.

While rclone sync is an option, you may also consider using rclone copy along with some parameters which have shown performance improvements when writing to a filespace (filespace as the destination).

--multi-thread-streams 4 --multi-thread-cutoff 1Mi --multi-thread-write-buffer-size 4Mi

Kudos to https://codingnotions.com/fully-automated-backup-rclone/