Migrating your Filespace S3 bucket to a different provider or region is achieved through reconfiguring your object storage settings for your Filespace with the LucidLink Client CLI move-s3 command.
Benefits of move-s3 include migrating large amounts of data from on-premises to a Filespace through vendor-provided services such as AWS Snowball or an intermediary S3-compatible appliance which can be transferred to the final object-store.
In this migration scenario, your Filespace would be configured with an on-premises endpoint:port with data ingested into the Filespace securely through the LucidLink Client in an encrypted manner, therefore protected during 3rd-party transfer.
When transitioning Filespaces between S3 providers or across regions, you are required to move all bucket data manually. Moving data could come in the form of a source bucket object copy or sync to the destination bucket (AWS CLI, s4cmd, MinIO Client or Rclone) however requires careful consideration of egress or other traffic-related transfer costs.
In certain circumstances data transfers could be performed through cloud provider Direct Connects, ExpressRoutes or Content Delivery Network (CDN) to attain a lower outbound data rate.
Note: native Bucket replication might be an option although depending on the vendor, often relies on versioning, and in certain implementations does not replicate deletes or delete markers
Usage: Lucid move-s3 --bucket-name name --access-key key --secret-key key --provider name [options] Reconfigure S3 object store connection settings for current filespace. Caution - read https://lucidlink.com/kb-move-s3/ before usage. Options: --password password Root user's password is required to use move-s3 --bucket-name name Use a specific bucket name. Make sure new bucket is an exact copy of the current bucket (see lucid status). (required) --http Use HTTP --https Use HTTPS --access-key key S3 access key (required) --secret-key key S3 secret key (required) --provider name Specify the name of the S3 compatible provider (AWS, DigitalOcean, ...) (required) --location-constraint Set a location constraint for the bucket. Required by some providers (AWS, GCP, ...). --region name Specify S3 region --endpoint name|IP:port Specify S3 compatible endpoint --virtual-addressing Use virtual hosted-style bucket addressing instead of path-style.
1. Identify bucket-name
lucid status
2. In our case, we will be moving between Wasabi Amsterdam to Amazon S3 Sydney. Execute move-s3
lucid move-s3 --bucket-name dd3b2831-2f34-411e-8de4-fbe73eef337c --access-key <access-key> --secret-key <secret-key> --provider AWS --region ap-southeast-2
3. Upon successful execution, you will receive the notification Daemon move S3 request sent.
4. Use lucid status once again to confirm your Filespace status reflects your move-s3 request.
5. Perform external bucket contents move. Your Filespace is now operating out of its new object storage provider or region.
Filespace data accessibility contingent on the objects being available in the current object store bucket at the time of any client data request.
Move object storage provider example for an endpoint:
lucid move-s3 --endpoint <url:port> --https --bucket-name <bucket> --region <region> --access-key <access-key --secret-key <secret-key> --provider <provider>