LucidLink Connect Overview

  • Updated

LucidLink Connect enables the connection of external data stores (such as S3 object storage) and the linking of stored assets directly within the filespace without the need to first download or copy the data into the filespace. Linked assets can then be streamed directly from any compatible LucidLink client (desktop, mobile, web) in the same manner as data added through conventional means.

Linking of assets via LucidLink Connect is achieved using the LucidLink API, allowing integration and automation opportunities for existing data in its native form. This may be from a standalone bucket, or from storage being managed by another service or application. Multiple data sources from multiple different providers can be linked in a single filespace.   

Only linking is performed via the API - data access itself is achieved through LucidLink client applications. Linking via CLI will be enabled in a future release.

LucidLink Connect is a purchased add-on available only to LucidLink Enterprise plans.

Key Benefits and Use Cases:

  • Access and stream existing external data (such as cloud object storage) instantly without duplication or copying
  • Integrate with platforms and tools backed by cloud object storage

Supported Access Methods and Storages

Storage/Access Type Support Status
S3-Compatible Storage / APIs

Supported
HTTP(S)-based URLs

Supported (client version 3.7.8703 and up / API container version 1.4.2 and up)
Azure Blob Storage

Planned

How It Works

LucidLink Connect provides two methods for linking external data:

  • LucidLink-managed data stores and external entries
  • Direct linking of external entries with HTTP(S)-based URLs

Key Terms

1. Data Stores

A Data Store contains the connection information and credentials needed to generate the pre-signed URLs that clients will use to stream data. The LucidLink hub manages the data store and the refresh lifecycle of any pre-signed URLs generated with it automatically. Each data store is:

  • Filespace-specific (not shared across filespaces)
  • Encrypted with both hub and admin keys for security
  • Configured once and can be reused for multiple files

Multiple data stores with multiple different storage providers can be configured per filespace (and even per provider if needed, for instance to configure multiple buckets or different sets of credentials scoped for different prefixes).

2. External Entries

An External Entry is a record in your filespace that points to an object or file in your cloud storage. Once linked, you can:

  • Instantly view and stream the file using normal filesystem operations
  • Access the file from any Lucid client (Web, Desktop, Mobile)

If an external entry is linked by:

  • providing a data store ID and object ID using the "Single Object File" type -> its refresh lifecycle is managed by the LucidLink hub automatically and no further action is required after linking
  • providing an HTTP(S)-based file URL using the "HTTP FIle" type -> its refresh lifecycle is not managed by the LucidLink hub and the user (or a third party automation configured by the user or administrator) is required to periodically obtain and update new file URLs to maintain access

The Quickstart guide below assumes you are using LucidLink-managed data stores. For more information about configuring LucidLink Integrations using the HTTP(S)-based file URL method, see this article. Descriptions of how the Connect feature functions apply to both methods unless otherwise stated.

Quickstart Guide

Prerequisites

  • LucidLink Enterprise workspace
  • Filespace version 3.6 or above
  • Filespace admin privileges
  • Service account and bearer token configured for use with LucidLink API
  • LucidLink Connect enabled for your filespace (contact sales)
  • Object storage bucket and access credentials for said bucket with GetObject permissions
  • LucidLink API container

Step 1: Create a Data Store

First, configure access to your S3 bucket by creating a data store:

POST /filespaces/{filespaceId}/external/data-stores
{
  "name": "my-s3-bucket",
  "kind": "S3DataStore",
  "s3StorageParams": {
    "accessKey": "AKIAIOSFODNN7EXAMPLE",
    "secretKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "bucketName": "my-company-data",
    "region": "us-east-1",
    "useVirtualAddressing": true,
    "urlExpirationMinutes": 10080
  }
}

LucidLink Connect requires only GetObject permissions for the bucket. Access credentials should always be scoped to allow only the minimum access required.

Example response:

{
    "id": "bXktZGF0YS1zdG9yZQ",
    "name": "my-s3-bucket",
    "kind": "S3DataStore",
    "s3StorageParams": {
        "accessKey": "AKIAIOSFODNN7EXAMPLE",
        "bucketName": "my-company-data",
        "region": "us-east-1",
        "endpoint": "https://s3.amazonaws.com",
        "useVirtualAddressing": true,
        "urlExpirationMinutes": 10080
    }
}

Use the ID from the response when linking files from this data store as shown in the next step.

Step 2: Link External Files

Once your data store is configured, link specific S3 objects to your filespace:

POST /filespaces/{filespaceId}/external/entries

{
  "path": "/reports/2024-annual-report.pdf",
  "kind": "SingleObjectFile",
  "dataStoreId": "bXktZGF0YS1zdG9yZQ",
  "singleObjectFileParams": {
    "objectId": "2024-annual-report.pdf"
  }
}

Example response:

{
  "data": {
    "id": "1234:5678",
    "parentId": "4294967294:0",
    "name": "2024-annual-report.pdf",
    "type": "file",
    "size": 1048576,
    "modifiedAt": "2025-10-21T10:30:00Z",
    "createdAt": "2025-10-20T08:00:00Z"
  }
}

The file will now appear at /reports/2024-annual-report.pdf in your filespace.

It is possible to link an object to multiple filespace entries at different paths in the same filespace, or even across multiple filespaces. Keep in mind that each link will be billed as additional storage.

Step 3: Access Your Files

External entries are denoted by the LucidLink Connect icon:

Icon.svg
The LucidLink Connect icon

This icon is intended to help you distinguish between external entries and LucidLink-native entries. They function like any other file in your filespace with the exception of being read-only, and can be accessed through:

Security & Encryption

LucidLink Connect Is Not Zero-Knowledge

Due to the fact that assets linked via LucidLink Connect are stored outside of LucidLink in their native format and without client-side encryption, they are not covered by LucidLink’s zero-knowledge guarantees.

This does not mean that LucidLink Connect is insecure - merely that it offers a more relaxed mode of data privacy on par with the data storage providers from whom the data is being streamed. 

More explicitly, this means that it is not technically impossible for the external storage provider, or for LucidLink with authorized access, to view or access external entries linked via LucidLink Connect (in contrast with the architecture of our base product). However, LucidLink does not read or download the contents of externally linked storage as part of normal operations.

As a matter of policy, customer file contents are not accessed during day-to-day use of LucidLink Connect. Because LucidLink Connect is not designed to make access to externally linked data cryptographically impossible, LucidLink retains a technical capability to access such data in exceptional circumstances where this is necessary to operate or support the service and where authorized internal access is used. Any such access is subject to LucidLink’s internal security controls, including restricted access, role-based permissions, and logging.

This differs from LucidLink’s standard layout, which is designed to prevent LucidLink from accessing customer data by design. Customers should therefore treat LucidLink Connect as operating under a trusted service-provider model rather than a strict zero-knowledge model.

LucidLink’s zero-knowledge guarantee continues to apply to all non-external data stored within the filespace even when LucidLink Connect is enabled. Assets that are externally linked via LucidLink Connect are handled differently, as described above, because they remain stored in customer-controlled external environments.

Note that in the case of the HTTP(S)-based method, LucidLink does not possess credentials to the storage and file URLs are encrypted in a manner that prevents access by LucidLink. While this is more constrained from a trust perspective than LucidLink-managed data stores, it is still not necessarily perfectly zero-knowledge since it is:

  • stored in its native form
  • without client-side encryption
  • in an external storage provider whose policies and procedures regarding access to stored data are outside LucidLink's control and cannot be guaranteed

Access Control

  • Filespace Admin: Required to create/delete data stores and link external entries
  • File Permissions: Standard Lucid permissions apply to external entries
  • Audit Trail: All operations are tracked in audit logs

Limitations & Best Practices

Current Limitations

  • Read-Only: Cannot modify or delete the original asset.
  • Individual Object Linking: Each operation links a single object - bulk link operations are planned for a future release
  • Versioning Not Supported: object versioning is not supported at release but may be enabled in future versions
  • Instant retrieval classes only: non-instant retrieval classes such as Glacier are not supported as with standard LucidLink entries
  • Snapshots: External entries are included in snapshots for the sake of accuracy and completeness; but due to the nature of snapshots as a point-in-time representation of the filespace metadata, their URLs will not be refreshed automatically and external entries in snapshots will cease to function once the URL expiration elapses. The data backing the entry will remain safe in the bucket.

External entries can be moved, renamed, copied, and/or deleted within the local filesystem. It is not possible to modify an external entry in place because external entries are read-only. The following bullets explain what will occur in each scenario.

  • Move
    • The external entry remains in the filesystem but is moved to a new location. 
    • Its access URL will continue to be refreshed by the hub (where applicable)
    • API requests will reflect the new path in the filesystem
    • The data backing the entry remains in place in the bucket
  • Rename
    • The external entry remains in the filesystem but is renamed
    • Its access URL will continue to be refreshed by the hub (where applicable)
    • API requests will reflect the new path in the filesystem / name 
    • The data backing the entry remains in place in the bucket
  • Delete
    • The external entry is removed from the filesystem
    • Its access URL will no longer be refreshed
    • API requests will no longer list the external entry ID
    • The data backing the entry remains in place in the bucket
  • Copy
    • The external entry remains in the filesystem, but the copied file is written to the bucket backing the filespace to which the Data Store is connected
    • The original external asset URL will continue to be refreshed (where applicable), but the copy will not be subject to refresh as it will have been copied into the filespace
    • API requests will reflect the original path for the original asset, but the copy will not be listed in API requests for external assets (since it has been copied into the filespace and is not an external asset)
    • The data backing the original external entry remains in the bucket, the copy’s data resides in the bucket backing the filespace in LucidLink-proprietary layout
  • Modify
    • The write operation will fail and you will receive an error because you cannot write back to the bucket via LucidLink Connect.

Best Practices

A Word On Saving

Depending on the behavior of your applications, it may be possible to save changes once you’ve opened an external entry for reading.

This is achieved through a combination of the filesystem operations listed above such as move/rename/delete, and effectively overwrites the original entry. The file that replaces it will be a LucidLink-native entry that is then stored in LucidLink-layout in the bucket the filespace was initialized against. The external entry in the filespace will cease to exist, but the data will remain safe in the bucket and can be linked again in a different location if desired.

You may also “save-as” in a different location, which is effectively the same as the “Copy” scenario described above.

The external entry icon described above is provided to help you distinguish and understand the behavior of applications in scenarios such as these.

API Deployment

  • For complete documentation on configuring Service Accounts, setting up the self-hosted API container, and using the LucidLink API; please refer to the LucidLink API series of articles.

Security

  • Use access credentials with minimal permissions (s3:GetObject or equivalent only)
  • Follow best practices for backup or replication of the source bucket - LucidLink is not responsible for the durability of data stored in customer-provided storage
  • Regularly rotate access keys (possible using a patch request with the DataStore endpoint)

Scalability

Related Resources

Support

If you need assistance with LucidLink Connect:

Was this article helpful?

0 out of 0 found this helpful