Skip to content

Configure S3

Connect SmartAGT to a private S3 bucket. S3 is required in containers and can be chosen on VMs, according to the storage mode.

Connect SmartAGT to a private S3 bucket. In a cluster, S3 is required: every Backend node shares the same bucket, from which each one reads the documents uploaded through the others. See the storage mode.

Prepare an HTTPS endpoint, a private bucket and access keys, plus the region if your provider requires it. The S3 account must be able to inspect the bucket, and read, write and delete its objects.

Procedure

1. Prepare TLS trust

With a private authority, install its trust chain on every Backend host before initialization. In containers, add it to the trust store of the image.

sudo install -m 644 s3-ca.crt /usr/local/share/ca-certificates/smartagt-s3.crt
sudo update-ca-certificates

In PowerShell as administrator:

Import-Certificate -FilePath '.\s3-ca.crt' -CertStoreLocation Cert:\LocalMachine\Root

2. Connect the bucket

The S3 access keys are secrets: they never appear in the YAML. Like the PostgreSQL password and the KEK, they are supplied through the environment.

  1. In the delivered Backend YAML, fill in the following block with the bucket settings, without the keys:

    smartagt:
      mediastore:
        backend: s3
        s3:
          endpoint_url: "https://storage.example.org"
          bucket: "smartagt-documents"
          region: "eu-west-1"
    
  2. At initialization, add the keys to the session before running init:

    read -rsp 'S3 access key: ' SMARTAGT_MEDIASTORE_S3_ACCESS__KEY; echo
    read -rsp 'S3 secret: ' SMARTAGT_MEDIASTORE_S3_SECRET__KEY; echo
    export SMARTAGT_MEDIASTORE_S3_ACCESS__KEY SMARTAGT_MEDIASTORE_S3_SECRET__KEY
    

    Also add SMARTAGT_MEDIASTORE_S3_ACCESS__KEY,SMARTAGT_MEDIASTORE_S3_SECRET__KEY to the --preserve-env list of the init command.

    $env:SMARTAGT_MEDIASTORE_S3_ACCESS__KEY = [Net.NetworkCredential]::new('', (Read-Host 'S3 access key' -AsSecureString)).Password
    $env:SMARTAGT_MEDIASTORE_S3_SECRET__KEY = [Net.NetworkCredential]::new('', (Read-Host 'S3 secret' -AsSecureString)).Password
    
  3. When creating the services, add the same keys to their environment:

    In /etc/smartagt/backend.env, after the password and the KEK:

    SMARTAGT_MEDIASTORE_S3_ACCESS__KEY=<S3 access key>
    SMARTAGT_MEDIASTORE_S3_SECRET__KEY=<S3 secret>
    

    Before the creation loop, read the keys:

    $s3Key = [Net.NetworkCredential]::new('', (Read-Host 'S3 access key' -AsSecureString)).Password
    $s3Secret = [Net.NetworkCredential]::new('', (Read-Host 'S3 secret' -AsSecureString)).Password
    

    Then add them to the Environment value of each service:

    Set-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Services\$($service.Name)" -Name Environment -Type MultiString -Value "SMARTAGT_CORE_DATABASE_PASSWORD=$password", "SMARTAGT_KEK=$kek", "SMARTAGT_MEDIASTORE_S3_ACCESS__KEY=$s3Key", "SMARTAGT_MEDIASTORE_S3_SECRET__KEY=$s3Secret"
    

    Before the creation loop, read the keys:

    $s3Key = [Net.NetworkCredential]::new('', (Read-Host 'S3 access key' -AsSecureString)).Password
    $s3Secret = [Net.NetworkCredential]::new('', (Read-Host 'S3 secret' -AsSecureString)).Password
    

    Then add them to the environment of each service:

    nssm set $service.Name AppEnvironmentExtra "SMARTAGT_CORE_DATABASE_PASSWORD=$password" "SMARTAGT_KEK=$kek" "SMARTAGT_MEDIASTORE_S3_ACCESS__KEY=$s3Key" "SMARTAGT_MEDIASTORE_S3_SECRET__KEY=$s3Secret"
    

In containers, inject both variables from the secrets manager of your infrastructure.

Expected result: the initialization and the services start without an S3 connection error.

The connection is declared during the Backend installation, identically on every node: Install the Backend.

  1. In the shared YAML of the Backend nodes, fill in the following block, without the access keys:

    smartagt:
      mediastore:
        backend: s3
        s3:
          endpoint_url: "https://storage.example.org"
          bucket: "smartagt-documents"
          region: "eu-west-1"
    
  2. Provide the keys through the environment, in the init session and then in the environment of the services of every node: SMARTAGT_MEDIASTORE_S3_ACCESS__KEY for the access key, SMARTAGT_MEDIASTORE_S3_SECRET__KEY for the secret.

Expected result: on every node, the services start without an S3 connection error.