Skip to content

Enable AI Gateway

AI Gateway opens the SmartAGT models to connected tools, through an OpenAI-compatible API. Its service is installed and started with the Backend. This procedure enables it in four steps.

  1. ServiceStarted with the Backend
  2. ActivationPublic URL and network
  3. KeyPermission and access key
  4. TestFirst call

Prerequisites

  • Backend installed, AI Gateway service included: Install the Backend.
  • Conversation model connected: Getting started.
  • DNS name used by the connected tools, for example gateway.example.test, resolving to this server, and port 8001 open for inbound traffic in the server firewall.
  • Backend installed, AI Gateway service created on the nodes with the gateway role: Install the Backend.
  • Conversation model connected: Getting started.
  • HTTPS load balancer in front of these nodes, listening on port 8001 under the name used by the connected tools, for example gateway.example.test. It forwards the client address in X-Forwarded-For, because the Network list filters this address.
  • Port 8001 of the gateway nodes open for inbound traffic from the load balancer only.

Procedure

1. Check the service

systemctl is-active SmartAGT-AI-Gateway-Service

Expected result: the command displays active.

Get-Service SmartAGT-AI-Gateway-Service

Expected result: the status is Running.

Declare the load balancer in the shared YAML of the Backend nodes, then restart the AI Gateway service of every gateway node, one node after the other:

smartagt:
  gateway:
    server:
      trusted_proxies:
        - <load balancer IP address>

Without this declaration, AI Gateway refuses the calls relayed by the load balancer.

sudo systemctl restart SmartAGT-AI-Gateway-Service
systemctl is-active SmartAGT-AI-Gateway-Service

Expected result: on every gateway node, the command displays active.

Restart-Service SmartAGT-AI-Gateway-Service
Get-Service SmartAGT-AI-Gateway-Service

Expected result: on every gateway node, the status is Running.

2. Enable AI Gateway

To enable AI Gateway, from the administration interface:

  1. Go to AI Factory, section Assistants, page AI Gateway.
  2. In Activation, enter the Public URL, for example https://gateway.example.test:8001.
  3. In Network, check that the addresses of the connected tools are in the list. By default, it allows the private networks 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16. For agent-based Zero Trust access, add its address range, for example 100.64.0.0/10.
  4. Turn on the switch at the top of the page, then click Save and confirm the activation.

The confirmation covers the retention of call content, which you can adjust in Call retention (14 days by default).

Expected result: the switch shows Enabled.

3. Create a key

Every call authenticates with an AI Gateway key, which only an account with the Use the AI Gateway permission can create. Nobody holds this permission by default.

To create a first key, from the administration interface:

  1. Open the record of the person, Permissions tab, and grant Use the AI Gateway.
  2. This person opens their settings, AI Gateway section, and clicks Create an AI Gateway key.
  3. Enter the Key name, the Connected tool and the Expiration date, then click Create the key.
  4. Copy the displayed key, which starts with sagt_gw_: it will never be shown again.

Expected result: the key appears in the list of keys of the person.

4. Test a call

From a workstation whose address is in the Network list:

curl -k https://gateway.example.test:8001/v1/models -H "Authorization: Bearer <key>"
curl.exe -k https://gateway.example.test:8001/v1/models -H "Authorization: Bearer <key>"

-k accepts the certificate generated by SmartAGT. After Install your certificates, replace it with --cacert <authority of your certificates>.

Expected result: the command displays, in JSON, the list of conversation models opened by AI Gateway.

Note

A 403 code reports an address missing from the Network list (address_not_allowed) or an AI Gateway that is not enabled (gateway_not_activated). A 401 code reports a missing, unknown or expired key. The app-gateway and access-gateway logs, in the Backend log folder, give the cause.

Special cases

Key for an application

For an application rather than a person, create a service account: IAM, Service accounts, record of the account, AI Gateway keys tab, Issue a key button. The first active key of a service account takes a license seat.

Opened models

By default, AI Gateway opens every conversation model. To restrict this catalog: Governance, section AI Gateway, page Models.

AI Gateway behind a proxy

Declare the proxy addresses in the Backend YAML, then restart the AI Gateway service:

smartagt:
  gateway:
    server:
      trusted_proxies:
        - <proxy IP address>

AI Gateway then reads the client address in the headers forwarded by this proxy, and the Network list allows this address. A call relayed by an undeclared proxy is refused. Only IPv4 addresses are accepted: Configure a reverse proxy.

Listener and certificate

By default, AI Gateway listens on port 8001 of every address of the server, always over HTTPS. The listener and your certificate are declared in the Backend YAML, under smartagt.gateway.server (host, port, certfile, keyfile): see the Advanced YAML of Install the Backend and Install your certificates.

Server dedicated to AI Gateway

On a server dedicated to AI Gateway, use the same Backend package, the same YAML and the KEK of the existing Backend. Apply step 1 of Install the Backend, skip steps 2 and 3, then in step 4 create only the AI Gateway service.

Node dedicated to AI Gateway

Declare it in deployment.nodes with the gateway role only, then follow the special case "Add a Backend node" of Install the Backend. Then add it to the load balancer nodes.