Enable AI Gateway
AI Gateway opens the SmartAGT models to connected tools, through an OpenAI-compatible API. Its service is installed and started with the Backend. This procedure enables it in four steps.
- ServiceStarted with the Backend
- ActivationPublic URL and network
- KeyPermission and access key
- TestFirst call
Prerequisites
- Backend installed, AI Gateway service included: Install the Backend.
- Conversation model connected: Getting started.
- DNS name used by the connected tools, for example
gateway.example.test, resolving to this server, and port8001open for inbound traffic in the server firewall.
- Backend installed, AI Gateway service created on the nodes with the
gatewayrole: Install the Backend. - Conversation model connected: Getting started.
- HTTPS load balancer in front of these nodes, listening on port
8001under the name used by the connected tools, for examplegateway.example.test. It forwards the client address inX-Forwarded-For, because the Network list filters this address. - Port
8001of thegatewaynodes open for inbound traffic from the load balancer only.
Procedure
1. Check the service
systemctl is-active SmartAGT-AI-Gateway-Service
Expected result: the command displays active.
Get-Service SmartAGT-AI-Gateway-Service
Expected result: the status is Running.
Declare the load balancer in the shared YAML of the Backend nodes, then restart the AI Gateway service of every gateway node, one node after the other:
smartagt:
gateway:
server:
trusted_proxies:
- <load balancer IP address>
Without this declaration, AI Gateway refuses the calls relayed by the load balancer.
sudo systemctl restart SmartAGT-AI-Gateway-Service
systemctl is-active SmartAGT-AI-Gateway-Service
Expected result: on every gateway node, the command displays active.
Restart-Service SmartAGT-AI-Gateway-Service
Get-Service SmartAGT-AI-Gateway-Service
Expected result: on every gateway node, the status is Running.
2. Enable AI Gateway
To enable AI Gateway, from the administration interface:
- Go to AI Factory, section Assistants, page AI Gateway.
- In Activation, enter the Public URL, for example
https://gateway.example.test:8001. - In Network, check that the addresses of the connected tools are in the list. By default, it allows the private networks
10.0.0.0/8,172.16.0.0/12and192.168.0.0/16. For agent-based Zero Trust access, add its address range, for example100.64.0.0/10. - Turn on the switch at the top of the page, then click Save and confirm the activation.
The confirmation covers the retention of call content, which you can adjust in Call retention (14 days by default).
Expected result: the switch shows Enabled.
3. Create a key
Every call authenticates with an AI Gateway key, which only an account with the Use the AI Gateway permission can create. Nobody holds this permission by default.
To create a first key, from the administration interface:
- Open the record of the person, Permissions tab, and grant Use the AI Gateway.
- This person opens their settings, AI Gateway section, and clicks Create an AI Gateway key.
- Enter the Key name, the Connected tool and the Expiration date, then click Create the key.
- Copy the displayed key, which starts with
sagt_gw_: it will never be shown again.
Expected result: the key appears in the list of keys of the person.
4. Test a call
From a workstation whose address is in the Network list:
curl -k https://gateway.example.test:8001/v1/models -H "Authorization: Bearer <key>"
curl.exe -k https://gateway.example.test:8001/v1/models -H "Authorization: Bearer <key>"
-k accepts the certificate generated by SmartAGT. After Install your certificates, replace it with --cacert <authority of your certificates>.
Expected result: the command displays, in JSON, the list of conversation models opened by AI Gateway.
Note
A 403 code reports an address missing from the Network list (address_not_allowed) or an AI Gateway that is not enabled (gateway_not_activated). A 401 code reports a missing, unknown or expired key. The app-gateway and access-gateway logs, in the Backend log folder, give the cause.
Special cases
Key for an application
For an application rather than a person, create a service account: IAM, Service accounts, record of the account, AI Gateway keys tab, Issue a key button. The first active key of a service account takes a license seat.
Opened models
By default, AI Gateway opens every conversation model. To restrict this catalog: Governance, section AI Gateway, page Models.
AI Gateway behind a proxy
Declare the proxy addresses in the Backend YAML, then restart the AI Gateway service:
smartagt:
gateway:
server:
trusted_proxies:
- <proxy IP address>
AI Gateway then reads the client address in the headers forwarded by this proxy, and the Network list allows this address. A call relayed by an undeclared proxy is refused. Only IPv4 addresses are accepted: Configure a reverse proxy.
Listener and certificate
By default, AI Gateway listens on port 8001 of every address of the server, always over HTTPS. The listener and your certificate are declared in the Backend YAML, under smartagt.gateway.server (host, port, certfile, keyfile): see the Advanced YAML of Install the Backend and Install your certificates.
Server dedicated to AI Gateway
On a server dedicated to AI Gateway, use the same Backend package, the same YAML and the KEK of the existing Backend. Apply step 1 of Install the Backend, skip steps 2 and 3, then in step 4 create only the AI Gateway service.
Node dedicated to AI Gateway
Declare it in deployment.nodes with the gateway role only, then follow the special case "Add a Backend node" of Install the Backend. Then add it to the load balancer nodes.