AYXZA Relay · Universal AI Gateway

Commercial section · 01 / 03

Perché AYXZA Relay

Adding AI to a product is no longer "calling an API". It means juggling five vendors, containing costs, guaranteeing uptime and protecting data. AYXZA Relay solves all of this with a single library.

The problem Without a gateway

✗Duplicated code for each provider (OpenAI, Anthropic, Gemini…)
✗Technical lock-in: changing model = rewriting the integration
✗Costs out of control, no per-tenant visibility
✗One provider down = service down
✗Sensitive data sent to the cloud, no on-prem option
✗RAG / knowledge base built from scratch every time

The solution With AYXZA Relay

+Single API: gw.chat() works with any registered model
+Provider switch via config, zero refactoring
+Costs tracked per call, user and model, with quotas and budgets per tenant
+Automatic fallback to a backup model, with a per-provider circuit breaker
+Local engines (Ollama, vLLM, llama.cpp) for sensitive data, with no external network
+Built-in Knowledge Base module with pgvector

Typical use cases

01

Multi-tenant SaaS

Per-customer cost tracking, per-plan limits, different models per tier. Out of the box.

02

On-premise enterprise

Local Ollama for confidential data, optional cloud fallback for non-sensitive tasks.

03

Corporate knowledge base

RAG over internal documents with pgvector, auto re-indexing, per-tenant isolation.

04

Agents & chains

Multi-step orchestration, native structured output (Gemini, OpenAI) and multimodal attachments (PDF, images) on prompts.

05

Provider migration

The model is changed from configuration or from the console, without touching application code.

06

High availability

Transparent fallback: if a provider does not answer, the request moves to the declared backup model with no code changes.

Governance and observability

01

Routing profiles

Each request stays within its declared class, local, European cloud or cloud, and moves through the candidates in order.

02

Resilience

Backup models, a per-provider circuit breaker, several keys per provider with an automatic pause for the failing one, and an on-demand health probe.

03

Web console

A web console manages providers, models, keys, prompts, quotas and the knowledge base, with a test bench for chat and embeddings.

04

Versioned prompts

Prompts are versioned and validated: every run records the version used and retries when the output fails its checks.

05

Documents and confidence

PDF and Word documents enter the knowledge base; answers carry a confidence that combines the model's judgement with the relevance of the retrieved passages.

06

Quotas and budgets

Spending quotas per user and per tenant, with actual spend set against the cap.

07

Data retention

Calls older than a given date are anonymised after a dry run, leaving the accounting figures intact.

08

OpenTelemetry

Native OpenTelemetry traces on every call, ready for any collector.

09

Tool loop

The model calls the application's functions and Relay manages the turns until the answer.

Technical section · 02 / 03

Architecture

Pure Python library, stateless where possible, with persistence on PostgreSQL + pgvector. No external service dependency beyond the LLM providers themselves.

Layered architecture

From the client application to providers, through the core gateway.

flowchart TB subgraph CLIENT["CLIENT APPLICATIONS"] A1["Web App"] A2["Backend API"] A3["CLI / Scripts"] end subgraph FORGE["AYXZA Relay · Gateway Library"] GW["Unified API
gw.chat · gw.embedding · gw.kb"] subgraph CORE["Core Services"] R["Profiles
routing"] T["Tracking
cost · tenant · model"] F["Fallback
provider failover"] K["Knowledge Base
RAG + pgvector"] end GW --> R R --> T T --> F GW -.-> K end subgraph PROV["LLM PROVIDERS"] P1["OpenAI"] P2["Anthropic"] P3["Google Gemini"] P4["Google Vertex"] P5["OpenAI-compatible
Ollama · vLLM · llama.cpp"] end DB[("PostgreSQL
+ pgvector")] CLIENT --> GW F --> P1 F --> P2 F --> P3 F --> P4 F --> P5 T --> DB K --> DB

Request flow with fallback

Primary provider unavailable, automatic failover to secondary.

sequenceDiagram autonumber participant App as Client App participant Relay as AYXZA Relay participant P1 as OpenAI primary participant P2 as Anthropic fallback participant DB as PostgreSQL App->>Relay: gw.chat(ChatRequest(profilo="standard")) Relay->>P1: POST /v1/chat/completions P1-->>Relay: 503 Service Unavailable Note over Relay: Hop to the backup model Relay->>P2: POST /v1/messages P2-->>Relay: 200 OK + tokens Relay->>DB: log(tenant, model, cost, latency) Relay-->>App: response (transparent)

RAG · Knowledge Base pipeline

Ingestion → chunking → embedding → retrieval → generation.

flowchart LR D["Documents
PDF · Word · text"] S["Source Registry
hash-based updates"] CH["Chunker
shared utility"] EM["Embedder
gemini-embedding-2
768-dim"] VS[("pgvector
HNSW index")] Q["User query"] QE["Query embedding"] RT["Retrieval
global + tenant chunks"] GEN["LLM Generation
any provider"] D --> S --> CH --> EM --> VS Q --> QE --> RT VS -.-> RT RT --> GEN

API in action

A few lines for chat, profile routing, structured output and answers from the knowledge base.

# 1. Setup: providers, models and keys are registered once
import os
from forge_ai import ChatRequest, ForgeAIConfig, Gateway, Message

config = ForgeAIConfig(
    database_url=os.environ["FORGE_AI_DATABASE_URL"],
    fernet_key=os.environ["FORGE_AI_FERNET_KEY"],
)

with Gateway(config) as gw:
    # 2. Direct chat on a provider and a model
    r = gw.chat(ChatRequest(
        provider="gemini",
        modello="gemini-2.5-flash",
        messaggi=[Message(role="user", content="Summarise this contract in 3 points")],
    ))
    print(r.testo, r.costo, r.provider, r.latenza_ms)

    # 3. Profile routing: candidates and fallback live in the configuration
    r = gw.chat(ChatRequest(
        profilo="standard",
        messaggi=[Message(role="user", content="...")],
    ))

    # 4. Native structured output with JSON Schema
    r = gw.chat(ChatRequest(
        provider="openai",
        modello="gpt-4o-mini",
        messaggi=[Message(role="user", content="Extract the data from this invoice...")],
        output_schema={"type": "object",
                       "properties": {"totale": {"type": "number"}, "iva": {"type": "number"}},
                       "required": ["totale", "iva"], "additionalProperties": False},
    ))

    # 5. Knowledge base: answer with confidence
    esito = gw.kb.interroga(
        "What is the expense reimbursement policy?",
        codice_prompt="qa_policy",
        app_codice="intranet",
    )
    print(esito.risposta, esito.confidenza_finale)

Capability matrix

Capability	OpenAI	Anthropic	Gemini	Vertex	Ollama
Chat completion	●	●	●	●	●
Streaming	●	—	●	—	●
Structured output (native)	●	—	●	●	—
Function calling	●	●	●	—	●
Embeddings	●	—	●	●	●
Vision · multimodal	●	●	●	●	●
On-premise · air-gapped	—	—	—	—	●

Requirements

Python 3.12+
PostgreSQL 16+ with the pgvector extension
API keys for the chosen providers (not needed for local engines)
A local OpenAI-compatible engine (Ollama, vLLM, llama.cpp) for on-premise models

Tech stack

SQLAlchemy 2.x
Pydantic v2 for validation and schemas
Alembic for database migrations
Retries with exponential backoff and jitter
Suite of more than 1,700 tests, with public contracts verified at every release

Supported providers · 03 / 03

Provider integrati

All major cloud LLM vendors, plus an OpenAI-compatible adapter reaching cloud engines (Groq, DeepSeek, OpenRouter) and local ones (Ollama, vLLM, llama.cpp), air-gapped deployments included.

OA

OpenAI

GPT and o-series models

Frontier reasoning, full function calling, native structured output.

AN

Anthropic

Claude Opus · Sonnet · Haiku

Long-context (200K+), excellent for long reasoning and document analysis.

GG

Google Gemini

Gemini 2.5 and 3 · gemini-embedding-2

Native multimodal, 768-dim embeddings, configurable thinking levels.

VX

Google Vertex AI

Enterprise GCP · configurable region

The same Gemini models on Google Cloud, with GCP billing and a processing region of your choice, European included.

OL

Ollama

Llama · Mistral · Qwen · local

100% local execution, zero external network, ideal for sensitive data and GDPR. The same OpenAI-compatible adapter also reaches cloud engines (Groq, DeepSeek, OpenRouter).

++

Custom provider

extensible api

Adding a new provider takes ~150 lines of Python implementing the BaseProvider interface.

Contact us

To evaluate an integration, discuss a use case or get a personalized demo on your product.

Contact us →

One SDK. Every AI model.