Aryan SinghandClaude Opus 5
cb985dcf9c
One endpoint in front of every free LLM tier, with automatic failover and no provider key ever leaving the box. Providers are data, not code: adding one is a JSON file plus an env var, never a TypeScript change. Ships 12 providers pre-written; `verify-provider --apply` derives the live model list from your own key. - core: registry, bucketed quota meter (rolling + UTC-calendar windows), key-pool store, 6-step model router with diagnostic rejection reasons - gateway: node:http, failover that commits to a stream only after 200 headers, SSE passthrough with usage sniffing, virtual keys with admin/inference roles - context engine: sqlite sessions and a 5-rung refit ladder, so a 128k thread keeps working when it fails over to an 8k model - cache: exact (sha256, safe-by-default) + opt-in semantic (flat Float32Array) - guard: inbound limits, quota forecasting, limit calibration from upstream headers, key health, secret/PII/injection scanning - dashboard: one self-contained HTML file, live SSE request log, token usage - sdk: zero-dep typed client, drop-in for the OpenAI SDK Zero runtime dependencies. 326 tests, 27-check smoke suite against a scripted mock upstream so routing is testable without spending quota. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
99 lines
3.9 KiB
Bash
99 lines
3.9 KiB
Bash
# Freeway — copy to `.env` and fill in whatever you have.
|
|
#
|
|
# Nothing here is required. Every provider you leave blank renders in the
|
|
# dashboard as a card telling you which variable to set and where to get a key.
|
|
#
|
|
# A single variable may hold SEVERAL comma-separated keys — that is the key
|
|
# pool. Quota is tracked per key, so a 429 on one benches only that one and the
|
|
# request retries on the next.
|
|
# MISTRAL_API_KEY=key-one,key-two,key-three
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Gateway
|
|
# ---------------------------------------------------------------------------
|
|
|
|
# Virtual keys your apps present. Real provider keys never leave the gateway.
|
|
# EMPTY = open, and the gateway then REFUSES to bind anything but 127.0.0.1.
|
|
# Generate one with: node -e "console.log('fw-'+crypto.randomUUID())"
|
|
FREEWAY_KEYS=
|
|
|
|
# FULL ACCESS. Everything under /api/*: reading stored conversations, writing
|
|
# provider credentials, spending quota. The keys above grant INFERENCE ONLY.
|
|
# Required to use the dashboard on anything but localhost — without it, /api/*
|
|
# is refused entirely on a non-loopback bind.
|
|
# node -e "console.log('fw-admin-'+crypto.randomUUID())"
|
|
FREEWAY_ADMIN_KEY=
|
|
|
|
# Browser origins allowed to call this gateway, comma-separated. Empty (the
|
|
# default) sends no CORS headers at all, which is correct for the bundled
|
|
# dashboard since it is same-origin. "*" is rejected: it would let any page you
|
|
# happen to visit drive your gateway from your own browser.
|
|
# FREEWAY_CORS_ORIGINS=https://myapp.example,https://staging.myapp.example
|
|
FREEWAY_CORS_ORIGINS=
|
|
|
|
HOST=127.0.0.1
|
|
PORT=8787
|
|
|
|
# priority | round-robin | least-used | fastest
|
|
FREEWAY_STRATEGY=priority
|
|
|
|
# Paths. Docker compose sets these; you rarely need them locally.
|
|
# FREEWAY_ROOT=/app
|
|
# FREEWAY_PROVIDERS=/app/providers
|
|
# FREEWAY_DATA=/app/data
|
|
# FREEWAY_CONFIG=/app/freeway.config.json
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Providers — verified free tiers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
# Mistral — https://console.mistral.ai/api-keys
|
|
# Free "Experiment" plan. Phone verification, no card. ~1B tokens/month.
|
|
# Note: Mistral no longer publishes exact rps/tpm, so providers/mistral.json
|
|
# carries nulls and the gateway learns the real numbers from response headers.
|
|
MISTRAL_API_KEY=
|
|
MISTRAL_API_KEY_2=
|
|
|
|
# Cloudflare Workers AI — https://dash.cloudflare.com/profile/api-tokens
|
|
# 10,000 neurons/day free. BOTH variables are required: the account id is
|
|
# interpolated into the base URL.
|
|
CLOUDFLARE_ACCOUNT_ID=
|
|
CLOUDFLARE_API_TOKEN=
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Providers — bundled, waiting for a key
|
|
# ---------------------------------------------------------------------------
|
|
|
|
# Groq — https://console.groq.com/keys ~30 rpm / 14.4k rpd, very fast
|
|
GROQ_API_KEY=
|
|
GROQ_API_KEY_2=
|
|
|
|
# Cerebras — https://cloud.cerebras.ai/platform ~1M tokens/day, no card
|
|
CEREBRAS_API_KEY=
|
|
|
|
# Google AI Studio — https://aistudio.google.com/apikey
|
|
# ~15 rpm / 1500 rpd on Flash. Native multimodal.
|
|
# NOTE: the free tier is not available in the EU, UK or Switzerland.
|
|
GOOGLE_AI_API_KEY=
|
|
|
|
# OpenRouter — https://openrouter.ai/keys only :free models cost nothing
|
|
OPENROUTER_API_KEY=
|
|
|
|
# GitHub Models — https://github.com/settings/tokens needs the models:read scope
|
|
GITHUB_MODELS_TOKEN=
|
|
|
|
# Together AI — https://api.together.xyz/settings/api-keys only -Free models
|
|
TOGETHER_API_KEY=
|
|
|
|
# Hugging Face — https://huggingface.co/settings/tokens small free credits
|
|
HF_TOKEN=
|
|
|
|
# NVIDIA NIM — https://build.nvidia.com/ free credits, not a free tier
|
|
NVIDIA_API_KEY=
|
|
|
|
# Scaleway — https://console.scaleway.com/iam/api-keys EU-hosted (Paris)
|
|
SCALEWAY_API_KEY=
|
|
|
|
# Ollama — your own machine, no key. Set the host to enable it.
|
|
# OLLAMA_HOST=http://127.0.0.1:11434
|