Freeway: self-hosted OpenAI-compatible gateway for free-tier providers

One endpoint in front of every free LLM tier, with automatic failover and no
provider key ever leaving the box.

Providers are data, not code: adding one is a JSON file plus an env var, never
a TypeScript change. Ships 12 providers pre-written; `verify-provider --apply`
derives the live model list from your own key.

- core: registry, bucketed quota meter (rolling + UTC-calendar windows),
  key-pool store, 6-step model router with diagnostic rejection reasons
- gateway: node:http, failover that commits to a stream only after 200 headers,
  SSE passthrough with usage sniffing, virtual keys with admin/inference roles
- context engine: sqlite sessions and a 5-rung refit ladder, so a 128k thread
  keeps working when it fails over to an 8k model
- cache: exact (sha256, safe-by-default) + opt-in semantic (flat Float32Array)
- guard: inbound limits, quota forecasting, limit calibration from upstream
  headers, key health, secret/PII/injection scanning
- dashboard: one self-contained HTML file, live SSE request log, token usage
- sdk: zero-dep typed client, drop-in for the OpenAI SDK

Zero runtime dependencies. 326 tests, 27-check smoke suite against a scripted
mock upstream so routing is testable without spending quota.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Aryan SinghandClaude Opus 5 committed 2026-08-16 22:41:49 -05:00
commit cb985dcf9c
86 files changed
+16350

No files matched your search

+62
View File
@@ -0,0 +1,62 @@
# Zero runtime dependencies means the runtime stage needs nothing but Node and
# the compiled output — no node_modules to copy, nothing to audit at runtime.
# ---- build ---------------------------------------------------------------
FROM node:24-alpine AS build
WORKDIR /app
# Dev dependencies are only typescript + @types/node, so this layer is small and
# caches well across source edits.
COPY package.json package-lock.json* ./
COPY packages/core/package.json ./packages/core/
COPY packages/gateway/package.json* ./packages/gateway/
COPY packages/sdk/package.json* ./packages/sdk/
RUN npm ci --ignore-scripts --no-audit --no-fund
COPY tsconfig.json tsconfig.base.json ./
COPY packages ./packages
RUN npm run build
# ---- runtime -------------------------------------------------------------
FROM node:24-alpine AS runtime
ENV NODE_ENV=production
WORKDIR /app
# `node` (uid 1000) ships with the image. Running as root would let a gateway
# compromise rewrite the mounted providers/ directory.
RUN mkdir -p /app/data && chown -R node:node /app
# Only the compiled output and each package's manifest — no sources, no tests.
COPY --from=build --chown=node:node /app/packages/core/dist ./packages/core/dist
COPY --from=build --chown=node:node /app/packages/core/package.json ./packages/core/
COPY --from=build --chown=node:node /app/packages/gateway/dist ./packages/gateway/dist
COPY --from=build --chown=node:node /app/packages/gateway/package.json ./packages/gateway/
COPY --from=build --chown=node:node /app/packages/gateway/public ./packages/gateway/public
COPY --from=build --chown=node:node /app/packages/sdk/dist ./packages/sdk/dist
COPY --from=build --chown=node:node /app/packages/sdk/package.json ./packages/sdk/
COPY --chown=node:node package.json ./
COPY --chown=node:node providers ./providers
COPY --chown=node:node freeway.config.json ./freeway.config.json
# `@freeway/core` resolves through npm's workspace symlinks, which live in the
# root node_modules — a directory this image otherwise has no reason to carry,
# since there are no runtime dependencies. Recreating just the links keeps the
# resolution working without shipping the toolchain.
RUN mkdir -p node_modules/@freeway \
&& ln -s ../../packages/core node_modules/@freeway/core \
&& ln -s ../../packages/gateway node_modules/@freeway/gateway \
&& ln -s ../../packages/sdk node_modules/@freeway/sdk \
&& chown -R node:node node_modules
USER node
EXPOSE 8787
# Bind all interfaces inside the container; docker-compose maps it to localhost.
ENV HOST=0.0.0.0 PORT=8787
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD node -e "fetch('http://127.0.0.1:'+(process.env.PORT||8787)+'/healthz').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"
CMD ["node", "packages/gateway/dist/bin.js"]