Aryan SinghandClaude Opus 5
cb985dcf9c
One endpoint in front of every free LLM tier, with automatic failover and no provider key ever leaving the box. Providers are data, not code: adding one is a JSON file plus an env var, never a TypeScript change. Ships 12 providers pre-written; `verify-provider --apply` derives the live model list from your own key. - core: registry, bucketed quota meter (rolling + UTC-calendar windows), key-pool store, 6-step model router with diagnostic rejection reasons - gateway: node:http, failover that commits to a stream only after 200 headers, SSE passthrough with usage sniffing, virtual keys with admin/inference roles - context engine: sqlite sessions and a 5-rung refit ladder, so a 128k thread keeps working when it fails over to an 8k model - cache: exact (sha256, safe-by-default) + opt-in semantic (flat Float32Array) - guard: inbound limits, quota forecasting, limit calibration from upstream headers, key health, secret/PII/injection scanning - dashboard: one self-contained HTML file, live SSE request log, token usage - sdk: zero-dep typed client, drop-in for the OpenAI SDK Zero runtime dependencies. 326 tests, 27-check smoke suite against a scripted mock upstream so routing is testable without spending quota. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
63 lines
2.8 KiB
Docker
63 lines
2.8 KiB
Docker
# Zero runtime dependencies means the runtime stage needs nothing but Node and
|
|
# the compiled output — no node_modules to copy, nothing to audit at runtime.
|
|
|
|
# ---- build ---------------------------------------------------------------
|
|
FROM node:24-alpine AS build
|
|
WORKDIR /app
|
|
|
|
# Dev dependencies are only typescript + @types/node, so this layer is small and
|
|
# caches well across source edits.
|
|
COPY package.json package-lock.json* ./
|
|
COPY packages/core/package.json ./packages/core/
|
|
COPY packages/gateway/package.json* ./packages/gateway/
|
|
COPY packages/sdk/package.json* ./packages/sdk/
|
|
RUN npm ci --ignore-scripts --no-audit --no-fund
|
|
|
|
COPY tsconfig.json tsconfig.base.json ./
|
|
COPY packages ./packages
|
|
RUN npm run build
|
|
|
|
|
|
# ---- runtime -------------------------------------------------------------
|
|
FROM node:24-alpine AS runtime
|
|
ENV NODE_ENV=production
|
|
WORKDIR /app
|
|
|
|
# `node` (uid 1000) ships with the image. Running as root would let a gateway
|
|
# compromise rewrite the mounted providers/ directory.
|
|
RUN mkdir -p /app/data && chown -R node:node /app
|
|
|
|
# Only the compiled output and each package's manifest — no sources, no tests.
|
|
COPY --from=build --chown=node:node /app/packages/core/dist ./packages/core/dist
|
|
COPY --from=build --chown=node:node /app/packages/core/package.json ./packages/core/
|
|
COPY --from=build --chown=node:node /app/packages/gateway/dist ./packages/gateway/dist
|
|
COPY --from=build --chown=node:node /app/packages/gateway/package.json ./packages/gateway/
|
|
COPY --from=build --chown=node:node /app/packages/gateway/public ./packages/gateway/public
|
|
COPY --from=build --chown=node:node /app/packages/sdk/dist ./packages/sdk/dist
|
|
COPY --from=build --chown=node:node /app/packages/sdk/package.json ./packages/sdk/
|
|
|
|
COPY --chown=node:node package.json ./
|
|
COPY --chown=node:node providers ./providers
|
|
COPY --chown=node:node freeway.config.json ./freeway.config.json
|
|
|
|
# `@freeway/core` resolves through npm's workspace symlinks, which live in the
|
|
# root node_modules — a directory this image otherwise has no reason to carry,
|
|
# since there are no runtime dependencies. Recreating just the links keeps the
|
|
# resolution working without shipping the toolchain.
|
|
RUN mkdir -p node_modules/@freeway \
|
|
&& ln -s ../../packages/core node_modules/@freeway/core \
|
|
&& ln -s ../../packages/gateway node_modules/@freeway/gateway \
|
|
&& ln -s ../../packages/sdk node_modules/@freeway/sdk \
|
|
&& chown -R node:node node_modules
|
|
|
|
USER node
|
|
|
|
EXPOSE 8787
|
|
# Bind all interfaces inside the container; docker-compose maps it to localhost.
|
|
ENV HOST=0.0.0.0 PORT=8787
|
|
|
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
|
|
CMD node -e "fetch('http://127.0.0.1:'+(process.env.PORT||8787)+'/healthz').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"
|
|
|
|
CMD ["node", "packages/gateway/dist/bin.js"]
|