Shared gateway-core, resume (coming soon), expanded tests & docs#590
Merged
Conversation
… docs
Consolidate the AI Gateway logic shared across the Cloudflare AI packages into a
new private, source-only workspace package (`@cloudflare/gateway-core`) that is
bundled inline into each consumer — no new runtime dependency. It houses the
provider registry, `cf-aig-*` header building, the resumable-stream engine, and
Workers AI SSE helpers, so routing/header/resume behavior stays in lockstep.
workers-ai-provider:
- AI Gateway delegate gains cross-vendor server-side fallback
(`fallback: { mode: "server" }`), header parity on the gateway path
(cacheKey/eventId/requestTimeoutMs/retries), and new `byokAlias` / `zdr`
universal-endpoint controls.
- Registry/header/SSE/resume logic now derive from gateway-core.
ai-gateway-provider:
- No longer deprecated/feature-frozen. Adds `byokAlias` / `zdr`, emits current
`cf-aig-cache-ttl` / `cf-aig-skip-cache` header names, and adds opt-in
resumable streaming on the binding/run path (shared engine).
- Provider list now derives from gateway-core; source simplified. `AiGatewayReties`
renamed to `AiGatewayRetries` (old name kept as a deprecated alias).
- type-check now covers test files; adds a creds-gated live e2e suite.
@cloudflare/tanstack-ai:
- Resumable streaming on the Workers AI catalog run path, gpt-oss forced
tool-call salvage, and `@tanstack/ai` peer bumps (multimodal MediaPrompt).
- Standardize the `format` script on oxfmt (drop the stray Biome usage).
Resume rollout:
- Resumable streaming is implemented but NOT generally available yet, so it is
marked "coming soon" across docs, package READMEs, changesets, and examples.
Docs:
- New `docs/` tree (concepts + per-package guides) with enriched examples; no
internal `gateway-core` leakage. Removes the internal test-coverage matrix.
Tooling:
- Standardize on oxlint + oxfmt; remove remaining Biome references.
Testing:
- Substantial new unit/integration/e2e coverage across all packages
(parse-options, auth, fallback, errors, request-shaping, resume,
tools/structured, abort, REST media, gateway providers).
Changesets included for the three published packages; gateway-core is private.
Co-authored-by: Cursor <cursoragent@cursor.com>
🦋 Changeset detectedLatest commit: 6f36e52 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
- Remove husky and the pre-push git hook. The hook only re-ran `pnpm install` to verify the lockfile was stable, which CI already enforces via `pnpm install --frozen-lockfile`. Drops the `husky` devDependency and the `prepare` script. - Fix the `.gitignore` escaping (`\*.tsbuildinfo` -> `*.tsbuildinfo`) so TypeScript build-info artifacts are actually ignored. - Run oxfmt across the repo (`pnpm run format`). Note: husky had set local `core.hooksPath` to `.husky/_`; with the dir removed that path is now a no-op (no hooks run). Left git config untouched. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidates the AI Gateway logic shared across the Cloudflare AI packages into a new private, source-only workspace package,
@cloudflare/gateway-core, that is bundled inline into each consumer (no new runtime dependency). It houses the provider registry,cf-aig-*header building, the resumable-stream engine, and Workers AI SSE helpers — so routing/header/resume behavior stays in lockstep acrossworkers-ai-provider,ai-gateway-provider, and@cloudflare/tanstack-ai.On top of the consolidation, this PR revives
ai-gateway-provider, brings resumable streaming to all three packages (marked coming soon — see below), standardizes tooling on oxlint/oxfmt, and adds a large amount of test and documentation coverage.Important
Resumable streaming is implemented but NOT generally available yet — the AI Gateway resume backend is still rolling out. It is therefore marked "coming soon" across docs, package READMEs, changesets, and examples. The API is in place so it can be adopted early, but should be treated as experimental until rollout completes.
What changed
@cloudflare/gateway-core(new, private)cf-aig-*header builders, resumable-stream engine, and Workers AI SSE helpers.private: true, source-only, bundled into consumers via tsdown — not published, no changeset.workers-ai-providerfallback: { mode: "server" }), winner selected viacf-aig-step.cacheKey/eventId/requestTimeoutMs/retries) and two new universal-endpoint controls:byokAlias(cf-aig-byok-alias) andzdr(cf-aig-zdr).gateway-core.ai-gateway-providerbyokAlias/zdr; emits the currentcf-aig-cache-ttl/cf-aig-skip-cacheheader names (oldcf-cache-ttl/cf-skip-cachedropped).gateway-core; source simplified.AiGatewayReties→AiGatewayRetries(old name kept as a deprecated alias — non-breaking).tsconfignow type-checks test files; adds a creds-gated live e2e suite.@cloudflare/tanstack-ai@tanstack/aipeer bumps (multimodalMediaPrompt).formatscript standardized on oxfmt (dropped a stray Biome usage).Docs
docs/tree: concepts (gateway routing, binding vs REST, resume) + per-package guides, with enriched examples.gateway-coreleakage in user-facing docs. Removed the internal test-coverage matrix.Tooling
Testing
parse-options,auth,fallback,errors,request-shaping,resume,tools/structured,abort, REST media, gateway providers.Validation (local)
nx run-many -t build packages/*)tsc --noEmit, all packages incl. tests)test:ci)oxlint)no-underscore-danglewarnings, unrelated)Test plan
nx affected -t lint test:ci type-check build)pnpm --filter <pkg> test:e2e) where credentials are availableexamples/workers-aiandexamples/tanstack-aiapps locally (binding + REST)Notes
@cloudflare/gateway-coreis intentionally private/source-only and excluded from changesets/publish.workers-ai-provider,ai-gateway-provider,@cloudflare/tanstack-ai).Made with Cursor