fix(models): report accurate limits in OpenRouter scans - #110855
Conversation
|
Codex review: needs maintainer review before merge. Reviewed August 3, 2026, 2:10 PM ET / 18:10 UTC. ClawSweeper reviewWhat this changesThe PR updates Merge readinessThis PR remains necessary: current Priority: P2 Review scores
Verification
How this fits togetherThe model scan fetches OpenRouter’s public model catalog and converts each entry into scan metadata for CLI output and optional capability probes. Normalized context and completion limits flow from the catalog parser into the displayed scan result and the probe model configuration. flowchart LR
A[OpenRouter model catalog] --> B[Model scan fetch]
B --> C[Limit normalization]
C --> D[Scan model records]
D --> E[CLI scan output]
D --> F[Capability probes]
Before merge
Agent review detailsSecurityNone. PR surfaceSource 0, Tests +102. Total +102 across 2 files. View PR surface stats
Review metrics
Stored data modelPersistent data-model change detected: Technical reviewBest possible solution: Merge this focused repair after the normal exact-head gates complete, keeping OpenRouter limit normalization centralized on the canonical positive-safe-integer and record-coercion contracts rather than adding a display-only or probe-only workaround. Do we have a high-confidence way to reproduce the issue? Yes, source-reproducible: current main demonstrably ignores Is this the best way to solve the issue? Yes. Parsing the provider payload once at the scan-record owner boundary fixes both output and optional probes; display-only or probe-only adjustments would leave the other consumer inconsistent. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against a67c52611e57. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (10 earlier review cycles; latest 8 shown)
|
1089411 to
df1009e
Compare
|
The precedence flip is right and it converges on the in-repo sibling, but // packages/normalization-core/src/number-coercion.ts:2-4
export function asFiniteNumber(value: unknown): number | undefined {
return typeof value === "number" && Number.isFinite(value) ? value : undefined;
}For contextWindow: entry.contextLength ?? baseModel.contextWindow,
maxTokens: entry.maxCompletionTokens ?? baseModel.maxTokens,
I want to be careful about how much this proves, because the loose validation isn't new — And I can't show you a captured OpenRouter payload containing On tests: the added fixture covers None of this is a blocker; reading |
|
@mushuiyu886 This morning's ClawSweeper pass turned the Source change. import {
- asFiniteNumber,
asDateTimestampMs,
+ asPositiveSafeInteger,
resolveTimerTimeoutMs,
} from "@openclaw/normalization-core/number-coercion";
const contextLength =
- asFiniteNumber(topProvider?.context_length) ??
- asFiniteNumber(obj.context_length) ??
+ asPositiveSafeInteger(topProvider?.context_length) ??
+ asPositiveSafeInteger(obj.context_length) ??
null;
const maxCompletionTokens =
- asFiniteNumber(topProvider?.max_completion_tokens) ??
- asFiniteNumber(obj.max_completion_tokens) ??
- asFiniteNumber(obj.max_output_tokens) ??
+ asPositiveSafeInteger(topProvider?.max_completion_tokens) ??
+ asPositiveSafeInteger(obj.max_completion_tokens) ??
+ asPositiveSafeInteger(obj.max_output_tokens) ??
null;Measured, same tree, same fixtures. Top-level
The two control rows are the ones that matter for scope: a well-formed Fixtures, including the mixed-field case the review asks for. Both fail on your current head — it("falls back when top-provider limits are malformed", async () => {
for (const topProvider of [
{ context_length: 0, max_completion_tokens: 0 },
{ context_length: -1, max_completion_tokens: -1 },
{ context_length: 8192.5, max_completion_tokens: 8192.5 },
{ context_length: null, max_completion_tokens: null },
]) {
const fetchImpl = createFetchFixture({
data: [
{
id: "acme/provider-limited:free",
name: "Provider Limited",
context_length: 32_768,
max_completion_tokens: 4096,
top_provider: topProvider,
supported_parameters: [],
pricing: { prompt: "0", completion: "0" },
},
],
});
const [result] = await scanOpenRouterModels({ fetchImpl, probe: false });
expect(result?.contextLength).toBe(32_768);
expect(result?.maxCompletionTokens).toBe(4096);
}
});
it("mixes provider context length with a top-level completion cap", async () => {
const fetchImpl = createFetchFixture({
data: [
{
id: "acme/provider-limited:free",
name: "Provider Limited",
context_length: 32_768,
max_completion_tokens: 4096,
top_provider: { context_length: 16_384, max_completion_tokens: 0 },
supported_parameters: [],
pricing: { prompt: "0", completion: "0" },
},
],
});
const [result] = await scanOpenRouterModels({ fetchImpl, probe: false });
expect(result?.contextLength).toBe(16_384);
expect(result?.maxCompletionTokens).toBe(4096);
});Validation with everything applied: Take it or adapt it — I'm not attached to the fixture wording, and the |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
@mushuiyu886 Heads-up: this branch started conflicting a few hours ago, and the review comment above predates it — it says GitHub reports the PR mergeable, but the API now returns What moved. Three-way merge against The conflicting hunk is your two limit blocks against the re-indented ones. No logic disagreement — both sides changed the same lines for unrelated reasons. The fix is still needed. Current const contextLength =
typeof obj.context_length === "number" && Number.isFinite(obj.context_length)
? obj.context_length
: null;So the top-level-only behaviour your PR corrects is exactly what Resolution. import {
asDateTimestampMs,
+ asPositiveSafeInteger,
resolveTimerTimeoutMs,
} from "@openclaw/normalization-core/number-coercion";Then, at const topProvider =
obj.top_provider && typeof obj.top_provider === "object"
? (obj.top_provider as Record<string, unknown>)
: undefined;
const contextLength =
asPositiveSafeInteger(topProvider?.context_length) ??
asPositiveSafeInteger(obj.context_length) ??
null;
const maxCompletionTokens =
asPositiveSafeInteger(topProvider?.max_completion_tokens) ??
asPositiveSafeInteger(obj.max_completion_tokens) ??
asPositiveSafeInteger(obj.max_output_tokens) ??
null;I applied exactly that on top of current What I did not check: I did not run the test file through the merged tree, so treat the source resolution as verified and the suite as still owing a run once you push. Your existing |
|
Resolved the current-main conflict in Post-merge verification:
Exact-head CI passed every product/check lane except I attempted to rerun the failed jobs, but GitHub rejected the contributor token because reruns require repository admin rights. A maintainer rerun of https://github.com/openclaw/openclaw/actions/runs/30747370605 is the remaining CI action; no product-code change is indicated by this failure. |
7dde713 to
d3c9ac5
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
Maintainer review complete for exact head
Proceeding through the repository's native prepare and merge verification flow. |
|
Merged via squash.
|
* origin/main: (25 commits) test(qa): cover plugin authoring contracts (#118821) fix(parallels): keep provider keys out of POSIX job logs (#118840) improve: reduce redundant Code Mode test work (#118815) fix(models): report primary OpenRouter limits in scans (#110855) test: speed up setup inference fixtures (#118811) fix(auto-reply): enforce canonical reset authorization (#118580) docs(gateway): fix invalid heartbeat visibility examples (#118827) fix(plugins): unify HTTP route conflict handling (#118203) test(agents): use prepared auth fixtures (#118816) fix(lmstudio): resolve JIT embedding variants fix(lmstudio): preserve embedding preload and model identity fix(ai): reject binary Codex websocket frames (#111138) fix(deps): bump brace-expansion override to 5.0.9 for HIGH advisory 1130705 (#118804) feat(cli): add session archive and delete commands (#118791) fix(openai): commit eligible final realtime transcription audio safely (#118782) fix(reef): restrict management commands to owners [AI] (#118578) fix(google): honor Cloud SDK credential location and Vertex billing project (#118745) fix(bedrock): preserve private embedding endpoints and AWS routing policy (#118744) fix(voice): make signed callback replay reservations durable and retryable (#118754) fix(openrouter): isolate custom proxy credentials and transport security (#118773) ...
…-state-mutation * origin/main: (22 commits) fix(delivery): renew stable producer leases (#118663) test(mcp): isolate catalog size checks from wall-clock load (#118871) test(ui): add credentials primary QA proof (#118790) fix(slack): prevent stalled presence polling from hanging shutdown (#117478) test(security): cover gateway shared auth modes (#118834) fix(macos): show gateway auth failures in app status (#118841) test(tui): route PTY gateway scenarios through explicit models (#118802) test(qa): add session transcript primary coverage (#118820) fix(browser): protect private observation media and Canvas trust boundaries (#118775) fix(build): restore SDK packages and signed macOS app builds (#118833) test(qa): cover plugin authoring contracts (#118821) fix(parallels): keep provider keys out of POSIX job logs (#118840) improve: reduce redundant Code Mode test work (#118815) fix(models): report primary OpenRouter limits in scans (#110855) test: speed up setup inference fixtures (#118811) fix(auto-reply): enforce canonical reset authorization (#118580) docs(gateway): fix invalid heartbeat visibility examples (#118827) fix(plugins): unify HTTP route conflict handling (#118203) test(agents): use prepared auth fixtures (#118816) fix(lmstudio): resolve JIT embedding variants ...
What problem this solves
openclaw models scanread only OpenRouter's catalog-wide context and outputlimits. When the primary provider published smaller limits under
top_provider, scan output could overstate the metadata used for display andoptional capability probes.
Canonical fix
top_provider.context_length,max_completion_tokens, andmax_output_tokens.extensions/openrouter/provider-catalog.ts.top_providerdescribes OpenRouter's primary-provider configuration; it is nota guarantee that every routed request will use one fixed provider.
No config, Plugin SDK, protocol, persistence, migration, or feature surface
changes.
Evidence
Exact rewritten head:
d3c9ac59bbb95539d76b1dd06b58b130fa9bd74fgoogle/gemma-4-26b-a4b-it:freereported catalog context262144and no completion cap.131072andcompletion cap
32768.131072and completion cap32768.fractional, unsafe, string, and null provider values; independent fallback;
and
max_output_tokens.git diff --check: clean.issues; correctness confidence 0.99; TruffleHog clean.
maina67c52611e5768ffe89dc2c4eea0cea0e26e34e6.Scope
Production delta:
+10/-10(net zero) insrc/agents/model-scan.ts.Tests add 102 lines.
Changelog not required: this corrects existing internal catalog normalization
without changing commands or configuration.