fix: catalog previews cut Chinese and Japanese summaries at the first Latin word - #3362
Conversation
… Latin word truncateText backtracks to the last space in the slice unconditionally. Scripts that do not separate words with spaces usually carry a single Latin space near the start of a summary, so that backtrack discards nearly the whole preview: across fixtures/public-corpus/corpus.jsonl, 25 catalog entries render with a handful of characters instead of their budget, one of them as just "|". Honour the word boundary only when it keeps most of the slice. All 1362 space-separated previews in the same corpus are unchanged.
|
@Yigtwxx is attempting to deploy a commit to the OpenClaw Foundation Team on Vercel. A member of the Team first needs to authorize it. |
|
Codex review: needs maintainer review before merge. Reviewed August 5, 2026, 5:43 PM ET / 21:43 UTC. ClawSweeper reviewWhat this changesThis PR changes catalog preview truncation so CJK summaries with an early Latin-space token retain the available preview text, and adds regression tests. Merge readinessKeep open. Current main and the latest release retain the unconditional space-backtracking behavior; this PR supplies a bounded shared-helper fix, regression tests, and sufficient real-browser evidence for ordinary maintainer merge review. Priority: P2 Review scores
Verification
How this fits togetherCatalog skill and plugin records feed shared card, search, publisher, home, and dashboard views. Those views use one frontend truncation helper to turn names and summaries into compact rendered previews. flowchart LR
A[Catalog records] --> B[Card and search views]
B --> C[Shared preview truncation]
C --> D[Word-boundary decision]
D --> E[Rendered catalog previews]
Before merge
Agent review detailsSecurityNone. Review metrics
Technical reviewBest possible solution: Land the shared-helper repair so every catalog preview surface handles CJK summaries consistently without duplicating per-card logic. Do we have a high-confidence way to reproduce the issue? Yes—current main’s unconditional last-space backtrack is source-proven, and the PR provides a concrete real-browser before/after path using live catalog records. Is this the best way to solve the issue? Yes—the shared helper is the narrowest maintainable layer because all affected card and listing views already route through it, while tests preserve the established word-boundary behavior for space-separated text. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 6381d789ab18. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (18 earlier review cycles; latest 8 shown)
|
Patrick-Erichsen
left a comment
There was a problem hiding this comment.
This fixes CJK catalog previews by avoiding a space backtrack when that boundary would discard most of the summary. The shared-helper implementation and regression coverage look correct, but the user-visible result has not been proven in ClawHub.
LOC: +77/-1 (2 files)
Finding: add a real-browser before/after capture from a running ClawHub instance with representative Chinese and Japanese fixtures.
Best-fix verdict: best pending visual proof. The shared preview helper is the right layer and preserves the old word-boundary behavior when it still yields a substantial preview.
Alternatives considered: special-casing CJK call sites duplicates formatting logic; disabling word-boundary truncation globally regresses Latin previews.
Code read: shared catalog preview helper, its established consumers, and added jsdom/corpus cases.
Remaining uncertainty: actual card layout and CSS behavior in the browser.
The kept-ratio fallback was added for CJK summaries whose only Latin space sits near the start, but it applied to every script. A space-separated summary ending in a long token — a URL, a compound word — lost its word boundary and was cut mid-token instead. Gate the ratio on the discarded tail actually being non-spacing script, reusing the character class the catalog search tokenizer already relies on in convex/lib/searchText.ts.
|
@Patrick-Erichsen thanks for the review. The real-browser capture you asked for is now in the PR, added in c53b786 under Evidence -> Real-browser before/after on a running ClawHub. How it was produced: Two comparisons are included:
That second commit also tightened the kept-ratio condition itself. Both halves of Ready for another look when you have time. |
What Problem This Solves
Fixes an issue where visitors browsing the catalog see Chinese, Japanese and Korean
summaries cut down to a handful of characters. The trigger is a summary that opens
with a Latin token — a product name, a protocol, a version number — followed by text
in a script that does not separate words with spaces.
The affected surface is every catalog card rendered through
truncateText: the homelisting sections, the skills and plugins catalogs, the search results list, publisher
cards, and the dashboard catalog view.
Why This Change Was Made
truncateTextslices to the budget and then backtracks to the last space inside thatslice, so a preview never ends mid-word. The rule assumes spaces mark word boundaries
throughout the text. In Chinese, Japanese and Korean they usually do not, and often the
only space in an entire summary sits immediately after an opening Latin token — so the
backtrack rewinds past everything else.
Taking
5gc-automationfromfixtures/public-corpus/corpus.jsonl: the summary is 104characters and its only space follows
5GC. At the 100-character budget the cardrenders
5GC….huangli-query-cnis worse — its preview is the single character|.The word-boundary rule is still right for space-separated text, so it is kept and only
skipped when two conditions hold at once: honouring it would discard most of the slice,
and the part being discarded is actually non-spacing script. The second condition keeps
the fallback off Latin text, and the character class it tests is the one the catalog
search tokenizer already uses (
CJK_REinconvex/lib/searchText.ts, character forcharacter). When the fallback does apply, the preview is cut at the budget, which is what
the repository's other two truncators already do:
convex/publisherAbuse.ts—value.slice(0, maxLength).trimEnd()convex/lib/skillSummary.ts—compact.slice(0, MAX_SUMMARY_CHARS - 3).trimEnd()src/lib/truncateText.tsis the only place in the repository that backtracks to aspace;
git grep 'lastIndexOf(" ")'returns one hit.Non-goals, kept out to hold the change to one concern:
src/andconvex/. The three call sites nowagree on behavior; consolidating them is a separate refactor.
User Impact
Catalog cards for Chinese, Japanese and Korean skills now show the same amount of text
as cards for English ones. Previously a publisher whose summary opened with a Latin
product name got a preview that carried none of their description, and a visitor
scanning the catalog had nothing to read on those rows. Cards for space-separated text
are byte-for-byte unchanged.
Evidence
Base commit:
a9d04bb0. The fix is not onmain—truncateTextthere still applieslastIndexOf(" ")unconditionally, and the file has no test.Measured against the repository's own corpus
fixtures/public-corpus/corpus.jsonl, 1250 real catalog records, run through the actualcall budgets (
displayName70,summary80 and 100):All 1362 space-separated previews in that corpus produce identical output before and
after, so the word-boundary behavior English summaries rely on is untouched.
Read out of a mounted
SkillCardSkillCardrendered with@testing-library/react, reading.skill-card-titleand.skill-card-summaryfrom the DOM. Same props on both sides; onlytruncateTextdiffers.
main5gc-automation5GC…— 4 charsclaude-code-evolution实施Claude…— 9 charsdeploy-helperdeploy-helperRendered summary on
main:and with this change:
Real-browser before/after on a running ClawHub
The measurements above are source-level. This one is the rendered catalog in a real
browser, against real catalog data - no jsdom, no fixture stubbing.
Setup:
bun run buildthenbun run preview, pointed at the same public Convexdeployment the repository's own Playwright job uses
(
VITE_CONVEX_URL=https://wry-manatee-359.convex.cloud). Chromium via Playwright thenloads
/skills?tab=trending,/skills?tab=newand/, and reads back every renderedpreview that ends in an ellipsis. Both arms are full rebuilds of the same tree; the only
difference is
src/lib/truncateText.ts.Two comparisons were run against the same three pages.
main(a9d04bb) against this branch — the CJK repair on real cards. Three renderedpreviews differ, all three CJK, each one going from a truncated fragment to the full
80-character budget of the card that renders them:
Every one of them is a live catalog record cut at the Latin space that follows its opening
token. No Latin-script preview differs between these two arms, so the word-boundary
behavior English cards rely on is untouched on real data as well.
The reviewed head (0b51dc3) against this branch — the review finding on real cards.
Both arms captured 31 truncated previews, 8 of them CJK. Exactly one preview differs,
and it is a real catalog record rather than a constructed string:
The reviewed head cut inside a URL query string; the boundary is restored. The other 22
Latin previews were already ending on a word and are unchanged, and all 8 CJK previews are
byte-identical across these two arms - which is the point: the CJK repair shown in the
first comparison survives this revision untouched. The record opens with an emoji, redacted to
<emoji>above; everythingelse in those two lines is reproduced exactly as rendered, including the trailing
ellipsis the helper emits.
Focused tests
src/lib/truncateText.test.tsis new — the module had no test file, despite beinginside the coverage
includeglob. On the parent commit the three budget assertionsfail and the five contract assertions pass:
With the change:
The two tests added in the latest revision cover the review finding on the kept-ratio
fallback: one asserts a long Latin word keeps its boundary, the other asserts a short CJK
tail is still discarded in favour of that boundary. Both halves of the condition are
load-bearing - replacing
keepsMostOfSlice || !discardsCJKwith either half alone turnsexactly one of them red:
CI
bun run lint,bun run deadcode:ci,bun run check:peers,bun audit,bun run llms:check,bun run check:release-workflow-action-pins— all pass.bun run format:checkfails onCLAUDE.mdand.agents/skills/autoreview/CLAUDE.md. That is pre-existing: the same two files, andonly those two, fail on
mainata9d04bb0. The two files in this PR are formattedwith
oxfmt.VITE_CONVEX_URL=https://example.invalid bun run ci:uniton this branch:14 failed | 431 passed | 1 skipped (446)files,24 failed | 5781 passedtests.On
main:14 failed | 430 passed | 1 skipped (445)files,24 failed | 5773 passedtests. The sorted list of failing files is identical on bothsides — they are the suites that shell out to
bun, plusconvex/lib/githubAccount.test.tsandsrc/routes/-management.test.tsx. This branchadds one file and ten passing tests and introduces no new failure.
The Vercel check on this PR stays pending OpenClaw Foundation authorization, as it does
on every fork PR.
Written with AI assistance. I ran the focused tests, the corpus measurement and the
mounted-component render above myself, diffed the full unit suite against
main, andcan maintain this code.