Skip to content

Make cacheMaxMemorySize: 0 and custom cache handlers fast in dev#94784

Merged
unstubbable merged 2 commits into
canaryfrom
hl/dev-tiered-cached-handler
Jun 16, 2026
Merged

Make cacheMaxMemorySize: 0 and custom cache handlers fast in dev#94784
unstubbable merged 2 commits into
canaryfrom
hl/dev-tiered-cached-handler

Conversation

@unstubbable

Copy link
Copy Markdown
Contributor

When Cache Components is enabled, next dev treats a 'use cache' value as a miss and renders as if the cache were empty whenever the read does not resolve right away. Two development configurations triggered that even for values that were already cached, so warm reloads streamed slowly instead of serving the cached value: cacheMaxMemorySize: 0 replaced the built-in default handler with a no-op stub, so nothing was cached at all, and custom cache handlers with a slow or remote get did not return in time.

For the size-0 case, development now uses a real in-memory handler instead of the no-op stub, and the 'use cache' wrapper forces a dynamic cache life (revalidate: 0, a 5-minute expire) for it, the same treatment private caches already receive, so every read serves the stale entry and re-warms a fresh one in the background. This also fixes the dev private handler, which was sized from cacheMaxMemorySize and so degraded to the no-op stub whenever cacheMaxMemorySize: 0 was set.

Custom cache handlers keep their configured cache life, since their backing owns it. Instead we put a fast built-in in-memory front handler in front of the configured one through the new TieredCacheHandler, which serves warm reads from the front, writes through to both tiers, and reconciles the front against the backing in the background, evicting the front entry when the backing no longer has it (the handler interface has no per-key delete, so it overwrites the entry with an already-expired copy). These dev-only handlers are kept out of the registered handler set and merged in only where tag operations iterate, so revalidateTag still reaches them.

Everything is gated on process.env.__NEXT_DEV_SERVER, so production is unchanged: cacheMaxMemorySize: 0 still caches nothing, private entries are still never persisted, and configured handlers are used directly. New development test suites cover the size-0 and custom-handler behavior.

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Tests Passed

Commit: d9732ce

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Stats cancelled

Commit: d9732ce
View workflow run

@unstubbable unstubbable force-pushed the hl/dev-tiered-cached-handler branch from 1c8d394 to 1a02771 Compare June 13, 2026 00:00
@unstubbable unstubbable changed the base branch from canary to hl/shell-shock June 13, 2026 00:00
@unstubbable unstubbable force-pushed the hl/dev-tiered-cached-handler branch from 1a02771 to 21e5258 Compare June 13, 2026 10:22
@unstubbable unstubbable force-pushed the hl/dev-tiered-cached-handler branch from 21e5258 to 5f096c7 Compare June 13, 2026 10:29
@unstubbable unstubbable marked this pull request as ready for review June 13, 2026 11:05
Comment thread packages/next/src/server/use-cache/tiered-cache-handler.ts Outdated
expire: 0,
value: new ReadableStream({
start(controller) {
controller.enqueue(new Uint8Array(1))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have a whole byte to play with, imho we should pick a funnier number like 42

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you testing whether I let my agent auto-address review feedback? 😉

@lubieowoce lubieowoce Jun 13, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah he figured it out, pack it up everybody

well, i'd assume you wouldn't outsource making something funny to an agent, that's way too important!

@unstubbable unstubbable force-pushed the hl/dev-tiered-cached-handler branch from 5f096c7 to a3d155f Compare June 13, 2026 13:07
Base automatically changed from hl/shell-shock to canary June 13, 2026 13:47
@unstubbable unstubbable force-pushed the hl/dev-tiered-cached-handler branch 2 times, most recently from 58c3656 to 1ff5b49 Compare June 13, 2026 14:16
@unstubbable unstubbable requested a review from lubieowoce June 13, 2026 19:42
When Cache Components is enabled, `next dev` treats a `'use cache'`
value as a miss and renders as if the cache were empty whenever the read
does not resolve right away. Two development configurations triggered
that even for values that were already cached, so warm reloads streamed
slowly instead of serving the cached value: `cacheMaxMemorySize: 0`
replaced the built-in default handler with a no-op stub, so nothing was
cached at all, and custom cache handlers with a slow or remote `get` did
not return in time.

For the size-0 case, development now uses a real in-memory handler
instead of the no-op stub, and the `'use cache'` wrapper forces a
dynamic cache life (`revalidate: 0`, a 5-minute `expire`) for it, the
same treatment private caches already receive, so every read serves the
stale entry and re-warms a fresh one in the background. This also fixes
the dev private handler, which was sized from `cacheMaxMemorySize` and
so degraded to the no-op stub whenever `cacheMaxMemorySize: 0` was set.

Custom cache handlers keep their configured cache life, since their
backing owns it. Instead we put a fast built-in in-memory front handler
in front of the configured one through the new `TieredCacheHandler`,
which serves warm reads from the front, writes through to both tiers,
and reconciles the front against the backing in the background, evicting
the front entry when the backing no longer has it (the handler interface
has no per-key delete, so it overwrites the entry with an
already-expired copy). These dev-only handlers are kept out of the
registered handler set and merged in only where tag operations iterate,
so `revalidateTag` still reaches them.

Everything is gated on `process.env.__NEXT_DEV_SERVER`, so production is
unchanged: `cacheMaxMemorySize: 0` still caches nothing, private entries
are still never persisted, and configured handlers are used directly.
New development test suites cover the size-0 and custom-handler
behavior.
@unstubbable unstubbable force-pushed the hl/dev-tiered-cached-handler branch from 1ff5b49 to 6075919 Compare June 16, 2026 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants