Skip to content

Use bolt hashmap freelist for metadata databases - #6943

Merged
tonistiigi merged 2 commits into
moby:masterfrom
thaJeztah:bolt_bolder
Jul 17, 2026
Merged

Use bolt hashmap freelist for metadata databases#6943
tonistiigi merged 2 commits into
moby:masterfrom
thaJeztah:bolt_bolder

Conversation

@thaJeztah

Copy link
Copy Markdown
Member

bolt documents the array freelist as potentially degrading significantly
for large fragmented databases, while the hashmap freelist is faster in
almost all circumstances. From the DB.FreelistType GoDoc:

// FreelistType sets the backend freelist type. There are two options. Array which is simple but endures
// dramatic performance degradation if database is large and fragmentation in freelist is common.
// The alternative one is using hashmap, it is faster in almost all circumstances
// but it doesn't guarantee that it offers the smallest page id available. In normal case it is safe.
// The default type is array
FreelistType FreelistType

While the default is still FreelistArrayType, the package shows that
the intent is to make FreelistMapType the default in future;
https://pkg.go.dev/go.etcd.io/bbolt@v1.5.0#pkg-constants

// TODO(ahrtr): eventually we should (step by step)
//  1. default to `FreelistMapType`;
//  2. remove the `FreelistArrayType`, do not export `FreelistMapType`
//     and remove field `FreelistType' from both `DB` and `Options`;

This keeps the on-disk format unchanged and only changes the in-memory
freelist implementation used after opening the DB.

@tonistiigi

Copy link
Copy Markdown
Member

cc @ZRHann via #6882

@thaJeztah

Copy link
Copy Markdown
Member Author

Doh; messed up a line; looks like I didn't push that fix; will do tomorrow

github.com/moby/buildkit/worker/runc
92.00 worker/runc/runc.go:100:4: syntax error: unexpected ) at end of statement

All imports aliased it, except for one.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
bolt documents the array freelist as potentially degrading significantly
for large fragmented databases, while the hashmap freelist is faster in
almost all circumstances. From the [DB.FreelistType] GoDoc:

    // FreelistType sets the backend freelist type. There are two options. Array which is simple but endures
    // dramatic performance degradation if database is large and fragmentation in freelist is common.
    // The alternative one is using hashmap, it is faster in almost all circumstances
    // but it doesn't guarantee that it offers the smallest page id available. In normal case it is safe.
    // The default type is array
    FreelistType FreelistType

While the default is still `FreelistArrayType`, the package shows that
the intent is to make `FreelistMapType` the default in future;
https://pkg.go.dev/go.etcd.io/bbolt@v1.5.0#pkg-constants

    // TODO(ahrtr): eventually we should (step by step)
    //  1. default to `FreelistMapType`;
    //  2. remove the `FreelistArrayType`, do not export `FreelistMapType`
    //     and remove field `FreelistType' from both `DB` and `Options`;

This keeps the on-disk format unchanged and only changes the in-memory
freelist implementation used after opening the DB.

[DB.FreelistType]: https://pkg.go.dev/go.etcd.io/bbolt@v1.5.0#DB.FreelistType

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@tonistiigi tonistiigi added this to the v0.32.0 milestone Jul 15, 2026

@tonistiigi tonistiigi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks safe to me.

@tonistiigi
tonistiigi merged commit d2698d3 into moby:master Jul 17, 2026
192 checks passed
@thaJeztah
thaJeztah deleted the bolt_bolder branch July 17, 2026 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants