Skip to content

Core: Write null for current-snapshot-id for V3+#12335

Merged
Fokko merged 6 commits into
apache:mainfrom
Fokko:fd-write-nulll
Mar 6, 2025
Merged

Core: Write null for current-snapshot-id for V3+#12335
Fokko merged 6 commits into
apache:mainfrom
Fokko:fd-write-nulll

Conversation

@Fokko

@Fokko Fokko commented Feb 19, 2025

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot added the core label Feb 19, 2025
Comment thread core/src/main/java/org/apache/iceberg/TableMetadataParser.java Outdated

@RussellSpitzer RussellSpitzer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit around putting version gating in a constant, lgtm otherwise

@Fokko Fokko merged commit f3b3ee4 into apache:main Mar 6, 2025
@Fokko

Fokko commented Mar 6, 2025

Copy link
Copy Markdown
Contributor Author

Moving this forward, thanks @nastra and @RussellSpitzer

@Fokko Fokko deleted the fd-write-nulll branch March 6, 2025 07:20
laskoviymishka added a commit to laskoviymishka/iceberg-go that referenced this pull request May 10, 2026
Closes apache#1002. v3 spec requires `"current-snapshot-id": null` for empty
tables, not the field omitted entirely (Java apache/iceberg#12335).
omitempty on the *int64 field was dropping the key when nil; metadataV3
now has a MarshalJSON that re-injects the key with explicit null after
the snapshots field — Java's declaration order — while preserving v1/v2
omitempty behaviour, since v1/v2 fixtures rely on the -1 sentinel.

The injection walks the marshaled JSON via json.Decoder so the
surrounding field order is preserved (a map round-trip would resort
alphabetically and break byte-level diffs against Java-produced
metadata.json).
laskoviymishka added a commit to laskoviymishka/iceberg-go that referenced this pull request May 10, 2026
Closes apache#1002. v3 spec requires `"current-snapshot-id": null` for empty
tables, not the field omitted entirely (Java apache/iceberg#12335).
omitempty on the *int64 field was dropping the key when nil; metadataV3
now has a MarshalJSON that re-injects it as explicit null at its
canonical declaration-order position — just before the first key that
follows it in commonMetadata — so empty-table output stays
byte-compatible with Java's metadata.json shape. v1/v2 keep omitempty
behaviour, since v1/v2 fixtures rely on the -1 sentinel.

The injection walks the marshaled JSON via json.Decoder so the
surrounding field order is preserved (a map round-trip would resort
alphabetically and break byte-level diffs against Java-produced
metadata.json).

Same MarshalJSON adds a symmetric guard for last-partition-id, which is
required for v2+ per spec but had the same omitempty tag. The parser's
preValidate/validate path already enforces non-nil on read; this is the
write-side equivalent so direct struct construction that skips the
builder can't silently emit a non-conformant file Java/PyIceberg would
reject. In practice the builder always sets the field, so the guard
only fires on malformed in-memory state.
laskoviymishka added a commit to laskoviymishka/iceberg-go that referenced this pull request May 13, 2026
Closes apache#1002. v3 spec requires `"current-snapshot-id": null` for empty
tables, not the field omitted entirely (Java apache/iceberg#12335).
omitempty on the shared commonMetadata field was dropping the key when
nil; metadataV3 now has a MarshalJSON that wraps the type alias inside
an outer struct redeclaring CurrentSnapshotID without omitempty, so the
field-name collision resolves to the shallower (always-emitted)
declaration. v1/v2 keep the omitempty path since their fixtures rely on
the -1 sentinel.

Same MarshalJSON adds a symmetric guard for last-partition-id, which is
required for v2+ per spec but had the same omitempty tag. The parser's
preValidate/validate path already enforces non-nil on read; this is the
write-side equivalent so direct struct construction that skips the
builder can't silently emit a non-conformant file Java/PyIceberg would
reject. In practice the builder always sets the field, so the guard
only fires on malformed in-memory state.
laskoviymishka added a commit to laskoviymishka/iceberg-go that referenced this pull request May 17, 2026
Closes apache#1002. v3 spec requires `"current-snapshot-id": null` for empty
tables, not the field omitted entirely (Java apache/iceberg#12335).
omitempty on the shared commonMetadata field was dropping the key when
nil; metadataV3 now has a MarshalJSON that wraps the type alias inside
an outer struct redeclaring CurrentSnapshotID without omitempty, so the
field-name collision resolves to the shallower (always-emitted)
declaration. v1/v2 keep the omitempty path since their fixtures rely on
the -1 sentinel.

Same MarshalJSON adds a symmetric guard for last-partition-id, which is
required for v2+ per spec but had the same omitempty tag. The parser's
preValidate/validate path already enforces non-nil on read; this is the
write-side equivalent so direct struct construction that skips the
builder can't silently emit a non-conformant file Java/PyIceberg would
reject. In practice the builder always sets the field, so the guard
only fires on malformed in-memory state.
laskoviymishka added a commit to apache/iceberg-go that referenced this pull request May 17, 2026
Closes #1002. v3 spec requires `"current-snapshot-id": null` for empty
tables, not the field omitted entirely (Java apache/iceberg#12335).
omitempty on the *int64 field was dropping the key when nil; metadataV3
now has a MarshalJSON that re-injects the key with explicit null after
the snapshots field — Java's declaration order — while preserving v1/v2
omitempty behaviour, since v1/v2 fixtures rely on the -1 sentinel.

The injection walks the marshaled JSON via json.Decoder so the
surrounding field order is preserved (a map round-trip would resort
alphabetically and break byte-level diffs against Java-produced
metadata.json).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Serialize null for current-snapshot-id when there is no current snapshot for ≥V3

3 participants