1979 Records with type annotations - #2566

The PR helped me reflect on further choices. Maybe we can treat them more independently:

First, I (now) think we should keep map:get and map:put unchanged. It can be a perfectly reasonable and deliberate choice to add a new entry to a record if it is regarded as a map. It is good that alternatives exist, such as map:put({ $map }, ...), but they are probably not very intuitive. Most importantly, though, the map prefix does not imply at all that records are to be treated differently under the hood. Finally, the behavior of other map functions will remain consistent. For example, we won’t need to care about consistency among other map functions, for example whether map:remove should raise an error.

Instead, I would suggest raising errors only for the lookup operator and dynamic function calls. In my experience, these two are used a lot, and interchangeably, for retrieving known entries, whereas map:get is mostly used for lookups of dynamic keys.

Next, I believe that a custom operator such as with is an appropriate solution to raise errors, even though there may be no record-specific function that backs the behavior (…yet, one never knows). Additionally, I would propose limiting this operator to records and raising an error for maps without record annotation. We already have so many other ways to create and update maps, and it might be helpful to let records evolve independently.

First, I (now) think we should keep map:get and map:put unchanged

There's certainly a substitutability argument that would favour that.

With the new rule, I would suggest dropping the following rule:

Entries in J whose key does not match the name of any field in R are discarded.

Otherwise, arbitrary maps can be coerced to arbitrary records (as the default value types are item()*), and code gets much more error-prone. An example:

let $r as record(lat, long) := { 'latitude': 0, 'longitude': 1 }

Similar as with other coercion rules, I think that existing data should not be dropped.

I have revised the PR in the light of points raised here, and in the discussion on 21 April 2026. The main changes are:

• Various obvious errors are corrected (for example WithExpr is now referenced in the grammar)
• The with operator now requires a record as its LH operand
• Dynamic function calls on records raise an error if the key is unknown
• Coercion to a record type no longer silently drops unwanted fields.

For the time being I have dropped the xsl:record instruction entirely. I will raise an issue to bring it back in a different form.

I have not (yet) made the changes to introduce "named typing" on record types. That is, the "instance of" test is still based on content comparison rather than on the type annotation. I have come to the conclusion that to use named typing for "instance of" tests, we also need to use some kind of named type relationship to test subsumption, which probably means we need to introduce a type hierarchy for named record types that includes derivation by restriction and/or extension. That's probably best handled in a separate proposal.

I tried generating parsers from the grammar as extended by this PR and ran into a couple of issues:

• the with keyword may need to be added to the exceptions in the unreserved-name constraint. Without that, I see LALR(2) shift/reduce conflicts for input such as with {

• even with that change, with appears to introduce additional LALR(2) conflicts when this grammar is combined with the XQuery Update Facility grammar. In particular, I see:

  LALR(2) conflict #1 (shift-reduce):
      WithExpr ::= InstanceofExpr ( 'with' InstanceofExpr )* .
      WithExpr ::= InstanceofExpr ( . 'with' InstanceofExpr )*
  

  This seems to arise from the coexistence of WithExpr with the following Update Facility grammar fragment:

  ReplaceExpr
           ::= 'replace' ( 'value' 'of' )? 'node' TargetExpr 'with' ExprSingle
  TargetExpr
           ::= ExprSingle
  

Separately, the XPath spec currently has a dead link to #parse-note-unreserved-name.

Interesting problem.

(We're going to hit the same issue with the Gizmo utility -- https://www.saxonica.com/documentation12/index.html#!gizmo/replace -- but that's our problem!)

I guess we could drop the precedence of "with" so it comes just after ",", and then make TargetExpr in XQUF map to whatever we call an "expression without with".

Or to put it another way:

Expr ::= WithExpr ++ "with"
WithExpr ::= ExprSingle ++ ","
XQUF_TargetExpr := ExprSingle

An alternative could be a classical function fn:with…

declare function with(
  $record  as record(*),
  $update  as map(*)
) as record(*) {
  fold-left(
    map:entries($update),
    $record,
    fn($r, $e) { record:replace-value($r, map:keys($e), map:items($e)) }
  )
};

let $coord as record(x, y, z) := { 'x': 1, 'y': 2: 'z': 3 }

(: before: $coord with { 'x': 4 } with { 'y': 5, 'z': 6 } :)
return $coord => with({ 'x': 4 }) => with({ 'y': 5, 'z': 6 })

…accompanied by fn:get for retrieving record entries (and both replacing dm:record-replace-value and dm:record-get-value).

My preference would be to avoid reusing the withkeyword here. From a grammar perspective it seems to introduce avoidable conflicts, especially in combination with XQUF.

If we want to keep a keyword for readability, something more specific like override or patch might work better, as it avoids overloading an existing keyword and more clearly reflects the “apply partial updates” semantics.

Happy to test again if there’s a concrete proposal along those lines.

An alternative could be a classical function fn:with…

But it seems odd to have such a function that's not in the "map" namespace.

Perhaps an operator symbol, say +:= ?

Perhaps an operator symbol, say +:= ?

A good idea; it looks intuitive to me:

let $coord as record(x, y, z) := { 'x': 1, 'y': 2: 'z': 3 }
return $coord +:= { 'x': 4 } +:= { 'y': 5, 'z': 6 }

One might be surprised that no new entries can be “added” this way. But it’s better than all other operators I get in mind (<-, <:=, |:=, 🛠️…).

Just a thought: JSON database systems/document stores have different conventions regarding null (which corresponds to the empty sequence in XQuery 3.1/4) vs. absent being the same or being distinguishable.

If we conflate the two in XQuery 4.0, then XQuery 4.0 might no longer be powerful enough to let the user control outputting nulls or absent values, or distinguishing them upon reading, when used together with systems that make a difference, restricting its usability to only systems that consider the two to be the same.

Does this only impact type matching, or also the semantics of record items themselves?

different conventions regarding null

We addressed that with the "null" option on parse-json.

different conventions regarding null

We addressed that with the "null" option on parse-json.

Ah yes, I see it. Very good indeed. Thank you.

At meeting 165, the CG agreed to merge this PR.