OpenParser

Overview

Map extracted fields back through the parsed document, schema, and evidence.

This is OpenParser's application of lineage@1, the vendor-neutral protocol for recording where a value came from. It is not a second protocol, and it is not the generic lineage overview — it is the extraction-specific profile of the same model.

OpenParser records how each extracted field traces back to the source document, the parsed evidence on the page, and the schema that asked for it. If the returned value is not the text as written, that rewrite is recorded as its own step. Human review can be attached later without erasing the machine trail.

That lets you inspect a field and answer: which words produced it, whether the value is a copy or a rewrite, how confident the read was, and who later confirmed or corrected it.

The rest of this section covers those pieces: reads and rewrites, evidence and confidence, and human review.

Set grounding to field on an extraction request. A successful result normally includes lineage beside output, parsed_document, and grounding.

What OpenParser emits

OpenParser emits:

  • source and parsed-document entities
  • shared source-element collections containing the full parsed transcription
  • OCR, schema-constrained extraction, and grounding-resolution activities
  • the extraction schema as a direct extraction input
  • one field entity per output leaf, keyed by its JSON Pointer (field:/lessee/0/name)
  • derivations connecting each field to the schema, the parsed document, and its verified source regions
  • when the returned value differs from the verbatim extraction, a source-text: entity and an implicit transform activity that produces the field — see extraction is always verbatim

Field entities use kind: decision because they are producer-settled answers, whether produced by a model or corrected by a person. The kind is not limited to human approvals.

The graph stays shallow: source document → parsed document → cited evidence regions → fields. A transform step adds one hop only when the returned value is not the verbatim text. Every field entity carries path (its RFC 6901 pointer), so outputs plus path is the field index — no separate lookup table is emitted. Use fields(doc) and fieldTrace(doc, pointer) from @openparser/lineage instead of walking the graph by hand.

This profile is identified by:

https://docs.openparser.dev/lineage/openparser

Nothing is stated twice

The profile stores each fact once and leaves the rest to be computed.

The complete extraction schema is the value of one schema:extraction artifact, referenced by field-producing derivations with role: output_schema. A field's path locates its title, description, type, and constraints inside that schema, so those definitions are not copied onto every field entity.

A field entity carries its RFC 6901 path, its value, and its confidence assertion. When the model explained the step that produced that entity, it also carries openparser:justification, which is not derivable from the graph and so is stored. That attribute is the read's reason when the value came off the page as written, and the rewrite's transform_claim.reason when it did not — the other account lives on the intermediate source-text: entity. The verbatim text an extraction read is that extract's output value — the field when the model copied, a source-text: entity when it rewrote — so nothing restates it as an attribute. The dotted display path, whether confidence was reported or derived, and whether the field is grounded are all recoverable from path, confidence, and the graph shape, so none of them is stored. Grounding in particular is a property of the graph: a field reached from a source region is grounded, one reached from none is ungrounded, and openparser:droppedSourceIds on the producer derivation marks the partial case. fields() and fieldTrace() in @openparser/lineage return the protocol-level derived values, so consumers should not reimplement the traversal. Because that partial case turns on an openparser: attribute, the grounding rule itself ships as fieldGrounding() in @openparser/lineage/openparser — the profile — rather than in lineage@1, which takes no position on when evidence is sufficient.

Model and tool identity lives on the agent an activity is associated with, never repeated as activity metadata. A derivation omits id, because one derivation produces exactly one output and output therefore identifies it — use derivationId(derivation) when a stable string is needed.

Documents ship no presentation strings. Labels are derived instead, and derived in one place: activityLabel(activity) from @openparser/lineage/openparser names an OpenParser activity from its type, and dottedPathFromPointer(pointer) from @openparser/lineage renders a field path for display. Reach for those rather than keeping a label map per dashboard, so a wording fix or a translation lands once. Core activityLabel stays vendor-neutral: an explicit name, otherwise the raw type. Going from a field path back to an entity is fieldEntityByPointer(doc, pointer); entity ids are not a stable contract, so path is the only locator to match on.

Every confidence assertion restates its scale, even when a producer only ever emits unit_interval. That is deliberate. A score without its scale is not interpretable, and assertions travel — copied into a review UI, logged, embedded in a support ticket — far more often than whole documents do. Making a reader walk back up to a document-level default to learn whether 0.9 means 90% or a raw logit trades a handful of bytes for a class of silent misreading.

Result-size fallback

If the lineage document would push the terminal result over its size limit, the extraction succeeds with grounding and omits lineage. Lineage is a derived view, so it is the first thing dropped and never the reason a job fails.

Continue the graph

Extracted field values preserve the document's wording, except where your schema asks for something else — and then the rewrite is a step in the graph rather than something OpenParser applies silently. Normalization beyond what the schema asked for is a downstream activity.

To represent downstream work:

  1. Add an entity for the normalized, calculated, or inferred value.
  2. Add the activity that produced it.
  3. Add a derivation from the new entity to its exact inputs.
  4. Associate the responsible service, model, or reviewer.
  5. Point outputs at the entities your application returns.

The original OCR evidence remains addressable throughout the chain. Human review follows the same append-only rule.

On this page