diff --git a/README.md b/README.md index 2f978a1..4d5fa11 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,7 @@ graph TD The `LayoutDocument` family (pages of positioned `LayoutItem`s — `text`/`image`/`rect`/`line`/`ellipse`/`path`/`link` in PDF user-space coordinates) no longer lives here: 4.0.0 demoted it to a pdf-codec-private model ([pdf-codec#65](https://github.com/ExaDev/pdf-codec/issues/65)), where the only codec that ever read or wrote it owns it outright. `documentFromJson` recognises old layout-document `$schema` URIs and throws a tombstone pointing at pdf-codec rather than failing as if the value were unrelated. Dependents stay on document-schema.js 3.x via semver until their own majors, so the demotion is not a cascade-breaker. -The package contains only [Zod](https://zod.dev) schemas, their inferred types, trivial schema-attached helpers (hex-colour conversion, recursive structural type guards, the style-resolution helpers of `src/definitions.ts`, the construct-marker balance check of `src/content.ts`), and one small structural interface (`ContentCodec`, see [Codecs](#codecs)). No XML, ZIP, PDF, or binary handling; the sole dependency is `zod`. +The package contains [Zod](https://zod.dev) schemas, their inferred types, trivial schema-attached helpers (hex-colour conversion, recursive structural type guards, the style-resolution helpers of `src/definitions.ts`, the construct-marker balance check of `src/content.ts`), one small structural interface (`ContentCodec`, see [Codecs](#codecs)), and the structural transform between the two encodings it defines (`decompose`/`flattenPackage`/`factorStyles`/`assemblePackage`, see [The package boundary](#the-package-boundary)). No format-specific behaviour and no I/O of any kind — no XML, ZIP, PDF, or binary handling; the sole dependency is `zod`. Two format-agnostic helpers live here because they operate on the content model itself: cell-addressing utilities in `src/a1.ts` (0-based row/column indices, row-first order matching `ContentSheetCell`'s `{row, column}`) and the `FontFace` interface in `src/font-port.ts` (`{family, bold, italic}`). @@ -102,7 +102,33 @@ The flat `ContentDocument` and the tree are **one format, two encodings**, relat 2. **Effective-property equality, universally** — resolve styles first, then compare: a factored and an unfactored serialisation of one document are equal (this is also why content hashing and structural diffing resolve first). 3. **Minting idempotence** — factoring a package a second time mints the identical styles table: `decompose(flatten(decompose(x))) === decompose(x)`. -The codecs do not change: they keep producing flat `ContentDocument`s (their natural reading shape); decomposition runs once at the package boundary in documents.js and flatten runs once where a builder consumes a package. +The codecs do not change: they keep producing flat `ContentDocument`s (their natural reading shape); decomposition runs once where a package is assembled and flatten runs once where a builder consumes one. Both directions live here — see [The package boundary](#the-package-boundary). + +## The package boundary + +The transform between the two encodings lives in this package, alongside the schemas that define them: + +```ts +import { assemblePackage, decompose, factorStyles, flattenPackage } from 'document-schema.js'; + +// The one call a construction site makes: decompose the flat content into the tree, splice the envelope +// onto the root, and mint a styles table over the result. `pages` is optional -- pass it once a layout +// pass has produced each rendered page's own size. +const pkg = assemblePackage(content, pages); + +// The inverse, with every style ref resolved away: a fully materialised, ref-free ContentDocument. +const flat = flattenPackage(pkg); + +// The two halves on their own, for a caller composing its own boundary. +const children = decompose(content); // flat -> the tree `children` a package carries +const reminted = factorStyles(pkg); // re-mint an already-assembled tree (idempotent) +``` + +`decompose` throws `ConstructMarkerImbalanceError` — carrying `src/content.ts`'s own `ConstructMarkerImbalance` payload, so a caller narrows with `instanceof` and reads the offending block index rather than parsing a message — when a container's `constructStart`/`constructEnd` markers do not pair up. Promotion is defined only over a balanced stream, so an unbalanced one is refused rather than repaired into a plausible tree. + +**This is a deliberate amendment to the "schemas only" charter, not a drift from it.** The transform is not business logic and not format-specific behaviour: it is the canonical, purely structural, zero-I/O relationship between the two shapes this package already defines, and its correctness contract *is* the three laws above. It lives here because it has to: `ooxml.js`, `odf.js`, `markdown-codec`, and `pdf-codec` all depend on this package and none of them depends on `documents.js`, so a codec whose public read/write functions speak `DocumentPackage` directly can only reach the transform if the transform sits at or below the schema layer. Everything the charter actually guards against — XML, ZIP, PDF, fonts, layout, bytes, filesystem — remains firmly out. + +The laws are pinned in `src/bijection.test.ts` over a corpus spanning every document kind, every leaf the tree vocabulary admits, and every grouping signal `decompose` reads (headings, list levels, and construct boundaries in each block flow that admits them). `documents.js` runs the same law harness over its own real-format corpus — reader output for every format it supports, editor builds, and conversion captures carrying a layout pass's real frames and pages — which is the complement this package cannot host, since every reader in it belongs to a package that depends on this one. ## Fidelity constructs @@ -199,7 +225,7 @@ Each is its own root field rather than three more tenants of `definitions`, for A styles entry carries `{ paragraph?, run? }` sub-objects of **resolved canonical properties only**: paragraph `alignment`/`list`/`spacingBeforePt`/`spacingAfterPt`/`lineSpacing`/`indentLeftPt`/`indentFirstLinePt`, run `bold`/`italic`/`underline`/`strike`/`fontFamily`/`sizePt`/`color`. Never `frames`, never `sourcePath`, never `styleId` (per-node facts — a position is a fact about a node, not a style), never a `basedOn` graph (the table is a dictionary, not a program) — and the ban list is **enforced by schema shape** (strict objects that reject those keys outright), not merely documented. -Resolution is one overlay chain — outermost ancestor group's style, each nearer group's style, the node's own direct properties; innermost wins, with the resolved run half applying one level further down as run defaults under each run's own properties. `src/definitions.ts` exports the pure helpers that implement it (`overlayStyleEntries`, `resolveStyleChain`, `applyParagraphStyleProperties`, `applyRunStyleProperties`); minting entries (the deterministic frequency pass that factors repeated property tuples into `s1`, `s2`, … refs) is documents.js's boundary behaviour, not this package's. +Resolution is one overlay chain — outermost ancestor group's style, each nearer group's style, the node's own direct properties; innermost wins, with the resolved run half applying one level further down as run defaults under each run's own properties. `src/definitions.ts` exports the pure helpers that implement it (`overlayStyleEntries`, `resolveStyleChain`, `applyParagraphStyleProperties`, `applyRunStyleProperties`), and `factorStyles` (`src/factor-styles.ts`) is the deterministic frequency pass that mints entries, factoring repeated property tuples into `s1`, `s2`, … refs — see [The package boundary](#the-package-boundary). Every module is also importable directly — `tsdown` builds one file per source module, and `package.json`'s `"./*"` export makes each individually resolvable: @@ -311,7 +337,7 @@ try { - [ooxml.js](https://github.com/ExaDev/ooxml.js) — `readDocx`/`readPptx`/`readXlsxContent` return types are typed against this package's schemas, not a local lookalike. - [odf.js](https://github.com/ExaDev/odf.js) — ODF typed readers return the same shared types, so ODF and OOXML speak the identical pivot. -- [documents.js](https://github.com/ExaDev/documents.js) — primary consumer of `ContentDocument` and `DocumentPackage`; its `DOCUMENT_FORMAT_CODECS` registry implements `ContentCodec` per format, and its package boundary runs decompose/flatten against the tree form. +- [documents.js](https://github.com/ExaDev/documents.js) — primary consumer of `ContentDocument` and `DocumentPackage`; its `DOCUMENT_FORMAT_CODECS` registry implements `ContentCodec` per format, and its conversion pipeline calls `assemblePackage`/`flattenPackage` from here at every package construction site. - [pdf-codec](https://github.com/ExaDev/pdf-codec) — owns its layout item model outright since 4.0.0; `readPdf`/`writePdf` operate on pdf-codec's own `LayoutDocument`, and this package's `ContentDocument` remains its content pivot. - [markdown-codec](https://github.com/ExaDev/markdown-codec) — `readMarkdown`/`writeMarkdown` read and write this package's `ContentDocument` directly. diff --git a/package.json b/package.json index cd648a9..c4128c8 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "document-schema.js", "version": "4.2.0", - "description": "The canonical, format-agnostic content and document-package schemas shared by ooxml.js, odf.js, documents.js, pdf-codec, and markdown-codec -- pure Zod schemas, no behaviour.", + "description": "The canonical, format-agnostic content and document-package schemas shared by ooxml.js, odf.js, documents.js, pdf-codec, and markdown-codec, plus the structural transform between them -- no format-specific or I/O behaviour.", "type": "module", "repository": { "type": "git", diff --git a/src/bijection.test.ts b/src/bijection.test.ts new file mode 100644 index 0000000..0defb9a --- /dev/null +++ b/src/bijection.test.ts @@ -0,0 +1,492 @@ +import { describe, expect, it } from 'vitest'; +import { canonicalise } from './canonicalise'; +import type { ConstructDescriptor } from './construct'; +import { + ContentDocumentSchema, + type ContentBlock, + type ContentDocument, + type ContentEmbeddedObject, + type ContentSheetCell, + type ContentSheetImage, + type ContentSheetPrintSettings, + type ContentShape, + type ContentVector, +} from './content'; +import { assemblePackage, factorStyles } from './factor-styles'; +import { flattenPackage } from './flatten'; +import type { PageSize } from './geometry'; +import { DocumentPackageSchema, type DocumentPackage } from './package'; + +// THE PACKAGE BOUNDARY'S MERGE GATE: the three bijection laws run over a corpus spanning every document kind, every leaf the tree vocabulary admits, and every grouping signal decompose reads. document-outline.js proved the laws property-wise over its local corpus in phase 1, and documents.js runs this same law harness over its own REAL corpus -- reader outputs for every format, editors per kind, onDocument captures carrying the layout pass's real frames and pages. That corpus cannot live here: every reader in it belongs to a package that depends on this one (ooxml.js, odf.js, markdown-codec, pdf-codec), so importing it would invert the dependency the schema layer exists to keep one-way. What lives here instead is the same harness over hand-built content covering the same structural ground, and documents.js's own suite stays the gate over real format output -- the two are complementary, not redundant: this one pins the transform against the schema's whole vocabulary, that one pins it against what codecs actually emit. +// +// The laws (stated on #20 and its errata, and in src/package.ts's own header): (i) flatten(assemble(c)) reproduces c exactly, up to one declared normalisation (a present-but-empty embeddedObjects array normalises to the field absent); (ii) effective-property equality universally -- the flat codec-exchange form flatten produces is fully materialised (zero style refs) and structurally identical to the unfactored original, so a factored and an unfactored serialisation of one document compare equal; (iii) minting idempotence -- assembling the flattened tree again (and factoring an already-factored package) mints the identical table and the identical tree. Never an identity assertion: decompose embeds the source's own node objects, so toBe would pass even for an implementation that mutated its input -- structural comparison over a pre-roundtrip structuredClone snapshot is what actually pins the values, and re-comparing the source against its snapshot additionally pins that neither direction mutates the input in place. + +function canon(value: unknown): unknown { + return JSON.parse(JSON.stringify(normaliseEmbeddedObjects(canonicalise(value)))); +} + +// The bijection's one declared normalisation: decompose concatenates a sheet's images and embedded objects into a single children array and flatten rebuilds embeddedObjects only when an embedded object exists, so a present-but-empty array -- schema-legal, emitted by no codec -- cannot survive the round trip and normalises to the field absent. Applied to BOTH sides of every comparison so law (i) stays an equivalence over canonical forms; the direction is pinned outright in decompose.test.ts. Recursive because a sheet can sit inside an embedded document, whose own sheets can carry the same field. +function normaliseEmbeddedObjects(value: unknown): unknown { + if (Array.isArray(value)) return value.map(normaliseEmbeddedObjects); + if (typeof value !== 'object' || value === null) return value; + const normalised: Record = {}; + for (const [key, child] of Object.entries(value)) { + if (key === 'embeddedObjects' && Array.isArray(child) && child.length === 0) continue; + normalised[key] = normaliseEmbeddedObjects(child); + } + return normalised; +} + +function expectStructurallyEqual(actual: unknown, expected: unknown): void { + expect(canon(actual)).toEqual(canon(expected)); +} + +// True when any object anywhere in the value is a tree group wrapper -- `{ node, children }`, the only shape a style ref can sit on. "The flat encoding is always fully materialised, refs live only on tree wrappers" is the invariant minting depends on and law (ii) asserts, and "no wrapper survived at all" is the strongest form of it: a ref has nowhere else to go. Stated structurally rather than as a scan for any key named `style`, because `style` is also an ordinary content field -- a ContentStroke's own solid/dashed/dotted/double, which a drawing page legitimately carries and which a key-name scan would misread as a leaked ref. +function containsGroupWrapper(value: unknown): boolean { + if (Array.isArray(value)) return value.some(containsGroupWrapper); + if (typeof value !== 'object' || value === null) return false; + if ('node' in value && 'children' in value) return true; + return Object.values(value).some(containsGroupWrapper); +} + +// One corpus entry: flat content, plus the pages a layout pass would have produced for the entries that carry fused frames (the wrapped-run case needs real per-page frames, and `pages` is what a frame's own pageIndex indexes into). +interface CorpusEntry { + readonly name: string; + readonly content: ContentDocument; + readonly pages?: readonly PageSize[]; +} + +// --- Shared fixture vocabulary --------------------------------------------------------------------------- + +const SECTION_GEOMETRY = { pageSize: { widthPt: 595, heightPt: 842 }, margins: { topPt: 72, rightPt: 72, bottomPt: 72, leftPt: 72 } }; +const SLIDE_SIZE = { widthPt: 960, heightPt: 540 }; +const SHAPE_FRAME = { xPt: 0, yPt: 0, widthPt: 400, heightPt: 300 }; +const PNG_BASE64 = 'aW1hZ2U='; +const PRINT_SETTINGS: ContentSheetPrintSettings = { + pageSize: { widthPt: 595, heightPt: 842 }, + margins: { topPt: 20, rightPt: 20, bottomPt: 20, leftPt: 20 }, + gridlines: true, + headers: true, + pageOrder: 'downThenOver', +}; + +interface ParagraphOptions { + readonly headingLevel?: number; + readonly listLevel?: number; + readonly numId?: string; + readonly alignment?: 'left' | 'center' | 'right' | 'justify'; + readonly indentLeftPt?: number; + readonly lineSpacing?: number; + readonly styleId?: string; + readonly sourcePath?: string; + readonly frames?: readonly { pageIndex: number; xPt: number; yPt: number; widthPt: number; heightPt: number }[]; + readonly bold?: boolean; + readonly sizePt?: number; +} + +function paragraph(text: string, options: ParagraphOptions = {}): ContentBlock { + return { + kind: 'paragraph', + runs: [{ text, ...(options.bold !== undefined ? { bold: options.bold } : {}), ...(options.sizePt !== undefined ? { sizePt: options.sizePt } : {}) }], + ...(options.headingLevel !== undefined ? { headingLevel: options.headingLevel } : {}), + ...(options.listLevel !== undefined ? { list: { level: options.listLevel, ...(options.numId !== undefined ? { numId: options.numId } : {}) } } : {}), + ...(options.alignment !== undefined ? { alignment: options.alignment } : {}), + ...(options.indentLeftPt !== undefined ? { indentLeftPt: options.indentLeftPt } : {}), + ...(options.lineSpacing !== undefined ? { lineSpacing: options.lineSpacing } : {}), + ...(options.styleId !== undefined ? { styleId: options.styleId } : {}), + ...(options.sourcePath !== undefined ? { sourcePath: options.sourcePath } : {}), + ...(options.frames !== undefined ? { frames: options.frames.map((frame) => ({ ...frame })) } : {}), + }; +} + +function shape(blocks: readonly ContentBlock[], overrides: Partial = {}): ContentShape { + return { frame: SHAPE_FRAME, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0, ...overrides, blocks: [...blocks] }; +} + +function wordprocessing(blocksPerSection: readonly (readonly ContentBlock[])[]): ContentDocument { + return { kind: 'wordprocessing', metadata: {}, sections: blocksPerSection.map((blocks) => ({ ...SECTION_GEOMETRY, blocks: [...blocks] })) }; +} + +// --- The corpus ------------------------------------------------------------------------------------------- + +function corpus(): readonly CorpusEntry[] { + const embeddedDrawing: ContentEmbeddedObject = { + objectKind: 'drawing', + document: { kind: 'drawing', metadata: {}, pages: [{ size: { widthPt: 300, heightPt: 300 }, shapes: [], vectors: [{ kind: 'rect', frame: { xPt: 1, yPt: 2, widthPt: 3, heightPt: 4 } }] }] }, + frame: { xPt: 10, yPt: 20, widthPt: 120, heightPt: 90 }, + }; + const table: ContentBlock = { + kind: 'table', + // A cell's own blocks stay flat in BOTH encodings -- decomposition treats a table as one leaf and never descends -- so the marker pair inside this cell must ride through untouched and unpromoted, which is the one place a construct is spelled the same way on both sides of the boundary. + rows: [ + { + cells: [ + { blocks: [paragraph('cell one'), { kind: 'constructStart', descriptor: { kind: 'field', instruction: 'PAGE' } }, paragraph('inside a cell construct'), { kind: 'constructEnd' }] }, + { blocks: [paragraph('cell two', { headingLevel: 2 })], colSpan: 2 }, + ], + }, + ], + columnWidthsPt: [80, 120], + }; + + const entries: CorpusEntry[] = [ + { name: 'empty wordprocessing document (no sections at all)', content: wordprocessing([]) }, + { name: 'wordprocessing section with no blocks', content: wordprocessing([[]]) }, + { + name: 'wordprocessing heading hierarchy with a level jump and a pop back to the root', + content: wordprocessing([[ + paragraph('front matter, before any heading'), + paragraph('Chapter', { headingLevel: 1 }), + paragraph('under the chapter'), + paragraph('Deep', { headingLevel: 4 }), + paragraph('under the deep heading'), + paragraph('Next chapter', { headingLevel: 1 }), + paragraph('under the next chapter'), + ]]), + }, + { + name: 'wordprocessing list nesting, closed by a plain paragraph and reopened', + content: wordprocessing([[ + paragraph('item one', { listLevel: 0, numId: 'n1' }), + paragraph('item one a', { listLevel: 1, numId: 'n1' }), + paragraph('item one a i', { listLevel: 2, numId: 'n1' }), + paragraph('item two', { listLevel: 0, numId: 'n1' }), + paragraph('a plain paragraph closes the list nesting'), + paragraph('a fresh item', { listLevel: 1, numId: 'n1' }), + ]]), + }, + { + name: 'wordprocessing with every block leaf kind the tree admits', + content: wordprocessing([[ + paragraph('Chapter', { headingLevel: 1 }), + table, + { kind: 'image', format: 'png', base64: PNG_BASE64, widthPt: 100, heightPt: 60, altText: 'a picture' }, + { kind: 'pageBreak' }, + { ...embeddedDrawing, kind: 'embeddedObject' }, + paragraph('after the leaves'), + ]]), + }, + { + name: 'multi-section wordprocessing (each section resets the heading stack)', + content: wordprocessing([ + [paragraph('Chapter', { headingLevel: 1 }), paragraph('body of chapter one')], + [paragraph('Method', { headingLevel: 2 }), paragraph('body of chapter two')], + ]), + }, + { + name: 'wordprocessing with repeated direct formatting on both halves (mints paragraph and run tuples)', + content: wordprocessing([[ + paragraph('one', { alignment: 'left', indentLeftPt: 20, bold: true, sizePt: 12 }), + paragraph('two', { alignment: 'left', indentLeftPt: 20, bold: true, sizePt: 12 }), + paragraph('three', { alignment: 'left', indentLeftPt: 20, bold: true, sizePt: 12 }), + ]]), + }, + { + name: 'wordprocessing whose repetition is nested under a heading (the ref lands on the heading group)', + content: wordprocessing([[ + paragraph('intro carries no mintable key'), + paragraph('Chapter', { headingLevel: 1, alignment: 'center' }), + paragraph('a', { alignment: 'center' }), + paragraph('b', { alignment: 'center' }), + ]]), + }, + { + name: 'wordprocessing carrying the ban-list fields alongside mintable ones', + // styleId, sourcePath, and frames repeat exactly as often as the mintable keys do, and must stay per-node throughout the round trip. + content: wordprocessing([[ + paragraph('one', { styleId: 'Body', sourcePath: 'word/document.xml#p1', indentLeftPt: 20, frames: [{ pageIndex: 0, xPt: 72, yPt: 700, widthPt: 451, heightPt: 14 }] }), + paragraph('two', { styleId: 'Body', sourcePath: 'word/document.xml#p1', indentLeftPt: 20, frames: [{ pageIndex: 0, xPt: 72, yPt: 680, widthPt: 451, heightPt: 14 }] }), + ]]), + }, + { + name: 'wordprocessing with fused frames across two pages and a populated pages array', + // The wrapped-run case: one paragraph rendered into two places by pagination, so its frames array names two different pages. + content: wordprocessing([[ + paragraph('wraps across the page boundary', { frames: [{ pageIndex: 0, xPt: 72, yPt: 90, widthPt: 451, heightPt: 14 }, { pageIndex: 1, xPt: 72, yPt: 760, widthPt: 451, heightPt: 14 }] }), + paragraph('lands on the second page', { frames: [{ pageIndex: 1, xPt: 72, yPt: 740, widthPt: 451, heightPt: 14 }] }), + ]]), + pages: [{ widthPt: 595, heightPt: 842 }, { widthPt: 595, heightPt: 842 }], + }, + { + name: 'wordprocessing carrying document metadata and a symbol table on the envelope', + content: { + kind: 'wordprocessing', + metadata: { title: 'Envelope', author: 'A. Author', keywords: ['one', 'two'], createdIso: '2026-01-15T00:00:00Z' }, + symbolTable: { symbols: [], units: [] }, + sections: [{ ...SECTION_GEOMETRY, blocks: [paragraph('body')] }], + }, + }, + { + name: 'presentation with several shapes, list nesting inside each, and a heading-styled leaf', + content: { + kind: 'presentation', + metadata: {}, + slides: [ + { + size: SLIDE_SIZE, + notes: 'notes ride the slide descriptor', + shapes: [ + shape([paragraph('top', { listLevel: 0 }), paragraph('nested', { listLevel: 1 })], { name: 'Body' }), + // headingLevel in a shape flow is deliberately not a grouping signal, so this paragraph stays a bare leaf carrying the field. + shape([paragraph('plain'), paragraph('heading-styled, still a leaf here', { headingLevel: 2 })], { rotationDeg: 15, paintOrder: 2 }), + ], + }, + { size: SLIDE_SIZE, notes: '', shapes: [] }, + ], + }, + }, + { + name: 'presentation with run formatting repeated across two shapes (mints on the slide wrapper)', + content: { + kind: 'presentation', + metadata: {}, + slides: [{ + size: SLIDE_SIZE, + notes: '', + shapes: [shape([paragraph('a', { bold: true, sizePt: 12 })]), shape([paragraph('b', { bold: true, sizePt: 12 })])], + }], + }, + }, + { + name: 'spreadsheet with a populated grid, anchored images, and an embedded document', + content: { + kind: 'spreadsheet', + metadata: {}, + sheets: [ + { + name: 'Data', + cells: [ + { row: 0, column: 0, value: { kind: 'string', value: 'label' }, displayText: 'label' }, + { row: 0, column: 1, value: { kind: 'number', value: 42.5 }, displayText: '42.50', formula: '=SUM(B2:B9)' }, + { row: 1, column: 0, value: { kind: 'boolean', value: true }, displayText: 'TRUE' }, + { row: 1, column: 1, value: { kind: 'empty' }, displayText: '', comment: { text: 'a note', author: 'A. Author' } }, + ] satisfies ContentSheetCell[], + columns: [{ index: 0, widthPt: 60 }, { index: 1, hidden: true }], + rows: [{ index: 0, heightPt: 12 }], + images: [{ kind: 'image', format: 'png', base64: PNG_BASE64, widthPt: 10, heightPt: 10, anchorRow: 0, anchorColumn: 0, offsetXPt: 2, offsetYPt: 3 }] satisfies ContentSheetImage[], + printSettings: PRINT_SETTINGS, + embeddedObjects: [embeddedDrawing], + }, + { name: 'Empty', cells: [], columns: [], rows: [], images: [], printSettings: PRINT_SETTINGS }, + ], + }, + }, + { + name: 'spreadsheet whose embeddedObjects array is present but empty (the one declared normalisation)', + content: { + kind: 'spreadsheet', + metadata: {}, + sheets: [{ name: 'Declared empty', cells: [], columns: [], rows: [], images: [], printSettings: PRINT_SETTINGS, embeddedObjects: [] }], + }, + }, + { + name: 'drawing page with a text shape and every vector primitive', + content: { + kind: 'drawing', + metadata: {}, + pages: [ + { + size: { widthPt: 300, heightPt: 300 }, + shapes: [shape([paragraph('a label on the drawing')])], + vectors: [ + { kind: 'rect', frame: { xPt: 1, yPt: 1, widthPt: 20, heightPt: 10 }, fill: { r: 1, g: 0.5, b: 0 } }, + { kind: 'ellipse', frame: { xPt: 30, yPt: 4, widthPt: 8, heightPt: 4 }, rotationDeg: 30 }, + { kind: 'line', from: { xPt: 1, yPt: 20 }, to: { xPt: 10, yPt: 20 }, stroke: { color: { r: 0, g: 0, b: 0 }, widthPt: 1, style: 'dashed' } }, + { + kind: 'path', + frame: { xPt: 0, yPt: 30, widthPt: 20, heightPt: 10 }, + subpaths: [{ start: { xPt: 0, yPt: 0 }, segments: [{ kind: 'line', to: { xPt: 10, yPt: 0 } }, { kind: 'cubic', control1: { xPt: 20, yPt: 0 }, control2: { xPt: 20, yPt: 10 }, to: { xPt: 10, yPt: 10 } }], closed: true }], + fillRule: 'evenodd', + }, + ] satisfies ContentVector[], + }, + { size: { widthPt: 300, heightPt: 300 }, shapes: [], vectors: [] }, + ], + }, + }, + { + name: 'formula document (the one tree shape with no container)', + content: { + kind: 'formula', + metadata: {}, + formula: { + mathml: [{ type: 'element', tag: 'math', attributes: [], children: [{ type: 'text', value: 'a/b' }] }], + starMath: '{a} over {b}', + presentation: { latex: '\\frac{a}{b}' }, + provenance: { source: 'odf:content.xml#Object1', editTrail: [] }, + }, + }, + }, + ]; + entries.push(...constructCorpus()); + return entries; +} + +// --- The construct-boundary corpus ------------------------------------------------------------------------ + +// 4.2.0 gave ContentBlock the constructStart/constructEnd marker pair, so a construct boundary is a flat-form signal decompose promotes to a construct group and flatten reproduces, exactly like a heading level or a list level. One entry per placement, so a failure names the case. + +const CONSTRUCT_SECTION = { pageSize: { widthPt: 595, heightPt: 842 }, margins: { topPt: 72, rightPt: 72, bottomPt: 72, leftPt: 72 } }; +const CONSTRUCT_SHAPE_FRAME = { xPt: 0, yPt: 0, widthPt: 400, heightPt: 300 }; +const CONSTRUCT_END: ContentBlock = { kind: 'constructEnd' }; + +function constructStart(descriptor: ConstructDescriptor): ContentBlock { + return { kind: 'constructStart', descriptor }; +} + +function constructParagraph(text: string, options: { headingLevel?: number; listLevel?: number; indentLeftPt?: number } = {}): ContentBlock { + return { + kind: 'paragraph', + runs: [{ text }], + ...(options.headingLevel !== undefined ? { headingLevel: options.headingLevel } : {}), + ...(options.listLevel !== undefined ? { list: { level: options.listLevel } } : {}), + ...(options.indentLeftPt !== undefined ? { indentLeftPt: options.indentLeftPt } : {}), + }; +} + +function constructSectionEntry(name: string, blocks: readonly ContentBlock[]): CorpusEntry { + return { name, content: { kind: 'wordprocessing', metadata: {}, sections: [{ ...CONSTRUCT_SECTION, blocks: [...blocks] }] } }; +} + +function constructShape(blocks: readonly ContentBlock[]): ContentShape { + return { frame: CONSTRUCT_SHAPE_FRAME, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0, blocks: [...blocks] }; +} + +function constructCorpus(): readonly CorpusEntry[] { + // Repeated indentLeftPt inside each construct region so the entries mint for real rather than round-tripping a styles-free tree: the ref lands on the construct group itself (the enclosing section's extent also holds the unindented paragraphs, so no ancestor can factor the key), which is what makes laws (ii) and (iii) bite on a construct wrapper and not just on the leaves under it. + const shapeWithConstruct = constructShape([ + constructParagraph('before the construct'), + constructStart({ kind: 'field', instruction: 'PAGE' }), + constructParagraph('in a shape construct', { indentLeftPt: 18 }), + constructParagraph('also in it', { indentLeftPt: 18 }), + CONSTRUCT_END, + ]); + return [ + constructSectionEntry('construct at a section root', [ + constructParagraph('before'), + constructStart({ kind: 'field', instruction: 'PAGE', cachedResult: '1' }), + constructParagraph('in a field', { indentLeftPt: 24 }), + constructParagraph('still in the field', { indentLeftPt: 24 }), + CONSTRUCT_END, + constructParagraph('after'), + ]), + constructSectionEntry('construct nested inside a heading group', [ + constructParagraph('Chapter', { headingLevel: 1 }), + constructParagraph('under the heading'), + constructStart({ kind: 'contentControl', controlType: 'richText', tag: 'body', alias: 'Body' }), + constructParagraph('in a content control', { indentLeftPt: 24 }), + constructParagraph('still in it', { indentLeftPt: 24 }), + CONSTRUCT_END, + constructParagraph('after the control, still under the heading'), + ]), + constructSectionEntry('construct nested inside a list group', [ + constructParagraph('item one', { listLevel: 0 }), + constructStart({ kind: 'anchor', anchorType: 'bookmark', name: 'b1' }), + constructParagraph('in a bookmark', { indentLeftPt: 24 }), + constructParagraph('still in it', { indentLeftPt: 24 }), + CONSTRUCT_END, + // A deeper item after the region: the round trip only reproduces it in place if stepping through the construct left the list stack alone. + constructParagraph('item two, nested', { listLevel: 1 }), + ]), + constructSectionEntry('two constructs of different kinds nested inside each other', [ + constructStart({ kind: 'provenance', change: 'insertion', author: 'A', dateIso: '2024-01-15T00:00:00Z' }), + constructParagraph('inserted'), + constructStart({ kind: 'link', target: { kind: 'external', uri: 'https://example.invalid/' }, title: 'Example' }), + constructParagraph('linked and inserted', { indentLeftPt: 24 }), + constructParagraph('also linked', { indentLeftPt: 24 }), + CONSTRUCT_END, + constructParagraph('inserted again'), + CONSTRUCT_END, + ]), + constructSectionEntry('construct with no children (an open marker immediately closed)', [ + constructParagraph('before'), + constructStart({ kind: 'division', name: 'empty', columnCount: 2 }), + CONSTRUCT_END, + constructParagraph('after'), + ]), + { + name: 'construct inside a presentation shape flow', + content: { kind: 'presentation', metadata: {}, slides: [{ size: SLIDE_SIZE, shapes: [shapeWithConstruct], notes: '' }] }, + }, + { + name: 'construct inside a drawing page shape flow', + content: { kind: 'drawing', metadata: {}, pages: [{ size: { widthPt: 300, heightPt: 300 }, shapes: [shapeWithConstruct], vectors: [] }] }, + }, + ]; +} + +describe('decompose/flatten bijection laws over the schema-vocabulary corpus', () => { + describe.each(corpus())('$name', ({ content, pages }) => { + it('law (i): flattenPackage(assemblePackage(c)) reproduces c exactly', () => { + expect(ContentDocumentSchema.safeParse(content).success).toBe(true); + const snapshot = structuredClone(content); + const tree = assemblePackage(content, pages); + expect(DocumentPackageSchema.safeParse(tree).success).toBe(true); + const flat = flattenPackage(tree); + expect(ContentDocumentSchema.safeParse(flat).success).toBe(true); + expectStructurallyEqual(flat, snapshot); + // decompose embeds the source's own nodes, so re-comparing the source against its snapshot also pins that neither direction of the round trip mutated the input in place. + expectStructurallyEqual(content, snapshot); + }); + + it('law (ii): the flat encoding is fully materialised and effective-equal to the original', () => { + const snapshot = structuredClone(content); + const tree = assemblePackage(content, pages); + const flat = flattenPackage(tree); + expect(containsGroupWrapper(flat)).toBe(false); + // Resolve-then-compare in the flatten-as-resolver form: materialising every ref away and comparing structurally IS the effective-property comparison, because gap-fill restoration is exactly what resolution does. + expectStructurallyEqual(flat, snapshot); + expectStructurallyEqual(content, snapshot); + }); + + it('law (iii): assembling the flattened tree again mints the identical table and tree', () => { + const first = assemblePackage(content, pages); + const second = assemblePackage(flattenPackage(first), first.pages); + expectStructurallyEqual(second, first); + // Factoring an already-factored package is the same law through the public re-mint entry point. + expectStructurallyEqual(factorStyles(first), first); + }); + + it('mints deterministically', () => { + expectStructurallyEqual(assemblePackage(content, pages), assemblePackage(content, pages)); + }); + }); + + // The gate must not pass vacuously: minting has to actually run over corpus documents, so at least one entry's tree carries a non-empty styles table and at least one wrapper ref. If this ever fails because no entry mints, the corpus has stopped exercising laws (ii) and (iii) and needs a real formatting-repetition fixture, not a weakened assertion. + it('the corpus exercises real minting (at least one entry carries a styles table)', () => { + const minting = corpus().filter((entry) => Object.keys(assemblePackage(entry.content, entry.pages).styles ?? {}).length > 0); + expect(minting.length).toBeGreaterThan(0); + }); + + // The same anti-vacuity guard, narrowed to the construct entries: laws (ii) and (iii) say nothing about construct groups unless a construct group actually carries a ref, and a construct entry that minted nothing would pass all three laws while proving only that its leaves round-trip. Every construct entry except the deliberately empty one is built to mint on its own construct wrapper, so this pins that the promotion and minting really do compose over the corpus rather than only in factor-styles.test.ts's single fixture. + it('the construct corpus mints refs onto the construct groups themselves', () => { + const withConstructRefs = constructCorpus().filter((entry) => constructGroupRefsOf(assemblePackage(entry.content, entry.pages)).length > 0); + expect(withConstructRefs.map((entry) => entry.name)).toEqual(constructCorpus().filter((entry) => entry.name !== 'construct with no children (an open marker immediately closed)').map((entry) => entry.name)); + }); + + // Every ContentDocument kind must be represented, or a kind could quietly stop being exercised while the suite still passed on the other four. + it('the corpus covers every document kind', () => { + expect([...new Set(corpus().map((entry) => entry.content.kind))].sort()).toEqual(['drawing', 'formula', 'presentation', 'spreadsheet', 'wordprocessing']); + }); +}); + +// Every style ref sitting on a construct-descriptor wrapper anywhere in a minted tree: a group node carrying a `kind` that is neither 'paragraph' nor a container discriminant is a ConstructDescriptor, which is exactly what construct groups (and nothing else) hold. +function constructGroupRefsOf(pkg: DocumentPackage): string[] { + const refs: string[] = []; + function walk(value: unknown): void { + if (Array.isArray(value)) { + for (const child of value) walk(child); + return; + } + if (typeof value !== 'object' || value === null) return; + if ('node' in value && 'children' in value && 'style' in value && typeof value.style === 'string') { + const node: unknown = value.node; + if (typeof node === 'object' && node !== null && 'kind' in node && typeof node.kind === 'string' && !['paragraph', 'section', 'slide', 'sheet', 'drawPage'].includes(node.kind)) { + refs.push(value.style); + } + } + for (const child of Object.values(value)) walk(child); + } + walk(pkg.children); + return refs; +} diff --git a/src/canonicalise.ts b/src/canonicalise.ts new file mode 100644 index 0000000..f4def56 --- /dev/null +++ b/src/canonicalise.ts @@ -0,0 +1,21 @@ +// Canonical key ordering for structural comparison and tuple identity, ported verbatim-in-spirit from document-outline.js's src/outline/hash.ts (the recipe the promotion plan locked as "the" canonicaliser -- one implementation across the family, not a second recipe that could drift). Rebuilds every plain object with its own keys sorted ascending by UTF-16 code unit (Array.prototype.sort's default comparison -- a total, implementation-specified-stable order), preserving arrays in order and primitives as-is. This removes construction-order differences between independently built but structurally identical content. `unknown` in, `unknown` out: the output is a fresh structure safe to hand to JSON.stringify, never a mutation of the input. Plain objects (the JSON-mappable class) are rebuilt with sorted keys; arrays are copied so the output never aliases the input. +// +// Not re-exported from the package's index barrel: it exists to give src/factor-styles.ts one tuple-identity recipe, and naming it there would invite a second caller to depend on the exact sort order as an API guarantee rather than as minting's internal determinism device. The `"./*"` subpath export still makes `document-schema.js/canonicalise` importable directly, per the README's "every module is also importable directly" -- omitting it from the barrel narrows what index.ts re-exports, not what the package publishes. +export function canonicalise(value: unknown): unknown { + if (Array.isArray(value)) return value.map(canonicalise); + if (isRecord(value)) { + const sorted: Record = {}; + for (const key of Object.keys(value).sort()) sorted[key] = canonicalise(value[key]); + return sorted; + } + return value; +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +// Tuple identity per the promotion plan's minting rules ("tuple identity reuses stableContentHash's canonicalise-then-stringify recipe -- no second recipe"): the canonical string IS the key, so equal-valued tuples built in different key orders land on one map slot and the tie-break ordering compares the same bytes the identity does. JSON.stringify drops undefined-valued properties, so an optional field left absent and one explicitly assigned undefined collapse to the same key -- both spellings mean "field absent" in the content schemas. +export function canonicalKey(value: unknown): string { + return JSON.stringify(canonicalise(value)); +} diff --git a/src/content.ts b/src/content.ts index cb756a3..15a2fac 100644 --- a/src/content.ts +++ b/src/content.ts @@ -84,7 +84,7 @@ export type ContentPageBreak = z.infer; // // THE BRACKET-MATCHING CONTRACT, stated once and binding on every producer and consumer: markers pair exactly as balanced parentheses do -- a `constructEnd` closes the nearest preceding still-open `constructStart` in the SAME block list, and the blocks between them are that construct's extent. That is the entire pairing mechanism; there is deliberately no id, name, or other pairing key on either marker. Matching never straddles a block list: a pair opened in a section's blocks closes in that same array, and a pair opened inside a table cell closes inside that cell -- which is also the only way a construct inside a table is expressible in EITHER encoding, since decomposition treats a table as one leaf and never descends into its cells. A block list whose markers do not balance (an end with no open start, or a start still open when the list ends) is malformed input rather than a shape to repair: see findConstructMarkerImbalance below, the one shared definition of that check. // -// BALANCE IS NECESSARY BUT NOT SUFFICIENT: a marker's extent must also not cross a heading-group or list-group scope boundary. Headings and list items carry no delimiter of their own in the flat form -- a heading paragraph's scope runs until the next paragraph in the same block list whose headingLevel is shallower than or equal to its own, and a list item's scope runs until nesting shallows back out, exactly the nesting decompose infers when it builds HeadingGroupNode/ListGroupNode (src/package-node.ts) from a flat list. A constructStart/constructEnd pair whose extent contains a paragraph that would close a heading or list scope already open when the pair started leaves decompose no single correct tree to build: nesting the construct group inside that still-closing scope strands the closing paragraph with no legal parent for it, while closing the scope at the constructStart and hoisting the construct group out silently moves everything after the closing paragraph out of a scope it belonged to -- two different, equally legal-looking trees from one balanced input, and neither is more correct than the other. findConstructMarkerImbalance below cannot see this: balance is a property of the marker pair alone, not of what sits between the two markers, and this package does not check it either -- unlike balance, detecting a scope crossing means walking the same heading/list nesting decompose already builds while constructing the tree, so decompose (documents.js) is the sole enforcement point. Exactly as it already rejects rather than repairs an unbalanced pair, decompose must reject a scope-crossing extent rather than silently choosing between the two divergent trees it could otherwise produce. A producer emitting a marker pair (ooxml.js, odf.js, markdown-codec, pdf-codec) must never open one inside a heading or list scope that some other block inside the extent then closes. +// BALANCE IS NECESSARY BUT NOT SUFFICIENT: a marker's extent must also not cross a heading-group or list-group scope boundary. Headings and list items carry no delimiter of their own in the flat form -- a heading paragraph's scope runs until the next paragraph in the same block list whose headingLevel is shallower than or equal to its own, and a list item's scope runs until nesting shallows back out, exactly the nesting decompose infers when it builds HeadingGroupNode/ListGroupNode (src/package-node.ts) from a flat list. A constructStart/constructEnd pair whose extent contains a paragraph that would close a heading or list scope already open when the pair started leaves decompose no single correct tree to build: nesting the construct group inside that still-closing scope strands the closing paragraph with no legal parent for it, while closing the scope at the constructStart and hoisting the construct group out silently moves everything after the closing paragraph out of a scope it belonged to -- two different, equally legal-looking trees from one balanced input, and neither is more correct than the other. findConstructMarkerImbalance below cannot see this: balance is a property of the marker pair alone, not of what sits between the two markers, and this package does not check it either -- unlike balance, detecting a scope crossing means walking the same heading/list nesting decompose already builds while constructing the tree, so decompose (src/decompose.ts) is the sole enforcement point. Exactly as it already rejects rather than repairs an unbalanced pair, decompose must reject a scope-crossing extent rather than silently choosing between the two divergent trees it could otherwise produce. A producer emitting a marker pair (ooxml.js, odf.js, markdown-codec, pdf-codec) must never open one inside a heading or list scope that some other block inside the extent then closes. // // WHY NO ID ON EITHER MARKER: an id would have to be minted by whichever producer emitted the pair and then reproduced byte-for-byte by flatten to satisfy the encoding pair's own first law, flatten(decompose(x)) === x (src/package.ts). A construct group carries a descriptor and its children and nothing else, so a marker id would be a value with no home on the tree side and no deterministic way back -- whereas a bare bracket has nothing to reproduce and nothing to get wrong. Bracket matching also already generalises to arbitrary nesting depth and to different construct kinds nested inside each other, which is the whole of what a pairing key would have bought. // @@ -272,7 +272,7 @@ export type ConstructMarkerImbalance = | { kind: 'unmatchedEnd'; index: number } | { kind: 'unclosedStart'; index: number }; -// The one shared definition of the bracket-matching contract's balance check (see the marker schemas above for the contract itself): returns the first place a block list's markers fail to match, or undefined when they balance. It lives here rather than in each consumer because at least three of them must agree exactly -- every codec that emits a pair, and documents.js's decompose, which promotes each matched pair into a construct group and so must reject a list it cannot promote instead of silently repairing one -- and because no schema can express it: balance is a property of a block list's sequence, not of any block in it, so ContentBlockSchema validating every member says nothing about whether the members pair up. +// The one shared definition of the bracket-matching contract's balance check (see the marker schemas above for the contract itself): returns the first place a block list's markers fail to match, or undefined when they balance. It lives here rather than in each consumer because at least three of them must agree exactly -- every codec that emits a pair, and this package's own decompose (src/decompose.ts), which promotes each matched pair into a construct group and so must reject a list it cannot promote instead of silently repairing one -- and because no schema can express it: balance is a property of a block list's sequence, not of any block in it, so ContentBlockSchema validating every member says nothing about whether the members pair up. // // Deliberately non-recursive. Each block list is its own bracket scope (a table cell's list matches independently of the list containing the table), so a caller walking nested lists calls this once per list -- which is exactly the walk decompose already performs -- rather than this helper duplicating that walk with its own idea of where the nested lists are. export function findConstructMarkerImbalance(blocks: readonly ContentBlock[]): ConstructMarkerImbalance | undefined { diff --git a/src/decompose.test.ts b/src/decompose.test.ts new file mode 100644 index 0000000..6a7c01f --- /dev/null +++ b/src/decompose.test.ts @@ -0,0 +1,369 @@ +import { describe, expect, it } from 'vitest'; +import type { ConstructDescriptor } from './construct'; +import type { ContentBlock, ContentDocument, ContentEmbeddedObject, ContentFormula, ContentSection, ContentShape, ContentSheetCell, ContentSheetImage, ContentSheetPrintSettings, ContentVector } from './content'; +import { ConstructMarkerImbalanceError, decompose, decomposeSection, decomposeSheet, isHeadingParagraph } from './decompose'; +import { flattenPackage } from './flatten'; +import type { DocumentPackage } from './package'; +import { isHeadingGroupNode, isListGroupNode, isSectionConstructGroupNode, type SheetGroupNode } from './package-node'; + +// The bijection laws (bijection.test.ts) pin round-trip fidelity, not grouping semantics -- a degenerate decompose whose section groups carried flat, ungrouped children would satisfy every law just as well. These tests pin the TREE SHAPE itself: mandatory section groups, per-container stacks, the never-cross-a-shape-boundary rule, and the ownership discipline. Ported from document-outline.js's phase-1 decompose tests, adapted to schema 4 (decompose takes the flat ContentDocument; the envelope rides the package root). + +const SECTION_GEOMETRY = { pageSize: { widthPt: 595, heightPt: 842 }, margins: { topPt: 72, rightPt: 72, bottomPt: 72, leftPt: 72 } }; + +function run(text: string): { text: string } { + return { text }; +} + +function paragraph(text: string, options: { headingLevel?: number; listLevel?: number } = {}): ContentBlock { + return { + kind: 'paragraph', + runs: [run(text)], + ...(options.headingLevel !== undefined ? { headingLevel: options.headingLevel } : {}), + // numId omitted deliberately on list paragraphs: schema 4.0.0 made it optional, and OOXML drawing paragraphs carry only a level -- the exact slide-body shape the presentation decomposition nests by. + ...(options.listLevel !== undefined ? { list: { level: options.listLevel } } : {}), + }; +} + +function table(text: string): ContentBlock { + return { kind: 'table', rows: [{ cells: [{ blocks: [paragraph(text)] }] }], columnWidthsPt: [80] }; +} + +function constructStart(descriptor: ConstructDescriptor): ContentBlock { + return { kind: 'constructStart', descriptor }; +} + +const CONSTRUCT_END: ContentBlock = { kind: 'constructEnd' }; + +function wordprocessingDoc(blocksPerSection: ContentBlock[][]): ContentDocument { + return { kind: 'wordprocessing', metadata: {}, sections: blocksPerSection.map((blocks) => ({ ...SECTION_GEOMETRY, blocks })) }; +} + +function shape(blocks: ContentBlock[]): ContentShape { + return { frame: { xPt: 0, yPt: 0, widthPt: 600, heightPt: 400 }, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0, blocks }; +} + +describe('wordprocessing decomposition', () => { + it('makes one section group per section, carrying the section geometry on the descriptor', () => { + const doc = wordprocessingDoc([[paragraph('first')], [paragraph('second')]]); + expect(decompose(doc)).toEqual([ + { node: { kind: 'section', pageSize: SECTION_GEOMETRY.pageSize, margins: SECTION_GEOMETRY.margins }, children: [paragraph('first')] }, + { node: { kind: 'section', pageSize: SECTION_GEOMETRY.pageSize, margins: SECTION_GEOMETRY.margins }, children: [paragraph('second')] }, + ]); + }); + + it('resets the heading stack at each section boundary instead of flowing sections into one tree', () => { + const doc = wordprocessingDoc([ + [paragraph('Chapter', { headingLevel: 1 }), paragraph('section one body')], + [paragraph('Method', { headingLevel: 2 }), paragraph('section two body')], + ]); + const [first, second] = decompose(doc); + // An H2 directly at a section root is the observable difference from a table-of-contents tree, where the same H2 would nest under the still-open H1 of the previous section: decompose groups per container, so each section starts with an empty stack. + expect(second).toEqual({ + node: { kind: 'section', pageSize: SECTION_GEOMETRY.pageSize, margins: SECTION_GEOMETRY.margins }, + children: [{ node: paragraph('Method', { headingLevel: 2 }), children: [paragraph('section two body')] }], + }); + expect(first).toEqual({ + node: { kind: 'section', pageSize: SECTION_GEOMETRY.pageSize, margins: SECTION_GEOMETRY.margins }, + children: [{ node: paragraph('Chapter', { headingLevel: 1 }), children: [paragraph('section one body')] }], + }); + }); + + it('nests headings, lists, and leaves with the stack semantics', () => { + const h1 = paragraph('Chapter', { headingLevel: 1 }); + const a = paragraph('A', { listLevel: 0 }); + const b = paragraph('B', { listLevel: 1 }); + const plain = paragraph('plain closes the list'); + const cells = table('x'); + const img: ContentBlock = { kind: 'image', format: 'png', base64: 'aW1hZ2U=', widthPt: 100, heightPt: 60 }; + const h4 = paragraph('Deep', { headingLevel: 4 }); + const doc = wordprocessingDoc([[h1, a, b, cells, plain, img, h4]]); + expect(decompose(doc)).toEqual([ + { + node: { kind: 'section', pageSize: SECTION_GEOMETRY.pageSize, margins: SECTION_GEOMETRY.margins }, + children: [ + { + node: h1, + children: [ + { node: a, children: [{ node: b, children: [cells] }] }, + plain, + img, + // An H4 with no open heading shallower than 4 attaches flat under the heading scope (its own group with the H1 still open above it), the builder's pop rule verbatim. + { node: h4, children: [] }, + ], + }, + ], + }, + ]); + }); + + it('decomposes an empty document to an empty root array (the envelope carries the kind)', () => { + expect(decompose({ kind: 'wordprocessing', metadata: {}, sections: [] })).toEqual([]); + expect(flattenPackage({ kind: 'wordprocessing', metadata: {}, children: [] })).toEqual({ kind: 'wordprocessing', metadata: {}, sections: [] }); + }); +}); + +describe('presentation decomposition', () => { + it('groups each shape separately and never flattens a slide across its shapes', () => { + const shapeA = shape([paragraph('A top', { listLevel: 0 }), paragraph('A nested', { listLevel: 1 })]); + const headingInShape = paragraph('a heading-styled paragraph is an ordinary leaf here', { headingLevel: 2 }); + const shapeB = shape([paragraph('B plain'), headingInShape]); + const doc: ContentDocument = { kind: 'presentation', metadata: {}, slides: [{ size: { widthPt: 960, heightPt: 540 }, shapes: [shapeA, shapeB], notes: 'notes ride the descriptor' }] }; + expect(decompose(doc)).toEqual([ + { + node: { kind: 'slide', size: { widthPt: 960, heightPt: 540 }, notes: 'notes ride the descriptor' }, + children: [ + { + node: { frame: shapeA.frame, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0 }, + children: [{ node: paragraph('A top', { listLevel: 0 }), children: [{ node: paragraph('A nested', { listLevel: 1 }), children: [] }] }], + }, + // headingLevel inside a shape is deliberately present and deliberately not a grouping signal: shapes carry list nesting only, so the paragraph stays a bare leaf carrying the field. + { + node: { frame: shapeB.frame, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0 }, + children: [paragraph('B plain'), headingInShape], + }, + ], + }, + ]); + }); +}); + +describe('spreadsheet decomposition', () => { + it('rides the grid on the sheet node and carries images then embedded objects as children', () => { + const printSettings: ContentSheetPrintSettings = { pageSize: { widthPt: 595, heightPt: 842 }, margins: { topPt: 20, rightPt: 20, bottomPt: 20, leftPt: 20 }, gridlines: true, headers: true, pageOrder: 'downThenOver' }; + const cell: ContentSheetCell = { row: 0, column: 0, value: { kind: 'number', value: 1 }, displayText: '1' }; + const image: ContentSheetImage = { kind: 'image', format: 'png', base64: 'aW1hZ2U=', widthPt: 10, heightPt: 10, anchorRow: 0, anchorColumn: 0, offsetXPt: 0, offsetYPt: 0 }; + const embedded: ContentEmbeddedObject = { objectKind: 'drawing', document: { kind: 'drawing', metadata: {}, pages: [] }, frame: { xPt: 0, yPt: 0, widthPt: 1, heightPt: 1 } }; + const emptyGrid = { cells: [], columns: [], rows: [], images: [], printSettings }; + const doc: ContentDocument = { + kind: 'spreadsheet', + metadata: {}, + sheets: [ + { name: 'Data', cells: [cell], columns: [{ index: 0, widthPt: 60 }], rows: [{ index: 0, heightPt: 12 }], images: [image], printSettings }, + { name: 'Model', ...emptyGrid, embeddedObjects: [embedded] }, + { name: 'Declared empty', ...emptyGrid, embeddedObjects: [] }, + ], + }; + expect(decompose(doc)).toEqual([ + { + // The descriptor carries the grid (cells/columns/rows/printSettings) minus the two arrays whose members became children. + node: { kind: 'sheet', name: 'Data', cells: [cell], columns: [{ index: 0, widthPt: 60 }], rows: [{ index: 0, heightPt: 12 }], printSettings }, + children: [image], + }, + { node: { kind: 'sheet', name: 'Model', cells: [], columns: [], rows: [], printSettings }, children: [embedded] }, + { node: { kind: 'sheet', name: 'Declared empty', cells: [], columns: [], rows: [], printSettings }, children: [] }, + ]); + // A present-but-empty embeddedObjects array round-trips to the field ABSENT: decompose concatenated images and embedded objects into one children array, so a declared-empty field is indistinguishable from an absent one -- the bijection's one declared normalisation, pinned here in its stripping direction. (images rebuilds as the always-present empty array because the flat form requires it.) + const flat = flattenPackage({ kind: 'spreadsheet', metadata: {}, children: doc.sheets.map(decomposeSheet) }); + expect(flat.kind).toBe('spreadsheet'); + if (flat.kind !== 'spreadsheet') throw new Error('expected a spreadsheet document'); + expect(flat.sheets[2]).not.toHaveProperty('embeddedObjects'); + expect(flat.sheets[1]).toHaveProperty('embeddedObjects'); + expect(flat.sheets[2]).toHaveProperty('images'); + }); + + it('refuses a style ref on a sheet group loudly -- a sheet holds no block flow to resolve it onto', () => { + // The schema permits style on every group node, but the spreadsheet arm builds no resolution chain (a sheet's children are images and embedded objects, not paragraphs), so a ref there could only be passed by silently -- losing the styled content with no signal. The refusal mirrors entryOf's missing-table rule: resolution runs completely or not at all. Minting never stamps such a ref (a sheet group's extent is always empty); this guard is for hand-built trees. + const sheet: SheetGroupNode = { node: { kind: 'sheet', name: 'Data', cells: [], columns: [], rows: [], printSettings: { pageSize: { widthPt: 595, heightPt: 842 }, margins: { topPt: 20, rightPt: 20, bottomPt: 20, leftPt: 20 }, gridlines: true, headers: true, pageOrder: 'downThenOver' } }, children: [], style: 's1' }; + expect(() => flattenPackage({ kind: 'spreadsheet', metadata: {}, children: [sheet] })).toThrow(/no block flow/); + }); +}); + +// document-schema.js 4.1.0 added SectionConstructGroupNode/ShapeConstructGroupNode (docx SDTs, ODF fields, tracked changes, and the rest of document-schema.js#22's fidelity-construct vocabulary) to the tree; 4.2.0 gave ContentBlock the matching flat carrier, the constructStart/constructEnd marker pair, so the boundary is now a promotion like every other grouping signal rather than a shape only a hand-built tree could hold. These tests pin the promotion's SHAPE -- which scope a construct attaches at, what its interior groups by, and what the outer stacks do while it is open. Round-trip fidelity itself is the bijection suite's job (bijection.test.ts carries the construct corpus entries). +describe('construct-boundary promotion', () => { + it('promotes a marker pair into one group whose children are the delimited region decomposed on its own', () => { + const insideHeading = paragraph('inside', { headingLevel: 1 }); + const doc = wordprocessingDoc([[paragraph('before'), constructStart({ kind: 'field', instruction: 'PAGE' }), insideHeading, paragraph('inside body'), CONSTRUCT_END, paragraph('after')]]); + expect(decompose(doc)).toEqual([ + { + node: { kind: 'section', pageSize: SECTION_GEOMETRY.pageSize, margins: SECTION_GEOMETRY.margins }, + children: [ + paragraph('before'), + // The construct's interior walks with FRESH heading/list stacks, so its own H1 groups inside it... + { node: { kind: 'field', instruction: 'PAGE' }, children: [{ node: insideHeading, children: [paragraph('inside body')] }] }, + // ...and closes with the region: `after` lands back at the section root rather than under the H1 the construct opened, which is what "the outer stacks are undisturbed" means in the one direction a leak would be invisible in the flat form. + paragraph('after'), + ], + }, + ]); + }); + + it('leaves the enclosing heading and list stacks exactly where they were', () => { + const h1 = paragraph('Chapter', { headingLevel: 1 }); + const first = paragraph('A', { listLevel: 0 }); + const second = paragraph('B', { listLevel: 1 }); + const doc = wordprocessingDoc([[h1, first, constructStart({ kind: 'anchor', anchorType: 'bookmark', name: 'b1' }), paragraph('inside'), CONSTRUCT_END, second]]); + expect(decompose(doc)).toEqual([ + { + node: { kind: 'section', pageSize: SECTION_GEOMETRY.pageSize, margins: SECTION_GEOMETRY.margins }, + children: [ + { + node: h1, + children: [ + { + node: first, + // B still nests under A across the intervening construct: stepping through a region neither pops the list stack (as a plain paragraph would) nor the heading stack. + children: [{ node: { kind: 'anchor', anchorType: 'bookmark', name: 'b1' }, children: [paragraph('inside')] }, { node: second, children: [] }], + }, + ], + }, + ], + }, + ]); + }); + + it('groups a construct inside a list item by list level alone -- the list-flow vocabulary its position admits', () => { + // A list group's children are ListChild, which admits a ShapeConstructGroupNode and no heading group at all, so a heading paragraph inside a construct inside a list is ordinary content -- exactly what it already is anywhere else in a list group's subtree. The section-root case above shows the other half: there the position is SectionChild, so the same marker pair promotes to a SectionConstructGroupNode whose interior does group headings. + const item = paragraph('A', { listLevel: 0 }); + const headingInside = paragraph('a heading-styled paragraph is an ordinary leaf here', { headingLevel: 2 }); + const doc = wordprocessingDoc([[item, constructStart({ kind: 'contentControl', controlType: 'richText' }), headingInside, CONSTRUCT_END]]); + expect(decompose(doc)).toEqual([ + { + node: { kind: 'section', pageSize: SECTION_GEOMETRY.pageSize, margins: SECTION_GEOMETRY.margins }, + children: [{ node: item, children: [{ node: { kind: 'contentControl', controlType: 'richText' }, children: [headingInside] }] }], + }, + ]); + }); + + it('nests constructs of different kinds to arbitrary depth, and admits an empty region', () => { + const doc = wordprocessingDoc([[ + constructStart({ kind: 'provenance', change: 'insertion', author: 'A' }), + constructStart({ kind: 'link', target: { kind: 'external', uri: 'https://example.invalid/' } }), + paragraph('deep'), + CONSTRUCT_END, + constructStart({ kind: 'division', name: 'empty' }), + CONSTRUCT_END, + CONSTRUCT_END, + ]]); + expect(decompose(doc)).toEqual([ + { + node: { kind: 'section', pageSize: SECTION_GEOMETRY.pageSize, margins: SECTION_GEOMETRY.margins }, + children: [ + { + node: { kind: 'provenance', change: 'insertion', author: 'A' }, + children: [ + { node: { kind: 'link', target: { kind: 'external', uri: 'https://example.invalid/' } }, children: [paragraph('deep')] }, + // An open marker immediately followed by its close is a real, schema-legal region with no content: it promotes to a group with no children rather than collapsing to nothing, so flatten reproduces the pair. + { node: { kind: 'division', name: 'empty' }, children: [] }, + ], + }, + ], + }, + ]); + }); + + it('promotes a construct in a shape flow the same way, on the shape vocabulary', () => { + const inner = shape([constructStart({ kind: 'field', instruction: 'PAGE' }), paragraph('in a shape'), CONSTRUCT_END]); + const doc: ContentDocument = { kind: 'presentation', metadata: {}, slides: [{ size: { widthPt: 960, heightPt: 540 }, shapes: [inner], notes: '' }] }; + expect(decompose(doc)).toEqual([ + { + node: { kind: 'slide', size: { widthPt: 960, heightPt: 540 }, notes: '' }, + children: [ + { + node: { frame: inner.frame, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0 }, + children: [{ node: { kind: 'field', instruction: 'PAGE' }, children: [paragraph('in a shape')] }], + }, + ], + }, + ]); + }); + + it('embeds the source descriptor object itself, and rebuilds the marker pair around it on the way back', () => { + // The ownership discipline, at the one node it cannot hold verbatim: PackageBlockLeaf excludes both marker kinds, so the tree keeps the descriptor (the same object, by identity) while the marker wrapper is reconstructed by flatten. A consumer holding both views still sees one descriptor. + const descriptor: ConstructDescriptor = { kind: 'field', instruction: 'PAGE' }; + const start: ContentBlock = { kind: 'constructStart', descriptor }; + const source: ContentSection = { ...SECTION_GEOMETRY, blocks: [start, paragraph('inside'), CONSTRUCT_END] }; + const group = decomposeSection(source); + const [constructGroup] = group.children; + if (constructGroup === undefined || !isSectionConstructGroupNode(constructGroup)) { + throw new Error('expected the marker pair to promote to a construct group'); + } + expect(constructGroup.node).toBe(descriptor); + const flat = flattenPackage({ kind: 'wordprocessing', metadata: {}, children: [group] }); + if (flat.kind !== 'wordprocessing') throw new Error('expected a wordprocessing document back'); + const section = flat.sections[0]; + if (section === undefined) throw new Error('expected one section back'); + expect(section.blocks).toEqual([start, paragraph('inside'), CONSTRUCT_END]); + const [rebuiltStart] = section.blocks; + if (rebuiltStart?.kind !== 'constructStart') throw new Error('expected the open marker back first'); + expect(rebuiltStart.descriptor).toBe(descriptor); + expect(rebuiltStart).not.toBe(start); + }); +}); + +// Promotion is defined only over a balanced marker stream, so an unbalanced one is refused outright rather than repaired into a plausible tree -- the same "fail loudly, never silently skip" rule the sheet-group style-ref guard above follows. The thrown error carries document-schema.js's own ConstructMarkerImbalance payload, so a caller gets the offending block index without parsing a message. +describe('construct marker imbalance', () => { + it('refuses a close marker that closes no open construct', () => { + const doc = wordprocessingDoc([[paragraph('before'), CONSTRUCT_END]]); + expect(() => decompose(doc)).toThrow(ConstructMarkerImbalanceError); + try { + decompose(doc); + } catch (error) { + if (!(error instanceof ConstructMarkerImbalanceError)) throw error; + expect(error.imbalance).toEqual({ kind: 'unmatchedEnd', index: 1 }); + } + }); + + it('refuses a block stream that ends with a construct still open', () => { + const doc = wordprocessingDoc([[constructStart({ kind: 'field', instruction: 'PAGE' }), paragraph('inside')]]); + expect(() => decompose(doc)).toThrow(ConstructMarkerImbalanceError); + try { + decompose(doc); + } catch (error) { + if (!(error instanceof ConstructMarkerImbalanceError)) throw error; + expect(error.imbalance).toEqual({ kind: 'unclosedStart', index: 0 }); + } + }); + + it('refuses an unbalanced shape flow too -- the check runs per container block stream, not per document', () => { + const doc: ContentDocument = { kind: 'presentation', metadata: {}, slides: [{ size: { widthPt: 960, heightPt: 540 }, shapes: [shape([CONSTRUCT_END])], notes: '' }] }; + expect(() => decompose(doc)).toThrow(ConstructMarkerImbalanceError); + }); +}); + +describe('drawing and formula decomposition', () => { + it('orders a draw page\'s children shapes-then-vectors and nests each shape\'s flow inside it', () => { + const vector: ContentVector = { kind: 'rect', frame: { xPt: 1, yPt: 2, widthPt: 3, heightPt: 4 } }; + const labelled = shape([paragraph('label')]); + const doc: ContentDocument = { kind: 'drawing', metadata: {}, pages: [{ size: { widthPt: 300, heightPt: 300 }, shapes: [labelled], vectors: [vector] }] }; + expect(decompose(doc)).toEqual([ + { + node: { kind: 'drawPage', size: { widthPt: 300, heightPt: 300 } }, + children: [ + { node: { frame: labelled.frame, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0 }, children: [paragraph('label')] }, + vector, + ], + }, + ]); + }); + + it('decomposes a formula document to its single ContentFormula leaf', () => { + const formula: ContentFormula = { mathml: [{ type: 'text', value: 'x' }], starMath: 'x' }; + const doc: ContentDocument = { kind: 'formula', metadata: {}, formula }; + expect(decompose(doc)).toEqual([formula]); + }); +}); + +// The ownership rule as a positive identity check: decompose embeds the document's own objects (leaves are the same references, never copies), and flatten emits those same objects back into block flow. The bijection laws deliberately never use toBe; this one deliberately does, because sharing IS the contract being pinned -- a consumer holding both views sees an edit through either. +describe('ownership', () => { + it('embeds the source nodes themselves, not copies', () => { + const heading = paragraph('Chapter', { headingLevel: 1 }); + const body = paragraph('body'); + const source: ContentSection = { ...SECTION_GEOMETRY, blocks: [heading, body] }; + const sectionGroup = decomposeSection(source); + const [headingGroup] = sectionGroup.children; + // A plain 'node'/'children' presence check no longer narrows out every non-anchor shape: since document-schema.js 4.1.0, a SectionConstructGroupNode carries both too. isHeadingGroupNode/isListGroupNode are the real schema guards, so reaching for them here (rather than reinventing the anchor-vs-construct narrow this test doesn't need to know about) both fixes the narrowing and states the assertion's actual intent. + if (headingGroup === undefined || !(isHeadingGroupNode(headingGroup) || isListGroupNode(headingGroup))) { + throw new Error('expected the heading paragraph to open the section flow'); + } + expect(headingGroup.node).toBe(heading); + expect(isHeadingParagraph(headingGroup.node)).toBe(true); + const pkg: DocumentPackage = { kind: 'wordprocessing', metadata: {}, children: [sectionGroup] }; + const flat = flattenPackage(pkg); + if (flat.kind !== 'wordprocessing') throw new Error('expected a wordprocessing document back'); + const [section] = flat.sections; + if (section === undefined) throw new Error('expected one section back'); + const [firstBlock, secondBlock] = section.blocks; + expect(firstBlock).toBe(heading); + expect(secondBlock).toBe(body); + }); +}); diff --git a/src/decompose.ts b/src/decompose.ts new file mode 100644 index 0000000..bf7b3e8 --- /dev/null +++ b/src/decompose.ts @@ -0,0 +1,229 @@ +import { + findConstructMarkerImbalance, + type ConstructMarkerImbalance, + type ContentBlock, + type ContentDocument, + type ContentDrawPage, + type ContentFormula, + type ContentParagraph, + type ContentSection, + type ContentShape, + type ContentSheet, + type ContentSlide, +} from './content'; +import type { ConstructDescriptor } from './construct'; +import type { + DrawPageGroupNode, + HeadingGroupNode, + HeadingParagraph, + ListGroupNode, + ListParagraph, + SectionChild, + SectionGroupNode, + ShapeChild, + ShapeGroupNode, + SheetGroupNode, + SlideGroupNode, +} from './package-node'; + +// The flat-to-tree half of the package boundary, ported from document-outline.js's phase-1 reference implementation (that repo's pre-re-charter git history) onto this package's own tree vocabulary -- src/package-node.ts is the same shape's schema-home port, so this module imports the group types instead of redeclaring them. The one forced adaptation beyond types: the reference decomposed a 3.x DocumentPackage's `content` field and rebuilt envelopes through a separate documentEnvelope helper, while the 4.x DocumentPackage IS the tree (the envelope -- kind, metadata, symbolTable -- rides the root), so decompose takes the flat ContentDocument directly and assemblePackage splices the envelope fields out of it. Everything else -- the stack semantics, the container rule, the ownership discipline -- is the reviewed reference verbatim. +// +// Decomposition follows the container rule: sections, slides, sheets, and draw pages each become one top-level group per container, a shape is its own group with its inner blocks grouped inside it (never a slide's paragraphs flattened across its shapes -- that is a table-of-contents projection, not a decomposition), a sheet's grid rides ON the sheet node while children carry its images and embedded objects, and an embedded document (the recursive ContentEmbeddedObject arm) stays intact as one leaf. The input's node objects are embedded, not copied -- decompose owns no content, it only wraps -- so the frames a layout pass stamped onto content survive into the tree as the same objects. The one node the tree cannot embed is a construct-boundary marker: PackageBlockLeaf excludes both marker kinds by construction, so a constructStart's own ConstructDescriptor becomes the group's `node` (that descriptor object IS embedded, uncopied) while the marker wrapper around it has no tree spelling and is rebuilt by flatten from the descriptor plus the group's extent. +// +// Construct boundaries (4.2.0's constructStart/constructEnd blocks in src/content.ts -- docx SDTs, ODF fields, tracked changes, and the rest of #22's fidelity-construct vocabulary) are promoted here, not passed through: a marker pair delimits a region of the block stream that becomes one construct group whose children are that region decomposed on its own. The region's extent is stated by the markers themselves rather than by any level comparison, which is why the block walks below run off a shared forward cursor rather than a plain for-of loop -- a nested walk consumes from exactly where its parent stopped and hands the position back by returning once it meets its own close marker. Promotion reaches exactly as far as grouping does, which is one container's own block flow: a marker inside a table cell's blocks or inside an embedded document rides through untouched on its leaf, unpromoted and unchecked, the same boundary a heading level inside a table cell already sits outside of. + +// Property-presence predicates over ContentParagraph, so decompose's branches narrow the paragraph itself (not just a property access) to the anchor types -- a plain property `!== undefined` check narrows the access, never the object, and `{ node: paragraph }` needs the narrowed object. The schema exports the anchor types but not these predicates; they are three lines of structural fact, not a second copy of a schema shape. +export function isHeadingParagraph(paragraph: ContentParagraph): paragraph is HeadingParagraph { + return paragraph.headingLevel !== undefined; +} + +export function isListParagraph(paragraph: ContentParagraph): paragraph is ListParagraph { + return paragraph.list !== undefined; +} + +// A container's block stream whose construct markers do not pair up: a constructEnd closing nothing, or a constructStart never closed. Promotion is defined only over balanced markers -- an unbalanced stream has no region for the group to span -- so decompose refuses it loudly rather than inventing a boundary (silently closing an unclosed construct at the container's end, or dropping a stray close) and losing the producer's real error in a plausible-looking tree. Balance is a per-container property, checked over one section's, shape's, or sheet's own block flow: a pair cannot span two containers, because each container decomposes independently. The payload is src/content.ts's own ConstructMarkerImbalance so the failure vocabulary stays the one the balance check already states rather than a second one restated here, and `imbalance.index` indexes that container's own blocks. +export class ConstructMarkerImbalanceError extends Error { + readonly imbalance: ConstructMarkerImbalance; + + constructor(imbalance: ConstructMarkerImbalance) { + super( + imbalance.kind === 'unmatchedEnd' + ? `decompose: the constructEnd marker at index ${imbalance.index} of this container's block flow closes no open construct` + : `decompose: the constructStart marker at index ${imbalance.index} of this container's block flow is never closed`, + ); + this.name = 'ConstructMarkerImbalanceError'; + this.imbalance = imbalance; + } +} + +// What decompose returns and every package arm's children hold: the per-kind roots the schema's DocumentPackage union states (section groups for wordprocessing, slide groups for presentation, sheet groups for spreadsheet, draw-page groups for drawing, the single ContentFormula leaf for formula). Spelled as the explicit union rather than the indexed access DocumentPackage['children'] so tooling that resolves types one hop at a time (this repo's ESLint typed rules) sees named imports instead of an index into the zod-inferred union. +export type PackageChildren = SectionGroupNode[] | SlideGroupNode[] | SheetGroupNode[] | DrawPageGroupNode[] | ContentFormula[]; + +// Decomposes a flat ContentDocument into the tree a DocumentPackage's children carry. The same node objects are embedded, never cloned -- the one exception being a construct-boundary marker pair, which has no tree spelling of its own and contributes its ConstructDescriptor (that object, uncopied) to the group it promotes to. Throws ConstructMarkerImbalanceError when a container's markers do not pair up. +export function decompose(content: ContentDocument): PackageChildren { + switch (content.kind) { + case 'wordprocessing': + return content.sections.map(decomposeSection); + case 'presentation': + return content.slides.map(decomposeSlide); + case 'spreadsheet': + return content.sheets.map(decomposeSheet); + case 'drawing': + return content.pages.map(decomposeDrawPage); + case 'formula': + return [content.formula]; + } +} + +// Section groups are mandatory, one per ContentSection: the descriptor keeps the section's own page geometry (which no rendered-pages array can hold), and the section's blocks become the group's children, grouped by the wordprocessing stack semantics -- but per section, because each section is its own container and the heading/list stacks reset at its boundary rather than flowing across sections the way a table-of-contents view deliberately does. Rest-destructuring lifts exactly `blocks` out, so a ContentSection field added by a future schema release rides the descriptor without this package being touched. +export function decomposeSection(section: ContentSection): SectionGroupNode { + const { blocks, ...rest } = section; + return { node: { kind: 'section', ...rest }, children: decomposeSectionBlocks(blocks) }; +} + +// One forward cursor over a container's flat block stream, shared by every walk below it. A construct region's extent is delimited by markers inside the stream, so a nested walk must start exactly where its parent stopped and leave the parent resuming exactly past the close marker it consumed -- which is precisely what one shared iterator is: the parent keeps pulling after the nested call returns, with no index to thread back or get wrong. An iterator rather than an index into `blocks` also keeps end-of-stream a real answer (the iterator protocol's own done flag) instead of an indexed read that noUncheckedIndexedAccess types as possibly-undefined and that would need a branch nothing can ever take. +type BlockCursor = Iterator; + +// Refuses a block stream whose construct markers do not pair up, before any grouping runs. The check is the schema's own findConstructMarkerImbalance rather than a second implementation folded into the walks, which is also what lets each walk return plainly at end-of-stream: over a balanced stream a nested walk always terminates on its own close marker, so no walk needs an unreachable "ran out mid-construct" arm of its own. +function assertBalancedConstructMarkers(blocks: readonly ContentBlock[]): void { + const imbalance = findConstructMarkerImbalance(blocks); + if (imbalance !== undefined) { + throw new ConstructMarkerImbalanceError(imbalance); + } +} + +// The wordprocessing stack semantics: a heading paragraph opens a group nested under the deepest open heading group with a strictly shallower level, popping equal-or-deeper groups closed (an H4 after an H2 becomes its direct child with no synthetic intermediates; an H1 after an H3 pops to the root); list paragraphs nest by list.level on the same stack semantics inside the innermost heading scope; non-paragraph blocks attach as leaves at the current depth without changing it; a plain paragraph -- no heading level, no list membership -- sits flat at its scope and closes the list nesting. headingLevel is the only heading signal read; a Heading styleId without headingLevel does not group. +function decomposeSectionBlocks(blocks: readonly ContentBlock[]): SectionChild[] { + assertBalancedConstructMarkers(blocks); + return walkSectionBlocks(blocks.values()); +} + +// Consumes a section flow off the cursor, returning at the constructEnd marker that closes the region it was called for (or at end of stream, for the container's own outermost call). +function walkSectionBlocks(cursor: BlockCursor): SectionChild[] { + const root: SectionChild[] = []; + // Heading groups currently open, deepest last. Each entry is the group itself -- a group carries both its anchor paragraph (and thereby its level) and its children, so it is the scope. An empty stack means the section root: content before any heading, and sections with no headings at all, attach directly to the section group's children. + const headingStack: HeadingGroupNode[] = []; + // List groups currently open inside the innermost heading scope. Reset by every heading and every plain paragraph: list nesting is a sub-structure of a heading group, never a bridge across groups or across intervening unlevelled paragraphs -- otherwise a later deeper item would traverse before an earlier sibling and document order would not survive flatten. + const listStack: ListGroupNode[] = []; + // The scope every non-heading attachment targets: the innermost open heading group's children, or the section root when no heading is open. + const headingScope = (): SectionChild[] => headingStack.at(-1)?.children ?? root; + for (let step = cursor.next(); step.done !== true; step = cursor.next()) { + const block = step.value; + if (block.kind === 'constructEnd') { + return root; + } + if (block.kind === 'constructStart') { + openConstructGroup(block.descriptor, cursor, listStack, headingScope()); + continue; + } + if (block.kind !== 'paragraph') { + const parent = listStack.at(-1); + (parent !== undefined ? parent.children : headingScope()).push(block); + continue; + } + if (isHeadingParagraph(block)) { + listStack.length = 0; + const level = block.headingLevel; + for (let top = headingStack.at(-1); top !== undefined && top.node.headingLevel >= level; top = headingStack.at(-1)) { + headingStack.pop(); + } + const group: HeadingGroupNode = { node: block, children: [] }; + const parent = headingStack.at(-1); + (parent !== undefined ? parent.children : root).push(group); + headingStack.push(group); + } else if (isListParagraph(block)) { + openListGroup(listStack, headingScope(), block); + } else { + listStack.length = 0; + headingScope().push(block); + } + } + return root; +} + +// A slide becomes one group whose children are one shape group per shape, in shape order -- never the slide's paragraphs taken across its shapes, which would silently discard the shape boundary the source format carries. notes rides the descriptor because it is slide-level data with no block-flow position, exactly like a section's own margins. +export function decomposeSlide(slide: ContentSlide): SlideGroupNode { + const { shapes, ...rest } = slide; + return { node: { kind: 'slide', ...rest }, children: shapes.map(decomposeShape) }; +} + +// A sheet's children are its images then its embedded objects (sibling arrays with no cross-array ordering field; this fixed order is what flatten's partition reverses). The grid and print settings ride ON the sheet descriptor -- they are addressable data, not block flow. embeddedObjects is optional on ContentSheet (a sheet may legitimately carry none), so absence spreads as nothing: children then hold images alone, and flatten reconstructs the field's absence from exactly that. A present-but-empty array also contributes no children -- the one spelling the concatenated children cannot distinguish from absence -- so it round-trips to the field absent: the bijection's one declared normalisation, stated in bijection.test.ts rather than left to be discovered as a law failure. +export function decomposeSheet(sheet: ContentSheet): SheetGroupNode { + const { images, embeddedObjects, ...rest } = sheet; + const children = [...images, ...(embeddedObjects ?? [])]; + return { node: { kind: 'sheet', ...rest }, children }; +} + +// A drawing page's children are its shape groups then its vectors: shapes are containers of their own (groups), vectors are textless primitives with no inner structure (leaves) that stay in the tree so structural diffing still sees them. +export function decomposeDrawPage(page: ContentDrawPage): DrawPageGroupNode { + const { shapes, vectors, ...rest } = page; + const children = [...shapes.map(decomposeShape), ...vectors]; + return { node: { kind: 'drawPage', ...rest }, children }; +} + +// A shape is its own group: its frame and insets ride the descriptor (blocks lifted out by the rest-destructure, so future ContentShape fields ride too), and its inner blocks group by list.level inside it. headingLevel is deliberately not read in a shape's flow -- slides and drawing pages have no heading hierarchy of their own, and list.level is the only depth signal their paragraphs actually carry. The list stack is per shape: list nesting never crosses the shape boundary either. +export function decomposeShape(shape: ContentShape): ShapeGroupNode { + const { blocks, ...rest } = shape; + return { node: rest, children: decomposeShapeBlocks(blocks) }; +} + +function decomposeShapeBlocks(blocks: readonly ContentBlock[]): ShapeChild[] { + assertBalancedConstructMarkers(blocks); + return walkShapeBlocks(blocks.values()); +} + +// Consumes a shape (or list-item) flow off the cursor, on the same return-at-my-close-marker contract walkSectionBlocks holds. +function walkShapeBlocks(cursor: BlockCursor): ShapeChild[] { + const root: ShapeChild[] = []; + const listStack: ListGroupNode[] = []; + for (let step = cursor.next(); step.done !== true; step = cursor.next()) { + const block = step.value; + if (block.kind === 'constructEnd') { + return root; + } + if (block.kind === 'constructStart') { + const parent = listStack.at(-1); + (parent !== undefined ? parent.children : root).push({ node: block.descriptor, children: walkShapeBlocks(cursor) }); + continue; + } + if (block.kind !== 'paragraph') { + const parent = listStack.at(-1); + (parent !== undefined ? parent.children : root).push(block); + continue; + } + if (isListParagraph(block)) { + openListGroup(listStack, root, block); + continue; + } + // A paragraph with no list membership sits directly under the shape group and closes any open list nesting, so document order survives the walk back out. This includes a paragraph carrying headingLevel: in a shape's flow that field is not a depth signal (see decomposeShape), so the paragraph is plain content here. + listStack.length = 0; + root.push(block); + } + return root; +} + +// Attaches the construct region a section-flow constructStart opened, recursing over the cursor to build its children and resuming the caller past the close marker that recursion consumed. Two things follow from a construct being a semantic wrapper rather than a container: it attaches at the CURRENT scope exactly as any other non-paragraph block does (the innermost open list group, else the innermost open heading scope), and it disturbs neither stack -- content after the close marker resumes at the same heading depth and the same list depth as content before the open marker. Its own children walk with FRESH stacks, the same reset a section boundary already performs, because the region is its own flow. Which stack it lands in also decides which group type it is, and the schema states both halves of that: the section flow's SectionChild admits a SectionConstructGroupNode (whose children are a full section flow, headings included), while a list group's ListChild admits only a ShapeConstructGroupNode (whose children are a list/shape flow, where a heading paragraph is ordinary content) -- a construct inside a list therefore groups its interior by list level alone, which is exactly the vocabulary a list group's subtree already has. +function openConstructGroup(descriptor: ConstructDescriptor, cursor: BlockCursor, listStack: readonly ListGroupNode[], headingScope: SectionChild[]): void { + const parent = listStack.at(-1); + if (parent === undefined) { + headingScope.push({ node: descriptor, children: walkSectionBlocks(cursor) }); + return; + } + parent.children.push({ node: descriptor, children: walkShapeBlocks(cursor) }); +} + +// Opens a list-item group anchored on `paragraph` at its list.level under the deepest open list group with a strictly shallower level (or directly under `scopeChildren` when none is open), popping equal-or-deeper groups closed -- the same stack semantics heading groups follow, on list.level's 0-based scale, so a level jump nests directly under the nearest shallower item with no synthetic intermediates. +function openListGroup( + listStack: ListGroupNode[], + scopeChildren: SectionChild[] | ShapeChild[], + paragraph: ListParagraph, +): void { + const level = paragraph.list.level; + for (let top = listStack.at(-1); top !== undefined && top.node.list.level >= level; top = listStack.at(-1)) { + listStack.pop(); + } + const group: ListGroupNode = { node: paragraph, children: [] }; + const parent = listStack.at(-1); + (parent !== undefined ? parent.children : scopeChildren).push(group); + listStack.push(group); +} diff --git a/src/definitions.ts b/src/definitions.ts index d7d5bb0..8901e9e 100644 --- a/src/definitions.ts +++ b/src/definitions.ts @@ -3,7 +3,7 @@ import { ColorSchema } from './color'; import { ContentListMembershipSchema, type ContentParagraph, type ContentRun } from './content'; import { AlignmentSchema } from './style'; -// The package-level definitions-table facility (ExaDev/document-schema.js#21): named tables at the DocumentPackage root whose entries tree nodes reference by string id, so repeated data is stated once and referenced many times. Styles were the first tenant (the StylesTableSchema below), and the tenant-generic DefinitionsTableSchema beside it is what let every later tenant land without this module changing: link, footnote, and comment definitions (ExaDev/markdown-codec#63, ExaDev/document-schema.js#22) ride the `definitions` field, and 4.1.0's three construct tables -- `layers`, `attachments`, `destinations` (ExaDev/document-schema.js#24) -- are three more root fields of this same generic type rather than three parallel shapes. This module defines the schemas and the pure resolution helpers only; minting entries (the frequency pass that factors repeated property tuples into table refs) is documents.js's boundary behaviour, not this package's. +// The package-level definitions-table facility (ExaDev/document-schema.js#21): named tables at the DocumentPackage root whose entries tree nodes reference by string id, so repeated data is stated once and referenced many times. Styles were the first tenant (the StylesTableSchema below), and the tenant-generic DefinitionsTableSchema beside it is what let every later tenant land without this module changing: link, footnote, and comment definitions (ExaDev/markdown-codec#63, ExaDev/document-schema.js#22) ride the `definitions` field, and 4.1.0's three construct tables -- `layers`, `attachments`, `destinations` (ExaDev/document-schema.js#24) -- are three more root fields of this same generic type rather than three parallel shapes. This module defines the schemas and the pure resolution helpers; the frequency pass that mints entries from repeated property tuples is src/factor-styles.ts, which consumes them. // The paragraph half of a style entry: exactly the canonical ContentParagraph direct properties that a style may carry, and nothing else. Deliberately strict rather than plain: strictObject REJECTS a smuggled extra key instead of silently stripping it, which is what makes the ban list a schema-shape guarantee rather than a documented convention -- frames, sourcePath, and styleId are per-node facts (a position is a fact about a node, not a style; sourcePath and styleId identify the node and its producer-side style), so an entry carrying any of them fails validation outright instead of parsing to a value that quietly dropped them (ExaDev/document-schema.js#21's errata). export const StyleParagraphPropertiesSchema = z.strictObject({ @@ -36,7 +36,7 @@ export const StyleEntrySchema = z.strictObject({ }); export type StyleEntry = z.infer; -// The styles tenant of the definitions facility: string id -> resolved entry. Ids are minted by the producer's factoring pass (s1, s2, ... in deterministic order -- minting determinism is documents.js's law iii), and a tree node's `style` ref names a key in exactly this record. +// The styles tenant of the definitions facility: string id -> resolved entry. Ids are minted by the factoring pass of src/factor-styles.ts (s1, s2, ... in deterministic order -- minting determinism is the encoding pair's law iii, src/package.ts), and a tree node's `style` ref names a key in exactly this record. export const StylesTableSchema = z.record(z.string(), StyleEntrySchema); export type StylesTable = z.infer; diff --git a/src/factor-styles.test.ts b/src/factor-styles.test.ts new file mode 100644 index 0000000..0047bdb --- /dev/null +++ b/src/factor-styles.test.ts @@ -0,0 +1,405 @@ +import { describe, expect, it } from 'vitest'; +import { canonicalise } from './canonicalise'; +import type { ContentBlock, ContentDocument, ContentParagraph, ContentRun } from './content'; +import { assemblePackage, factorStyles, mint } from './factor-styles'; +import { flattenPackage } from './flatten'; +import { DocumentPackageSchema, type DocumentPackage } from './package'; +import type { SectionConstructGroupNode, SectionGroupNode, ShapeConstructGroupNode, ShapeGroupNode, SlideGroupNode } from './package-node'; + +// The minting rules as focused fixtures: the >=2 frequency threshold, the paragraph/run namespaces, the ban list (frames/sourcePath/styleId never enter a tuple), refs on wrappers only, the frozen-key rule for nested wrappers, chain-scoped stripping for nodes aliased under sibling wrappers, entry ordering and determinism, and idempotence. The bijection corpus (bijection.test.ts) re-runs the effective-equality and idempotence laws over real reader/conversion output; these tests pin the mechanism itself on minimal hand-built documents. + +const SECTION = { pageSize: { widthPt: 595, heightPt: 842 }, margins: { topPt: 72, rightPt: 72, bottomPt: 72, leftPt: 72 } }; + +function run(text: string, properties: Record = {}): ContentRun { + return { text, ...properties }; +} + +function paragraph(runs: readonly ContentRun[], properties: Record = {}): ContentParagraph { + return { kind: 'paragraph', runs: [...runs], ...properties }; +} + +function wordprocessingDoc(blocks: ContentBlock[], metadata: Record = {}): ContentDocument { + return { kind: 'wordprocessing', metadata, sections: [{ ...SECTION, blocks }] }; +} + +function canon(value: unknown): unknown { + return JSON.parse(JSON.stringify(canonicalise(value))); +} + +// Recovers every group wrapper carrying a style ref, in tree order, as [ref, node-kind] pairs -- the shape assertions below read the minted tree through it rather than by index-walking. +function refsOf(pkg: DocumentPackage): { readonly ref: string; readonly nodeKind: string }[] { + const found: { readonly ref: string; readonly nodeKind: string }[] = []; + function walk(value: unknown): void { + if (Array.isArray(value)) { + for (const child of value) walk(child); + return; + } + if (typeof value !== 'object' || value === null) return; + const record = value as Record; + if ('node' in record && 'children' in record && typeof record.style === 'string') { + const node = record.node; + const kind = typeof node === 'object' && node !== null && 'kind' in node && typeof node.kind === 'string' ? node.kind : 'shape/anchor'; + found.push({ ref: record.style, nodeKind: kind }); + } + for (const child of Object.values(record)) walk(child); + } + walk(pkg.children); + return found; +} + +function containsKeyAnywhere(value: unknown, key: string): boolean { + if (Array.isArray(value)) return value.some((child) => containsKeyAnywhere(child, key)); + if (typeof value !== 'object' || value === null) return false; + for (const [k, child] of Object.entries(value)) { + if (k === key) return true; + if (containsKeyAnywhere(child, key)) return true; + } + return false; +} + +describe('factorStyles minting', () => { + it('factors a paragraph tuple occurring twice onto a wrapper ref and strips it from both positions', () => { + const doc = wordprocessingDoc([ + paragraph([run('one')], { indentLeftPt: 20, alignment: 'left' }), + paragraph([run('two')], { indentLeftPt: 20, alignment: 'left' }), + paragraph([run('three')], { indentLeftPt: 20, alignment: 'right' }), + ]); + const minted = assemblePackage(doc); + // The section wrapper is the one scope whose extent holds both indentLeftPt-20-left positions (bare leaves at section root), and alignment+indent are carried by every extent paragraph, so one entry covers both keys for the two matching positions. The third paragraph keeps its differing alignment inline (present-wins), and shares the indent through the same entry only if its tuple matched -- it does not (alignment differs), so it stays fully inline. + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'section' }]); + const styles = minted.styles ?? {}; + expect(Object.keys(styles)).toEqual(['s1']); + expect(styles.s1).toEqual({ paragraph: { alignment: 'left', indentLeftPt: 20 } }); + expect(minted.kind).toBe('wordprocessing'); + if (minted.kind !== 'wordprocessing') throw new Error('expected wordprocessing'); + const strip1 = minted.children[0]?.children[0]; + const strip2 = minted.children[0]?.children[1]; + const keep3 = minted.children[0]?.children[2]; + expect(strip1).toMatchObject({ kind: 'paragraph', runs: [{ text: 'one' }] }); + expect(strip1).not.toHaveProperty('indentLeftPt'); + expect(strip1).not.toHaveProperty('alignment'); + expect(strip2).not.toHaveProperty('indentLeftPt'); + expect(keep3).toMatchObject({ alignment: 'right', indentLeftPt: 20 }); + expect(DocumentPackageSchema.safeParse(minted).success).toBe(true); + }); + + it('mints nothing for singletons -- a ref plus its entry is larger than the inline tuple', () => { + const doc = wordprocessingDoc([ + paragraph([run('only')], { indentLeftPt: 20 }), + paragraph([run('other')], { indentLeftPt: 40 }), + ]); + const minted = assemblePackage(doc); + expect(minted.styles).toBeUndefined(); + expect(refsOf(minted)).toEqual([]); + }); + + it('never factors the ban list: frames, sourcePath, and styleId stay per-node', () => { + const frames = [{ pageIndex: 0, xPt: 10, yPt: 20, widthPt: 30, heightPt: 5 }]; + const doc = wordprocessingDoc([ + paragraph([run('one', { bold: true })], { styleId: 'Heading1', sourcePath: 'word/document.xml#one', frames }), + paragraph([run('two', { bold: true })], { styleId: 'Heading1', sourcePath: 'word/document.xml#one', frames }), + ]); + const minted = assemblePackage(doc); + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'section' }]); + // All three ban-list keys are identical on both paragraphs and so occur twice, but each is a per-node fact: no entry carries any of them and no paragraph loses any of them. bold occurs twice AND is carried by both extent runs, so the run half legitimately mints. + expect(minted.styles?.s1).toEqual({ run: { bold: true } }); + expect(containsKeyAnywhere(minted.children, 'styleId')).toBe(true); + expect(containsKeyAnywhere(minted.styles, 'styleId')).toBe(false); + expect(containsKeyAnywhere(minted.children, 'sourcePath')).toBe(true); + expect(containsKeyAnywhere(minted.styles, 'sourcePath')).toBe(false); + expect(containsKeyAnywhere(minted.children, 'frames')).toBe(true); + expect(containsKeyAnywhere(minted.styles, 'frames')).toBe(false); + }); + + it('mints run tuples on the wrapper whose extent covers the runs, stripping them from the runs', () => { + const body = shapeBlocks(paragraph([run('a', { bold: true, sizePt: 12 })]), paragraph([run('b', { bold: true, sizePt: 12 })])); + const doc: ContentDocument = { kind: 'presentation', metadata: {}, slides: [{ size: { widthPt: 960, heightPt: 540 }, shapes: [body], notes: '' }] }; + const minted = assemblePackage(doc); + // Outermost-first: the slide wrapper's extent already covers both shape flows, so it (not the deeper shape group) carries the ref -- one entry styles every run in the slide, which is exactly the "slide body text" case the rule exists for. + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'slide' }]); + expect(minted.styles?.s1).toEqual({ run: { bold: true, sizePt: 12 } }); + const flat = flattenPackage(minted); + if (flat.kind !== 'presentation') throw new Error('expected presentation'); + const runs = flat.slides[0]!.shapes[0]!.blocks.flatMap((block) => (block.kind === 'paragraph' ? block.runs : [])); + expect(runs).toEqual([run('a', { bold: true, sizePt: 12 }), run('b', { bold: true, sizePt: 12 })]); + }); + + it('freezes an ancestor\'s minted key for nested wrappers -- a deeper different value never shadows the ref that restores it', () => { + const h1 = paragraph([run('Chapter')], { headingLevel: 1, indentLeftPt: 20 }); + const body1 = paragraph([run('one')], { indentLeftPt: 20 }); + const body2 = paragraph([run('two')], { indentLeftPt: 20 }); + const h2 = paragraph([run('Part')], { headingLevel: 2, indentLeftPt: 40 }); + const body3 = paragraph([run('three')], { indentLeftPt: 40 }); + const body4 = paragraph([run('four')], { indentLeftPt: 40 }); + const doc = wordprocessingDoc([h1, body1, body2, h2, body3, body4]); + const minted = assemblePackage(doc); + // The section mints {indentLeftPt: 20} (three positions: the H1 anchor and its two body leaves -- the H2 branch carries a different value and stays inline). indentLeftPt is then frozen for every wrapper below, so the H2 group -- whose extent shares {indentLeftPt: 40} three times -- mints nothing: re-minting the key with 40 would silently rewrite the value the section's ref restores for the stripped 20-positions in nothing, but would shadow it for any nested stripped position, and freezing is the rule that keeps the two namespaces apart. + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'section' }]); + expect(minted.styles?.s1).toEqual({ paragraph: { indentLeftPt: 20 } }); + if (minted.kind !== 'wordprocessing') throw new Error('expected wordprocessing'); + // The H2 nests INSIDE the still-open H1 group (decompose's stack), so find it by text anywhere in the tree. + const h2Group = findGroupByText(minted, 'Part'); + expect(h2Group).toMatchObject({ node: { kind: 'paragraph', headingLevel: 2, indentLeftPt: 40 } }); + expect(h2Group).not.toHaveProperty('style'); + // Resolution restores the stripped positions exactly and leaves the H2 branch alone: gap-fill on the section's chain returns indentLeftPt 20 to the stripped three, and the inline 40s win where they sit. + const flat = flattenPackage(minted); + if (flat.kind !== 'wordprocessing') throw new Error('expected wordprocessing'); + const indents = flat.sections[0]!.blocks.map((block) => (block.kind === 'paragraph' ? block.indentLeftPt : undefined)); + expect(indents).toEqual([20, 20, 20, 40, 40, 40]); + }); + + it('orders entries by descending frequency and mints deterministically', () => { + const doc: ContentDocument = { + kind: 'wordprocessing', + metadata: {}, + sections: [ + { ...SECTION, blocks: [paragraph([run('a')], { alignment: 'center' }), paragraph([run('b')], { alignment: 'center' })] }, + { ...SECTION, blocks: [paragraph([run('c')], { lineSpacing: 1.5 }), paragraph([run('d')], { lineSpacing: 1.5 }), paragraph([run('e')], { lineSpacing: 1.5 })] }, + ], + }; + const minted = assemblePackage(doc); + // lineSpacing occurs on three positions (section two), alignment on two (section one): the more frequent entry takes s1 regardless of document order, and the same input mints the identical tree twice. + expect(minted.styles && Object.keys(minted.styles)).toEqual(['s1', 's2']); + expect(minted.styles?.s1).toEqual({ paragraph: { lineSpacing: 1.5 } }); + expect(minted.styles?.s2).toEqual({ paragraph: { alignment: 'center' } }); + expect(canon(assemblePackage(doc))).toEqual(canon(minted)); + }); + + it('is idempotent: factoring a second time mints the identical table and tree', () => { + const doc = wordprocessingDoc([ + paragraph([run('one', { bold: true })], { indentLeftPt: 20, alignment: 'left' }), + paragraph([run('two', { bold: true })], { indentLeftPt: 20, alignment: 'left' }), + ]); + const once = assemblePackage(doc); + const twice = factorStyles(once); + expect(canon(twice)).toEqual(canon(once)); + expect(twice.styles).toEqual(once.styles); + }); + + it('carries a package\'s definitions table through re-factoring untouched', () => { + const doc = wordprocessingDoc([ + paragraph([run('one')], { indentLeftPt: 20 }), + paragraph([run('two')], { indentLeftPt: 20 }), + ]); + const minted = assemblePackage(doc); + // definitions is package-root caller data the flat ContentDocument cannot spell, so re-factoring must hand it back verbatim -- dropping it would silently lose the table on every factorStyles round trip. Minting still runs: the indent tuple mints s1 alongside the carried definitions. + const withDefinitions: DocumentPackage = { ...minted, definitions: { tenantNote: { kind: 'tenant-note' } } }; + const refactored = factorStyles(withDefinitions); + expect(refactored.definitions).toEqual({ tenantNote: { kind: 'tenant-note' } }); + expect(refactored.styles?.s1).toEqual({ paragraph: { indentLeftPt: 20 } }); + expect(DocumentPackageSchema.safeParse(refactored).success).toBe(true); + }); + + it('keeps flat output free of refs and effective-equal to the unfactored form, combining halves on one wrapper', () => { + const doc = wordprocessingDoc([ + paragraph([run('one', { bold: true, sizePt: 14 })], { indentLeftPt: 20, alignment: 'left' }), + paragraph([run('two', { bold: true, sizePt: 14 })], { indentLeftPt: 20, alignment: 'left' }), + paragraph([run('three', { sizePt: 14 })], { indentLeftPt: 20, alignment: 'left' }), + ]); + const minted = assemblePackage(doc); + // Every extent paragraph carries alignment+indentLeftPt and every run carries sizePt, so the section's one entry combines the paragraph half (three stripped positions) with the run half (three stripped runs); bold occurs on only two of the three runs, so it is not common and stays inline everywhere. + expect(minted.styles?.s1).toEqual({ paragraph: { alignment: 'left', indentLeftPt: 20 }, run: { sizePt: 14 } }); + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'section' }]); + const flat = flattenPackage(minted); + expect(containsKeyAnywhere(flat, 'style')).toBe(false); + expect(canon(flat)).toEqual(canon(doc)); + // Law (ii) in its direct form: the factored tree's materialised flat form IS the unfactored content, key for key (a wrapper carrying a ref the styles table does not back is malformed, so the comparison runs against the original document rather than a ref-stripped tree). + }); + + it('strips by copying, never mutating the input content', () => { + const p1 = paragraph([run('one')], { indentLeftPt: 20 }); + const p2 = paragraph([run('two')], { indentLeftPt: 20 }); + const doc = wordprocessingDoc([p1, p2]); + const snapshot = structuredClone(doc); + assemblePackage(doc); + expect(doc).toEqual(snapshot); + expect(p1.indentLeftPt).toBe(20); + expect(p2.indentLeftPt).toBe(20); + }); + + it('strips an aliased node at every position whose own chain minted -- identical tuple, both sibling wrappers mint', () => { + const shared = paragraph([run('a')], { alignment: 'center', indentLeftPt: 20 }); + const doc: ContentDocument = { + kind: 'wordprocessing', + metadata: {}, + sections: [ + { ...SECTION, blocks: [shared, paragraph([run('b')], { alignment: 'center', indentLeftPt: 20 })] }, + { ...SECTION, blocks: [shared, paragraph([run('c')], { alignment: 'center', indentLeftPt: 20 })] }, + ], + }; + const minted = assemblePackage(doc); + // Both sections' extents hold two matching positions (the shared node plus a sibling leaf), so each mints the identical entry content and shares ONE table entry through the canonical key -- two refs, one row. Global factored bookkeeping would mark the shared node done at the first section, leaving the second position's chain ref-less while a node-keyed strip still took its properties; branch-scoped, both positions resolve their own ref back. + expect(refsOf(minted)).toEqual([ + { ref: 's1', nodeKind: 'section' }, + { ref: 's1', nodeKind: 'section' }, + ]); + expect(minted.styles?.s1).toEqual({ paragraph: { alignment: 'center', indentLeftPt: 20 } }); + const flat = flattenPackage(minted); + expect(canon(flat)).toEqual(canon(doc)); + }); + + it('strips an aliased node by its own branch\'s key set when sibling wrappers mint divergent entries', () => { + const shared = paragraph([run('a')], { alignment: 'center', indentLeftPt: 20 }); + const doc: ContentDocument = { + kind: 'wordprocessing', + metadata: {}, + sections: [ + { ...SECTION, blocks: [shared, paragraph([run('b')], { alignment: 'center' })] }, + { ...SECTION, blocks: [shared, paragraph([run('c')], { alignment: 'center', indentLeftPt: 20 })] }, + ], + }; + const minted = assemblePackage(doc); + // Section one's extent shares only alignment (its second paragraph carries no indent), so it mints the alignment-only entry and strips just that key off the shared node's first position; section two's extent shares both keys and mints the wider entry. Whichever section plans second would overwrite a node-keyed global strip's key set, leaving the first position stripped of indentLeftPt with a ref that restores only alignment; per-wrapper strips keep each position's strip the set its own ref restores. + expect(minted.styles?.s1).toEqual({ paragraph: { alignment: 'center' } }); + expect(minted.styles?.s2).toEqual({ paragraph: { alignment: 'center', indentLeftPt: 20 } }); + if (minted.kind !== 'wordprocessing') throw new Error('expected wordprocessing'); + const flat = flattenPackage(minted); + if (flat.kind !== 'wordprocessing') throw new Error('expected wordprocessing'); + expect(flat.sections[0]!.blocks[0]).toMatchObject({ alignment: 'center', indentLeftPt: 20 }); + expect(flat.sections[1]!.blocks[0]).toMatchObject({ alignment: 'center', indentLeftPt: 20 }); + expect(canon(flat)).toEqual(canon(doc)); + }); + + it('leaves an aliased node fully inline at a position whose own chain minted nothing', () => { + const shared = paragraph([run('a')], { indentLeftPt: 20 }); + const doc: ContentDocument = { + kind: 'wordprocessing', + metadata: {}, + sections: [ + { ...SECTION, blocks: [shared, paragraph([run('b')], { indentLeftPt: 20 })] }, + { ...SECTION, blocks: [shared] }, + ], + }; + const minted = assemblePackage(doc); + // Section one mints (two matching positions); section two's extent is the aliased node alone -- a singleton, below the threshold -- so its chain carries no ref and the node must keep every property inline there: a node-keyed global strip would strip it at BOTH positions with no ref to restore the second. + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'section' }]); + expect(minted.styles?.s1).toEqual({ paragraph: { indentLeftPt: 20 } }); + if (minted.kind !== 'wordprocessing') throw new Error('expected wordprocessing'); + expect(minted.children[1]?.children[0]).toMatchObject({ kind: 'paragraph', indentLeftPt: 20 }); + const flat = flattenPackage(minted); + expect(canon(flat)).toEqual(canon(doc)); + }); + + it('mints nothing for a formula package (one leaf, no wrappers)', () => { + const doc: ContentDocument = { kind: 'formula', metadata: {}, formula: { mathml: [] } }; + const minted = assemblePackage(doc); + expect(minted.styles).toBeUndefined(); + expect(DocumentPackageSchema.safeParse(minted).success).toBe(true); + }); + + it('factors a paragraph tuple nested inside a construct group\'s children onto the construct group\'s own ref (document-schema.js 4.1.0)', () => { + // mint() run directly on a hand-built tree, rather than through assemblePackage, so extentOf/flowExtent's construct-group recognition is asserted on exactly the tree shape stated here -- independent of which flat marker placement decompose would have promoted to it. + const outside = paragraph([run('outside')], { alignment: 'right' }); + const insideA = paragraph([run('a')], { indentLeftPt: 20 }); + const insideB = paragraph([run('b')], { indentLeftPt: 20 }); + const constructGroup: SectionConstructGroupNode = { node: { kind: 'contentControl', controlType: 'richText' }, children: [insideA, insideB] }; + const sectionGroup: SectionGroupNode = { node: { kind: 'section', ...SECTION }, children: [outside, constructGroup] }; + const pkg: DocumentPackage = { kind: 'wordprocessing', metadata: {}, children: [sectionGroup] }; + const minted = mint(pkg); + // Rule 1 (every extent paragraph must carry a minted key) means the section's own three-paragraph extent shares no key across all three -- outside lacks indentLeftPt, insideA/insideB lack alignment -- so the section wrapper itself mints nothing. Only once the walk descends INTO the construct group's own two-paragraph extent (proof extentOf/flowExtent recurse into a construct group's children rather than stopping at or skipping it) does indentLeftPt become common there and mint. + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'contentControl' }]); + expect(minted.styles?.s1).toEqual({ paragraph: { indentLeftPt: 20 } }); + if (minted.kind !== 'wordprocessing') throw new Error('expected wordprocessing'); + const mintedSection = minted.children[0]; + if (mintedSection === undefined) throw new Error('expected the section group'); + const mintedConstruct = mintedSection.children[1]; + if (mintedConstruct === undefined || !('node' in mintedConstruct) || !('children' in mintedConstruct)) { + throw new Error('expected the construct group to survive minting'); + } + expect(mintedConstruct.style).toBe('s1'); + expect(mintedConstruct.children[0]).not.toHaveProperty('indentLeftPt'); + expect(mintedConstruct.children[1]).not.toHaveProperty('indentLeftPt'); + }); + + it('factors a paragraph tuple nested inside a shape-flow construct group onto the construct group\'s own ref (document-schema.js 4.1.0)', () => { + // The section-flow test above exercises rebuildSectionConstructGroup and the isConstructGroup arm in rebuildSectionChild; this mirrors it through the shape/list-flow vocabulary instead -- a ShapeConstructGroupNode sat inside a ShapeGroupNode's own children, nested under a SlideGroupNode -- so rebuildShapeConstructGroup and the isConstructGroup dispatch arm in rebuildListChild get their own coverage rather than riding untested on the section-flow rebuilder's coattails. + const outside = paragraph([run('outside')], { alignment: 'right' }); + const insideA = paragraph([run('a')], { indentLeftPt: 20 }); + const insideB = paragraph([run('b')], { indentLeftPt: 20 }); + const constructGroup: ShapeConstructGroupNode = { node: { kind: 'contentControl', controlType: 'richText' }, children: [insideA, insideB] }; + const shapeGroup: ShapeGroupNode = { node: { frame: { xPt: 0, yPt: 0, widthPt: 400, heightPt: 300 }, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0 }, children: [outside, constructGroup] }; + const slideGroup: SlideGroupNode = { node: { kind: 'slide', size: { widthPt: 960, heightPt: 540 }, notes: '' }, children: [shapeGroup] }; + const pkg: DocumentPackage = { kind: 'presentation', metadata: {}, children: [slideGroup] }; + const minted = mint(pkg); + // Neither the slide's nor the shape's own extent shares a key across all three paragraphs (outside lacks indentLeftPt, insideA/insideB lack alignment), so both mint nothing; only descending into the construct group's own two-paragraph extent makes indentLeftPt common there and mints. + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'contentControl' }]); + expect(minted.styles?.s1).toEqual({ paragraph: { indentLeftPt: 20 } }); + if (minted.kind !== 'presentation') throw new Error('expected presentation'); + const mintedSlide = minted.children[0]; + if (mintedSlide === undefined) throw new Error('expected the slide group'); + const mintedShape = mintedSlide.children[0]; + if (mintedShape === undefined) throw new Error('expected the shape group'); + const mintedConstruct = mintedShape.children[1]; + if (mintedConstruct === undefined || !('node' in mintedConstruct) || !('children' in mintedConstruct)) { + throw new Error('expected the construct group to survive minting'); + } + expect(mintedConstruct.style).toBe('s1'); + expect(mintedConstruct.children[0]).not.toHaveProperty('indentLeftPt'); + expect(mintedConstruct.children[1]).not.toHaveProperty('indentLeftPt'); + }); + + it('mints through a construct promoted from flat marker blocks, and the tree it produces flattens back', () => { + // The two tests above hand mint() a tree directly; this one comes the whole way round -- flat content carrying a constructStart/constructEnd pair, through assemblePackage (decompose then mint), and back through flattenPackage. It is the end-to-end proof that minting and the promotion compose: a construct group manufactured by decompose is an ordinary mint wrapper, and a minted tree containing one is still flattenable. + const doc = wordprocessingDoc([ + paragraph([run('outside')], { alignment: 'right' }), + { kind: 'constructStart', descriptor: { kind: 'contentControl', controlType: 'richText' } }, + paragraph([run('a')], { indentLeftPt: 20 }), + paragraph([run('b')], { indentLeftPt: 20 }), + { kind: 'constructEnd' }, + ]); + const minted = assemblePackage(doc); + expect(DocumentPackageSchema.safeParse(minted).success).toBe(true); + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'contentControl' }]); + expect(minted.styles?.s1).toEqual({ paragraph: { indentLeftPt: 20 } }); + expect(canon(flattenPackage(minted))).toEqual(canon(doc)); + }); + + it('resolves an ancestor heading\'s ref onto a paragraph nested inside a construct -- a construct extends the style chain, never resets it', () => { + // The chain axis, stated on its own because it is the one place a construct differs from the section/slide/sheet/draw-page roots: those start a brand new empty chain, a construct extends the incoming one. A construct is a semantic wrapper sitting inside ambient content, so `inside` must come back carrying the heading group's factored alignment exactly as `a` (its sibling outside the construct) does -- if flatten reset the chain at the construct boundary, `inside` would flatten back stripped and law (i) would fail on it. + const doc = wordprocessingDoc([ + // `intro` carries no alignment, so the SECTION wrapper's own four-paragraph extent shares no mintable key and mints nothing -- which is what puts the ref on the heading group specifically rather than on an ancestor that happens to cover everything. + paragraph([run('intro')]), + paragraph([run('Chapter')], { headingLevel: 1, alignment: 'center' }), + paragraph([run('a')], { alignment: 'center' }), + { kind: 'constructStart', descriptor: { kind: 'field', instruction: 'PAGE' } }, + paragraph([run('inside')], { alignment: 'center' }), + { kind: 'constructEnd' }, + ]); + const minted = assemblePackage(doc); + expect(refsOf(minted)).toEqual([{ ref: 's1', nodeKind: 'paragraph' }]); + expect(minted.styles?.s1).toEqual({ paragraph: { alignment: 'center' } }); + const heading = findGroupByText(minted, 'Chapter'); + if (heading === undefined) throw new Error('expected the heading group'); + expect(heading.style).toBe('s1'); + // The nested paragraph was stripped by the heading's entry (no ref of its own on the construct group), and flatten restores it from that ancestor entry. + expect(containsKeyAnywhere(minted.children, 'alignment')).toBe(false); + expect(canon(flattenPackage(minted))).toEqual(canon(doc)); + }); +}); + +// Finds the first group wrapper anywhere in the tree whose anchor paragraph's first run text matches -- the frozen-key test's H2 group sits nested inside the H1 group, not at any fixed depth. +function findGroupByText(pkg: DocumentPackage, text: string): { node: unknown; style?: string } | undefined { + let found: { node: unknown; style?: string } | undefined; + function walk(value: unknown): void { + if (found !== undefined) return; + if (Array.isArray(value)) { + for (const child of value) walk(child); + return; + } + if (typeof value !== 'object' || value === null) return; + if ('node' in value && 'children' in value) { + const node = value.node as ContentParagraph; + if (node.kind === 'paragraph' && node.runs[0]?.text === text) { + found = value; + return; + } + } + for (const child of Object.values(value)) walk(child); + } + walk(pkg.children); + return found; +} + +function shapeBlocks(...blocks: ContentBlock[]): { frame: { xPt: number; yPt: number; widthPt: number; heightPt: number }; insetLeftPt: number; insetTopPt: number; insetRightPt: number; insetBottomPt: number; blocks: ContentBlock[] } { + return { frame: { xPt: 0, yPt: 0, widthPt: 400, heightPt: 300 }, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0, blocks: [...blocks] }; +} diff --git a/src/factor-styles.ts b/src/factor-styles.ts new file mode 100644 index 0000000..58e58d7 --- /dev/null +++ b/src/factor-styles.ts @@ -0,0 +1,549 @@ +import { canonicalKey } from './canonicalise'; +import type { ContentDocument, ContentParagraph, ContentRun } from './content'; +import { decomposeDrawPage, decomposeSection, decomposeSheet, decomposeSlide } from './decompose'; +import type { StyleEntry, StyleParagraphProperties, StyleRunProperties, StylesTable } from './definitions'; +import { flattenPackage } from './flatten'; +import type { PageSize } from './geometry'; +import type { DocumentPackage } from './package'; +import type { + DrawPageGroupNode, + HeadingGroupNode, + HeadingParagraph, + ListChild, + ListGroupNode, + SectionChild, + SectionConstructGroupNode, + SectionGroupNode, + ShapeConstructGroupNode, + ShapeGroupNode, + SlideGroupNode, +} from './package-node'; + +// The styles minting half of the package boundary (#21's factoring pass): assemblePackage = decompose then factorStyles, and the two are separate passes so minting idempotence stays independently testable. Minting walks the freshly decomposed tree, finds property tuples that repeat, and hoists each onto a wrapper ref + styles-table entry -- pure compression over the one tree a conversion just built, never a second authority for content. +// +// The mintable property sets are exactly the schema's own style halves (StyleParagraphProperties / StyleRunProperties) minus `list`: frames, sourcePath, and styleId are per-node facts the schema's strict entry objects already refuse outright (the ban list), and `list` is additionally excluded here because it is a grouping signal -- decompose's own stack semantics and the anchor schema (ListParagraphSchema requires list on a list group's node) read it off the node object, so a membership factored into the table would move structure the tree itself must keep stating. headingLevel is not a StyleParagraphProperties field at all, so heading anchors are equally safe by construction. +// +// Exactness -- the promotion's law (ii), that a factored and an unfactored serialisation of one document resolve to the same effective properties -- is guaranteed by two rules, both consequences of the resolution helpers being gap-fill-never-overwrite (applyParagraphStyleProperties / applyRunStyleProperties): +// +// 1. A minted tuple's keys must be carried by EVERY paragraph (for the paragraph half) and EVERY run of every extent paragraph (for the run half) of the wrapper's whole subtree extent -- the block-flow paragraphs resolution overlays the ref onto (group anchors and bare paragraph leaves; a table leaf's cell paragraphs and an embedded document's own content are outside the walk, exactly as they are outside resolution). A key a node already carries wins over the entry whatever its value, so unstripped extent nodes are untouched; a key a node lacks would be FILLED, so the every-node-carries-it condition is what makes the fill a no-op for everyone except the stripped positions it restores. +// 2. A key already minted by an ancestor wrapper's entry is frozen for every wrapper below it: resolution overlays the chain outermost-first with the nearest entry winning, so a nested entry re-minting an ancestor's key with a different value would silently rewrite the value the ancestor's ref restores for its own stripped positions. Freezing keeps already-factored keys out of every deeper candidate. +// +// Minting order and identity per the plan's locked rules: wrappers are visited outermost-first (pre-order, document order); a wrapper mints at most one entry (optionally carrying both halves) when some paragraph tuple and/or run tuple occurs on two or more positions in its extent that no wrapper on its root-to-leaf chain has already factored (the factored bookkeeping is branch-scoped, never global -- see Branch); the best tuple at a wrapper is the most frequent, tie-broken by first occurrence in document order -- a total rule, since a position joins exactly one tuple group, so two distinct tuples can never share a first occurrence. Identical entries minted at several wrappers share one table entry; ids are s1, s2, ... in (descending total frequency, first wrapper visit) order -- itself total, because one wrapper mints at most one entry, so distinct entries always have distinct first visits -- and the pass is deterministic. +// +// The whole pass is a pure function of the MATERIALISED content: factorStyles flattens its input first (resolving any refs it already carries), so factoring a second time computes the identical plan over the identical values and mints the identical table -- law (iii), minting idempotence, holds by construction. Stripping copies only the paragraphs and runs whose keys moved to a table entry (never the caller's nodes in place -- decompose embedded those, and the layout pass's frames ride on them); every other node in the minted tree is the same object the flat content owns. Strips apply per chain, not per node object: the same paragraph or run object may legally sit at positions under two sibling wrappers (a caller-built document that pushes one node into two sections is a two-position tree once serialised), and each position is stripped only by a wrapper whose ref that position's own chain resolves -- flatten resolves per position, so minting strips per position too (see WrapperStrips). + +// The paragraph half's mintable keys, in the schema's own declaration order (StyleParagraphProperties minus `list`; see the module doc for why list never factors). +const PARAGRAPH_STYLE_KEYS = ['alignment', 'spacingBeforePt', 'spacingAfterPt', 'lineSpacing', 'indentLeftPt', 'indentFirstLinePt'] as const; + +// The run half's mintable keys -- StyleRunProperties' full field set, schema declaration order. +const RUN_STYLE_KEYS = ['bold', 'italic', 'underline', 'strike', 'fontFamily', 'sizePt', 'color'] as const; + +type ParagraphKey = (typeof PARAGRAPH_STYLE_KEYS)[number]; +type RunKey = (typeof RUN_STYLE_KEYS)[number]; + +// The wrapper kinds that can carry a ref and hold block-flow paragraphs. SheetGroupNode fits the wrapper shape but its children are images and embedded objects -- no paragraphs, an always-empty extent -- so it never mints and is excluded from the walk's type. SectionConstructGroupNode/ShapeConstructGroupNode (4.1.0) join the set on equal footing: each carries the same `{ node, style?, children }` shape as every other wrapper here, and neither needs a dedicated dispatch arm below -- their node is never the 'paragraph'/'slide'/'drawPage' discriminant any other wrapper matches on, so both fall straight through to extentOf/childWrappers' shared "no anchor of its own" default, the same default a plain SectionGroupNode already relies on. +type MintWrapper = SectionGroupNode | SlideGroupNode | DrawPageGroupNode | ShapeGroupNode | HeadingGroupNode | ListGroupNode | SectionConstructGroupNode | ShapeConstructGroupNode; + +// One child position of any block flow: the union of the section, list, and shape flows' child vocabularies. ListChild and ShapeChild are the identical type (ListGroupNode | ShapeConstructGroupNode | PackageBlockLeaf) since 4.1.0, no longer a sub-range of SectionChild (which carries SectionConstructGroupNode instead) -- so the extent walk needs both halves explicitly to serve all three flows with one function. +type FlowChild = SectionChild | ListChild; + +// Per-kind narrowers over MintWrapper. These exist because TypeScript does not narrow a union from a comparison against a NESTED discriminant (`wrapper.node.kind === 'section'` narrows wrapper.node at best, never `wrapper`) -- the identical reason src/package-node.ts writes per-kind predicates, and an explicit guard is what narrows the wrapper itself. A shape group is the no-kind arm (ContentShape carries no kind field); heading and list groups share the 'paragraph' node discriminant and stay one arm because the minting walk treats every anchor alike. SectionGroupNode, SectionConstructGroupNode, and ShapeConstructGroupNode get no guard of their own: none of their node kinds ('section', or one of the six construct kinds) matches any check below, so all three fall through to the shared "no anchor" default at the foot of extentOf/childWrappers. +function isShapeGroupWrapper(wrapper: MintWrapper): wrapper is ShapeGroupNode { + return !('kind' in wrapper.node); +} + +function isAnchorGroupWrapper(wrapper: MintWrapper): wrapper is HeadingGroupNode | ListGroupNode { + return 'kind' in wrapper.node && wrapper.node.kind === 'paragraph'; +} + +function isSlideGroupWrapper(wrapper: MintWrapper): wrapper is SlideGroupNode { + return 'kind' in wrapper.node && wrapper.node.kind === 'slide'; +} + +function isDrawPageGroupWrapper(wrapper: MintWrapper): wrapper is DrawPageGroupNode { + return 'kind' in wrapper.node && wrapper.node.kind === 'drawPage'; +} + +// Heading vs list within the anchor arm, discriminated the way decompose constructs them (a paragraph carrying both signals becomes a heading anchor -- headings win). +function isHeadingGroup(group: HeadingGroupNode | ListGroupNode): group is HeadingGroupNode { + return group.node.headingLevel !== undefined; +} + +// A section- or shape-flow child position whose own node is a construct descriptor rather than a paragraph -- the same structural narrow flatten.ts uses (node.kind is never 'paragraph' for a ConstructDescriptor), needed here in the rebuild walk below to dispatch a construct-group position to its own rebuilder rather than treating it as a heading/list anchor. +function isConstructGroup(child: SectionChild | ListChild): child is SectionConstructGroupNode | ShapeConstructGroupNode { + return 'node' in child && 'children' in child && child.node.kind !== 'paragraph'; +} + +// Assembles the tree-form DocumentPackage every construction site reports: decompose the flat content into its children, splice the envelope fields (kind, metadata, symbolTable) out of the content onto the root, carry `pages` when a layout pass produced rendered page sizes, then mint the styles table over the result. `pages` is spread-copied because the schema's array field is mutable while callers hand us readonly views of the layout engine's own array. +export function assemblePackage(content: ContentDocument, pages?: readonly PageSize[]): DocumentPackage { + const envelope = { + metadata: content.metadata, + ...(content.symbolTable !== undefined ? { symbolTable: content.symbolTable } : {}), + ...(pages !== undefined ? { pages: [...pages] } : {}), + }; + switch (content.kind) { + case 'wordprocessing': + return mint({ kind: 'wordprocessing', ...envelope, children: content.sections.map(decomposeSection) }); + case 'presentation': + return mint({ kind: 'presentation', ...envelope, children: content.slides.map(decomposeSlide) }); + case 'spreadsheet': + return mint({ kind: 'spreadsheet', ...envelope, children: content.sheets.map(decomposeSheet) }); + case 'drawing': + return mint({ kind: 'drawing', ...envelope, children: content.pages.map(decomposeDrawPage) }); + case 'formula': + // A formula package's single child is a leaf -- no wrappers, no paragraphs -- so minting is necessarily a no-op; it routes through mint anyway so the return shape stays one code path. + return mint({ kind: 'formula', ...envelope, children: [content.formula] }); + } +} + +// Re-factors an already-assembled package. The input is flattened first (materialising its refs), so this both re-mints a minted package to the identical table (law iii) and factors any hand-built or round-tripped tree a caller hands in. `pages` and `definitions` ride the input through: neither has a spelling on the flat ContentDocument, so the flatten step cannot carry them and the reassembled tree would otherwise drop them silently. Minting never reads `definitions` -- the table is per-document caller data, not style content the pass has any business rewriting. +export function factorStyles(pkg: DocumentPackage): DocumentPackage { + const reassembled = assemblePackage(flattenPackage(pkg), pkg.pages); + if (pkg.definitions === undefined) return reassembled; + return { ...reassembled, definitions: pkg.definitions }; +} + +// --- The plan: extents, candidates, selection ----------------------------------------------------------- + +// The paragraphs a wrapper's ref would overlay onto: the wrapper's own anchor (heading and list groups) plus, recursively, nested group anchors and bare paragraph leaves inside the block flow. This is exactly flatten.ts's resolution extent -- the same walk boundary, the same exclusions -- because exactness is proven against exactly the nodes resolution touches. +function extentOf(wrapper: MintWrapper): ContentParagraph[] { + if (isShapeGroupWrapper(wrapper)) { + // A shape group: no anchor of its own, its list-flow children carry everything. + return flowExtent(wrapper.children); + } + if (isAnchorGroupWrapper(wrapper)) { + return [wrapper.node, ...flowExtent(wrapper.children)]; + } + if (isSlideGroupWrapper(wrapper)) { + return wrapper.children.flatMap(extentOf); + } + if (isDrawPageGroupWrapper(wrapper)) { + const paragraphs: ContentParagraph[] = []; + for (const child of wrapper.children) { + if ('node' in child) paragraphs.push(...extentOf(child)); + } + return paragraphs; + } + // A section group or a construct group (section or shape variant): no anchor of its own, its whole flow is the extent -- a construct descriptor is never a paragraph, so it never contributes a paragraph itself, exactly like a plain section group's descriptor. + return flowExtent(wrapper.children); +} + +// The block-flow extent of one section/heading/list/shape/construct child list: nested heading, list, and construct groups contribute their anchors (construct groups contribute none of their own) and recurse, bare paragraph leaves contribute themselves, every other leaf (tables, images, page breaks, embedded objects) contributes nothing. +function flowExtent(children: readonly FlowChild[]): ContentParagraph[] { + const paragraphs: ContentParagraph[] = []; + for (const child of children) { + if ('node' in child && 'children' in child) { + paragraphs.push(...extentOf(child)); + } else if (child.kind === 'paragraph') { + paragraphs.push(child); + } + } + return paragraphs; +} + +// A tuple of just the keys in `keys` that the paragraph actually carries -- the candidate identity for the paragraph namespace. Absent keys are omitted, not set to undefined, so canonicalKey treats both spellings of absence identically. +function paragraphTuple(paragraph: ContentParagraph, keys: readonly ParagraphKey[]): StyleParagraphProperties { + const tuple: Record = {}; + for (const key of keys) { + if (paragraph[key] !== undefined) tuple[key] = paragraph[key]; + } + return tuple; +} + +function runTuple(run: ContentRun, keys: readonly RunKey[]): StyleRunProperties { + const tuple: Record = {}; + for (const key of keys) { + if (run[key] !== undefined) tuple[key] = run[key]; + } + return tuple; +} + +// The keys (from the mintable set, minus the ancestor-frozen ones) that EVERY paragraph in the extent carries -- rule 1's every-node-carries-it condition, computed before grouping so candidates can only form over keys the whole extent shares. +function commonParagraphKeys(extent: readonly ContentParagraph[], frozen: ReadonlySet): readonly ParagraphKey[] { + return PARAGRAPH_STYLE_KEYS.filter((key) => !frozen.has(key) && extent.every((paragraph) => paragraph[key] !== undefined)); +} + +function commonRunKeys(extent: readonly ContentParagraph[], frozen: ReadonlySet): readonly RunKey[] { + const runs = extent.flatMap((paragraph) => paragraph.runs); + if (runs.length === 0) return []; + return RUN_STYLE_KEYS.filter((key) => !frozen.has(key) && runs.every((run) => run[key] !== undefined)); +} + +interface ParagraphCandidate { + readonly tuple: StyleParagraphProperties; + readonly keys: readonly ParagraphKey[]; + readonly positions: ContentParagraph[]; +} + +interface RunCandidate { + readonly tuple: StyleRunProperties; + readonly keys: readonly RunKey[]; + readonly positions: ContentRun[]; +} + +// Groups the not-yet-factored positions by their restricted tuple and returns the best candidate -- the one occurring on two or more positions, most frequent first, then earliest first position -- or undefined when no tuple reaches the threshold. The frequency threshold is the plan's own economy rule: a singleton ref plus its table entry is larger than the inline tuple it would replace. +function bestParagraphCandidate(extent: readonly ContentParagraph[], keys: readonly ParagraphKey[], factored: ReadonlySet): ParagraphCandidate | undefined { + const groups = new Map(); + for (const paragraph of extent) { + if (factored.has(paragraph)) continue; + const tuple = paragraphTuple(paragraph, keys); + if (Object.keys(tuple).length === 0) continue; + const key = canonicalKey(tuple); + const existing = groups.get(key); + if (existing === undefined) groups.set(key, { tuple, keys, positions: [paragraph] }); + else existing.positions.push(paragraph); + } + return bestGroup(groups); +} + +function bestRunCandidate(extent: readonly ContentParagraph[], keys: readonly RunKey[], factored: ReadonlySet): RunCandidate | undefined { + const groups = new Map(); + for (const paragraph of extent) { + for (const run of paragraph.runs) { + if (factored.has(run)) continue; + const tuple = runTuple(run, keys); + if (Object.keys(tuple).length === 0) continue; + const key = canonicalKey(tuple); + const existing = groups.get(key); + if (existing === undefined) groups.set(key, { tuple, keys, positions: [run] }); + else existing.positions.push(run); + } + } + return bestGroup(groups); +} + +// The shared (most frequent, earliest) choice over a tuple -> positions grouping. Map iteration order is first-occurrence insertion order, so iterating in insertion order and taking the first strictly-more-frequent group resolves frequency first and falls back to document order for ties. The rule is total at two arms: a position joins exactly one tuple group, so two distinct tuples can never share a first occurrence. +function bestGroup(groups: Map): T | undefined { + let best: T | undefined; + for (const group of groups.values()) { + if (group.positions.length < 2) continue; + if (best === undefined || group.positions.length > best.positions.length) best = group; + } + return best; +} + +// --- The plan walk and the apply phase ------------------------------------------------------------------- + +// The strips one minting wrapper's selection recorded: the keys its entry takes off each paragraph and each run it factored. Held per wrapper and consulted per chain during the rebuild, so the same node object aliased at positions under two sibling wrappers is stripped by each branch's own minter (or by neither) -- flatten resolves refs per position, so minting must strip per position too. A single node-keyed strip map cannot express that: it would apply one branch's strip at every position, and a position whose own chain minted nothing (or minted a different key set) would lose the stripped properties with no ref to restore them. +interface WrapperStrips { + readonly paragraphs: Map; + readonly runs: Map; +} + +// The mutable working state of one minting run: the refs the rebuild stamps and the per-wrapper strips it applies. The plan's chain-scoped inputs (frozen keys, factored positions) live in Branch instead, threaded through plan() copy-on-descend so they never survive into the rebuild. +interface MintState { + // wrapper object identity -> the entry id its ref names (consumed by the rebuild walk). + readonly wrapperRefs: Map; + // minted wrapper object identity -> the strips its own entry recorded (the rebuild walk threads the records of its chain, innermost first). + readonly wrapperStrips: Map; +} + +// The chain-scoped planning context one plan() call sees: the keys an ancestor's entry froze for everything below it, and the positions an ancestor's entry already factored (restoring those is that ancestor ref's job). Copy-on-descend, so a factored position is invisible to every SIBLING branch: the same paragraph or run object may legally sit at positions under two sibling wrappers (a caller-built document that pushes one node into two sections is a two-position tree once serialised), and chain-global bookkeeping would let the first wrapper's mint suppress the second branch's own mint -- leaving the second position's chain ref-less while a strip still took its properties, silently breaking law (i). Branch-scoped, the sibling mints the identical entry content, shares the table entry through the canonical key, and carries its own ref, so both positions resolve back. +interface Branch { + readonly frozenParagraphs: ReadonlySet; + readonly frozenRuns: ReadonlySet; + readonly factoredParagraphs: ReadonlySet; + readonly factoredRuns: ReadonlySet; +} + +// One accumulated table entry: its resolved content, the wrappers referencing it, and the ordering inputs (total stripped positions and the first wrapper's pre-order visit index). +interface MintedEntry { + readonly content: StyleEntry; + readonly wrappers: MintWrapper[]; + frequency: number; + firstVisit: number; +} + +// Visits one wrapper outermost-first: selects at most one entry here, records its strips against this wrapper, freezes its keys for everything below, then recurses into the child wrappers with the branch bookkeeping extended (copy-on-descend, so sibling branches stay independent). `visit.index` numbers wrappers in pre-order -- the "first occurrence" arm of the entry ordering rule. +function plan(wrapper: MintWrapper, visit: { index: number }, branch: Branch, state: MintState, entries: Map): void { + const extent = extentOf(wrapper); + const paragraphCandidate = extent.length > 0 ? bestParagraphCandidate(extent, commonParagraphKeys(extent, branch.frozenParagraphs), branch.factoredParagraphs) : undefined; + const runCandidate = extent.length > 0 ? bestRunCandidate(extent, commonRunKeys(extent, branch.frozenRuns), branch.factoredRuns) : undefined; + + const nextFrozenParagraphs = new Set(branch.frozenParagraphs); + const nextFrozenRuns = new Set(branch.frozenRuns); + const nextFactoredParagraphs = new Set(branch.factoredParagraphs); + const nextFactoredRuns = new Set(branch.factoredRuns); + if (paragraphCandidate !== undefined || runCandidate !== undefined) { + const content: StyleEntry = { + ...(paragraphCandidate !== undefined ? { paragraph: paragraphCandidate.tuple } : {}), + ...(runCandidate !== undefined ? { run: runCandidate.tuple } : {}), + }; + const contentKey = canonicalKey(content); + const existing = entries.get(contentKey); + if (existing === undefined) { + entries.set(contentKey, { content, wrappers: [wrapper], frequency: (paragraphCandidate?.positions.length ?? 0) + (runCandidate?.positions.length ?? 0), firstVisit: visit.index }); + } else { + existing.wrappers.push(wrapper); + existing.frequency += (paragraphCandidate?.positions.length ?? 0) + (runCandidate?.positions.length ?? 0); + } + const strips: WrapperStrips = { paragraphs: new Map(), runs: new Map() }; + if (paragraphCandidate !== undefined) { + for (const paragraph of paragraphCandidate.positions) { + nextFactoredParagraphs.add(paragraph); + strips.paragraphs.set(paragraph, paragraphCandidate.keys); + } + for (const key of paragraphCandidate.keys) nextFrozenParagraphs.add(key); + } + if (runCandidate !== undefined) { + for (const run of runCandidate.positions) { + nextFactoredRuns.add(run); + strips.runs.set(run, runCandidate.keys); + } + for (const key of runCandidate.keys) nextFrozenRuns.add(key); + } + state.wrapperStrips.set(wrapper, strips); + } + + visit.index += 1; + const next: Branch = { frozenParagraphs: nextFrozenParagraphs, frozenRuns: nextFrozenRuns, factoredParagraphs: nextFactoredParagraphs, factoredRuns: nextFactoredRuns }; + for (const child of childWrappers(wrapper)) { + plan(child, visit, next, state, entries); + } +} + +// The direct child wrappers of a wrapper, in document order -- the pre-order walk's recursion set. +function childWrappers(wrapper: MintWrapper): MintWrapper[] { + if (isShapeGroupWrapper(wrapper) || isAnchorGroupWrapper(wrapper)) { + // A shape's flow and a heading/list group's flow share the loop: nested groups are the child wrappers, leaves are not. An explicit loop rather than filter's type-guard overload because children arrives as a union of array types, whose filter signature TypeScript resolves without the predicate. + const wrappers: MintWrapper[] = []; + for (const child of wrapper.children) { + if ('node' in child && 'children' in child) wrappers.push(child); + } + return wrappers; + } + if (isSlideGroupWrapper(wrapper)) { + return [...wrapper.children]; + } + if (isDrawPageGroupWrapper(wrapper)) { + const shapes: ShapeGroupNode[] = []; + for (const child of wrapper.children) { + if ('node' in child) shapes.push(child); + } + return shapes; + } + // A section group: its flow's nested groups are the child wrappers. + const wrappers: MintWrapper[] = []; + for (const child of wrapper.children) { + if ('node' in child && 'children' in child) wrappers.push(child); + } + return wrappers; +} + +// The entry point over a whole tree: plan (outermost-first, freezing keys and factoring positions down each chain), order the entries, then rebuild the tree stamping refs and stripping keys per chain. Exported beyond assemblePackage's own internal use so a caller holding an already-tree-form package can mint over it directly without a flatten/decompose round trip first. +export function mint(pkg: DocumentPackage): DocumentPackage { + const state: MintState = { + wrapperRefs: new Map(), + wrapperStrips: new Map(), + }; + const entries = new Map(); + const visit = { index: 0 }; + const rootBranch: Branch = { frozenParagraphs: new Set(), frozenRuns: new Set(), factoredParagraphs: new Set(), factoredRuns: new Set() }; + switch (pkg.kind) { + case 'wordprocessing': + for (const root of pkg.children) plan(root, visit, rootBranch, state, entries); + break; + case 'presentation': + for (const root of pkg.children) plan(root, visit, rootBranch, state, entries); + break; + case 'drawing': + for (const root of pkg.children) plan(root, visit, rootBranch, state, entries); + break; + // A spreadsheet's roots are sheet groups (no block flow, never minted) and a formula package's single child is a leaf: neither holds a wrapper to visit. + case 'spreadsheet': + case 'formula': + break; + } + if (entries.size === 0) { + return pkg; + } + // Entry ids in (descending total frequency, first wrapper visit) order -- the deterministic table order the plan locks. The comparator is total at two arms: one wrapper mints at most one entry, so distinct entries always have distinct first visits and a further tie-break arm could never bind. + const ordered = [...entries.values()].sort((a, b) => b.frequency - a.frequency || a.firstVisit - b.firstVisit); + const styles: StylesTable = {}; + ordered.forEach((entry, index) => { + const id = `s${index + 1}`; + styles[id] = entry.content; + for (const wrapper of entry.wrappers) state.wrapperRefs.set(wrapper, id); + }); + // Per-arm spreads rather than one spread of the union: a literal containing a union spread widens its discriminant-narrowed properties and stops assigning to DocumentPackageSchema's inferred type, so each arm rebuilds itself with its own children type. + switch (pkg.kind) { + case 'wordprocessing': + return { ...pkg, styles, children: pkg.children.map((group) => rebuildSectionGroup(group, [], state)) }; + case 'presentation': + return { ...pkg, styles, children: pkg.children.map((group) => rebuildSlideGroup(group, [], state)) }; + case 'drawing': + return { ...pkg, styles, children: pkg.children.map((group) => rebuildDrawPageGroup(group, [], state)) }; + // No wrapper was visited for these arms (sheets hold no block flow; a formula package holds one leaf), so entries is empty and the early return above has already fired -- the arms exist for switch totality only. + case 'spreadsheet': + case 'formula': + return { ...pkg, styles }; + } +} + +// The strip records of every minted wrapper on the current rebuild chain, outermost first (each rebuild level appends its own record before walking its children). A node's strip is the LAST record naming it -- the chain is outermost-first, so the last is the innermost, and branch-scoped factoring gives each chain at most one minter per node anyway, which is what makes the per-position rule exact: an aliased node is stripped by its own branch's record and never by a sibling's. +type ChainStrips = readonly WrapperStrips[]; + +// The chain one level deeper than `group`: unchanged when this wrapper minted nothing, extended by its own strips when it did. +function innerChain(group: MintWrapper, chain: ChainStrips, state: MintState): ChainStrips { + const own = state.wrapperStrips.get(group); + return own === undefined ? chain : [...chain, own]; +} + +// The strip a wrapper on `chain` recorded against `paragraph`, or undefined when no wrapper on the chain factored it (the paragraph rides through as the same object). +function paragraphStripsOf(chain: ChainStrips, paragraph: ContentParagraph): readonly ParagraphKey[] | undefined { + let result: readonly ParagraphKey[] | undefined; + for (const strips of chain) { + const found = strips.paragraphs.get(paragraph); + if (found !== undefined) result = found; + } + return result; +} + +function runStripsOf(chain: ChainStrips, run: ContentRun): readonly RunKey[] | undefined { + let result: readonly RunKey[] | undefined; + for (const strips of chain) { + const found = strips.runs.get(run); + if (found !== undefined) result = found; + } + return result; +} + +// One slide group: stamp its ref when minted, rebuild its shapes below it, and return the same object when neither changed. +function rebuildSlideGroup(group: SlideGroupNode, chain: ChainStrips, state: MintState): SlideGroupNode { + const inner = innerChain(group, chain, state); + const children = group.children.map((shape) => rebuildShapeGroup(shape, inner, state)); + const ref = state.wrapperRefs.get(group); + const unchanged = ref === undefined && children.every((child, index) => child === group.children[index]); + return unchanged ? group : { node: group.node, ...(ref !== undefined ? { style: ref } : {}), children }; +} + +// One draw-page group: its shape children rebuild, its vector leaves pass through unchanged (no paragraphs to strip, no ref to carry -- a vector is a leaf). +function rebuildDrawPageGroup(group: DrawPageGroupNode, chain: ChainStrips, state: MintState): DrawPageGroupNode { + const inner = innerChain(group, chain, state); + const children = group.children.map((child) => ('node' in child ? rebuildShapeGroup(child, inner, state) : child)); + const ref = state.wrapperRefs.get(group); + const unchanged = ref === undefined && children.every((child, index) => child === group.children[index]); + return unchanged ? group : { node: group.node, ...(ref !== undefined ? { style: ref } : {}), children }; +} + +// One section group: stamp its ref when minted, rebuild its flow below it, and return the same object when neither changed. +function rebuildSectionGroup(group: SectionGroupNode, chain: ChainStrips, state: MintState): SectionGroupNode { + const inner = innerChain(group, chain, state); + const children = group.children.map((child) => rebuildSectionChild(child, inner, state)); + const ref = state.wrapperRefs.get(group); + const unchanged = ref === undefined && children.every((child, index) => child === group.children[index]); + return unchanged ? group : { node: group.node, ...(ref !== undefined ? { style: ref } : {}), children }; +} + +// One section-flow child position: a construct group recurses through its own rebuilder (no anchor to narrow), a heading group recurses through the section-flow vocabulary, a list group through the list-flow vocabulary (its own children are ListChild, the shared list/shape vocabulary), a bare paragraph leaf is copied only when stripped, every other leaf passes through as the same object. +function rebuildSectionChild(child: SectionChild, chain: ChainStrips, state: MintState): SectionChild { + if (isConstructGroup(child)) { + return rebuildSectionConstructGroup(child, chain, state); + } + if ('node' in child && 'children' in child) { + return isHeadingGroup(child) ? rebuildHeadingGroup(child, chain, state, rebuildSectionChild) : rebuildListGroup(child, chain, state, rebuildListChild); + } + if (child.kind === 'paragraph') { + return rebuildParagraph(child, chain); + } + return child; +} + +// One list-flow child position -- the shared vocabulary of list-group children and shape flows. +function rebuildListChild(child: ListChild, chain: ChainStrips, state: MintState): ListChild { + if (isConstructGroup(child)) { + return rebuildShapeConstructGroup(child, chain, state); + } + if ('node' in child && 'children' in child) { + return rebuildListGroup(child, chain, state, rebuildListChild); + } + if (child.kind === 'paragraph') { + return rebuildParagraph(child, chain); + } + return child; +} + +// A shape group: no anchor of its own, its list-flow children rebuilt through the shared walk. +function rebuildShapeGroup(group: ShapeGroupNode, chain: ChainStrips, state: MintState): ShapeGroupNode { + const inner = innerChain(group, chain, state); + const children = group.children.map((child) => rebuildListChild(child, inner, state)); + const ref = state.wrapperRefs.get(group); + const unchanged = ref === undefined && children.every((child, index) => child === group.children[index]); + return unchanged ? group : { node: group.node, ...(ref !== undefined ? { style: ref } : {}), children }; +} + +// A construct group sat in a section flow: no anchor of its own (its node is a ConstructDescriptor, never a paragraph), so it rebuilds exactly like rebuildSectionGroup -- stamp its own ref when minted, rebuild its section-flow children below it. +function rebuildSectionConstructGroup(group: SectionConstructGroupNode, chain: ChainStrips, state: MintState): SectionConstructGroupNode { + const inner = innerChain(group, chain, state); + const children = group.children.map((child) => rebuildSectionChild(child, inner, state)); + const ref = state.wrapperRefs.get(group); + const unchanged = ref === undefined && children.every((child, index) => child === group.children[index]); + return unchanged ? group : { node: group.node, ...(ref !== undefined ? { style: ref } : {}), children }; +} + +// A construct group sat in a shape or list-item flow: the same shape as rebuildSectionConstructGroup, over the list-flow vocabulary instead. +function rebuildShapeConstructGroup(group: ShapeConstructGroupNode, chain: ChainStrips, state: MintState): ShapeConstructGroupNode { + const inner = innerChain(group, chain, state); + const children = group.children.map((child) => rebuildListChild(child, inner, state)); + const ref = state.wrapperRefs.get(group); + const unchanged = ref === undefined && children.every((child, index) => child === group.children[index]); + return unchanged ? group : { node: group.node, ...(ref !== undefined ? { style: ref } : {}), children }; +} + +function rebuildHeadingGroup(group: HeadingGroupNode, chain: ChainStrips, state: MintState, rebuildChild: (child: SectionChild, chain: ChainStrips, state: MintState) => SectionChild): HeadingGroupNode { + const inner = innerChain(group, chain, state); + const anchor = rebuildParagraph(group.node, inner); + assertHeadingAnchor(anchor); + const children = group.children.map((child) => rebuildChild(child, inner, state)); + const ref = state.wrapperRefs.get(group); + const unchanged = ref === undefined && anchor === group.node && children.every((child, index) => child === group.children[index]); + return unchanged ? group : { node: anchor, ...(ref !== undefined ? { style: ref } : {}), children }; +} + +function rebuildListGroup(group: ListGroupNode, chain: ChainStrips, state: MintState, rebuildChild: (child: ListChild, chain: ChainStrips, state: MintState) => ListChild): ListGroupNode { + const inner = innerChain(group, chain, state); + const anchor = rebuildParagraph(group.node, inner); + assertListAnchor(anchor); + const children = group.children.map((child) => rebuildChild(child, inner, state)); + const ref = state.wrapperRefs.get(group); + const unchanged = ref === undefined && anchor === group.node && children.every((child, index) => child === group.children[index]); + return unchanged ? group : { node: anchor, ...(ref !== undefined ? { style: ref } : {}), children }; +} + +// rebuildParagraph is typed on the loose ContentParagraph, so a rebuilt anchor comes back with its REQUIRED grouping signal widened to optional; these assertions re-narrow it without a cast, exactly as flatten.ts does for resolved anchors. Stripping only ever deletes mintable style keys (never headingLevel or list -- see the module doc), so the signal always survives; the throw is the loud guard if that contract ever broke. +function assertHeadingAnchor(paragraph: ContentParagraph): asserts paragraph is HeadingParagraph { + if (paragraph.headingLevel === undefined) throw new Error("factorStyles: stripping dropped a heading anchor's headingLevel"); +} + +function assertListAnchor(paragraph: ContentParagraph): asserts paragraph is ListGroupNode['node'] { + if (paragraph.list === undefined) throw new Error("factorStyles: stripping dropped a list anchor's list membership"); +} + +// One paragraph (leaf or anchor): stripped -- copied sans its minted keys -- when a wrapper on its chain factored it (chain-scoped, so an aliased position is stripped by its own branch's minter, never another branch's), with its runs rebuilt through the same copy-or-share rule. Returns the same object when nothing under it changed. +function rebuildParagraph(paragraph: ContentParagraph, chain: ChainStrips): ContentParagraph { + const strips = paragraphStripsOf(chain, paragraph); + const base = strips === undefined ? paragraph : stripParagraphKeys(paragraph, strips); + let changed = base !== paragraph; + const runs: ContentRun[] = []; + for (const run of base.runs) { + const runStrips = runStripsOf(chain, run); + const rebuilt = runStrips === undefined ? run : stripRunKeys(run, runStrips); + changed ||= rebuilt !== run; + runs.push(rebuilt); + } + if (!changed) return paragraph; + return { ...base, runs }; +} + +// Copies a paragraph sans the named keys -- copy-then-delete, never destructuring the keys out (an unused binding) and never mutating the input (decompose embedded the caller's own node objects, and the layout pass's frames ride on them). Every mintable paragraph key is optional on ContentParagraph, so the deletes are type-honest. +function stripParagraphKeys(paragraph: ContentParagraph, keys: readonly ParagraphKey[]): ContentParagraph { + const copy: ContentParagraph = { ...paragraph }; + for (const key of keys) delete copy[key]; + return copy; +} + +function stripRunKeys(run: ContentRun, keys: readonly RunKey[]): ContentRun { + const copy: ContentRun = { ...run }; + for (const key of keys) delete copy[key]; + return copy; +} diff --git a/src/flatten.test.ts b/src/flatten.test.ts new file mode 100644 index 0000000..d5b6f50 --- /dev/null +++ b/src/flatten.test.ts @@ -0,0 +1,151 @@ +import { describe, expect, it } from 'vitest'; +import type { ContentParagraph, ContentSection } from './content'; +import { decomposeSection } from './decompose'; +import { flattenPackage } from './flatten'; +import type { StylesTable } from './definitions'; +import type { DocumentPackage } from './package'; +import type { HeadingGroupNode, SectionGroupNode, ShapeGroupNode, SlideGroupNode } from './package-node'; + +// flattenPackage entered directly, on trees a caller hands in rather than ones assemblePackage just built. The bijection suite already pins the round trip over a styles-free-or-freshly-minted tree; what only a direct entry can reach is the resolver's own semantics (which chain a position resolves against, and that gap-fill never overwrites) and the two refusals a hand-built tree can trigger. decompose.test.ts covers the third refusal, a style ref on a sheet group. + +const SECTION = { pageSize: { widthPt: 595, heightPt: 842 }, margins: { topPt: 72, rightPt: 72, bottomPt: 72, leftPt: 72 } }; + +function paragraph(text: string, properties: Partial = {}): ContentParagraph { + return { kind: 'paragraph', runs: [{ text }], ...properties }; +} + +function wordprocessingPackage(children: SectionGroupNode[], styles?: StylesTable): DocumentPackage { + return { kind: 'wordprocessing', metadata: {}, ...(styles !== undefined ? { styles } : {}), children }; +} + +function sectionBlocks(pkg: DocumentPackage): ContentSection['blocks'] { + const flat = flattenPackage(pkg); + if (flat.kind !== 'wordprocessing') throw new Error('expected a wordprocessing document back'); + const section = flat.sections[0]; + if (section === undefined) throw new Error('expected one section back'); + return section.blocks; +} + +describe('flattenPackage style resolution', () => { + it('overlays the chain outermost-first, so the nearest group wins over a further-out one', () => { + const body = paragraph('body'); + const headingGroup: HeadingGroupNode = { node: { kind: 'paragraph', headingLevel: 1, runs: [{ text: 'Chapter' }] }, style: 'inner', children: [body] }; + const pkg = wordprocessingPackage( + [{ node: { kind: 'section', ...SECTION }, style: 'outer', children: [headingGroup] }], + { outer: { paragraph: { indentLeftPt: 10, alignment: 'left' } }, inner: { paragraph: { indentLeftPt: 40 } } }, + ); + // Both positions sit under outer+inner: alignment comes from the only entry that carries it, indentLeftPt from the nearer one. + expect(sectionBlocks(pkg)).toEqual([ + { kind: 'paragraph', headingLevel: 1, runs: [{ text: 'Chapter' }], indentLeftPt: 40, alignment: 'left' }, + { kind: 'paragraph', runs: [{ text: 'body' }], indentLeftPt: 40, alignment: 'left' }, + ]); + }); + + it('fills gaps only -- a property the node already carries survives whatever the entry says', () => { + const pkg = wordprocessingPackage( + [{ node: { kind: 'section', ...SECTION }, style: 's1', children: [paragraph('keeps its own', { indentLeftPt: 99 }), paragraph('takes the entry\'s')] }], + { s1: { paragraph: { indentLeftPt: 20 } } }, + ); + expect(sectionBlocks(pkg).map((block) => (block.kind === 'paragraph' ? block.indentLeftPt : undefined))).toEqual([99, 20]); + }); + + it('applies a resolved entry\'s run half to every run of every paragraph it resolved for', () => { + const pkg = wordprocessingPackage( + [{ node: { kind: 'section', ...SECTION }, style: 's1', children: [{ kind: 'paragraph', runs: [{ text: 'a' }, { text: 'b', bold: false }] }] }], + { s1: { run: { bold: true, sizePt: 11 } } }, + ); + const [block] = sectionBlocks(pkg); + if (block?.kind !== 'paragraph') throw new Error('expected a paragraph back'); + // The second run's own `bold: false` is a carried value, not a gap, so the entry does not overwrite it. + expect(block.runs).toEqual([{ text: 'a', bold: true, sizePt: 11 }, { text: 'b', bold: false, sizePt: 11 }]); + }); + + it('leaves an unreferenced subtree\'s own objects untouched -- no chain, no copy', () => { + const untouched = paragraph('no ref anywhere above me'); + const pkg = wordprocessingPackage([{ node: { kind: 'section', ...SECTION }, children: [untouched] }]); + expect(sectionBlocks(pkg)[0]).toBe(untouched); + }); + + it('resolves a shape group\'s ref through the slide group above it', () => { + const shape: ShapeGroupNode = { + node: { frame: { xPt: 0, yPt: 0, widthPt: 400, heightPt: 300 }, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0 }, + style: 'shape', + children: [paragraph('in the shape')], + }; + const slide: SlideGroupNode = { node: { kind: 'slide', size: { widthPt: 960, heightPt: 540 }, notes: '' }, style: 'slide', children: [shape] }; + const flat = flattenPackage({ + kind: 'presentation', + metadata: {}, + styles: { slide: { run: { fontFamily: 'Inter' } }, shape: { paragraph: { alignment: 'center' } } }, + children: [slide], + }); + if (flat.kind !== 'presentation') throw new Error('expected a presentation back'); + expect(flat.slides[0]?.shapes[0]?.blocks).toEqual([{ kind: 'paragraph', runs: [{ text: 'in the shape', fontFamily: 'Inter' }], alignment: 'center' }]); + }); + + it('refuses a ref it cannot resolve loudly, rather than skipping that level of the chain', () => { + const pkg = wordprocessingPackage([{ node: { kind: 'section', ...SECTION }, style: 'missing', children: [paragraph('x')] }], { s1: { paragraph: { alignment: 'left' } } }); + expect(() => flattenPackage(pkg)).toThrow(/names no entry in the styles table/); + }); + + it('refuses a ref on a package that carries no styles table at all', () => { + // Resolution runs completely or not at all: a tree stating a ref against no table is malformed, and passing the ref by silently would drop the styling it names with no signal. + const pkg = wordprocessingPackage([{ node: { kind: 'section', ...SECTION }, style: 's1', children: [paragraph('x')] }]); + expect(() => flattenPackage(pkg)).toThrow(/no styles table/); + }); +}); + +describe('flattenPackage cardinality guards', () => { + it('refuses a formula package holding anything other than exactly one ContentFormula', () => { + const formula = { mathml: [] }; + expect(() => flattenPackage({ kind: 'formula', metadata: {}, children: [] })).toThrow(/exactly one ContentFormula/); + expect(() => flattenPackage({ kind: 'formula', metadata: {}, children: [formula, formula] })).toThrow(/exactly one ContentFormula/); + expect(flattenPackage({ kind: 'formula', metadata: {}, children: [formula] })).toEqual({ kind: 'formula', metadata: {}, formula }); + }); +}); + +describe('flattenPackage envelope handling', () => { + it('carries metadata and symbolTable back onto the flat document, and drops the tree-only pages array', () => { + // `pages` and the package tables have no spelling on a flat ContentDocument, so flatten states the envelope it can carry and nothing else -- factorStyles is what rides pages and definitions across a re-factoring. + const flat = flattenPackage({ + kind: 'wordprocessing', + metadata: { title: 'Envelope' }, + symbolTable: { symbols: [], units: [] }, + pages: [{ widthPt: 595, heightPt: 842 }], + definitions: { n1: { kind: 'footnote' } }, + children: [{ node: { kind: 'section', ...SECTION }, children: [] }], + }); + expect(flat).toEqual({ kind: 'wordprocessing', metadata: { title: 'Envelope' }, symbolTable: { symbols: [], units: [] }, sections: [{ ...SECTION, blocks: [] }] }); + }); + + it('rebuilds a draw page\'s shapes-then-vectors partition and a sheet\'s images-then-embedded-objects one', () => { + const vector = { kind: 'rect', frame: { xPt: 1, yPt: 2, widthPt: 3, heightPt: 4 } } as const; + const drawing = flattenPackage({ + kind: 'drawing', + metadata: {}, + children: [{ + node: { kind: 'drawPage', size: { widthPt: 300, heightPt: 300 } }, + children: [{ node: { frame: { xPt: 0, yPt: 0, widthPt: 10, heightPt: 10 }, insetLeftPt: 0, insetTopPt: 0, insetRightPt: 0, insetBottomPt: 0 }, children: [] }, vector], + }], + }); + if (drawing.kind !== 'drawing') throw new Error('expected a drawing back'); + expect(drawing.pages[0]?.shapes).toHaveLength(1); + expect(drawing.pages[0]?.vectors).toEqual([vector]); + }); + + it('is the exact inverse of decomposeSection for a section whose flow uses every grouping signal', () => { + const source: ContentSection = { + ...SECTION, + blocks: [ + paragraph('Chapter', { headingLevel: 1 }), + paragraph('item', { list: { level: 0 } }), + paragraph('nested item', { list: { level: 1 } }), + { kind: 'constructStart', descriptor: { kind: 'field', instruction: 'PAGE' } }, + paragraph('inside the field'), + { kind: 'constructEnd' }, + paragraph('plain'), + ], + }; + expect(sectionBlocks(wordprocessingPackage([decomposeSection(source)]))).toEqual(source.blocks); + }); +}); diff --git a/src/flatten.ts b/src/flatten.ts new file mode 100644 index 0000000..deda681 --- /dev/null +++ b/src/flatten.ts @@ -0,0 +1,208 @@ +import { + ContentFormulaSchema, + type ContentBlock, + type ContentDocument, + type ContentDrawPage, + type ContentEmbeddedObject, + type ContentParagraph, + type ContentSection, + type ContentShape, + type ContentSheet, + type ContentSheetImage, + type ContentSlide, + type ContentVector, +} from './content'; +import { + applyParagraphStyleProperties, + applyRunStyleProperties, + resolveStyleChain, + type StyleEntry, + type StylesTable, +} from './definitions'; +import type { DocumentPackage } from './package'; +import type { + HeadingGroupNode, + ListChild, + ListGroupNode, + SectionChild, + SectionConstructGroupNode, + ShapeConstructGroupNode, + ShapeGroupNode, +} from './package-node'; + +// The tree-to-flat half of the package boundary, ported from document-outline.js's phase-1 reference (pre-re-charter history) onto the 4.x tree types, with the styles resolution the reference left as a separate effective/effectiveTree seam fused directly into the walk: the flat codec-exchange form is ALWAYS fully materialised (no table, no refs -- #21's own rule), so materialising and restructuring are one pass here. Resolution semantics are the reviewed reference's: a group's ref plus every ancestor group's ref overlays onto each paragraph in that group's subtree -- group anchors (heading and list groups carry ContentParagraph anchors) and bare paragraph leaves alike -- with the chain ordered outermost first so the nearest group's entry wins over further-out ones and the paragraph's own direct properties win over everything (applyParagraphStyleProperties / applyRunStyleProperties fill gaps, never overwrite). The run half of a resolved entry applies to every run of each paragraph it resolved for. The walk's boundary is the block flow: a table leaf's cell paragraphs and an embedded document's own content are leaf-local payload this walk does not rewrite, exactly as resolution does not. +// +// For a styles-free package the walk emits the SAME node objects the tree embeds (no copies -- the ownership discipline decompose.ts states), so flattenPackage(assemblePackage(c)) shares every content node with c unless minting factored a property tuple onto a wrapper ref (those paragraphs come back as resolved copies carrying identical values). The one node with no object to share is a construct boundary: PackageBlockLeaf excludes both marker kinds, so a construct group's constructStart/constructEnd pair is rebuilt here from the group's own ConstructDescriptor (that descriptor object IS the one decompose embedded, handed straight back on the rebuilt marker) rather than carried through as the marker object the flat form arrived with. +// +// A construct group's style chain is CUMULATIVE, not reset: the recursion extends the incoming chain with the group's own ref exactly as a heading or list group does, never starts a fresh one the way a section/slide/sheet/draw-page root does. A construct is a semantic wrapper nested inside ambient content -- a field, a content control, a tracked-change span -- so a paragraph inside it must still inherit the enclosing heading's or section's factored properties, as though the construct were not there. That is a different axis from the heading/list STACK reset decompose performs when it walks a construct's interior: that reset is about which markers group structurally, this is about which refs resolve, and the two are deliberately independent. + +// A group's chain extended by its own ref when it carries one: the array passed to everything inside the group, which is how a group's style applies to its whole subtree. Outermost-first order, so resolveStyleChain's overlay fold makes the nearest entry win over further-out ones. +function chainWithRef(chain: readonly string[], group: { readonly style?: string }): readonly string[] { + return group.style === undefined ? chain : [...chain, group.style]; +} + +// The resolved entry a chain names, or undefined for an empty chain. resolveStyleChain itself is the loud refusal on a ref the styles table does not carry: consistency between refs and the table is the producer's responsibility, and once resolution runs it runs completely or not at all. +function entryOf(styles: StylesTable | undefined, chain: readonly string[]): StyleEntry | undefined { + if (chain.length === 0) return undefined; + if (styles === undefined) { + throw new Error('flattenPackage: a group carries a style ref but the package has no styles table'); + } + return resolveStyleChain(styles, chain); +} + +// Applies one resolved entry to one paragraph: the entry's paragraph half fills the paragraph's own gaps, its run half fills each run's gaps. Pure -- unchanged halves return the same objects (applyParagraphStyleProperties itself returns the input paragraph when the entry has no paragraph half). +function applyEntry(entry: StyleEntry, paragraph: ContentParagraph): ContentParagraph { + const withParagraph = applyParagraphStyleProperties(entry.paragraph, paragraph); + const runProperties = entry.run; + if (runProperties === undefined) return withParagraph; + return { ...withParagraph, runs: withParagraph.runs.map((run) => applyRunStyleProperties(runProperties, run)) }; +} + +// The exact inverse of decompose: a pre-order walk over the tree reconstituting sections, slides, sheets, and pages in document order, re-emitting every group-represented paragraph as an ordinary block (a heading or list group's anchor paragraph IS the block; it was never copied, only wrapped) and every construct group as the constructStart/constructEnd marker pair that delimited it, with every style ref resolved away into materialised direct properties. Leaf nodes pass through as the same objects. The result is schema-valid against ContentDocumentSchema and structurally identical to the source document the tree was assembled from -- the bijection law flattenPackage(assemblePackage(c)) reproduces c exactly, pinned in bijection.test.ts. +export function flattenPackage(pkg: DocumentPackage): ContentDocument { + const styles = pkg.styles; + const envelope = { + metadata: pkg.metadata, + ...(pkg.symbolTable !== undefined ? { symbolTable: pkg.symbolTable } : {}), + }; + switch (pkg.kind) { + case 'wordprocessing': + return { + kind: 'wordprocessing', + ...envelope, + sections: pkg.children.map((group): ContentSection => ({ + ...untag(group.node), + blocks: flattenSectionChildren(styles, chainWithRef([], group), group.children), + })), + }; + case 'presentation': + return { + kind: 'presentation', + ...envelope, + slides: pkg.children.map((group): ContentSlide => { + const chain = chainWithRef([], group); + return { ...untag(group.node), shapes: group.children.map((shape) => flattenShape(styles, chain, shape)) }; + }), + }; + case 'spreadsheet': + return { + kind: 'spreadsheet', + ...envelope, + sheets: pkg.children.map((group): ContentSheet => { + // The schema allows a style ref on every group node, but a sheet group holds no block flow, so a chain built here has nothing to resolve onto -- refuse rather than pass the ref by silently, the same all-or-nothing rule as entryOf's missing-table refusal below. Minting never stamps a ref on a sheet (its extent is always empty); the guard is for hand-built trees. + if (group.style !== undefined) { + throw new Error('flattenPackage: a sheet group carries a style ref but a sheet holds no block flow to resolve it onto'); + } + const images: ContentSheetImage[] = []; + const embedded: ContentEmbeddedObject[] = []; + for (const child of group.children) { + // ContentSheetImage carries a `kind` ('image') and ContentEmbeddedObject carries none, so the property's presence partitions the two sibling arrays exactly as decompose concatenated them. + if ('kind' in child) images.push(child); + else embedded.push(child); + } + // embeddedObjects is rebuilt only when the sheet actually carried embedded objects, so a sheet whose field was absent round-trips with it absent again -- absent-versus-present is content here, not a default to fill in. The one declared exception: a present-but-empty array (schema-legal, emitted by no codec) is indistinguishable from an absent field once decompose has concatenated images and embedded objects into one children array, so it normalises to absent (bijection.test.ts declares the normalisation on both sides). + return { ...untag(group.node), images, ...(embedded.length > 0 ? { embeddedObjects: embedded } : {}) }; + }), + }; + case 'drawing': + return { + kind: 'drawing', + ...envelope, + pages: pkg.children.map((group): ContentDrawPage => { + const chain = chainWithRef([], group); + const shapes: ContentShape[] = []; + const vectors: ContentVector[] = []; + for (const child of group.children) { + // Shape groups carry `node`; vectors do not -- the presence check reverses decompose's fixed shapes-then-vectors concatenation without re-inspecting payloads. + if ('node' in child) shapes.push(flattenShape(styles, chain, child)); + else vectors.push(child); + } + return { ...untag(group.node), shapes, vectors }; + }), + }; + case 'formula': { + // A formula document is the one tree shape with no container: exactly one node, the ContentFormula leaf itself. + const first = pkg.children[0]; + if (pkg.children.length !== 1 || first === undefined || !ContentFormulaSchema.safeParse(first).success) { + throw new Error('flattenPackage: a formula package takes exactly one ContentFormula node'); + } + return { kind: 'formula', ...envelope, formula: first }; + } + } +} + +// Strips a container descriptor's tree-only `kind` tag, keeping every other field by spread rather than by naming them: decompose rest-spreads each flat container's fields into its descriptor (minus the arrays that became children), so a container field added by a future schema release rides the descriptor, and flatten must hand it back without this package ever naming it. Copy-then-delete rather than destructuring the tag out, because the repo's lint bans unused bindings outright and a destructured-away tag would be exactly that. +function untag(descriptor: D): Omit { + const copy: { kind?: D['kind'] } & Omit = { ...descriptor }; + delete copy.kind; + return copy; +} + +function flattenShape(styles: StylesTable | undefined, chain: readonly string[], group: ShapeGroupNode): ContentShape { + // A shape group's descriptor needs no untagging -- ContentShape carries no kind, so the descriptor is every field except blocks and the blocks ride straight back on. + return { ...group.node, blocks: flattenListChildren(styles, chainWithRef(chain, group), group.children) }; +} + +// One section-flow child walk: nested heading/list groups recurse with their extended chain, a construct group re-emits its boundary marker pair around its own recursed extent (see below), a bare paragraph leaf resolves against the incoming chain (it carries no ref of its own -- refs are legal only on group wrappers), and every other leaf is its own payload, untouched. +function flattenSectionChildren(styles: StylesTable | undefined, chain: readonly string[], children: readonly SectionChild[]): ContentBlock[] { + const blocks: ContentBlock[] = []; + for (const child of children) { + if (isHeadingGroup(child)) { + const own = chainWithRef(chain, child); + blocks.push(resolveAnchor(styles, own, child.node), ...flattenSectionChildren(styles, own, child.children)); + } else if (isListGroup(child)) { + const own = chainWithRef(chain, child); + blocks.push(resolveAnchor(styles, own, child.node), ...flattenListChildren(styles, own, child.children)); + } else if (isConstructGroup(child)) { + const own = chainWithRef(chain, child); + blocks.push({ kind: 'constructStart', descriptor: child.node }, ...flattenSectionChildren(styles, own, child.children), { kind: 'constructEnd' }); + } else if (child.kind === 'paragraph') { + const entry = entryOf(styles, chain); + blocks.push(entry === undefined ? child : applyEntry(entry, child)); + } else { + blocks.push(child); + } + } + return blocks; +} + +// The shared vocabulary of shape flows and list-group children (ListChild: ListGroupNode | ShapeConstructGroupNode | ContentBlock), so one walk serves both. +function flattenListChildren(styles: StylesTable | undefined, chain: readonly string[], children: readonly ListChild[]): ContentBlock[] { + const blocks: ContentBlock[] = []; + for (const child of children) { + if (isListGroup(child)) { + const own = chainWithRef(chain, child); + blocks.push(resolveAnchor(styles, own, child.node), ...flattenListChildren(styles, own, child.children)); + } else if (isConstructGroup(child)) { + const own = chainWithRef(chain, child); + blocks.push({ kind: 'constructStart', descriptor: child.node }, ...flattenListChildren(styles, own, child.children), { kind: 'constructEnd' }); + } else if (child.kind === 'paragraph') { + const entry = entryOf(styles, chain); + blocks.push(entry === undefined ? child : applyEntry(entry, child)); + } else { + blocks.push(child); + } + } + return blocks; +} + +// Structural narrows over the already-typed child unions, avoiding a widening round-trip through the schema's unknown-taking guards inside this module's own walks: every group kind carries `node`+`children`, no block leaf does. +function isHeadingGroup(child: SectionChild): child is HeadingGroupNode { + return 'node' in child && 'children' in child && child.node.kind === 'paragraph' && child.node.headingLevel !== undefined; +} + +function isListGroup(child: SectionChild | ListChild): child is ListGroupNode { + return 'node' in child && 'children' in child && child.node.kind === 'paragraph' && child.node.list !== undefined; +} + +// A construct group's own node is a ConstructDescriptor (contentControl/field/anchor/link/provenance/division), never a paragraph -- discriminated off the same node.kind property the heading/list narrows above read, since a ConstructDescriptor's `kind` is always disjoint from `'paragraph'`. Section and shape construct groups share this one narrow: the emitted marker pair is identical for both, and each call site's own argument type is what picks the flow the extent recurses through, so the guard never needs to tell the two apart itself. +function isConstructGroup(child: SectionChild | ListChild): child is SectionConstructGroupNode | ShapeConstructGroupNode { + return 'node' in child && 'children' in child && child.node.kind !== 'paragraph'; +} + +// One group anchor under its own chain: an empty chain leaves the anchor object as-is (the ownership discipline -- no copies when nothing resolves), anything else resolves the entry and applies it; the anchor's required grouping signal survives gap-fill by construction, and a resolved heading/list anchor keeps its narrowed type through the shared ContentParagraph return. +function resolveAnchor(styles: StylesTable | undefined, chain: readonly string[], anchor: HeadingGroupNode['node'] | ListGroupNode['node']): ContentParagraph { + const entry = entryOf(styles, chain); + if (entry === undefined) return anchor; + return applyEntry(entry, anchor); +} diff --git a/src/index.ts b/src/index.ts index b2c0eef..706fc37 100644 --- a/src/index.ts +++ b/src/index.ts @@ -16,3 +16,10 @@ export * from './text-layout'; export * from './font-port'; export * from './math-layout'; export * from './a1'; + +// --- The package boundary: the structural transform between this package's own two encodings of one document -- the flat ContentDocument every codec reads and writes, and the DocumentPackage tree a serialised artefact carries. assemblePackage is the one helper a construction site calls (decompose then factorStyles), decompose/flattenPackage are the two directions exposed for a caller composing its own boundary, and factorStyles re-mints an already-assembled tree to the identical styles table (minting is idempotent). Together they satisfy the three laws src/package.ts states: strict structural round-trip both directions for a styles-free package, effective-property equality universally (flattenPackage materialises refs away), and minting idempotence -- pinned in bijection.test.ts. +// +// Named exports rather than the `export *` every module above uses, because these four modules also carry helpers that are not meant as this barrel's public surface: decompose's per-container helpers (which factor-styles calls), factor-styles' own mint entry point, and the canonicaliser behind tuple identity. Curating the names re-exported here is a deliberate act, not a side effect of exporting a module's contents wholesale -- but per the README's "every module is also importable directly", `package.json`'s `"./*"` subpath export still makes each of those helpers reachable by importing the module directly (e.g. `document-schema.js/decompose`), so this curation narrows the index barrel, not the package's overall published surface. +export { decompose, ConstructMarkerImbalanceError, type PackageChildren } from './decompose'; +export { flattenPackage } from './flatten'; +export { assemblePackage, factorStyles } from './factor-styles'; diff --git a/src/package.ts b/src/package.ts index 681001b..d8bacad 100644 --- a/src/package.ts +++ b/src/package.ts @@ -7,7 +7,7 @@ import { DrawPageGroupSchema, SectionGroupSchema, SheetGroupSchema, SlideGroupSc // DocumentPackage is the single hierarchical artefact: structure, layout, and content fused in one tree (ExaDev/document-schema.js#20). The root carries what no tree node can -- the document kind (moved up from the retired flat `content` field; the empty documents are legal, so the kind cannot be inferred from the children and the envelope keeps it explicit), the required metadata, the optional document-level symbolTable (the same shared fields every ContentDocument arm spreads -- one declaration, spliced in from src/content.ts), and the envelope's optional tables and arrays: `pages` (each rendered page's own size, indexed to match every content node's own `frames[].pageIndex` -- present once a layout pass has run, absent for a content-only package), and the package-level table facility of src/definitions.ts -- `styles`, `definitions`, and the three construct tables `layers`/`attachments`/`destinations` added in 4.1.0. Everything structural hangs off `children`: one group per top-level container (a section, slide, sheet, or draw page), each holding its own content tree -- see src/package-node.ts for the node vocabulary and its structural discrimination rule. -// The package tree and the flat ContentDocument are one format in two encodings, related by three laws (stated on the issues and proven property-wise by document-outline.js's decompose/flatten over real corpus documents, with documents.js re-running the same assertions over its own corpus at the package boundary): (i) strict structural equality holds both directions for a table-free package -- decompose(flatten(pkg)) and flatten(decompose(pkg)) reproduce it exactly; (ii) effective-property equality holds universally -- once styles are resolved (resolve-then-compare, src/definitions.ts), a factored and an unfactored serialisation of one document compare equal; (iii) minting is idempotent -- factoring a second time mints the identical table. The codecs keep producing flat ContentDocuments (their natural reading shape); decomposition runs once at the package boundary in documents.js and flatten runs once where a builder consumes a package. +// The package tree and the flat ContentDocument are one format in two encodings, related by three laws (stated on the issues, proven property-wise by document-outline.js's decompose/flatten over real corpus documents, run here over the whole schema vocabulary in src/bijection.test.ts, and re-run by documents.js over its own real-format corpus): (i) strict structural equality holds both directions for a table-free package -- decompose(flatten(pkg)) and flatten(decompose(pkg)) reproduce it exactly; (ii) effective-property equality holds universally -- once styles are resolved (resolve-then-compare, src/definitions.ts), a factored and an unfactored serialisation of one document compare equal; (iii) minting is idempotent -- factoring a second time mints the identical table. The codecs keep producing flat ContentDocuments (their natural reading shape); decomposition runs once where a package is assembled and flatten runs once where a builder consumes one. Both directions live in this package (src/decompose.ts, src/flatten.ts, src/factor-styles.ts) so that every codec can reach them without depending on a package that depends on it. // This is a genuinely breaking shape change from the previous `{ formatVersion, content, pages }` envelope, which is why it rides a major (4.0.0). The old envelope's `formatVersion` field is gone with no replacement field: a serialised package states its version through the release-pinned $schema URI its dumper stamped (documentPackageWithSchema, src/schema-io.ts), and an ingesting documentFromJson dispatches on that URI -- the URI is the version, not a hand-kept integer. ContentDocument (the flat codec-exchange form) survives unchanged in role minus its own retired formatVersion literal, and nothing about the content model itself changed: every block, run, cell, and frame field a 3.x package carried still validates in its old flat shape -- only the envelope around it moved. diff --git a/test/smoke.test.mjs b/test/smoke.test.mjs index 1b00a60..e3aea1a 100644 --- a/test/smoke.test.mjs +++ b/test/smoke.test.mjs @@ -6,17 +6,21 @@ import { fileURLToPath } from 'node:url'; import { describe, expect, it } from 'vitest'; describe('smoke: ESM/CJS parity', () => { - it('loads the ESM build and exposes the tree-form package and node schemas', async () => { + it('loads the ESM build and exposes the tree-form package and node schemas plus the package-boundary transform', async () => { const esm = await import('../dist/index.js'); expect(typeof esm.DocumentPackageSchema.parse).toBe('function'); expect(typeof esm.PackageNodeSchema.safeParse).toBe('function'); expect(typeof esm.ConstructDescriptorSchema.safeParse).toBe('function'); expect(typeof esm.SectionConstructGroupSchema.safeParse).toBe('function'); expect(typeof esm.resolveStyleChain).toBe('function'); + for (const name of ['assemblePackage', 'decompose', 'factorStyles', 'flattenPackage']) { + expect(typeof esm[name]).toBe('function'); + } + expect(typeof esm.ConstructMarkerImbalanceError).toBe('function'); expect(esm.LayoutDocumentSchema).toBeUndefined(); }); - it('loads the CJS build and exposes the tree-form package and node schemas', () => { + it('loads the CJS build and exposes the tree-form package and node schemas plus the package-boundary transform', () => { const require = createRequire(import.meta.url); const cjs = require('../dist/index.cjs'); expect(typeof cjs.DocumentPackageSchema.parse).toBe('function'); @@ -24,6 +28,10 @@ describe('smoke: ESM/CJS parity', () => { expect(typeof cjs.ConstructDescriptorSchema.safeParse).toBe('function'); expect(typeof cjs.SectionConstructGroupSchema.safeParse).toBe('function'); expect(typeof cjs.resolveStyleChain).toBe('function'); + for (const name of ['assemblePackage', 'decompose', 'factorStyles', 'flattenPackage']) { + expect(typeof cjs[name]).toBe('function'); + } + expect(typeof cjs.ConstructMarkerImbalanceError).toBe('function'); expect(cjs.LayoutDocumentSchema).toBeUndefined(); }); }); diff --git a/test/workers/document-schema.test.ts b/test/workers/document-schema.test.ts index 0976670..01af5a8 100644 --- a/test/workers/document-schema.test.ts +++ b/test/workers/document-schema.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from 'vitest'; -import { ContentDocumentSchema, DocumentPackageSchema, findConstructMarkerImbalance, resolveStyleChain } from '../../src'; +import { assemblePackage, ContentDocumentSchema, decompose, DocumentPackageSchema, factorStyles, findConstructMarkerImbalance, flattenPackage, resolveStyleChain } from '../../src'; // Proves document-schema.js's Zod schemas and helpers parse inside a Cloudflare Workers isolate (workerd, via @cloudflare/vitest-pool-workers) with no Node-only APIs. The package is pure Zod by design -- no node:fs, no Buffer, no process -- and zod is isomorphic, so if any schema (or its zod dependency) touched a Node-only API the workerd isolate would throw rather than these passing. This is the runtime complement to the static node test suite. describe('document-schema.js under the Cloudflare Workers runtime', () => { @@ -112,4 +112,31 @@ describe('document-schema.js under the Cloudflare Workers runtime', () => { expect(findConstructMarkerImbalance(parsedBlocks)).toBeUndefined(); expect(findConstructMarkerImbalance(parsedBlocks.slice(1))).toStrictEqual({ kind: 'unmatchedEnd', index: 1 }); }); + + // The package boundary is the one part of the published surface that is behaviour rather than schema, so it needs the same runtime proof: it walks and rebuilds plain objects with no platform API at all, and running the whole assemble/mint/flatten round trip here turns that from a design claim into a workerd-executed fact. + it('runs the flat/tree transform end to end -- decompose, mint, and flatten back -- inside the isolate', () => { + const content = { + kind: 'wordprocessing', + metadata: {}, + sections: [ + { + pageSize: { widthPt: 612, heightPt: 792 }, + margins: { topPt: 0, rightPt: 0, bottomPt: 0, leftPt: 0 }, + blocks: [ + { kind: 'paragraph', headingLevel: 1, runs: [{ text: 'Chapter' }], indentLeftPt: 20 }, + { kind: 'paragraph', runs: [{ text: 'first' }], indentLeftPt: 20 }, + { kind: 'paragraph', runs: [{ text: 'second' }], indentLeftPt: 20 }, + ], + }, + ], + } as const; + const parsed = ContentDocumentSchema.parse(content); + expect(decompose(parsed)).toHaveLength(1); + const tree = assemblePackage(parsed); + expect(DocumentPackageSchema.safeParse(tree).success).toBe(true); + // Repeated indentLeftPt across the section's whole extent, so minting genuinely runs rather than short-circuiting on a styles-free tree. + expect(tree.styles).toEqual({ s1: { paragraph: { indentLeftPt: 20 } } }); + expect(factorStyles(tree).styles).toEqual(tree.styles); + expect(flattenPackage(tree)).toStrictEqual(parsed); + }); });