cliff-python

Python parser, serializer, validator and converter for CLIFF 1.0 / 1.1 — the Contextual Localization Integrated File Format.

---

CLIFF keeps translation context — namespace, clan, group path, entry id, type, emotion, width — next to every string, in a line-oriented format that stays cheap for translators and LLMs alike. cliff-python is the reference implementation of that format for Python: one typed data model, one parser, and every converter a localization pipeline needs.

Parse	Strict mode rejects exactly what the grammar rejects; tolerant mode applies the seven documented repairs of Appendix C and reports each one.
Serialize	Canonical output that round-trips, preserving the spec version the document declared.
Validate	Required fields, closed vocabularies, status rules, duplicate ids, ICU brace balance, layout, display width, and style.
Convert	JSON, YAML, CSV, PO, XLIFF, Fluent, Android strings.xml, iOS Localizable.strings — in both directions.
Typed	Ships py.typed; the public API is annotated and checked with mypy.

The parser, serializer, validator and converters target CLIFF 1.1 and are checked against the normative examples of both 1.0 and 1.1 plus the cliff-test conformance fixtures.

Installation

pip install cliff-python

Optional converter support (YAML) is a separate extra, so the core package has no dependencies:

pip install "cliff-python[converters]"

Requires Python 3.11 or newer.

Quick start

import cliff_format

text = '''CLIFF 1.1
namespace: demo
clan: settings
source-language: en-US
target-language: zh-CN

[Video]
type: label

<Resolution>
source: "Resolution"
target: "分辨率"
status: final
'''

doc = cliff_format.parse(text)
print(doc.spec_version)              # 1.1 — the version the document declared
print(doc.header.namespace)          # demo
print(doc.groups[0].path)            # Video
print(doc.groups[0].entries[0].id)   # Resolution

print(cliff_format.serialize(doc))   # canonical CLIFF

for issue in cliff_format.validate(text):
    print(issue.line, issue.category, issue.message)

Strict and tolerant parsing

CLIFF 1.1 defines two readings of the same syntax, and cliff_format exposes both. Strict is the default; the mode is never chosen for you.

Strict (parse, load) rejects everything the grammar rejects, and accepts everything it accepts:

• identifiers may use A-Z a-z 0-9 _ - and never ., are case-sensitive, and are never rewritten — bad_id and BadVersion stay as written, because an identifier is a translation match key;
• tags (type, emotion, status, variant) are the one thing 1.1 did not relax: lowercase words from a closed vocabulary;
• one optional trailing , or ; per line is standard syntax, means nothing, and is never re-emitted by the serializer;
• a file-layout mismatch is a warning — the layout is a recommendation;
• resource limits are enforced.

Tolerant (parse(..., tolerant=True), parse_tolerant) adds the seven relaxations of Appendix C and reports every repair:

Relaxation	Input	Result
C.2.1 bare scalar in a list-typed field	emotion: objective	["objective"]
C.2.2 repeated field in one scope	two target: lines	concatenated / appended in order
C.2.3 quoted tag	status: "final"	final
C.2.4 quoted entry id	<"resolution">	resolution
C.2.5 identifier with a reserved character	<Save & Load>	Save-Load (+ collision handling)
C.2.6 version-line spelling	cliff 1.1.0	CLIFF 1.1
C.2.7 quoted key	"context": "…"	context: "…"

C.2.7 removes the quotes and nothing else: the enclosed text must be a name, no escape is processed, and the key is then checked against the legal keys of its scope exactly as a bare key is. A quoted word that is not a legal key, or a key quoted in a scope that does not allow it, is still an error (§C.5) — the relaxation cannot legalize a word.

C.2.5 applies to an identifier that begins with a name character (ignoring surrounding whitespace), or that uses the quoted form of C.2.4. Angle brackets that enclose anything else are not an entry marker: a stray closing tag </terms> — what a model writes when it treats a section like an XML element — or <.hidden> is rejected under C.5 rather than normalized. Without that limit the reading turned /terms into the entry terms and then failed the document on a required field of an entry the file never contained, which is the guessing C.5 forbids. An identifier that is empty or whitespace-only is unchanged: C.3 step 7 still gives the scope's fallback name.

Every repair is recorded, so nothing is fixed silently:

doc, corrections = cliff_format.parse_tolerant(text)
for c in corrections:
    print(c.line, c.category, c.message, c.before, "->", c.after)

Categories: list-shape, field-repeat, tag-quote, name-quote, name-normalized, id-collision, version, casing. A category names the operation, not the clause, so name-quote covers both a quoted entry id (C.2.4) and a quoted key (C.2.7).

The mode never guesses. Appendix C.5 lists what a tolerant parser MUST refuse — a missing required field, a word outside a closed vocabulary, an unbalanced ICU brace, a malformed line, an unsupported version — and those raise CliffParseError with a line number and a category, exactly as in strict mode.

Two guarantees make the mode usable in a pipeline:

# 1. the repaired document is valid under the strict grammar
once = cliff_format.serialize(doc)
assert not [i for i in cliff_format.validate(once) if i.category not in ("warning", "extension")]

# 2. canonical output is a fixed point
assert cliff_format.serialize(cliff_format.parse(once)) == once

Command line

cliff-format parse path/to/file.cliff
cliff-format serialize path/to/file.cliff
cliff-format validate path/to/file.cliff
cliff-format convert path/to/file.cliff --format po
cliff-format convert path/to/file.po --from po --format cliff

# Tolerant reading of a model's output, with the repair report on stderr
cliff-format parse path/to/model-output.cliff --tolerant
cliff-format validate path/to/model-output.cliff --tolerant
cliff-format convert path/to/model-output.cliff --tolerant --format json

# Opt-in checks: a layout mismatch as an error, style deviations as warnings
cliff-format validate path/to/file.cliff --check-layout
cliff-format validate path/to/file.cliff --style
cliff-format validate path/to/file.cliff --check-width

# Canonical output for a specific version, or with a canonical id style applied
cliff-format serialize path/to/file.cliff --spec-version 1.0
cliff-format serialize path/to/file.cliff --style

Supported convert formats: cliff, json, yaml, csv, po, xliff, fluent, android, ios.

validate exits 1 when the file has errors and 0 when it only has warnings, extension notes, corrections, or style findings. Every diagnostic carries a line number, a category, and the offending line.

Converters

Bidirectional for JSON, YAML, CSV, PO, XLIFF 2.x, Fluent, Android strings.xml and iOS Localizable.strings:

po_text = cliff_format.to_po(doc)
doc = cliff_format.from_po(po_text)

# XLIFF 2.0 / 2.1 / 2.2; 2.2 also writes the glossary module for variant: glossary
xml = cliff_format.to_xliff(doc, version="2.2")

CLIFF carries context as first-class data, and every converter moves that data through the target format's own documented metadata channel — never through an invented one:

Format	Channel used for CLIFF attributes	Channel used for context
JSON / YAML / CSV	native fields of the CLIFF-shaped document	native field
XLIFF 2.x	metadata module (mda:metadata / mda:metaGroup category="cliff" / mda:meta), plus state on segment for status	mda:meta type="context" and a plain note
XLIFF 2.2 glossary	additionally the glossary module (gls:glossEntry) for variant: glossary	gls:definition
gettext PO	extracted comments #. cliff:<key>: <value>; #: for references; msgctxt for the group path and entry id	plain #. comment
Fluent	message comment # cliff:<key> = <value> (attributes are translatable content, so they are not used for metadata)	plain # comment
Android strings.xml	XML comment above the resource, cliff:<key>: <value>	first line of that comment
iOS Localizable.strings	/* ... */ block comment above the pair, cliff:<key>: <value>	first line of that comment

Because formats without a group level cannot express inheritance, the exporters fold the group values into each entry (the effective context, type, emotion and max-width) so nothing is lost.

The reverse direction is deliberately conservative. An importer reads only the metadata the source format actually defines: namespaced cliff: entries become CLIFF attributes, an ordinary comment becomes context, and everything else falls back to a documented default (type: sentence, status derived from whether a translation exists). A converter never invents context, emotion, references, widths or reviewers that the source file did not contain.

Round-trip fidelity

JSON, YAML, CSV and XLIFF preserve the whole CLIFF data model, so cliff -> format -> cliff is lossless. The remaining formats have no slot for some CLIFF fields:

Format	Fidelity	Dropped on the way back
JSON / YAML / CSV	yes	—
XLIFF 2.0 / 2.1 / 2.2	yes	— (CLIFF-only fields travel as cliff:-categorised notes)
PO	entry level	header prose (variant, version, title, info, standard, dependency) and the group structure
Fluent	entry level	header prose and the group structure; Fluent attributes of foreign files cannot be represented
Android strings.xml	entry level	header prose and the group structure
iOS Localizable.strings	entry level	header prose and the group structure

Importers keep foreign identifiers that are already valid CLIFF 1.1 names — InvSwordIron and bad_id are valid names, and rewriting one silently would break a translation memory. Only characters outside the name set are replaced, by the documented normalization of specification Appendix C.3, and duplicates are disambiguated with a numeric suffix so every generated document validates.

API

Function	Description
cliff_format.parse(text, path=None, tolerant=False, corrections=None, **limits)	Parse CLIFF text into CliffDocument.
cliff_format.parse_tolerant(text, path=None)	Parse in tolerant mode; returns (document, corrections).
cliff_format.load(path, tolerant=False)	Read and parse a .cliff file.
cliff_format.serialize(doc, style=False)	Serialize a CliffDocument to canonical CLIFF text.
cliff_format.validate(text, check_width=False, check_layout=False, style=False, tolerant=False)	Validate CLIFF text and return list[ValidationIssue].
cliff_format.validate_document(doc, ...)	Validate an already parsed document.
cliff_format.effective_context / effective_type / effective_emotion / effective_max_width	Resolve a group-inherited value for one entry.
cliff_format.is_name / is_tag / is_style_identifier / normalize_identifier / strip_line_terminator	The identifier rules of §5.5 and Appendix C.3, and the terminator rule of §5.6.
cliff_format.to_dict / from_dict	Convert between CliffDocument and a JSON-shaped dict.
cliff_format.to_json / from_json	Convert between CliffDocument and JSON text.

Diagnostics

Category	Meaning
syntax, semantic, vocabulary, icu, id	Errors: the file does not conform.
warning	A recommendation (layout mismatch, glossary shape, width overflow).
extension	An x- field was ignored, as the specification requires.
correction	A tolerant parse repaired something; reports before → after.
style	A deviation from the style guide (--style); never an error.

Resource limits

parse and load accept max_lines, max_line_length and max_string_length; the defaults bound a document at 1 000 000 lines, 1 MiB per line and 1 MiB per decoded string. Exceeding a limit raises a located error instead of exhausting memory. The limits apply in strict and tolerant mode alike (§19).

Repository layout

cliff-python/
├── pyproject.toml
├── src/
│   └── cliff_format/
│       ├── identifiers.py   # name / tag / terminator rules (§5.5, §5.6, Appendix C.3)
│       ├── vocabulary.py    # the closed tag vocabularies (§12, §13)
│       ├── model.py         # dataclasses for the CLIFF data model
│       ├── parser.py        # single-pass line parser, strict and tolerant
│       ├── serializer.py    # canonical serializer
│       ├── validator.py     # semantic validation, inheritance helpers, style and layout checks
│       ├── converter.py     # bidirectional format converters
│       └── cli.py           # cliff-format command line
├── tools/
│   └── regenerate_examples.py
├── examples/
└── tests/

Conversion samples live in examples/cliff_to_<format>/ and examples/<format>_to_cliff/. The generated side is produced by tools/regenerate_examples.py; run it after changing a converter, and python tools/regenerate_examples.py --check verifies the samples are current.

Links

• CLIFF specification
• Conformance fixtures
• Changelog
• Issues

License

MIT — see LICENSE. The CLIFF wordmark under docs/assets/ comes from the CLIFF project and is licensed under the MIT License as well.