Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
nullptr meaning "a fresh empty one"). `context()` returns a `GraphContext&`
and is never null, so stages need no null check; `sharedContext()` hands out
the `std::shared_ptr` for composites that forward it.
- **`dsl::toDot`** — renders a parsed graph as a Graphviz DOT digraph, the same
picture as `toMermaid` with the same node ids (stages as boxes listing their
arguments, edges as ellipses, a box per dead end, slot names on the links of
a named group). `dot -Tsvg` renders it as an image, `graph-easy --as=boxart`
in a console.

### Changed
- A `.<key>` on anything but `in` and `out` (e.g. `msg.x`) is now a diagnostic;
Expand Down
4 changes: 4 additions & 0 deletions EXAMPLE.md
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,10 @@ flowchart LR
s4 --> e4
```

`dsl::toDot` renders the same picture, with the same node ids, as a Graphviz
DOT digraph. Pipe it into `dot -Tsvg -o pipeline.svg` for an image, or into
`graph-easy --as=boxart` to draw it in a console.

## 7. The same pipeline in JSON

The JSON format remains supported and uses the same registered stages. JSON
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,9 @@ drops the message, and everything downstream of that edge is skipped.
- **`validateDslGraph<In, Out>(text)`** — the same checks as construction,
returned as a list of `line:column` diagnostics instead of a thrown
**`GraphError`**.
- **`dsl::parseGraphProgram`** / **`dsl::toMermaid`** — parse a graph into its
node/edge form and render it as a Mermaid flowchart. The parser is built on
- **`dsl::parseGraphProgram`** / **`dsl::toMermaid`** / **`dsl::toDot`** —
parse a graph into its node/edge form and render it as a Mermaid flowchart or
a Graphviz DOT digraph. The parser is built on
[lexy](https://github.com/foonathan/lexy); the earlier hand-written parser is
deprecated (`dsl::parseGraphProgramHandwritten` in `GraphLangHandwritten.hpp`).

Expand Down Expand Up @@ -434,7 +435,8 @@ from `typeid(...).name()`, so how they read depends on the compiler.

`dsl::toMermaid(dsl::parseGraphProgram(text))` (or
`dsl::toMermaid(graph.program())`) renders a graph as a Mermaid flowchart,
listing each stage's arguments.
listing each stage's arguments. `dsl::toDot` renders the same picture as a
Graphviz DOT digraph, for `dot -Tsvg` or, in a console, `graph-easy --as=boxart`.

### Current limitations

Expand Down
40 changes: 38 additions & 2 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,15 @@ Examples use the generic stages of the [README](README.md) (`Parse`,
| 4 | [Injectable registry, duplicate detection](#4-injectable-registry-duplicate-detection) | low | mostly additive | unique names |
| 5 | [Documentation: 0..n outputs, large messages](#5-documentation-0n-outputs-large-messages) | doc only | — | — |
| 6 | [Known limitations to lift](#6-known-limitations-to-lift) | low–medium | additive | JSON config; read `GraphOutputs` carefully |
| 7 | [`dsl::toAscii`: console rendering](#7-dsltoascii-console-rendering) | low | additive | `toDot` into `graph-easy --as=boxart` |

Items of the same list that are done, and therefore not repeated here:
**type-checked merge inputs** (`TypedMergeFilter` / `UniformMergeFilter` /
`registerTypedMergeFilter`), **named merge slots**
(`(raw: msg, checked: valid) -> Merge`, `MergeStage::mergeInputNames()`),
**several named graph inputs** (`in.<key>`, `GraphInputs`), the
**end-of-stream hook** (`MessageFilter::finish()`), all described in the
README, and **examples for
**end-of-stream hook** (`MessageFilter::finish()`), the **Graphviz export**
(`dsl::toDot`), all described in the README, and **examples for
the newer features** — `apps/statefulPipeline` and `apps/compositePipeline`,
walked through in [EXAMPLE.md](EXAMPLE.md) sections 8 and 9.

Expand Down Expand Up @@ -178,3 +179,38 @@ work items:
behind `out.<key>` are checked when they are read (`get<T>` throws
`std::bad_any_cast`), not when the graph is built. Checking them up front
needs a way to declare the expected type per key.

## 7. `dsl::toAscii`: console rendering

**Today.** Printing a graph without an external tool is not possible; neither
diagnostics nor a debugging session can show the graph's shape as text.

**Proposal.** A native text rendering, in two steps:

1. **Text listing** (cheap, do first): one node per line with branch
characters in the style of `git log --graph`, merges showing their slots.
Readable for any DAG and easy to snapshot-test.

```text
in
└─► Parse ──► msg
├─► Validate ──► valid
└─┐
(msg, valid) ─► Summarize ──► out.stats
```

2. **2D box layout** (only if the listing is not enough): a layered layout —
columns by longest path from `in`, node order within a column chosen to
reduce crossings, edges routed on a character grid with box-drawing
characters (and a pure-ASCII fallback). Linear chains are trivial; merges,
splits and crossing edges are where the effort is.

```cpp
std::string toAscii(const GraphProgram& program);
```

**Compatibility.** Additive.

**Workaround today.** `dsl::toDot` piped into `graph-easy --as=boxart`
(Graph::Easy), or `dsl::toMermaid` into `mermaid-ascii`, which supports only a
subset of Mermaid.
148 changes: 130 additions & 18 deletions libs/filterGraph/core/filterGraph/GraphLang.hpp
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
#pragma once

#include <filterGraph/core/filterGraph/FilterRegistry.hpp>
Expand Down Expand Up @@ -49,7 +49,8 @@
// - `#` starts a comment that runs to the end of the line.
//
// This header parses (with a lexy-based parser) and structurally validates a
// program into a node/edge IR (GraphProgram) and renders it with toMermaid;
// program into a node/edge IR (GraphProgram) and renders it with toMermaid or
// toDot;
// DslFilterGraph instantiates, type-checks and runs it. The original
// hand-written parser is deprecated (GraphLangHandwritten.hpp).
namespace filterGraph::dsl {
Expand Down Expand Up @@ -1014,6 +1015,65 @@

namespace detail {

// The name each edge is shown under in a rendering: output edges (internally
// "$outN") become `out`, `out.<key>` or `out[<index>]`; every other edge keeps
// its own name.
inline std::unordered_map<std::string, std::string> edgeDisplayNames(const GraphProgram& program)
{
std::unordered_map<std::string, std::string> names;
for (const auto& output : program.outputs)
{
names[output.edge] = output.key ? "out." + *output.key
: program.outputs.size() > 1 ? std::format("out[{}]", output.index)
: std::string{"out"};
}
return names;
}

inline std::string displayName(const std::unordered_map<std::string, std::string>& names, const std::string& edge)
{
auto it = names.find(edge);
return it != names.end() ? it->second : edge;
}

// A stage's config as "key=value" lines, strings unquoted, for diagram labels.
inline std::vector<std::string> configLines(const StageNode& stage)
{
std::vector<std::string> lines;
if (stage.config.is_object())
{
for (const auto& item : stage.config.items())
{
const std::string value = item.value().is_string() ? item.value().get<std::string>() : item.value().dump();
lines.push_back(item.key() + '=' + value);
}
}
return lines;
}

// Escapes text for a quoted DOT string.
inline std::string dotLabel(std::string_view text)
{
std::string label;
for (char c : text)
{
if (c == '"' || c == '\\')
{
label += '\\';
label += c;
}
else if (c == '\n')
{
label += "\\n";
}
else
{
label += c;
}
}
return label;
}

// Escapes the characters that would end or confuse a quoted Mermaid label.
inline std::string mermaidLabel(std::string_view text)
{
Expand Down Expand Up @@ -1048,13 +1108,7 @@
// keywords such as `end`.
inline std::string toMermaid(const GraphProgram& program)
{
std::unordered_map<std::string, std::string> outputLabels;
for (const auto& output : program.outputs)
{
outputLabels[output.edge] = output.key ? "out." + *output.key
: program.outputs.size() > 1 ? std::format("out[{}]", output.index)
: std::string{"out"};
}
const auto displayNames = detail::edgeDisplayNames(program);

std::string nodes;
std::string links;
Expand All @@ -1065,10 +1119,8 @@
{
return it->second;
}
std::string id = std::format("e{}", edgeIds.size());
auto label = outputLabels.find(edge);
nodes += std::format(" {}([\"{}\"])\n", id,
detail::mermaidLabel(label != outputLabels.end() ? label->second : edge));
std::string id = std::format("e{}", edgeIds.size());
nodes += std::format(" {}([\"{}\"])\n", id, detail::mermaidLabel(detail::displayName(displayNames, edge)));
edgeIds.emplace(edge, id);
return id;
};
Expand All @@ -1079,13 +1131,9 @@
const StageNode& stage = program.stages[i];

std::string label = detail::mermaidLabel(stage.type);
if (stage.config.is_object())
for (const auto& line : detail::configLines(stage))
{
for (const auto& item : stage.config.items())
{
const std::string value = item.value().is_string() ? item.value().get<std::string>() : item.value().dump();
label += std::format("<br/>{}={}", detail::mermaidLabel(item.key()), detail::mermaidLabel(value));
}
label += "<br/>" + detail::mermaidLabel(line);
}
nodes += std::format(" s{}[\"{}\"]\n", i, label);

Expand Down Expand Up @@ -1116,4 +1164,68 @@
return "flowchart LR\n" + nodes + links;
}

// Renders a parsed program as a Graphviz DOT digraph: the same picture as
// toMermaid, with the same node ids. Stages are boxes (listing their config),
// edges are ellipses, every `-> end` gets its own double-bordered box, and the
// links of a named group carry the slot names. Render it with `dot -Tsvg`, or
// in a console with `graph-easy --as=boxart`.
inline std::string toDot(const GraphProgram& program)
{
const auto displayNames = detail::edgeDisplayNames(program);

std::string nodes;
std::string links;
std::unordered_map<std::string, std::string> edgeIds;

auto edgeId = [&](const std::string& edge) -> std::string {
if (auto it = edgeIds.find(edge); it != edgeIds.end())
{
return it->second;
}
std::string id = std::format("e{}", edgeIds.size());
nodes += std::format(" {} [shape=ellipse, label=\"{}\"];\n", id,
detail::dotLabel(detail::displayName(displayNames, edge)));
edgeIds.emplace(edge, id);
return id;
};

std::size_t deadEnds = 0;
for (std::size_t i = 0; i < program.stages.size(); ++i)
{
const StageNode& stage = program.stages[i];

std::string label = detail::dotLabel(stage.type);
for (const auto& line : detail::configLines(stage))
{
label += "\\n" + detail::dotLabel(line);
}
nodes += std::format(" s{} [shape=box, label=\"{}\"];\n", i, label);

for (std::size_t slot = 0; slot < stage.inputs.size(); ++slot)
{
const std::string& input = stage.inputs[slot];
if (stage.slotNames.empty())
{
links += std::format(" {} -> s{};\n", edgeId(input), i);
}
else
{
links += std::format(" {} -> s{} [label=\"{}\"];\n", edgeId(input), i,
detail::dotLabel(stage.slotNames[slot]));
}
}
if (stage.output)
{
links += std::format(" s{} -> {};\n", i, edgeId(*stage.output));
}
else
{
nodes += std::format(" d{} [shape=box, peripheries=2, label=\"end\"];\n", deadEnds);
links += std::format(" s{} -> d{};\n", i, deadEnds);
++deadEnds;
}
}
return "digraph filterGraph {\n rankdir=LR;\n" + nodes + links + "}\n";
}

} // namespace filterGraph::dsl
27 changes: 27 additions & 0 deletions tests/FilterGraphTests/FilterGraphTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -915,6 +915,33 @@ TEST_CASE("toMermaid generates node ids and labels outputs, config and dead ends
REQUIRE(mermaid.find('$') == std::string::npos);
}

TEST_CASE("toDot renders the same picture as toMermaid", "[GraphDsl]")
{
auto program = dsl::parseGraphProgram("in -> RunDouble -> out.twice\nin -> RunRecord(label=tap) -> end\n");
REQUIRE(program.ok());

const std::string dot = dsl::toDot(program);
REQUIRE(dot.starts_with("digraph filterGraph {\n rankdir=LR;\n"));
REQUIRE(dot.ends_with("}\n"));
REQUIRE(dot.find("e0 [shape=ellipse, label=\"in\"];") != std::string::npos);
REQUIRE(dot.find("[shape=ellipse, label=\"out.twice\"];") != std::string::npos);
REQUIRE(dot.find("s1 [shape=box, label=\"RunRecord\\nlabel=tap\"];") != std::string::npos);
REQUIRE(dot.find("d0 [shape=box, peripheries=2, label=\"end\"];") != std::string::npos);
REQUIRE(dot.find("e0 -> s0;") != std::string::npos);
REQUIRE(dot.find("s1 -> d0;") != std::string::npos);
REQUIRE(dot.find('$') == std::string::npos);
}

TEST_CASE("toDot escapes quotes and backslashes in labels", "[GraphDsl]")
{
auto program = dsl::parseGraphProgram("in -> RunRecord(label=\"say \\\"hi\\\" \\\\ bye\") -> out\n");
REQUIRE(program.ok());
REQUIRE(program.stages[0].config["label"] == "say \"hi\" \\ bye");

const std::string dot = dsl::toDot(program);
REQUIRE(dot.find("label=\"RunRecord\\nlabel=say \\\"hi\\\" \\\\ bye\"") != std::string::npos);
}

// ---------------------------------------------------------------------------
// Parser diagnostics: the hand-written and lexy parsers must agree exactly
// ---------------------------------------------------------------------------
Expand Down
13 changes: 13 additions & 0 deletions tests/FilterGraphTests/NamedSlotTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,19 @@ TEST_CASE("Inconsistent slot declarations of a stage are reported", "[NamedSlots
REQUIRE(diagnostics[0].message == "stage 'SlotBroken' declares 3 slot names for 2 slot types");
}

TEST_CASE("toDot labels a named group's links with the slot names", "[NamedSlots]")
{
const auto program = dsl::parseGraphProgram("in -> SlotPlusOne -> small\n"
"in -> SlotTimesTen -> large\n"
"(bonus: small, base: large) -> SlotDifference -> out\n");
REQUIRE(program.ok());

const std::string dot = dsl::toDot(program);
REQUIRE(dot.find("-> s2 [label=\"bonus\"];") != std::string::npos);
REQUIRE(dot.find("-> s2 [label=\"base\"];") != std::string::npos);
REQUIRE(dot.find("e0 -> s0;") != std::string::npos);
}

TEST_CASE("The parsed program and toMermaid carry the slot names", "[NamedSlots]")
{
const auto program = dsl::parseGraphProgram("in -> SlotPlusOne -> small\n"
Expand Down
Loading