Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
arguments, edges as ellipses, a box per dead end, slot names on the links of
a named group). `dot -Tsvg` renders it as an image, `graph-easy --as=boxart`
in a console.
- **`dsl::toAscii`** — prints a parsed graph as an indented text listing, with
no external tool: one tree per graph input, each line a stage followed by the
edge it produces (or `end`), fan-out as siblings, and each merge under its
(named) group once its inputs have been shown. `AsciiStyle::plain` (the
default) draws with ASCII, `AsciiStyle::unicode` with box-drawing characters.

### Changed
- A `.<key>` on anything but `in` and `out` (e.g. `msg.x`) is now a diagnostic;
Expand Down
12 changes: 12 additions & 0 deletions EXAMPLE.md
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,18 @@ flowchart LR
DOT digraph. Pipe it into `dot -Tsvg -o pipeline.svg` for an image, or into
`graph-easy --as=boxart` to draw it in a console.

Without any tool, `dsl::toAscii(pipeline.program())` prints it as a listing,
one tree per input:

```text
in
+-> Uppercase -> shouted
| `-> Print(prefix="[tap] ") -> end
`-> Reverse -> reversed
`-> MinLength(minLength=3) -> long
`-> Print(prefix="[main] ") -> out
```

## 7. The same pipeline in JSON

The JSON format remains supported and uses the same registered stages. JSON
Expand Down
23 changes: 20 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,10 @@ drops the message, and everything downstream of that edge is skipped.
- **`validateDslGraph<In, Out>(text)`** — the same checks as construction,
returned as a list of `line:column` diagnostics instead of a thrown
**`GraphError`**.
- **`dsl::parseGraphProgram`** / **`dsl::toMermaid`** / **`dsl::toDot`** —
parse a graph into its node/edge form and render it as a Mermaid flowchart or
a Graphviz DOT digraph. The parser is built on
- **`dsl::parseGraphProgram`** / **`dsl::toMermaid`** / **`dsl::toDot`** /
**`dsl::toAscii`** — parse a graph into its node/edge form and render it as a
Mermaid flowchart, a Graphviz DOT digraph or a console listing. The parser is
built on
[lexy](https://github.com/foonathan/lexy); the earlier hand-written parser is
deprecated (`dsl::parseGraphProgramHandwritten` in `GraphLangHandwritten.hpp`).

Expand Down Expand Up @@ -437,6 +438,22 @@ from `typeid(...).name()`, so how they read depends on the compiler.
`dsl::toMermaid(graph.program())`) renders a graph as a Mermaid flowchart,
listing each stage's arguments. `dsl::toDot` renders the same picture as a
Graphviz DOT digraph, for `dot -Tsvg` or, in a console, `graph-easy --as=boxart`.
`dsl::toAscii` prints it as a text listing that needs no tool at all:

```text
in
`-> Parse -> msg
+-> Validate -> valid
| `-> Summarize (merge, see below)
`-> Summarize (merge, see below)

(msg, valid)
`-> Summarize -> out.stats
```

`toAscii(program, dsl::AsciiStyle::unicode)` draws the same listing with
box-drawing characters (`├─►`, `└─►`); a Windows console shows them only with
the UTF-8 code page (`chcp 65001`).

### Current limitations

Expand Down
48 changes: 19 additions & 29 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,16 @@ Examples use the generic stages of the [README](README.md) (`Parse`,
| 4 | [Injectable registry, duplicate detection](#4-injectable-registry-duplicate-detection) | low | mostly additive | unique names |
| 5 | [Documentation: 0..n outputs, large messages](#5-documentation-0n-outputs-large-messages) | doc only | — | — |
| 6 | [Known limitations to lift](#6-known-limitations-to-lift) | low–medium | additive | JSON config; read `GraphOutputs` carefully |
| 7 | [`dsl::toAscii`: console rendering](#7-dsltoascii-console-rendering) | low | additive | `toDot` into `graph-easy --as=boxart` |
| 7 | [2D box layout for console rendering](#7-2d-box-layout-for-console-rendering) | low | additive | `toAscii` listing; `toDot` into `graph-easy --as=boxart` |

Items of the same list that are done, and therefore not repeated here:
**type-checked merge inputs** (`TypedMergeFilter` / `UniformMergeFilter` /
`registerTypedMergeFilter`), **named merge slots**
(`(raw: msg, checked: valid) -> Merge`, `MergeStage::mergeInputNames()`),
**several named graph inputs** (`in.<key>`, `GraphInputs`), the
**end-of-stream hook** (`MessageFilter::finish()`), the **Graphviz export**
(`dsl::toDot`), all described in the README, and **examples for
(`dsl::toDot`) and the **console listing** (`dsl::toAscii`), all described in
the README, and **examples for
the newer features** — `apps/statefulPipeline` and `apps/compositePipeline`,
walked through in [EXAMPLE.md](EXAMPLE.md) sections 8 and 9.

Expand Down Expand Up @@ -180,37 +181,26 @@ work items:
`std::bad_any_cast`), not when the graph is built. Checking them up front
needs a way to declare the expected type per key.

## 7. `dsl::toAscii`: console rendering
## 7. 2D box layout for console rendering

**Today.** Printing a graph without an external tool is not possible; neither
diagnostics nor a debugging session can show the graph's shape as text.
**Today.** `dsl::toAscii` prints a graph as an indented listing: one tree per
graph input, fan-out as siblings, and each merge as a reference in the trees of
its inputs plus a tree of its own under its group. It is readable for any
graph, but a merge's inputs are not drawn next to each other, so the shape of a
graph with many merges takes some reading.

**Proposal.** A native text rendering, in two steps:
**Proposal.** A drawn layout, as a second style of `toAscii` (or a sibling
function): boxes for stages, arrows for edges, laid out in layers:

1. **Text listing** (cheap, do first): one node per line with branch
characters in the style of `git log --graph`, merges showing their slots.
Readable for any DAG and easy to snapshot-test.
- columns by longest path from the inputs;
- node order within a column chosen to reduce crossings;
- edges routed on a character grid, with `AsciiStyle::unicode` box drawing and
a plain-ASCII fallback.

```text
in
└─► Parse ──► msg
├─► Validate ──► valid
└─┐
(msg, valid) ─► Summarize ──► out.stats
```

2. **2D box layout** (only if the listing is not enough): a layered layout —
columns by longest path from `in`, node order within a column chosen to
reduce crossings, edges routed on a character grid with box-drawing
characters (and a pure-ASCII fallback). Linear chains are trivial; merges,
splits and crossing edges are where the effort is.

```cpp
std::string toAscii(const GraphProgram& program);
```
Linear chains are trivial; merges, splits and crossing edges are where the
effort is. Worth doing only if the listing turns out not to be enough.

**Compatibility.** Additive.

**Workaround today.** `dsl::toDot` piped into `graph-easy --as=boxart`
(Graph::Easy), or `dsl::toMermaid` into `mermaid-ascii`, which supports only a
subset of Mermaid.
**Workaround today.** The `toAscii` listing, or `dsl::toDot` piped into
`graph-easy --as=boxart` (Graph::Easy).
183 changes: 181 additions & 2 deletions libs/filterGraph/core/filterGraph/GraphLang.hpp
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
#pragma once

#include <filterGraph/core/filterGraph/FilterRegistry.hpp>
Expand Down Expand Up @@ -49,8 +49,8 @@
// - `#` starts a comment that runs to the end of the line.
//
// This header parses (with a lexy-based parser) and structurally validates a
// program into a node/edge IR (GraphProgram) and renders it with toMermaid or
// toDot;
// program into a node/edge IR (GraphProgram) and renders it with toMermaid,
// toDot or toAscii;
// DslFilterGraph instantiates, type-checks and runs it. The original
// hand-written parser is deprecated (GraphLangHandwritten.hpp).
namespace filterGraph::dsl {
Expand Down Expand Up @@ -1228,4 +1228,183 @@
return "digraph filterGraph {\n rankdir=LR;\n" + nodes + links + "}\n";
}

// The characters toAscii draws with: plain ASCII, or Unicode box drawing
// (UTF-8, which a Windows console shows only with code page 65001).
enum class AsciiStyle
{
plain,
unicode,
};

namespace detail {

struct AsciiGlyphs
{
std::string_view branch; // a reader that has siblings below it
std::string_view last; // the last reader of an edge
std::string_view pipe; // continues a branch past a nested subtree
std::string_view blank; // indents below a last reader
std::string_view arrow; // stage -> produced edge
};

inline AsciiGlyphs asciiGlyphs(AsciiStyle style)
{
if (style == AsciiStyle::unicode)
{
// ├─► └─► │ ─►
return {"\xE2\x94\x9C\xE2\x94\x80\xE2\x96\xBA ", "\xE2\x94\x94\xE2\x94\x80\xE2\x96\xBA ", "\xE2\x94\x82 ",
" ", " \xE2\x94\x80\xE2\x96\xBA "};
}
return {"+-> ", "`-> ", "| ", " ", " -> "};
}

// A stage as written in the DSL: `Type`, or `Type(k=v, k2="s")`.
inline std::string stageCall(const StageNode& stage)
{
std::string text = stage.type;
if (stage.config.is_object() && !stage.config.empty())
{
std::string args;
for (const auto& item : stage.config.items())
{
args += (args.empty() ? "" : ", ") + item.key() + '=' + item.value().dump();
}
text += '(' + args + ')';
}
return text;
}

} // namespace detail

// Renders a parsed program as an indented text listing, for a console or a log:
// one tree per graph input, where each line is a stage reading the edge above
// it, followed by the edge it produces (or `end`). Fan-out shows as siblings.
// A merge appears in the trees of its inputs as a reference, and gets a tree of
// its own, headed by its group, once all of its inputs have been shown:
//
// in
// `-> Parse -> msg
// +-> Validate -> valid
// | `-> Summarize (merge, see below)
// `-> Summarize (merge, see below)
//
// (msg, valid)
// `-> Summarize -> out.stats
inline std::string toAscii(const GraphProgram& program, AsciiStyle style = AsciiStyle::plain)
{
const auto glyphs = detail::asciiGlyphs(style);
const auto displayNames = detail::edgeDisplayNames(program);

std::unordered_map<std::string, std::vector<std::size_t>> readers;
for (std::size_t i = 0; i < program.stages.size(); ++i)
{
for (const auto& input : program.stages[i].inputs)
{
auto& stages = readers[input];
if (stages.empty() || stages.back() != i) // `(a, a) -> Merge` reads `a` once
{
stages.push_back(i);
}
}
}

std::vector<bool> printed(program.stages.size(), false);
std::unordered_set<std::string> shown; // edges whose tree position is known
std::string text;

auto isMerge = [&](std::size_t i) { return program.stages[i].inputs.size() > 1; };

auto stageLine = [&](std::size_t i) {
const StageNode& stage = program.stages[i];
return detail::stageCall(stage) + std::string{glyphs.arrow} +
(stage.output ? detail::displayName(displayNames, *stage.output) : std::string{"end"}) + '\n';
};

auto printReaders = [&](auto&& self, const std::string& edge, const std::string& prefix) -> void {
auto it = readers.find(edge);
if (it == readers.end())
{
return;
}
for (std::size_t k = 0; k < it->second.size(); ++k)
{
const std::size_t i = it->second[k];
const bool last = k + 1 == it->second.size();
text += prefix + std::string{last ? glyphs.last : glyphs.branch};
if (printed[i] || isMerge(i))
{
text += program.stages[i].type + (printed[i] ? " (see above)\n" : " (merge, see below)\n");
continue;
}
printed[i] = true;
text += stageLine(i);
if (const auto& output = program.stages[i].output)
{
shown.insert(*output);
self(self, *output, prefix + std::string{last ? glyphs.blank : glyphs.pipe});
}
}
};

auto separate = [&] {
if (!text.empty())
{
text += '\n';
}
};

for (const auto& input : program.inputs)
{
separate();
text += detail::displayName(displayNames, input.edge) + '\n';
shown.insert(input.edge);
printReaders(printReaders, input.edge, "");
}

// Then the merges, each once all of its inputs are shown. Stages that never
// become ready (only in an invalid program, e.g. a cycle or a missing
// producer) are printed in order at the end, so every stage appears.
for (;;)
{
std::optional<std::size_t> next;
for (std::size_t i = 0; i < program.stages.size() && !next; ++i)
{
if (!printed[i] && isMerge(i) &&
std::ranges::all_of(program.stages[i].inputs, [&](const auto& edge) { return shown.contains(edge); }))
{
next = i;
}
}
for (std::size_t i = 0; i < program.stages.size() && !next; ++i)
{
if (!printed[i])
{
next = i;
}
}
if (!next)
{
break;
}

const StageNode& stage = program.stages[*next];
std::string group;
for (std::size_t slot = 0; slot < stage.inputs.size(); ++slot)
{
group += (slot == 0 ? "" : ", ") + (stage.slotNames.empty() ? "" : stage.slotNames[slot] + ": ") +
detail::displayName(displayNames, stage.inputs[slot]);
}
separate();
text += (stage.fanIn || stage.inputs.size() != 1 ? '(' + group + ')' : group) + '\n';
printed[*next] = true;
text += std::string{glyphs.last} + stageLine(*next);
if (stage.output)
{
shown.insert(*stage.output);
printReaders(printReaders, *stage.output, std::string{glyphs.blank});
}
}
return text;
}

} // namespace filterGraph::dsl
42 changes: 42 additions & 0 deletions tests/FilterGraphTests/FilterGraphTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -932,6 +932,48 @@ TEST_CASE("toDot renders the same picture as toMermaid", "[GraphDsl]")
REQUIRE(dot.find('$') == std::string::npos);
}

TEST_CASE("toAscii lists fan-out as siblings and a merge under its group", "[GraphDsl]")
{
auto program = dsl::parseGraphProgram("in -> RunDouble -> msg\n"
"msg -> RunDropOdd -> valid\n"
"(msg, valid) -> RunDescribe -> out.stats\n");
REQUIRE(program.ok());

REQUIRE(dsl::toAscii(program) == "in\n"
"`-> RunDouble -> msg\n"
" +-> RunDropOdd -> valid\n"
" | `-> RunDescribe (merge, see below)\n"
" `-> RunDescribe (merge, see below)\n"
"\n"
"(msg, valid)\n"
"`-> RunDescribe -> out.stats\n");
}

TEST_CASE("toAscii shows arguments, dead ends and outputs, in plain or Unicode characters", "[GraphDsl]")
{
auto program = dsl::parseGraphProgram("in -> RunRecord(n=3, s=\"x\") -> end\nin -> RunDouble -> out\n");
REQUIRE(program.ok());

REQUIRE(dsl::toAscii(program) == "in\n"
"+-> RunRecord(n=3, s=\"x\") -> end\n"
"`-> RunDouble -> out\n");
REQUIRE(dsl::toAscii(program, dsl::AsciiStyle::unicode) ==
"in\n"
"\xE2\x94\x9C\xE2\x94\x80\xE2\x96\xBA RunRecord(n=3, s=\"x\") \xE2\x94\x80\xE2\x96\xBA end\n"
"\xE2\x94\x94\xE2\x94\x80\xE2\x96\xBA RunDouble \xE2\x94\x80\xE2\x96\xBA out\n");
}

TEST_CASE("toAscii prints every stage of an invalid program once", "[GraphDsl]")
{
auto program = dsl::parseGraphProgram("a -> X -> b\nb -> Y -> a\n");
REQUIRE_FALSE(program.ok());

REQUIRE(dsl::toAscii(program) == "a\n"
"`-> X -> b\n"
" `-> Y -> a\n"
" `-> X (see above)\n");
}

TEST_CASE("toDot escapes quotes and backslashes in labels", "[GraphDsl]")
{
auto program = dsl::parseGraphProgram("in -> RunRecord(label=\"say \\\"hi\\\" \\\\ bye\") -> out\n");
Expand Down
Loading
Loading