Query Engine

The fourth of ArcFlow's eight layers. Owns the parser, planner, and executor that turn declarative Cypher / GQL into work against the World Graph, the Perception Lake, and the World Store underneath.

The Query Engine is what makes the storage layers feel like a single graph. An agent writes one Cypher pattern; the planner decides which parts of it become in-memory traversal, which parts become columnar scans, and how to combine the results — none of which is visible at the query surface.

What the engine provides#

Why "one query, two storage shapes" works#

The boundary between the Lake and the Graph is invisible at the query surface. A pattern like:

MATCH (p:Player {team: 'Alpha'})-[:OBSERVED_IN]->(f:Frame)
WHERE f.speed > 5.0
RETURN p.name, count(f) AS observations

— mixes a Graph-resident class (Player) with a Lake-resident class (Frame). The planner reads the catalog, decides that (:Player) is an in-memory probe against the Graph's label index, and that (f:Frame) WHERE f.speed > 5.0 is a columnar scan with speed > 5.0 pushed down to the storage layer. The traversal edge [:OBSERVED_IN] runs against the Graph's CSR adjacency. Three execution shapes, one query.

This is the load-bearing property of the architecture: agents do not pick "the graph engine" or "the analytics engine." They write graph queries, and the planner picks the right execution shape per pattern.

What the engine does NOT do#

• It does not own storage. Bytes live in the World Store; observation discipline lives in the Perception Lake; typed entities live in the World Graph.
• It does not own change notification. That is the Live Surface.
• It does not own delivery semantics. That is the Event Bus.
• It does not own behavior firing. That is the Behavior Engine.

The Query Engine is read-modify-write at the query level — Cypher in, results out, mutations applied. Everything change-driven is downstream.

Conformance + dialect#

ArcFlow targets the ratified ISO/IEC 39075 GQL standard while keeping openCypher source compatibility. Existing Neo4j-shaped Cypher runs unchanged for the conforming subset. ArcFlow's additions sit cleanly inside both grammars — see GQL Conformance and WorldCypher for the binding contract.

Query hints — explicit lane selection#

The planner picks a compute lane automatically for each algorithm call. A HINT clause overrides that choice. Supported lanes: auto, cpu, cuda, metal, gpu.cuda, gpu.metal.

CALL algo.pageRank() HINT lane=gpu.cuda YIELD nodeId, score
CALL algo.triangleCount() HINT lane=cpu YIELD count

If the requested lane is unavailable on the host (e.g. gpu.cuda on Apple Silicon), the engine falls back silently to the next-best lane and records the actual lane used. The chosen lane comes back on the result envelope as result.transport_outcome.lane — use it to detect silent fallbacks in tests, or to log which path each query actually took.

Telemetry — result.io_stats#

Every result envelope carries an io_stats field with pruning and I/O telemetry. Use it to verify predicate pushdown is firing and to tune lakehouse layouts.

pruning_efficiency near 1.0 means the planner skipped almost all row groups via statistics. A value near 0 means scans walked through most of the data — usually a sign that the query lacks selectivity on the columns the lakehouse layout was sorted by.

See also#

• WorldCypher — the surface this engine consumes.
• Graph Patterns — how to read a Cypher pattern.
• Query Results — typed columns; numbers stay numbers.
• EXPLAIN — inspect the plan the engine chose.