Introduction

Bring embedded graph queries to your data lake. Learn to map relational tables, query with three graph languages, and use OrchidDB from Rust, Java, Python, JavaScript/TypeScript, Elixir, or C++.

Start with your data

OrchidDB is a graph query layer built in Rust, with DataFusion planning and Apache Arrow results. Compile SQL and execute through your own engine, or use the standalone CLI with bundled DuckDB and default Iceberg support. Describe nodes, relationships, and properties over your existing tables, then ask graph questions in Cypher, Gremlin, or SPARQL.

A graph mapping connects your application vocabulary to physical columns. A customer can become a Person, an order row can define an ORDERED relationship, and a SQL view can expose a selected group of accounts. Your database remains the source of those rows.

Choose a starting point

Version 0.1.0 is available as a CLI installer and published language packages. On macOS ARM64, install the CLI and run the quickstart without a Rust toolchain or source checkout:

curl -fsSL https://install.orchiddb.com | bash
export PATH="$HOME/.local/bin:$PATH"
orchiddb --version

Three languages, one graph

Cypher describes patterns between nodes:

MATCH (p:Person)-[:FOLLOWS]->(friend:Person)
WHERE p.name = 'alice'
RETURN friend.name
ORDER BY friend.name

Gremlin describes a traversal through the same relationships:

g.V().hasLabel('Person').has('name', 'alice')
  .out('FOLLOWS').values('name').order()

SPARQL uses RDF rules over application columns or an ontology mapping over graph labels:

PREFIX ex: <https://example.com/>
SELECT ?name WHERE {
  ?person a ex:Person ; ex:name "alice" ; ex:follows ?friend .
  ?friend ex:name ?name .
}
ORDER BY ?name

The language frontends produce a shared Graph IR and relational plan. Execution uses SQL regions and native operators as supported by the selected API. Read SQL compilation to inspect or execute generated SQL, and core concepts for the shared execution model.

Follow a learning path

Begin with installation, run the quickstart, then choose a client example. The mapped graph tutorial demonstrates GraphEngine using a customer/order dataset.

For application integration, continue with typed parameters, Arrow results, and transactions. The CLI reference, mapping reference, and Rust API provide the details to keep nearby while coding.

Project links

Website · GitHub · Community

Installation

Use the CLI for DuckDB and Iceberg, or embed a language client around your own engine. Version 0.1.0 is published. Packaged native binaries currently target macOS ARM64; source builds are also available.

Binary installer

Install the released CLI with bundled DuckDB. No Rust toolchain or separate database installation is needed:

curl -fsSL https://install.orchiddb.com | bash
export PATH="$HOME/.local/bin:$PATH"
orchiddb --version

The installer downloads from OrchidDB-cli releases, verifies SHA-256 checksums, and installs the executable and its bundled DuckDB library into ~/.local/bin. It never runs sudo or edits your shell configuration. The export above updates the current shell; add it to your shell configuration to keep the command available in new terminals.

Version 0.1.0 provides a macOS ARM64 (Apple Silicon) binary. Other platforms require a source build. The macOS binary is ad-hoc signed, not notarized.

After installation, run ~/.local/bin/orchiddb --version. Add ~/.local/bin to your PATH to use orchiddb directly. Direct download.

curl -fsSL https://install.orchiddb.com | ORCHIDDB_VERSION=v0.1.0 ORCHIDDB_INSTALL_DIR="$HOME/.local/bin" bash

Without a version, the installer selects the most recently created published release, including prereleases. Review the installer before running it. Checksums come from the same release publisher as the archive.

Run the installer again to upgrade, or set ORCHIDDB_VERSION to select a specific release. Keep the installed lib/libduckdb.dylib alongside the executable on macOS. Continue with the quickstart to run a query without cloning a repository.

Published packages

The following 0.1.0 releases are available. The CLI includes DuckDB 1.5.2; embedded clients use an application-owned database driver.

Packaged native compilers in this release target macOS ARM64. The Python wheel requires macOS 26 or newer and Python 3.10+; the Node.js client requires Node.js 20+; Java requires Java 17+ and an ARM64 JVM. Rust compiles from source through Cargo. See client setup for driver dependencies, native library setup, and runnable examples.

From source

The standalone CLI lives in OrchidDB-cli:

git clone https://github.com/OrchidDB/OrchidDB-cli.git
cd OrchidDB-cli
cargo install --locked --path . --bin orchiddb
orchiddb query examples/people.json --init examples/setup.sql --format table

Use Rust with edition 2024 support, Git, and a C/C++ build toolchain. The first build compiles bundled DuckDB. Add Cargo’s executable directory (usually $HOME/.cargo/bin) to PATH.

Queries load the official Iceberg extension by default. First use requires network access to install the extension. Your setup SQL configures catalogs, credentials, views, plugins, and UDFs. See the quickstart and CLI reference.

Language clients

Start with client setup and runnable examples. These clients compile SQL without a DuckDB driver dependency. Your application supplies the engine. For Java, add the API and macOS ARM64 compiler JAR dependencies from Maven Central, then call NativeSqlCompiler.load(). Maven supplies the compiler automatically: no manual native binary download or Rust build is required. See Java installation for the complete dependency block and Arrow setup. The Java 0.1.0 compiler package supports macOS ARM64 JVMs only. The shared compiler instructions below apply to source builds of the other bindings.

Shared native compiler

Python, Node.js, Elixir, and C++ source builds need OrchidDB-native. Check out the revision in your client’s NATIVE_REVISION, then the compiler’s matching core revision. Run this from the client repository root (requires new sibling checkout directories):

git clone https://github.com/OrchidDB/OrchidDB-native.git ../orchiddb-native
git -C ../orchiddb-native checkout "$(cat NATIVE_REVISION)"
git clone https://github.com/OrchidDB/OrchidDB.git ../orchiddb
git -C ../orchiddb checkout "$(cat ../orchiddb-native/CORE_REVISION)"
cargo build --locked --release --manifest-path ../orchiddb-native/Cargo.toml

Set the path for the current shell, then run your client’s example:

# macOS; use liborchiddb_compiler.so on Linux
export ORCHIDDB_NATIVE_LIBRARY="$(cd ../orchiddb-native/target/release && pwd)/liborchiddb_compiler.dylib"

The library compiles metadata and query text only; result data never crosses this boundary. Release workflows bundle it into Python wheels, npm packages, and C++ archives. Elixir loads it explicitly. No binding downloads a compiler implicitly at runtime.

Core compiler library

Install the published core compiler directly from crates.io:

[dependencies]
orchiddb = { version = "=0.1.0", default-features = false }

See SQL compilation. For application integration, prefer the Rust client.

Rust GraphEngine

The optional GraphEngine runtime supports managed storage and mapped application tables through one engine, including RDF reads and updates. The API tutorials describe the current core checkout. New RDF catalog, composite-key, physical-layout, and collection-source APIs require a matching source build; the published 0.1.0 binaries may bundle an earlier core revision. Their dependencies use an explicit duckdb feature:

[dependencies]
orchiddb = { path = "../orchiddb", features = ["duckdb"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
arrow = "58.2.0"
duckdb = { version = "1.10502.0", features = ["bundled"] }

Run its separate example from the core checkout:

cargo run --locked --features duckdb --example managed_graph

The standalone CLI instructions above refer to OrchidDB-cli, not the legacy core binary.

Quickstart

Query existing tables through a graph mapping. The standalone CLI bundles DuckDB and loads Iceberg by default; no graph store is created.

Install the CLI

curl -fsSL https://install.orchiddb.com | bash
export PATH="$HOME/.local/bin:$PATH"
orchiddb --version

The published v0.1.0 binary supports macOS ARM64 (Apple Silicon) and includes DuckDB. No Rust toolchain or source checkout is needed. See installation for version pinning, other platforms, and source builds.

Create the input files

In a working directory, save this as setup.sql:

CREATE TABLE people(id BIGINT, name VARCHAR);
INSERT INTO people VALUES (1, 'Ada'), (2, 'Grace');

Save this as people.json, describing the table schema and its graph mapping:

{
  "version": 1,
  "dialect": "duckdb",
  "language": "cypher",
  "query": "MATCH (p:Person) RETURN p.name AS name ORDER BY name",
  "tables": [{"name": "people", "columns": [
    {"name": "id", "data_type": "int64", "nullable": false},
    {"name": "name", "data_type": "string", "nullable": true}
  ]}],
  "nodes": [{"label": "Person", "table": "people", "id": "id",
    "properties": {"name": "name"}}]
}

Execute the query

orchiddb query people.json --init setup.sql --format table --no-iceberg

The result contains Ada and Grace. This table-only example uses --no-iceberg to skip extension setup. By default, the CLI loads the official Iceberg extension; its first installation requires network access.

Inspect SQL or export Arrow

orchiddb compile people.json
orchiddb query people.json --init setup.sql --no-iceberg > people.arrow

Compilation does not open a database. Query output defaults to an Arrow IPC stream, so use --format table for terminal output. Each invocation above uses a fresh in-memory database.

Use your data lake

Replace the example setup with your own catalog configuration and a view over an Iceberg table:

CREATE VIEW people AS
SELECT id, name FROM iceberg_scan('/path/to/table/metadata/v1.metadata.json');

Keep the graph mapping aligned with the view’s real schema, and omit --no-iceberg when querying it. The compiler sees the metadata; DuckDB accesses the source data.

Embed in your application

Choose Rust, Java, Python, JavaScript/TypeScript, Elixir, or C++. These clients compile the same graph-language requests but leave engine configuration and execution to your application. Results use Arrow batches.

For persisted graph records, see managed graphs. For reads and writes over existing tables through the shared engine, see mapped graphs.

Core concepts

Understand the graph model, the mapping contract, and how a graph query reaches a relational engine.

Nodes and relationships

A node has an identity, labels, and properties. A relationship connects a source node to a destination node and has a type and properties. A label such as Person identifies a group of nodes; a type such as FOLLOWS identifies a group of relationships.

In a mapped graph, these concepts come from your schema. A node’s identity can be a table’s primary key. Relationship endpoints can be foreign-key columns. Properties can use graph names different from their physical column names.

Query execution

OrchidDB uses one mapping catalog and shared language planning. The language clients execute queries through application-provided database sessions. The SQL API exposes generated SQL when applications need it directly. GraphEngine owns a DuckDB connection and executes the shared relational DAG, with either managed or mapped storage.

Managed and mapped data

Mapped engines use the schemas registered in their mappings to plan queries, while DuckDB reads the source rows. Registering a schema describes the table to the planner.

Graph IR

Graph IR (graph intermediate representation) is OrchidDB’s shared logical query plan for Cypher, Gremlin, and SPARQL. It describes what a query should do using graph operations such as node scans, relationship expansion, filtering, projection, aggregation, and path traversal. It represents the query, not the stored graph data.

Each language frontend parses and validates a query, then translates it into a tree of these operations. Operators introduce bindings for nodes, relationships, and values; expressions refer to those bindings to read properties, compare values, or compute results. The plan also carries rules for missing values, matching, and paths so that sharing a representation preserves each language’s behavior.

For example:

MATCH (p:Person)-[:FOLLOWS]->(q:Person)
WHERE p.name = 'Ada'
RETURN q.name

Conceptually, this becomes a scan of Person nodes, a filter for p.name, an outgoing FOLLOWS expansion to another Person, and a projection of q.name. This is a simplified description, not literal explain output. Relational lowering uses the graph mappings to turn scans into table reads, a single-hop expansion into joins on endpoint identities, and property access into column expressions.

Graph IR gives the languages a common planning layer, so relational lowering and SQL generation can be shared instead of implemented separately for every language and SQL dialect. Keeping graph operations and their rules explicit also lets later stages preserve graph identity, duplicate results, and path constraints when translating them into relational operations.

Graph IR is lowered before execution. The SQL API returns SQL for supported reads and rejects queries that cannot be fully lowered. GraphEngine executes the lowered DataFusion plan with eligible SQL regions in DuckDB and remaining native kernels. Graph IR itself is not interpreted at runtime. The execution guide shows how to inspect it.

SQL islands

A SQL island is a region of the lowered relational plan that executes as SQL in DuckDB and returns Arrow data. DataFusion schedules the managed execution DAG and its remaining native kernels. Graph IR is a compiler representation; it is not interpreted at runtime.

Query text
    ↓
Language parser and planner
    ↓
Graph IR
    ↓
Relational lowering
    ↓
DataFusion execution DAG + eligible DuckDB SQL regions
    ↓
Arrow results

Vocabulary and storage

An ontology mapping associates RDF class and predicate IRIs with graph labels, properties, and relationship types. A graph mapping then associates those concepts with physical tables and columns. This lets SPARQL use the same property graph as Cypher and Gremlin.

Alternatively, add RdfMapping rules directly to the same GraphMapping to expose application columns and composite keys as RDF terms. An ontology is optional. Existing quad tables remain supported through RdfDatasetMapping. Both use the same engine and transaction owner; see RDF over application tables.

Identity and ordering

Choose stable, non-null node IDs. For mapped relationships, source and destination values refer to IDs of the configured endpoint labels. Provide a relationship ID when individual edges need an explicit identity.

Request order in the query whenever the consumer needs it: use Cypher ORDER BY, Gremlin order(), or SPARQL ORDER BY. Explicit order also makes examples and paginated application output reproducible.

Map existing tables

Expose existing DuckDB tables as a graph, then query them with Cypher and Gremlin.

The relational dataset

This tutorial models three people, their orders, and a social relationship. The users table supplies Person nodes. The orders table supplies both Order nodes and ORDERED relationships. The follows table supplies FOLLOWS relationships.

CREATE TABLE users (id BIGINT, name VARCHAR, age BIGINT);
INSERT INTO users VALUES
  (1, 'alice', 30), (2, 'bob', 28), (3, 'carol', 41);
CREATE TABLE orders (order_id BIGINT, user_id BIGINT, total DOUBLE);
INSERT INTO orders VALUES
  (100, 1, 50.0), (101, 1, 120.0), (102, 2, 80.0), (103, 3, 500.0);
CREATE TABLE follows (src BIGINT, dst BIGINT);
INSERT INTO follows VALUES (1, 2), (1, 3), (2, 3);

Save this as schema.sql beside your application’s Cargo.toml, or download the sample schema.

Register schemas and mappings

Use the dependencies from installation. This complete src/main.rs sets up the tutorial database and queries it:

use std::sync::Arc;
use arrow::datatypes::{DataType, Field, Schema};
use orchiddb::ir::rel::mapping::{EdgeMapping, ForeignKeyEndpoint, GraphMapping, NodeMapping};
use orchiddb::engine::GraphEngine;

#[tokio::main]
async fn main() -> Result<(), String> {
    let mut mapping = GraphMapping::new();
    mapping.register_table_schema("users", Arc::new(Schema::new(vec![
        Field::new("id", DataType::Int64, false),
        Field::new("name", DataType::Utf8, true),
        Field::new("age", DataType::Int64, true),
    ])));
    mapping.register_table_schema("orders", Arc::new(Schema::new(vec![
        Field::new("order_id", DataType::Int64, false),
        Field::new("user_id", DataType::Int64, false),
        Field::new("total", DataType::Float64, true),
    ])));
    mapping.register_table_schema("follows", Arc::new(Schema::new(vec![
        Field::new("src", DataType::Int64, false),
        Field::new("dst", DataType::Int64, false),
    ])));
    mapping.map_node(NodeMapping::table("Person", "users", "id")
        .property("id", "id").property("name", "name").property("age", "age"));
    mapping.map_node(NodeMapping::table("Order", "orders", "order_id")
        .property("total", "total"));
    mapping.map_edge(EdgeMapping::table(
        "ORDERED", "orders", "user_id", "order_id", "Person", "Order"
    ).foreign_key(ForeignKeyEndpoint::Destination));
    mapping.map_edge(EdgeMapping::table(
        "FOLLOWS", "follows", "src", "dst", "Person", "Person"
    ));

    let connection = duckdb::Connection::open_in_memory().map_err(|e| e.to_string())?;
    connection.execute_batch(include_str!("../schema.sql")).map_err(|e| e.to_string())?;
    let mut graph = GraphEngine::mapped(connection, Arc::new(mapping))?;
    let result = graph.cypher(
        "MATCH (p:Person)-[:ORDERED]->(o:Order) WHERE o.total > 100.0 \
         RETURN p.name, o.total ORDER BY o.total"
    ).await?;
    for row in 0..result.returned.batch.num_rows() {
        let values = result.returned.batch.columns().iter().map(|column|
            arrow::util::display::array_value_to_string(column, row)
                .map_err(|error| error.to_string())
        ).collect::<Result<Vec<_>, _>>()?;
        println!("{}", values.join(" | "));
    }
    Ok(())
}

Expected rows:

alice | 120.0
carol | 500.0

Download the complete Rust example to use with the sample schema.

Open an existing database

For a database that already contains these tables, open its connection and pass it to the engine. Register the same schemas and mappings, then query immediately:

let connection = duckdb::Connection::open("warehouse.duckdb")
    .map_err(|error| error.to_string())?;
let mut graph = GraphEngine::mapped(connection, Arc::new(mapping))?;

The mapping describes the source schema; the engine owns the connection to its rows. Use matching table names and Arrow types on both sides.

Traverse with Gremlin

With the same mapped engine:

let friends = graph.gremlin(
    "g.V().hasLabel('Person').has('name','alice').out('FOLLOWS').values('name')"
).await?;

Alice follows bob and carol.

Write through the mapping

Use cypher and gremlin to insert, update, and delete rows in the same mapped tables. Node labels select node tables; relationship types select relationship tables and endpoint columns. See mapped writes for examples, identity rules, and transaction behavior.

Share the engine with RDF

Add RdfMapping rules to the same GraphMapping before constructing the engine. Use sparql_query for typed RDF results or sparql_dataset for Arrow results and statistics. All languages share the connection and transaction methods. See RDF over application tables. MappedGraphEngine remains a compatibility facade; new applications use GraphEngine::mapped.

Mapping reference

Define graph labels, relationship types, identities, and properties with Rust builders or TOML.

Node mappings

NodeMapping::table(label, table, id_column) maps a label to a table or view. Add properties with .property(graph_name, column_name).

NodeMapping::table("Person", "customers", "customer_id")
    .property("name", "full_name")
    .property("age", "age")

Use stable IDs within each label. The graph label and identity together distinguish elements. Property names are the vocabulary used in queries; column names belong to your physical schema.

Relationship mappings

The edge constructor takes the relationship type, source table, source ID column, destination ID column, source label, and destination label:

EdgeMapping::table(
    "ORDERED", "orders", "customer_id", "order_id", "Person", "Order"
)
.foreign_key(ForeignKeyEndpoint::Destination)

One physical table can supply both nodes and relationships. In this example, an order is a node, and its customer_id also defines a relationship from a person to that order.

Use a join table for a many-to-many relationship:

EdgeMapping::table(
    "FOLLOWS", "follows", "src", "dst", "Person", "Person"
)

JSON documents and collections

Register document columns with compiler type json or the Rust Arrow datatype ir::functions::domain::json_type(). JSON strings remain distinct from ordinary text. Collection descriptors can use expand expressions such as json.elements(profile, '$.friends'), field expressions, optional outer expansion, and ordinality. The resulting relation can supply ordinary nodes or relationships. See JSON documents and relationships for a complete mapping, typed extraction, and containment relationships.

Computed relationships and search

Define a relationship from source and target labels, scalar property expressions, eligibility predicates, and per-source ranking. Cypher and Gremlin traverse these read-only relationships with the same syntax as stored edges. The planner maps logical scoring functions to native execution or the owning SQL engine.

The Search guide covers vector similarity, BM25, indexed candidates, MaxSim reranking, and PostgreSQL/pgvector and duckdb-lance index bindings.

Register source schemas

GraphMapping::register_table_schema accepts an Arrow SchemaRef. Describe physical column names, data types, and nullability. Use register_table when you already have a DataFusion table provider.

TOML format

[node.Person]
table = "users"
id = "id"

[node.Person.properties]
name = "name"
age = "age"

[node.Order]
table = "orders"
id = "order_id"

[node.Order.properties]
total = "total"

[edge.ORDERED]
table = "orders"
src = "user_id"
dst = "order_id"
src_label = "Person"
dst_label = "Order"
foreign_key = "dst"

Load it with GraphMapping::from_toml(text) and serialize with mapping.to_toml(). Register the physical table schemas after loading. TOML contains the mapping definition; table providers are registered by the application.

For a query-backed source, use a query key in place of table:

[node.Adult]
query = "SELECT id, name, age FROM users WHERE age >= 18"
id = "id"

[node.Adult.properties]
name = "name"
age = "age"

Design a mapping

Choose names that match your application domain. Keep node IDs stable, match endpoint column types to their node ID types, and expose the properties that queries should use. Use .with_id(...) for table-backed relationships that participate in property updates.

For SQL-defined labels and reusable physical views, continue to views and SQL sources.

Writable one-to-many relationships

Use .foreign_key(ForeignKeyEndpoint::Destination) when the destination node’s row holds the reference to the source node. Use Source for the reverse graph direction. This is explicit: an ordinary table-backed edge continues to own its own row.

use orchiddb::ir::rel::mapping::{EdgeMapping, ForeignKeyEndpoint, NodeMapping};

mapping.map_node(NodeMapping::table("Customer", "customers", "id")
    .property("id", "id"));
mapping.map_node(NodeMapping::table("Order", "orders", "id")
    .property("id", "id")
    .property("total", "total"));
mapping.map_edge(EdgeMapping::table(
    "HAS_ORDER", "orders", "customer_id", "id", "Customer", "Order"
).foreign_key(ForeignKeyEndpoint::Destination));

The existing schema can enforce the relationship:

CREATE TABLE customers (id BIGINT PRIMARY KEY);
CREATE TABLE orders (
    id BIGINT PRIMARY KEY,
    customer_id BIGINT NOT NULL REFERENCES customers(id),
    total DOUBLE NOT NULL
);

Create both the child and the required relationship in one statement:

MATCH (c:Customer {id: 42})
CREATE (c)-[:HAS_ORDER]->(o:Order {id: 101, total: 29.95})
RETURN o

The runtime combines the node properties and the relationship’s FK assignment into one INSERT INTO orders (id, customer_id, total) .... It creates no other table. An edge to an existing child updates that row. Edge identities are the child primary keys, qualified by relationship type; do not supply independent relationship IDs. A child can have at most one live edge per FK mapping.

Equivalent TOML:

[edge.HAS_ORDER]
table = "orders"
src = "customer_id"
dst = "id"
src_label = "Customer"
dst_label = "Order"
foreign_key = "dst"

edge_id is derived from the child endpoint and may be omitted. If supplied, it must be the complete ordered child key. foreign_key = "src" selects the source endpoint as the child.

Deleting the relationship clears its nullable, non-primary-key FK components, preserving the child row, shared key components, and other required columns. An FK with no nullable non-key component rejects this operation. To reparent a child, delete the old relationship and create its replacement in the same statement; only the final FK value is written. Deleting the child removes its FK relationship with it (use the graph language’s detach-delete operation where required). Deleting a parent does not implicitly delete its children; remaining database constraints apply.

An FK with any NULL component produces no graph edge in either SQL scans or native traversals. Newly created child rows without a relationship receive NULL for the FK, so a NOT NULL constraint fails; SQL defaults on that FK do not invent an implicit relationship. Required properties and links must be complete within a single statement, even in an explicit transaction.

Column write ownership must be unambiguous. Shared primary-key columns may appear in node properties and multiple FKs; they are immutable and all supplied values must agree. Other FK columns and relationship property columns must not also be node properties or owned by another relationship mapping. The child and parent must be table-backed; each ordered FK component must match its corresponding parent-key type. Query-backed writes and cyclic writes requiring deferred constraints are not supported. Compatible rows are batched with referenced rows inserted first. DuckDB constraints remain binding: in particular, updating an indexed FK column on a row referenced by another table can be rejected by DuckDB even when its primary key does not change. The engine propagates that error and rolls back; it does not disable constraints or rewrite unrelated rows.

Composite keys

Use ordered column arrays for node IDs, edge IDs, and either endpoint:

mapping.map_node(NodeMapping::table("Customer", "customers", ["tenant", "id"])
    .property("tenant", "tenant").property("id", "id").property("name", "name"));
mapping.map_node(NodeMapping::table("Order", "orders", ["tenant", "id"])
    .property("tenant", "tenant").property("id", "id").property("total", "total"));
mapping.map_edge(EdgeMapping::table(
    "HAS_ORDER", "orders", ["tenant", "customer_id"], ["tenant", "id"],
    "Customer", "Order",
).foreign_key(ForeignKeyEndpoint::Destination));

Here orders(tenant, customer_id) references customers(tenant, id), while the order’s own primary key is (tenant, id). No relationship table is created. The shared tenant component cannot change when reparenting an existing order.

[node.Customer]
table = "customers"
id = ["tenant", "id"]
[node.Customer.properties]
tenant = "tenant"
id = "id"
name = "name"

[node.Order]
table = "orders"
id = ["tenant", "id"]
[node.Order.properties]
tenant = "tenant"
id = "id"
total = "total"

[edge.HAS_ORDER]
table = "orders"
src = ["tenant", "customer_id"]
dst = ["tenant", "id"]
src_label = "Customer"
dst_label = "Order"
foreign_key = "dst"

Columns match by position, not name. All primary-key components are required, non-null scalars; nested keys and duplicate/empty column lists are rejected. Existing scalar constructor arguments and serialized strings remain supported. Rust callers accessing mapping fields directly now receive KeyColumns; use .columns() to inspect the ordered names. Ordinary relationship tables can use .with_id(["tenant", "edge_id"]) and composite endpoints too.

Cypher supplies all key components through their declared property aliases:

CREATE (:Customer {tenant:'acme', id:42, name:'Alice'})
       -[:HAS_ORDER]->(:Order {tenant:'acme', id:101, total:29.95})

Gremlin accepts and returns composite IDs as ordered lists:

g.addV('Customer').property(T.id, ['acme', 42]).property('name', 'Alice')
g.V().hasLabel('Customer').hasId(eq(['acme', 42]))

Use eq(tuple) for one composite ID: Gremlin’s ordinary hasId(list) overload expands the list as multiple candidate IDs. The runtime keeps component types and boundaries through joins, traversals, paths, and serialization. No implicit key generation is introduced. If a child’s FK consists entirely of its primary-key columns, create its relationship explicitly in the same statement as the child.

A complete runnable example is examples/composite_keys.rs: cargo run --features duckdb --example composite_keys.

Collection columns as logical tables

A collection source expands a native list column or a typed list-valued expression into a named, read-only relation. It is an inline logical plan, so no database view needs to be created. Node, edge, RDF, and query-backed mappings can reference its name.

Mapping example

For orders(id BIGINT, items STRUCT(item_id BIGINT, sku VARCHAR, quantity BIGINT)[]):

{
  "collection_sources": [{
    "name": "order_items",
    "table": "orders",
    "column": "items",
    "parent_columns": {"order_id": "id"},
    "fields": {"item_id": "item_id", "sku": "sku", "quantity": "quantity"}
  }],
  "nodes": [{
    "label": "Item",
    "table": "order_items",
    "id": ["order_id", "item_id"],
    "properties": {"order_id": "order_id", "sku": "sku", "quantity": "quantity"}
  }]
}

The parent column and field maps are output alias → source column/field. For a scalar list, replace fields with "element": "value". Supply exactly one of element or nonempty fields.

The compiler’s existing string schema declarations accept list:string, list:int64, and list:struct:{"item_id":"int64","sku":"string","quantity":"int64"}. The JSON object after struct: maps field names to recursive type strings. The Rust API accepts Arrow List, LargeList, or FixedSizeList providers directly.

mapping.register_collection_source(collection_source)?;

The type is orchiddb::ir::rel::collection_source::CollectionSource. Register the parent schemas/providers first, followed by partition-layout sources, then collection sources. A collection’s parent can be a physical table, a logical source with equivalent physical layouts, or a representation source exposing a supported list column. Register each dependency before its dependent source. Replacing a parent provider refreshes dependent collection schemas; incompatible replacements leave the collection unbound instead of retaining stale data.

TOML round-trips definitions in [collection_sources] with catalog holding a JSON array string, like the layout catalog. Register physical providers after loading TOML.

Plans and partition selection

MATCH (i:Item)
WHERE i.order_id = 42 AND i.quantity > 1
RETURN i.sku, i.quantity

The plan contains the parent scan, list projection, Unnest, field projection, and remaining filters. These appear in CompiledSql.logical_plan and execution stats.logical_plan. Physical-layout choices remain in layout_selections. A parent filter such as order_id = 42 can push beneath expansion and choose a partitioned parent table. An element filter such as quantity > 1 applies after expansion and does not prune parent partitions using scalar element statistics.

Run cargo run --example collection_table_plans for a complete mapping and the actual generated logical plan and SQL. The runnable example supplies a complete compiler request.

Semantics and limits

• By default, null and empty lists produce zero rows. outer: true retains the parent with null child columns. Null list elements produce a row with null element values. Graph identities with null components are excluded by the existing graph mapping rules.
• Parent columns repeat for each element. Duplicates remain relational duplicates; RDF applies its usual triple-set semantics. Result order is unspecified without ORDER BY.
• Explicit child keys are required for graph identity. Use a stable element key, often combined with the parent key. A parent key alone is not a child identity. Repeated values that duplicate a declared graph key are rejected when whole elements are materialized; they require a distinct child key even though relational projections retain duplicate rows. An explicit ordinality column can distinguish occurrences within a parent; positions change when the collection is reordered.
• One collection is expanded per definition. Use expand instead of column for a typed expression such as json.elements(profile, '$.friends'); fields then contains scalar expressions using the as row alias. Parent and child fields may retain nested values. Collection sources can reference other derived sources, with cycles rejected. No implicit text-to-JSON coercion or zipped-array expansion occurs.
• Define physical layout alternatives on the parent tables. Parent-table uniqueness and row counts are not child-table facts. Collection-level supplied constraints are not consumed by the optimizer in this version; parent constraints remain within the input plan.
• Writes through collection-backed node/edge mappings are rejected. Update the containing physical row through its owner instead.
• Native lists use the engine’s UNNEST support; JSON row functions have PostgreSQL and DuckDB transformations. Unsupported SQL operations fail explicitly. Runtime DuckDB metadata lookups use the parent’s unfiltered layout choice; query planning can choose using pushed parent filters.
• Layout byte/file estimates describe the parent scan, not the expanded row count. There is no new estimate for collection length or element selectivity.

Physical layout alternatives

The merged engine accepts several equivalent physical tables under one logical table name. Node, edge, RDF, and query-backed mappings refer to that logical name. The selector simplifies expressions, pushes predicates through relational projections, and chooses a physical table independently for each scan. Self-joins can select different layouts.

Register physical providers first, then call GraphMapping::register_logical_source. The SQL compiler accepts the same descriptors in its optional logical_sources array. Existing mappings need no changes. The logical source’s schema comes from default_table; alternatives must have the same column names, order, types and nullability. Use caller-owned views to normalize different physical schemas.

{
  "name": "events",
  "default_table": "events_by_id",
  "layouts": [
    {
      "table": "events_by_id",
      "specs": [{"spec_id": 0, "fields": []}],
      "partitions": [{"spec_id": 0, "bytes": 2000000, "files": 20}]
    },
    {
      "table": "events_by_region",
      "specs": [{
        "spec_id": 0,
        "fields": [{
          "source_column": "region",
          "source_id": 2,
          "field_id": 1000,
          "transform": {"kind": "identity"}
        }]
      }],
      "partitions": [
        {"spec_id": 0, "bytes": 1000000, "files": 10, "values": {"1000": "east"}},
        {"spec_id": 0, "bytes": 1000000, "files": 10, "values": {"1000": "west"}}
      ]
    }
  ]
}

A filter region = 'east' selects events_by_region. The logical plan contains TableScan: events_by_region, and the emitted SQL uses that table. All original predicates remain effective; metadata chooses a table rather than filtering the returned rows itself.

CompiledSql, DagStats, and ExecStats expose layout_selections. Each decision includes the logical source, selected physical table, estimated bytes/files/rows, selected partition specs, and candidate estimates/rejection reasons. logical_plan shows the resolved plan alongside the existing physical plan and execution measurements. Nested DAG statistics retain their layout decisions. Estimates describe planned scans, not measured Iceberg I/O.

Partition metadata

Each partition summary has a spec_id, bytes, files, optional delete_bytes, optional transformed values keyed by partition field ID, and optional inclusive source-column bounds:

{"spec_id": 0, "bytes": 1000, "files": 1, "bounds": {"event_id": {"min": "10", "max": "99"}}}

Values and bounds are scalar text, parsed using the source or transform output type. JSON null partition values represent nulls. Temporal transform outputs are integer offsets from 1970; bucket outputs are integer bucket IDs. Descriptors retain source IDs and partition field IDs. Each summary uses its own spec, including after partition evolution.

Supported descriptors: identity, bucket with buckets, truncate with width, year, month, day, hour, void, and unknown with name. Bucket hashing follows Iceberg’s hash requirements. Unsupported type/transform combinations contribute no pruning estimate. Comparisons, literal IN lists, conjunctions and disjunctions participate in estimation; unknown expressions retain their partitions.

Cost is estimated surviving bytes plus delete bytes plus 64 KiB per surviving file. Equal costs retain the default layout. Missing statistics are unknown, not zero; a default with unknown statistics is retained. An empty partition list describes a known empty layout. Statistics must summarize all files in that layout. The engine does not discover Iceberg metadata or scan data to construct these summaries; supply them from the caller’s catalog or manifest integration.

Optional generated statistics can supply source row/byte costs when a layout has no partition summaries. File counts remain unknown; when not all candidates supply file counts, layout comparison uses bytes consistently. Collected sample extrema never become partition bounds. Generation is explicit and does not discover Iceberg manifests.

Optional generation fields on the logical source and layouts exclude layouts whose generation differs. Optional layout snapshot strings are included in diagnostics. Omit both for ordinary catalogs; declaring alternatives asserts that they contain equivalent rows, including duplicate multiplicity. Snapshot strings are metadata provenance and do not issue snapshot-pinning SQL.

TOML round trips the descriptors as JSON in [logical_sources] / catalog, following the existing constraint-catalog convention. Register physical providers after parsing the TOML to bind the sources. Runtime scalar-key lookups use the same selector. Logical sources are read-only in mapped graph persistence; update physical tables and refresh their declarations/statistics separately.

Map multiple sources to one table

Use representation_sources when the same rows are available in different forms, such as a collection and a flat table. Map the graph label to their shared name. For tables that differ only in partitioning, use physical layout alternatives.

This example uses order_items from the collection mapping above and a physical order_items_flat(oid, line, product, qty) table:

{
  "representation_sources": [{
    "name": "items",
    "default_representation": "nested",
    "representations": [
      {
        "name": "nested",
        "source": {"kind": "table", "name": "order_items"}
      },
      {
        "name": "flat",
        "source": {"kind": "table", "name": "order_items_flat"},
        "columns": {"order_id": "oid", "item_id": "line",
                    "sku": "product", "quantity": "qty"}
      }
    ]
  }],
  "nodes": [{
    "label": "Item", "table": "items", "id": ["order_id", "item_id"],
    "properties": {"order_id": "order_id", "sku": "sku", "quantity": "quantity"}
  }]
}

columns maps shared column names to source columns; omit it when the names already match. Each source must return the same rows, including duplicates, with matching column names and types. Queries use Item; OrchidDB chooses which source to read using the query filters and available statistics. default_representation names the source to use when costs are unknown.

For a SQL definition, replace source with {"kind": "query", "sql": "SELECT oid AS order_id, line AS item_id, product AS sku, qty AS quantity FROM order_items_flat"}. The query can include joins or grouping. Register a stored copy of its results as another source with the same output columns.

Register physical schemas and collection sources first. In Rust, use GraphMapping::register_representation_source; TOML stores the JSON array in [representation_sources] under catalog. Node, edge and RDF mappings can all use the shared name. These mappings are read-only; update their underlying data.

Optional generation strings on the shared source and each alternative identify matching versions of the data. Set them together when refreshing sources; the default must match. Optional statistics on each alternative uses the partition metadata format, with a table field identifying each input. For collections, average_list_length supplies the average number of elements per parent when generated statistics are unavailable.

JSON documents and relationships

MATCH (person:Person)-[link:KNOWS]->(friend:Person)
RETURN person.name, friend.name, link.since

A document’s array can supply the rows behind a relationship. Map the parent ID, extract each array element’s target ID and properties, then traverse those edges with ordinary Cypher or Gremlin. JSON functions also work directly in expressions:

RETURN json.value(JSON '{"author":{"name":"Ada"}}', '$.author.name') AS author,
       json.value(JSON '{"count":7}', '$.count', 'int64') AS count

JSON values have their own type. Use a JSON '...' literal or json.parse(text); ordinary strings remain strings. Register a document column with schema type json in a compiler request, or use ir::functions::domain::json_type() for an Arrow schema in Rust.

Turn an array into edges

Suppose each person stores a document like this in profile:

{"friends":[{"id":2,"since":2020},{"id":3,"since":2023}]}

Use jsonb for the PostgreSQL column or JSON for the DuckDB column:

-- PostgreSQL
CREATE TABLE people (id BIGINT PRIMARY KEY, name TEXT, profile JSONB);

-- DuckDB
CREATE TABLE people (id BIGINT PRIMARY KEY, name VARCHAR, profile JSON);

Both tables can hold these example rows:

INSERT INTO people VALUES
  (1, 'Ada', '{"friends":[{"id":2,"since":2020},{"id":3,"since":2023}]}'),
  (2, 'Grace', '{"friends":[]}'),
  (3, 'Katherine', '{"friends":[]}');

The mapping below expands profile.friends into a named relation. Its fields become the source ID, target ID, and relationship properties:

[node.Person]
table = "people"
id = "id"

[node.Person.properties]
name = "name"
profile = "profile"

[collection_sources]
catalog = '''[
  {
    "name": "friend_links",
    "table": "people",
    "expand": "json.elements(profile, '$.friends')",
    "as": "item",
    "parent_columns": {"source_id": "id"},
    "ordinality": "position",
    "fields": {
      "target_id": "json.value(item.value, '$.id', 'int64')",
      "since": "json.value(item.value, '$.since', 'int64')"
    }
  }
]'''

[edge.KNOWS]
table = "friend_links"
src = "source_id"
dst = "target_id"
src_label = "Person"
dst_label = "Person"
edge_id = ["source_id", "position"]

[edge.KNOWS.properties]
since = "since"

Load the mapping with GraphMapping::from_toml and register the people schema or provider, as in the mapping reference. For the JSON compiler, put the same collection descriptor in the collection_sources array and use data_type: "json" for profile.

The opening query now returns Ada’s two friendships. Repeated array elements remain separate occurrences. position starts at one and distinguishes those edges; use an embedded stable relationship ID instead if identity must survive array reordering. json.elements itself exposes a zero-based index field.

Missing paths, SQL-null documents, and empty arrays produce no rows. A present value of the wrong kind is an error: json.elements requires an array. Set "outer": true on a collection source to retain parents with no child rows; the generated child fields and ordinality are null. Null endpoints do not form ordinary graph edges. A collection can use another collection as its source to expand nested documents.

Read and construct documents

MATCH (p:Person)
WHERE json.exists(p.profile, '$.friends')
RETURN p.name,
       json.array_length(p.profile, '$.friends') AS friend_count,
       json.query(p.profile, '$.friends') AS friends

json.query returns JSON. json.value returns a scalar string by default; pass a literal type name such as int64, float64, or boolean for a typed result. Missing paths return SQL null. An existing JSON null is preserved by json.query, while json.value returns SQL null for JSON null, objects, and arrays.

Construct values without string concatenation:

RETURN json.object('name', 'Ada', 'tags', json.array('graph', 'sql')) AS document

Strings stay JSON strings; JSON arguments embed their document value. SQL-null constructor values become JSON null. Object keys must be non-null strings. Duplicate object keys keep the last value.

json.transform converts a document to an explicitly typed nested value:

RETURN json.transform(
  JSON '{"id":7,"tags":["graph","sql"]}',
  JSON '{"id":"int64","tags":["string"]}'
) AS record

A schema object declares fields, a one-element schema array declares a list, and a type-name string declares a scalar. Missing fields become typed nulls. Invalid type conversions produce an error. The schema can also be supplied as a constant string.

Use documents in relationship rules

For example, map a document node’s payload and a requirement node’s criteria as JSON properties, then define containment:

[edge.MATCHES]
source = "Document"
target = "Requirement"
predicate = "json.contains(source.payload, target.criteria)"

MATCH (document:Document)-[:MATCHES]->(requirement:Requirement)
RETURN document.id, requirement.id

Containment checks the requested structure. Objects match a subset of their fields; arrays match an unordered subset, ignoring repeated requested values. {"a":{"b":1}} does not contain {"b":1} at the root. Use json.query first when containment should be tested within a particular subtree.

Function catalog

The row functions return lists of records when used as scalar expressions and can supply collection-source rows. Each record contains value, index, key, path, parent_path, and depth. Aggregates retain null inputs as JSON null; empty input returns SQL null. Specify ordering when array order or the last value of a repeated object key matters.

Execution and supported paths

OrchidDB has native implementations of these functions. PostgreSQL and DuckDB lower supported operations into their own JSON expressions and row-producing operations, so document work can compose with surrounding joins and filters. Both adapters support document reads, constructors, equality, containment, merge patch, aggregates, and collection expansion. DuckDB also lowers nested json.transform results and text validation. PostgreSQL lowers path mutations and scalar/list transforms; transforms containing structs execute natively. DuckDB path mutations currently execute natively. SQL generation reports unsupported combinations explicitly; backend errors are not retried with different semantics.

Use definite paths such as $.friends[0].id, $["key.with.dots"], or JSON Pointer /friends/0/id for portable SQL examples. The native path evaluator also supports wildcards, recursive descent, unions, slices, and basic filters. Array positions are zero based; negative JSONPath positions count backward. Mutation functions require definite paths. See the detailed JSON reference for path grammar, null behavior, mutation rules, and engine capability limits.

Statistics

Statistics describe the size and distribution of mapped data so OrchidDB can estimate how much work a query will do. They help it choose sources, order joins and filters, and decide how to fetch records during graph traversal. Cypher, Gremlin and SPARQL all use the same statistics and optimizer.

Statistics are optional. Generate them explicitly, then reuse the resulting snapshot for subsequent queries.

When to use statistics

Generate statistics when OrchidDB has meaningful choices to make: several physical sources for one mapping, joins with very different input sizes, collections that expand into many rows, or repeated traversals. They are particularly useful with external data such as Iceberg, where the embedded database may have little information about the mapped sources.

For simple queries over one source, a database with good statistics can often plan the SQL well on its own. Collection is less useful there, especially for one-off queries. Keep the database’s optimizer enabled: OrchidDB chooses mapped sources and shapes the query, then the database optimizes the SQL it receives.

Generation

Generation reads available source metadata and bounded samples through the existing database connection. It can make several requests to gather source sizes, column value distributions, key frequencies, relationship connectivity and collection lengths. The shared collector produces one snapshot; there are no statistics tiers or collection settings to tune.

Generate after registering the mappings and loading the data. Collection happens separately from query compilation, so compiling subsequent queries uses the snapshot without scanning the data again.

Optimizer behavior

The optimizer estimates the rows and work involved at each step of a query. It uses those estimates to:

• Choose among mapped tables, collection sources and SQL definitions containing the same logical rows.
• Order connected joins and choose build sides to reduce intermediate results.
• Order eligible filters so selective, inexpensive predicates run earlier.
• Filter collection parents using a selective joined source before expanding their elements.

These choices affect the generated SQL. During native graph execution, statistics also help choose between scanning and caching a source or fetching records by key, using the number of keys needed by the current traversal.

See the mapping reference for declaring alternative sources.

Snapshot reuse

Keep a snapshot in the client or engine to reuse it across queries. It can also be saved and loaded in another session with the same mappings and schemas. Cypher, Gremlin and SPARQL queries share it; generation is not repeated for each language or query.

Refresh the snapshot after substantial changes to data volume, value distributions or collection sizes. Regenerate it when mappings or schemas change. Loading a saved snapshot avoids collection; generating a new one replaces the estimates used for subsequent planning.

Views and SQL sources

Use SQL views and query-backed labels to shape a graph over existing relational data.

Map a physical view

A view can select, rename, or derive the columns that form a graph. Create the view in the same DuckDB session used by your mapped engine:

CREATE VIEW adult_users AS
SELECT id, name, age FROM users WHERE age >= 18;

Register adult_users with its Arrow schema, then map it like a table:

mapping.map_node(
    NodeMapping::table("Adult", "adult_users", "id")
        .property("name", "name")
        .property("age", "age")
);

The SQL generated by OrchidDB references the view. DuckDB evaluates the view definition against its underlying data.

Define a query-backed label

For a label defined directly by SQL, use NodeMapping::query:

mapping.map_node(
    NodeMapping::query(
        "Adult",
        "SELECT id, name, age FROM users WHERE age >= 18",
        "id",
    ).property("name", "name").property("age", "age")
);

Register the schemas of the tables referenced by the SQL. Query-backed definitions are incorporated into the generated relational query.

MATCH (p:Adult)
WHERE p.age > 30
RETURN p.name, p.age
ORDER BY p.age DESC

Define relationships with SQL

EdgeMapping::query takes the same endpoint information as a table mapping. A SQL expression can filter or reshape an existing relation before exposing it to graph queries:

mapping.map_edge(EdgeMapping::query(
    "LARGE_ORDER",
    "SELECT order_id, user_id, total FROM orders WHERE total > 100.0",
    "user_id", "order_id", "Person", "Order",
).with_id("order_id").property("total", "total"));

Work with attached data

Use the DuckDB connection to attach your data sources and create stable local views over them. Register the view schemas with OrchidDB and point mappings at those views. This keeps source connection details in the database setup and graph vocabulary in the mapping.

For example, after your session exposes a source as warehouse.customers, create a local view:

CREATE VIEW graph_customers AS
SELECT customer_id AS id, full_name AS name
FROM warehouse.customers;

Map graph_customers to Person using id and name. The same mapping can then be used by all graph languages.

Inspect the generated query

Use the SQL API to inspect generated SQL and logical plans without execution. Existing callers of the MappedGraphEngine compatibility facade can also call explain_cypher to inspect how graph properties, filters, and relationships refer to the source SQL:

let sql = graph.explain_cypher(
    "MATCH (p:Person) WHERE p.age > 30 RETURN p.name"
).await?;
println!("{sql}");

Managed graphs

Create an application-owned graph with persistence, mutations, and transactional query execution.

Open a graph

use orchiddb::engine::GraphEngine;

let mut graph = GraphEngine::open("social.duckdb")?;

Use GraphEngine::in_memory() for an ephemeral graph, such as a test fixture or a request-local calculation. Both constructors return Result<GraphEngine, String>.

Create nodes and relationships

CREATE (:Person {name:'Alice', age:30})
  -[:KNOWS {since:2020}]->
  (:Person {name:'Bob', age:28})

Run the statement with graph.cypher(query).await?. A write outside an explicit transaction commits atomically.

Create a relationship between existing nodes by matching their properties:

MATCH (a:Person), (b:Person)
WHERE a.name = 'Alice' AND b.name = 'Bob'
CREATE (a)-[:FOLLOWS]->(b)

Use parameters for values supplied by your application.

Read and update properties

MATCH (p:Person)
WHERE p.name = 'Alice'
SET p.age = 31
RETURN p.name, p.age

A subsequent query sees the updated value:

MATCH (p:Person)
RETURN p.name, p.age
ORDER BY p.name

Merge and delete

Use MERGE when the statement should match or create the specified pattern:

MERGE (p:Person {name:'Carol'})
RETURN p.name

Use DETACH DELETE when removing a node together with its incident relationships:

MATCH (p:Person)
WHERE p.name = 'Carol'
DETACH DELETE p

Import an in-memory graph

replace_graph accepts an Arrow-backed PropertyGraph and installs it as the engine’s graph in one transaction. This is useful when an application constructs a graph through the catalog APIs and then wants managed execution and persistence.

graph.replace_graph(property_graph)?;

This operation replaces the stored graph. Use it at an intentional import or reset boundary.

Inspect execution

Managed queries return QueryResult, which includes an Arrow result, an execution backend, and statistics:

let result = graph.cypher("MATCH (p:Person) RETURN p.name").await?;
println!("backend: {:?}", result.backend);
println!("rows: {}", result.returned.batch.num_rows());

Continue to transactions and storage for multi-statement work, or execution and plans for read policies.

Transactions and storage

Group graph changes, control commit boundaries, and maintain a persistent managed graph.

Explicit transactions

GraphEngine provides begin, commit, rollback, and in_transaction. Queries inside a transaction see earlier writes from that transaction.

graph.begin()?;
let write = graph.cypher(
    "CREATE (:Person {name:'Alice'})"
).await;
match write {
    Ok(_) => graph.commit()?,
    Err(error) => {
        graph.rollback()?;
        return Err(error);
    }
}

For several statements, keep them within the same transaction and commit after all application checks succeed. Dropping an engine with an active transaction discards its uncommitted work.

Statement atomicity

A failed graph statement restores its prior graph state. Writes executed outside an explicit transaction have an atomic commit boundary of their own. This makes individual operations straightforward while allowing applications to group related work explicitly.

Multiple engine instances

Separate engines use DuckDB snapshot isolation. An engine outside a transaction refreshes its committed snapshot when it queries. Within a transaction, the snapshot provides a consistent view together with that transaction’s writes.

If writers conflict, roll back the failed transaction and retry the complete unit of work against a fresh snapshot. Apply application-level retry limits and ensure that external side effects are coordinated with successful commits.

Persistence and checkpoints

Managed writes persist changed node, edge, and label records transactionally. A checkpoint compacts those records into a graph snapshot:

graph.checkpoint()?;

A checkpoint participates in an active transaction, so a rollback also rolls back the compaction. replace_graph installs a new graph checkpoint as part of its replacement operation.

Mapped engine transactions

GraphEngine::mapped uses the same begin, commit, and rollback API for Cypher, Gremlin, and SPARQL. All statements share one DuckDB connection. Reads see earlier writes from any of these languages in the transaction.

graph.begin()?;
let changed = graph.cypher(
    "MATCH (p:Person) WHERE p.age >= 30 SET p.age = p.age + 1 RETURN p.age"
).await;
match changed {
    Ok(result) => {
        graph.commit()?;
        println!("returned {} rows", result.returned.batch.num_rows());
    }
    Err(error) => {
        graph.rollback()?;
        return Err(error);
    }
}

Outside an explicit transaction, mapped statements commit atomically or roll back on failure/cancellation. A failed mapped execution inside an explicit transaction marks it failed; call rollback before continuing. Do not assume that only the failing statement was undone and then commit earlier work.

Existing MappedGraphEngine callers retain executor_mut().begin(), commit(), and rollback() on the compatibility facade. These are not methods on GraphEngine. For typed RDF reads use sparql_query; for updates use sparql_update(update, dataset, base). Neither needs a second RDF engine.

Back up a graph file

For a simple file backup workflow, stop writers, close connections to the database, and then copy the database file. Keep the backup together with the application revision and mapping configuration needed to interpret it. Test restores by opening a copied database and running representative reads.

Upgrading a local database

Stop other processes using the file and keep a backup before upgrading. On open, OrchidDB recognizes a prior graph-table namespace by its reserved schema and validates the checkpoint before renaming tables in a transaction. Checkpoints and incremental records are preserved. Ambiguous or corrupt stores are rejected instead of being replaced with an empty graph. Older typed Arrow metadata is accepted while new metadata uses the OrchidDB namespace.

Rebuild JVM artifacts together with the native orchiddb-jvm-store executable; Java packages, configuration keys, and environment variables now use OrchidDB.

Update mapped tables

Use GraphEngine::mapped to read and write application tables with Cypher, Gremlin, and SPARQL. All languages use the same mapping catalog, DuckDB connection, and transaction owner.

Create mapped rows

Declare every writable property, including a property alias for keys supplied through Cypher. For example, add .property("id", "id") to the tutorial’s Person mapping:

graph.cypher(
    "CREATE (p:Person {id:4, name:'dave', age:29}) RETURN p.name"
).await?;

Mapped keys must be supplied explicitly. Scalar and ordered composite keys are supported; there is no automatic integer allocation or primary-key default generation. For Gremlin, T.id supplies the mapped key without requiring a property alias:

graph.gremlin("g.addV('Person').property(T.id,4).property('name','dave').property('age',29)").await?;

These are alternative ways to create the same row; do not execute both against the same existing key. Every composite-key component must be non-null. Identity remains qualified by the mapped label or relationship type.

Constant defaults on omitted mapped columns are visible during the statement. Volatile/function defaults on omitted mapped values are currently rejected. Unmapped source columns retain their database defaults and constraints. Unknown properties are not stored in supplemental columns or overflow tables.

Update and return values

let result = graph.cypher(
    "MATCH (p:Person) WHERE p.id=4 \
     SET p.age=p.age+1, p.name='David' RETURN p.name,p.age"
).await?;
println!("{} rows", result.returned.batch.num_rows());

Use cypher_with_params to bind values. SET p += {...} assigns properties; SET p = {...} clears omitted writable mapped properties to SQL NULL. Identity and endpoint columns remain protected. Primary keys are immutable. Scalar and composite keys use the same typed identity rules.

Relationships and deletion

Independent edge-row mappings need explicit edge IDs for writes. Their endpoint columns match node keys in declared order. An FK-backed relationship instead uses its child row’s key as the edge identity: creating a child and its required relationship becomes one complete row insertion. Deleting that relationship clears its nullable non-key FK components; it does not delete the child row. See FK-backed relationships.

MATCH (p:Person {id:4})
DETACH DELETE p

Node deletion checks mapped incident relationships. Database constraints still apply. Properties must have one write owner: the tutorial’s total belongs to Order, so update o.total, not an unmapped ORDERED.total edge property.

MATCH (:Person)-[:ORDERED]->(o:Order)
WHERE o.total > 100.0
SET o.total = o.total + 10.0
RETURN o.total

RDF updates

Add writable RdfMapping rules with complete physical keys, then call sparql_update(update, "default", None). RDF inserts and deletes translate to mapped row transitions; they do not populate a separate triple store. A scalar property replacement requires deleting the old RDF value and inserting the new one in the same request. See RDF updates.

Query-backed mappings, partition-layout logical sources, and collection-backed node/edge mappings are read-only. Updates must target writable physical rows. No write is redirected to managed storage when a mapping cannot represent it.

Transactions

begin, commit, and rollback on GraphEngine cover all three languages. Statements outside an explicit transaction commit atomically; a failed mapped execution within an explicit transaction requires rollback before further work. See mapped transactions.

Compatibility APIs

MappedGraphEngine::cypher_update and cypher_update_with_params remain for callers needing an affected-row count. They accept one property assignment over a mapped target, without RETURN, CREATE, DELETE, MERGE, or map replacement. They execute through the shared runtime, not a separate direct-SQL update engine. Use GraphEngine::cypher for multiple assignments and returned values.

The compatibility facade also retains execute_sql and executor_mut. GraphEngine owns its connection: perform relational setup before passing the connection to GraphEngine::mapped, or consume the engine with into_executor to recover it for subsequent SQL work.

Search

MATCH (seed:Document {id: $id})-[similarity:SIMILAR_TO]->(related)
MATCH (related)-[:WRITTEN_BY]->(author)
RETURN related.title, author.name, similarity.score
ORDER BY similarity.score DESC

Find documents similar to a seed, then follow their authors through the graph. SIMILAR_TO is a computed relationship: its edges come from a search rule. WRITTEN_BY is an ordinary stored relationship. Both compose in the same query.

Define the relationship once and use the matching scoring operation with PostgreSQL/pgvector, Lance through DuckDB, Quickwit, Elasticsearch, or native scoring. The examples below assume documents have IDs, titles, body text, and embeddings; map WRITTEN_BY separately as an ordinary stored relationship.

Define the relationship

Map your columns to graph properties and describe how to select neighbors:

[node.Document]
table = "documents"
id = "id"

[node.Document.properties]
id = "id"
title = "title"
body = "body"
embedding = "embedding"

[edge.SIMILAR_TO]
source = "Document"
target = "Document"
order_by = [{ expression = "score", direction = "desc" }]
limit_per_source = 10

[edge.SIMILAR_TO.properties]
score = "vector.cosine_similarity(source.embedding, target.embedding)"

Each document has up to ten outgoing edges, ranked by cosine similarity, with a score property on each edge. Expressions reference the mapped properties using source and target. This rule includes the seed among eligible matches; see Optional filters to exclude it or restrict eligible documents.

Load this TOML mapping with GraphMapping::from_toml and register your tables as described in the mapping reference. Add one of the backend configurations below to use an existing search index. Each [[source_metadata]] block belongs at the top level of the same mapping file; declare each table once.

The relationship also works in Gremlin:

g.V().has('Document', 'id', 7).
  outE('SIMILAR_TO').as('similarity').
  inV().as('document').select('document', 'similarity')

Ranking belongs to the relationship: filtering its destinations later does not refill its top ten, and reverse traversal follows the same directed edges. Computed edges are read-only. approximate_allowed is the default; exact requests exhaustive vector ranking.

PostgreSQL with pgvector

Use a PostgreSQL documents table with a native vector(N) embedding column, where N matches your embedding dimension. Enable the installed extension and create a cosine index on your existing table:

CREATE EXTENSION IF NOT EXISTS vector;
CREATE INDEX documents_cosine ON documents
  USING hnsw (embedding vector_cosine_ops);

Bind the column in the mapping:

[[source_metadata]]
table = "documents"
format = "pgvector"
indexes = [{ column = "embedding", metric = "cosine" }]

Assign documents to a PostgreSQL engine in the compiler request. The SIMILAR_TO definition and Cypher query stay the same. For same-engine data, the search portion has this SQL shape (aliases and score guards simplified):

SELECT source.id AS source_id, hits.*
FROM documents AS source
CROSS JOIN LATERAL (
  SELECT target.id AS target_id,
         1.0 - (target.embedding <=> source.embedding) AS score
  FROM documents AS target
  ORDER BY target.embedding <=> source.embedding ASC NULLS LAST
  LIMIT 10
) AS hits
WHERE source.id = $1;

Descending similarity becomes ascending distance so PostgreSQL can use the index. The remaining graph joins can stay in the same SQL island. Dot product uses <#> and L2 distance uses <->; choose the matching index operator class and mapping metric. See pgvector indexing for index options. exact deliberately avoids ANN index selection.

Lance through DuckDB

Use a Lance dataset at /data/documents.lance with the mapped columns and fixed-size embeddings, such as FLOAT[384]. In the DuckDB connection that will execute the searches, load duckdb-lance and expose the dataset as documents:

INSTALL lance FROM community;
LOAD lance;
CREATE VIEW documents AS SELECT * FROM '/data/documents.lance';

Create a cosine index if the dataset does not already have one. This small-data example uses one partition; size the index for your dataset:

CREATE INDEX documents_cosine ON '/data/documents.lance' (embedding)
  USING IVF_FLAT WITH (num_partitions = 1, metric_type = 'cosine');

Use this binding instead of the pgvector binding:

[[source_metadata]]
table = "documents"
format = "lance"
options = { uri = "/data/documents.lance" }
indexes = [{ column = "embedding", metric = "cosine", options = { nprobes = 1, refine_factor = 2 } }]

The table and URI must identify the same dataset. The binding metric must match the index. nprobes controls IVF partitions searched; refine_factor controls candidate refinement. duckdb-lance’s SQL reference describes these backend options and index creation.

Assign documents to a DuckDB engine. OrchidDB reads the selected seed, binds its embedding into lance_vector_search, and joins the hits into the graph query. For a three-dimensional example seed, the search statement has this shape; actual vector dimensions must match your dataset:

SELECT target.id,
       list_cosine_similarity([0.1, 0.9, 0.2], target.embedding) AS score
FROM lance_vector_search(
  '/data/documents.lance', 'embedding', [0.1, 0.9, 0.2]::FLOAT[3],
  k = 10, use_index = true, nprobs = 1, refine_factor = 2, prefilter = true
) AS target;

The Cypher query and relationship rule stay unchanged. Install duckdb-lance for the DuckDB version running your queries. See the detailed reference for version compatibility and verification details.

Candidate filters must be pushable comparisons, null checks, or conjunctions. Lance exact currently supports L2; exact cosine/dot requests fail planning because the extension’s non-indexed search does not expose a metric selector.

BM25 full-text search

Use document text as the query to define a lexical relationship over the same nodes:

[edge.TEXT_MATCH]
source = "Document"
target = "Document"
order_by = [{ expression = "score", direction = "desc" }]
limit_per_source = 10

[edge.TEXT_MATCH.properties]
score = "text.bm25(source.body, target.body)"

For indexed Lance retrieval, create a text index and replace the earlier documents source metadata with the combined vector and text declaration:

CREATE INDEX documents_text ON '/data/documents.lance' (body) USING INVERTED;

[[source_metadata]]
table = "documents"
format = "lance"
options = { uri = "/data/documents.lance" }
indexes = [
  { column = "embedding", metric = "cosine", options = { nprobes = 1, refine_factor = 2 } },
  { column = "body", metric = "bm25" },
]

Query the relationship normally:

MATCH (seed:Document {id: $id})-[r:TEXT_MATCH]->(related)
RETURN related.title, r.score
ORDER BY r.score DESC

For a seed whose body is graph retrieval, the backend search is shaped like:

SELECT target.id, target._score AS score
FROM lance_fts('/data/documents.lance', 'body', 'graph retrieval',
               k = 10, prefilter = true) AS target;

Lance supplies its indexed corpus statistics, tokenizer, and scores. Without a declared text index, native, PostgreSQL, and ordinary DuckDB execution compute portable BM25 over the mapped target corpus before pair filters and ranking. That mode is exhaustive scoring. pgvector does not provide a BM25 index, and OrchidDB does not substitute PostgreSQL ts_rank for BM25.

Quickwit and Elasticsearch

Enable the optional quickwit or elasticsearch Cargo feature to execute BM25 relationships against a remote index. The text.bm25 relationship definition stays the same. Assign an index table to the remote engine, or declare a remote index over a SQL-owned document table. Search results then compose with stored edges and properties in your SQL engines.

The Quickwit and Elasticsearch guide covers engine registration, index configuration, authentication, and execution. Both adapters return the engine’s indexed BM25 scores and preserve filters before top-k.

Retrieve candidates, then rerank with MaxSim

Map a tokens property to a column containing each document’s token vectors (for example, add tokens = "token_embeddings" under [node.Document.properties]). Supply embeddings from your model; normalize them if your ColBERT scoring requires cosine token similarity.

This relationship retrieves up to 200 BM25 candidates with the text binding above, then selects ten using MaxSim:

[edge.RELEVANT_TO]
source = "Document"
target = "Document"
order_by = [{ expression = "score", direction = "desc" }]
limit_per_source = 10

[edge.RELEVANT_TO.candidates]
order_by = [{ expression = "lexical", direction = "desc" }]
limit_per_source = 200

[edge.RELEVANT_TO.candidates.properties]
lexical = "text.bm25(source.body, target.body)"

[edge.RELEVANT_TO.properties]
score = "vector.maxsim(source.tokens, target.tokens)"

Traverse RELEVANT_TO in place of SIMILAR_TO in the opening query. Both score and lexical are available as edge properties. To retrieve vector candidates instead, replace the candidate’s lexical expression with vector.cosine_similarity(source.embedding, target.embedding) and use the pgvector or Lance vector binding. Candidate property names must be distinct from final property names.

MaxSim sums the best document-token inner product for each query token. Eligibility filters belong in the candidate predicate when they must apply before retrieval. Final-stage predicates filter only retrieved candidates. Removing the candidate stage makes MaxSim exhaustive over the target relation.

Available scores

All built-in scores have native implementations and PostgreSQL/DuckDB SQL mappings:

Native scoring and ordinary DuckDB scoring evaluate eligible pairs, which can be expensive for large collections. PostgreSQL vector search can use an existing matching pgvector index even without an explicit index declaration. Lance search requires the source metadata shown above. Portable BM25 and MaxSim score their inputs exhaustively; use indexed candidates to limit reranking work.

Vectors may be float32/float64 lists or fixed-size lists. Null inputs produce null scores; zero-norm cosine inputs also produce null. Indexed search requires valid query vectors with dimensions matching the index.

Optional filters

A relationship needs no predicate field unless you want to restrict eligible matches. For example, add this line under [edge.SIMILAR_TO] to exclude the seed:

predicate = "target.id <> source.id"

To restrict matches to the same tenant, map tenant_id on the nodes and use:

predicate = "source.tenant_id = target.tenant_id AND target.id <> source.id"

These are relationship rules: they apply before selecting its top ten. A WHERE clause in a consuming Cypher query filters the resulting edges and does not replace discarded matches. With a candidate stage, put eligibility filters under [edge.RELEVANT_TO.candidates] so they apply before candidate selection. A filter on the final stage applies only to those candidates.

Expressions support arithmetic, comparisons, Boolean logic, CASE, and registered scalar functions. Indexed retrieval supports the filters its backend can execute; unsupported filters produce an error.

Using multiple engines

Assign tables to engines in the compiler request. The graph query can combine search results with relationships stored in other engines; OrchidDB composes the searches, transfers, and graph joins. Index errors are reported to the caller.

See the detailed search reference for the compiler API, native execution, custom scoring functions, and backend constraints. To add support for another database, see Engine adapters.

Quickwit and Elasticsearch

MATCH (question:Question {id: $id})-[match:SIMILAR_TO]->(document:Document)
MATCH (document)-[:WRITTEN_BY]->(author:Person)
RETURN document.title, author.name, match.score
ORDER BY match.score DESC

Use Quickwit or Elasticsearch to retrieve documents, then follow ordinary graph relationships stored in SQL. Both are optional engines. The search relationship describes scoring and ranking; OrchidDB constructs the remote requests and joins their results into the remaining SQL islands.

Enable the engine you use:

[dependencies]
orchiddb = { path = "../orchiddb", features = ["quickwit", "elasticsearch"] }

The features are independent and disabled by default. Applications that execute DuckDB locally also enable duckdb; applications supplying their own SQL sessions do not need it.

Define the relationship

Map Question to your query-input table and Document to your document table or index. This example assumes both expose body and tenant properties, and that WRITTEN_BY is already mapped as a stored relationship.

[edge.SIMILAR_TO]
source = "Question"
target = "Document"
predicate = "source.tenant = target.tenant"
order_by = [{ expression = "score", direction = "desc" }]
limit_per_source = 10

[edge.SIMILAR_TO.properties]
score = "text.bm25(source.body, target.body)"

This selects up to ten matching documents from the same tenant for each question. The tenant condition applies before retrieval. A WHERE clause on the consuming Cypher query filters the selected edges without refilling their top ten.

Quickwit and Elasticsearch supply their own indexed corpus statistics and text analysis. Scores are backend scores; their numeric values need not match across engines. Indexed retrieval returns matching documents, not unrelated documents with zero scores. A null query produces no matches.

Gremlin traverses the same relationship:

g.V().has('Question', 'id', 7).
  outE('SIMILAR_TO').as('match').
  inV().as('document').select('document', 'match')

Register an index as an engine-owned table

In the compiler request, register the engine and assign the index table to it. This excerpt uses Quickwit; replace quickwit with elasticsearch to select Elasticsearch.

{
  "engines": {
    "local": {"dialect": "duckdb"},
    "text": {"dialect": "quickwit"}
  },
  "execution_engine": "local",
  "tables": [{
    "name": "documents",
    "engine": "text",
    "columns": [
      {"name": "id", "data_type": "int64", "nullable": false},
      {"name": "body", "data_type": "string"},
      {"name": "tenant", "data_type": "string"},
      {"name": "title", "data_type": "string"}
    ]
  }],
  "source_metadata": [{
    "table": "documents",
    "format": "quickwit",
    "options": {"index": "documents"},
    "indexes": [{"column": "body", "metric": "bm25"}]
  }]
}

The complete request also includes the query, node and edge mappings, and the SQL tables used for questions and authors. Table names for engine-owned indexes identify their actual remote indexes. Stored fields must match the declared column types, and graph IDs must be unique and non-null.

Quickwit needs a stored key, a text field with term frequencies and fieldnorms: true, and appropriate indexed/fast fields for filtering and ordering. Use a raw tokenizer for exact string comparisons such as tenant IDs. See Quickwit index configuration. For Elasticsearch, use text for BM25 content and keyword for exact string filters. Keep the required fields available in _source.

Use a remote index over SQL-owned documents

Keep the documents table assigned to your SQL engine and name the index engine in its source metadata:

[[source_metadata]]
table = "documents"
format = "quickwit"
options = { engine = "text", index = "document_index", key_field = "document_id", text_field = "body" }
indexes = [{ column = "body", metric = "bm25" }]

Use format = "elasticsearch" with an Elasticsearch engine. The remote key must correspond to the mapped document ID. OrchidDB retrieves keys and scores, then joins the selected keys to the authoritative SQL table. Titles and other properties therefore come from SQL, even if older copies exist in the index. Fields used for eligibility must be indexed consistently with their SQL values. Maintaining index freshness and unique document keys is the application’s responsibility. Missing SQL documents do not refill the ranked candidate set.

Connect and execute

Connections and credentials belong to the application, not the compiled plan. The HTTP sessions implement federation::Session alongside your SQL session:

use orchiddb::remote::transport::{
    Authentication, HttpOptions, QuickwitSession,
};

let mut options = HttpOptions::new("https://quickwit.example.com");
options.authentication = Authentication::Bearer(token);
let text = QuickwitSession::with_options(options)?;

sessions.insert("text".into(), Box::new(text));
// Register your SQL session under "local" as well.
let batches = orchiddb::federation::execute(&compiled, &mut sessions).await?;

For Elasticsearch, use ElasticsearchSession and the same options. Basic, bearer-token, and API-key authentication are supported. A caller-supplied reqwest::Client can provide custom certificate roots. Configure request timeouts, page size, and result limits through HttpOptions.

Ordinary remote scans use paginated snapshot reads. Configured limits reject oversized results rather than silently truncating them. HTTP errors, malformed responses, and partial search failures propagate to the caller.

Language clients

Python, JavaScript, JVM, C++, and Elixir provide RemoteEngine adapters for these services. Register one alongside your application-owned SQL adapter and use the client’s federated-query entry point. Keep the remote engine open across queries and close it when the application no longer needs it. These clients use the same validated HTTP transport as Rust, including pagination, typed results, and error handling. Each client README includes connection and query examples.

The Rust client exposes the adapters through its independent quickwit and elasticsearch features. The CLI accepts a separate engine options file:

{
  "text": {"endpoint": "http://localhost:7280", "page_size": 1000}
}

orchiddb query request.json --engines engines.json --init setup.sql --no-iceberg

The request’s engines registry selects Quickwit or Elasticsearch; the options file supplies connection details for that engine ID. Credentials belong in this application configuration, never in the compiled query.

Compose retrieval and reranking

Use the candidate-stage relationship syntax to retrieve candidates from either engine and rerank their token embeddings with MaxSim. The candidate stage keeps the indexed BM25 score, while the final stage selects the highest reranked scores. Token embeddings must be available from the mapped document source; OrchidDB does not generate embeddings.

API choice and supported operations

Portable BM25 relationships use each engine’s structured search endpoint. Quickwit’s Elasticsearch-compatible endpoint exposes the score metadata and non-scoring Boolean filters required by these relationships. Elasticsearch is not a dependency of Quickwit.

Quickwit’s native search API remains available for explicitly requested native queries. Its hit response does not expose the same score metadata; native text queries and scored relationship retrieval therefore have distinct interfaces.

The adapters push supported projections, comparisons, Boolean filters, ordering, and limits into remote requests. SQL null semantics are preserved, including missing fields under negation. Expressions that cannot be pushed stay in a valid SQL island where possible. Unsupported eligibility expressions inside indexed top-k retrieval fail explicitly.

Advanced native queries can return a typed JSON response through the request operation interface and then compose with JSON functions. This makes backend query DSL features and aggregations available without claiming that every backend operation has an automatic Cypher-to-query-DSL translation.

The Rust relational API exposes quickwit.search and elasticsearch.search as typed TableFunction operations, and quickwit.query and elasticsearch.query as complete JSON responses. Quickwit additionally provides native_search and native_query for its native API. These names are relational API operations, not new Cypher scalar functions. Native search requires an explicit finite max_hits; use the structured search operation for paginated enumeration. Automatic aggregate pushdown and automatic vector relationship lowering are not currently provided by these adapters.

For an exact Quickwit relationship, a single target key supplies the secondary tie-break order: Quickwit permits at most two sort criteria, including the score. Index aliases or mappings changed during a session require clear_metadata_cache() before their new schema is used.

Verify against running services

The repository includes pinned local services and live tests that create their own indexes:

docker compose -f tests/remote-engines.compose.yml up -d
QUICKWIT_URL=http://127.0.0.1:17280 \
ELASTICSEARCH_URL=http://127.0.0.1:19200 \
cargo test --features duckdb,quickwit,elasticsearch \
  --test remote_engines -- --ignored

The tests require reachable services; they do not silently pass when one is absent. The compose configuration binds unauthenticated test services to localhost and is intended only for local testing.

Cypher

Express graph patterns, filter properties, and shape tabular results with Cypher.

Conformance

OrchidDB passes all 3,897 scenarios in the pinned openCypher 2024.3 Technology Compatibility Kit. The conformance comparison includes individual outcomes, upstream assertions, execution timings, and downloadable evidence.

Match a pattern

Cypher uses parentheses for nodes and brackets for relationships. A colon names a label or relationship type, and an arrow gives direction:

MATCH (p:Person)-[:ORDERED]->(o:Order)
RETURN p.name, o.total
ORDER BY o.total

These queries use the mapped tutorial dataset. The pattern follows each person’s orders and returns the person’s name with the order total.

Filter and project

Use WHERE to select matches and AS to name result columns:

MATCH (p:Person)
WHERE p.age >= 30
RETURN p.name AS name, p.age AS age
ORDER BY age DESC

The tutorial returns carol, then alice. Use explicit column aliases for stable application-facing result names.

Read relationship properties

Bind a relationship to a variable to use its properties:

MATCH (p:Person)-[r:ORDERED]->(:Order)
WHERE r.total > 100.0
RETURN p.name, r.total
ORDER BY r.total

A mapped relationship property resolves to the column configured with EdgeMapping::property.

Aggregate matches

Non-aggregate expressions define the grouping keys:

MATCH (p:Person)-[r:ORDERED]->(:Order)
RETURN p.name AS name, sum(r.total) AS spent
ORDER BY name

Use count(...) to count matches and DISTINCT when the query should remove duplicate results.

Compose query stages

WITH introduces an intermediate projection. It is useful for filtering an aggregate before returning the final result:

MATCH (p:Person)-[:ORDERED]->(o:Order)
WITH p.name AS name, sum(o.total) AS spent
WHERE spent > 100.0
RETURN name, spent
ORDER BY spent DESC

Preserve optional matches

OPTIONAL MATCH keeps the input row when its optional pattern has no match:

MATCH (p:Person)
OPTIONAL MATCH (p)-[:FOLLOWS]->(friend:Person)
RETURN p.name, friend.name
ORDER BY p.name, friend.name

In the tutorial, carol has no outgoing FOLLOWS row, so her optional friend binding is null.

Bound a traversal

A variable-length relationship can express paths over several hops:

MATCH (p:Person)-[:FOLLOWS*1..2]->(friend:Person)
WHERE p.name = 'alice'
RETURN DISTINCT friend.name
ORDER BY friend.name

Choose explicit bounds that match the application’s question. The distinct projection returns each friend’s name once even if several paths reach that friend.

Parameters and writes

Pass application inputs through cypher_with_params and reference them as $name or $minimum. See parameters for a complete example.

For graph-owned writes, use managed graphs. For native updates over mapped tables, use update properties.

Gremlin

Traverse nodes and relationships step by step with Gremlin, then project the values your application needs.

Conformance

OrchidDB passes all 1,511 scenarios in the pinned Apache TinkerPop 3.7.4 Gremlin suite. The complete run uses one OrchidDB instance. The conformance comparison includes individual outcomes, upstream assertions, execution timings, and downloadable evidence.

Start a traversal

OrchidDB accepts Gremlin traversal text through gremlin(query).await. In the standalone CLI request, set "language": "gremlin". The optional managed core binary uses --language gremlin.

JVM callbacks and TinkerPop vertex programs execute as operators in the same SQL IR/DataFusion pipeline. Typed callbacks can be supplied through gremlin_with_bindings. The compatibility matrix reports one OrchidDB outcome per scenario.

g.V().hasLabel('Person').values('name')

g.V() starts at vertices, hasLabel selects a label, and values projects a property. Examples here use the mapped tutorial.

Filter vertices

g.V().hasLabel('Person').has('name', 'alice').values('age')

The tutorial returns 30. Predicates express comparisons:

g.V().hasLabel('Person').has('age', gt(29)).values('name').order()

This selects alice and carol.

Follow relationships

g.V().hasLabel('Person').has('name', 'alice')
  .out('FOLLOWS').values('name').order()

Use out(type) to follow relationships from source to destination and in(type) to walk toward their source. Use both(type) when the question concerns either direction.

For an edge property, traverse the edges themselves:

g.V().hasLabel('Person').has('name', 'alice')
  .outE('ORDERED').values('total')

This returns alice’s two order totals, 50 and 120.

Reduce values

g.V().hasLabel('Person').has('name', 'alice')
  .out('ORDERED').values('total').sum()

The sum is 170. Use count() to count traversers at the current step:

g.V().hasLabel('Person').count()

The tutorial contains three people.

Order and deduplicate

g.V().hasLabel('Person').out('FOLLOWS')
  .values('name').dedup().order()

dedup() removes repeated values from this projection. order() makes the output order explicit. Use limit(n) or range(start, end) when selecting a window of traversal results.

Name traversal positions

as records a binding for later select steps:

g.V().hasLabel('Person').as('person')
  .out('ORDERED').as('order')
  .select('person', 'order')

Bindings are useful when the result needs information from multiple stages. For simple application output, project properties directly with values.

Repeated traversal

A bounded repeat expresses a repeated graph step:

g.V().hasLabel('Person').has('name', 'alice')
  .repeat(out('FOLLOWS')).times(2).values('name')

This asks for people reached after exactly two outgoing steps. Use a bound that matches the domain question, and inspect the execution plan when integrating a traversal into an application.

SPARQL

Query application tables with RDF vocabulary through the common GraphEngine.

Choose the data model

For a property graph, define an OntologyMapping that maps class and predicate IRIs to graph labels and properties. Use it with GraphEngine::sparql. The older MappedGraphEngine::sparql method remains available on the compatibility facade.

For explicit column, identity, and named-graph mappings, add RdfMapping rules to GraphMapping and call GraphEngine::sparql_query. The RDF guide covers ordinary tables, composite foreign keys, and updates. On mapped engines, existing ontology builders translate to these same rules; patterns do not require an explicit type root.

Define a vocabulary

This vocabulary describes the people and social relationships in the mapped tutorial:

use orchiddb::ir::plan::Direction;
use orchiddb::language::sparql::OntologyMapping;

let ontology = OntologyMapping::new()
    .class("https://example.com/Person", "Person")
    .property("https://example.com/name", "Person", "name")
    .property("https://example.com/age", "Person", "age")
    .relationship_between(
        "https://example.com/follows", "FOLLOWS", Direction::Out,
        "Person", "Person",
    );

The ontology guide explains identity properties and the builder methods.

Select properties

PREFIX ex: <https://example.com/>
SELECT ?name ?age WHERE {
  ?person a ex:Person ; ex:name ?name ; ex:age ?age .
  FILTER(?age >= 30)
}
ORDER BY ?name

The prefix abbreviates IRIs. a expresses an RDF type. Variables begin with ?, and FILTER applies a condition to the matched bindings.

Run the query through a mapped engine:

let result = graph.sparql(
    "PREFIX ex: <https://example.com/> \
     SELECT ?name WHERE { ?p a ex:Person ; ex:name ?name . } ORDER BY ?name",
    ontology.clone(),
).await?;

result.returned.batch contains Arrow data, and result.returned.fields contains output binding names. The ontology convenience method returns legacy scalar columns. For typed RDF terms, use sparql_query(query, dataset); for Arrow RDF term metadata and execution statistics, use sparql_dataset(query, dataset).

Join relationships

PREFIX ex: <https://example.com/>
SELECT ?name WHERE {
  ?person a ex:Person ; ex:name "alice" ; ex:follows ?friend .
  ?friend ex:name ?name .
}
ORDER BY ?name

Shared variables connect patterns. The ontology resolves ex:follows to the mapped FOLLOWS relationship.

Shape a result

Use SELECT DISTINCT to remove repeated projected bindings. Use ORDER BY for result ordering and LIMIT with OFFSET for a result window.

PREFIX ex: <https://example.com/>
SELECT DISTINCT ?name WHERE {
  ?person a ex:Person ; ex:name ?name .
}
ORDER BY ?name
LIMIT 20
OFFSET 0

Query named graphs

RDF datasets can include a graph column. Use GRAPH to choose a named graph and FROM or FROM NAMED to define the active dataset for a query. See RDF datasets for the source mapping and examples.

Ontology mappings

Connect RDF vocabulary to property-graph labels, identities, properties, and relationship directions.

Class mappings

OntologyMapping::class(iri, label) associates a class IRI with a graph label:

let ontology = OntologyMapping::new()
    .class("https://example.com/Person", "Person");

A pattern such as ?p a ex:Person then selects nodes with the Person label.

Subject identity

Use class_with_identity when a graph property provides the externally visible subject:

let ontology = OntologyMapping::new()
    .class_with_identity("https://example.com/Person", "Person", "iri");

Map that graph property to your physical identity column in the graph mapping:

mapping.map_node(
    NodeMapping::table("Person", "people", "id")
        .property("iri", "person_iri")
        .property("name", "full_name")
);

The relational ID identifies the node for graph joins; the identity property supplies the vocabulary-facing subject.

Property predicates

property(iri, domain_label, property) maps a predicate to a graph property:

let ontology = OntologyMapping::new()
    .class("https://example.com/Person", "Person")
    .property("https://example.com/name", "Person", "name")
    .property("https://example.com/age", "Person", "age");

The graph mapping resolves name and age to physical columns. This keeps the RDF vocabulary independent of database column names.

Relationship predicates

Use relationship_between to make both endpoint labels explicit:

use orchiddb::ir::plan::Direction;

let ontology = OntologyMapping::new()
    .class("https://example.com/Person", "Person")
    .class("https://example.com/Order", "Order")
    .relationship_between(
        "https://example.com/ordered", "ORDERED", Direction::Out,
        "Person", "Order",
    );

The direction defines how a subject-to-object triple follows the graph relationship. Use Direction::In for a vocabulary predicate that follows the inverse graph direction.

relationship(iri, rel_type, direction) configures the predicate without explicit endpoint labels. Use the endpoint-aware builder when those labels are known in your vocabulary.

Reuse one vocabulary

Pass the same ontology to managed or mapped graph SPARQL methods. This allows application queries to share vocabulary while the graph mapping determines physical storage.

Shared RDF catalog

For mapped storage, GraphEngine::sparql(query, ontology) translates ontology builders into the same relational RDF rules used by GraphMapping::map_rdf. To install those rules in a named dataset before engine construction, call ontology.apply_to(&mut mapping, "default")?, then use sparql_query or sparql_dataset. Query patterns need not start with an explicit type assertion. An ontology is optional when rules already map RDF terms directly to columns. See RDF mappings for composite identities and writable row rules.

RDF over application tables

Use GraphEngine for Cypher, Gremlin, and SPARQL over the same application tables. Add RDF vocabulary to GraphMapping: each rule describes a subject, predicate, object, and optional graph. SPARQL patterns resolve to scans and joins of those sources. OrchidDB does not create or populate a triple store. An ontology or explicit rdf:type root is not required for direct column mappings.

The complete Rust example creates a small application table, queries typed RDF terms, updates through SPARQL, reads the change through Cypher in the same transaction, and rolls it back. Use the dependencies in installation.

Map columns and composite identities

Given customers(tenant BIGINT, id BIGINT, name VARCHAR), register its schema and map its name column:

use std::sync::Arc;
use arrow::datatypes::{DataType, Field, Schema};
use orchiddb::engine::GraphEngine;
use orchiddb::ir::rel::mapping::{GraphMapping, NodeMapping, schema_only_provider};
use orchiddb::ir::rel::rdf_mapping::{RdfMapping, RdfTermMapping as Term};

let connection = duckdb::Connection::open("application.duckdb")
    .map_err(|e| e.to_string())?;
let mut mapping = GraphMapping::new();
mapping.register_table("customers", schema_only_provider(Arc::new(Schema::new(vec![
    Field::new("tenant", DataType::Int64, false),
    Field::new("id", DataType::Int64, false),
    Field::new("name", DataType::Utf8, true),
]))));
mapping.map_node(NodeMapping::table("Customer", "customers", ["tenant", "id"])
    .property("name", "name"));
mapping.map_rdf(RdfMapping::table(
    "customers", Term::template("urn:customer:", ["tenant", "id"]),
    "https://example.com/name", Term::literal("name"),
).writable(["tenant", "id"]));
let mut graph = GraphEngine::mapped(connection, Arc::new(mapping))?;
let result = graph.sparql_query(
    "SELECT ?name WHERE { ?customer <https://example.com/name> ?name }",
    "default",
).await?;

Term::iri(column) reads existing IRIs. Term::template(prefix, columns) encodes ordered key components as UTF-8 hexadecimal separated by /: tenant 1, id 7 become urn:customer:31/37. Binary key columns use their original bytes. The encoding is reversible and escapes separators. A null component emits no statement. Blank-node templates add a declared scope.

Literal rules infer RDF datatypes from registered column types. Use the Literal variant to specify a datatype, constant language, or language column. Language tags are normalized to lowercase. Native numeric columns expose their stored value; a text column can preserve an original lexical spelling.

Map relationships

An orders table can expose a composite foreign key directly:

mapping.map_rdf(RdfMapping::table(
    "orders", Term::template("urn:customer:", ["tenant", "customer_id"]),
    "https://example.com/hasOrder", Term::template("urn:order:", ["tenant", "id"]),
).writable(["tenant", "id"]));
mapping.map_rdf(RdfMapping::table(
    "orders", Term::template("urn:order:", ["tenant", "id"]),
    "https://example.com/total", Term::literal("total"),
));

Register the orders schema and these rules before constructing the engine. Partial-null foreign keys emit no relationship. Independent edge tables work the same way: map their source and destination key columns. Registered SQL views can provide joins or computed read projections.

PREFIX ex: <https://example.com/>
SELECT ?name ?total WHERE {
  ?customer ex:name ?name ; ex:hasOrder ?order .
  ?order ex:total ?total .
  FILTER(?total > 100)
}

No type assertion is required. Constant predicates prune unrelated mappings. Compatible composite identity mappings retain native key columns for joins. Variable predicates enumerate applicable rules. Overlapping statements are removed at graph boundaries; query solution multiplicity follows SPARQL.

Default and named graphs

Rules without a graph belong to the default graph. Add .graph(Term::constant("urn:team")) or .graph(Term::iri("graph_iri")) for named graphs. A null mapped graph expression emits no statement. GRAPH ?g, FROM, and FROM NAMED use the same rules. An explicit named-graph registry preserves empty graphs; configure it through RdfDatasetMapping::map_writable_named_graphs and attach it with GraphMapping::with_rdf_mapping before adding further RDF rules.

Updates and transactions

Call graph.sparql_update(update, "default", None).await?. Writable rules declare a complete physical row key. Subject and object identities determine key/FK values; property inserts for a new subject combine into one row insertion. Deleting a property clears its column, deleting an FK relationship unlinks its non-key columns, and deleting an all-key edge or class assertion deletes its row. DELETE/INSERT replaces values in a single row transition, including required columns. Defaults and database constraints still apply.

A scalar column cannot represent two different simultaneous RDF values. To replace a value, delete the old statement and insert the new one in the same request. Ambiguous targets, missing key components, and constraint failures abort the request. No fallback storage is created.

begin, commit, and rollback cover all three query languages. SPARQL writes are immediately visible to Cypher and Gremlin on that connection. A failed explicit transaction must be rolled back. Automatic transactions roll back on errors or cancellation.

Inspect execution and results

sparql_query returns typed SparqlResults: SELECT solutions, an ASK boolean, or CONSTRUCT/DESCRIBE graph triples. sparql_dataset returns QueryResult, with Arrow data in returned.batch and execution statistics in stats. sparql_sql shows generated SQL. into_executor returns the connection for further relational work.

Application-owned quad tables remain a supported source shape. Existing RdfDatasetMapping configurations attach through GraphMapping::new().with_rdf_mapping(mapping). The old RdfGraphEngine API is a compatibility facade delegating to GraphEngine; it owns no separate executor.

Migration from RDF-specific engine construction

New code constructs GraphEngine::mapped(connection, Arc::new(mapping)) once. GraphEngine re-exports SparqlResults and RdfTermValue; callers do not need an RDF-specific engine to decode terms. Existing facades retain their signatures:

The dataset argument selects a declared dataset (usually "default"); it is separate from named graphs selected by GRAPH. sparql_query returns typed terms without the statistics wrapper. sparql_dataset exposes the logical/physical plans, generated SQL, and physical table choices. The ontology convenience method sparql(query, ontology) continues to return legacy scalar columns.

Logical sources and collections

RDF rules can reference the same partition-layout logical sources and expanded collection tables as node and edge mappings. Register physical providers first, then layout definitions, then collection sources. RDF filters participate in physical layout selection; collection parent filters may push beneath Unnest, while element filters remain after expansion. Null mapped terms emit no statement; duplicate statements are removed at RDF graph boundaries.

Use these derived sources for reads. Writable RDF rules should reference the physical row owner and supply a complete key; declaring a derived mapping writable does not make its expansion or layout alternatives writable. See collection mappings and physical layouts.

Execution boundary

SPARQL uses the shared relational DAG and DuckDB transaction owner. It does not create another connection for RDF queries or updates. Remote SERVICE, including SERVICE SILENT, is unsupported. Native maps/JSON are not implicitly RDF sources; use explicit columns, term rules, or supported collection mappings.

Equivalent source representations

RDF rules can use a representation source as their table. The shared relational optimizer can then choose a collection expansion, a SQL derivation, or an equivalent materialized table. sparql_dataset exposes the choice in stats.representation_selections; RDF term construction and graph semantics remain above that choice. Representation sources are read-only even if an RDF rule declares itself writable.

Portable scalar functions

Use fn.<name>(...) to call scalar functions in your queries. These functions cover numbers, text, dates and times, arrays, maps, and records. OrchidDB can execute supported calls in DuckDB or PostgreSQL and evaluate other calls in its query runtime when using mixed execution.

Browse the function catalog for all 165 functions and their DuckDB and PostgreSQL support.

Call a function

In Cypher, use functions in expressions such as the values returned by a query:

RETURN fn.sqrt(9.0) AS root, fn.upper('hello') AS greeting

Pass a property to apply a function to each matching row:

MATCH (d:Document)
RETURN d.title AS title, fn.upper(d.title) AS uppercase_title

Function aliases are supported. For example, fn.char_length and fn.character_length refer to the same function.

The same fn. names are available in SQL expressions used by query-backed mappings and computed relationships. Existing language-specific function names keep their own behavior. This catalog covers scalar functions; aggregates, window functions, and table functions use their existing catalogs.

Choose a function

See the function catalog for the supported argument types and options for each engine.

Where functions execute

OrchidDB checks each call’s arguments before placing it in SQL. All 165 functions have a DuckDB and PostgreSQL SQL path, but some argument types or options require evaluation in OrchidDB. Functions that inspect types or metadata can be resolved while the query is compiled.

For example, a simple, case-sensitive fn.regexp_like pattern such as '[a-z]+' can execute in either database. Patterns with Unicode character classes require evaluation in OrchidDB because the databases interpret those classes differently.

In mixed execution, a call without a compatible SQL mapping runs in OrchidDB’s query runtime. SQL-only compilation reports an error identifying the function, engine, and unsupported case. It cannot use runtime evaluation to complete the query.

SQL support preserves the function’s result types and behavior. Some functions use SQL expressions containing several operations, so their cost can differ from a database’s built-in function.

Compatibility limits

Consult the catalog before relying on a specific SQL overload. Restrictions can depend on the argument types, number of arguments, and constant options.

• Regular expressions: SQL mappings support a case-sensitive subset of constant patterns. Captures, alternation, Unicode classes, lazy quantifiers, and empty matches require native evaluation.
• Arrays: Many SQL mappings support flat arrays of booleans, signed integers, or text. Support for nested arrays and other element types varies by function.
• PostgreSQL values: PostgreSQL does not support NUL characters in text or arbitrary nanosecond timestamp precision. Database value ranges also apply.
• Text: Unicode case conversion can produce multiple output characters for one input character. Padding and translation use grapheme boundaries; edit distance counts Unicode code points.

The function catalog lists the restrictions for each function so you can check whether a call can run in your selected engine.

Function catalog

All 165 portable scalar functions have a DuckDB and PostgreSQL SQL path. The table describes the supported scope for each engine; availability is checked for each call’s argument types and options. See Portable scalar functions for usage and native fallback.

*Lowered functions resolve Arrow schema/query metadata or become casts during compilation. Runtime mappings are specific to arity, argument types, and options. Calls without a matching mapping execute natively; SQL-only compilation rejects them. Every listed function has a native implementation. Aliases resolve to the same identity.

Parameters and values

Bind application input as typed Cypher values and reuse query text across calls.

Bind a read parameter

Parameters are passed in a BTreeMap<String, Value> and referenced by $name in query text:

use std::collections::BTreeMap;
use orchiddb::ir::Value;

let params = BTreeMap::from([
    ("name".into(), Value::String("Alice".into())),
]);
let result = graph.cypher_with_params(
    "MATCH (p:Person) WHERE p.name = $name RETURN p.name, p.age",
    &params,
).await?;

Map keys contain the name without $. GraphEngine and MappedGraphEngine both provide this method.

Bind a write parameter

For a managed graph:

let params = BTreeMap::from([
    ("name".into(), Value::String("Alice".into())),
    ("age".into(), Value::Int(30)),
]);
graph.cypher_with_params(
    "CREATE (:Person {name:$name, age:$age})",
    &params,
).await?;

For a mapped property update, pass the same map shape to cypher_update_with_params.

Choose value types

Use a value type that matches the property being compared or assigned. The source table’s types remain part of the mapped graph contract.

Query text and values

Keep the query structure fixed and bind values separately. This preserves types and avoids constructing query strings from application input. Labels, relationship types, property names, and other structural choices belong in application-controlled query text.

Supply every referenced parameter before executing. Binding happens before query execution, and a missing parameter produces an error identifying the missing input.

Nulls and lists

Use Cypher IS NULL and IS NOT NULL when testing for missing values. For a list parameter, use a list-oriented query expression such as membership:

let params = BTreeMap::from([(
    "names".into(),
    Value::List(vec![
        Value::String("Alice".into()),
        Value::String("Bob".into()),
    ]),
)]);
let result = graph.cypher_with_params(
    "MATCH (p:Person) WHERE p.name IN $names RETURN p.name ORDER BY p.name",
    &params,
).await?;

Arrow results

Read Arrow batches, handle nulls, and inspect execution metadata from graph queries.

Client Arrow results

Language clients return results from your engine, without passing data through the compiler. Rust uses RecordBatchReader, Python a PyArrow reader, Java ArrowResult, JavaScript an async batch iterator, and C++/Elixir the Arrow C Stream interface.

Rust/PyArrow batches retain their buffers after reader closure. Java’s reusable vectors are borrowed until the next batch or close. C++ batches have independent release callbacks; Elixir’s stream pointer is valid only inside its callback. JavaScript follows the producer’s lifetime contract. Always close results on early exit and keep the caller connection alive until result production finishes.

Arrow removes per-cell row conversion from the result boundary, but does not promise zero-copy or streaming execution by the database. See client examples.

Result containers

GraphEngine methods cypher, gremlin, sparql, and sparql_dataset return QueryResult for both managed and mapped storage. Its returned field holds ReturnedBatches with fields and batch. sparql_query instead returns decoded SparqlResults. Only the compatibility facades retain their older direct ReturnedBatches interfaces (MappedGraphEngine reads and RdfGraphEngine::sparql).

A batch is an Arrow RecordBatch: a schema plus equally sized column arrays. Read its schema to inspect each column’s physical Arrow type.

Print rows

Arrow’s display helper is convenient for CLI output and diagnostics:

let batch = &result.returned.batch;
println!("{}", result.returned.fields.join("\t"));
for row in 0..batch.num_rows() {
    let cells = batch.columns().iter().map(|column|
        arrow::util::display::array_value_to_string(column, row)
            .map_err(|error| error.to_string())
    ).collect::<Result<Vec<_>, _>>()?;
    println!("{}", cells.join("\t"));
}

For a compatibility facade returning ReturnedBatches, bind batch to &result.batch instead.

sparql_query returns SparqlResults::Solutions { variables, rows }, Boolean(bool), or Graph(Vec<[RdfTermValue; 3]>). Unbound solution values are None; RDF terms preserve IRI, blank-node, datatype, and language identity. The raw sparql_dataset batch includes __rdf:term:kind:?variable, __rdf:term:datatype:?variable, and __rdf:term:language:?variable columns alongside lexical values. returned.fields lists visible variables. The ontology convenience method sparql(query, ontology) keeps the older scalar-column contract.

Read typed values

When the application expects a string column, check its type and handle nulls:

use arrow::array::{Array, StringArray};

let names = batch.column(0).as_any().downcast_ref::<StringArray>()
    .ok_or_else(|| "expected a UTF-8 name column".to_string())?;
for row in 0..names.len() {
    if names.is_null(row) {
        println!("no name");
    } else {
        println!("{}", names.value(row));
    }
}

Use Int64Array, Float64Array, or other Arrow arrays according to the result schema. Check nullability before reading a nullable value.

Stable application output

Choose aliases in the query to make result fields explicit. Add an order when row ordering matters. Project the properties your consumer needs so downstream code can operate on a small, well-defined schema.

MATCH (p:Person)
RETURN p.name AS name, p.age AS age
ORDER BY name

Inspect execution metadata

For a GraphEngine query, including mapped RDF reads:

println!("backend: {:?}", result.backend);
println!("SQL islands: {}", result.stats.islands);
println!("island rows: {}", result.stats.island_rows);
println!("DataFusion operators: {}", result.stats.datafusion_ops);
println!("logical plan: {}", result.stats.logical_plan);
println!("physical plan: {}", result.stats.physical_plan);
println!("table choices: {:?}", result.stats.layout_selections);

The backend is DuckDb, Hybrid, or DataFusion. See execution and plans for how the read policy selects the execution path.

Execution and plans

Inspect Graph IR and generated SQL, choose a read policy, and understand execution statistics.

Managed read policies

GraphEngine defaults to ReadMode::Hybrid. It lowers Graph IR to a DataFusion relational DAG, executes eligible SQL regions in DuckDB, and schedules native kernels through DataFusion. To require a complete SQL read, set ReadMode::SqlOnly:

use orchiddb::engine::ReadMode;

graph.set_read_mode(ReadMode::SqlOnly);
let result = graph.cypher(
    "MATCH (p:Person) RETURN p.name"
).await?;

SqlOnly applies to managed-storage reads. Mapped execution currently uses the shared DAG path regardless of this setting; it lowers source reads to SQL islands and schedules remaining operators through DataFusion. RDF dataset queries use the same mapped execution path.

Inspect Graph IR

The legacy core CLI’s --explain flag parses and plans a query, then prints Graph IR:

cargo run --features duckdb --bin orchiddb -- --explain --query \
  "MATCH (p:Person)-[:KNOWS]->(friend) RETURN p.name, friend.name"

Use this to inspect scans, expansions, filters, and projections. Explain mode does not execute the query or open a database.

From Rust, build a plan through the language frontend and format it:

use orchiddb::language::cypher;

let parsed = cypher::parse_query(
    "MATCH (p:Person) RETURN p.name"
).map_err(|error| error.to_string())?;
let plan = cypher::CypherPlanner::new().plan(&parsed)
    .map_err(|error| error.to_string())?;
println!("{}", orchiddb::ir::explain(&plan));

Inspect generated SQL

For SQL inspection without execution, use compiler::compile or compile_json and inspect CompiledSql.sql and logical_plan. The existing MappedGraphEngine compatibility facade also offers explain_cypher (the following snippet uses that facade):

let sql = graph.explain_cypher(
    "MATCH (p:Person)-[:ORDERED]->(o:Order) \
     WHERE o.total > 100.0 RETURN p.name, o.total"
).await?;
println!("{sql}");

Graph property names resolve to the physical columns in your mapping. Use this output when reviewing source joins, filters, and query-backed definitions.

Execution metadata

GraphEngine query results identify their backend for managed or mapped storage:

ExecStats records datafusion_ops, islands, island_rows, sql_queries, native_source_queries, native_source_rows, and cost. It also exposes logical_plan, physical_plan, constraint_proofs, layout_selections, and representation_selections. The latter records the chosen derived or materialized input, candidate costs, and decision reasons. Use fully_pushed_down() to inspect whether all recorded query operators were delegated to SQL. Backend labels alone do not establish that.

Logical plans show selected physical tables and collection Unnest operators. Layout estimates describe parent-table bytes/files; they are not measured I/O or expanded collection cardinalities. SQL island rows are intermediate rows; use the result batch for the number of returned rows.

For RDF, sparql_dataset returns these statistics in QueryResult.stats. sparql_query decodes typed terms and does not return the stats wrapper. sparql_sql(query, dataset) produces SQL without executing the query.

SQL timeout

use std::time::Duration;
graph.set_sql_timeout(Duration::from_secs(10));

This setting applies to individual DuckDB queries and setup work. Treat the SQL timeout as one part of your application’s overall request budget.

Planner and executor boundaries

Language planners produce GraphPlan values. RelBackend handles relational lowering; SQL preparation turns the lowered plan into a dialect-specific program. SqlExecutor supplies the SQL execution boundary. Managed execution compiles to a DataFusion relational DAG with explicit DuckDB regions and native kernels.

Graph IR is retained for planning and inspection. It has no standalone recursive interpreter, and execution errors do not trigger an interpreter fallback. SPARQL SERVICE, including SERVICE SILENT, is unsupported; no remote SPARQL HTTP calls are made.

Use the engine APIs for application workflows. Use these lower-level types when integrating a custom planner or execution target. See the Rust API for the module map.

SQL generation and execution

orchiddb::compiler is the database-free entry point for clients that own their SQL engine. It parses Cypher, Gremlin or mapped SPARQL, lowers through Graph IR and relational IR, and returns SQL plus output field names. It never opens a connection, registers functions, mutates a schema, or executes a SQL statement.

[dependencies]
orchiddb = { path = "../orchiddb" }

All language clients use the same compiler. The Java binding exposes it through JNI. Its JNI library uses this configuration: DuckDB and libduckdb-sys are absent from the native dependency graph. Existing managed engine/CLI builds retain their optional duckdb feature; it must now be enabled explicitly.

Versioned request

compile(CompileRequest) is the typed Rust API. compile_json(&str) provides the same API for language bindings. Both return errors before client execution. The JSON protocol is version 1; unknown fields and unsupported versions fail.

{
  "version": 1,
  "dialect": "duckdb",
  "language": "cypher",
  "query": "MATCH (p:Person) WHERE p.name=$name RETURN p.name AS name",
  "parameters": {"name": "Ada"},
  "tables": [{
    "name": "\"people\"",
    "columns": [
      {"name": "id", "data_type": "int64", "nullable": false},
      {"name": "name", "data_type": "string", "nullable": true}
    ]
  }],
  "nodes": [{
    "label": "Person", "table": "\"people\"", "id": "id",
    "properties": {"name": "name"}
  }]
}

let response = orchiddb::compiler::compile_json(request_json).await?;
// Response contains version, dialect, sql, fields, logical_plan,
// constraint_proofs, and layout_selections.
// The client decides whether, where and how to execute that SQL.

Table names use SQL identifier syntax, including quoted qualified identifiers. Bindings must quote individual identifier parts. Column/property names are literal strings. Schemas contain metadata only; no source data is passed into the compiler. Mapped identities may use a non-null scalar or an ordered tuple of non-null scalars: booleans, integers, floats, strings, binary, decimals, dates, times, timestamps, durations and intervals. Floating-point keys follow the target engine’s equality semantics. Each label retains its native component types. Heterogeneous scans and joins use typed struct variants to preserve types and component boundaries; label equality accompanies identity comparisons. Node id and edge id, source, and target accept either a column-name string or an ordered array of column names, for example "id": ["tenant", "id"]. Endpoint columns correspond positionally to the referenced node key. Arrays must be nonempty with distinct column names; a one-column array is a scalar key. Bindings must ensure ID uniqueness/non-nullability and referential integrity in their actual data. Mapped identities do not require a physical primary-key index; supported index types depend on the target database.

Optional edges describe label, table, id, source/target columns, source_label/target_label, and property-to-column mappings. An FK-backed edge also sets foreign_key to "src" or "dst" (the endpoint whose row owns the FK), uses the child table, and supplies the child key as its id. Partially NULL FKs do not produce graph edges. These declarations describe reads in this API; compile does not execute mutations. Gremlin tuple-ID projection may require the unified runtime’s native operator rather than standalone SQL compilation.

Optional computed_relationships define expression-based edges with predicates, score properties, per-source ranking, and candidate stages. source_metadata describes physical formats, options, and indexes for the owning engine to lower. Same-engine PostgreSQL search stays in a correlated SQL island; dependent search transfers include a template that clients bind with op: "bind_operation" before the ordinary bind operation. See the search reference for the protocol and generated SQL, or Search for an overview.

With the optional quickwit and elasticsearch features, engine descriptors may select these non-SQL adapters. Their transfers carry typed structured request operations. The execution_engine remains a SQL engine for graph joins and expressions around those operations. See Quickwit and Elasticsearch for configuration and execution; endpoints and credentials remain in caller-owned sessions, outside the compiler request.

Optional functions describe language-facing name, SQL target, parameter type list, return type (returns), and aggregate flag. The caller installs real implementations in its engine. SPARQL’s optional ontology maps classes, properties and directed relationships; see the public request structs for the complete contract.

Supported schema types are boolean, int8/int16/int32/int64, uint8/uint16/uint32/uint64, float32/float64, string, binary, json, date, time and duration (microseconds), interval (month/day/nanosecond), timestamp (microseconds, no timezone), and decimal:precision:scale with precision up to 38. Native list and struct schema declarations use list:<element-type> and struct:<JSON field-to-type object>, for example list:string or list:struct:{"sku":"string","quantity":"int64"}. These type strings recurse. The json type describes a document column, stored as PostgreSQL JSONB or DuckDB JSON; see JSON documents and relationships for functions and collection mappings. Collection sources can compose to expand nested lists or documents. Unsupported source types require a caller-owned cast/view. Unsupported SQL dialects fail explicitly; DuckDB and PostgreSQL rendering are currently implemented.

Permission filtering

The JSON protocol can filter mapped nodes against an application-owned flat grants relation. A top-level authorization identifies the current subject. Each node permission_scopes entry names the node source column to compare and a grants relation with its resource type and permission. The compiler emits the membership filter at the node scan, before graph traversal. Multiple scopes combine with OR, which supports direct object grants alongside project or tenant grants.

{
  "authorization": {"subject_type":"user", "subject_id":"alice"},
  "nodes": [{
    "label":"Document", "table":"documents", "id":"id",
    "properties":{"title":"title", "project_id":"project_id"},
    "permission_scopes":[
      {"resource_column":"id", "relation":{"table":"effective_grants", "resource_type":"document", "permission":"view"}},
      {"resource_column":"project_id", "relation":{"table":"effective_grants", "resource_type":"project", "permission":"view"}}
    ]
  }]
}

Include the grants source and all its columns in tables. Flat relation column names default to resource_type, resource_rel, resource_id, subject_type, subject_rel, and subject_id; callers can override every column name. Scope columns must exist on their mapped node. Requests with protected nodes and no principal fail closed. The relation must already contain effective grants for the principal, including any group or nested-set resolution. The compiler and clients do not connect to or synchronize a permission system.

Collection-backed logical tables

The optional collection_sources array defines named relations by expanding a native list column. Each definition supplies name, parent table, list column, optional parent_columns (output alias to parent column), and either element (a scalar element’s output name) or fields (output alias to struct field). Node, edge, and RDF mappings reference the resulting relation by name.

{
  "collection_sources": [{
    "name": "person_tags",
    "table": "people",
    "column": "tags",
    "parent_columns": {"person_id": "id"},
    "element": "tag"
  }]
}

This is a request fragment; declare people.id and people.tags in tables, using list:string for the tags column. No database view is created or required. For a full struct-list mapping, identity rules, and limitations, see collection columns as logical tables.

The compiler registers physical schemas, then optional logical_sources (physical-layout alternatives), then collection and representation sources in dependency order. Parent filters can guide layout selection below UNNEST; element filters apply after expansion. logical_plan exposes the expansion and layout_selections reports physical choices and parent-scan byte/file estimates. Those estimates do not describe expanded row counts. Include collection and layout definitions and their metadata in plan cache keys. These definitions do not enable writes.

Equivalent derived and materialized inputs

The optional representation_sources array registers multiple complete, equivalent inputs for one mapped relation. Inputs can be named tables/collections or SQL queries, including joins and grouped summaries. The optimizer chooses per occurrence after pushing filters, and exposes its decisions in representation_selections, alongside layout_selections and logical_plan. The shared engine uses the same selection and exposes it in execution statistics.

Boundaries

The API specializes SQL to typed Cypher parameter values. It does not return JDBC placeholders. Bindings must treat generated SQL as potentially sensitive and include parameters, schema, ontology, function declarations, mappings, dialect, engine routing, authorization principal, and permission scopes in their plan cache keys. Gremlin/SPARQL parameter bindings are not yet implemented. Conformance evidence applies to its recorded API and revision; SQL emission must also fit the target dialect.

Writes, opaque extensions, and unlowerable operations fail. SPARQL SERVICE, including SERVICE SILENT, is unsupported in both compiler and managed runtime; OrchidDB does not make remote SPARQL HTTP requests. Internal table materializations are rejected by inspecting the logical plan before unparsing; they are never collected or executed. Constant seed rows in mapped plans are emitted as SQL expressions instead of private tables.

Regression tests: cargo test --test sql_compiler --test execution --test federation.

Cross-engine reads

Current source builds accept an engines registry, an execution_engine, and an engine on each table. Engine IDs name caller-owned connections; supported dialects are duckdb and postgres. Tables without an engine use the execution engine, whose dialect must match the request.

The compiler pushes eligible source subtrees into SQL islands, including filters, projections, same-engine joins, and aggregates. Cross-engine joins execute on the selected execution engine. CompiledSql.transfers describes each source query and its typed result columns.

Use a federation helper, such as Rust’s federation::execute or Java’s FederatedQuery.query, to execute these plans. It buffers source results and binds them as typed common table expressions in the final query. It creates no tables or views. Large transfers increase memory use and SQL statement size. Connections keep their own snapshots and transaction policy; this read interface provides no distributed writes or snapshot.

See the mixed-engine request and client APIs for a complete configuration.

Caller-owned data flow

For a plan without transfers, you can execute CompiledSql.sql directly. For a common adapter boundary, implement execution::SqlSession: declare a dialect, a driver error type, and a result type implementing Arrow RecordBatchReader that can borrow the session. Its async query method executes the SQL; execution::execute first rejects protocol/dialect mismatches. It performs no schema discovery, setup statements, data copying, buffering or commits. Futures may stay on the calling thread; Send and background scheduling are not required. Your adapter controls streaming errors, cancellation and drop cleanup.

The DuckDB application example declares its own driver dependency and implements the interface with a borrowed connection and native Arrow reader. Retained Rust batches own their buffers after advancing or dropping the reader. It registers a SQL function, queries uncommitted caller data, and verifies caller rollback afterward:

cargo run --manifest-path examples/duckdb-client/Cargo.toml

For SQL output alone, with no DuckDB build:

cargo run --example compile_sql

Resolve metadata and compile against the same session/schema version used for execution. The dialect check cannot distinguish two databases using the same SQL dialect; engine identity and routing belong to your application.

RDF rules in compiler requests

SPARQL requests accept rdf rules and an optional dataset (default default). The same tables, nodes, and edges catalog can serve all three languages. For example, a rule over registered customer columns is:

{"table":"customers",
 "subject":{"kind":"template","prefix":"urn:customer:","columns":["tenant","id"]},
 "predicate":{"kind":"constant","value":"https://example.com/name"},
 "object":{"kind":"literal","column":"name"}}

Term kinds are iri (column), template (prefix and ordered columns), blank (scope and ordered columns), literal (column with optional datatype, language, or language_column), and constant (value with optional datatype or language). Rules may specify graph and dataset. Runtime writes additionally require writable: true and a complete key array. The compiler remains read-only. Existing ontology declarations translate into the same relational rules.

Requests using rdf retain typed RDF output: each variable’s lexical column is accompanied by __rdf:term:kind:?variable, __rdf:term:datatype:?variable, and __rdf:term:language:?variable. A null kind means unbound. fields lists the visible variable names. Legacy ontology requests retain their scalar result columns and native numeric types.

Engine adapters

An engine adapter translates OrchidDB’s typed relational plan into operations its engine can execute. SQL adapters rewrite expressions and relations into SQL; request adapters produce typed structured requests. Both use the same federation planner and value exchange. Implementations are Rust code; configuration selects a registered implementation and supplies source facts.

Search uses this same extension point. There is no separate search-backend registry. The Search guide shows pgvector and duckdb-lance source metadata and relationship declarations.

Register an engine

Implement orchiddb::ir::rel::sql::DialectAdapter, also exported as EngineAdapter. Required methods are name, parser_dialect, unparser_dialect, and sql_type. Supply exchange_literal for typed inputs crossing islands or binding dependent operations. Methods return explicit errors for unsupported types and operations.

Register a static implementation before compiling queries:

use orchiddb::ir::rel::sql::SqlDialect;

static WAREHOUSE: Warehouse = Warehouse;
let dialect = SqlDialect::register(&WAREHOUSE)?;

Warehouse is your adapter type. Its name() becomes the compiler’s dialect value, including engine descriptors in a federated request. Runtime connections remain caller-owned and implement the existing session interface. Registering the same adapter instance twice is allowed; replacing a registered name is not.

The complete tested adapter implements a third dialect, scalar mappings, AST rewriting, a table-function transformation, and typed input binding without modifying core dispatch.

For an engine with a non-SQL protocol, implement orchiddb::operations::RequestAdapter and register it with operations::register(Arc::new(adapter)). lower returns a PreparedOperation with an optional source plan, a RequestTemplate, and an Arrow output schema. The optional owner hook identifies an explicitly selected engine instance. Returning None allows the planner to consider smaller valid islands; a recognized invalid operation returns an error.

Request templates carry adapter-owned JSON and typed parameter bindings at JSON pointer locations. User values are inserted as values, never interpolated into query text. An operation may provide a relational continuation containing a typed RequestResult leaf. For example, a remote index returns document keys and scores, then a continuation joins those keys to authoritative SQL properties. The planner checks schemas and replaces the leaf with the exchanged relation.

The optional Quickwit and Elasticsearch adapters use this interface. The final execution engine is still a SQL engine, which executes the remaining joins and expressions. A request engine is never registered as a pretend SQL dialect.

Transform expressions and relations

Use function_mapping to provide a logical function’s SQL expression and ordering mapping. Use rewrite_expression and rewrite_query for SQL AST transformations that need code. Custom dialects do not inherit DuckDB function implementations: unsupported functions must be rejected or explicitly mapped.

For a row-producing operation, implement lower_relation. This example method renames a logical table function and delegates SQL construction to the shared relational lowering:

fn lower_relation(
    &self,
    plan: &datafusion::logical_expr::LogicalPlan,
    context: &orchiddb::ir::rel::sql::lowering::LoweringContext,
) -> orchiddb::ir::rel::sql::SqlResult<
    Option<orchiddb::ir::rel::sql::lowering::RelationLowering>,
> {
    use datafusion::logical_expr::LogicalPlan;
    use orchiddb::ir::rel::{
        dependent::TableFunction,
        sql::{SqlError, lowering::builtin_lower_relation},
    };
    if let LogicalPlan::Extension(extension) = plan {
        if let Some(function) = extension.node.as_any().downcast_ref::<TableFunction>() {
            if function.name != ["expand_numbers"] {
                return Err(SqlError::Unsupported("warehouse table function".into()));
            }
            let mut physical = function.clone();
            physical.name = vec!["warehouse_range".into()];
            return builtin_lower_relation(&physical.into_plan(), context);
        }
    }
    Ok(None)
}

The return value can be any of:

Rewrites must preserve the output schema. Dependent templates must preserve the output schema, use the owning dialect, and bind one parameter per source column. For custom extension nodes with multiple inputs, relation_placement identifies which source and target inputs determine ownership. Ordinary relational plans continue through the existing SQL-island planner.

Table functions and binding

TableFunction::new takes a multipart name, typed Expr arguments, an optional source plan, an ArgumentBinding, and the function’s output DFSchemaRef. Its resulting schema contains source columns followed by function output columns. Arguments are checked against the source schema. Source and output column names must be distinct; project unique aliases before constructing the function when joined inputs contain duplicate names.

• ArgumentBinding::Correlated allows the engine to consume source columns inside the same statement.
• ArgumentBinding::PrepareTime requires values before the engine prepares the statement, so lowering creates a dependent operation.

LoweringContext.mode can also request a bound operation when ownership crosses engines. SqlTemplate::bind uses typed values and the registered dialect’s literal codec. Nulls, empty collections, numeric types, and nested values need explicit encodings; unsupported representations fail.

Predicates remain above a table-function boundary until an adapter explicitly implements a valid pushdown. Moving a filter after a top-k operation changes its meaning; accepting a pushdown means preserving its placement and semantics.

Search and source metadata

GraphMapping::register_source_metadata and the compiler’s source_metadata array describe each table’s physical format, source options, and indexes:

[[source_metadata]]
table = "documents"
format = "vendor_collection"
options = { collection = "articles" }
indexes = [{ column = "embedding", metric = "cosine", options = { access_method = "vendor_ann" } }]

Custom format and option names are retained for the owning adapter to validate. Use GraphMapping::try_to_toml() when serializing arbitrary source options. JSON null options remain valid metadata but cannot be represented in TOML; this method returns an error instead of dropping them. The legacy to_toml() method retains its infallible signature and panics on unrepresentable metadata. A search transformation inspects RankedJoin: it carries query/document expressions, eligibility, per-source keys, score ordering, limit, retrieval requirements, and the selected source/index metadata. Return the engine’s index-usable SQL AST or a dependent operation. Reject unsupported metrics, filters, or retrieval requirements explicitly. PostgreSQL’s rule uses distance ordering; DuckDB’s Lance rule emits bound search calls.

JSON functions and relationships use these same interfaces: typed scalar expressions use AST rewrites, collection expansion uses relational nodes, and nested document values cross engine boundaries through domain codecs. A future domain such as geometry can provide its own native functions, SQL transformations, and value codecs without adding a separate execution backend.

Execute dependent operations

Compiled transfers expose an optional operation descriptor with its owner engine, input columns, and SQL template. federation::execute handles these through caller-owned sessions. JSON clients execute source SQL, call op: "bind_operation" with the source rows, execute returned statements on the specified engine, and feed output rows into the ordinary bind operation.

Non-SQL transfers expose request instead. bind_operation returns structured requests for these operations, while retaining sql for existing SQL operations. A closed request has no source SQL and executes once. A correlated request binds each source row. Sessions implement execute_request and may override execute_requests to batch execution without changing row semantics. Returned Arrow batches must match the declared transfer schema.

The former search descriptor, bind_search command, and search_indexes configuration are accepted as compatibility inputs. New plans use the general operation names. Execution errors propagate without retrying a different access method.

Client APIs

Map your existing tables, compile a Cypher, Gremlin, or mapped SPARQL read query to SQL, then consume native Arrow batches from your own engine. Normal compilation receives schema and mapping metadata. Explicit statistics generation additionally sends bounded source samples to the shared statistics builder. Clients do not bundle DuckDB.

Version 0.1.0 is published on crates.io, PyPI, npm, Maven Central, and Hex, with C++ binaries on GitHub Releases. See the package links and installation commands. Each repository includes working source examples; packaged native compilers currently target macOS ARM64.

Shared execution contract

1. Your application opens/configures the database, plugins, UDFs, transactions, and caches.
2. Supply table schemas and graph mappings to the version-1 compiler.
3. Compile a supported read query into SQL and output field names.
4. Execute through a compatible engine or federation helper and consume its result.
5. Release result resources; the connection remains yours.

Current source builds support DuckDB, Postgres, and cross-engine reads through caller-owned connections. The compiler emits source SQL islands and a final query for the selected execution engine. Federation helpers buffer source results and bind them as typed SQL values; they create no database tables and provide no distributed snapshot or transaction. See cross-engine reads. ClickHouse SQL is not implemented.

Parameters are specialized into SQL. Cache keys must include their values, schema, node/edge/RDF mappings, collection definitions, physical layouts and partition metadata, ontology, functions, dialect, engine routing, authorization principal and permission scopes, and installed statistics revision. Recompile after these change. Gremlin/SPARQL parameter bindings are not implemented. Generated SQL must fit the target dialect. Consult the recorded conformance results for the exact APIs and revisions tested.

Composite keys in source builds

The core compiler accepts ordered column arrays for node IDs, edge IDs, and endpoints; see SQL compilation and the composite mapping example. Existing single-column mappings remain valid. Cypher and Gremlin endpoint joins preserve all components and exclude partially NULL foreign keys.

Use a compiler built from the composite-key sources when sending column arrays. The published 0.1.0 packages and standalone example manifests select the released compiler. Use a matching source revision for new request fields such as rdf, logical_sources, collection_sources, representation_sources, and statistics; older packaged compilers may reject them. API availability is determined by the loaded library revision.

Rust

Repository and example. Add the client to your application:

cargo add orchiddb-client@0.1.0 --no-default-features

SqlSession declares a dialect and returns an Arrow RecordBatchReader. Its result may borrow the session. Retained batches own reference-counted buffers after reader drop. The driver is your dependency.

let plan = orchiddb_client::compile(request).await?;
let mut batches = orchiddb_client::execute(&mut your_session, &plan).await?;
for batch in &mut batches {
    let batch = batch?;
    // Read typed Arrow arrays from batch.
}

In a client checkout, run cargo run --manifest-path examples/Cargo.toml --features bundled for mappings, a caller-defined function, a borrowed connection, and rollback. The standalone example resolves OrchidDB 0.1.0 from crates.io. The example owns its DuckDB dependency; --features bundled builds that driver.

Java

Repository · Arrow guide.

Use Java 17+ and add both published Maven dependencies:

<dependency>
  <groupId>com.orchiddb</groupId>
  <artifactId>orchiddb-java</artifactId>
  <version>0.1.0</version>
</dependency>
<dependency>
  <groupId>com.orchiddb</groupId>
  <artifactId>orchiddb-java</artifactId>
  <version>0.1.0</version>
  <classifier>macos-aarch64</classifier>
  <scope>runtime</scope>
</dependency>

var compiler = io.orchiddb.NativeSqlCompiler.load();

Maven resolves the compiler JAR; load() verifies, extracts and loads it automatically. No manual binary download, library path, Rust toolchain or source checkout is needed. Version 0.1.0 includes macOS ARM64 JVMs only; other platform compiler JARs are not published for this version.

Run ./scripts/run-example.sh ArrowBatches from a Java client checkout. Its standalone example POM downloads the published Maven packages and requires no Rust checkout. Run ./scripts/run-gremlin-example.sh for the optional Gremlin module.

For a complete application example, see ArrowBatches.java.

JdbcEngine.withArrow takes your JDBC connection, Arrow allocator, and driver export callback. Graph.queryArrow returns an ArrowResult. Java vectors are borrowed: consume them before advancing or closing the result. Copy/transfer data explicitly if it must outlive that scope. Your parent allocator and connection remain caller-owned.

Java requires an Arrow-compatible JVM setup, including --add-opens=java.base/java.nio=ALL-UNNAMED; the example launcher handles it. Supply your own JDBC driver and Arrow memory implementation as described in the Arrow guide; DuckDB JDBC is test-only in the client repository. The optional com.orchiddb:orchiddb-gremlin:0.1.0 dependency adds fluent TinkerPop traversal integration; other clients accept Gremlin text.

Current Java sources also expose PermissionRelation, NodeMapping.protectWith, and Authorization for permission filtering. Applications supply effective grants from their chosen permission system; OrchidDB applies them before traversal and projection.

Python

Repository and complete example. Install from PyPI:

python -m pip install "orchiddb[arrow]==0.1.0"

To run the checked-in example in a fresh virtual environment, use python -m pip install -r examples/requirements.txt followed by python examples/people.py.

The published wheel bundles the compiler for macOS 26+ on Apple Silicon. Supply your own database driver. Other platforms require a source build.

from orchiddb import Compiler, DuckDBEngine, Graph
# connection, tables, and nodes are supplied by your application.
graph = Graph(Compiler(), DuckDBEngine(connection), tables=tables, nodes=nodes)
with graph.query_arrow("MATCH (p:Person) RETURN p.name AS name") as reader:
    for batch in reader:
        print(batch.num_rows)

The context closes the reader, not the connection. Retained PyArrow batches own their buffers. Do not reuse the borrowed connection while the result is open. Implement ArrowEngine for another Arrow-capable backend. Release workflows build platform wheels for PyPI.

JavaScript / TypeScript

Repository. Node.js 20+:

npm install @orchiddb/client@0.1.0 apache-arrow@17

From a client checkout, run cd examples && npm ci && npm start. Its package.json installs the published client and the application-owned DuckDB-Wasm driver.

Apache Arrow is an optional peer dependency needed for the Arrow interfaces and TypeScript declarations. For compilation alone, install only @orchiddb/client.

The published package includes the macOS ARM64 compiler. Compiler.compile(request) returns a SQL plan. Your ExecutionEngine.execute(plan) returns a schema, async Arrow batch iterator, and close() method. The batches(result) helper closes resources on completion, failure, and early exit.

The example uses DuckDB-Wasm’s native Arrow stream. The binding itself runs in Node.js; it is not a browser compiler. Compilation is synchronous, so use a worker for latency-sensitive services. Pass exact signed 64-bit parameters as bigint, for example 9007199254740993n. Unsafe integer Numbers, non-finite Numbers, and bigint values outside signed int64 are rejected, including inside nested parameters. Follow the producer’s batch lifetime contract. Version 0.1.0 is available on npm as @orchiddb/client.

Elixir

Repository. With Elixir, Erlang headers, make, and a C compiler:

cd examples
mix deps.get
# Set ORCHIDDB_NATIVE_LIBRARY to the downloaded compiler library.
mix run duckdb.exs

The standalone example uses Hex package orchiddb 0.1.0. Download the macOS ARM64 compiler, extract it, and point ORCHIDDB_NATIVE_LIBRARY at its lib/liborchiddb_compiler.dylib. Other platforms require a source build. OrchidDB.compile(request) returns {:ok, plan} or {:error, reason}. The C NIF runs compilation on a dirty CPU scheduler.

Optional OrchidDB.query_arrow(connection, request, callback) uses ADBC. Its Arrow C Stream pointer is valid only inside the callback and must not escape it. The connection stays caller-owned. Integration tests execute compiled queries through the real DuckDB ADBC driver and verify native Arrow ingestion, large integers, nulls, caller rollback, and cleanup after consumer errors. Add {:orchiddb, "~> 0.1.0"} to your Mix dependencies to install the published Hex package. The native compiler is still required separately.

C++

Download the 0.1.0 macOS ARM64 binary package (SHA-256 checksums). It includes C++17 headers, CMake configuration, the JSON dependency, and the compiled native library. No Rust build is required. Other platforms can build from source.

Extract the archive and pass its directory as -DCMAKE_PREFIX_PATH=/path/to/extracted/package. In your application’s CMakeLists.txt:

find_package(OrchidDB CONFIG REQUIRED)
target_link_libraries(your_app PRIVATE OrchidDB::orchiddb)

Set ORCHIDDB_NATIVE_LIBRARY to the extracted lib/liborchiddb_compiler.dylib before running your application.

ExecutionEngine supplies an ArrowResult; stream, schema, and batch use separate move-only RAII wrappers and release callbacks. Batches can outlive the stream.

The DuckDB adapter and integration example show ownership and rollback. To run against the downloaded package, configure cmake -S examples -B examples/build with CMAKE_PREFIX_PATH pointing to the extracted archive and DUCKDB_INCLUDE_DIR / DUCKDB_LIBRARY pointing to your DuckDB installation. Then run cmake --build examples/build and ctest --test-dir examples/build --output-on-failure. The published CMake archive includes the native compiler.

Data lake access and performance

The CLI bundles DuckDB and loads its official Iceberg extension by default. Embedded clients leave catalog, credentials, extensions, and SQL views to the application. Map a view over iceberg_scan like any other table.

Arrow avoids per-cell row-object conversion at the interface. It does not guarantee zero-copy, bounded memory, streaming database execution, or cancellation support for every driver. See Arrow results for lifetime rules and GraphEngine result containers.

Rust API

Find the public engine methods, result types, mapping builders, and planner entry points.

SQL generation

Use compiler::compile or compiler::compile_json to return SQL without a database driver. The optional execution::SqlSession interface accepts caller-owned sessions and result types. See SQL compilation for runnable examples.

The engine APIs below require features = ["duckdb"].

GraphEngine

Import from orchiddb::engine. Cypher, Gremlin, ontology SPARQL, and sparql_dataset methods are asynchronous and return Result<QueryResult, String>; typed RDF queries return SparqlResults. Lifecycle methods are synchronous.

QueryResult contains returned: ReturnedBatches, backend: ExecutionBackend, and stats: ExecStats. stats.logical_plan, stats.layout_selections, and stats.representation_selections show the chosen inputs.

MappedGraphEngine compatibility facade

New applications use GraphEngine::mapped. For existing callers, import from orchiddb::mapped_engine and construct with MappedGraphEngine::new(executor, Arc::new(mapping)).

Read methods return Result<ReturnedBatches, String>. The mapped tutorial demonstrates source schema registration and engine construction.

SPARQL on GraphEngine

RdfGraphEngine remains a delegating compatibility facade. New applications use GraphEngine::mapped with RDF rules on GraphMapping. See RDF mappings.

Data and mapping types

Planning and execution APIs

Graph IR is lowered before execution. The shared execution runtime uses DataFusion physical operators and native kernels; there is no standalone Graph IR interpreter or legacy island target API.

Generate symbol documentation

Build the crate’s Rust documentation locally to browse complete signatures and type definitions:

cargo doc --locked --no-deps

Open target/doc/orchiddb/index.html in a browser. Enable optional features in the Cargo command when documenting their types.

Error handling

Engine methods use Result so the application can handle failures at each boundary. Add application context when logging a failure, preserve the engine’s error message, and roll back an explicit transaction before retrying conflicting work. See transactions for the control flow.

CLI reference

The standalone OrchidDB-cli bundles DuckDB and loads the official Iceberg extension by default. It compiles graph reads to SQL and returns Arrow results.

Install

Install the released v0.1.0 CLI on macOS ARM64 (Apple Silicon):

curl -fsSL https://install.orchiddb.com | bash
export PATH="$HOME/.local/bin:$PATH"
orchiddb --version

See installation for pinned versions, direct downloads, and source builds. The quickstart provides complete input files and runs with the installed binary, without a repository checkout.

Invocation

orchiddb compile REQUEST.json
orchiddb query REQUEST.json [--database FILE] [--init SQL_FILE] [--format arrow|table] [--no-iceberg]
orchiddb --version

REQUEST.json uses compiler protocol version 1, including language, query, dialect, table schemas, and graph mappings. The CLI executes DuckDB SQL only. compile prints SQL without opening a database or installing extensions.

Query options

Use space-separated option values. The request is a file, not inline query text. This binary does not accept the legacy managed CLI’s --query, --language, or --explain flags.

Run a query

Save people.json and setup.sql from the quickstart, then run:

orchiddb query people.json --init setup.sql --format table --no-iceberg
orchiddb compile people.json
orchiddb query people.json --init setup.sql --no-iceberg > people.arrow

The setup creates source tables and the request maps them to a Person label. Arrow IPC goes to stdout; errors go to stderr. A failure returns a nonzero exit status. See the quickstart for complete input files.

Iceberg and engine setup

Queries first load the cached Iceberg extension or install the signed official extension from DuckDB’s extension service. Installation failure is reported; there is no silent fallback. Use --no-iceberg for queries that do not need Iceberg.

Your initialization file can configure credentials, plugins, UDFs, attached catalogs, and views:

CREATE VIEW people AS
SELECT id, name FROM iceberg_scan('/path/to/table/metadata/v1.metadata.json');

Map that view in your request. Actual storage access requires your own credentials and catalog setup. Setup SQL is executed as supplied. The CLI owns its database connection; applications that already own an engine should use a language client.

Managed runtime CLI

Core retains a separate legacy binary behind --features duckdb, used by some managed graph examples. Its mutation/query flags are different. The CLI described here executes generated read SQL; it does not expose graph mutations. Match command usage and conformance evidence to the executable revision.

Conformance comparison

Compare Cypher with Neo4j Community, PuppyGraph and ArcadeDB, Gremlin with SQLg, PuppyGraph, JanusGraph and ArcadeDB, and SPARQL with Apache Jena. The recorded percentages apply to the exact interfaces, revisions, and fixtures listed in the evidence. They are not a blanket capability guarantee for every API or SQL dialect. SQL emission rejects writes and operations that cannot lower to SQL; remote SERVICE calls are unsupported throughout OrchidDB. See SQL API boundaries.

OrchidDB results include the complete optimized release run: 6,382 passes, no failures, and unchanged exclusions. Peer results retain their recorded source revisions. Current compiler and managed runtime APIs reject SPARQL SERVICE, including SERVICE SILENT; OrchidDB does not issue remote SPARQL HTTP requests.

Upstream compatibility evidence

This comparison uses independently maintained test scenarios and expected results. It records the original upstream identifiers, fixtures, source revisions, outcomes and available diagnostics. Engines are grouped by supported language, using free editions and marking paid capabilities separately. The SPARQL peer, Apache Jena, uses the permissive Apache 2.0 license.

All upstream scenarios appear in the report, including cases that were skipped or could not be executed. Passes, failures, unsupported interfaces, adapter limitations and timeouts are distinct outcomes. Timing records local scenario wall time and available query or step measurements. Everything runs locally; GitHub Actions only builds and publishes the static documentation and committed evidence. Download the full catalog and result files for reproducibility and investigation.

Comparison report

Open the full conformance report for suite totals, feature matrices, per-scenario results, query costs, timings, and downloadable JSON and CSV evidence. The report is generated from committed results; building the docs does not run the test suites.

The interactive report is separate from this book so the guides remain small and easy to search. Its tables and downloads also work without JavaScript.

Cypher

Inspect the openCypher results.

ArcadeDB comparison. We reproduced its published 97.8% result on 26.9.1; our harness records 91.9%. The difference comes from error checks, identity-based side-effect checks, path representations, fixture formatting and exclusions. Its harness accepts any exception for expected errors and any list as a path; our error-category mapping and path normalization also need qualification. Nineteen failures occur before the tested query because its embedded API rejects a fixture’s trailing semicolon; all 19 pass with only that terminator removed. These are fixture/API compatibility failures, not 19 demonstrated query defects. Read the scenario-level reconciliation.

Gremlin

Inspect the TinkerPop results.

SPARQL

Inspect the W3C SPARQL results.

Reproduce a run

See the conformance runner documentation for the pinned sources, adapters, and local commands. Read the scope and method alongside each result; passing an imported corpus is not standards certification.

Historical evidence

Project identifiers and local paths in the downloadable records were normalized on 2026-09-25. Outcomes and timings remain those of the recorded runs; the name change is not a new conformance run. Original source and binary hashes remain unchanged. Where local paths changed a case definition, a separate normalized_case_sha256 identifies the displayed definition while case_sha256 retains its original value. The original records are available in the source snapshot.

Query cost

Query cost in the report ranks OrchidDB scenarios by measured work and shows measurement coverage. Each scenario includes its cost; compact JSON evidence includes aggregate costs and coverage, and the CSV includes metric version, coverage, work units and query request time.

Version 1 adds SQL output rows, native input/output rows, source rows, SQL output memory in rounded-up KiB, and 100 units per SQL execution or source request. It measures work at execution boundaries, not internal database scans or all DataFusion operators. Partial costs are lower bounds; unmeasured work is shown explicitly. Fixture and observation queries are excluded from rankings. Costs do not change pass/fail outcomes or compare performance between products. Gremlin attribution version 2 excludes graph initializers, parameter lookups and assertion-helper requests. The published costs come from the new complete release run; comparisons use the prior recorded requests with corrected helper attribution.

The first measured run provides a baseline. Use --cost-baseline with the local conformance runner to compare later runs with matching case definitions, query text, metric version and complete coverage.

Configuration

Configure engine execution, Cargo features, and development checks for a reproducible integration.

Engine settings

Keep configuration near engine construction so applications can see the persistence and execution policy in one place.

Environment variables

Treat database connection URLs as application secrets. The managed and mapped DuckDB storage modes are configured through their constructors and setters.

Build features

The default library build has no database driver. Enable the optional duckdb feature for GraphEngine execution. It includes the bundled executor and the high-level engine APIs. The postgres feature includes the PostgreSQL SQL executor.

cargo build --locked
cargo build --locked --features postgres
cargo check --locked --no-default-features --lib

Run development checks

cargo fmt --all -- --check
cargo test --locked --lib --bins
RUST_MIN_STACK=16777216 cargo test --locked --features duckdb \
  --test engine --test unified_mapped_engine --test rdf_relational --test rdf_engine

See the repository’s docs/verification.md for local validation and conformance reproduction.

Reproduce an application setup

Keep the Cargo lockfile, schema definitions, graph mapping, ontology mapping, and query text under version control with the application. Parameter values are supplied at execution time. Record the execution policy and relevant timeout settings alongside deployment configuration.

For a standalone mapping artifact, use TOML mappings. For schema evolution, update the registered Arrow schemas when physical column names, types, or nullability change.

Glossary

A compact reference for the terms used throughout OrchidDB’s documentation.

Graph terms

Mapping terms

Execution terms

RDF terms

Return to the introduction to choose a guide, or use the search button to find a specific API or topic.