datamog

Datamog

Datamog

Datamog is an educational Datalog dialect that translates into SQL. It supports Horn clauses with extensional predicate declarations, stratified negation, and aggregates, and compiles rules into views (including recursive views for recursive predicates). A first-class value column type — the union of every shape (null, booleans, integers, floats, strings, arrays, objects), with subscript / slice / iteration / coercion / construction primitives and structural equality maintained across every backend — lets programs work with nested data directly, without pre-flattening. The project ships with three SQL backends (Postgres, SQLite, sql.js), two non-SQL in-memory evaluators (native and seminaive), and a VS Code extension for editor support.

Runtime Support

Datamog’s TypeScript packages are Bun-only when consumed directly. The workspace publishes TypeScript source entry points and intentionally uses Bun APIs such as Bun.file, Bun.sql, and bun:sqlite; Node.js runtime compatibility is not currently a goal. Use Bun 1.3 or newer for development and for running the CLI/packages directly.

The VS Code extension is the exception at install time: it is packaged as bundled JavaScript for VS Code’s extension host, so users of the .vsix do not need Bun installed.

Syntax

# Declare extensional predicates (backed by tables)
extensional parent(name: string, child: string).

# Define rules (Horn clauses)
ancestor(X, Y) :- parent(X, Y).
ancestor(X, Y) :- parent(X, Z), ancestor(Z, Y).

# Query
?- ancestor("alice", X).

Values

A value column holds the union of every shape: null, booleans, integers, floats, strings, arrays, and objects. Datamog gives you a small toolkit to read, project, iterate over, and (in narrow ways) construct them, all from inside a rule body. The name “JSON” is reserved for the syntax (JSONL files, parsing strings) and the on-disk representation; the language type is just value.

extensional event(payload: value).

# Destructure an event into flat columns. Wrong-shape access → NULL.
request(Id, Method, Path, Status) :-
    event(E),
    Id = as_integer(E["id"]),
    Method = as_string(E["method"]),
    Path = as_string(E["path"]),
    Status = as_integer(E["status"]).

# Iterate every key/value pair of a nested object value.
event_header(Id, Key, V) :-
    event(E),
    Id = as_integer(E["id"]),
    object_entry(E["headers"], Key, Raw),
    V = as_string(Raw).

# Construct: parse a string, or use array / object literals to
# assemble shapes directly. Primitives flowing into a `value` slot
# auto-lift, so no explicit primitive-to-value conversion is needed.
#   parse_json : string → value (NULL on malformed input)
#   [...] / {"k": v, ...} — array / object literals
parsed(S, V)   :- raw(S),               V = parse_json(S).
record(Id, V)  :- request(Id, M, P, _), V = {"method": M, "path": P}.

The toolkit:

Datamog maintains structural equality across every backend: PostgreSQL stores value as JSONB and gets canonicalisation natively; SQLite / sql.js store it as canonical TEXT (object keys sorted recursively, numbers normalised on insert) so textual equality coincides with structural; the in-memory evaluators canonicalise via the same function. The one v1 cross-backend variance is parse_json on SQLite / sql.js, which doesn’t sort object keys — see the spec for details.

Two loaders feed value columns: JSONL with a single-value-column declaration consumes each line as one row; a standalone <predicate>.json file loads the whole file as one row. The Working with values tutorial chapter walks through a complete example, and the json-events, json-config, and parse-json CLI examples are runnable end-to-end.

Packages

Package Description
datamog-parser Langium grammar, generated parser, and AST type definitions
datamog-core Program analyzer (safety checking, dependency graph, recursion detection, type inference)
datamog-engine SQL translator, executor, and pluggable loader interface
datamog-backend-postgres Postgres backend (via Bun.sql)
datamog-backend-sqlite SQLite backend (via bun:sqlite, in-memory by default)
datamog-backend-sqljs sql.js backend (SQLite compiled to WASM via sql.js)
datamog-backend-native Native in-memory backend that interprets Datalog directly via a naive evaluator (no SQL)
datamog-backend-seminaive Seminaive in-memory evaluator — same observable semantics as native but with delta-aware iteration (no SQL)
datamog-csv Loader plugin for CSV files
datamog-jsonl Loader plugin for JSONL files
datamog-json Loader plugin for whole-file JSON (single-row tables)
datamog-gsheet Loader plugin for Google Sheets
datamog-mermaid Loader plugin for Mermaid graph/flowchart files (.mmd)
datamog-repl Incremental REPL session engine (declarations/rules/queries accumulate); drives the interactive CLI REPL and the --repl --json JSONL protocol
datamog-cli Command-line interface
datamog-playground Browser-based playground (Preact, CodeMirror, sql.js — no server needed)
datamog-vscode VS Code extension with syntax highlighting and diagnostics

Playground

The playground is a browser-based IDE for Datamog — write programs, attach CSV/JSONL data or CORS-enabled CSV URLs, and run them entirely client-side. The full pipeline (parse → analyze → translate → execute) runs in a Web Worker; SQL execution uses sql.js (SQLite compiled to WASM) and the native / seminaive evaluators run directly in JS. No installation required.

Try it online: max-schaefer.github.io/datamog

Highlights:

A lightweight embeddable variant (packages/playground/src/embed/) turns the datamog code blocks in a Markdown tutorial into live, editable mini-playgrounds — it runs on the main thread with the pure-TS native/seminaive backend (no Web Worker, no WASM), so several can share one page. See doc/embed-tutorials/.

The playground is automatically deployed to GitHub Pages on every push to main.

bun run playground:dev    # start dev server
bun run playground:build  # production build (static files in packages/playground/dist/)

Usage

Start the interactive REPL:

bun run datamog

Run with the CLI (no database setup needed — uses in-memory SQLite by default):

bun run datamog packages/cli/examples/family/family.dl

Or preview the generated SQL:

bun run datamog --dry-run packages/cli/examples/family/family.dl

Select a backend explicitly:

bun run datamog --backend sqlite packages/cli/examples/family/family.dl
bun run datamog --backend sqljs packages/cli/examples/family/family.dl
bun run datamog --backend native packages/cli/examples/family/family.dl
DATABASE_URL=postgres://localhost:5432/mydb bun run datamog --backend postgres program.dl

The native backend skips SQL entirely: it runs a naive bottom-up evaluator over in-memory relations, which is slower than the SQL backends for large programs but makes the Datalog semantics (stratum-by-stratum fixed-point iteration, rule-body enumeration, stratified negation) easy to trace step by step. --dry-run is not supported for this backend since there’s no SQL to print.

The CLI auto-discovers data files in the same directory as the .dl file: <predicate>.csv, <predicate>.jsonl, <predicate>.json (loaded as one row, requires a single value column), or <predicate>.mmd (Mermaid graph). You can override individual predicates with --extensional name=source, where source is a local file path, an HTTP(S) URL ending in .csv, .jsonl, .json, or .mmd, a Google Sheets URL — see the datamog-gsheet README for Google Sheets setup instructions — or a GitHub shorthand github:owner/repo/path (gh: alias), which expands to a raw.githubusercontent.com URL with the ref defaulting to HEAD (pin a branch/tag/commit with a trailing #ref).

Examples

The packages/cli/examples/ directory holds 40+ runnable programs — transitive closure, stratified negation, aggregates, mutual recursion, classic puzzles, JSON/value handling, propositional logic, and Boolean-circuit solvers. Run any of them with:

bun run datamog packages/cli/examples/<name>/<name>.dl

Some use non-linear recursion (rejected by the SQL backends); those carry a native-only marker file and run on --backend native or --backend seminaive.

Programmatic API

import { DatamogExecutor } from "datamog-engine";
import { CsvLoader } from "datamog-csv";
import { create as createBackend } from "datamog-backend-sqlite";

const backend = await createBackend();
const executor = new DatamogExecutor(backend, [
  new CsvLoader({ directory: "./data" }),
]);

const source = await Bun.file("family.dl").string();
const results = await executor.execute(source);

for (const result of results) {
  console.log(result.sql);
  console.table(result.rows);
}

await backend.close();

VS Code Extension

The datamog-vscode package provides a VS Code extension with syntax highlighting, live parse-error diagnostics, and semantic validation (arity mismatches, unsafe variables, unstratifiable negation, etc.).

Build and install

bun run build:vscode
code --install-extension packages/vscode-extension/datamog.vsix

For development, open the packages/vscode-extension folder in VS Code and press F5 to launch an Extension Development Host with the extension loaded.

Features

Documentation

Development

See DEVELOPMENT.md for the full local development guide.

bun install          # install dependencies
bun test             # run all tests
bun run typecheck    # tsc -b across the workspace (project references)
bun run e2e          # Playwright e2e suite for the playground (auto-installs Chromium on first run)
bun run check        # lint and format check (biome)
bun run check:fix    # auto-fix lint and format issues

Tutorial slides

Per-chapter Marp decks live under doc/walkthrough/slides/. Build PDFs (gitignored) with:

bun run slides:build   # one-shot
bun run slides:watch   # rebuild on change

License

MIT © 2026 Max Schaefer.

Trademarks

The Datamog mascot is original art, a play on the Mercedes-Benz Unimog; “Mercedes-Benz” and “Unimog” are trademarks of Mercedes-Benz Group AG. The Part 1 course material uses Pokémon as a running example; “Pokémon” and Pokémon names, types, moves, and abilities are trademarks of Nintendo, Creatures Inc., and GAME FREAK inc. All such marks are used only nominatively, for educational illustration; this project is not affiliated with or endorsed by their owners.

This site is open source. Improve this page.