delta_engine¶

delta-engine: declarative schema management for Delta Lake tables.

This is the curated runtime entry point. The preferred user imports are:

from delta_engine.schema import DeltaTable, Column, Integer
from delta_engine.databricks import build_spark_engine
from delta_engine import Engine

Only backend-neutral runtime types live here. Schema declarations belong in delta_engine.schema and Databricks helpers belong in delta_engine.databricks.

Submodules¶

Exceptions¶

`DuplicateTableDefinitionError`	Raised when one sync receives two definitions for the same table.
`SyncFailedError`	Raised when one or more tables failed during sync.

Classes¶

`Engine`	High-level orchestrator to plan and execute changes.
`ExecutionFailure`	Details about a statement that failed while executing.
`Failure`	A failure that can render itself as display lines, tagged with its phase.
`FailurePhase`	The sync phase that produced a failure. Ordered so the earliest wins.
`ForeignKeyFailure`	A foreign key constraint that could not be applied, failing its whole table.
`ReadFailure`	Failure reading current catalog state for a table.
`SyncReport`	Aggregate report for a run across all tables.
`TableRunReport`	Per-table report with outcomes and a single phase-ordered failure stream.
`TableRunStatus`	High-level status of a table's sync run.
`ValidationFailure`	Description of a validation rule failure.

Functions¶

`render_diff`(→ str)	Render every table's planned changes as +/-/~ blocks, under a DIFF title.
`render_report`(→ str)	Render the run: title, optional dry-run banner, status grid, failures section, footer.

Package Contents¶

exception DuplicateTableDefinitionError(qualified_name: delta_engine.domain.model.QualifiedName)¶

Bases: ValueError

Raised when one sync receives two definitions for the same table.

qualified_name¶

class Engine(reader: delta_engine.application.ports.CatalogStateReader, executor: delta_engine.application.ports.PlanExecutor)¶

High-level orchestrator to plan and execute changes.

The engine coordinates reading current state from a catalog, computing a diff of desired vs observed state, accepting or rejecting that diff at the validated planning boundary, resolving FK dependencies with full failure context, and executing accepted plans using the provided adapters.

reader¶

executor¶

sync(*tables: delta_engine.application.ports.DesiredTableSource, dry_run: bool = False) → delta_engine.application.report.SyncReport¶

Synchronize all registered tables to their desired state.

Runs the phases as a chain, each transforming the per-table runs: read → diff → plan → compile → resolve → execute. Each TableRunReport is born in the read phase and accretes its diff, plan, compiled SQL, failures, and execution as later phases run.

A table that fails an early phase carries that failure forward and is skipped by execution; its partial run is still included in the report.

Parameters:

*tables – The table specifications to synchronize. Duplicate qualified names raise DuplicateTableDefinitionError before any phase runs.
dry_run – When True, run read → diff → plan → compile → resolve but skip execution (zero catalog mutations). Every run’s execution stays None while its plan still records the actions compiled from the observed snapshot, and the report is returned instead of raising SyncFailedError when a pre-execution phase fails.

Returns:

The aggregate SyncReport for the run.

Raises:

DuplicateTableDefinitionError – If two table specifications have the same qualified name. No phase has run when this is raised.
SyncFailedError – On a real run (dry_run=False), if any table fails to read, validate, resolve foreign keys, or execute. The report is available on the exception’s report attribute. A dry run never raises.

class ExecutionFailure¶

Bases: Failure

Details about a statement that failed while executing.

phase: ClassVar[FailurePhase]¶

statement_index: int¶

exception_type: str¶

message: str¶

statement: str¶

format_lines() → tuple[str, Ellipsis]¶: Return one or more human-readable lines describing this failure.

headline() → str¶: Return a compact one-line summary without the detail message, for the report grid.

class Failure¶

Bases: abc.ABC

A failure that can render itself as display lines, tagged with its phase.

phase: ClassVar[FailurePhase]¶

abstractmethod format_lines() → tuple[str, Ellipsis]¶: Return one or more human-readable lines describing this failure.

abstractmethod headline() → str¶: Return a compact one-line summary without the detail message, for the report grid.

class FailurePhase¶

Bases: enum.IntEnum

The sync phase that produced a failure. Ordered so the earliest wins.

READ = 1¶

PLANNING = 2¶

FOREIGN_KEY = 3¶

EXECUTION = 4¶

class ForeignKeyFailure¶

Bases: Failure

A foreign key constraint that could not be applied, failing its whole table.

phase: ClassVar[FailurePhase]¶

table: delta_engine.domain.model.QualifiedName¶

local_columns: tuple[str, Ellipsis]¶

references: delta_engine.domain.model.QualifiedName¶

reason: ForeignKeyFailureReason¶

format_lines() → tuple[str, Ellipsis]¶: Return one or more human-readable lines describing this failure.

headline() → str¶: Return a compact one-line summary without the detail message, for the report grid.

class ReadFailure¶

Bases: Failure

Failure reading current catalog state for a table.

phase: ClassVar[FailurePhase]¶

exception_type: str¶

message: str¶

format_lines() → tuple[str, Ellipsis]¶: Return one or more human-readable lines describing this failure.

headline() → str¶: Return a compact one-line summary without the detail message, for the report grid.

exception SyncFailedError(report: delta_engine.application.report.SyncReport)¶

Bases: Exception

Raised when one or more tables failed during sync.

report¶

class SyncReport¶

Aggregate report for a run across all tables.

started_at: datetime.datetime¶

ended_at: datetime.datetime¶

table_reports: tuple[TableRunReport, Ellipsis]¶

dry_run: bool = False¶

property has_failures: bool¶: Return True if any table failed in the run.

property has_changes: bool¶

True if any table’s plan holds actions.

States facts about planned changes only: a table that failed validation contributes failures, not changes. The CI gate idiom is report.has_failures or report.has_changes.

property planned_sql_statements: dict[str, tuple[str, Ellipsis]]¶: Dotted table name → the SQL its plan compiles to; no-op tables omitted.

property failures_by_table: dict[delta_engine.domain.model.QualifiedName, tuple[delta_engine.application.failures.Failure, Ellipsis]]¶: Mapping of qualified table name to its failures (if any).

to_dict() → dict[str, Any]¶: Project the whole run as plain, JSON-serialisable data; tables in run order.

class TableRunReport¶

Per-table report with outcomes and a single phase-ordered failure stream.

Carries the exact SQL statements its plan compiles to (planned_sql_statements), populated on every run — dry or real — so a dry run can preview the DDL. Planned is not executed: a table blocked after planning (for example by a foreign-key failure) still reports the SQL its plan compiles to.

qualified_name: delta_engine.domain.model.QualifiedName¶

desired: delta_engine.domain.model.DesiredTable¶

read: delta_engine.application.ports.CatalogState¶

plan: delta_engine.domain.plan.ActionPlan¶

planned_sql_statements: tuple[str, Ellipsis] = ()¶

failures: tuple[delta_engine.application.failures.Failure, Ellipsis] = ()¶

execution: delta_engine.application.ports.ExecutionSummary | None = None¶

property status: TableRunStatus¶: Status of the earliest phase that failed; SUCCESS when nothing failed.

property has_failures: bool¶: True if the table did not fully succeed.

property has_changes: bool¶: True when the plan holds actions — drift was found and validated.

to_dict() → dict[str, Any]¶

Project this table’s run as plain, JSON-serialisable data.

The field names are a public stability contract (see the run report reference doc); changing them is a breaking change.

class TableRunStatus¶

Bases: enum.StrEnum

High-level status of a table’s sync run.

SUCCESS = 'SUCCESS'¶

READ_FAILED = 'READ_FAILED'¶

PLANNING_FAILED = 'PLANNING_FAILED'¶

FOREIGN_KEY_FAILED = 'FOREIGN_KEY_FAILED'¶

EXECUTION_FAILED = 'EXECUTION_FAILED'¶

class ValidationFailure¶

Bases: Failure

Description of a validation rule failure.

phase: ClassVar[FailurePhase]¶

rule_name: str¶

message: str¶

format_lines() → tuple[str, Ellipsis]¶: Return one or more human-readable lines describing this failure.

headline() → str¶: Return a compact one-line summary without the detail message, for the report grid.

render_diff(report: delta_engine.application.report.SyncReport) → str¶: Render every table’s planned changes as +/-/~ blocks, under a DIFF title.

render_report(report: delta_engine.application.report.SyncReport) → str¶: Render the run: title, optional dry-run banner, status grid, failures section, footer.