delta_engine.schema =================== .. py:module:: delta_engine.schema .. autoapi-nested-parse:: Public schema declaration surface. Import table declarations, columns, data types, properties, and key helpers from here when defining desired Delta table schemas. Attributes ---------- .. autoapisummary:: delta_engine.schema.Self Classes ------- .. autoapisummary:: delta_engine.schema.DeltaTable delta_engine.schema.ForeignKey delta_engine.schema.Property delta_engine.schema.Array delta_engine.schema.Binary delta_engine.schema.Boolean delta_engine.schema.Byte delta_engine.schema.Date delta_engine.schema.Decimal delta_engine.schema.Column delta_engine.schema.Double delta_engine.schema.Float delta_engine.schema.Integer delta_engine.schema.Long delta_engine.schema.Map delta_engine.schema.Short delta_engine.schema.String delta_engine.schema.Struct delta_engine.schema.StructField delta_engine.schema.Timestamp delta_engine.schema.TimestampNtz delta_engine.schema.Variant Module Contents --------------- .. py:class:: DeltaTable(catalog: str, schema: str, name: str, columns: collections.abc.Iterable[delta_engine.domain.model.DesiredColumn], comment: str = '', properties: collections.abc.Mapping[str, str | None] | None = None, tags: collections.abc.Mapping[str, str] | None = None, partitioned_by: collections.abc.Iterable[str] = (), clustered_by: collections.abc.Iterable[str] = (), primary_key: collections.abc.Sequence[str] | None = None, foreign_keys: collections.abc.Iterable[ForeignKey] | None = None, scope: delta_engine.application.scopes.ScopeName = 'full') Defines a Delta table schema. ``scope`` selects how much of the table the declaration manages: the whole table (default), catalog metadata only, or tags only. Note on dropping columns: Delta only permits ``ALTER TABLE ... DROP COLUMN`` when ``delta.columnMapping.mode`` is ``name``. Declare it in ``properties`` on any table whose columns may be dropped; a sync that drops a column without it fails at validation with a message naming the property. .. py:property:: catalog :type: str Unity Catalog catalog name. .. py:property:: schema :type: str Schema (database) name within the catalog. .. py:property:: name :type: str Table name. .. py:property:: columns :type: tuple[delta_engine.domain.model.DesiredColumn, Ellipsis] Declared columns, in declaration order. .. py:property:: comment :type: str Table-level comment (empty string when unset). .. py:property:: properties :type: collections.abc.Mapping[str, str | None] Declared table properties. A ``None`` value asserts the property must be absent from the table. .. py:property:: tags :type: collections.abc.Mapping[str, str] Declared table tags. .. py:property:: partitioned_by :type: tuple[str, Ellipsis] Partition column names, in declaration order. .. py:property:: clustered_by :type: tuple[str, Ellipsis] Clustering key column names, in declaration order. .. py:property:: primary_key :type: tuple[str, Ellipsis] Column names declared as the primary key, in declaration order. .. py:property:: foreign_keys :type: tuple[delta_engine.domain.model.ForeignKeyConstraint, Ellipsis] Foreign key constraints declared on this table. .. py:class:: ForeignKey Public declaration of a foreign key relationship. ``columns`` accepts one local column name for a single-column parent key, a sequence of local names for a same-name key, or an explicit ``{local: referenced}`` mapping. Sequence and mapping orders are irrelevant; ambiguous composite keys require the explicit form. Identifiers are normalized to lowercase when the declaration is constructed. The physical constraint name is generated by the engine and is not part of this declaration. ``references`` is another :class:`DeltaTable`, or the :data:`Self` sentinel for a self-referential key. See the architecture explanation doc for why the reference is an object rather than a name. The referenced table must live in the same catalog as the declaring table — information_schema is per-catalog, so a cross-catalog constraint could be created but never observed afterwards. .. py:attribute:: columns :type: str | collections.abc.Sequence[str] | collections.abc.Mapping[str, str] .. py:attribute:: references :type: DeltaTable | _SelfReference .. py:data:: Self :type: Final .. py:class:: Property Bases: :py:obj:`enum.StrEnum` The Delta table properties a user may declare on a table. .. py:attribute:: COLUMN_MAPPING_MODE :value: 'delta.columnMapping.mode' .. py:attribute:: CHANGE_DATA_FEED :value: 'delta.enableChangeDataFeed' .. py:attribute:: DELETED_FILE_RETENTION_DURATION :value: 'delta.deletedFileRetentionDuration' .. py:attribute:: LOG_RETENTION_DURATION :value: 'delta.logRetentionDuration' .. py:attribute:: DATA_SKIPPING_NUM_INDEXED_COLS :value: 'delta.dataSkippingNumIndexedCols' .. py:attribute:: TYPE_WIDENING :value: 'delta.enableTypeWidening' .. py:class:: Array Bases: :py:obj:`DataType` Array of homogeneous ``element`` values. .. py:attribute:: element :type: DataType .. py:class:: Binary Bases: :py:obj:`DataType` Sequence-of-bytes type. .. py:class:: Boolean Bases: :py:obj:`DataType` Boolean truth value type. .. py:class:: Byte Bases: :py:obj:`DataType` 8-bit signed integer type (TINYINT). .. py:class:: Date Bases: :py:obj:`DataType` Calendar date without time or timezone. .. py:class:: Decimal Bases: :py:obj:`DataType` Fixed-precision decimal type. :ivar precision: Total number of digits (1-38, Delta/Spark limit). :ivar scale: Digits to the right of the decimal point. .. py:attribute:: precision :type: int .. py:attribute:: scale :type: int :value: 0 .. py:class:: Column Immutable column declaration with a canonical-lowercase name. Exposed to users as ``Column`` through ``delta_engine.schema``; the domain name states the desired/observed side explicitly, mirroring :class:`ObservedColumn`. :ivar name: Column name, lowercased on construction. Identifiers are case-insensitive on the platform, so two names differing only in case are the same column. :ivar data_type: Logical data type of the column. :ivar nullable: Whether the column accepts ``NULL`` values. :ivar comment: Optional column comment. :ivar tags: Read-only mapping of Unity Catalog column tag keys to values. Tag keys are case-sensitive and are stored verbatim (never lowercased, unlike the column name). :ivar renamed_from: The column's previous name, declaring a rename. Inert unless the old name is observed and the new one is not, so it is safe to keep on declarations that continue to manage column structure, and correct on fresh environments. .. py:attribute:: name :type: str .. py:attribute:: data_type :type: delta_engine.domain.model.data_type.DataType .. py:attribute:: nullable :type: bool :value: True .. py:attribute:: comment :type: str :value: '' .. py:attribute:: tags :type: collections.abc.Mapping[str, str] .. py:attribute:: renamed_from :type: str | None :value: None .. py:class:: Double Bases: :py:obj:`DataType` 64-bit floating point type. .. py:class:: Float Bases: :py:obj:`DataType` 32-bit floating point type. .. py:class:: Integer Bases: :py:obj:`DataType` 32-bit signed integer type. .. py:class:: Long Bases: :py:obj:`DataType` 64-bit signed integer type. .. py:class:: Map Bases: :py:obj:`DataType` Dictionary of ``key`` to ``value`` elements. .. py:attribute:: key :type: DataType .. py:attribute:: value :type: DataType .. py:class:: Short Bases: :py:obj:`DataType` 16-bit signed integer type (SMALLINT). .. py:class:: String Bases: :py:obj:`DataType` Unicode string type. .. py:class:: Struct Bases: :py:obj:`DataType` Struct of named fields; identity is the ordered (name, type) tuple. .. py:attribute:: fields :type: collections.abc.Sequence[StructField] .. py:class:: StructField One named field inside a :class:`Struct`. Field nullability and comments are deliberately not modeled: catalog DDL strings do not round-trip them reliably, so both desired and observed structs normalize to name + type. Declared fields are created nullable. .. py:attribute:: name :type: str .. py:attribute:: data_type :type: DataType .. py:class:: Timestamp Bases: :py:obj:`DataType` Timestamp with date and time (timezone handling is engine-specific). .. py:class:: TimestampNtz Bases: :py:obj:`DataType` Timestamp with date and time, no timezone. .. py:class:: Variant Bases: :py:obj:`DataType` Semi-structured value type (Databricks VARIANT).