diff --git a/doc/user/content/ingest-data/postgres/source-versioning.md b/doc/user/content/ingest-data/postgres/source-versioning.md index ee372e72050ad..4778ea7ee266a 100644 --- a/doc/user/content/ingest-data/postgres/source-versioning.md +++ b/doc/user/content/ingest-data/postgres/source-versioning.md @@ -20,7 +20,7 @@ table schema changes seamlessly, specifically: - Adding a column in the upstream database. - Dropping a column in the upstream database. -- Dropping a `PRIMARY KEY` or `UNIQUE` constraint in the upstream database. +- Dropping a `PRIMARY KEY`, `UNIQUE`, or `NOT NULL` constraint in the upstream database. This guide walks you through how to handle these changes without any downtime in Materialize. @@ -188,10 +188,8 @@ has been altered. ## Handle upstream constraint drop -{{< warn-if-unreleased "v26.42" >}} - {{< public-preview >}} -Excluding constraints with `EXCLUDE CONSTRAINTS` or `EXCLUDE ALL CONSTRAINTS` +Excluding constraints with `EXCLUDE CONSTRAINTS`, `EXCLUDE ALL NOT NULL CONSTRAINTS`, or `EXCLUDE ALL CONSTRAINTS` {{< /public-preview >}} Materialize ignores the following constraints: foreign key, `CHECK`, and @@ -215,14 +213,43 @@ CREATE TABLE v4.T Materialize does not record the excluded constraint as a key of `v4.T`. +#### Excluding nullability constraints + `EXCLUDE CONSTRAINTS` only accepts `PRIMARY KEY` and `UNIQUE` constraint -names. A `NOT NULL` constraint on a specific column cannot be excluded by -name. To drop a `NOT NULL` constraint safely, use `EXCLUDE ALL CONSTRAINTS` -instead, which records no constraints at all: the table has no keys and every -column is nullable, so any later constraint drop is safe. +names. Individual `NOT NULL` constraints cannot be excluded by name. To drop +`NOT NULL` constraints safely, use `EXCLUDE ALL NOT NULL CONSTRAINTS`: ```sql CREATE SCHEMA v4; +CREATE TABLE v4.T + FROM SOURCE my_source(REFERENCE public.T) WITH (EXCLUDE ALL NOT NULL CONSTRAINTS); +``` + +Note that when excluding `NOT NULL` constraints: + +- Primary key columns remain `NOT NULL`, unless the primary key is also + excluded with `EXCLUDE CONSTRAINTS`. +- Columns in a `UNIQUE` constraint become nullable. This means Materialize no + longer uses that constraint as a key when optimizing queries, which might + reduce query performance. `UNIQUE NULLS NOT DISTINCT` constraints + (PostgreSQL 15+) are still used as keys. +- `PRIMARY KEY` and `UNIQUE` constraints are still recorded. Dropping one + upstream still puts the table into an error state unless it is also + excluded: + + ```sql + CREATE TABLE v4.T + FROM SOURCE my_source(REFERENCE public.T) + WITH (EXCLUDE ALL NOT NULL CONSTRAINTS, EXCLUDE CONSTRAINTS ('t_pkey')); + ``` + +#### Excluding all constraints + +To record no constraints at all, use `EXCLUDE ALL CONSTRAINTS`: the table has +no keys and every column is nullable, so any later constraint drop is safe. +Queries that relied on those keys for optimization may run slower. + +```sql CREATE TABLE v4.T FROM SOURCE my_source(REFERENCE public.T) WITH (EXCLUDE ALL CONSTRAINTS); ``` diff --git a/doc/user/data/examples/create_table_postgres.yml b/doc/user/data/examples/create_table_postgres.yml index 8881822c8dfbd..f71b1ee27c4d4 100644 --- a/doc/user/data/examples/create_table_postgres.yml +++ b/doc/user/data/examples/create_table_postgres.yml @@ -10,6 +10,7 @@ | EXCLUDE COLUMNS ( [, ...]) | EXCLUDE CONSTRAINTS ('' [, ...]) | EXCLUDE ALL CONSTRAINTS + | EXCLUDE ALL NOT NULL CONSTRAINTS | PARTITION BY ( [, ...]) [, ...] )] @@ -67,8 +68,9 @@ |--------|-------------| | `TEXT COLUMNS ( [, ...])` |*Optional.* If specified, decode data as `text` for the listed column(s),such as for unsupported data types. See also [supported types](#supported-data-types). | | `EXCLUDE COLUMNS ( [, ...])`| *Optional.* If specified,exclude the listed column(s) from the table, such as for unsupported data types. See also [supported types](#supported-data-types).| - | `EXCLUDE CONSTRAINTS ('' [, ...])` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, do not record the listed upstream `PRIMARY KEY` or `UNIQUE` constraint(s) as keys of the table, so that dropping them upstream does not put the table into an error state. Constraint names are string literals matched exactly, including case. `NOT NULL` constraints cannot be excluded by name, use `EXCLUDE ALL CONSTRAINTS` instead. See [Handle upstream constraint drop](/ingest-data/postgres/source-versioning/#handle-upstream-constraint-drop). | - | `EXCLUDE ALL CONSTRAINTS` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, record no upstream constraints: the table has no keys and every column is nullable, so dropping any `PRIMARY KEY`, `UNIQUE`, or `NOT NULL` constraint upstream does not put the table into an error state. Cannot be combined with `EXCLUDE CONSTRAINTS`. | + | `EXCLUDE CONSTRAINTS ('' [, ...])` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, do not record the listed upstream `PRIMARY KEY` or `UNIQUE` constraint(s) as keys of the table, so that dropping them upstream does not put the table into an error state. Constraint names are string literals matched exactly, including case. `NOT NULL` constraints cannot be excluded by name; use `EXCLUDE ALL NOT NULL CONSTRAINTS` instead. See [Handle upstream constraint drop](/ingest-data/postgres/source-versioning/#handle-upstream-constraint-drop). | + | `EXCLUDE ALL CONSTRAINTS` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, record no upstream constraints: the table has no keys and every column is nullable, so dropping any `PRIMARY KEY`, `UNIQUE`, or `NOT NULL` constraint upstream does not put the table into an error state. Cannot be combined with `EXCLUDE CONSTRAINTS`. To keep keys and only relax nullability, use `EXCLUDE ALL NOT NULL CONSTRAINTS` instead. | + | `EXCLUDE ALL NOT NULL CONSTRAINTS` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, ingest every column as nullable, so that dropping any `NOT NULL` constraint upstream does not put the table into an error state. `PRIMARY KEY` and `UNIQUE` constraints are still recorded; to drop one of those safely, also list it in `EXCLUDE CONSTRAINTS`. Columns in a `PRIMARY KEY` remain `NOT NULL`, unless the primary key is also excluded with `EXCLUDE CONSTRAINTS`. Columns in a `UNIQUE` constraint become nullable, so that constraint is no longer used as a key for query optimization, unless it is a `UNIQUE NULLS NOT DISTINCT` constraint. See [Handle upstream constraint drop](/ingest-data/postgres/source-versioning/#handle-upstream-constraint-drop). | | `PARTITION BY ( [, ...])` | {{< include-md file="content/headless/partition-by-option-description.md" >}} | results: | {{< include-md