Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 35 additions & 8 deletions doc/user/content/ingest-data/postgres/source-versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ table schema changes seamlessly, specifically:

- Adding a column in the upstream database.
- Dropping a column in the upstream database.
- Dropping a `PRIMARY KEY` or `UNIQUE` constraint in the upstream database.
- Dropping a `PRIMARY KEY`, `UNIQUE`, or `NOT NULL` constraint in the upstream database.

This guide walks you through how to handle these changes without any downtime in Materialize.

Expand Down Expand Up @@ -188,10 +188,8 @@ has been altered.

## Handle upstream constraint drop

{{< warn-if-unreleased "v26.42" >}}

{{< public-preview >}}
Excluding constraints with `EXCLUDE CONSTRAINTS` or `EXCLUDE ALL CONSTRAINTS`
Excluding constraints with `EXCLUDE CONSTRAINTS`, `EXCLUDE ALL NOT NULL CONSTRAINTS`, or `EXCLUDE ALL CONSTRAINTS`
{{< /public-preview >}}

Materialize ignores the following constraints: foreign key, `CHECK`, and
Expand All @@ -215,14 +213,43 @@ CREATE TABLE v4.T

Materialize does not record the excluded constraint as a key of `v4.T`.

#### Excluding nullability constraints

`EXCLUDE CONSTRAINTS` only accepts `PRIMARY KEY` and `UNIQUE` constraint
names. A `NOT NULL` constraint on a specific column cannot be excluded by
name. To drop a `NOT NULL` constraint safely, use `EXCLUDE ALL CONSTRAINTS`
instead, which records no constraints at all: the table has no keys and every
column is nullable, so any later constraint drop is safe.
names. Individual `NOT NULL` constraints cannot be excluded by name. To drop
`NOT NULL` constraints safely, use `EXCLUDE ALL NOT NULL CONSTRAINTS`:

```sql
CREATE SCHEMA v4;
CREATE TABLE v4.T
FROM SOURCE my_source(REFERENCE public.T) WITH (EXCLUDE ALL NOT NULL CONSTRAINTS);
```

Note that when excluding `NOT NULL` constraints:

- Primary key columns remain `NOT NULL`, unless the primary key is also
excluded with `EXCLUDE CONSTRAINTS`.
- Columns in a `UNIQUE` constraint become nullable. This means Materialize no
longer uses that constraint as a key when optimizing queries, which might
reduce query performance. `UNIQUE NULLS NOT DISTINCT` constraints
(PostgreSQL 15+) are still used as keys.
- `PRIMARY KEY` and `UNIQUE` constraints are still recorded. Dropping one
upstream still puts the table into an error state unless it is also
excluded:

```sql
CREATE TABLE v4.T
FROM SOURCE my_source(REFERENCE public.T)
WITH (EXCLUDE ALL NOT NULL CONSTRAINTS, EXCLUDE CONSTRAINTS ('t_pkey'));
```

#### Excluding all constraints

To record no constraints at all, use `EXCLUDE ALL CONSTRAINTS`: the table has
no keys and every column is nullable, so any later constraint drop is safe.
Queries that relied on those keys for optimization may run slower.

```sql
CREATE TABLE v4.T
FROM SOURCE my_source(REFERENCE public.T) WITH (EXCLUDE ALL CONSTRAINTS);
```
Expand Down
6 changes: 4 additions & 2 deletions doc/user/data/examples/create_table_postgres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
| EXCLUDE COLUMNS (<column_name> [, ...])
| EXCLUDE CONSTRAINTS ('<constraint_name>' [, ...])
| EXCLUDE ALL CONSTRAINTS
| EXCLUDE ALL NOT NULL CONSTRAINTS
| PARTITION BY (<column_name> [, ...])
[, ...]
)]
Expand Down Expand Up @@ -67,8 +68,9 @@
|--------|-------------|
| `TEXT COLUMNS (<column_name> [, ...])` |*Optional.* If specified, decode data as `text` for the listed column(s),such as for unsupported data types. See also [supported types](#supported-data-types). |
| `EXCLUDE COLUMNS (<column_name> [, ...])`| *Optional.* If specified,exclude the listed column(s) from the table, such as for unsupported data types. See also [supported types](#supported-data-types).|
| `EXCLUDE CONSTRAINTS ('<constraint_name>' [, ...])` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, do not record the listed upstream `PRIMARY KEY` or `UNIQUE` constraint(s) as keys of the table, so that dropping them upstream does not put the table into an error state. Constraint names are string literals matched exactly, including case. `NOT NULL` constraints cannot be excluded by name, use `EXCLUDE ALL CONSTRAINTS` instead. See [Handle upstream constraint drop](/ingest-data/postgres/source-versioning/#handle-upstream-constraint-drop). |
| `EXCLUDE ALL CONSTRAINTS` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, record no upstream constraints: the table has no keys and every column is nullable, so dropping any `PRIMARY KEY`, `UNIQUE`, or `NOT NULL` constraint upstream does not put the table into an error state. Cannot be combined with `EXCLUDE CONSTRAINTS`. |
| `EXCLUDE CONSTRAINTS ('<constraint_name>' [, ...])` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, do not record the listed upstream `PRIMARY KEY` or `UNIQUE` constraint(s) as keys of the table, so that dropping them upstream does not put the table into an error state. Constraint names are string literals matched exactly, including case. `NOT NULL` constraints cannot be excluded by name; use `EXCLUDE ALL NOT NULL CONSTRAINTS` instead. See [Handle upstream constraint drop](/ingest-data/postgres/source-versioning/#handle-upstream-constraint-drop). |
| `EXCLUDE ALL CONSTRAINTS` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, record no upstream constraints: the table has no keys and every column is nullable, so dropping any `PRIMARY KEY`, `UNIQUE`, or `NOT NULL` constraint upstream does not put the table into an error state. Cannot be combined with `EXCLUDE CONSTRAINTS`. To keep keys and only relax nullability, use `EXCLUDE ALL NOT NULL CONSTRAINTS` instead. |
| `EXCLUDE ALL NOT NULL CONSTRAINTS` | ***Public preview.** This option is under active development and may have stability or performance issues.* *Optional.* If specified, ingest every column as nullable, so that dropping any `NOT NULL` constraint upstream does not put the table into an error state. `PRIMARY KEY` and `UNIQUE` constraints are still recorded; to drop one of those safely, also list it in `EXCLUDE CONSTRAINTS`. Columns in a `PRIMARY KEY` remain `NOT NULL`, unless the primary key is also excluded with `EXCLUDE CONSTRAINTS`. Columns in a `UNIQUE` constraint become nullable, so that constraint is no longer used as a key for query optimization, unless it is a `UNIQUE NULLS NOT DISTINCT` constraint. See [Handle upstream constraint drop](/ingest-data/postgres/source-versioning/#handle-upstream-constraint-drop). |
| `PARTITION BY (<column_name> [, ...])` | {{< include-md file="content/headless/partition-by-option-description.md" >}} |
results: |
{{< include-md
Expand Down
Loading