CHANGELOG.md

Changelog

All notable changes to CipherStash Proxy will be documented in this file.

The format is based on Keep a Changelog.

Unreleased

Added

• Application-generated EQL values in statements: SQL literals and bound parameters may now carry EQL v3 storage payloads or query-only SEM operands produced by an application. Proxy authenticates every stored ciphertext, requires its authenticated descriptor to name the inferred destination column, and independently re-derives every SEM term before forwarding it without double encryption. Query-only operands contain no ciphertext to authenticate, so Proxy instead validates their version, identifier, term shape, column capabilities, and syntactic query role; they are rejected in storage positions. This includes bare SteVec selector hashes matching ^[0-9a-f]{32}$: in JSON selector query positions a match is treated as already hashed, while a non-match remains plaintext and is encrypted normally. A matching plaintext selector is inherently ambiguous and is intentionally treated as already hashed. Invalid payloads fail closed with one generic, transaction-aborting error so validation details cannot be used as an oracle.

  Compatibility note: on every encrypted column type, Proxy reserves three JSON object shapes for application-generated EQL: storage payloads with top-level v and i plus at least one of c, h, or sv; scalar query payloads with v and i plus at least one of hm, bf, ob, or op; and SteVec query payloads whose only top-level key is sv. An object matching one of these shapes is validated as EQL rather than encrypted as plaintext, and there is no opt-out. Query-only shapes are rejected in storage positions. Before upgrading, audit plaintext application writes for these key combinations; Invalid encrypted value includes a jsonb scan predicate.

Changed

• Upstream TLS verification for client traffic: Connections with with_tls_verification enabled now use a cached snapshot of the system root certificates, loaded once when Proxy starts. Unlike Proxy's background database connections, pg-proto client-traffic connections do not apply operating-system revocation checks or enterprise verification policy. Restart Proxy after changing the system trust store.

Fixed

• Configured default keyset selection: when a connection has not selected a keyset explicitly, Proxy now scopes encryption and decryption to CS_DEFAULT_KEYSET_ID. Previously it passed no keyset to ZeroKMS and could silently use the account default instead, deriving different searchable-encryption terms when the two defaults differed. Before upgrading, verify that the configured and account defaults are intentional; values written by an affected version under the unintended account default must be decrypted with that old keyset and re-encrypted under the configured default.

• PostgreSQL protocol error handling after the pg-proto migration: Proxy now rejects require_tls configurations that omit a certificate, preserves PostgreSQL transaction state when statement mapping fails, returns decryption failures as PostgreSQL errors without dropping the connection, and reloads changed schemas only after PostgreSQL confirms the transaction boundary. Prepared-statement replacement also preserves existing portals and overlapping statement metrics.

3.0.1 - 2026-08-05

Added

• Multi-step JSON paths on encrypted columns, everywhere: col -> 'a' -> 'b' now works wherever a single field access does — in the select list, in an ordering comparison (<, <=, >, >=), and in the ->, ->> and jsonb_path_query_first spellings mixed freely, to any depth, with each step written as a literal or a placeholder. Previously only exact equality accepted a multi-step path and everything else was rejected. A chain is a single path into a single document, so it is now rewritten to one field access keyed on the whole composed path ($.a.b) instead of the nested accessors that would search an already-extracted entry and return NULL. Exact equality continues to fold the path and the value into one needle, which remains the stronger match.

  Two shapes are still rejected rather than answered. A path split across a subquery, CTE or view (SELECT a -> 'foo' FROM (SELECT col -> 'bar' AS a FROM t) s) cannot be composed at all — the extracted value does not carry the path that produced it, and the document it came from is not in scope — so write the whole path in one expression. A placeholder step in front of a literal final step (col -> $1 -> 'b') cannot be composed either, because a literal is encrypted before any parameter is bound; parameterise the final step too (col -> $1 -> $2), or write the whole path as literals.

• ANY/ALL over an array of encrypted values: col = ANY(ARRAY['a', 'b']) now works on an encrypted column whose domain supports the operator, rewritten elementwise to the same term form as the scalar comparison — each array element is encrypted like any other comparison operand. ARRAY[…] written in the statement is the supported spelling; an encrypted subquery projection (ANY(SELECT enc FROM …)) or a bare array parameter (ANY($1)) is rejected with an explanatory error, where previously the subquery form was forwarded unrewritten and silently matched nothing.

Fixed

• One placeholder used as the JSON selector of two different paths: col -> 'a' -> $1 = $2 alongside col -> 'b' -> $1 = $3 silently kept only one of the two paths, so one of the predicates was matched against the wrong field. The path a selector placeholder keys is recorded against the parameter it arrives in — at Bind time the parameter number is all Proxy has — so two different paths for one parameter cannot both be honoured. This is now reported as an error naming the parameter, rather than answered from whichever path was recorded last.

• Literals and params that escape type checking now fail closed: a literal or parameter whose type was never worked out during type checking used to be silently assumed to be plaintext — so a value in a clause the type checker did not cover could skip encryption without any error. Proxy now only makes that assumption where it is provably safe: a value that only flows to the client through a SELECT projection (SELECT 'lit', SELECT $1), or comparison operands that relate to nothing but each other (WHERE 1 = 1, 1 IN (1, 2), and friends — anything encrypted arriving in such a comparison types it concretely first). Anywhere else the statement is rejected with an error naming the value. As part of this, WHERE/HAVING/join ON conditions and ORDER BY/GROUP BY ordinals are now explicitly typed as plaintext where they appear, and an encrypted column used bare as a boolean condition (for example WHERE enc_col) is rejected instead of being forwarded to the database.

• UPDATE … SET … FROM with same-named columns: an UPDATE was rejected as ambiguous when a table in the FROM clause had a column with the same name as the column being assigned. The assignment now always refers to the table being updated, so these statements work and the assigned value gets the target column's type — encrypted or not.

• Encrypted values as row counts are rejected: an encrypted column used in LIMIT, OFFSET, or FETCH (for example LIMIT enc_col) is now rejected with a type error instead of being forwarded to the database.

• Statements Proxy cannot type-check fail with a clear error: a statement Proxy admits for type checking but has no support for is now rejected immediately with an error naming the statement, instead of surfacing later as an opaque resolution error. No currently-supported statement is affected.

Security

• Chained JSON field accessors sent the intermediate field name to the database in plaintext: WHERE col -> 'a' -> 'b' = $1 on an encrypted JSON column emitted eql_v3.jsonb_contains(col -> 'a', …), so the field name a appeared in the statement text PostgreSQL received (and in its logs), and native jsonb -> was applied to the encrypted payload — which also made the predicate match nothing. A chain is now treated as the single path it is: $.a.b of the whole document, folded into the one encrypted needle and matched against the bare column. Chains of any depth are supported, in the ->, ->> and jsonb_path_query_first spellings, with = and <>, and with each step written as a literal or a placeholder.

• A NULL JSON selector forwarded the compared value to the database in plaintext: WHERE col -> $1 = $2 with $1 bound NULL builds no needle, so $2 was never encrypted — and it was then sent to PostgreSQL exactly as the client bound it, putting the plaintext comparand on the wire and into the server log when the column's domain CHECK rejected it. An encrypted operand that produced no ciphertext is now bound NULL, which is also what the SQL means: a comparison against NULL is NULL, so the query returns no rows.

• A surviving eql_v2_encrypted column was served as plaintext: after migrating to EQL v3, a column still declared with EQL v2's eql_v2_encrypted type had no v3 domain identity, so Proxy fell back to treating it as an ordinary plaintext column — no encryption on writes, no decryption on reads. An application writing to such a column stored plaintext in a database it believed was encrypted, and saw no error doing so; the only signal was a single warning at startup. This is the shape of a partly-completed migration, where most columns move to v3 domains and one is left behind.

  Proxy now refuses every statement referencing a table that has such a column, with an error naming the column, its type and the need to migrate it. Other tables are unaffected, so one unmigrated column no longer costs you a deployment. The refusal is not subject to the mapping_errors_enabled passthrough — it applies whatever CS_DEVELOPMENT__ENABLE_MAPPING_ERRORS is set to. See Unmappable encrypted column.

3.0.0 - 2026-08-05

Changed

• EQL v3 (searchable encryption): Proxy now targets EQL v3. Encrypted columns are declared with self-configuring, typed jsonb domains (for example eql_v3_text_search, eql_v3_integer_ord, eql_v3_json_search) that encode both the scalar type and the column's searchable capabilities in the column type itself, replacing EQL v2's opaque eql_v2_encrypted composite type and its separate eql_v2_configuration table. The bundled cipherstash-client is upgraded to 0.42.0 and EQL to 3.0.4. Existing v2-encrypted data and schemas must be migrated to v3.

Added

• Encrypted full-text match with @@: The @@ operator is now supported on encrypted text columns whose domain carries a match (bloom-filter) term, rewritten to the EQL v3 eql_v3.match_term form.

• SELECT DISTINCT on an encrypted column now deduplicates: DISTINCT used to compare whole encrypted payloads, whose ciphertext is randomised per row, so equal plaintexts never collapsed and DISTINCT silently returned duplicates. It is now keyed on the column's equality term — SELECT DISTINCT ON (eql_v3.eq_term(col)) col … — so one row is returned per distinct plaintext. Deduplication is equality, so a column whose domain carries no equality term (eql_v3_boolean, for instance, which is storage-only) is now rejected with a capability error rather than silently returning every row.

• SELECT DISTINCT ordered by an encrypted column: SELECT DISTINCT … ORDER BY <encrypted column> now works. Ordering an encrypted column requires its ordering term, which PostgreSQL will not accept under DISTINCT unless it also appears in the select list, so the query is rewritten to project the term from a subquery and order the outer query by it. The term is never returned to the client and column names are preserved. Two shapes remain unsupported and are reported as such: SELECT DISTINCT ON (…) and SELECT DISTINCT *, both when combined with ORDER BY on an encrypted column — list the columns explicitly for the latter.

• Equality on encrypted JSON fields: WHERE col -> 'field' = 'value' now works on encrypted JSON columns, in both the simple and extended query protocols, and in the ->> and jsonb_path_query_first(col, path) = value spellings. <> is supported as the negation. The field and the value are combined into a single encrypted value-selector needle and matched by containment, so a query never reveals the field and value separately. Matching is exact and case-sensitive; the value must be a JSON scalar (comparing a whole object or array to a field is rejected — use containment with @> instead).

Fixed

• Statement errors no longer desync the connection: when a statement failed inside the proxy (an unsupported operation on an encrypted column, for instance), the error was written straight to the client and could overtake responses still in flight from the server — with connection pools and prepared-statement caching, the client then saw a protocol error (unexpected message from server in tokio_postgres) instead of the proxy's message, typically right after an encrypted statement had run on the same connection. The proxy now delivers statement errors through the server, so clients always receive the proxy's actual error message, in order, and the connection remains usable.

• A param bound as both a stored value and a query operand: UPDATE t SET enc = $1 WHERE enc = $1 failed with a domain CHECK violation. The two occurrences need different payloads — the stored one carries the ciphertext, the query one only search terms — but the role was tracked per input param, so marking the param as a query operand stripped the ciphertext from the value being stored. The role is now taken from the rewritten statement, per occurrence.

• JSON selector params when the client declares its own types: a client that sends param OIDs in Parse (pgx in cache_describe mode, for example) got function eql_v3.jsonb_path_exists(eql_v3_json_search, jsonb) does not exist. A JSON field selector is passed to the rewritten function as bare text, but was being declared as jsonb like every other encrypted operand. Affects ->, ->>, jsonb_path_exists, jsonb_path_query and jsonb_path_query_first.

• Binary-format text operands on encrypted JSON fields: a TEXT/VARCHAR operand arriving in binary format was handed straight to the JSON decoder and rejected, even though the same value in text format was accepted. Textual types are now read as a string first and then given the text format's treatment, so Alice behaves like "Alice".

• Aggregates over a grouped encrypted column: SELECT MIN(enc) FROM t GROUP BY enc produced grouped_value(eql_v3.min(enc)) — an aggregate inside an aggregate, which PostgreSQL rejects. An aggregate already returns one value per group, so it is no longer lifted; only a direct projection of the grouped column is.

• SELECT * with GROUP BY on an encrypted column is now rejected with an explanatory error instead of PostgreSQL's "column must appear in the GROUP BY clause". A wildcard hides the projected columns, so the grouped column cannot be projected through eql_v3.grouped_value — list the columns explicitly. This matches the existing treatment of SELECT DISTINCT *.

• SELECT DISTINCT * skipped the encrypted-column protection: a wildcard hides the columns DISTINCT deduplicates on, so neither the equality-term keying nor the capability check applied and duplicates were returned silently. The wildcard is now expanded to its columns, which are keyed like any other; a wildcard hiding a column with no equality term is rejected.

• @@ with the encrypted column on the right: 'pattern' @@ col produced match_term('pattern') @> match_term(col) — a backwards containment, with the pattern never encrypted, that silently matched nothing. @@ is symmetric in PostgreSQL, so both spellings now produce the same query.

• Encrypt config could pick up a same-named table from another schema: the config is keyed on (table, column) while the schema query scanned every schema, so a table of the same name elsewhere — another tenant's, a staging copy — could overwrite the served one and give a column the wrong domain config or drop its encryption. The scan is now limited to the connection's search path, in precedence order.

• A prepared statement name reused for an unmapped statement: Parse rebinds its name, but a statement Proxy does not map — BEGIN, COMMIT, or anything needing no type check — left the previous statement cached under that name. The next Bind for the name was then rewritten against a statement the client never parsed, failing with Rewritten statement binds parameter 1, but only 0 were provided. Affects any client that reuses the unnamed prepared statement across a transaction, which includes pgbench in extended mode and psycopg with prepare=False.

• LIKE/ILIKE capability checking: LIKE and ILIKE on an encrypted column are now gated by the column's token-match capability. Previously these predicates bypassed capability checking and were silently accepted on columns that do not support fuzzy match; they are now rejected with a capability error.

• Upserts with ON CONFLICT DO UPDATE now encrypt the update path: INSERT … ON CONFLICT (…) DO UPDATE SET enc = … previously left the DO UPDATE assignments untouched, so a plaintext value landed in the encrypted column unencrypted whenever the conflict path ran. Assignments are now typed and encrypted exactly like a plain UPDATE … SET, excluded.<col> references resolve to the column's encrypted type, and comparisons in the DO UPDATE … WHERE predicate are rewritten to their search terms. A conflict target naming an encrypted column (ON CONFLICT (enc)) is rejected: uniqueness there would be judged on the randomised ciphertext, so the conflict would never fire.

• Window functions over an encrypted column: the window's ORDER BY is now checked against the column's ordering capability (previously it was silently left ordering on raw ciphertext, whose order differs on every insert), and named window definitions (OVER w with WINDOW w AS (PARTITION BY enc …)) get the same equality-term and ordering-term treatment as inline OVER (…) clauses, which previously escaped both checking and rewriting. RANGE frames with an offset over an encrypted sort key are rejected, since no search term supports the arithmetic they need; ROWS and GROUPS frames work.

• count(DISTINCT enc) now counts distinct plaintexts: the deduplication previously ran on whole encrypted payloads, whose ciphertext is randomised per row, so every value looked distinct and the count silently equalled the row count. The argument is now rewritten to the column's equality term. DISTINCT with an encrypted argument in any other aggregate is rejected (the substitution would change that aggregate's result), as is an aggregate-internal ORDER BY (array_agg(x ORDER BY enc)) on a column with no ordering term — where the capability exists, the key is rewritten to its ordering term.

• WITHIN GROUP (ORDER BY enc) is now rejected: an ordered-set aggregate (percentile_disc, mode, …) computes its result from the sort key, so on an encrypted column it would hand the client an opaque search term. Previously the clause escaped type checking entirely.

• SELECT … INTO an encrypted column is now rejected: the statement copies data into a table the encryption schema has never seen, leaving unreachable ciphertext there. Native-only projections pass through as before.

2.2.4 - 2026-06-18

Fixed

• ZeroKMS authentication failures ~15 minutes after startup (access keys): Fixed the root cause of access tokens never being renewed when authenticating with an access key. The token's lifetime was misread, so renewal never triggered and every encrypt/decrypt operation began failing (ZeroKMS error: Request not authorized, "Could not decrypt data") roughly 15 minutes — the token lifetime — after connecting, recovering only on restart. Tokens now renew correctly ahead of expiry. This resolves the remaining cases not addressed by the 2.2.3 fix.

2.2.3 - 2026-06-17

Fixed

• ZeroKMS authentication failures ~15 minutes after startup: Fixed an issue in the access-key authentication path where, after an in-flight request was interrupted at the wrong moment (for example, a client disconnecting mid-query), access-token renewal could stall. This caused ZeroKMS error: Request not authorized on all encrypt/decrypt operations roughly 15 minutes (the access-token lifetime) after connecting — connections worked on startup and then began failing in lockstep.

2.2.2 - 2026-06-01

Fixed

• Passthrough mode memory leak: Fixed a per-statement memory leak that occurred in passthrough mode (empty encrypt config), where per-statement queues were not drained. Long-running connections could grow unbounded and eventually OOM. (#400)

2.2.1 - 2026-05-14

Added

• OPE (Order-Preserving Encryption) index: New ope index type alongside the existing ore for range and ORDER BY queries on encrypted columns. Drop-in alternative to ore — pick one per column. See the encrypted indexes documentation for configuration.

2.2.0-alpha.1 - 2026-03-25

Changed

• Log target renamed: KEYSET log target renamed to ZEROKMS. The environment variable CS_LOG__KEYSET_LEVEL is now CS_LOG__ZEROKMS_LEVEL.

Removed

• Log target removed: PROXY log target and CS_LOG__PROXY_LEVEL environment variable have been removed.

Added

• Cipher cache miss metric: New Prometheus counter cipherstash_proxy_keyset_cipher_cache_miss_total tracks cache misses requiring cipher initialization. This complements the cipherstash_proxy_keyset_cipher_cache_hits_total metric, and can be used to calculate cache hit/miss ratio.
• Cipher init duration metric: New Prometheus histogram cipherstash_proxy_keyset_cipher_init_duration_seconds tracks cipher initialization time including ZeroKMS network calls.
• Encrypt/decrypt timing: Debug logs for encrypt_eql and decrypt_eql now include duration_ms.
• Cache eviction logging: ScopedCipher cache eviction events are now logged under the ZEROKMS target.
• Slow cipher init warning: Cipher initialization taking longer than 1 second triggers a warning log.

2.1.22 - 2026-02-05

Added

• Configurable slow database response threshold: The "Slow database response" log threshold is now configurable via CS_LOG__SLOW_DB_RESPONSE_MIN_DURATION_MS (default: 100ms). This controls per-message logging for individual slow reads from the PostgreSQL server.

2.1.21 - 2026-02-04

Changed

• Updated cipherstash-client to v0.33.0. Adds array_index_mode configuration for STE-VEC indexes, which controls how arrays are indexed in JSONB data. Defaults to all (generating item, wildcard, and positional selectors), preserving backwards compatibility with existing configurations.

2.1.20 - 2026-01-29

Added

• Slow statement logging: Enable with CS_LOG__SLOW_STATEMENTS=true to log detailed timing breakdowns when queries exceed a configurable threshold (default 2 seconds). Includes breakdown of parse, encrypt, server wait, and decrypt phases.
• Prometheus slow statement counter: New cipherstash_proxy_slow_statements_total metric increments when slow statements are detected.
• Prometheus histogram labels: Duration histograms now include statement_type, protocol, mapped, and multi_statement labels for granular performance analysis.
• Term filters for STE-VEC indexes: Support for term_filters configuration in eql_v2.add_search_config(), enabling case-insensitive JSONB queries with the downcase filter.

Changed

• Updated cipherstash-client to v0.32.2.
• GitHub Actions jobs now timeout after 30 minutes.
• ARM64 builds migrated to Blacksmith runners.

2.1.9 - 2026-01-10

Added

• Encryption sanity checks for improved error detection.
• Developer documentation updates.

Changed

• Updated cipherstash-client to v0.31.1.

2.1.8 - 2025-12-15

Changed

• Refactored EQL encryption logic.
• JSONB containment operator transformation improvements.
• Testing across multiple PostgreSQL versions.

2.1.7 - 2025-11-27

Added

• Security documentation.

Changed

• Improved ZeroKMS error handling.
• Database connection CLI arguments now optional.

2.1.6 - 2025-09-05

Fixed

• Accurate cipher cache sizing.
• JSONB encrypted type protocol fixes.

Changed

• Module restructuring.

2.1.5 - 2025-08-21

Added

• SET command for keyset_id configuration.
• Configurable cipher caching using async Moka.

2.1.4 - 2025-08-08

Changed

• Updated EQL to v2.1.8.

2.1.3 - 2025-08-01

Added

• Helm chart support.
• JSONB operator integration tests.
• Comprehensive proxy/EQL showcase crate.

2.1.2 - 2025-07-16

Fixed

• Common Table Expression (CTE) table resolution in EQL mapper.

2.1.1 - 2025-07-15

Added

• JSON indexing for EQL v2.
• Prometheus metrics collection.
• Multiple integration test frameworks.

2.0.10 - 2025-06-26

Added

• SET command to disable mapping.

2.0.9 - 2025-06-20

Changed

• Upgraded container base image to Ubuntu 25.10.
• Updated sqltk dependency to v0.10.0.

2.0.8 - 2025-06-18

Added

• Version string sent to ZeroKMS/CTS requests.

Fixed

• Type-related issues in sqlparser.

Changed

• Release workflow now triggers on release events.

2.0.7 - 2025-06-12

Added

• Language-specific tests in integration suite.
• PostgreSQL custom and domain type identifier handling.

Fixed

• Docker image build processes in GitHub Actions.

2.0.6 - 2025-06-09

Added

• TLS and Docker configuration documentation.
• Expanded test coverage for order and group operations.

Changed

• URL encoding for usernames in Docker entrypoint.
• Preference for CRN over workspace_id and region.

Removed

• Order and group transformers.

2.0.5 - 2025-05-27

Fixed

• Cache usage in release artifact building.

2.0.4 - 2025-05-26

Added

• OIDC support.

Fixed

• Special character handling in database configuration.
• "Insufficient data left in message" errors with null values.

2.0.3 - 2025-05-26

Fixed

• Tests now ignore CS_ environment variables during configuration validation.

Changed

• Added environment debugging to AWS Marketplace release workflow.

2.0.2 - 2025-05-22

Added

• Multi-platform Docker image builds.

Changed

• Updated EQL to v2.0.1.

2.0.1 - 2025-05-21

Added

• Encryption configuration validation.
• pgbench performance testing integration.
• ZeroKMS and CTS host configuration options.
• GROUP BY SQL transformations.
• EQL v2 decryption support.
• Enhanced column configuration verification.

Fixed

• Connection termination messaging.

Changed

• Upgraded to Rust 1.86.0 compatibility.
• Upgraded sqltk to v0.8.0.

2.0.0 - 2025-03-27

Added

• Initial release of CipherStash Proxy.
• Transparent proxy for PostgreSQL with automatic encryption/decryption.
• Support for queries over encrypted values (equality, comparison, ordering).
• Docker container deployment.
• Integration with CipherStash ZeroKMS.
• Encrypt Query Language (EQL) for indexing and searching encrypted data.