panic: makeslice: len out of range when reading JSON column with deprecated v0 wire format

[amalakar]

Observed

1. Create a JSON column and insert 4 rows with heterogeneous value types across the same JSON
   paths (e.g. latency_ms is an array in one row, a string in another).
2. SELECT the column back using the Native protocol with clickhouse-go v2.
3. The driver panics or returns a wire decode error:
   • panic: runtime error: makeslice: len out of range (in production with large datasets)
   • read data: failed to decode block: unexpected value N for boolean
   • clickhouse [process]: unexpected packet N

Expected behaviour

The SELECT should return all rows without error.

Code example

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/ClickHouse/clickhouse-go/v2"
)

// rows contains heterogeneous JSON documents. The same key appears with
// different value types across rows (Int64, String, Array, Bool, Float64,
// nested object, timestamp) so that each JSON path acquires multiple
// inferred CH types — the condition that triggers the discriminator
// index shift in the v0 deprecated decoder.
var rows = []string{
	`{"latency_ms":[]}`,
	`{"cpu_pct":1,"region":"x","request_id":1,"response_times":[],"service":"x","tags":[]}`,
	`{"counts":[],"created_at":"x","ended_at":"x","errors":[],"labels":[],"span_ids":[],"started_at":"x","trace_id":"x","user_id":"x"}`,
	`{"attachments":[],"config":{"host":"host-149","nested":{"extra":null,"name":"svc-147","parent":null},"value":null},"context":{"name":"ctx-273","parent":null,"root":false},"durations":[],"logged_at":"x","metadata":{"active":true,"ids":[null,null],"ts":"2026-05-04T15:18:01Z"},"metrics":{"primary":{"count":542265,"enabled":true,"parent":null},"samples":[18485,18893],"secondary":{"extra":null,"label":"label-166","parent":null}},"org_id":"x","stack_frames":[],"updated_at":"x"}`,
}

func main() {
	conn, err := clickhouse.Open(&clickhouse.Options{
		Addr:     []string{"127.0.0.1:9002"},
		Auth:     clickhouse.Auth{Database: "default", Username: "default", Password: "password"},
		Protocol: clickhouse.Native,
	})
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()

	ctx := context.Background()
	if err := conn.Ping(ctx); err != nil {
		log.Fatalf("ping: %v", err)
	}

	exec := func(q string) {
		if err := conn.Exec(ctx, q); err != nil {
			log.Fatalf("%s\n  %v", q, err)
		}
	}
	exec("DROP TABLE IF EXISTS test_json_table")
	exec("CREATE TABLE test_json_table (raw JSON(max_dynamic_paths=32)) ENGINE = MergeTree ORDER BY tuple()")

	batch, err := conn.PrepareBatch(ctx, "INSERT INTO test_json_table (raw)")
	if err != nil {
		log.Fatal(err)
	}
	for _, row := range rows {
		if err := batch.Append(row); err != nil {
			log.Fatal(err)
		}
	}
	if err := batch.Send(); err != nil {
		log.Fatal(err)
	}

	result, err := conn.Query(ctx, "SELECT raw FROM test_json_table SETTINGS max_memory_usage=0")
	if err != nil {
		log.Fatal(err)
	}
	defer result.Close()

	var n int
	for result.Next() {
		var raw clickhouse.JSON
		if err := result.Scan(&raw); err != nil {
			log.Fatal(err)
		}
		n++
	}
	if err := result.Err(); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("read %d rows OK\n", n)
}

Run with:

docker run --rm -p 9002:9000 -p 8123:8123 \
  -e CLICKHOUSE_PASSWORD=password \
  clickhouse/clickhouse-server:26.2

go run .

Error log

read data: failed to decode block from 127.0.0.1:9002 (conn_id=1, compression=none): unexpected value 97 for boolean

In production with larger datasets the panic form is observed:

panic: runtime error: makeslice: len out of range

goroutine 120050 [running]:
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*JSON).decodeObjectData_v1
    lib/column/json_deprecated.go:111
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*JSON).Decode
    lib/column/json.go:768
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*Dynamic).decodeData_v1
    lib/column/dynamic_deprecated.go:174
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*Dynamic).Decode
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*Array).Decode
    lib/column/array.go:227
  github.com/ClickHouse/clickhouse-go/v2/lib/proto.(*Block).Decode
    lib/proto/block.go:232
  github.com/ClickHouse/clickhouse-go/v2.(*connect).readData
    conn_process.go:107

Root cause (hypothesis)

ClickHouse 26.2 sends JSON column data in the deprecated v0 wire format
(output_format_native_use_flattened_dynamic_and_json_serialization=0 by default). In this format
each JSON path with multiple inferred types is encoded as a Dynamic column with N variant types.

Dynamic.decodeHeader_v1 reads the N type names from the wire, then inserts "SharedVariant" and
re-sorts the list alphabetically. This shifts the driver's discriminator→column index mapping
relative to what the server wrote. For example, if a path has variants ["Array(Int64)", "String"],
the server uses discriminator 1 = String, but the driver maps discriminator 1 → SharedVariant
(which sorts between "Array..." and "String..."). Every subsequent column read is then offset,
causing the decoder to consume the wrong bytes — producing a makeslice panic, an unexpected-boolean
error, or an unexpected-packet error depending on what those bytes look like to the next decoder.

Details

Environment

• clickhouse-go version: v2.45.0
• Interface: ClickHouse API (clickhouse.Open)
• Go version: 1.26.0
• Operating system: macOS / darwin arm64
• ClickHouse version: 26.2.15.4
• Is it a ClickHouse Cloud? No (also reproduces on self-hosted)
• ClickHouse Server non-default settings: none
  (output_format_native_use_flattened_dynamic_and_json_serialization=0 is the default)
• CREATE TABLE statement:

CREATE TABLE test_json_table (raw JSON(max_dynamic_paths=32))
ENGINE = MergeTree ORDER BY tuple()

[abonander]

panic: runtime error: makeslice: len out of range

goroutine 120050 [running]:
  github.com/ClickHouse/clickhouse-go/v2/lib/column.(*JSON).decodeObjectData_v1
    lib/column/json_deprecated.go:111

Are you certain this is the correct stack trace for this panic? There's no make() call at that line to produce this panic:

clickhouse-go/lib/column/json_deprecated.go

Line 111 in 1422cc5

err := col.Decode(reader, rows)

[amalakar]

@abonander there is a possibility the stack trace is not accurate for the issue I am describing. I tried to piece together what I saw in production logs, vs what I can reproduce.

[abonander]

When running in a debugger, I found the boolean error is in decodeBlockInfo() here:

clickhouse-go/lib/proto/block.go

Lines 267 to 269 in 13b74cb

	if _, err := reader.Bool(); err != nil {
	return err
	}

Not really sure what that means, yet. It seems like a prior packet may not have been fully consumed and we're reading junk data.

[ShamsRKhan]

We hit this in production-shaped testing, root-caused it in the driver source, and have deterministic minimal repros for the two routes that trigger it. Posting here rather than opening a new issue because the symptoms in this thread — makeslice: len out of range, unexpected value N for boolean, unexpected packet N, and the "junk data" a commenter mentions — all fall out of the same two gaps.

Environment: server 26.3.17, driver v2.48.0, native protocol.

Root cause

(a) lib/column/json_deprecated.go — shared data is skipped, desyncing the stream.

After decoding typed and dynamic paths, Decode reads only the outer offsets and never consumes the shared-data payload:

// SharedData per row, ignored for now. May cause stream offset issues if present
_, err := reader.ReadRaw(8 * rows) // one UInt64 per row

That comment is accurate. One UInt64 per row is the offsets array; the values it points at are left in the stream. Every column decoded after the JSON column in that block then reads from the wrong position. That is the mechanism behind unexpected packet N and the makeslice panic in this thread — a length field parsed out of misaligned bytes is arbitrary, so where you land depends on what follows the JSON column in the block.

(b) lib/column/sharedvariant.go — SharedVariant is a String passthrough.

// SharedVariant deprecated. Use Dynamic/JSON serialization version 3.
type SharedVariant struct {
	name       string
	stringData String
}

Every method delegates to stringData. So any value stored as SharedVariant — which happens when a path exceeds max_dynamic_types — is handed to Go as its raw type-tagged binary inside a string, with no error from Query, Scan, or rows.Err().

This is the nastier of the two. No panic, no failed query, just wrong values. We observed "\x0e\x00\x00\x00\x00\x00\x00&@" — a 0x0E type tag followed by little-endian float64 11.0 — being served as a value on a live dashboard.

Reproduction

Plain Go, no deps beyond the driver. R2 runs before R1 because R1's stream desync can leave the pooled connection broken.

package main

import (
	"context"
	"fmt"
	"log"
	"strings"

	"github.com/ClickHouse/clickhouse-go/v2"
	"github.com/ClickHouse/clickhouse-go/v2/lib/chcol"
)

func main() {
	ctx := context.Background()
	conn, err := clickhouse.Open(&clickhouse.Options{
		Addr: []string{"localhost:9000"},
		Auth: clickhouse.Auth{Database: "default", Username: "default"},
	})
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()

	exec := func(q string) {
		if err := conn.Exec(ctx, q); err != nil {
			log.Fatalf("setup: %s: %v", q, err)
		}
	}

	// ---- R2: max_dynamic_types exhaustion -> SharedVariant --------------
	// Deterministic: one part, one INSERT, no merges involved. 'satellites'
	// holds Int64 and Float64 against a type budget of 1, so one of them is
	// stored as SharedVariant.
	fmt.Println("=== R2: SharedVariant (silent raw bytes) ===")
	exec(`DROP TABLE IF EXISTS r2`)
	exec(`CREATE TABLE r2 (id UInt64, doc JSON(max_dynamic_paths = 8, max_dynamic_types = 1))
	      ENGINE = MergeTree ORDER BY id`)
	exec(`INSERT INTO r2 VALUES (1, '{"satellites": 11}'), (2, '{"satellites": 12.5}')`)

	// Control: the server has the values, and knows both types.
	ctl, err := conn.Query(ctx, `SELECT toString(getSubcolumn(doc,'satellites')),
	                                    dynamicType(getSubcolumn(doc,'satellites'))
	                             FROM r2 ORDER BY id`)
	if err != nil {
		log.Fatal(err)
	}
	for ctl.Next() {
		var v, t string
		if err := ctl.Scan(&v, &t); err != nil {
			log.Fatal(err)
		}
		fmt.Printf("  server: %-6s %s\n", v, t)
	}
	ctl.Close()

	// (i) subcolumn read
	sub, err := conn.Query(ctx, `SELECT getSubcolumn(doc,'satellites') FROM r2 ORDER BY id`)
	if err != nil {
		log.Fatal(err)
	}
	for i := 0; sub.Next(); i++ {
		var v chcol.Dynamic
		if err := sub.Scan(&v); err != nil { // never fires
			fmt.Printf("  subcolumn row %d: scan error %v\n", i, err)
			break
		}
		fmt.Printf("  subcolumn row %d: %T %q\n", i, v.Any(), fmt.Sprint(v.Any()))
	}
	if err := sub.Err(); err != nil { // never fires
		fmt.Printf("  subcolumn rows.Err: %v\n", err)
	}
	sub.Close()

	// (ii) whole-column read
	whole, err := conn.Query(ctx, `SELECT doc FROM r2 ORDER BY id`)
	if err != nil {
		fmt.Printf("  whole-column: query error %v\n", err)
	} else {
		for i := 0; whole.Next(); i++ {
			var d chcol.JSON
			if err := whole.Scan(&d); err != nil {
				fmt.Printf("  whole-column row %d: scan error %v\n", i, err)
				break
			}
			v := d.ValuesByPath()["satellites"]
			if dyn, ok := v.(chcol.Dynamic); ok {
				v = dyn.Any()
			}
			fmt.Printf("  whole-column row %d: %T %q\n", i, v, fmt.Sprint(v))
		}
		if err := whole.Err(); err != nil {
			fmt.Printf("  whole-column rows.Err: %v\n", err)
		}
		whole.Close()
	}
	fmt.Println("  EXPECTED: int64(11) and float64(12.5) from both reads")

	// ---- R1: max_dynamic_paths overflow -> shared data ------------------
	// 12 distinct paths into an 8-path budget. Runs last: its stream desync
	// can leave the pooled connection broken.
	fmt.Println("\n=== R1: shared data (stream desync) ===")
	var b strings.Builder
	b.WriteString("{")
	for i := 0; i < 11; i++ {
		fmt.Fprintf(&b, `"k%02d":%d,`, i, i)
	}
	b.WriteString(`"satellites":11}`)

	exec(`DROP TABLE IF EXISTS r1`)
	exec(`CREATE TABLE r1 (id UInt64, doc JSON(max_dynamic_paths = 8))
	      ENGINE = MergeTree ORDER BY id`)
	exec(fmt.Sprintf(`INSERT INTO r1 (id, doc) VALUES (1, '%s')`, b.String()))

	rows, err := conn.Query(ctx, `SELECT doc FROM r1`)
	if err != nil {
		fmt.Printf("  OBSERVED: %v\n", err)
	} else {
		for rows.Next() {
			var d chcol.JSON
			if err := rows.Scan(&d); err != nil {
				fmt.Printf("  OBSERVED: scan error %v\n", err)
				break
			}
			fmt.Printf("  decoded: %v\n", d.ValuesByPath())
		}
		if err := rows.Err(); err != nil {
			fmt.Printf("  OBSERVED: rows.Err %v\n", err)
		}
		rows.Close()
	}
	fmt.Println("  EXPECTED: the row decodes with all 12 paths")

	// In some runs the connection is left broken: an unrelated query fails too.
	if err := conn.Exec(ctx, `SELECT 1`); err != nil {
		fmt.Printf("  OBSERVED: unrelated query on same conn now fails: %v\n", err)
	}
}

Observed

Verbatim output of the program above (server 26.3.17, driver v2.48.0):

R2, no error anywhere on the path:

server: 11     Int64
server: 12.5   Float64

subcolumn row 0: int64 "11"                                <- correct
subcolumn row 1: string "\x0e\x00\x00\x00\x00\x00\x00)@"   <- 0x0E tag + LE float64 12.5

whole-column row 0: int64 "11"
whole-column row 1: string "\x0e\x00\x00\x00\x00\x00\x00)@"

The value that lost the dynamic-type slot arrives as raw bytes; its neighbor decodes correctly — same path, same column, same result set — so a single result set mixes correct values and garbage, and nothing distinguishes them at the call site.

R1:

decoded: map[k00:{0 Int64} k01:{1 Int64} ... k07:{7 Int64}]
OBSERVED: rows.Err clickhouse [process]: unexpected packet 57

Four of the twelve paths (k08–k10, satellites) are silently ABSENT from the decoded row — the shared-data overflow — and the stream then desyncs at rows.Err. Which face R1 shows varies with block layout: across variations of this shape we have also seen the query fail outright with unexpected packet 4 and the pooled connection stay broken for subsequent unrelated queries. Single part, single row, no FINAL, no type mixing needed for any of them.

One more data point on severity: in production-shaped testing we watched a live dashboard serve the raw-bytes values for a while, and the symptom later vanished on its own after background merges reorganised the parts. Readability tracks per-part storage layout, so the same query over the same logical data can return correct values or garbage depending on when it runs. That is a large part of why this is hard to catch in testing.

Master

Both files are byte-identical on master — checked against pseudo-version v2.48.1-0.20260812104646-3a8dafcff5e2. Neither gap is fixed there.

The wire format is decodable

clickhouse-connect (Python) fully decodes both encodings — see datatypes/binary_value.py. So this is a driver gap rather than anything missing server-side, and there is a working reference implementation to check against.

Two asks

1. Default output_format_native_use_flattened_dynamic_and_json_serialization on when the server is >= 25.6. We verified end-to-end that it avoids both routes. That closes the practical problem for most users without waiting on a decoder.
2. Until SharedVariant decoding is implemented, make it error loudly rather than pass raw bytes through as a string. Root cause (a) at least announces itself. Root cause (b) does not, and silent corruption is strictly worse than a failed query — we shipped wrong numbers to a dashboard for a while without a single error in the logs.

Happy to test patches against our reproduction, or to open a separate issue for either root cause if you would rather track them apart.