sidecar: re-marshaling the request body as map[string]any reorders nested JSON keys (tools schema) #2590

What happened

The P/D sidecar decodes the request body into map[string]any and marshals it again for the prefill and decode legs (pkg/sidecar/proxy/connector_nixlv2.go, and the same pattern in connector_shared_storage.go, connector_p2p.go, connector_mooncake.go, connector_sglang.go, decode.go). encoding/json sorts map keys at every depth when marshaling a map, so the key order of every nested object the sidecar does not touch is lost.

tools[].function.parameters.properties is the visible case. Chat templates render tools into the prompt verbatim ({{ tools | tojson }}), so the model sees the parameter schema in alphabetical order instead of the order the client sent. Schema order matches the training distribution; a reordered schema can degrade tool-calling accuracy (missing or mis-mapped parameters).

Providers are tested for this. The MiniMax-Provider-Verifier Scenario-Check-Pass-Rate metric checks whether the provider preserves the original JSON key order in tool definitions and is expected to be 100%; reordering parameters.properties (alphabetical sorting) fails it. A P/D deployment fronted by the sidecar fails this check.

Reproduction

Pure encoding/json, no sidecar needed:

original := []byte(`{"tools":[{"type":"function","function":{"name":"example","parameters":{"properties":{"some-parameter":{"type":"string"},"xyz":{"type":"string"},"123":{"type":"string"},"another-parameter":{"type":"string"}}}}}]}`)
var req map[string]any
json.Unmarshal(original, &req)
req["kv_transfer_params"] = map[string]any{"do_remote_decode": true}
out, _ := json.Marshal(req)
fmt.Println(string(out))

Output orders properties as 123, another-parameter, some-parameter, xyz.

Expected

Fields the sidecar does not modify (tools, messages content, response_format, chat_template_kwargs, ...) are forwarded byte-for-byte to prefill and decode.

Proposed fix

Two approaches, each in its own PR. They fix the same issue; only one should merge.

A. Decode only inspected fields, keep the rest as json.RawMessage (#2591)

Decode the top level into map[string]json.RawMessage. Fields the sidecar reads or rewrites (stream, max_tokens, kv_transfer_params, ...) are decoded into Go values; every other field stays a json.RawMessage, so json.Marshal emits its original bytes. messages is decoded lazily by the two code paths that read it (EC item extraction, chunked decode).

• Keeps the existing map[string]any model; connectors change one call each.
• Requires an explicit allowlist of inspected fields. A field read by a connector but missing from the list decodes as json.RawMessage, so a req[field].(bool) type assertion silently reports the field as absent.

B. Edit the raw body in place with sjson (#2595)

Keep the validated body as []byte. Connectors apply their edits with sjson.SetBytes / DeleteBytes through a small requestBody helper and read values with gjson. Every field the sidecar does not set is forwarded byte-for-byte, including number formatting and escaping.

• No allowlist to maintain.
• NIXLv2 no longer saves and restores stream and token-limit fields between legs; each leg is built from the original bytes.
• Larger diff: every connector's body handling changes from map access to path edits. gjson and sjson were already indirect dependencies and become direct ones.

Both PRs include a NIXLv2 test asserting that tools and messages reach prefill and decode byte-for-byte.