July release candidate (DO NOT MERGE) - #919
Draft
ChristianPavilonis wants to merge 325 commits into
Draft
Conversation
On the SSAT proxy path the browser calls /auction against the trusted-server edge domain (e.g. ts.example.com), which was leaking into ext.trusted_server.request_host on the outbound Prebid Server request. That field must track the publisher's own domain instead, matching site.domain/publisher.domain and what PBS's trusted_server verification module expects.
A client framework can replace the ad divs after GPT slots were bound to them: the publisher's React app serves ids like `ad-header-0-_R_ssr_` and swaps them for client ids (`ad-header-0-_r_1_`) during hydration. GPT is left holding slots whose element no longer exists — GAM reports "defineSlot was called without a corresponding DIV", still fetches a creative for them, and the bid is silently wasted with nowhere to render. Waiting for the divs to merely exist cannot help, because at adInit() time the server-rendered divs are present and are only later replaced. So rather than delaying the initial ad request, detect the swap after the fact: a debounced MutationObserver armed after adInit() looks for TS-defined slots whose element left the document and re-runs adInit(), which destroys the orphans and re-binds against the live DOM — reusing the publisher's own slot for that div when they have since defined one. Bounded deliberately, since each re-bind re-requests the affected slots: a 250ms quiet period, a 5s watch window (measured: the swap lands ~2-3s in, so the existing 2s SPA wait would miss it), and at most two re-binds per page load, with the attempt budget shared across the adInit() the watcher itself triggers so it cannot loop. Verified against the live page through the dev proxy: two orphaned slots were detected and re-bound, leaving zero orphans, with all three slots tracing to live client-id elements. (cherry picked from commit e1badbb)
The panel collapsed each slot to a single row with a ×N counter, so a publisher page that refreshes its slots on every render (autoblog's ad-service refreshes from its own slotRenderEnded handler) showed a climbing number instead of what actually happened. Keep an append-only `window.tsjs.renderLog` alongside the per-slot `renders` registry and render it newest-first, one entry per render, with a wall-clock time and a `#N` sequence. The log is trimmed to the most recent entries so a page that refreshes indefinitely cannot grow it without bound; `renders` still collapses per slot for "did this ever render" checks. Also badge every slot that actually shows a creative, not just confirmed TS renders. Gating the badge on `ok` meant production — where inject_adm_for_testing is off and TS only applies GAM targeting — never displayed one at all, since every slot is honestly `gam-only`. The badge now carries its status colour and mark: green ✓ for a confirmed TS render, blue ◐ for gam-only. Slots with nothing on screen (`empty`) or nothing visible (`hidden`) stay unbadged, as there is no creative to label. (cherry picked from commit 013f5e6)
The server-side auction runs once per navigation, but the slotRenderEnded handler read the winning bid out of window.tsjs.bids on every render. That map never changes, so each publisher-driven GAM refresh re-stamped the page-load auction's id, bidder and adm hash and labelled itself `ssat` — claiming a render the server-side auction never produced. A slot refreshed six times over a minute showed six `ssat` rows all pointing at one auction. Scope the claim to the render that actually consumes it: adInit arms a per-slot flag when it applies bid targeting, and the first slotRenderEnded clears it. Later renders are recorded as `gam-refresh` with the stale attribution dropped from both the record and the DOM markers, since GAM re-requested the slot on its own and the returned creative cannot be traced to any Trusted Server auction. These rows are where the client-side /auction path will report real attribution via Prebid's bidWon once that bundle ships; labelling them `ssat` hid that gap instead of showing it. The /auction recording path is unchanged. (cherry picked from commit c4999f7)
Three trace-panel fixes, all surfaced by the gam-refresh rows the previous commit introduced: - Restore the `gam:filled`/`gam:empty` marker on gam-refresh rows. It was gated on `path === 'ssat'`, which hid GAM's own fill signal on exactly the rows where "did GAM fill it this time" is the whole question. - Stop rendering absent attribution as `? · ?` and `auction ?`. An unattributed GAM refresh carries no bidder/hash/auction id by design, so the row now says `no TS attribution` and drops the auction segment rather than looking like a failed lookup. - Give each render a page-global `seq`, shown as `#N` on both the panel row and the on-creative badge so the two point at each other, and mark the row still live for its slot as `◂ current`. The per-slot render count keeps its own `×N`. seq is module-scoped, not stored on window.tsjs, so a re-executed bundle restarts the sequence instead of handing two renders one number. (cherry picked from commit c0811bc)
The visible row text already says "no TS attribution" for a gam-refresh, but the badge title and row hover tooltip still rendered the same absent fields as `?`, which reads like a failed lookup rather than "there is nothing to attribute here by design". Switch both to `—`, matching the `gam_empty ?? '—'` convention the row tooltip already used elsewhere in the same list. (cherry picked from commit b4908cd)
The pbRender bridge has two branches for serving a winning SSAT bid into GAM's Universal Creative: fetch from PBS Cache, or use `bid.adm` directly when present (added by the SSAT inline-creative work, and the only path production actually exercises for bidders that carry markup inline). The PBS Cache branch calls recordBridgeRender after replying; the inline-adm branch replied, fired win/billing beacons, and logged success, but never called it — so this render path, the strongest confirmation signal SSAT has (TS supplies the exact bytes GAM's own creative asked for by name), was invisible to the trace panel. Every SSAT row capped at gam-only even when this branch was serving real creative. Add the missing call, matching the PBS Cache branch's placement. Add regression coverage on both branches — neither had a trace assertion before, so this gap could recur silently on either one. (cherry picked from commit 25316dd)
…re absent Confirmed live on autoblog: Kargo's response carries neither a Prebid Cache UUID nor an `adid`, so hb_adid was omitted for every SSAT bid from that bidder. That broke the pbRender bridge's reverse lookup (adId -> hb_adid) for GAM's Universal Creative postMessage protocol — verified in the browser that GAM sends real 'Prebid Request' messages with real adIds on this account, but window.tsjs.bids[slot].hb_adid was undefined for all three winning bids, so the bridge could never match any of them. Confirmed rendering (injected: true) was structurally unreachable for this bidder regardless of what GAM did. bid.bid_id (the OpenRTB bid's own `id`, always present per spec) already flows through the pipeline unused for this purpose. It is unique per bid instance rather than a creative identifier, but that is exactly what hb_adid needs here: a stable value GAM's Universal Creative echoes back so the bridge can find the winning bid. Add it as the last-resort fallback, after cache_id, the APS renderer's bid id, and ad_id — all three still take priority where present, locked in by test. (cherry picked from commit 281df38)
# Conflicts: # crates/trusted-server-core/src/ec/prebid_eids.rs # crates/trusted-server-core/src/integrations/prebid.rs # crates/trusted-server-core/src/publisher.rs # crates/trusted-server-core/src/settings.rs # crates/trusted-server-js/lib/src/integrations/prebid/index.ts # crates/trusted-server-js/lib/test/build-prebid-external.test.mjs # crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts # docs/guide/integrations/prebid.md
build_auction_request derived publisher.domain, site.domain, and the page URL host from the incoming request Host header. On the SSAT proxy path that header is the trusted-server edge host (e.g. the staging domain), which then leaked into the outbound OpenRTB bid request and, through it, into injected creatives and the IAS brand-safety pixel. Source these fields from settings.publisher.domain instead, matching what convert_tsjs_to_auction_request already does on the /auction endpoint path. Closes #936
…n requests) into rc/july # Conflicts: # crates/trusted-server-core/src/publisher.rs
hb_adid is not unique per bid: absent PBS Cache it falls back to a creative id a bidder may reuse across slots (observed: three IX bids sharing one id). The bridge matched the first bid whose hb_adid equalled the requested adId, then rejected on the slot-ownership guard, so every slot but the first rendered blank. Resolve the bid by the requesting slot and verify its hb_adid matches the request, so each slot renders its own creative regardless of duplicate ids. The adId check still blocks a slot A iframe from pulling slot B's creative and beacons.
The pbRender bridge sized every inline response from the first configured slot format, while the winning creative's own width/height were emitted only inside the testing-only debug_bid. A multi-size slot whose winner is not the first format therefore rendered at the wrong size (clipping or whitespace). Emit w/h in the normal bid map and AuctionBidData, and prefer them in the inline bridge response, falling back to the first slot format only when absent.
The inline render path forwarded adm without resolving the auction-price
macro. URL rewriting then serialized query pairs, encoding the literal
${AUCTION_PRICE} into %24%7BAUCTION_PRICE%7D inside the signed proxy/click
URL — so trackers received an encoded macro rather than the clearing
price, and signing locked the wrong value.
Add expand_auction_price_macro and call it from build_bid_map before
sanitize_creative_html and rewrite_inline_creative_html, using the exact
winning CPM. Only the clear-price token is expanded; the encrypted
${AUCTION_PRICE:B64} variant is left for the DSP.
rewrite_inline_creative_html hard-coded https://{publisher.domain},
discarding the incoming scheme, subdomain, and port. publisher.domain
cannot carry a port, deployments may serve a subdomain, and Axum/Viceroy
dev runs over HTTP with a port — so inline proxy/click URLs resolved
against the wrong origin with no render-time fallback.
Thread the trusted request origin (scheme://host, host including any
port) through write_bids_to_state and build_bid_map into
rewrite_inline_creative_html. Initial navigation derives it from the
buffered request host/scheme; SPA page-bids from RequestInfo. Falls back
to the configured publisher domain only when the origin is unknown.
Covers HTTP localhost with a port, a request subdomain differing from
publisher.domain, and a non-default HTTPS port.
extractCachedAdm reduced the cached bid to its adm string, so the cache
fallback sized every render from the first slot format and left price
macros unresolved. Replace it with parseCachedBid, which retains the
cached creative dimensions (w/h or width/height) and clearing price.
The fallback now sizes from the cached dimensions (slot format only when
absent) and expands ${AUCTION_PRICE} from the cached price before
responding. Raw-markup bodies stay supported as the adm-only variant.
Firing a cached win-notification URL is deferred: it is a billing side
effect and the exact cache field/dedup contract needs a real PBS Cache
payload to verify before emitting.
The design named rewrite_creative_html and omitted the render-metadata
requirements the code now enforces; the plan's tasks were unchecked and
predated the shipped divergences. Add an "Implementation reconciliation"
section to the design (inline rewriter, request-origin URLs, w/h and
${AUCTION_PRICE} render metadata, structured cache decode) and update the
components table; mark the plan superseded-but-completed pointing at it.
Threading the request origin pushed the private collect_stream_auction helper to 8 arguments. Its arguments mirror the AuctionCollectCtx fields the caller destructures, so a parameter struct would only duplicate that context; suppress the lint instead.
CodeQL flagged the click guard's navigation and href-persist sinks: the inputs are creative-controlled DOM attributes, so a javascript: value in data-tsclick or href could reach location.href or be written back as an anchor href. Resolve every candidate URL against the pinned trusted base and require an http(s) scheme before navigating or persisting, failing closed otherwise. Also replace an as-any cast in the new click test now that main lints the full JS package (#984).
CodeQL still flagged the href write in persistRebuiltClick: it validated the candidate URL but then wrote the original creative-controlled string. Write the sanitizer's resolved output instead — the http(s)-checked URL absolutized against the pinned trusted base. Beyond closing the taint flow, an absolute href keeps the anchor's default navigation working inside the srcdoc iframe, where a relative value would resolve against about:srcdoc. Tests updated to expect the absolute forms.
- Navigate the observer-repaired click. The mutation observer writes the GET rebuild fallback to href while keeping the canonical signed click in data-tsclick; a later click canonicalized the fallback against that canonical URL, failed the base comparison, and navigated the pre-mutation click. Remember the pending rebuild per anchor and navigate it, and skip no-op attribute writes that would otherwise wake the observer in a loop. - Accept origin-form request targets in the shared signed-target parser. Browsers send /path?query and the Axum adapter forwards it verbatim, so url::Url::parse rejected it as relative — breaking /first-party/click, /first-party/proxy and GET /first-party/sign there, including the second hop of the new rebuild redirect chain. - Inject the click-guard runtime into body-less creative fragments. lol_html matches no <body> in a bare fragment, so common adm shapes shipped without the guard while surviving bidder script could still mutate rewritten links. - Bound rewritten output, not just raw input: rewriting expands every URL into a signed proxy/click URL, so a sub-cap creative could amplify well past it. Reject once the output exceeds the cap. - Fail closed on rewriter errors instead of returning partially rewritten markup, matching the sanitizer. - Treat an explicit empty adm as a supplied creative, not an absent one, so it cannot re-enable the raw PBS Cache fallback without a rejection. - Stamp the first-party origin into the srcdoc document from the parent page and prefer it, then location.origin, over the inherited document.baseURI, which honours a publisher <base> and is not a trustworthy boundary. - Allow proxied assets to load cross-origin. The opaque creative origin makes /first-party/proxy cross-origin, blocking CORS-mode subresources; assets are fetched without client credentials, so a wildcard allow is sound. - Docs: correct the remaining stale auction-orchestration sections, note that #982 only affects deployments with renderGuard enabled, document the cache coordinate policy, and scope the Spin route test's comment to what it asserts.
The srcdoc/opaque-origin conditions the click-guard recovery depends on cannot be reproduced in jsdom: document.baseURI stays an ordinary http URL and window.origin is a real origin, so the about:srcdoc branch and the CORS-blocked POST are never exercised. The Playwright harness already runs in CI, so pin the path there: build the sandboxed srcdoc the client builds (stamped origin, runtime, root-relative signed click), mutate the link, click it, and assert the iframe navigates to /first-party/proxy-rebuild carrying the mutation.
This reverts commit 8589431.
The trace test imported node:child_process after vitest, tripping import/order under eslint; auction.ts and gpt/index.ts carried non-prettier line wrapping that the failed eslint step masked in CI. The CHANGELOG Changed section listed the APS inventory override and renderer preservation entries twice after an earlier merge splice.
Brings in the squashed versions of PR #957 (per-section gam_unit_path templates) and PR #967 (decoupled prebid tsjs shim). rc/july already carried pre-squash versions of both feature branches, so every conflict was the same feature at two different review stages. Conflict resolution: - creative_opportunities.rs, publisher.rs, settings.rs, docs/guide/configuration.md, CHANGELOG.md — took main's refined gam_unit_path work: the 100-byte dynamic render cap (MAX_DYNAMIC_GAM_UNIT_PATH_BYTES), Option-returning render_gam_unit_path/build_slot_json, match_renderable_slots, the narrowed gam_network_id requirement (only when a default path or {network_id} template consumes it), section_segment finalization marker, and the accompanying tests. - creative_opportunities.rs validate_runtime — kept rc/july's providers.aps compatibility warning on top of main's validation body. - prebid index.ts and its test — kept rc/july's render-trace and APS renderer additions (installPrebidRenderTrace, apsRenderer fixture, extra mocks); the rest matches main. Verified: cargo fmt, all six clippy targets, test-fastly / test-axum / test-cloudflare / test-spin, the parity suite, JS vitest, and JS + docs prettier checks all pass.
…e absent Bidders that return neither a Prebid Cache UUID nor an `adid` produced no `hb_adid` in `window.tsjs.bids` at all. `adInit` only sets targeting keys that exist on the bid, so GAM never received an `hb_adid` key, the Universal Creative's `%%PATTERN:hb_adid%%` expanded to empty, and the render bridge rejected the resulting `Prebid Request` message for want of an ad ID. The line item won and served its wrapper, but the creative never rendered. Add `Bid::bid_id`, populated from the OpenRTB bid object's own `id`, and use it as the last-resort `hb_adid` source. Per spec `id` is mandatory, so this closes the gap for every bidder. It is unique per bid instance rather than a creative identifier, which is exactly what `hb_adid` needs here: a stable value GAM echoes back verbatim so the bridge can find this winning bid. `cache_id` and `ad_id` keep priority in that order — locked in by test, since the Universal Creative treats `hb_adid` as the Prebid Cache lookup key whenever `hb_cache_host`/`hb_cache_path` are present. `bid_id` is carried as its own field rather than folded into `ad_id`, which is exposed raw in the debug bid and would mislead consumers treating it as a creative identifier. Verified: cargo fmt, all six clippy targets, test-fastly / test-axum / test-cloudflare / test-spin, the parity suite, JS vitest, and JS + docs prettier checks all pass. The new bid_map test was confirmed to fail with the fallback removed.
rc/july already carried `Bid::bid_id` (added by the APS OpenRTB and render
tracing work), so most conflicts were the fix branch re-adding a field this
branch already has — resolved in favour of rc/july, which populates it for
real instead of `None`.
The substantive resolution is the `hb_adid` source chain in `build_bid_map`.
rc/july scopes the bid ID to typed renderer bids via `renderer_bid_id`; the fix
branch adds it as an unconditional last resort. Both are kept:
cache_id → renderer_bid_id → ad_id → bid_id
Non-APS bidders that return neither a Prebid Cache UUID nor `adid` now reach
the last resort instead of shipping no `hb_adid` at all, while APS renderer
bids keep resolving through their selected upstream bid ID as before.
Test resolutions keep rc/july's fixtures (`ordinary-ad-id`, the `creative_id`
and `renderer` fields) and take the fix branch's tightened assertions and the
`should-be-ignored-bid-id` values that prove `cache_id` and `ad_id` still win.
Both APS renderer tests are preserved alongside the two new precedence tests.
Verified: cargo fmt, all six clippy targets, test-fastly / test-axum /
test-cloudflare / test-spin, the parity suite, JS vitest, and JS + docs
prettier checks all pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidates the July release-candidate changes for review and integration testing before the individual changes merge to
main.Included pull requests
Already included through
mainPending Review / Merge