Skip to content

Fix and document the Tinybird benchmark runner - #952

Closed
fallintoplace wants to merge 4 commits into
ClickHouse:mainfrom
fallintoplace:fix-tinybird-runner-auth
Closed

Fix and document the Tinybird benchmark runner#952
fallintoplace wants to merge 4 commits into
ClickHouse:mainfrom
fallintoplace:fix-tinybird-runner-auth

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace the invalid placeholder authorization header with required TINYBIRD_TOKEN and TINYBIRD_HOST environment variables
  • fail on shell, HTTP, malformed JSON, and non-numeric timing errors
  • document the Tinybird CLI setup, data import, pipe generation, token creation, and benchmark invocation
  • use the host selected by tb auth -i, preserve runner failures when logging with tee, and ignore local credentials and virtual environments

Why

AUTH_HEADER=<TOKEN> is parsed as shell redirection, so the runner exits before sending any benchmark requests. The previous instructions also did not cover the setup needed to create the 43 pipe endpoints and run the benchmark.

Validation

  • bash -n tinybird/run.sh
  • verified missing token and host values fail before any request
  • mocked successful Tinybird responses and verified the output contains 43 arrays with three numeric timings each
  • verified malformed and non-numeric timing responses fail
  • git diff --check

Comment thread tinybird/README.md Outdated
# Querying the data

Once the data is inserted you can create the endpoints needed to run the benchmark using pipes. `run.sh` will iterate through each endpoint.
Set `TINYBIRD_TOKEN` to a Tinybird token with read access before running the script.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a Tinybird account but the UI is rather complex and this README covers it only rudimentary.

@fallintoplace Since you seem familiar with Tinybird, maybe I could ask you to add

  • the steps to install the Tinybird CLI in section "Creating an account" above,
  • the steps to create and retrieve a "Tinybird Token" (l. 35),
  • the steps to run the INSERT (l. 24),
  • the exact invocation of run.sh (the PR message has an example)

@rschu1ze rschu1ze self-assigned this Jun 30, 2026
@fallintoplace
fallintoplace force-pushed the fix-tinybird-runner-auth branch from 60ec13d to 56fae7b Compare July 30, 2026 17:35
@fallintoplace fallintoplace changed the title Fix Tinybird benchmark runner auth Fix and document the Tinybird benchmark runner Jul 30, 2026
@rschu1ze rschu1ze removed their assignment Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants