Skip to content

[API/SDK] Add ASCII helpers and remove regex from InstrumentMetaDataValidator - #4715

Open
ltowarek wants to merge 6 commits into
open-telemetry:mainfrom
ltowarek:instrument-validator-without-regex
Open

ltowarek wants to merge 6 commits into
open-telemetry:mainfrom
ltowarek:instrument-validator-without-regex

Conversation

@ltowarek

@ltowarek ltowarek commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #4570.

InstrumentMetaDataValidator checks every instrument name with std::regex. Before GCC 16, libstdc++'s regex uses one stack frame per character (GCC bug 86164), so a long name can overflow a small stack. On an ESP32-S3 (GCC 15.2), a 255-character name overflows a 4 KB task.

Changes

  1. Add common/detail/ascii.h with small ASCII helpers (IsAsciiAlpha, HexDigitValue, LowerHexDigit, ...) and use them instead of <cctype> and hand-written copies. <cctype> depends on the locale and is undefined for negative char. In the "C" locale nothing changes, and a test checks every byte. Under other locales, bytes above 0x7F are no longer treated as letters or spaces. TraceFlags::ToLowerBase16 keeps its uppercase output.
  2. Replace the regex with a loop. This also fixes two regex bugs: it accepted a name with an embedded NUL, and it read a string_view past its end. Fix TestUnit, which called ValidateName.

Benchmarks

main (a273994) vs this PR (0e8dda2) , GCC 13.3 -O3, i5-4590, one core, median of 5 runs.

Inject is 5-24% faster and Baggage::ToHeader 11-15% faster. Extract changes by -3% to +4%.

propagator_benchmark, baggage_benchmark and span_id_benchmark
Benchmark main (ns) this PR (ns) Delta
B3PropagatorExtract 220.7 227.1 +2.9%
B3PropagatorExtractEmptyContext 31.0 31.4 +1.3%
B3PropagatorInject 44.9 37.3 -17.0%
B3PropagatorInjectEmptyContext 11.0 11.0 +0.2%
B3PropagatorMultiHeaderExtract 210.8 213.6 +1.3%
B3PropagatorMultiHeaderExtractEmptyContext 43.4 44.3 +2.2%
B3PropagatorMultiHeaderInject 75.5 71.9 -4.8%
B3PropagatorMultiHeaderInjectEmptyContext 11.0 11.0 -0.0%
BaggagePropagatorExtract 492.4 480.1 -2.5%
BaggagePropagatorInject 354.7 291.5 -17.8%
BaggageToHeader180Entries 38,010.3 32,177.5 -15.3%
BaggageToHeaderTenEntries 2,122.8 1,892.7 -10.8%
CompositeTraceContextBaggageExtract 844.7 846.7 +0.2%
CompositeTraceContextBaggageInject 404.9 354.3 -12.5%
CreateBaggageFrom180Entries 51,652.6 52,970.0 +2.6%
CreateBaggageFromTenEntries 2,526.7 2,517.6 -0.4%
ExtractBaggageHavingTenEntries 0.4 0.4 +0.1%
ExtractBaggageWith180Entries 0.4 0.4 +0.0%
HttpTraceContextExtract 277.8 278.8 +0.4%
HttpTraceContextExtractEmptyContext 12.4 12.1 -2.1%
HttpTraceContextExtractWithTraceState 526.4 513.6 -2.4%
HttpTraceContextInject 48.9 37.1 -24.2%
HttpTraceContextInjectEmptyContext 11.0 11.0 -0.1%
HttpTraceContextInjectWithTraceState 120.6 108.6 -10.0%
JaegerPropagatorExtract 238.1 247.8 +4.1%
JaegerPropagatorExtractEmptyContext 16.3 16.3 +0.2%
JaegerPropagatorInject 45.0 37.8 -15.9%
JaegerPropagatorInjectEmptyContext 11.0 11.0 +0.0%
SetValueBaggageWith180Entries 14,439.7 14,096.6 -2.4%
SetValueBaggageWithTenEntries 568.6 575.3 +1.2%
SpanIdConstructor 1.7 1.7 +0.7%
SpanIdDefaultConstructor 1.7 1.7 +0.1%
SpanIdIsValid 2.1 2.1 +0.2%
SpanIdToLowerBase16 8.5 7.7 -9.2%

For significant contributions please make sure you have completed the following items:

  • CHANGELOG.md updated for non-trivial changes
  • Unit tests have been added
  • Changes in public API reviewed

…debase

Character classes in specifications and protocols are defined over ASCII,
yet callers used <cctype>, whose results depend on the global locale and
which is undefined for a negative char, and several of them passed bytes
from headers, configuration files and environment variables straight
through. Others repeated the same range checks and hexadecimal decoding
and encoding by hand.

common/detail/ascii.h provides constexpr IsAsciiDigit, IsAsciiLower,
IsAsciiUpper, IsAsciiAlpha, IsAsciiAlnum, IsAsciiHexDigit, IsAsciiControl,
IsAsciiSpace, ToAsciiLower, ToAsciiUpper, HexDigitValue, LowerHexDigit and
UpperHexDigit, plus EncodeLowerHex and EncodeUpperHex for byte sequences.
TraceState, Baggage, StringUtil::Trim, EnvironmentCarrier, env_variables,
DocumentNode, PercentDecode, InstrumentDescriptor comparison and hashing,
the OpenTelemetry trace state sampler, the environment entity detector,
the OTLP and HTTP client header maps, the curl HTTP time parser and debug
log, UrlDecoder, the HTTP test server, the ETW provider GUID, the
Prometheus sanitizers and the W3C test server use them. The digit table
in trace/propagation/detail/hex.h becomes HexDigitValue's lookup table:
propagators decode trace and span IDs on every request, and the lookup
is faster there than range checks. The encoders replace the hexadecimal
encoding tables and loops in TraceId, SpanId, TraceFlags, Baggage, the
service detector's UUID formatter and the OTLP HTTP and file clients,
and the digit helpers replace them in the trace state sampler and the
ETW UUID. Each keeps its case.

ascii_test checks each classifier against <cctype> in the "C" locale for
all 256 byte values, and each encoder against printf.
Before GCC 16, libstdc++'s std::regex matcher recurses once per input
character (GCC bug 86164), so validating a 255-character instrument name
needs a deep stack and overflows small thread stacks, such as the few KB
an RTOS task gets. Names and units are now checked with a loop that
accepts the same patterns. Unlike the regex, which read the view as a C
string, it checks exactly the view: an embedded NUL is rejected, and a
view that is not NUL-terminated is no longer read past its end, which
accepted an empty view over a valid name and rejected a valid view
followed by invalid characters. Unlike the former non-regex fallback, it
does not read the first character of an empty name, and it uses the
ASCII helpers instead of the locale-dependent std::isalpha and
std::isalnum. meter.h no longer includes <regex> through the validator's
header.

TestUnit checked its valid units with ValidateName; it now calls
ValidateUnit. New cases cover an embedded NUL, non-ASCII characters
(including as the first character of a name), an empty unit and views
that are not NUL-terminated; each fails with the regex. A benchmark
covers construction and validation.
cppcheck reports their callers in SpanId, TraceId and TraceFlags as
reading the uninitialized buffers that the B3 propagator passes in. The
call sites go back to writing two digits per byte with LowerHexDigit and
UpperHexDigit.
The opentracing shim propagation test declared an unused common namespace
alias and included attribute_value.h only for it. The validator
benchmark passes names as nostd::string_view.
@codecov

codecov Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.17355% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 86.83%. Comparing base (cdd17eb) to head (2bc5c85).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...nclude/opentelemetry/ext/http/server/http_server.h 66.67% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4715      +/-   ##
==========================================
+ Coverage   86.81%   86.83%   +0.02%     
==========================================
  Files         525      526       +1     
  Lines       20736    20757      +21     
==========================================
+ Hits        18000    18022      +22     
+ Misses       2736     2735       -1     
Files with missing lines Coverage Δ
api/include/opentelemetry/baggage/baggage.h 98.24% <100.00%> (-0.06%) ⬇️
api/include/opentelemetry/common/detail/ascii.h 100.00% <100.00%> (ø)
api/include/opentelemetry/common/string_util.h 90.00% <100.00%> (ø)
...elemetry/context/propagation/environment_carrier.h 100.00% <100.00%> (ø)
...clude/opentelemetry/trace/propagation/detail/hex.h 100.00% <100.00%> (ø)
api/include/opentelemetry/trace/span_id.h 100.00% <100.00%> (ø)
api/include/opentelemetry/trace/trace_flags.h 100.00% <100.00%> (ø)
api/include/opentelemetry/trace/trace_id.h 100.00% <100.00%> (ø)
api/include/opentelemetry/trace/trace_state.h 98.95% <100.00%> (ø)
...de/opentelemetry/exporters/otlp/otlp_environment.h 100.00% <100.00%> (ø)
... and 16 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ltowarek
ltowarek marked this pull request as ready for review October 19, 2026 05:09
@ltowarek
ltowarek requested a review from a team as a code owner October 19, 2026 05:09
@mateenali66

Copy link
Copy Markdown
Member

reproduced both validator bugs on main. asan flags a heap over-read in regex_match on a view that isn't NUL-terminated, and a 3-byte view over "abc def" is rejected. the new tests fail 7 of 7 against main's validator and pass here. Thank you

@lalitb lalitb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks.

@marcalff marcalff left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the fix.

Independently of the issue with regexp, this fix removes all the hard coded helpers from the code base and replaces them with a clean implementation.

This is a most welcome change, great work.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants