[ManuelOsburg]

No description provided.

[kriboe90]

Hi Manuel,

thanks for adding the feature. For traceability can you maybe elaborate the feature and model behind that here?

Thank you!

[ManuelOsburg]

Hi Kristian,

sure — here is the feature and the model behind it.

What the feature adds. Step 3 computes one additional quantity per LED and image, bgsub_sum_col_value: the local background $\hat B$ is estimated as the median of the positive pixels in the outer 2-px border ring of the LED search area, and then subtracted from every pixel before summing:

$$\hat B = \mathrm{median}\lbrace p_i \in \mathrm{ring} \mid p_i > 0 \rbrace, \qquad S_{\mathrm{bgsub}} = \sum_{\mathrm{box}} \max(p_i - \hat B,\ 0)$$

Median → robust against LED tails and hot pixels; positive-only → raw Bayer arrays mask foreign-channel pixels with exactly 0, which would otherwise pull the median to ~0; clip at 0 → masked pixels stay untouched. The analysis uses it only when reference_property = bgsub_sum_col_val is set in config_analysis.ini — the default pipeline is unchanged (verified bit-identical by the new acceptance suite 07, which also covers reading pre-existing 5-column step-3 files).

The model behind it. The search-area sum contains, besides the LED, an additive background (ambient/stray light plus a small estimator noise floor) that does not dim with smoke. Because this floor $F$ is present in the reference image and the measurement image, the measured transmission becomes (exactly, for a constant floor):

$$T_{\mathrm{meas}} = \frac{S_0,T + F}{S_0 + F} = T + f,(1-T), \qquad f = \frac{F}{S_0+F}$$

which propagates to the extinction coefficient as $\Delta k/k \approx -f,(e^{\tau}-1)/\tau$: the bias is always negative (extinction underestimated) and grows with optical depth. The ring measures $\hat B$ locally and per image, so the correction itself does not rely on the floor being constant in time or on any assumption about its origin.

Empirical basis (Hasenheide data, camera F1L). The LED-free border ring falls only to 0.86 of its reference value while the LED signal falls to 0.51 — direct evidence of light in the box that the smoke does not dim. The floor fraction measured independently from the ring time series ($f = 9.4\ %$ for a dim probe channel) predicts a $-11\ %$ bias via the formula above; $-12.6\ %$ was observed. For the bright production channels $f \approx 1.4\ %$, i.e. a $-2$ to $-4\ %$ effect. In a synthetic-twin test with a constant floor injected into the acceptance-test images, sum_col_val follows the formula prediction within 1 pp across $f = 0\ldots 32\ %$ while bgsub_sum_col_val stays within $+0.2\ %$ of the known ground truth.

[ManuelOsburg]

Hi Kristian,

two test-only commits were added, so the step 3 output and bgsub_sum_col_val itself are unchanged:

• 7b65569: the bit-identity test in suite 07 never read the stripped step 3 files. read_hdf only builds all_parameters.h5 if it does not exist, and the binary from the previous suites was reused. The test now deletes the binary before each analysis run.
• 630fcec: the acceptance test data are generated with a fixed seed. The rmse of bgsub on the clean images varied between runs, from 0.039 to 0.066 against the bound of 0.06.

An update on the model, from the Hasenheide 2.0 data:

• With the small search areas (16–20 px), the 2-px border ring lies in the halo of the LED itself. The ring median was 163 / 358 counts, compared with 6 / 10 counts 25–60 px next to the strip. The halo scales with the LED and largely cancels in I/I0. So bgsub removes a mixture of halo and additive pedestal rather than the pedestal alone.
• In practice this works out well in weakly sooting tests. There, bgsub and a pedestal estimate from side bands next to the strip agree within 0–2 percentage points in LEDSA/LEM. Under dense smoke they differ: LEDSA/LEM on a plateau was 0.80–0.85 raw, 0.90–0.94 with bgsub and 0.96–1.03 with the side-band estimate.
• The "median of the pixels > 0" also overestimates a pedestal of a few counts. Many channel pixels are noise clipped at 0 after the black level subtraction, 35–45 % under smoke.

In my fork I built opt-in follow-ups on top of this PR. The defaults stay bit-identical:

• a side-band pedestal (sidebg_sum_col_val),
• a ring variant with a mean over the channel pixels including zeros (bgsub2_sum_col_val),
• a per-ray validity marking (-rv),
• an automatic reference image selection,
• a few small fixes.

If you are interested, I would offer them as separate PRs once #130 and #131 are merged. Would you merge these two as they are now?

Best,
Manuel