Does the shape of the star-formation distribution tell the two codes apart?

§1Question

The joint study asked whether retuning four feedback parameters can make IllustrisTNG and SIMBA agree on every galaxy observable at once. Its star-formation outputs were two numbers per mass bin: the quenched fraction and the median specific star formation rate of the galaxies that are still forming stars. A distribution has more structure than that: how many galaxies are in between the blue cloud and the red sequence (the green valley), and how wide the star-forming sequence is.

Two new outputs per stellar-mass bin: the green-valley fraction (the share of galaxies with specific star formation rate between 10⁻¹¹ and 10^−10.5 per year) and the width of the star-forming sequence (the 84th minus the 16th percentile of log specific star formation rate among the galaxies above 10^−10.5 per year). Three questions with rules fixed in advance: is the distance between the codes for these outputs alone structural, removable or neither (Q1); do they add separation to the quenched fraction and the main sequence (Q2); do they add separation to the whole joint set (Q3)?

§2Data

Simulations

The 2,054 CAMELS simulations of the earlier studies (IllustrisTNG and SIMBA; 1000 Latin-hypercube simulations and 27 seeds at the fiducial parameters each), snapshots at z = 0, 0.95 and 2.15.

Outputs

Ten bins of 0.25 dex in stellar mass from 10⁹ M_☉, read from three arrays of each Subfind catalogue by range requests (data/ensembles/camels-distshape.json, scripts/ingest_camels_distshape.py); a bin needs at least 10 galaxies. 214 of 620 outputs (these and the earlier study's) are eligible; the shape outputs are eligible at z = 0 and 0.95 and not at z = 2.15, where too few bins hold enough galaxies.

Method

The estimators of the joint study, unchanged: polynomial ridge emulators per code, the noise left when everything else is fixed, the distance minimised over both codes' four parameters, 100 random half-sample splits per set and the margin of one noise unit.

§3Result

• Q1 - shape alone. The green-valley fraction and the width of the star-forming sequence (blocks: 4, outputs: 18) are removable: retuning within the CAMELS ranges brings the two codes within one noise unit on these outputs. Half-sample between-code distance, emulator error removed: median 0.00 noise units (5th to 95th percentile 0.00 to 0.10); same-code controls at most 0.00; margin 1. On all simulations: 0.00 corrected, 0.24 uncorrected, 0.51 at the fiducial parameters.
• Q2 - shape beyond the usual two. Adding the shape outputs to the quenched fraction and the star-forming main sequence adds nothing detectable: the rise beyond what the same-code controls show is below 0.1 noise units even at its largest. Alone, the quenched fraction and the star-forming main sequence is removable; with the shape outputs the set is removable. Excess of the change of the raw between-code distance over the mean change of the two same-code controls, split by split: median -0.163, 5th to 95th percentile -0.207 to -0.099 noise units (n = 100 splits). The sign is negative because each block has the same weight: blocks on which the two codes already agree lower the root mean distance of the set they join. That is dilution, not evidence that the shape closes the gap. The unadjusted change of the between-code distance: median -0.160 (-0.193 to -0.101); of the controls: IllustrisTNG 0.010 (-0.010 to 0.032), SIMBA 0.004 (-0.017 to 0.032).
• Q3 - shape in the joint set. Adding the shape outputs to the joint set of the earlier study adds nothing detectable: the rise beyond what the same-code controls show is below 0.1 noise units even at its largest. Alone, the joint set of the earlier study is structural; with the shape outputs the set is structural. Excess of the change of the raw between-code distance over the mean change of the two same-code controls, split by split: median -0.245, 5th to 95th percentile -0.255 to -0.221 noise units (n = 100 splits). The sign is negative because each block has the same weight: blocks on which the two codes already agree lower the root mean distance of the set they join. That is dilution, not evidence that the shape closes the gap. The unadjusted change of the between-code distance: median -0.245 (-0.252 to -0.219); of the controls: IllustrisTNG 0.001 (-0.003 to 0.008), SIMBA -0.001 (-0.004 to 0.005).
• Validations. Not all hold: V1, V4; read the conclusions above with the qualification below and in section 2. V1: the emulators of the shape outputs are less accurate than the seed noise (median cross-validation error 1.36 and 1.26 times the seed noise; the share within 1.5 times is 0.83 and 0.78, the rule asks for 0.8 each). The emulators' error is part of the noise scale, so it widens the unit in which distances are measured; a low distance therefore means no difference larger than that combined noise could be shown, and the power to see a small shape difference is limited. V4: the point distance of the shape set changes from 0.328 with 4 random starts to 0.213 with 8 (54%). Both are far below the margin of 1: when a distance is this small the surface the search minimises is flat and noisy, and different starts end in different places. A larger number of starts can only lower a found minimum, so the verdict removable does not depend on it; the check after the run (docs/CHECKS-quenching-shape.md) tests this with up to 64 starts.

What the shape adds, in the same splits. The excess is the change of the between-code distance minus the mean change of the two same-code controls; "adds separation" needs its 5th percentile above zero and an equally strong verdict, "adds nothing detectable" needs its 95th percentile below 0.1 noise units.

§4Validation

Sensitivity analyses (reported, none used to choose):

Deviations from the protocol and implementation choices:

• Emulator degrees were chosen on all eligible outputs of the earlier study together with the eligible shape outputs (protocol section 4 says "degree chosen by cross-validation" without naming the outputs): IllustrisTNG 3, SIMBA 3; the earlier study chose 3 and 3.
• Sensitivity S3 (lower threshold) was computed from the same arrays during the same ingest (family gvwide), not by a second ingest, and not used to choose anything.
• Sensitivity S1 removes the two lowest mass bins from every mass-binned family in the sets it is applied to (the same rule as the earlier study's S8), not only from the shape families.
• 100 random splits per set, as the protocol states; the earlier study used 200 for its joint set. The split streams are the earlier study's (same seed, as the estimators are reused unchanged).
• Two simulations per code and set (LH_0 and CV_0, the first snapshot) were printed when the ingest was tested; no distance, emulator or verdict was computed before this run.
• The wording of the report was revised after the run (clearer sentences, an explanation of the negative excess and of the two validations that do not hold); no number was recomputed, and the first output is in the git history.

§5Limits

• Two codes, CAMELS resolution. 25 Mpc/h boxes with 256³ elements: the lowest mass bins hold galaxies of a few tens of star particles, where a green-valley share can be driven by single events. The validation table reports how many outputs rest on few galaxies.
• Instantaneous rates, zero counted as quenched. A galaxy with no star formation at the snapshot is counted as quenched, so the quenched and green-valley shares both depend on how often a code assigns exactly zero.
• Two summaries of a distribution that has more. The position of the red peak or the colour is a different study.
• Estimators not calibrated for this layout. They were calibrated for the joint study's outputs; a result of "adds nothing detectable" means this method did not detect more separation, not that there is none.
• Subfind subhaloes are not galaxies on the sky. No aperture, selection or survey effect is modelled.

§6Reproduce

python3 scripts/ingest_camels_distshape.py       # about 10 minutes; range reads of the public catalogues (needs fsspec, aiohttp)
python3 scripts/analyze_quenching_shape.py       # writes data/analysis/quenching-shape.json and the report
python3 scripts/test_quenching_shape.py          # machinery tests on synthetic worlds and the stale-output check
node scripts/build-site.mjs                      # regenerates this page from the JSON