Study · Matching definitions
Can documented definitions close the gap in the z ≈ 0 mass function?
In plain words
Putting 9 simulations' stellar mass functions on the same footing did not measurably shrink the disagreement between them.
We put every simulation's stellar mass function on the same footing: the same way of counting stars and the same aperture. Then we asked how much of the disagreement between them goes away. The best guess is 0.14 of it (1 would be all of it). But the 95% range, −0.89 to 0.51, includes zero. By the rule we wrote down before looking, the answer is "not detected". The data cannot say whether matching definitions explains part of the gap.
Abstract
Simulated stellar mass functions at z ≈ 0 differ between simulations. With a protocol written before any of the numbers existed, each of 9 simulations is expressed in one IMF, without added mass scatter and in a 30 pkpc aperture, using only documented definitions and aperture maps measured inside single runs. The statistic R is the fraction of the spread across simulations (in the stellar mass at fixed abundance) that disappears. R = 0.14 (95% interval −0.89 to 0.51; permutation p = 0.08): not detected by the rule fixed in advance.
- Status
- Pre-registered · result
- Protocol
- PROTOCOL-harmonization.md
- Analysis
- ANALYSIS-harmonization.md
- Script
- scripts/analyze_harmonization.py
- Dataset
- v0.24.0
Limits in brief. One relation at one epoch, 9 simulations. The test could not have seen an aperture effect of the size measured inside single runs, so "not detected" is not evidence that definitions do not matter; the power analysis in §4 says what could have been seen.
Question
Do the documented differences in IMF, mass scatter and stellar-mass aperture explain the spread between simulated z ≈ 0 stellar mass functions?
The definitions study asks a similar question and cannot answer it: it reuses each curve in many pairs, has few matched pairs per test, and 0 of 18 tests survive correction. This study replaces it with one statistic on one relation at one epoch, specified before it was computed.
Data and rule
Each of the 9 simulations’ z ≈ 0 stellar mass functions is expressed in one IMF, without added mass scatter and in a 30 pkpc aperture. Only documented definitions and aperture maps measured inside single runs are used. The coordinate is the stellar mass at fixed abundance. R is the fraction of the spread across simulations that disappears after harmonizing.
The rule is fixed before the statistic exists. The protocol records what had been looked at when it was written, and each amendment is dated with what had been seen at the time.
Result
Result: not detected. R = 0.14, 95% interval −0.89 to 0.51, permutation p = 0.08.
| Operator alone | R | 95% interval |
|---|---|---|
| imf | 0.26 | 0.00 to 0.52 |
| scatter | −0.33 | −1.13 to 0.00 |
| aperture | 0.01 | −0.02 to 0.06 |
Limits
- Power. Injecting aperture effects of the measured size (and up to twice that) into the same simulations is not detected in 80% of draws by the same rule (largest detection rate 3%), so a non-detection of aperture effects here is uninformative about whether they exist.
- Sampling noise is not modelled. Few galaxies per bin in the tail inflate the dispersion before and after harmonization alike and lower the ceiling on R; a sensitivity analysis in the analysis document addresses it.
- One relation, one epoch. The result says nothing about other relations or redshifts.
- Documented definitions only. Where a simulation’s paper does not state a definition, no value is assumed (Methods, limits).
The pre-specification, every amendment with what had been seen when it was made, the sensitivity analyses and the deviations are in ANALYSIS-harmonization.md.
Reproduce
python3 scripts/analyze_harmonization.py # writes the analysis document and data/analysis/harmonization.json
python3 scripts/test_harmonization.py
node scripts/build-site.mjs # regenerates this page from the JSON