# Reproduce the great tit clock training-reuse study

The study is an exact reconstruction and conditional sensitivity analysis of the developmental clock in Haller et al. (2025), DOI https://doi.org/10.1111/1755-0998.14128. It does not develop a new independently validated clock or establish a physiological development rate.

## Download and run

Download `https://research.thecabin.io/round-004/results/029_reproducibility.tar.gz` and compare its SHA256 with `029_package_manifest.json` beside it. Extract the archive, then enter `great-tit-clock-study/`.

Use Linux x86-64, Docker, and a Python environment with the versions in `requirements-study.txt`. Python 3.11 is recommended for available binary wheels of these pinned versions:

```bash
python3.11 -m venv .venv
.venv/bin/pip install -r requirements-study.txt
PYTHON_BIN="$PWD/.venv/bin/python" bash run_study.sh
```

The original and verification runs used Python 3.12.3 with the listed package versions, including Ubuntu's matplotlib build, in an existing isolated environment. The clean verification used a new output directory, physically copied and hashed inputs, and freshly compiled R dependencies. It reused the installed Python environment; it was not a fresh operating-system or Python-dependency installation. Floating point and rendered-file bytes may vary on other BLAS/font/platform builds; see the expected scientific values below.

The Docker image is pinned to:

```
rocker/tidyverse@sha256:470ba4d09f861cddef6e0a6e4d07849463f1a1ab817f05357a89027932513bfc
```

Docker may need to download this large image once if absent. R is 4.0.3; glmnet is 5.0. The five exact R dependency source tarballs are included and hash-checked. The first run compiles them into `inputs/r-library/`, using one CPU and a 2 GiB memory limit. Later runs reuse that library. Analysis containers run with `--network none`, one CPU, one BLAS/OpenMP thread and nice 19. The Python stages also use one BLAS thread and nice 19. No GPU, credentials, BirdNET, web service or scheduler is required. Allow several minutes for a first source build and several minutes for analysis. Keep several GB free for the cached Docker image and temporary compilation.

The archive includes all measured input files required by this analysis; it does not require re-fetching Dryad during a normal run. `BIO_DISCOVERY_CACHE` can select a different directory with the same `great_tit_clock/`, `great_tit_broods/` and `r-sources/` layout. `CLOCK_R_LIBRARY` can select an already built compatible library. `PYTHON_BIN` selects the interpreter. Outputs go beside the copied scripts, in this round's `results/` and `figures/`; run a copy if preserving an existing output set. None of the earlier three study pipelines is called.

## What runs

1. `verify_inputs.py` checks all 12 exact input/source files against `evidence/029_input_manifest.json`.
2. `native_clock_fit.R` reconstructs the full published clock; `verify_native_calibration.py` requires the exact nonzero support and maximum coefficient discrepancy below 1e-10 **before exclusion fits**.
3. `native_clock_exclusions.R` executes individual, origin-brood, cross-foster-pair and rearing-brood exclusions, foldwise filtering, half/double lambda, and tight-convergence controls.
4. `clock_reuse_pilot.py` reproduces published-coefficient predictions and the original REML treatment model.
5. `summarize_clock_exclusions.py` evaluates the frozen practical gates and runs 5,000 brood and pair bootstrap resamples for each fitting variant.
6. `clock_controls.py` performs the independent Python/KKT numerical checks, metadata joins, 5,000 cluster bootstrap draws for each treatment specification, library adjustment, unseen-group subsets and leave-one-group influence.
7. `verify_membership.py` checks that all 34 overlapping sample IDs have identical measured methylation values at every shared clock site. The 33 age-14 predictions must also match the fixed clock.
8. `render_clock_study.py` creates `study.html` and four SVG/PDF/PNG figure sets from the measured outputs.

`clock_math.py` supplies the independent numerical and bootstrap helpers. The original `clock_exclusion_validation.py` is retained as a historical failed calibration attempt; its standalone main fails the exact-support gate and is intentionally **not** called by the completed workflow. It must not be mistaken for the primary native pipeline. Failed development logs are retained separately for transparency.

## Expected scientific results

- 67 training samples from 66 individuals; 11,116 candidates, 3,184 filtered CpGs, 115 selected clock sites.
- 34 training IDs among 102 deposited test samples; 33 among 96 age-14 birds. Shared methylation measurement difference: exactly zero.
- Full native coefficient discrepancy versus the published supplement: about 1.02e-14.
- The same 33 birds represent 22 origin broods and 15 cross-foster pairs.
- Mean full-fit minus excluded prediction: individual **0.6366253368 days**, origin brood **0.7096771442 days**, pair **0.7616731987 days**.
- Individual origin-brood-bootstrap 95% interval **[0.2468486980, 1.1076774734]**; brood exclusion **[0.3029542429, 1.1821780050]**. Both primary practical gates pass.
- Pair exclusion with pair bootstrap: **[0.3729577616, 1.2623567619]**.
- SD among the same 33 predictions: **0.0774920849 days** full-fit and **1.2752498168 days** brood-excluded.
- Reduced-control original REML estimate **0.5823084534 days**; held-out-only **0.6707370362**; hybrid brood exclusion **0.7668550716**. The positive direction remains.
- All primary and sensitivity results are in `029_validation_summary.json`; full model/cluster controls are in `029_controls_summary.json`.

For numeric verification on another platform, use an absolute tolerance of 1e-6 days for predictions and contrasts, and allow 1e-5 for the custom REML variance optimization. Do not require byte equality across differing Python, BLAS or plotting builds. The exact support calibration is deliberately stricter for the pinned native image.

## Prespecified and exploratory parts

The pilot protocol, larger-test protocol and solver/grouping amendment are preserved under `evidence/`. The larger protocol was committed in `16c5e39`; the native solver amendment and supplemental cross-foster-pair controls were committed in `e987f48` before exclusion outputs. The strict Python support gate failed first; that failure was retained rather than silently changing the gate. All primary biological thresholds were unchanged.

The same underlying data informed the pilot and the larger computational exclusions. This is not an independent cohort replication. The original hyperparameters and candidate selection remain fixed; foldwise filtering is a sensitivity control. Bootstrap intervals condition on learned clocks, do not refit the corpus in each bootstrap draw, and do not establish universal clock accuracy. Hybrid treatment analyses combine predictions from different fitted models. The study describes these restrictions throughout.

## Inputs, credit and licenses

- Haller et al. matrices and metadata: Dryad DOI https://doi.org/10.5061/dryad.wstqjq2wq, frozen version 344507; CC0. Original public version archive: `https://datadryad.org/api/v2/versions/344507/download`.
- Haller et al. coefficient supplement: https://www.ebi.ac.uk/biostudies/files/S-EPMC12415945/MEN-25-e14128-s001.xlsx, corresponding to the CC BY 4.0 source article. Original filename is recorded; the unmodified packaged copy is named `clock_coefficients.xlsx`.
- Author notebook: Zenodo https://zenodo.org/records/14192661. It is cited and hash-recorded; the notebook PDF is not needed for execution and is not copied into this package.
- Sepers et al. (2023) brood metadata: Dryad DOI https://doi.org/10.5061/dryad.s7h44j1bv, frozen version 261370; CC0. Corresponding article: https://doi.org/10.1111/mec.16973. Unmodified UTF-16 data and README are included.
- Five unmodified R source packages: `iterators`, `foreach`, `shape`, `RcppEigen`, `glmnet`. Each original tarball retains its DESCRIPTION, notices and license files. Source URLs and hashes are in `029_r_sources_manifest.json`; the pinned image and Python libraries retain their upstream licenses.

This is a research reanalysis, not an endorsement by the source authors. No manuscript submission or contact with those authors has been made. The novelty reviews explicitly credit earlier cross-validation, wildlife-clock and grouped-validation work.
