Calibration

CalibrationProblem

Defines a full calibration problem: model inputs, parameters to infer, observed data, and how to compare simulated to observed output.

Fields

• inputs::InputFolders: Base model configuration shared across all calibration runs.

• parameters::Vector{CalibrationParameter}: Parameters to calibrate, stored as CalibrationParameter objects that track both the original user-supplied variation and the derived LatentVariation used internally. Pass any combination of DistributedVariation, CoVariation{DistributedVariation}, or LatentVariation{<:Distribution} to the constructors — conversion is automatic.

• observed_data: Observed summary statistic in whatever form the distance function expects as its second argument.

• summary_statistic: a QoI, a vector of them, or a plain function — ideally one that declares it takes a Simulation, f(s::Simulation) or (s::Simulation) -> …, since one that does not is warned about. In every case the measurement is made once per simulation and the replicates are combined by reduce (mean for a plain function; a QoI is how you choose otherwise, and its reduce receives every replicate's value, so a step that must happen after averaging goes there). A single QoI or a plain function reports its value directly; a vector of QoIs reports a Dict keyed by QoI name.

  The annotation matters because the previous contract called a bare function once per monad and let it aggregate however it liked. An unannotated argument is ambiguous between the two, and reinterpreting one silently would change results without raising. It is warned about rather than refused, since refusing every unannotated function would also reject sim -> measure(sim), the natural way to write a new-contract lambda. The warning is transitional and goes in v0.10.

• distance::Function: (simulated, observed) → Float64. simulated is the return value of summary_statistic; observed is observed_data. Built-in: mseDistance — handles Dict, Vector, and scalar inputs.

• n_replicates::Int: Number of replicate simulations to run per proposed particle (default 1). Values > 1 reduce stochastic noise in each particle evaluation at the cost of N× more compute.

• reference_variation_id::VariationID: Base variation ID establishing fixed parameter values that apply to every particle evaluation. Obtain from a reference monad: createTrial(inputs, fixed_dvs...; n_replicates=0).variation_id.

Examples

# Short run for testing — set max_time via a reference
ref = createTrial(inputs, DiscreteVariation(["overall","max_time"], 12.0); n_replicates=0)

# Your own per-simulation measurement: reach into this simulation's output and return a number.
# Parsing raw output is the backend's job, so in practice you would call the loader your simulator
# package provides (keyed by `simulationID`); this reads a summary file the simulator wrote.
function countDefaultCells(sim::Simulation)
    counts = joinpath(pathToOutputFolder(sim), "final_cell_counts.csv") |> CSV.File |> DataFrame
    return Float64(only(counts[counts.cell_type .== "default", :count]))
end

# A vector of QoIs reports a `Dict` keyed by QoI name, so `observed` is keyed the same way.
observed = Dict("default" => 100.0)
problem = CalibrationProblem(
    ref,
    [DistributedVariation(:config, xml_path, Uniform(1e-7, 1e-4))],
    observed,
    [QoI("default", countDefaultCells)],
    mseDistance
)

# Covaried parameters — one latent CDF draw moves both targets together
problem2 = CalibrationProblem(
    ref,
    [CoVariation(dv_birth_rate, dv_death_rate)],
    observed,
    summary_fn,
    mseDistance
)

CalibrationParameter

Internal type pairing an AbstractCalibrationSource (the original user-supplied variation) with the derived LatentVariation{<:Distribution} used by the ABC-SMC algorithm.

CalibrationParameter objects are stored in a CalibrationProblem and are passed through the calibration loop. The source is used only for:

• Writing human-readable display CSVs with interpretable target parameter values.
• Serializing to problem.jld2 via JLD2 for resumeABC without re-supplying the original problem.

Users never construct CalibrationParameter directly — it is created automatically when building a CalibrationProblem from DistributedVariation, CoVariation{DistributedVariation}, or LatentVariation{<:Distribution} arguments.

Fields

• source::Union{DVSource,CVSource,LVSource}: The original variation type for provenance.
• lv::LatentVariation{<:Distribution}: The derived latent variation used internally by the ABC-SMC loop.

AbstractCalibrationMethod

Abstract supertype for calibration methods. Concrete subtypes define the algorithm and its settings.

Current implementations:

• ABCSMC: Approximate Bayesian Computation — Sequential Monte Carlo

Future implementations may include GP-accelerated ABC, Bayesian optimization, etc.

ABCSMC

Settings for ABC-SMC (Approximate Bayesian Computation — Sequential Monte Carlo) calibration (Toni et al. 2009, Beaumont et al. 2009).

Fields

• population_size::Int: Number of accepted particles per generation (default 100).
• max_nr_populations::Int: Maximum number of SMC generations (default 10).
• minimum_epsilon::Float64: Stop when the maximum accepted distance drops to or below this value (default 0.01). Acts as a hard floor: the algorithm always runs a full generation at this threshold before stopping, ensuring all accepted particles satisfy it.
• epsilon_quantile::Float64: Quantile of accepted distances used to set the next generation's threshold when no epsilon_schedule is supplied (default 0.5, i.e. median). Ignored when epsilon_schedule is provided.
• perturbation_kernel::AbstractKernel: Kernel for perturbing resampled particles. Accepts GaussianKernel, ComponentwiseKernel, LocalNNKernel, or LocalNNCovKernel. Default GaussianKernel() uses twice the weighted covariance (Beaumont et al. 2009).
• epsilon_schedule::Union{Nothing,Vector{Float64}}: Optional explicit decreasing sequence of acceptance thresholds. When supplied, generation t uses epsilon_schedule[t-1] instead of the adaptive quantile rule. If the schedule is shorter than max_nr_populations, the adaptive rule resumes for remaining generations. All values must be positive and strictly decreasing. Default nothing.
• min_acceptance_rate::Float64: Stop after any generation whose acceptance rate (accepted / proposed) falls below this value. 0.0 disables this check (default 0.0).
• min_epsilon_decrease::Float64: Stop after any generation where the proportional (relative) decrease in epsilon is less than this value. Formally, stop when (ε_{t-1} - ε_t) / ε_{t-1} < min_epsilon_decrease. A value of 0.1 requires at least a 10 % reduction each generation; 0.0 disables this check (default 0.0).
• min_ess_fraction::Float64: Stop after any generation where the effective sample size as a fraction of population_size falls below this value. ESS = 1 / Σwᵢ². A value of 0.2 stops when fewer than 20 % of particles carry meaningful weight. 0.0 disables this check (default 0.0).
• accept_overflow::Bool: When true, keep all particles that pass the epsilon threshold in a batch, even if their count exceeds population_size. population_size then acts as a minimum rather than an exact target. When false (default), any overflow from the final batch is discarded so each generation contains exactly population_size particles. The ESS stopping criterion always compares against population_size regardless of this flag; the log line reports ESS as a fraction of the actual accepted count.
• cdf_grid_k::Union{Nothing,Int}: Base grid resolution for CDF-grid snapping. When set, proposal CDF coordinates are snapped to the dyadic grid {j/2^k : j=1,...,2^k-1} at generation 1, with effective resolution doubling each generation (2^(k+t-1) points). The snap box radius at generation t is 1/2^(k+t). Bank entries within the snap box are reused without re-simulation; if none, the proposal snaps to the grid for a fresh simulation. Duplicate monad proposals within a generation are allowed — the same monad can appear as multiple particles (each receiving its own weight). The bank is updated between generations with all newly evaluated monads. Generation 1 uses a Sobol low-discrepancy sequence for good prior coverage. nothing (default) disables snapping. At runtime, the effective base resolution is raised to satisfy (2^k-1)^d ≥ population_size; an @info is emitted when this correction fires.
• max_evaluations::Union{Nothing,Int}: Maximum total number of evaluated particles (monads) across the entire calibration run. Each proposal sent to evaluate_batch counts as one evaluation, regardless of whether the monad was a fresh simulation or a bank reuse. The budget is enforced before each batch is dispatched: a planned batch that would exceed the budget is trimmed to exactly the remaining allowance, so the run never evaluates more than max_evaluations simulations. The completed portion of the current generation is then saved (possibly with fewer than population_size accepted particles) and the run stops. If the budget is smaller than population_size, even generation 1 is trimmed. nothing (default) disables the budget. Persisted to method.toml.
• store_rejected::Bool: When true, each GenerationResult (for generations t > 1) stores all rejected proposal CDF coordinates in rejected_proposals::DataFrame (same column names as particles; converted to target space at plot time). Useful for the :transition visualization recipe, which can also recover rejected proposals from disk via a lazy lookup when this is false (default). Not persisted to disk; always nothing on resume and for generation 1.

Examples

# Fully adaptive run
method = ABCSMC(population_size=200, max_nr_populations=15, minimum_epsilon=0.005)

# Manual epsilon schedule with acceptance-rate safety stop
method = ABCSMC(
    population_size = 100,
    epsilon_schedule = [50.0, 20.0, 5.0, 1.0],
    min_acceptance_rate = 0.01,
)

result = runCalibration(method, problem)

GaussianKernel(scale = 2.0)

Full multivariate Gaussian perturbation kernel using scale × weighted_covariance (Beaumont et al. 2009).

scale may be a scalar Float64 (constant across generations) or a Vector{Float64} generation schedule: generation t uses scale[min(t, end)].

Default scale=2.0 preserves the Beaumont et al. rule of thumb.

ComponentwiseKernel(scale = 2.0)

Diagonal-covariance perturbation kernel: independent 1D Gaussians per parameter, using scale × weighted_variance for each dimension. More robust than GaussianKernel in high dimensions where off-diagonal covariance estimation is noisy with small populations.

scale semantics are identical to GaussianKernel.

LocalNNKernel(; k = 10, scale = 1.0)

Per-particle bandwidth kernel: h_j = scale × dist(θ_j, θ_j^{(k)}) where θ_j^{(k)} is the k-th nearest neighbor of particle j (Chebyshev metric). All particles share the global weighted covariance shape Σ_global; only the scalar bandwidth h_j varies per particle. Proposal for parent j: N(θ_j, h_j² × Σ_global).

Bandwidth shrinks automatically as the particle cloud concentrates, so an explicit generation schedule is not needed. Requires only one Cholesky factorization per generation.

For fully local covariance (adapts direction as well as scale), see LocalNNCovKernel.

LocalNNCovKernel(; k = 10, scale = 1.0)

Per-particle local covariance kernel: for each previous-generation particle j, its perturbation kernel is N(θ_j, scale × Σ_local,j) where Σ_local,j is the sample covariance of particle j's k nearest neighbors. Unlike LocalNNKernel, the covariance direction adapts locally — useful for banana-shaped or anisotropic posteriors.

Cost: N Cholesky factorizations per generation (one per particle). For N ≤ 5000, d ≤ 10, this is fast in practice.

Calibration

Represents a calibration run tracked in the database.

Created automatically by runABC. The associated output folder at data/outputs/calibrations/{id}/ contains:

• generations/{t}/monads.csv: monad IDs evaluated per generation (written before each batch for crash safety).
• generations/{t}/failed_simulations.csv and generations/{t}/failed_monads.csv: IDs of simulations that failed in that generation and of the monads they belong to. Written only when something failed.
• generations/{t}/particles.csv: human-readable per-generation results — target parameter values (and latent parameter samples for user-supplied LatentVariations), weights, distances, and monad IDs.
• generations/{t}/metadata.toml: generation-level metadata (epsilon, acceptancerate, ess, nevaluations).
• generations/{t}/cdfs.csv: raw CDF coordinates for each accepted particle; used by resumeABC to reconstruct the internal particle state exactly.
• method.toml: serialized ABC-SMC settings (used by resumeABC).
• problem.jld2: full serialized CalibrationProblem; enables resumeABC(Calibration(id)) with no further arguments.
• parameters.toml: human-readable mapping from display column names (used in generations/ CSVs) to database column names, with prior strings.

Fields

• id::Int: Unique ID, matched to the calibrations table in the database.

GenerationResult

Result of a single ABC-SMC generation.

Fields

• t::Int: Generation index (1-based).
• particles::DataFrame: One row per accepted particle; columns are latent CDF coordinates (internal representation used by the ABC-SMC algorithm).
• weights::Vector{Float64}: Normalized importance weights (sum to 1).
• distances::Vector{Float64}: Distance for each accepted particle.
• max_epsilon_accepted::Float64: The largest distance this generation accepted. Never exceeds epsilon_threshold, and is what the stopping criteria compare against.
• n_evaluations::Int: Total proposals evaluated, including rejected ones.
• monad_ids::Vector{Int}: Monad IDs for each accepted particle.
• acceptance_rate::Float64: Fraction of proposals that passed the epsilon threshold (n_accepted_total / n_evaluations). When accept_overflow=false, this equals length(distances) / n_evaluations; when accept_overflow=true, it may be slightly higher because overflow particles are counted but n_evaluations includes the full batch.
• ess::Float64: Effective sample size, 1 / Σwᵢ². Equals population_size when weights are uniform (generation 1) and decreases as weights concentrate.
• epsilon_threshold::Union{Nothing,Float64}: The cutoff this generation was run against — epsilon_schedule[t-1] if one was supplied, otherwise max(minimum_epsilon, quantile(previous_distances, epsilon_quantile)). nothing for generation 1, which accepts every proposal it evaluates, and for generations recorded before this was stored. Distinct from max_epsilon_accepted: at the default epsilon_quantile of 0.5 the threshold is a median of the previous generation's distances while max_epsilon_accepted is a maximum of this one's, so the two coincide only when epsilon_quantile == 1.0.
• proposal_distances::Union{Nothing,DataFrame}: Reserved for the per-generation distance of every evaluated proposal, accepted or not. Currently always nothing — nothing populates it yet.
• rejected_proposals::Union{Nothing,DataFrame}: CDF-coordinate DataFrame of all rejected proposals in this generation (same column names as particles). Populated only when ABCSMC(store_rejected=true); always nothing for generation 1 (all Sobol proposals are accepted) and always nothing on resume. Used by the :transition visualization recipe; see also the lazy disk fallback in _lazyLoadRejected.

ABCResult

Holds the result of an ABC-SMC calibration run.

Fields

• calibration::Calibration: The calibration record (DB entry + folder).
• generations::Vector{GenerationResult}: Results per SMC generation, in order. Each GenerationResult.particles stores raw latent CDF coordinates.
• parameters::Vector{CalibrationParameter}: The calibrated parameters (same as stored in the CalibrationProblem), used to convert CDF coordinates to interpretable target values in posterior.
• method::ABCSMC: The settings used for this run.

The run-level accessors take an ABCResult directly, so reaching for .calibration is optional: Sampling, monadIDs, simulationIDs, tag!, untag!, tags, hasTag, tagsTable, calibrationsTable and deleteCalibration all forward to it.

Examples

result = runABC(problem)
df, weights = posterior(result)                # final generation, target-value format
df, weights = posterior(result; generation=2)  # specific generation

tag!(result, "project" => "immune-escape")     # same as tag!(result.calibration, ...)
simulationsTable(Sampling(result))

ConvergenceSummary(result::ABCResult)
ConvergenceSummary(cal::Calibration)

Per-generation convergence table for an ABC-SMC run. Supports plot(ConvergenceSummary(result)) via the RecipesBase recipe in visualize.jl, and behaves like a DataFrame for property access (cs.max_epsilon_accepted, etc.).

Columns

• t: Generation index.
• max_epsilon_accepted: The largest distance the generation accepted.
• epsilon_threshold: The cutoff it was run against; nothing for generation 1 and for generations recorded before this was stored.
• acceptance_rate: Fraction of proposals accepted.
• n_accepted: Number of accepted particles (equals population_size when accept_overflow=false; may be larger when accept_overflow=true).
• ess: Effective sample size (1 / Σwᵢ²).
• ess_fraction: ess / n_accepted — values near 1 mean uniform weights.
• n_evaluations: Total proposals evaluated (including rejected).

Examples

cs = ConvergenceSummary(result)
cs = ConvergenceSummary(Calibration(42))
plot(cs)

runCalibration(method::ABCSMC, problem::CalibrationProblem; description="") → ABCResult

Run ABC-SMC calibration. See ABCSMC for method settings.

The full CalibrationProblem is serialized to problem.jld2 enabling resumeABC(Calibration(id)) with no further arguments. Per-generation results are saved in two forms:

• generations/{t}/particles.csv: human-readable target parameter values.
• generations/{t}/cdfs.csv: raw CDF coordinates for exact resume.

Arguments

• run_kwargs::NamedTuple=(;): forwarded to each run(sampling; quiet=true, ...) call.
• description::String="": stored in the calibrations DB row.
• progress::Symbol=:auto: console-feedback verbosity. One of :auto, :none, :generation, :batch, :bar. :auto resolves to :bar on an interactive terminal and :generation otherwise.
• on_monad_failure::Symbol=:reject: what to do when a proposed monad has no successful simulation, so no distance can be computed for it. :reject records the distance as missing, which ABC-SMC never accepts, and continues; :error stops the run. Either way the failed simulation and monad IDs are recorded per generation in generations/{t}/failed_simulations.csv and generations/{t}/failed_monads.csv.

Examples

method = ABCSMC(population_size=200, max_nr_populations=5)
result = runCalibration(method, problem)
df, weights = posterior(result)

runABC(problem::CalibrationProblem; method=nothing, kwargs...) → ABCResult

Run ABC-SMC calibration on problem, a convenience wrapper over runCalibration.

Method settings may be given either as a ready-made ABCSMC via method=, or as loose keyword arguments naming its fields — but not both. The loose form forwards to the ABCSMC constructor, so every field it accepts is accepted here, with the same defaults.

Arguments

• problem::CalibrationProblem: the model, parameters, observed data and distance to calibrate.

Keywords

• method::Union{Nothing,ABCSMC}=nothing: the method to run. When nothing, one is built from the remaining keyword arguments.
• any field of ABCSMC — population_size, max_nr_populations, minimum_epsilon, epsilon_quantile, perturbation_kernel, epsilon_schedule, min_acceptance_rate, min_epsilon_decrease, min_ess_fraction, accept_overflow, cdf_grid_k, max_evaluations, store_rejected — forwarded to its constructor. See ABCSMC for the meaning and default of each. An unrecognized keyword raises an ArgumentError naming it.
• description::String="": free-text prose stored in the calibrations DB row and shown by calibrationsTable. For labels you intend to search on, prefer tags.
• tags=(): key => value pairs applied to the calibration before any simulation is dispatched, so they survive an interrupted run. Queryable with findTrials(Calibration; tags=...).
• run_kwargs::NamedTuple=(;): forwarded to each run(sampling; ...) call.
• progress::Symbol=:auto: console-feedback verbosity (:auto, :none, :generation, :batch, :bar). :auto shows a live progress bar on an interactive terminal and per-generation milestones otherwise.
• on_monad_failure::Symbol=:reject: what to do when a proposed monad has no successful simulation. :reject records its distance as missing (so the particle is never accepted) and continues; :error stops the run. Failed simulation and monad IDs are recorded per generation either way, in generations/{t}/failed_simulations.csv and generations/{t}/failed_monads.csv.

Returns

An ABCResult holding the calibration, its generations, the parameters and the method.

Examples

# Loose keywords
result = runABC(problem; population_size=200, max_nr_populations=5)
df, weights = posterior(result)
println("Posterior mean: ", sum(df[!, "overall/max_time"] .* weights))

# Or a ready-made method, plus labels to find the run again
result = runABC(problem; method=ABCSMC(population_size=200, store_rejected=true),
                tags=("project" => "immune-escape",))

# Resume after a crash
result2 = resumeCalibration(result.calibration)

resumeABC(calibration::Calibration; method=nothing, kwargs...) → ABCResult

ABC-specific alias for resumeCalibration; every argument has the same meaning.

resumeCalibration(calibration::Calibration, method=nothing; kwargs...)
resumeABC(calibration::Calibration; method=nothing, kwargs...)
          problem::Union{Nothing,CalibrationProblem}=nothing,
          method::Union{Nothing,ABCSMC}=nothing,
          run_kwargs::NamedTuple=(;)) → ABCResult

Resume a stopped or crashed ABC-SMC calibration from saved generation files.

Loads ABCSMC settings from method.toml and the calibration problem from problem.jld2 (both written by the original run), then continues from the next generation.

When problem= is required

If the original CalibrationProblem contained any anonymous functions (summary_statistic, distance, or any LatentVariation maps / inverse_maps defined as lambdas), JLD2 cannot serialize them and saves only a partial manifest. In that case resumeABC requires the full problem to be re-supplied:

resumeABC(Calibration(42); problem=my_problem)

The re-supplied problem is validated against the saved manifest:

• Structural check: parameter count, source types, names, targets, distributions.
• Behavioral check (all parameters): re-supplied maps are applied to the stored CDF particle data and the outputs are compared against the recorded target values.
• Round-trip check (LV parameters with inverse_maps): verifies inverse_map(map(lp)) ≈ lp on stored particles.

Even when problem.jld2 contains a full CalibrationProblem (all named functions), the behavioral and round-trip checks are re-run on any LVSource parameters, because JLD2 saves only the function's name — if the function was redefined between runs the new definition is used silently. Passing problem= in this case forces full validation.

Arguments

• problem: The original CalibrationProblem. Required when anonymous functions were present at save time; optional otherwise (loads from problem.jld2).
• method: Replace the saved settings wholesale. If nothing, they are loaded from method.toml. To change some settings and keep the rest, pass them as keywords instead — resumeCalibration(cal; max_nr_populations=15) — since a method object supplies every field, so any field it does not name takes a constructor default rather than the value the run used. Passing both a method object and individual settings is an error.
• Any ABCSMC field may be given as a keyword; it patches the saved value for that one field. Whenever the effective settings differ from method.toml, the file is rewritten to match and the changed keys are reported, so a later resume does not revert to the original run's values.

What a changed setting does to a resumed run

Nothing already on disk is recomputed — a resume only ever appends generations — so a changed setting takes effect from the next generation onward. What that means in practice differs by field:

No setting is refused. A generation is the unit of change, and none of these can take effect part-way through one — so the honest description is "from the next generation", not "immediately".

• run_kwargs: Forwarded to each run(sampling; ...) call.
• progress: Console-feedback verbosity (:auto, :none, :generation, :batch, :bar); same semantics as in runABC.
• on_monad_failure: :reject (default) or :error; same semantics as in runABC.

Examples

# Session restart — only calibration ID needed (all named functions)
result = resumeABC(Calibration(42))

# Re-supply the problem when anonymous functions were used
result = resumeABC(Calibration(42); problem=my_problem)

# Raise the generation cap, keeping every other saved setting
result = resumeCalibration(Calibration(42); max_nr_populations=15)

# Replace the settings wholesale (every unnamed field takes its default)
result = resumeCalibration(Calibration(42), ABCSMC(population_size=64, max_nr_populations=15))

run(method::ABCSMC, problem::CalibrationProblem; kwargs...) → ABCResult
run(calibration::Calibration[, method]; kwargs...) → ABCResult

Run or continue a calibration through run, alongside run(::AbstractTrial) and run(::GSAMethod, ...).

The first form starts a fresh run and is runCalibration(method, problem). The second continues an existing one and is resumeCalibration(calibration, method); as there, an ABCSMC field passed as a keyword patches the saved method rather than replacing it.

posterior(result::ABCResult; generation::Union{Int,Symbol}=:final)

Extract posterior samples from an ABCResult.

Converts the internal CDF-coordinate particles to target-parameter space using the stored CalibrationParameter objects. For DVSource / CVSource parameters the columns are the actual calibrated parameter values; for LVSource parameters the latent parameter samples are prepended.

Returns

• df::DataFrame: One row per particle; columns are target parameter names. For LVSource parameters (user-supplied LatentVariation), the latent parameter samples are included as well.
• weights::Vector{Float64}: Importance weights (sum to 1).

Arguments

• generation: Integer generation index (1-based) or :final for the last generation.

Examples

result = runABC(problem)
df, weights = posterior(result)                # final generation
df, weights = posterior(result; generation=1)  # first generation
println("Posterior mean: ", sum(df[!, "overall/max_time"] .* weights))

posterior(calibration::Calibration; generation::Union{Int,Symbol}=:final)

Extract posterior samples directly from disk for a completed calibration run.

Reads the human-readable generations/{t}/particles.csv file for the requested generation. Returns only the parameter columns (strips weight, distance, monad_id).

Useful when you have only the calibration ID (e.g. after a session restart) and don't have an in-memory ABCResult.

Returns

• df::DataFrame: One row per particle; columns are target parameter names.
• weights::Vector{Float64}: Importance weights (sum to 1).

Examples

# Retrieve results after a session restart
df, weights = posterior(Calibration(42))
df, weights = posterior(Calibration(42); generation=3)

calibrationsTable(; tags=false, include_auto_tags=false)
calibrationsTable(target...; kwargs...)

One row per calibration run: CalibrationID, DateTime, Method, Description.

target may be a vector of calibration IDs, one or more Calibrations, a vector of them, or the ABCResult runABC returns. Omitted, every run in the project.

Per-generation convergence numbers are ConvergenceSummary; the parameters themselves posterior.

Keyword Arguments

• tags::Bool: append one tag:<key> column per tag key in use (see appendTags!). Only tags on the run itself appear — a calibration contains no simulations to inherit from.
• include_auto_tags::Bool: also pivot the mm: provenance keys. Requires tags=true.

Examples

calibrationsTable()
calibrationsTable(; tags = true)      # what each run was for
calibrationsTable([1, 2, 3])

printCalibrationsTable(args...; sink=println, kwargs...)

Print calibrationsTable. sink receives the DataFrame (default println, or e.g. CSV.write("calibrations.csv")).

simulationIDs(calibration::Calibration[, generation::Integer])
simulationIDs(result::ABCResult[, generation::Integer])

IDs of every simulation a calibration run produced, or one generation's.

Equal to simulationIDs(Sampling(calibration)), but records nothing.

Examples

simulationsTable(simulationIDs(result))

monadIDs(calibration::Calibration[, generation::Integer])
monadIDs(result::ABCResult[, generation::Integer])

Sorted IDs of the monads a calibration run evaluated, or those of one generation.

Monads deleted after losing every simulation are excluded, so this is what the run actually produced. It only reads; Sampling is what makes a view addressable.

Examples

result = runABC(problem)
monadIDs(result)        # every surviving monad
monadIDs(result, 3)     # just generation 3

deleteCalibration(target; delete_subs=false)

Delete calibration runs: the calibrations database row, its tags, and its output folder.

target may be a calibration ID, a vector of IDs, a Calibration, a vector of them, or the ABCResult runABC returns.

delete_subs defaults to false, unlike the trial-level deleters — a run's monads are shared, through the SimulationBank and use_previous, so they may predate it and outlive it. Pass true to remove them and their simulations too.

Note that the folder holds the generation CSVs, the serialized problem and the method settings, so deleting a run discards its posterior: posterior and resumeABC stop working for it.

Examples

deleteCalibration(result)                  # bookkeeping only
deleteCalibration(3; delete_subs = true)   # and every monad it evaluated

SimulationBank

Pre-built registry of existing monads whose calibrated parameters lie strictly within the prior support (0,1)^d in CDF space.

Built once at calibration start by _buildSimulationBank and reused across all generations for approximate simulation reuse (CDF-grid snapping).

Fields

• monad_ids::Vector{Int}: Monad IDs of eligible entries.
• cdf_coords::Matrix{Float64}: n_latent_dims × n_monads matrix of CDF-space coordinates for each eligible monad (columns correspond to monad_ids).
• param_names::Vector{String}: Latent parameter names matching rows of cdf_coords.
• tree::Union{Nothing,NNTree}: KD-tree (Chebyshev metric) built from cdf_coords for O(log n) L∞ box queries. nothing when the bank is empty.

endpointPopulationCounts(monad_id::Int; cell_types=nothing, include_dead::Bool=false)

Built-in summary statistic: mean final-snapshot cell counts across all replicates in a monad.

Returns a Dict{String,Float64} mapping cell type name → mean count.

This is a monad-level function: it takes a monad ID and does its own averaging. Since ModelManager 0.9 a summary_statistic measures a single Simulation and ModelManager reduces the replicates, so this is no longer a valid summary_statistic argument — passing it fails when the first monad is measured. Use endpointPopulationCountQoI, which measures the same quantity in that shape. Keep this one for analysing a monad directly.

Arguments

• monad_id: ID of the monad whose replicates to average.
• cell_types: Optional Vector{String} to restrict which cell types are included. If nothing, all cell types present in the simulation are included.
• include_dead: Whether to include dead cells in the count (default false).

Examples

counts = endpointPopulationCounts(monad_id; cell_types=["tumor", "immune"])

# For calibration, use the QoI form instead — it measures one simulation, as ModelManager 0.9 requires
problem = CalibrationProblem(inputs, parameters, observed,
                             endpointPopulationCountQoI(; cell_types=["tumor", "immune"]),
                             mseDistance)

endpointPopulationFractions(monad_id::Int; cell_types=nothing, include_dead::Bool=false)

Built-in summary statistic: mean final-snapshot cell fractions (out of total live cells) across all replicates in a monad.

Returns a Dict{String,Float64} mapping cell type name → mean fraction.

This is a monad-level function and, since ModelManager 0.9, not a valid summary_statistic argument — see endpointPopulationCounts for why. Use endpointPopulationFractionQoI for calibration and keep this one for analysing a monad directly.

Arguments

• monad_id: ID of the monad whose replicates to average.
• cell_types: Optional Vector{String} to restrict which cell types are included. If nothing, all cell types present in the simulation are included.
• include_dead: Whether to include dead cells in the denominator (default false).

meanPopulationTimeSeries(monad_id::Int; cell_types=nothing, include_dead::Bool=false)

Built-in summary statistic: mean population time series across all replicates in a monad.

Returns a Dict{String,Vector{Float64}} mapping cell type name → mean count over time. The time axis is shared across replicates (an error is thrown if they differ). This is a monad-level function and, since ModelManager 0.9, not a valid summary_statistic argument — see endpointPopulationCounts for why. Use meanPopulationTimeSeriesQoI when calibrating against time-series data, and keep this one for analysing a monad directly. The corresponding observed_data values should be Vector{Float64} on the same time grid.

Arguments

• monad_id: ID of the monad whose replicates to average.
• cell_types: Optional Vector{String} to restrict which cell types are included. If nothing, all cell types present in the simulation are included.
• include_dead: Whether to include dead cells in the count (default false).

Examples

series = meanPopulationTimeSeries(monad_id; cell_types=["tumor"])

# For calibration, use the QoI form instead — it measures one simulation, as ModelManager 0.9 requires
problem = CalibrationProblem(inputs, parameters, observed,
                             meanPopulationTimeSeriesQoI(; cell_types=["tumor"]),
                             mseDistance)

endpointPopulationCountQoI(; cell_types=nothing, include_dead::Bool=false)

Return a QoI giving mean final-snapshot counts per cell type across a monad's replicates.

Its value is a Dict{String,Float64} of cell type → mean count: the same thing endpointPopulationCounts returns, so observed_data does not change between them.

Keyword Arguments

• cell_types: restrict to these cell types. nothing (default) measures every cell type present.
• include_dead: whether to include dead cells in the count (default false).

Examples

problem = CalibrationProblem(inputs, parameters, observed, endpointPopulationCountQoI(), mseDistance)

endpointPopulationFractionQoI(; cell_types=nothing, include_dead::Bool=false)

Return a QoI giving mean final-snapshot fractions of total cells per cell type across a monad's replicates.

Its value is a Dict{String,Float64} of cell type → mean fraction, matching endpointPopulationFractions. The ratio is taken per simulation and only then averaged — mean-of-ratios, not ratio-of-means — which is why it fits a per-simulation compute at all.

Keyword Arguments

• cell_types: restrict to these cell types. nothing (default) measures every cell type present.
• include_dead: whether to include dead cells in the denominator (default false).

Examples

problem = CalibrationProblem(inputs, parameters, observed, endpointPopulationFractionQoI(), mseDistance)

meanPopulationTimeSeriesQoI(; cell_types=nothing, include_dead::Bool=false)

Return a QoI giving the mean population time series per cell type across a monad's replicates.

Its value is a Dict{String,Vector{Float64}} on the shared time grid, matching meanPopulationTimeSeries, so a CalibrationProblem using it wants observed_data values on that same grid.

Unlike the two endpoint builders, a replicate lacking a cell type is excluded from that cell type's average rather than contributing a zero — MonadPopulationTimeSeries divides by the number of replicates having the cell type, and zero-filling would divide by a larger denominator.

Keyword Arguments

• cell_types: restrict to these cell types. nothing (default) measures every cell type present.
• include_dead: whether to include dead cells in the count (default false).

Examples

problem = CalibrationProblem(inputs, parameters, observed, meanPopulationTimeSeriesQoI(), mseDistance)

mseDistance(simulated, observed)

Built-in distance functions for use as distance in a CalibrationProblem. Three calling conventions are supported:

• mseDistance(sim::Dict{String,<:Any}, obs::Dict{String,<:Any}) — mean of per-key MSE contributions, averaged across all keys in obs. Scalar keys contribute (sim−obs)²; vector keys contribute mean((sim .- obs).^2). Keys missing from sim are treated as zero. Keys in sim not in obs are ignored (with a one-time warning).

• mseDistance(sim::AbstractVector{<:Real}, obs::AbstractVector{<:Real}) — sum of squared differences Σ(simᵢ−obsᵢ)². Throws DimensionMismatch when lengths differ.

• mseDistance(sim::Real, obs::Real) — squared difference (sim − obs)².