Skip to content

v1/v2 equivalence of problems with priors on log-scaled parameters #333

Open

Description

@dweindl

Relates to #327, #328.

We require a problem's v1/ and v2/ encodings to be equivalent. Four problems have v1 priors on log-scaled parameters. For these, a plain v2 encoding cannot preserve both the optimization problem (objective, optimum) and the Bayesian model (prior, posterior, evidence). This issue proposes how to encode them and how to define "equivalent". While checking the priors against the original implementations, I also found some unrelated errors (see the end).

Notation. x is a parameter's value on linear scale, as used in the model and by PEtab bounds and nominal values (v1 and v2). θ is its value on the v1 parameterScale, e.g. θ = log10 x for log10; this is what v1 optimizers work with. Note that the v2 spec's objective section uses θ for the linear parameter values, i.e. our x.

The problem

In v1, parameterScaleNormal(μ, σ) on a log10-scaled parameter x means log10(x) ~ N(μ, σ). The original implementations (D2D) and pyPESTO add this term to the objective:

-log N(log10 x; μ, σ)        (a density over θ = log10 x)

See D2D arCollectRes.m L344–345 and pyPESTO get_parameter_prior_dict.

Parameters without a prior default to parameterScaleUniform, which the v1 spec calls "equivalent to no prior". It is flat in θ, i.e. log-uniform in x.

PEtab v2 has no parameterScale (spec). Priors are always densities over x, and the MAP objective is -log L(x) - log π(x) (spec). The same distribution written as a density over x is N(log10 x; μ, σ) / (x ln 10), i.e. log-normal(μ ln10, σ ln10). The two objectives differ by ln(x ln10). That term is not constant, so the optimum moves: a MAP is not invariant under reparametrization. Hence a v2 prior on x can match the v1 objective or the v1 distribution, but not both.

This is the pdf1/pdf2 question from PEtab-dev/PEtab#402, which is still open. Because v1 never settled it, tools disagree on the v1 objective:

  • D2D and pyPESTO evaluate the density over θ, without a Jacobian.
  • libpetab's v1 Prior evaluates the density over x, i.e. with the Jacobian (distributions.py).
  • petab1to2, as of petab1to2: rescale noise and priors when converting log10 to log libpetab-python#535 and #536, follows libpetab (code):
    • Explicit priors become log-normal(μ ln10, σ ln10), which keeps the distribution.
    • Unspecified priors on log-scaled parameters become uniform(lb, ub), which keeps the objective; #536 says this explicitly.
    • A converted problem that has both kinds of parameters is therefore equivalent to v1 under neither reading.

Effect of the current conversion on the MAP, ignoring the data (i.e. for parameters the data don't determine). In general the shift is −σ²·ln10 decades:

Parameter Prior (log10) v1 optimum v2 optimum after petab1to2
Bachmann init_EpoRJAK2 N(0.60, 0.19) 3.97 3.26
Schwen ka1 N(−4.0, 3) 9.8e-5 1e-5 (lower bound)
Raimundez xi_ka_* N(0, 3) 1 1e-5 (lower bound)

Affected:

  • Bachmann_MSB2011: 1 parameter.
  • Isensee_JCB2018: 20.
  • Raimundez_PCB2020: 17, or 16 after the fix below.
  • Schwen_PONE2014: 6.

All of these priors are parameterScaleNormal on log10. The same four problems also have about 260 log10-scaled estimated parameters without a prior (see "Parameters without a prior" below).

Not affected:

  • Problems without priors. The likelihood is invariant under reparametrization. #535 showed agreement within 4e-15 for all models with log10 observables. An AMICI 1.1 simulation of Boehm_JProteomeRes2014 agrees within 4e-9, which is solver tolerance.
  • The laplace/uniform priors in Lang_PLOSComputBiol2024 and Raimundez_PCB2020, which are on lin parameters.

The originals use the θ-density. I checked all four problems against the publications and the original D2D implementations. Each one evaluates ((log10 x − μ)/σ)² without a Jacobian, and the benchmark v1/ encodings match them. The published optima are therefore those of the θ-density reading.

Options

The options are compared with the v1 problem read as a model in θ, the D2D/pyPESTO convention. A separate numeric review confirmed them on a toy model.

v2 encoding Optimum and objective Bayesian model (posterior, evidence) Other cons
A log-normal(μ ln10, σ ln10) (current petab1to2) MAP moves by −σ²·ln10 decades: 0.02–0.09 for σ ≤ 0.2, onto the bound for σ = 3. The objective differs by the non-constant ln(x ln10), so reference optima and profiles would disagree with the publications Identical –
B log-normal(μ ln10 + (σ ln10)², σ ln10) Same; the objective differs by a constant (N(ln x; m, s) ∝ x · LN(x; m+s², s)) Different. On a toy model, posterior means of log10 x shift by up to 4.3 decades and the evidence by up to a factor of 10 The prior no longer states the intended belief: for σ = 3 it is centred 20.7 decades above μ. This is what PEtab#402 and v2 set out to remove
C Estimate log10_<id> with normal(μ, σ), compute <id> = 10^log10_<id>, as the v2 spec recommends Same, including the gradient/Hessian in θ. The objective differs by the truncation constant −log Z, at most 1.08 per prior and 0 for σ ≤ 0.2 Identical IDs, bounds and nominal values change for 43–44 parameters; the model (or observables) changes
D Prior as a pseudo-measurement (libpetab priors_to_measurements, libpetab-python#307) Same Undefined: v2 sees an ML problem The prior becomes data: blurs ML vs. MAP and changes the number of data points used by BIC. Not a v2 prior

Suggestion: C for the parameters with an explicit prior. It is the only option that keeps both the published optimum and the Bayesian model, and it is the route the v2 spec itself prescribes. Reject B, because it changes the Bayesian model and misstates the prior, and D, because it turns the prior into data.

Implementation notes for C:

  • Not every prior parameter is a model parameter. In Isensee_JCB2018, b_Calpha_global and b_pRII_global appear only in observable formulas, and the four s_pRII_* only as observableParameters overrides in the measurement table. These need 10^(...) in the observable formulas rather than an SBML rule.
  • A tool heuristic such as "log-transform if lb ≥ 0" would double-transform a log10_* parameter with non-negative log bounds. Currently this affects only xi_pPDE in the Isensee model-selection variant, with bounds [0, 3]. The optimizationScale hint below covers this.

Parameters without a prior

v1's default (flat in θ) is log-uniform in x. v2's default, uniform(lb, ub), keeps the MAP, since both are constant objective terms. As a prior, however, it is very different: for bounds such as [1e-5, 1e3] it puts 90% of the mass in the top decade. For the ~260 such parameters, the v2 Bayesian model and evidence therefore differ from v1's. Encoding them as log-uniform(lb, ub) restores the distribution but moves the MAP, which is the same dilemma as above. Only reparametrizing them as well preserves both: log10_<id> with uniform(log10 lb, log10 ub), called "C-all" here.

Suggestion:

  • Keep the v2 default in v2/, since the collection is mainly used for optimization. State in metadata.yaml that the prior distribution of these parameters differs from v1's.
  • If needed for Bayesian use, a fully faithful C-all variant can be added under resources/ later.

Model selection:

  • AIC, BIC and likelihood-ratio tests on the negative log-likelihood are unaffected by all options except D.
  • Comparisons of MAP objective values or evidence across models are affected by the constants (−Σ log Z, −Σ ln(ub − lb)), which differ between models. This applies, for example, to the Isensee variants in resources/.
  • The published Isensee model selection used D2D's merit function, which includes the prior terms. D2D's reported objective is not normalized either: it adds log(2π) per prior but no log σ term (arGetMerit.m L140). So no v1 reference evidence exists to compare against.

Proposed policy

To replace the equivalence wording in #328:

v1/ and v2/ encode the same estimation problem under the v1 reading used by the original implementations: priors are densities on parameterScale, without a Jacobian (the D2D/pyPESTO convention). Both must have the same optimum. As functions of the model parameters, their objective functions must agree up to a documented additive constant. Documented reparametrizations (e.g. estimating log10_k with k = 10^log10_k) are allowed. Where a v1 prior cannot be expressed as a v2 prior on the linear parameter without moving the optimum, reparametrize. Parameters without a v1 prior use the v2 default uniform(lb, ub). That default is objective-equivalent but not distribution-equivalent to v1's flat-in-scale default, and this must be stated.

Per problem, document in metadata.yaml:

  • the reparametrized parameters, with their old and new IDs, bounds and nominal values;
  • the objective constant;
  • the caveat about the default prior;
  • a non-normative recommended optimization and start-point scale (e.g. optimizationScale), because v2 drops parameterScale and initialization priors, while benchmark results depend on optimizing on log scale and sampling start points on log scale (parameterScaleUniform was originally introduced for the latter, see PEtab#402).

PEtab v2 itself

The v2 objective is well defined; the problem above only arises in v1→v2 conversion. The spec has two errors and one point worth clarifying:

  1. Infinite bounds. lowerBound/upperBound are NUMERIC, which excludes inf (L116, L903–L911). Yet the spec recommends bounds of 0 and inf to avoid truncating a log-normal prior (L936–L938). Action: allow ±inf bounds, or drop that sentence.
  2. gamma domain. It is listed as (−∞, ∞) (L999–L1003) and should be (0, ∞). Action: fix.
  3. Default uniform prior. If any parameter has a prior, every other parameter defaults to uniform(lb, ub) (L929–L931). This doesn't change optima. It does tie objective values and evidence to the bounds, and it equates "no prior" with "uniform prior" (cf. PEtab#402). Action: clarify.

Upstream actions beyond the spec:

  • libpetab: document petab1to2's mixed prior convention, or add an objective-preserving (reparametrizing) option.
  • libpetab: its v1 Prior gives the default parameterScaleUniform a non-constant ln(x ln10) term, contradicting the v1 spec's "equivalent to no prior".
  • PEtab#402: close with a documented decision on the v1 reading.

Other findings from the prior audit (separate issues/PRs)

  • Raimundez_PCB2020
    • The d_* priors should be laplace 0;6, not 0;3. D2D's L1 term |p−μ|/σ is added to χ² = −2 log posterior (arCollectRes.m L388–L393). D2D's PEtab exporter has the same factor-2 error (arExportPEtab.m L363ff).
    • The ki_AKT__MKN1 prior was never active originally: ki_AKT is not a parameter of the published model (code: 10.5281/zenodo.2908234, ParameterEstimation/parallelizeMultistarts.m, Models/model_NoCrossTalk_ALL.def).
    • The paper describes ML estimation (eq. 1), but the code uses priors.
  • Lang_PLOSComputBiol2024
    • sCDKN1B has a nominal value of 2e-10, above its upper bound of 1.99e-10. The cause is a bound-shifting bug in the authors' sacess_to_petab.py L62–L71, which shifts around the old nominal value instead of the optimum.
    • Seven log-scaled parameters have nominal value 0, below their lower bound of 1e-24: kDpApc_1, kDpE2f1 and kPhC25A (estimated), and kPhNup, kDpNup_1, kIm_2 and kEx_2 (fixed).
  • Bachmann_MSB2011
    • σ = 0.1935 comes from D2D's arLogNtoN, which is off by a factor of √ln10 for the intended CV of 30% (it should be 0.1275).
    • The published fit used 0.1935: the SI prior is N(0.6, 0.2²), and Table S16 gives 0.5995. Keep the value and document it.
  • Isensee_JCB2018, Schwen_PONE2014: the encodings match the D2D originals. For Isensee, the priors come from Results/BestFit/workspace_pars_only.mat. For Schwen, see Setup.m L35–L38.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions