diff --git a/README.md b/README.md index 41380b5..8fa8720 100644 --- a/README.md +++ b/README.md @@ -5,42 +5,39 @@ This is a draft for a PEtab NLME extension. It adds the following to the existing PEtab specification. -## Individual table +## YAML configuration file -This table is an addition to the existing PEtab format. It introduces a classification of individuals (most fine grained grouping) into several (more coarse grained) groups. Random effects can be specific to each individual or to the groups they belong to. +In addition to existing attributes, this extension uses `covariance_files` to define distributions of random effects. Each distribution has a `distributionId`, `distributionType` (e.g. normal, laplace) and `covariance` (defined in a TSV file). Distributions have zero mean. The `randomEffectFormula` in the random effect table can be used to shift the location of the distribution. -### Detailed field description - -- `individualId` [STRING] -- `${groupId}` [STRING, OPTIONAL] - - Further columns may be used to group individuals into different categories. +## Random effect table -## Parameter table - -In addition to the existing descriptions of fixed effects, the NLME extension adds the following columns to describe random effects. +In addition to the existing descriptions of fixed effects in the parameter table, the NLME extension adds a random effect table to describe random effects. ### Detailed field description - `randomVariableId` [STRING] - ID of the random variable. If left empty, there is no random effect defined on the parameter of this row. + ID of the random variable. -- `randomEffectFormula` [STRING] +- `distributionId` [STRING] - Random effect function as plain text formula expression. Must contain the `parameterId` and the `randomVariableId` and must not contain any other symbols (or should we allow that???). + Specifies the Id of the distribution from which this random effect is drawn. Any Id from the distributions listed in the YAML configuration file is allowed. -- `distributionType` [STRING: 'normal' or 'laplace', OPTIONAL] +- `estimate` [0|1] - Assumed random effect distribution. Only normal and laplace distributions are currently allowed (but we could allow any term in some ontology for distributions). Defaults to normal. Distributions are assumed to be zero-centered. Covariances are defined in the covariance table. + 1 or 0, depending on, if the parameter is estimated (1) or set to zero (to shift a random effect away from zero, see `randomEffectFormula`). -- `groupType` [STRING] +- `parameterId` [STRING] - Defines how random effects are grouped together. Any column name from the individual table is allowed, except for covariate (i.e. numeric) columns. + ID of the associated fixed effect. + +- `randomEffectFormula` [STRING] + + Random effect function as plain text formula expression. Must contain the `parameterId` and the `randomVariableId`. May contain other `randomVariableId`s from other distributions. Must not contain any other symbols (or should we allow that???). ## Covariance tables -These tables are an addition to the existing PEtab format to describe the covariances of the random effects. Each `groupType` for which covariances are estimated has its own table. Four formats are allowed +These tables are an addition to the existing PEtab format to describe the covariances of the distribtions from which the random effects are drawn. Each `distributionId` has its own table. Four formats are allowed - Matrix format @@ -57,3 +54,7 @@ These tables are an addition to the existing PEtab format to describe the covari - No table If no table is provided, only the variances, but no covariances are estimated. + +## Measurement table + +In addition to existing collumns, the measurement table must contain every `distributionId` from the YAML configuration file as a column. Each data point (i.e. row) is assigned to one entity for each `distributionId`. This allows grouping of data points. As a simple example, the YAML configuration file defines just one `distributionId` that could be called `patient`. Rows that share the same entry in the `patient` column belong to the same patient. Therefore, they must share the same value for all random effects that are drawn from the `patient` distribution. diff --git a/covariance_tumorStage.tsv b/covariance_anotherDistribution.tsv similarity index 100% rename from covariance_tumorStage.tsv rename to covariance_anotherDistribution.tsv diff --git a/covariance_individual.tsv b/covariance_distributionA.tsv similarity index 100% rename from covariance_individual.tsv rename to covariance_distributionA.tsv diff --git a/covariance_sex.tsv b/covariance_yetAnotherDistribution.tsv similarity index 100% rename from covariance_sex.tsv rename to covariance_yetAnotherDistribution.tsv diff --git a/individuals.tsv b/individuals.tsv deleted file mode 100644 index f25acc8..0000000 --- a/individuals.tsv +++ /dev/null @@ -1,4 +0,0 @@ -individualId tumorStage sex age -p1 s1 f 60 -p2 s1 m 60 -p3 s2 m 70 diff --git a/measurements.tsv b/measurements.tsv index 29a5524..f2a34e2 100644 --- a/measurements.tsv +++ b/measurements.tsv @@ -1,13 +1,13 @@ -observableId simulationConditionId individualId measurement time -obs_B high_dose p1 9.4 1 -obs_B high_dose p1 1.5 10 -obs_B low_dose p1 1.8 1 -obs_B low_dose p1 1.06 10 -obs_B high_dose p2 9.6 1 -obs_B high_dose p2 1.52 10 -obs_B low_dose p2 2 1 -obs_B low_dose p2 1.08 10 -obs_B high_dose p3 9 1 -obs_B high_dose p3 1.4 10 -obs_B low_dose p3 1.6 1 -obs_B low_dose p3 1.04 10 +observableId simulationConditionId distributionA anotherDistribution yetAnotherDistribution measurement time +obs_B high_dose patient1 stage1 blue 9.4 1 +obs_B high_dose patient1 stage1 blue 1.5 10 +obs_B low_dose patient1 stage1 blue 1.8 1 +obs_B low_dose patient1 stage1 blue 1.06 10 +obs_B high_dose patient2 stage1 brown 9.6 1 +obs_B high_dose patient2 stage1 brown 1.52 10 +obs_B low_dose patient2 stage1 brown 2 1 +obs_B low_dose patient2 stage1 brown 1.08 10 +obs_B high_dose patient3 stage2 brown 9 1 +obs_B high_dose patient3 stage2 brown 1.4 10 +obs_B low_dose patient3 stage2 brown 1.6 1 +obs_B low_dose patient3 stage2 brown 1.04 10 diff --git a/parameters.tsv b/parameters.tsv index e11a9ab..f88e209 100644 --- a/parameters.tsv +++ b/parameters.tsv @@ -1,9 +1,9 @@ -parameterId parameterScale lowerBound upperBound nominalValue estimate objectivePriorType objectivePriorParameters randomVariableId randomEffectFormula distributionType groupType -k1 log10 0.1 10 1 1 laplace 0; 1 u1 k1*exp(u1) normal individualId -k2 log10 0.1 10 1 1 laplace 0; 1 u2 k2 + u2 normal individualId -k3 log10 0.1 10 1 1 laplace 0; 1 u3 k3 * u3 normal individualId -k4 log10 0.1 10 1 1 laplace 0; 1 u4 k4 + u4 laplace tumorStage -k5 log10 0.1 10 1 1 laplace 0; 1 u5 k5 + u5 laplace tumorStage -k6 log10 0.1 10 1 1 laplace 0; 1 u6 k6 + u6 laplace tumorStage -k7 log10 0.1 10 1 1 laplace 0; 1 u7 k7 + u7 normal sex -k8 log10 0.1 10 1 1 laplace 0; 1 +parameterId parameterScale lowerBound upperBound nominalValue estimate objectivePriorType objectivePriorParameters +k1 log10 0.1 10 1 1 laplace 0; 1 +k2 log10 0.1 10 1 1 laplace 0; 1 +k3 log10 0.1 10 1 1 laplace 0; 1 +k4 log10 0.1 10 1 1 laplace 0; 1 +k5 log10 0.1 10 1 1 laplace 0; 1 +k6 log10 0.1 10 1 1 laplace 0; 1 +k7 log10 0.1 10 1 1 laplace 0; 1 +k8 log10 0.1 10 1 1 laplace 0; 1 diff --git a/petab_nlme_example.yaml b/petab_nlme_example.yaml index 537ff3c..226c7ce 100644 --- a/petab_nlme_example.yaml +++ b/petab_nlme_example.yaml @@ -1,5 +1,6 @@ format_version: 2.0.0 parameter_file: parameters.tsv +random_effect_file: random_effects.tsv problems: - condition_files: - conditions.tsv @@ -11,15 +12,16 @@ problems: location: model.xml observable_files: - observables.tsv - individual_files: - - individuals.tsv covariance_files: - covariance: - group: individual - locaion: covariance_individual.tsv - covariance: - group: tumorStage - location: covariance_tumorStage.tsv - covariance: - group: sex - location: covariance_sex.tsv + distribution: + distributionId: distributionA + distributionType: normal + covariance: distributionA.tsv + distribution: + distributionId: anotherDistribution + distributionType: lognormal + covariance: covariance_anotherDistribution.tsv + distribution: + distributionId: yetAnotherDistribution + distributionType: laplace + covariance: covariance_yetAnotherDistribution.tsv diff --git a/random_effects.tsv b/random_effects.tsv new file mode 100644 index 0000000..377fb2e --- /dev/null +++ b/random_effects.tsv @@ -0,0 +1,8 @@ +randomVariableId distributionId estimate parameterId randomEffectFormula +u1 distributionA 1 k1 k1*exp(u1+u4) +u2 distributionA 1 k2 k2 + u2 +u3 distributionA 1 k3 k3 * u3 +u4 anotherDistribution 1 k1 k1*exp(u1+u4) +u5 anotherDistribution 1 k5 k5 + u5 +u6 anotherDistribution 1 k6 k6 + u6 +u7 yetAnotherDistribution 1 k7 k7 + u7