The classes below all implement the ProblemLogLikelihood interface,
and can calculate a log-likelihood based on some time-series Problem
and an assumed noise model.
Calculates a log-likelihood assuming AR(1) (autoregressive order 1) errors.
In this error model, the ith error term
\(\epsilon_i = x_i - f_i(\theta)\) is assumed to obey the following
relationship.
\[\epsilon_i = \rho \epsilon_{i-1} + \nu_i\]
where \(\nu_i\) is IID Gaussian white noise with variance
\(\sigma^2 (1-\rho^2)\). Therefore, this likelihood is appropriate when
error terms are autocorrelated, and the parameter \(\rho\)
determines the level of autocorrelation.
This model is parameterised as such because it leads to a simple marginal
distribution \(\epsilon_i \sim N(0, \sigma)\).
This class treats the error at the first time point (i=1) as fixed, which
simplifies the calculations. For sufficiently long time-series, this
conditioning on the first observation has at most a small effect on the
likelihood. Further details as well as the alternative unconditional
likelihood are available in [1] , chapter 5.2.
problem – A SingleOutputProblem or MultiOutputProblem. For a
single-output problem two parameters are added (rho, sigma),
for a multi-output problem 2 * n_outputs parameters are added.
This model is parameterised as such because it leads to a simple marginal
distribution \(\epsilon_i \sim N(0, \sigma)\).
Due to the complexity of the exact ARMA(1,1) likelihood, this class
calculates a likelihood conditioned on initial values. This topic is
discussed further in [2] , chapter 5.6. Thus, for a time series defined at
points \(i=1,...,N\), summation begins at \(i=3\), and the
conditional log-likelihood is
problem – A SingleOutputProblem or MultiOutputProblem. For a
single-output problem three parameters are added (rho, phi, sigma),
for a multi-output problem 3 * n_outputs parameters are added.
problem – A SingleOutputProblem or MultiOutputProblem. For a
single-output problem one parameter is added sigma, where
sigma is scale, for a multi-output problem n_outputs
parameters are added.
Calculates a log-likelihood assuming independent Gaussian noise at
each time point, and that data above and below certain limits are censored.
In other words, any data values less than the lower limit are only known to
be less than or equal to it; and any values greater than the upper limit
are only known to be greater than or equal to it. This likelihood is useful
for censored data - see for instance [3]. The parameter sigma represents
the standard deviation of the noise on each output.
For a noise level of sigma, and left and right censoring (below the
lower limit cutoff and above the upper limit), the likelihood
becomes:
where \(x_j\) is the sampled data at time \(j\) and \(f_j\) is
the simulated data at time \(j\). The functions \(\phi\) and
\(\Phi\) are the standard normal distribution probability density
function and cumulative distribution function respectively.
Calculates the log-likelihood assuming a mixed error model of a
Gaussian base-level noise and a Gaussian heteroscedastic noise.
For a time series model \(f(t| \theta)\) with parameters \(\theta\)
, the ConstantAndMultiplicativeGaussianLogLikelihood assumes that the
model predictions \(X\) are Gaussian distributed according to
where \(\epsilon\) is a i.i.d. standard Gaussian random variable
\[\epsilon \sim \mathcal{N}(0, 1).\]
For each output in the problem, this likelihood introduces three new scalar
parameters: a base-level scale \(\sigma _{\text{base}}\); an
exponential power \(\eta\); and a scale relative to the model output
\(\sigma _{\text{rel}}\).
The resulting log-likelihood of a constant and multiplicative Gaussian
error model is
where \(n_t\) is the number of measured time points in the time series,
\(X^{\text{obs}}_i\) is the observation at time point \(t_i\), and
\(\sigma _{\text{tot}, i}=\sigma _{\text{base}} +\sigma _{\text{rel}}
f(t_i| \theta)^\eta\) is the total standard deviation of the error at time
\(t_i\).
where \(n_o\) is the number of outputs of the model,
\(X^{\text{obs}}_{ij}\) is the observation at time point \(t_i\)
of output \(j\), and
\(\sigma _{\text{tot}, ij}=\sigma _{\text{base}, j} +
\sigma _{\text{rel}, j}f_j(t_i| \theta)^{\eta _j}\) is the total standard
deviation of the error at time \(t_i\) of output \(j\).
problem – A SingleOutputProblem or MultiOutputProblem. For a
single-output problem three parameters are added
(\(\sigma _{\text{base}}\), \(\eta\),
\(\sigma _{\text{rel}}\)),
for a multi-output problem \(3n_o\) parameters are added
(\(\sigma _{\text{base},1},\ldots , \sigma _{\text{base},n_o},
\eta _1,\ldots , \eta _{n_o}, \sigma _{\text{rel},1}, \ldots ,
\sigma _{\text{rel},n_o})\).
Calculates a log-likelihood assuming independent Gaussian-distributed noise
at each time point where \(p(\sigma)\propto 1/\sigma\) has been
integrated out of the joint posterior of \(p(\theta,\sigma|X)\),
Note that this is exactly the same statistical model as
pints.GaussianLogLikelihood with a uniform prior on
\(\sigma\) in the logarithmic transformed space.
This is also known as Jeffrey’s prior, as the standard deviation is a scale
parameter, as discussed here: https://doi.org/10.1093/gji/ggaa168.
A possible advantage of this log-likelihood compared with using a
pints.GaussianLogLikelihood, is that it has one fewer parameters
(\(sigma\)) which may speed up convergence to the posterior
distribution, especially for multi-output problems which will have
n_outputs fewer parameter dimensions.
The log-likelihood is given in terms of the sum of squared errors:
where \(\Gamma(a)\) is the incomplete gamma function.
This log-likelihood is inherently a Bayesian method since it assumes a
uniform prior on \(\sigma\) _in the logarithmic transformed space.
However using this likelihood in optimisation routines should yield the
same estimates as the full pints.GaussianLogLikelihood.
Calculates a log-likelihood assuming independent Gaussian-distributed noise
at each time point where \(\sigma\sim U(a,b)\) has been integrated out
of the joint posterior of \(p(\theta,\sigma|X)\) in the same way as in
pints.GaussianIntegratedLogUniformLogLikelihood,
Note that this is exactly the same statistical model as
pints.GaussianLogLikelihood with a uniform prior on
\(\sigma\).
Similar to pints.GaussianIntegratedLogUniformLogLikelihood, a
possible advantage of this log-likelihood compared with using a
pints.GaussianLogLikelihood, is that it has one fewer parameters
(\(sigma\)) which may speed up convergence to the posterior
distribution, especially for multi-output problems which will have
n_outputs fewer parameter dimensions.
The log-likelihood is given in terms of the sum of squared errors:
This log-likelihood is inherently a Bayesian method since it assumes a
uniform prior on \(\sigma\sim U(a,b)\). However using this likelihood
in optimisation routines should yield the same estimates as the full
pints.GaussianLogLikelihood.
Calculates a log-likelihood assuming independent Gaussian noise at each
time point, using a known value for the standard deviation (sigma) of that
noise:
Calculates a log-likelihood assuming independent Gaussian noise at each
time point, and adds a parameter representing the standard deviation
(sigma) of the noise on each output.
For a noise level of sigma, the likelihood becomes:
problem – A SingleOutputProblem or MultiOutputProblem. For a
single-output problem a single parameter is added, for a multi-output
problem n_outputs parameters are added.
Calculates a log-likelihood assuming independent log-normal noise at each
time point, and adds a parameter representing the standard deviation
(sigma) of the noise on the log scale for each output.
Specifically, the sampling distribution takes the form:
As such, the optional parameter mean_adjust (if true) adjusts the mean of
the distribution so that its expectation is \(f(\theta, t)\). This
shifted distribution is of the form:
problem – A SingleOutputProblem or MultiOutputProblem. For a
single-output problem a single parameter is added, for a multi-output
problem n_outputs parameters are added.
mean_adjust – A Boolean. Adjusts the distribution so that its mean corresponds to the
function value. By default, this parameter if False.
Calculates the log-likelihood for a time-series model assuming a
heteroscedastic Gaussian error of the model predictions
\(f(t, \theta )\).
This likelihood introduces two new scalar parameters for each dimension of
the model output: an exponential power \(\eta\) and a scale
\(\sigma\).
A heteroscedascic Gaussian noise model assumes that the observable
\(X\) is Gaussian distributed around the model predictions
\(f(t, \theta )\) with a standard deviation that scales with
\(f(t, \theta )\)
problem – A SingleOutputProblem or MultiOutputProblem. For a
single-output problem two parameters are added (\(\eta\),
\(\sigma\)), for a multi-output problem 2 times \(n_o\)
parameters are added.
Calculates a log-likelihood assuming independent Student-t-distributed
noise at each time point, and adds two parameters: one representing the
degrees of freedom (nu), the other representing the scale (sigma).
For a noise characterised by nu and sigma, the log likelihood is of
the form:
problem – A SingleOutputProblem or MultiOutputProblem. For a
single-output problem two parameters are added (nu,sigma), where
nu is the degrees of freedom and sigma is scale, for a
multi-output problem 2*n_outputs parameters are added.