Log-PDFs

LogPDFs are callable objects that represent distributions, including likelihoods and Bayesian priors and posteriors. They are unnormalised, i.e. their area does not necessarily sum up to 1, and for efficiency reasons we always work with the logarithm e.g. a log-likelihood instead of a likelihood.

Example:

p = pints.GaussianLogPrior(mean=0, variance=1)
x = p(0.1)

Overview:

• LogPDF
• LogPrior
• LogPosterior
• PooledLogPDF
• ProblemLogLikelihood
• SumOfIndependentLogPDFs

class pints.LogPDF

Represents the natural logarithm of a (not necessarily normalised) probability density function (PDF).

All LogPDF types are callable: when called with a vector argument p they return some value log(f(p)) where f(p) is an unnormalised PDF. The size of the argument p is given by n_parameters().

evaluateS1(x)

Evaluates this LogPDF, and returns the result plus the partial derivatives of the result with respect to the parameters.

The returned data is a tuple (L, L') where L is a scalar value and L' is a sequence of length n_parameters.

Note that the derivative returned is of the log-pdf, so L' = d/dp log(f(p)), evaluated at p=x.

This is an optional method that is not always implemented.

n_parameters()

Returns the dimension of the space this LogPDF is defined over.

class pints.LogPrior

Represents the natural logarithm log(f(theta)) of a known probability density function f(theta).

Priors are usually normalised (i.e. the integral f(theta) over all points theta in parameter space sums to 1), but this is not a strict requirement.

Extends LogPDF.

cdf(x)

Returns the cumulative density function at point(s) x.

x should be an n x d array, where n is the number of input samples and d is the dimension of the parameter space.

convert_from_unit_cube(u)

Converts samples u uniformly drawn from the unit cube into those drawn from the prior space, typically by transforming using LogPrior.icdf().

u should be an n x d array, where n is the number of input samples and d is the dimension of the parameter space.

convert_to_unit_cube(x)

Converts samples from the prior x to be drawn uniformly from the unit cube, typically by transforming using LogPrior.cdf().

x should be an n x d array, where n is the number of input samples and d is the dimension of the parameter space.

evaluateS1(x)

Evaluates this LogPDF, and returns the result plus the partial derivatives of the result with respect to the parameters.

The returned data is a tuple (L, L') where L is a scalar value and L' is a sequence of length n_parameters.

Note that the derivative returned is of the log-pdf, so L' = d/dp log(f(p)), evaluated at p=x.

This is an optional method that is not always implemented.

icdf(p)

Returns the inverse cumulative density function at cumulative probability/probabilities p.

p should be an n x d array, where n is the number of input samples and d is the dimension of the parameter space.

mean()

Returns the analytical value of the expectation of a random variable distributed according to this LogPDF.

n_parameters()

Returns the dimension of the space this LogPDF is defined over.

sample(n=1)

Returns n random samples from the underlying prior distribution.

The returned value is a NumPy array with shape (n, d) where n is the requested number of samples, and d is the dimension of the prior.

class pints.LogPosterior(log_likelihood, log_prior)

Represents the sum of a LogPDF and a LogPrior defined on the same parameter space.

As an optimisation, if the LogPrior evaluates as -inf for a particular point in parameter space, the corresponding LogPDF will not be evaluated.

Extends LogPDF.

Parameters:

• log_likelihood – A LogPDF, defined on the same parameter space.
• log_prior – A LogPrior, representing prior knowledge of the parameter space.

evaluateS1(x)

Evaluates this LogPDF, and returns the result plus the partial derivatives of the result with respect to the parameters.

The returned data has the shape (L, L') where L is a scalar value and L' is a sequence of length n_parameters.

This method only works if the underlying :class:`LogPDF` and :class:`LogPrior` implement the optional method :meth:`LogPDF.evaluateS1()`!

log_likelihood()

Returns the LogLikelihood used by this posterior.

log_prior()

Returns the LogPrior used by this posterior.

n_parameters()

See LogPDF.n_parameters().

class pints.PooledLogPDF(log_pdfs, pooled)

Combines \(m\) LogPDFs, each with \(n\) parameters, into a single LogPDF where \(k\) parameters are “pooled” (i.e. have the same value for each LogPDF), so that the resulting combined LogPDF has \(m (n - k) + k\) independent parameters.

This is useful for e.g. modelling the time-series of multiple individuals (each individual defines a separate LogPDF), and some parameters are expected to be the same across individuals (for example, the noise parameter across different individuals within the same experiment).

For two LogPDFs \(L _1\) and \(L _2\) with four parameters \((\psi ^{(1)}_1, \psi ^{(1)}_2, \psi ^{(1)}_3, \psi ^{(1)}_4)\) and \((\psi ^{(2)}_1, \psi ^{(2)}_2, \psi ^{(2)}_3, \psi ^{(2)}_4)\) respectively, a pooling of the second and third parameter \(\psi _2 := \psi ^{(1)}_2 = \psi ^{(2)}_2\), \(\psi _3 := \psi ^{(1)}_3 = \psi ^{(2)}_3\) results in a pooled log-pdf of the form

\[L(\psi ^{(1)}_1, \psi ^{(1)}_4, \psi ^{(2)}_1, \psi ^{(2)}_4, \psi _2, \psi _3 | D_1, D_2) = L _1(\psi ^{(1)}_1, \psi _2, \psi _3, \psi ^{(1)}_4 | D_1) + L _2(\psi ^{(2)}_1, \psi _2, \psi _3, \psi ^{(2)}_4 | D_2),\]

\(D_i\) is the measured time-series of individual \(i\). As \(k=2\) parameters where pooled across the log-likelihoods, the pooled log-likelihood has six parameters in the following order: \((\psi ^{(1)}_1, \psi ^{(1)}_4, \psi ^{(2)}_1, \psi ^{(2)}_4, \psi _2, \psi _3)\).

Note that the input parameters of a PooledLogPDF are not just a simple concatenation of the parameters of the individual LogPDFs. The pooled parameters are only listed once and are moved to the end of the parameter list. This avoids inputting the value of the pooled parameters at mutliple positions. Otherwise the order of the parameters is determined firstly by the order of the likelihoods and then by the order of the parameters of those likelihoods.

Extends LogPDF.

Parameters:

• log_pdfs – A sequence of LogPDF objects.
• pooled – A sequence of booleans indicating which parameters across the likelihoods are pooled (True) or remain unpooled (False).

Example

pooled_log_likelihood = pints.PooledLogPDF(
    log_pdfs=[
        pints.GaussianLogLikelihood(problem1),
        pints.GaussianLogLikelihood(problem2)],
    pooled=[False, True])

evaluateS1(parameters)

See LogPDF.evaluateS1().

The partial derivatives of the pooled log-likelihood with respect to unpooled parameters equals the partial derivative of the corresponding indiviudal log-likelihood.

\[\frac{\partial L}{\partial \psi} = \frac{\partial L_i}{\partial \psi},\]

where \(L\) is the pooled log-likelihood, \(\psi\) an unpooled parameter and \(L _i\) the individual log-likelihood that depends on \(\psi\).

For a pooled parameter \(\theta\) the partial derivative of the pooled log-likelihood equals to the sum of partial derivatives of all individual log-likelihoods

\[\frac{\partial L}{\partial \theta} = \sum _{i=1}^n\frac{\partial L_i}{\partial \theta}.\]

Here \(n\) is the number of individual log-likelihoods.

This method only works if all the underlying :class:`LogPDF` objects implement the optional method :meth:`LogPDF.evaluateS1()`!

n_parameters()

See LogPDF.n_parameters().

class pints.ProblemLogLikelihood(problem)

Represents a log-likelihood on a problem’s parameter space, used to indicate the likelihood of an observed (fixed) time-series given a particular parameter set (variable).

Extends LogPDF.

Parameters:problem – The time-series problem this log-likelihood is defined for.

evaluateS1(x)

Evaluates this LogPDF, and returns the result plus the partial derivatives of the result with respect to the parameters.

The returned data is a tuple (L, L') where L is a scalar value and L' is a sequence of length n_parameters.

Note that the derivative returned is of the log-pdf, so L' = d/dp log(f(p)), evaluated at p=x.

This is an optional method that is not always implemented.

n_parameters()

See LogPDF.n_parameters().

class pints.SumOfIndependentLogPDFs(log_likelihoods)

Calculates a sum of LogPDF objects, all defined on the same parameter space.

This is useful for e.g. Bayesian inference using a single model evaluated on two independent data sets D and E. In this case,

\[\begin{split}f(\theta|D,E) &= \frac{f(D, E|\theta)f(\theta)}{f(D, E)} \\ &= \frac{f(D|\theta)f(E|\theta)f(\theta)}{f(D, E)}\end{split}\]

Extends LogPDF.

Parameters:log_likelihoods – A sequence of LogPDF objects.

Example

log_likelihood = pints.SumOfIndependentLogPDFs([
    pints.GaussianLogLikelihood(problem1),
    pints.GaussianLogLikelihood(problem2),
])

evaluateS1(x)

See LogPDF.evaluateS1().

This method only works if all the underlying :class:`LogPDF` objects implement the optional method :meth:`LogPDF.evaluateS1()`!

n_parameters()

See LogPDF.n_parameters().