Nelder-Mead

class pints.NelderMead(x0, sigma0=None, boundaries=None)

Nelder-Mead downhill simplex method.

Implementation of the classical algorithm by [1], following the presentation in Algorithm 8.1 of [2].

This is a deterministic local optimiser. In most update steps it performs either 1 evaluation, or 2 sequential evaluations, so that it will not typically benefit from parallelisation.

Generates a “simplex” of n + 1 samples around a given starting point, and evaluates their scores. Next, each iteration consists of a sequence of operations, typically the worst sample y_worst is replaced with a new point:

y_new = mu + delta * (mu - y_worst)
mu = (1 / n) * sum(y), y != y_worst

where delta has one of four values, depending on the type of operation:

• Reflection (delta = 1)

• Expansion (delta = 2)

• Inside contraction (delta = -0.5)

• Outside contraction (delta = 0.5)

Note that the delta values here are common choices, but not the only valid choices.

A fifth type of iteration called a “shrink” is occasionally performed, in which all samples except the best sample y_best are replaced:

y_i_new = y_best + ys * (y_i - y_best)

where ys is a parameter (typically ys = 0.5).

The initialisation of the initial simplex was copied from [3].

References

[1]

A simplex method for function minimization Nelder, Mead 1965, Computer Journal https://doi.org/10.1093/comjnl/7.4.308

[2]

Introduction to derivative-free optimization Andrew R. Conn, Katya Scheinberg, Luis N. Vicente 2009, First edition. ISBN 978-0-098716-68-9 https://doi.org/10.1137/1.9780898718768

[3]

SciPy on GitHub https://github.com/scipy/scipy/

ask()

See: pints.Optimiser.ask().

f_best()

See: pints.Optimiser.f_best().

f_guessed()

For optimisers in which the best guess of the optimum (see x_guessed()) differs from the best-seen point (see x_best()), this method returns an estimate of the objective function value at x_guessed.

Notes:

1. For many optimisers the best guess is simply the best point seen during the optimisation, so that this method is equivalent to f_best().

2. Because x_guessed is not required to be a point that the optimiser has visited, the value f(x_guessed) may be unkown. In these cases, an approximation of f(x_guessed) may be returned.

fbest()

Deprecated alias of f_best().

n_hyper_parameters()

Returns the number of hyper-parameters for this method (see TunableMethod).

name()

See: pints.Optimiser.name().

needs_sensitivities()

Returns True if this methods needs sensitivities to be passed in to tell along with the evaluated error.

running()

See: pints.Optimiser.running().

set_hyper_parameters(x)

Sets the hyper-parameters for the method with the given vector of values (see TunableMethod).

Parameters:

x – An array of length n_hyper_parameters used to set the hyper-parameters.

stop()

See: pints.Optimiser.stop().

tell(fx)

See: pints.Optimiser.tell().

x_best()

See: pints.Optimiser.x_best().

x_guessed()

Returns the optimiser’s current best estimate of where the optimum is.

For many optimisers, this will simply be the point for which the minimal error or maximum likelihood was observed, so that x_guessed = x_best. However, optimisers like pints.CMAES and its derivatives, maintain a separate “best guess” value that does not necessarily correspond to any of the points evaluated during the optimisation.

xbest()

Deprecated alias of x_best().