From Clojure, the syntax for calling an inference algorithm on a query is (doquery
:method foo args & method-opts)
. For example, we could run the smc
algorithm with
:number-of-particles
set to 100 on a program foo
defined by (defquery foo [x] ...)
as follows
(doquery :smc
foo [xval]
:number-of-particles 100)
When a query
object is passed to doquery
or conditional
the user has a choice of what kind of inference should be used to perform the characterization of the denoted conditional distribution. In general we recommend rmh
if all observes occur at the end of the program, smc
with as many particles as can fit in memory if observes occur with the program, and ipmcmc
for programs with internal observes that are too large for smc
to solve well.
CRITICAL: the default arguments supplied for all inference methods are the minimum required. This design make interactive debugging easier but nearly absolutely ensures poor inference performance unless non-default parameter values are chosen. For instance smc
performance will only get better with more particles; ultimately how many particles to use should be chosen so as to maximize the joint criteria of how long you want to wait and how good you want your results to be.
Anglican provides 4 general categories of inference methods and several instances of each:
The current list of algorithms that are included with Anglican is
Method | Type | Description |
---|---|---|
importance |
IS | Importance sampling (likelihood weighting) |
smc |
IS | Sequential Monte Carlo |
pcascade |
IS | Particle cascade (asynchronous sequential Monte Carlo) |
pgibbs |
PMCMC | Particle Gibbs (iterated conditional SMC) |
pimh |
PMCMC | Particle independent Metropolis-Hastings |
pgas |
PMCMC | Particle Gibbs with ancestor sampling |
ipmcmc |
PMCMC | Interacting particle Markov chain Monte Carlo |
lmh |
MCMC | Lightweight Metropolis-Hastings |
rmh |
MCMC | Random-walk Lightweight Metropolis-Hastings |
almh |
MCMC | Adaptive scheduling lightweight Metropolis-Hastings |
palmh |
MCMC | Parallelised adaptive scheduling lightweight Metropolis-Hastings |
plmh |
MCMC | Parallelised lightweight Metropolis-Hastings |
bamc |
MAP | Bayesian Ascent Monte Carlo |
siman |
MAP | MAP estimation via simulated annealing |
bbvb |
VI | Black Box Variational Inference |
Some algorithms have optional arguments, which can be specified with (doquery :method prog args :option value)
. These are
Method | Keyword | Default | Description |
---|---|---|---|
smc |
:number-of-particles | 1 | Number of particles |
rmh |
:alpha | 0.5 | Probability of using a local MCMC move |
:sigma | 1 | Spread of the local move | |
ipmcmc |
:number-of-particles | 2 | Number of particles per sweep |
:number-of-nodes | 32 | Number of nodes running SMC and CSMC | |
:number-of-csmc-nodes | (/ :number-of-nodes 2) | Number of nodes CSMC | |
:all-particles? | true | Return all particles or just one per node | |
:pool | (+ (npus) 2) | Number of threads to use | |
pgibbs |
:number-of-particles | 2 | |
pgas |
:number-of-particles | 2 | |
pimh |
:number-of-particles | 2 | |
plmh |
:number-of-threads | 2 | Number of processor threads |
palmh |
:number-of-threads | 2 | |
pcascade |
:number-of-threads | 16 | |
:number-of-particles | :number-of-threads/2 | Initial number of particles | |
bamc |
:predict-candidates | false | Output samples with non-increasing log-weight |
siman |
:predict-candidates | false | |
:cooling-rate | 0.99 | Cooling rate (should be less than 1) | |
:cooling-schedule | :exponential | Cooling schedule, :exponential or :lundy-mees | |
bbvb |
:number-of-particles | 100 | Number of particles |
:base-step-size | 1.0 | Gradient descent stepsize | |
:adagrad | true | Use adagrad adaptive gradient ascent | |
:robbins-monro | 0.0 | Stepsize gradient decay |