Benchmarks
The following benchmarks show the performance of NestedSamplers.jl. As with any statistical inference package, the likelihood function will often dominate the runtime. This is important to consider when comparing packages across different languages- in general a custom Julia likelihood function may be faster than the same code written in Python/numpy. As an example, compare the relative timings of these two simple Guassian likelihoods
using BenchmarkTools
using PyCall
# julia version
gauss_loglike(X) = sum(x -> exp(-0.5 * x^2) / sqrt(2π), X)
# python version
py"""
import numpy as np
def gauss_loglike(X):
return np.sum(np.exp(-0.5 * X ** 2) / np.sqrt(2 * np.pi))
"""
gauss_loglike_py = py"gauss_loglike"
xs = randn(100)
@btime gauss_loglike($xs)
611.971 ns (0 allocations: 0 bytes)
26.813747896467206
@btime gauss_loglike_py($xs)
13.129 μs (6 allocations: 240 bytes)
26.81374789646721
In certain cases, you can use language interop tools (like PyCall.jl) to use Julia likelihoods with Python libraries.
Setup and system information
The benchmark code can be found in the bench
folder. The system information at the time these benchmarks were ran is
julia> versioninfo()
Julia Version 1.7.1
Commit ac5cc99908* (2021-12-22 19:35 UTC)
Platform Info:
OS: macOS (x86_64-apple-darwin20.5.0)
CPU: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-12.0.1 (ORCJIT, skylake)
Environment:
JULIA_NUM_THREADS = 1
Highly-correlated multivariate Guassian
This benchmark uses Models.CorrelatedGaussian
and simply measures the time it takes to fully sample down to dlogz=0.01
. This benchmark is exactly the same as the first benchmark detailed in the JAXNS paper.
Timing
Accuracy
The following shows the Bayesian evidence estmiate as compared to the true value