Initializing HMC with Pathfinder

The MCMC warm-up phase

When using MCMC to draw samples from some target distribution, there is often a lengthy warm-up phase with 2 phases:

converge to the typical set (the region of the target distribution where the bulk of the probability mass is located)
adapt any tunable parameters of the MCMC sampler (optional)

While (1) often happens fairly quickly, (2) usually requires a lengthy exploration of the typical set to iteratively adapt parameters suitable for further exploration. An example is the widely used windowed adaptation scheme of Hamiltonian Monte Carlo (HMC) in Stan [3], where a step size and positive definite metric (aka mass matrix) are adapted. For posteriors with complex geometry, the adaptation phase can require many evaluations of the gradient of the log density function of the target distribution.

Pathfinder can be used to initialize MCMC, and in particular HMC, in 3 ways:

Initialize MCMC from one of Pathfinder's draws (replace phase 1 of the warm-up).
Initialize an HMC metric adaptation from the inverse of the covariance of the multivariate normal approximation (replace the first window of phase 2 of the warm-up).
Use the inverse of the covariance as the metric without further adaptation (replace phase 2 of the warm-up).

This tutorial demonstrates all three approaches with DynamicHMC.jl and AdvancedHMC.jl. Both of these packages have standalone implementations of adaptive HMC (aka NUTS) and can be used independently of any probabilistic programming language (PPL). Both the Turing and Soss PPLs have some DynamicHMC integration, while Turing also integrates with AdvancedHMC.

For demonstration purposes, we'll use the following dummy data:

using LinearAlgebra, Pathfinder, Random, StatsFuns, StatsPlots

Random.seed!(91)

x = 0:0.01:1
y = @. sin(10x) + randn() * 0.2 + x

scatter(x, y; xlabel="x", ylabel="y", legend=false, msw=0, ms=2)

We'll fit this using a simple polynomial regression model:

\[\begin{aligned} \sigma &\sim \text{Half-Normal}(\mu=0, \sigma=1)\\ \alpha, \beta_j &\sim \mathrm{Normal}(\mu=0, \sigma=1)\\ \hat{y}_i &= \alpha + \sum_{j=1}^J x_i^j \beta_j\\ y_i &\sim \mathrm{Normal}(\mu=\hat{y}_i, \sigma=\sigma) \end{aligned}\]

We just need to implement the log-density function of the resulting posterior.

struct RegressionProblem{X,Z,Y}
    x::X
    J::Int
    z::Z
    y::Y
end
function RegressionProblem(x, J, y)
    z = x .* (1:J)'
    return RegressionProblem(x, J, z, y)
end

function (prob::RegressionProblem)(θ)
    (; σ, α, β) = θ
    (; z, y) = prob
    lp = normlogpdf(σ) + logtwo
    lp += normlogpdf(α)
    lp += sum(normlogpdf, β)
    y_hat = muladd(z, β, α)
    lp += sum(eachindex(y_hat, y)) do i
        return normlogpdf(y_hat[i], σ, y[i])
    end
    return lp
end

J = 3
dim = J + 2
model = RegressionProblem(x, J, y)
ndraws = 1_000;

DynamicHMC.jl

To use DynamicHMC, we first need to transform our model to an unconstrained space using TransformVariables.jl and wrap it in a type that implements the LogDensityProblems interface (see DynamicHMC's worked example):

using DynamicHMC, ForwardDiff, LogDensityProblems, LogDensityProblemsAD, TransformVariables
using TransformedLogDensities: TransformedLogDensity

transform = as((σ=asℝ₊, α=asℝ, β=as(Array, J)))
P = TransformedLogDensity(transform, model)
∇P = ADgradient(:ForwardDiff, P)

ForwardDiff AD wrapper for TransformedLogDensity of dimension 5, w/ chunk size 5

Pathfinder can take any object that implements this interface.

result_pf = pathfinder(∇P)

Single-path Pathfinder result
  tries: 1
  draws: 5
  fit iteration: 19 (total: 27)
  fit ELBO: -116.38 ± 0.41
  fit distribution: MvNormal{Float64, Pathfinder.WoodburyPDMat{Float64, LinearAlgebra.Diagonal{Float64, Vector{Float64}}, Matrix{Float64}, Matrix{Float64}, Pathfinder.WoodburyPDFactorization{Float64, LinearAlgebra.Diagonal{Float64, Vector{Float64}}, LinearAlgebra.QRCompactWYQ{Float64, Matrix{Float64}, Matrix{Float64}}, LinearAlgebra.UpperTriangular{Float64, Matrix{Float64}}}}, Vector{Float64}}(
dim: 5
μ: [-0.3519374274430237, 0.2469006000838245, 0.058275882597590235, 0.11655176210247765, 0.17482764793925842]
Σ: [0.009946200741163912 -0.010857736662224778 … 0.0945761989658503 -0.05406126770592312; -0.010857736662224775 0.04217029814718195 … -0.21130899138871223 0.11256047339424753; … ; 0.0945761989658503 -0.21130899138871223 … 2.477040093076655 -1.4323279373309448; -0.05406126770592312 0.11256047339424753 … -1.4323279373309448 0.8357026950008384]
)

init_params = result_pf.draws[:, 1]

5-element Vector{Float64}:
 -0.17659426520397614
  0.13410956190418802
 -0.27880094938697114
  1.3210092582940742
 -0.43308396799178983

inv_metric = result_pf.fit_distribution.Σ

5×5 Pathfinder.WoodburyPDMat{Float64, LinearAlgebra.Diagonal{Float64, Vector{Float64}}, Matrix{Float64}, Matrix{Float64}, Pathfinder.WoodburyPDFactorization{Float64, LinearAlgebra.Diagonal{Float64, Vector{Float64}}, LinearAlgebra.QRCompactWYQ{Float64, Matrix{Float64}, Matrix{Float64}}, LinearAlgebra.UpperTriangular{Float64, Matrix{Float64}}}}:
  0.0099462  -0.0108577  -0.0160873   0.0945762  -0.0540613
 -0.0108577   0.0421703   0.0336147  -0.211309    0.11256
 -0.0160873   0.0336147   0.097474   -0.439433    0.249701
  0.0945762  -0.211309   -0.439433    2.47704    -1.43233
 -0.0540613   0.11256     0.249701   -1.43233     0.835703

Initializing from Pathfinder's draws

Here we just need to pass one of the draws as the initial point q:

result_dhmc1 = mcmc_with_warmup(
    Random.default_rng(),
    ∇P,
    ndraws;
    initialization=(; q=init_params),
    reporter=NoProgressReport(),
)

(posterior_matrix = [-0.35511738176064195 -0.5138553553954529 … -0.39284308115853467 -0.3943363729060238; 0.3060054956100503 0.4476331651499723 … 0.14908539440475727 0.14189620993262822; … ; 0.526550648917556 0.25347681105778647 … -0.36230642943718566 -0.31927734829896903; 0.3131251335477015 0.14319593830243318 … 0.6680271714035524 0.6525725911815745], tree_statistics = DynamicHMC.TreeStatisticsNUTS[DynamicHMC.TreeStatisticsNUTS(-115.39857852877277, 6, turning at positions -62:1, 0.9998633859279341, 63, DynamicHMC.Directions(0x9ed9ffc1)), DynamicHMC.TreeStatisticsNUTS(-117.71730896445258, 5, turning at positions -28:3, 0.9984654640741676, 31, DynamicHMC.Directions(0xdf28c6e3)), DynamicHMC.TreeStatisticsNUTS(-119.34095850978332, 6, turning at positions -53:10, 0.9990628615048124, 63, DynamicHMC.Directions(0x012f25ca)), DynamicHMC.TreeStatisticsNUTS(-122.21032472332335, 4, turning at positions -4:-19, 0.9684688726920223, 31, DynamicHMC.Directions(0xbed82e6c)), DynamicHMC.TreeStatisticsNUTS(-115.57814273719835, 6, turning at positions -18:45, 0.9996995795489445, 63, DynamicHMC.Directions(0xd1dfc66d)), DynamicHMC.TreeStatisticsNUTS(-116.11589077187067, 6, turning at positions -2:61, 0.9078118483560944, 63, DynamicHMC.Directions(0x2b019f3d)), DynamicHMC.TreeStatisticsNUTS(-115.96457069627621, 6, turning at positions -62:1, 0.8728583370983598, 63, DynamicHMC.Directions(0x89a394c1)), DynamicHMC.TreeStatisticsNUTS(-117.54449542109303, 6, turning at positions -58:5, 0.7535391771971093, 63, DynamicHMC.Directions(0x1be982c5)), DynamicHMC.TreeStatisticsNUTS(-117.06403817678692, 6, turning at positions -49:14, 0.9987077072525765, 63, DynamicHMC.Directions(0x3cbe5c8e)), DynamicHMC.TreeStatisticsNUTS(-117.99333700001863, 6, turning at positions -10:53, 0.9933094623669495, 63, DynamicHMC.Directions(0x10f318f5))  …  DynamicHMC.TreeStatisticsNUTS(-116.86287022541863, 6, turning at positions -46:17, 0.936947905078784, 63, DynamicHMC.Directions(0x25b497d1)), DynamicHMC.TreeStatisticsNUTS(-119.08525250605203, 5, turning at positions -14:-45, 0.9733164545713815, 63, DynamicHMC.Directions(0x147fc692)), DynamicHMC.TreeStatisticsNUTS(-117.91630169397129, 6, turning at positions -58:5, 0.8327663314690851, 63, DynamicHMC.Directions(0x02a35505)), DynamicHMC.TreeStatisticsNUTS(-119.38264072706787, 5, turning at positions -14:-45, 0.9848718648907713, 63, DynamicHMC.Directions(0x21e26d12)), DynamicHMC.TreeStatisticsNUTS(-115.57487492479184, 6, turning at positions -10:53, 0.9995991762398503, 63, DynamicHMC.Directions(0x7f5600b5)), DynamicHMC.TreeStatisticsNUTS(-115.72283494919103, 6, turning at positions 45:108, 0.998961823134643, 127, DynamicHMC.Directions(0x9ce0106c)), DynamicHMC.TreeStatisticsNUTS(-116.13351810669845, 5, turning at positions 26:57, 0.9471995034412938, 63, DynamicHMC.Directions(0x736f4cf9)), DynamicHMC.TreeStatisticsNUTS(-114.63951082870615, 6, turning at positions 0:63, 0.8816855493403424, 63, DynamicHMC.Directions(0xbce1f9ff)), DynamicHMC.TreeStatisticsNUTS(-113.4555163220938, 6, turning at positions -2:61, 0.9891820462426757, 63, DynamicHMC.Directions(0xce06373d)), DynamicHMC.TreeStatisticsNUTS(-114.97985376589294, 6, turning at positions -32:31, 0.9383495730472687, 63, DynamicHMC.Directions(0x7443319f))], logdensities = [-113.12071771622887, -116.61352161268769, -117.2830302663124, -112.70492141254182, -114.03534970062894, -114.30878561068549, -113.52196335327253, -114.6928512624253, -114.5926391860889, -115.59437910163918  …  -113.08521871484561, -112.82357473175298, -115.05347896192983, -114.74840695555592, -113.81891756542707, -112.98349813818623, -113.28840708769623, -112.99745161906044, -112.88872777454799, -113.32122214634329], κ = Gaussian kinetic energy (Diagonal), √diag(M⁻¹): [0.07386015267732474, 0.12631860513414883, 0.956823707659201, 0.7299841060408273, 0.5489147747407914], ϵ = 0.05488687345900733)

Initializing metric adaptation from Pathfinder's estimate

To start with Pathfinder's inverse metric estimate, we just need to initialize a DynamicHMC.GaussianKineticEnergy object with it as input:

result_dhmc2 = mcmc_with_warmup(
    Random.default_rng(),
    ∇P,
    ndraws;
    initialization=(; q=init_params, κ=GaussianKineticEnergy(inv_metric)),
    warmup_stages=default_warmup_stages(; M=Symmetric),
    reporter=NoProgressReport(),
)

(posterior_matrix = [-0.40515806164711005 -0.3015207027421393 … -0.36928828667560765 -0.2948031312947311; 0.4752869054972702 -0.0007623108539365486 … 0.4433866299755978 0.05562199957539912; … ; 0.7114336779146463 -0.5998551737278297 … 1.2025601499415097 -0.788617062309446; 0.015800369239446166 0.4597569000178074 … -0.5301160842496415 0.7693134917309549], tree_statistics = DynamicHMC.TreeStatisticsNUTS[DynamicHMC.TreeStatisticsNUTS(-117.24578783638374, 3, turning at positions -4:3, 0.9129288820770928, 7, DynamicHMC.Directions(0xc83b9a03)), DynamicHMC.TreeStatisticsNUTS(-118.12743152696343, 3, turning at positions 0:7, 0.8934353553941321, 7, DynamicHMC.Directions(0x2889cabf)), DynamicHMC.TreeStatisticsNUTS(-119.86204168734905, 4, turning at positions -14:1, 0.8470033928624826, 15, DynamicHMC.Directions(0x96ff3f61)), DynamicHMC.TreeStatisticsNUTS(-120.29462751109564, 4, turning at positions -3:12, 0.8980948758022241, 15, DynamicHMC.Directions(0xd1ba64dc)), DynamicHMC.TreeStatisticsNUTS(-116.66417025984705, 3, turning at positions -4:3, 0.9568703328499841, 7, DynamicHMC.Directions(0xecef6903)), DynamicHMC.TreeStatisticsNUTS(-118.60003904414688, 3, turning at positions 0:7, 0.9817317203618057, 7, DynamicHMC.Directions(0xeaa14067)), DynamicHMC.TreeStatisticsNUTS(-116.91756359529086, 4, turning at positions -10:5, 0.9676563615341496, 15, DynamicHMC.Directions(0x9e69b145)), DynamicHMC.TreeStatisticsNUTS(-118.82698109561717, 3, turning at positions -5:2, 0.9191438428784892, 7, DynamicHMC.Directions(0x8e79b6f2)), DynamicHMC.TreeStatisticsNUTS(-121.7434107515719, 2, turning at positions 0:3, 0.8433629192619967, 3, DynamicHMC.Directions(0x2c9f5167)), DynamicHMC.TreeStatisticsNUTS(-121.78046351632915, 3, turning at positions -1:6, 0.9999999999999999, 7, DynamicHMC.Directions(0x3cbbdd0e))  …  DynamicHMC.TreeStatisticsNUTS(-116.46703667912422, 4, turning at positions 0:15, 0.9633587619468066, 15, DynamicHMC.Directions(0x98e7312f)), DynamicHMC.TreeStatisticsNUTS(-117.7075374301976, 3, turning at positions -5:-12, 0.7318808729837325, 15, DynamicHMC.Directions(0x62fdfe93)), DynamicHMC.TreeStatisticsNUTS(-116.78815133489977, 4, turning at positions -13:2, 0.9531902155536204, 15, DynamicHMC.Directions(0xd4eb8c32)), DynamicHMC.TreeStatisticsNUTS(-116.24249548925228, 3, turning at positions -1:6, 0.9384361025421881, 7, DynamicHMC.Directions(0x745701fe)), DynamicHMC.TreeStatisticsNUTS(-117.4992596359232, 4, turning at positions -1:14, 0.9051362768126034, 15, DynamicHMC.Directions(0xafc892de)), DynamicHMC.TreeStatisticsNUTS(-119.40997936034192, 4, turning at positions -12:3, 0.8953615570450499, 15, DynamicHMC.Directions(0x23fef483)), DynamicHMC.TreeStatisticsNUTS(-117.73625455275845, 4, turning at positions -8:7, 0.9283234637747341, 15, DynamicHMC.Directions(0x9b607637)), DynamicHMC.TreeStatisticsNUTS(-117.60414285393823, 3, turning at positions 0:7, 0.9201958784622233, 7, DynamicHMC.Directions(0x871a626f)), DynamicHMC.TreeStatisticsNUTS(-117.34947680910899, 3, turning at positions -1:6, 0.97692785945346, 7, DynamicHMC.Directions(0x30edb31e)), DynamicHMC.TreeStatisticsNUTS(-115.70607543107337, 4, turning at positions -5:10, 1.0, 15, DynamicHMC.Directions(0x7dbf994a))], logdensities = [-115.63837047409372, -114.75598579706347, -114.56343855951047, -113.54814999569095, -116.01681069182753, -113.97778011236372, -115.32018381515084, -117.94690011571625, -121.3335142888974, -114.1108786446355  …  -114.36911545940434, -113.5726294914352, -114.99841395920215, -113.17073446078822, -115.37460389974667, -116.27075123149675, -113.57632891097788, -114.09165785672312, -114.71351687871481, -113.99313373782154], κ = Gaussian kinetic energy (Symmetric), √diag(M⁻¹): [0.06660129511681978, 0.1500330643051528, 0.9882644796533832, 0.8151679366307755, 0.5791905326585098], ϵ = 0.3579277658878891)

We also specified that DynamicHMC should tune a dense Symmetric matrix. However, we could also have requested a Diagonal metric.

Use Pathfinder's metric estimate for sampling

To turn off metric adaptation entirely and use Pathfinder's estimate, we just set the number and size of the metric adaptation windows to 0.

result_dhmc3 = mcmc_with_warmup(
    Random.default_rng(),
    ∇P,
    ndraws;
    initialization=(; q=init_params, κ=GaussianKineticEnergy(inv_metric)),
    warmup_stages=default_warmup_stages(; middle_steps=0, doubling_stages=0),
    reporter=NoProgressReport(),
)

(posterior_matrix = [-0.4006457770414287 -0.3098118616646564 … -0.36018797271673797 -0.4589851529484841; 0.2678340499115404 0.27029630471335464 … 0.3011278402682173 0.39089531827678287; … ; 0.11515285599431158 -0.2795266402597436 … -0.411403374832905 1.1206204528548225; 0.32061758703079724 0.39925750894717593 … 0.8252394666574091 -0.5163836397102695], tree_statistics = DynamicHMC.TreeStatisticsNUTS[DynamicHMC.TreeStatisticsNUTS(-116.49784423101646, 4, turning at positions 2:9, 0.9945979929022739, 23, DynamicHMC.Directions(0xa527ea31)), DynamicHMC.TreeStatisticsNUTS(-113.78389538719082, 4, turning at positions -21:-28, 0.9992853594660515, 31, DynamicHMC.Directions(0xb518e0e3)), DynamicHMC.TreeStatisticsNUTS(-113.94828618381534, 2, turning at positions -2:-5, 0.9281821921274337, 7, DynamicHMC.Directions(0x80eaf8a2)), DynamicHMC.TreeStatisticsNUTS(-117.20440598571676, 3, turning at positions -1:6, 0.9544048973013793, 7, DynamicHMC.Directions(0xc73d57ce)), DynamicHMC.TreeStatisticsNUTS(-119.00444613598198, 4, turning at positions 16:23, 0.9350764556147072, 31, DynamicHMC.Directions(0x1d270c77)), DynamicHMC.TreeStatisticsNUTS(-121.26906555495185, 3, turning at positions -5:2, 0.7919233587299283, 7, DynamicHMC.Directions(0xa8706b62)), DynamicHMC.TreeStatisticsNUTS(-121.73403886818953, 2, turning at positions 4:7, 0.9733552967422268, 7, DynamicHMC.Directions(0x2b10bfb7)), DynamicHMC.TreeStatisticsNUTS(-119.28504642098932, 3, turning at positions -7:0, 0.8241815854344565, 7, DynamicHMC.Directions(0x78201c98)), DynamicHMC.TreeStatisticsNUTS(-124.03451257481471, 3, turning at positions -2:5, 0.9739198902146304, 7, DynamicHMC.Directions(0x6d2864dd)), DynamicHMC.TreeStatisticsNUTS(-119.56845146426076, 2, turning at positions 4:7, 0.9999999999999999, 7, DynamicHMC.Directions(0x59f2eabf))  …  DynamicHMC.TreeStatisticsNUTS(-115.65913125305711, 3, turning at positions -6:1, 0.9115449797142486, 7, DynamicHMC.Directions(0x3bd57909)), DynamicHMC.TreeStatisticsNUTS(-113.69802027709432, 3, turning at positions -2:5, 0.8668472220922557, 7, DynamicHMC.Directions(0xbafb840d)), DynamicHMC.TreeStatisticsNUTS(-114.38608629240531, 3, turning at positions -1:-8, 0.9819252497669955, 15, DynamicHMC.Directions(0xc8ec6487)), DynamicHMC.TreeStatisticsNUTS(-120.74901857787641, 5, turning at positions -19:-34, 0.8839116384794224, 47, DynamicHMC.Directions(0x662b57cd)), DynamicHMC.TreeStatisticsNUTS(-116.69628828325901, 5, turning at positions -24:-31, 0.9434187242499033, 47, DynamicHMC.Directions(0xfd3e3910)), DynamicHMC.TreeStatisticsNUTS(-118.56633458291763, 3, turning at positions 0:7, 0.941516449061952, 7, DynamicHMC.Directions(0x442b10df)), DynamicHMC.TreeStatisticsNUTS(-114.61443953733055, 5, turning at positions -10:-17, 0.9953954433970494, 39, DynamicHMC.Directions(0xb7353156)), DynamicHMC.TreeStatisticsNUTS(-114.66278997047601, 4, turning at positions 18:25, 0.954361032360652, 31, DynamicHMC.Directions(0x5e5efb39)), DynamicHMC.TreeStatisticsNUTS(-113.88767589993654, 4, turning at positions 12:19, 1.0, 23, DynamicHMC.Directions(0x54195e7b)), DynamicHMC.TreeStatisticsNUTS(-117.3870048048878, 5, turning at positions -37:-40, 0.8628928194441827, 47, DynamicHMC.Directions(0x4136da47))], logdensities = [-112.94150164685594, -112.65248453087281, -113.7615326773123, -115.6509895704956, -114.87261521643524, -119.10906450452926, -114.04696867257347, -118.11544553080198, -118.93445485394422, -113.23510091560918  …  -112.43240969212668, -112.88758824357684, -113.81466667225361, -113.6675065744535, -114.7356360345121, -113.1548443979893, -113.36284892653613, -113.10231377654719, -113.0002376204635, -115.27878612162216], κ = Gaussian kinetic energy (WoodburyPDMat), √diag(M⁻¹): [0.09973064093428816, 0.20535407993799867, 0.3122083359083897, 1.5738615228401307, 0.9141677608627633], ϵ = 0.4440173911163078)

AdvancedHMC.jl

Similar to Pathfinder and DynamicHMC, AdvancedHMC can also work with a LogDensityProblem:

using AdvancedHMC

nadapts = 500;

Initializing from Pathfinder's draws

metric = DiagEuclideanMetric(dim)
hamiltonian = Hamiltonian(metric, ∇P)
ϵ = find_good_stepsize(hamiltonian, init_params)
integrator = Leapfrog(ϵ)
kernel = HMCKernel(Trajectory{MultinomialTS}(integrator, GeneralisedNoUTurn()))
adaptor = StepSizeAdaptor(0.8, integrator)
samples_ahmc1, stats_ahmc1 = sample(
    hamiltonian,
    kernel,
    init_params,
    ndraws + nadapts,
    adaptor,
    nadapts;
    drop_warmup=true,
    progress=false,
)

([[-0.49601571978022696, 0.3516936628617788, 0.4426165085755207, -0.012798878805417933, 0.05692265531426957], [-0.4160612366455508, 0.18020449646608255, 0.4780437187788644, 0.30093064698941163, -0.059437647435650394], [-0.19694090621058952, 0.26219923068464895, 0.47098977986834745, 0.22904182473394805, 0.008644228463192972], [-0.4050174876088776, 0.47622351378021915, 0.48236960042483173, 0.21754364016534525, -0.10156563946126865], [-0.2876224928068998, 0.12357310833929629, 0.6353403506194931, 0.23091153569109205, 0.024149391730259605], [-0.42602290349188987, 0.3582681221890307, 0.6104942372113089, 0.23060445731413945, -0.10062477194712313], [-0.44991830043318376, 0.2806785800749659, -0.5315255239062858, -0.29903494311309964, 0.6974451380503486], [-0.23133263891916672, 0.4398825637149451, 0.23083562455878825, -0.03646405830485531, 0.21267986979415734], [-0.22864730821942436, 0.020295925883694207, 0.09647613227537728, 0.07726302645323976, 0.34127384300194824], [-0.43467189598942657, 0.046437768980872554, 0.14473257015628926, 0.026215003885756497, 0.2803151111559735]  …  [-0.4184027360468004, 0.2669430699278612, 1.3839031655771579, 0.09067885752344856, -0.2684734481323344], [-0.27364725996467565, 0.3787632863972153, 1.2703649444013263, 0.0593194861106531, -0.18493363036744387], [-0.35029911058462637, 0.4704818086707959, 1.3157887658300187, -0.0026672036391307864, -0.23125039932057007], [-0.2175139630252042, 0.44151533494614015, 1.3344243526063242, -0.04003348857651552, -0.2512342960859379], [-0.4895640929080781, 0.15637581024692054, 1.4794315203306398, 0.19064044105662264, -0.283620757362977], [-0.464190007005124, 0.31943895153645113, 1.8813252521739834, 0.021711920809132174, -0.3725890690753484], [-0.3277360536744764, 0.3179415819380844, 1.7234179923883055, -0.1058253547552422, -0.3041600592992404], [-0.3794212678847367, 0.1134062232314722, 1.4285511578808159, 0.012646025953028238, -0.1649206673706872], [-0.3271748689383702, 0.35678465229314754, -0.5469109365249575, 0.7388389713134396, -0.08601024770370502], [-0.34418969763570184, 0.19173037914272165, -0.11687278326190698, -1.163900376612295, 1.0880408633778347]], NamedTuple[(n_steps = 15, is_accept = true, acceptance_rate = 0.927787545568205, log_density = -115.08491989771058, hamiltonian_energy = 115.81824653201315, hamiltonian_energy_error = 0.22207624732116926, max_hamiltonian_energy_error = 0.2571993106970467, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 23, is_accept = true, acceptance_rate = 0.995378583176479, log_density = -112.777678857418, hamiltonian_energy = 116.46120087611632, hamiltonian_energy_error = -0.24413220874339459, max_hamiltonian_energy_error = -0.2543280443890552, tree_depth = 4, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.52848994194592, log_density = -115.0368813488538, hamiltonian_energy = 115.80336552050359, hamiltonian_energy_error = 0.610088935803816, max_hamiltonian_energy_error = 1.226366745296957, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 11, is_accept = true, acceptance_rate = 0.9350866842453935, log_density = -114.77017904604885, hamiltonian_energy = 117.79627003860708, hamiltonian_energy_error = -0.21383884823168842, max_hamiltonian_energy_error = -0.5441487833610523, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 15, is_accept = true, acceptance_rate = 0.9740179686054908, log_density = -113.87310288200294, hamiltonian_energy = 116.17746664002202, hamiltonian_energy_error = -0.041637739215033776, max_hamiltonian_energy_error = -0.32260315693487485, tree_depth = 4, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 11, is_accept = true, acceptance_rate = 0.9620087358608141, log_density = -113.79660609228601, hamiltonian_energy = 115.46018776866487, hamiltonian_energy_error = -0.1028075735563192, max_hamiltonian_energy_error = -0.2103471320070298, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 51, is_accept = true, acceptance_rate = 0.8236205089971347, log_density = -115.15407995573482, hamiltonian_energy = 117.1768856683029, hamiltonian_energy_error = 0.5563505789811529, max_hamiltonian_energy_error = 0.7295006431010052, tree_depth = 5, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 47, is_accept = true, acceptance_rate = 0.8653117244985359, log_density = -116.17374022303464, hamiltonian_energy = 117.50346077224202, hamiltonian_energy_error = 0.35827814474703246, max_hamiltonian_energy_error = -0.8322344738441103, tree_depth = 5, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 11, is_accept = true, acceptance_rate = 0.8784017185246565, log_density = -114.95082147124579, hamiltonian_energy = 122.3475043734504, hamiltonian_energy_error = -0.6092569497841254, max_hamiltonian_energy_error = 1.1755259887095093, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.9623262202494008, log_density = -114.3074267395098, hamiltonian_energy = 116.37279228672087, hamiltonian_energy_error = -0.061669415974890285, max_hamiltonian_energy_error = 0.21090360137893072, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false)  …  (n_steps = 15, is_accept = true, acceptance_rate = 0.9808790987091704, log_density = -113.55160328347957, hamiltonian_energy = 115.54294828619716, hamiltonian_energy_error = -0.09911030789659492, max_hamiltonian_energy_error = -0.137232163406523, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 23, is_accept = true, acceptance_rate = 0.6296288486057082, log_density = -115.21102239096687, hamiltonian_energy = 116.66739779774969, hamiltonian_energy_error = 0.764883864545908, max_hamiltonian_energy_error = 1.1784242913824556, tree_depth = 4, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 1.0, log_density = -114.88284185136163, hamiltonian_energy = 115.80098509487928, hamiltonian_energy_error = -0.3634294844049606, max_hamiltonian_energy_error = -0.5770241507651974, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.9291243095362045, log_density = -115.45188923249502, hamiltonian_energy = 116.33853766529342, hamiltonian_energy_error = -0.09078772814196157, max_hamiltonian_energy_error = 0.2029698796420405, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 15, is_accept = true, acceptance_rate = 0.9757908010852153, log_density = -115.7298323341413, hamiltonian_energy = 116.80292467460004, hamiltonian_energy_error = 0.0837000612440022, max_hamiltonian_energy_error = -0.12437716760676665, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 19, is_accept = true, acceptance_rate = 0.9631656896003987, log_density = -115.84971196465041, hamiltonian_energy = 118.18959488756234, hamiltonian_energy_error = 0.1256329914031511, max_hamiltonian_energy_error = -0.20110342629855893, tree_depth = 4, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 1.0, log_density = -114.17900094532358, hamiltonian_energy = 117.17440239869993, hamiltonian_energy_error = -0.14498626752681787, max_hamiltonian_energy_error = -0.311967859371336, tree_depth = 3, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 31, is_accept = true, acceptance_rate = 0.984648303762743, log_density = -113.76055816959004, hamiltonian_energy = 116.18519013774342, hamiltonian_energy_error = -0.0980807258425358, max_hamiltonian_energy_error = -0.1453578420279058, tree_depth = 4, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 63, is_accept = true, acceptance_rate = 0.9961974473292537, log_density = -112.8599931042722, hamiltonian_energy = 114.47150859546777, hamiltonian_energy_error = -0.06927487902589746, max_hamiltonian_energy_error = -0.08820214501098178, tree_depth = 6, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false), (n_steps = 63, is_accept = true, acceptance_rate = 0.9631260536765327, log_density = -113.61545469978066, hamiltonian_energy = 114.88967158406315, hamiltonian_energy_error = 0.08137734500766669, max_hamiltonian_energy_error = 0.09139380530302788, tree_depth = 6, numerical_error = false, step_size = 0.040189804855371915, nom_step_size = 0.040189804855371915, is_adapt = false)])

Initializing metric adaptation from Pathfinder's estimate

We can't start with Pathfinder's inverse metric estimate directly. Instead we need to first extract its diagonal for a DiagonalEuclideanMetric or make it dense for a DenseEuclideanMetric:

metric = DenseEuclideanMetric(Matrix(inv_metric))
hamiltonian = Hamiltonian(metric, ∇P)
ϵ = find_good_stepsize(hamiltonian, init_params)
integrator = Leapfrog(ϵ)
kernel = HMCKernel(Trajectory{MultinomialTS}(integrator, GeneralisedNoUTurn()))
adaptor = StepSizeAdaptor(0.8, integrator)
samples_ahmc2, stats_ahmc2 = sample(
    hamiltonian,
    kernel,
    init_params,
    ndraws + nadapts,
    adaptor,
    nadapts;
    drop_warmup=true,
    progress=false,
)

([[-0.3542583119280188, 0.21509243096686848, -1.1245500417632324, -0.37968014597119615, 0.9538683646253557], [-0.45471625143280076, 0.17333894598657348, -1.1782814701779518, 0.08042845202280363, 0.6668637504627436], [-0.2497848756375956, 0.2840082796978076, -1.0485652142361594, -0.6047743444224716, 0.9870923469835599], [-0.2669606886307623, 0.12306216933157373, -1.2265499760791017, 0.7935135443122733, 0.2664608255891088], [-0.33615498870097477, 0.14913257836521931, -1.2707328040150354, 0.49599120157116827, 0.3510603696738369], [-0.3989737155246304, 0.17548813128176197, -1.1530784188922132, -0.10255862382845193, 0.7531586015948349], [-0.49966370335956395, 0.32318018998841497, -1.7279936742641078, 1.3621467665595492, -0.17475819455582287], [-0.32955583632819924, 0.17766336656381643, -1.6362092569375368, 1.1409121358277496, 0.11658318581636198], [-0.32974861266396976, 0.18849131586529622, -1.266094164872395, -0.22458443034058728, 0.8357683173998162], [-0.32676382783612273, 0.3645465881282004, -1.4573284652363916, 0.37109872762342866, 0.5124012123120021]  …  [-0.3191643580854772, 0.1452230614018443, 1.707661400101134, 0.12458716537403772, -0.3416418105089809], [-0.3560748703197543, 0.3023586550583869, 1.9286618957013797, -0.16359712995634834, -0.2641656588566582], [-0.38360173466941266, 0.1400514282879115, 0.7978678589900917, 0.04276941140778062, 0.03498481132297804], [-0.2688131431702196, 0.3236681271802563, 0.767946250134151, -0.19844079196695485, 0.08367632459800767], [-0.3998080664199487, 0.18488288312893553, 0.36523959102058245, 0.1500679763958635, 0.12595246879315558], [-0.3377943684989902, 0.3598009759436781, 0.4017231135333215, -0.8113433942496833, 0.576786062540531], [-0.2947611117921396, 0.2802115881859889, -0.40721773319139265, 2.099535859610114, -0.983237251494876], [-0.3318283668238215, 0.10384660764780888, 0.9501286923944721, -1.247900175336073, 0.8091235934858331], [-0.21971079824874543, 0.14149025209151384, 0.49103028912903546, 1.297203615689788, -0.725981545055259], [-0.39566558299456256, 0.2889775883958418, 0.13827486543080558, 0.030513891469831966, 0.200127715207985]], NamedTuple[(n_steps = 7, is_accept = true, acceptance_rate = 0.15564753188688601, log_density = -113.67233430521479, hamiltonian_energy = 119.50450283536127, hamiltonian_energy_error = 0.0, max_hamiltonian_energy_error = 4.234537174646547, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.8487076444240957, log_density = -114.41456041136102, hamiltonian_energy = 118.1102136159144, hamiltonian_energy_error = 0.032077181174528846, max_hamiltonian_energy_error = 0.2834051343345436, tree_depth = 2, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.951459993421919, log_density = -114.38022933016659, hamiltonian_energy = 116.8263019153856, hamiltonian_energy_error = -0.019320961531903436, max_hamiltonian_energy_error = 0.13446881756600249, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.7597496396778585, log_density = -114.89043787108305, hamiltonian_energy = 117.2998379203381, hamiltonian_energy_error = 0.21872640903173135, max_hamiltonian_energy_error = 0.5663375717985417, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.9981063859144454, log_density = -114.47208379767416, hamiltonian_energy = 115.79670830993409, hamiltonian_energy_error = 0.00879357937550651, max_hamiltonian_energy_error = -0.2386142352510774, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.9637462114315839, log_density = -113.3907393661242, hamiltonian_energy = 116.00365803286839, hamiltonian_energy_error = -0.34545293232918084, max_hamiltonian_energy_error = -0.34545293232918084, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.7750764597880518, log_density = -119.57421340003366, hamiltonian_energy = 121.73328964504192, hamiltonian_energy_error = 0.5036054604058791, max_hamiltonian_energy_error = 0.5036054604058791, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 3, is_accept = true, acceptance_rate = 0.9144583179498397, log_density = -114.39190349735448, hamiltonian_energy = 120.39335716475068, hamiltonian_energy_error = -0.13396055305256027, max_hamiltonian_energy_error = -0.3716342992816095, tree_depth = 2, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.9982384763266194, log_density = -113.76073090498468, hamiltonian_energy = 114.58636146991049, hamiltonian_energy_error = -0.17719350562613556, max_hamiltonian_energy_error = -0.21891342215469933, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.8806528186293817, log_density = -114.794504331819, hamiltonian_energy = 116.59801055004625, hamiltonian_energy_error = 0.13691090801812322, max_hamiltonian_energy_error = 0.2237231310927683, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false)  …  (n_steps = 7, is_accept = true, acceptance_rate = 1.0, log_density = -113.96496814885586, hamiltonian_energy = 116.40710379811779, hamiltonian_energy_error = -0.3038412692922776, max_hamiltonian_energy_error = -0.3038412692922776, tree_depth = 2, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 9, is_accept = true, acceptance_rate = 0.5654576076555401, log_density = -114.23633358505055, hamiltonian_energy = 116.42370319084094, hamiltonian_energy_error = -0.2656616386172601, max_hamiltonian_energy_error = 1.6523405717748005, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 31, is_accept = true, acceptance_rate = 0.9561788249075521, log_density = -112.78693523569363, hamiltonian_energy = 115.66062495063785, hamiltonian_energy_error = -0.054339056784357354, max_hamiltonian_energy_error = 0.13966125194133383, tree_depth = 4, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.4745870687756013, log_density = -113.35117347342369, hamiltonian_energy = 118.219292096801, hamiltonian_energy_error = 0.20507421010155724, max_hamiltonian_energy_error = 2.2569718507538568, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 15, is_accept = true, acceptance_rate = 0.8626481106972211, log_density = -113.15302855749525, hamiltonian_energy = 115.93951567672393, hamiltonian_energy_error = -0.02476516707965004, max_hamiltonian_energy_error = 0.30791717028760957, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.8889265897577296, log_density = -113.62017296956759, hamiltonian_energy = 115.42909352260853, hamiltonian_energy_error = 0.1489391572057741, max_hamiltonian_energy_error = 0.21650629250574127, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 23, is_accept = true, acceptance_rate = 0.7809207845204434, log_density = -115.33599126542742, hamiltonian_energy = 117.25520256173627, hamiltonian_energy_error = 0.5505483309000141, max_hamiltonian_energy_error = 0.6150835086544504, tree_depth = 4, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 15, is_accept = true, acceptance_rate = 0.8761724403674196, log_density = -114.87417840127675, hamiltonian_energy = 117.62769588046608, hamiltonian_energy_error = -0.32348141183514656, max_hamiltonian_energy_error = 0.6031762511654648, tree_depth = 3, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.4142124097112424, log_density = -115.22218650284564, hamiltonian_energy = 121.09778018192144, hamiltonian_energy_error = 0.9160616974241549, max_hamiltonian_energy_error = 2.1037257848388435, tree_depth = 2, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false), (n_steps = 31, is_accept = true, acceptance_rate = 0.9365306817532942, log_density = -112.40675554478646, hamiltonian_energy = 115.22900739099408, hamiltonian_energy_error = -1.1903525316027839, max_hamiltonian_energy_error = -1.1973719878102287, tree_depth = 4, numerical_error = false, step_size = 0.5455056329823196, nom_step_size = 0.5455056329823196, is_adapt = false)])

Use Pathfinder's metric estimate for sampling

To use Pathfinder's metric with no metric adaptation, we need to use Pathfinder's own Pathfinder.RankUpdateEuclideanMetric type, which just wraps our inverse metric estimate for use with AdvancedHMC:

nadapts = 75
metric = Pathfinder.RankUpdateEuclideanMetric(inv_metric)
hamiltonian = Hamiltonian(metric, ∇P)
ϵ = find_good_stepsize(hamiltonian, init_params)
integrator = Leapfrog(ϵ)
kernel = HMCKernel(Trajectory{MultinomialTS}(integrator, GeneralisedNoUTurn()))
adaptor = StepSizeAdaptor(0.8, integrator)
samples_ahmc3, stats_ahmc3 = sample(
    hamiltonian,
    kernel,
    init_params,
    ndraws + nadapts,
    adaptor,
    nadapts;
    drop_warmup=true,
    progress=false,
)

([[-0.43387780650465124, 0.10550823918177005, 0.9134794766797871, -0.8165449067177877, 0.5665972466889763], [-0.34512821891639767, 0.053046415550849935, 0.575053852864992, 0.8833751301360624, -0.4239248661974121], [-0.3151649601807399, 0.13399106264096378, 0.050834298821545325, 0.10336182808029981, 0.227369010330464], [-0.3539674491912428, 0.17274483370186283, 0.3113910746395255, -1.477432684298363, 1.0737677294915833], [-0.39520100148919296, 0.43053318860526635, -0.008082431068161589, 0.8533715892763426, -0.2505202123076114], [-0.26138745826121856, 0.10760133812475105, 0.0027313615369467664, 0.6959774160681337, -0.12149944852205757], [-0.3474884030876259, 0.23769632642349509, -1.3202826677938355, -1.3136137652429203, 1.628889357741358], [-0.4168002377914013, 0.2715000136098215, -1.7784754364788935, 0.9848117957213707, 0.12874295211202624], [-0.2713769307451645, 0.2548443782220083, -1.5205329114431831, -1.1442471689794556, 1.6208418008278191], [-0.18100031019515903, 0.2635116300672349, -1.727903221872193, 0.9024925800970207, 0.20130278738211616]  …  [-0.20382267656386233, 0.31362293905875815, -1.8137717785340475, 1.220188133059982, 0.041892718154873165], [-0.25657344852256, 0.09954156381431162, -0.15021928273242496, 0.10612065515778468, 0.353464778513216], [-0.311268544801874, 0.19022057712264023, -0.5887238402832486, -0.8091151836539505, 1.0783700861553098], [-0.2807347028165323, 0.1344857934668664, 0.9340637111744585, -0.27550252938566666, 0.23647098982404452], [-0.2863172676094762, 0.11859439772589456, -0.7936701878470681, -0.6395782700244661, 1.0669546946815278], [-0.2778496118092527, 0.2711713577696458, -0.9491698373814063, 0.006600751049656461, 0.5572095033605413], [-0.31906470456173447, 0.2707097130098051, -0.7008507479617964, -0.548279675745085, 0.8496848170457563], [-0.3440406829307183, 0.19494304535809856, -1.0374405745321977, -0.21090247854577915, 0.7542361814927614], [-0.3600303513110096, 0.18576690120391348, -0.11031787174844841, -0.18182940851275375, 0.4274140021391084], [-0.35542377774169936, 0.43250618075542896, -0.43205567275252504, 1.6814961405336974, -0.7854027186127652]], NamedTuple[(n_steps = 31, is_accept = true, acceptance_rate = 0.9977941860060714, log_density = -114.37608418205954, hamiltonian_energy = 115.49638318729656, hamiltonian_energy_error = -0.0008490302911070557, max_hamiltonian_energy_error = -0.148848432326389, tree_depth = 4, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 3, is_accept = true, acceptance_rate = 0.8151954173491237, log_density = -113.70129519718394, hamiltonian_energy = 115.9050805876096, hamiltonian_energy_error = 0.16123179123813713, max_hamiltonian_energy_error = 0.52004889412909, tree_depth = 2, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 63, is_accept = true, acceptance_rate = 0.9953524446310917, log_density = -112.57957868830503, hamiltonian_energy = 113.83331722812696, hamiltonian_energy_error = -0.23044774507425814, max_hamiltonian_energy_error = -0.33023535627737033, tree_depth = 5, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.8943193072009846, log_density = -118.09797874744575, hamiltonian_energy = 118.41423164106682, hamiltonian_energy_error = 0.2542142670788081, max_hamiltonian_energy_error = 0.2542142670788081, tree_depth = 3, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.8632734149588763, log_density = -119.71887326673846, hamiltonian_energy = 120.56006336208094, hamiltonian_energy_error = 0.5266482078241239, max_hamiltonian_energy_error = 0.5266482078241239, tree_depth = 3, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 1.0, log_density = -113.49017843353876, hamiltonian_energy = 119.67370573741103, hamiltonian_energy_error = -0.46787898668630135, max_hamiltonian_energy_error = -0.7252409522537135, tree_depth = 2, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 27, is_accept = true, acceptance_rate = 0.9457060724160997, log_density = -115.51067451936443, hamiltonian_energy = 117.47565255123699, hamiltonian_energy_error = 0.018367059130042662, max_hamiltonian_energy_error = -0.3712975482010137, tree_depth = 4, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.9371374036726066, log_density = -116.18242929765371, hamiltonian_energy = 121.53728204460762, hamiltonian_energy_error = -0.21910000739521251, max_hamiltonian_energy_error = 0.22174760696991314, tree_depth = 2, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.8939882617478471, log_density = -117.59385848882002, hamiltonian_energy = 118.13153382446374, hamiltonian_energy_error = 0.2592397440838283, max_hamiltonian_energy_error = 0.2592397440838283, tree_depth = 3, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 23, is_accept = true, acceptance_rate = 0.8225424646982363, log_density = -116.91725622842979, hamiltonian_energy = 121.09599538896339, hamiltonian_energy_error = 0.24381590488997062, max_hamiltonian_energy_error = 0.6950651291011383, tree_depth = 4, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false)  …  (n_steps = 23, is_accept = true, acceptance_rate = 0.9282132426188189, log_density = -116.57803326266277, hamiltonian_energy = 120.70403022554282, hamiltonian_energy_error = 0.030299475363264605, max_hamiltonian_energy_error = -0.4801846399895595, tree_depth = 4, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 23, is_accept = true, acceptance_rate = 0.9838368988690165, log_density = -113.65502178511632, hamiltonian_energy = 120.5653409980844, hamiltonian_energy_error = -0.31244006535501967, max_hamiltonian_energy_error = -0.43355428003613383, tree_depth = 4, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 39, is_accept = true, acceptance_rate = 0.9969369795809385, log_density = -113.87038929159162, hamiltonian_energy = 114.83031520653498, hamiltonian_energy_error = -0.009996146558805208, max_hamiltonian_energy_error = -0.1079843821727593, tree_depth = 5, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 47, is_accept = true, acceptance_rate = 0.9996044800451261, log_density = -113.66048202393856, hamiltonian_energy = 115.45501548966476, hamiltonian_energy_error = -0.06405609416687241, max_hamiltonian_energy_error = -0.12120321384813337, tree_depth = 5, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 39, is_accept = true, acceptance_rate = 0.9907392381607445, log_density = -114.35018458372393, hamiltonian_energy = 114.8312458519669, hamiltonian_energy_error = 0.02403240532414941, max_hamiltonian_energy_error = -0.04702538615741503, tree_depth = 5, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.9756653919637278, log_density = -113.25500950232909, hamiltonian_energy = 115.75311206765997, hamiltonian_energy_error = -0.0316866064476784, max_hamiltonian_energy_error = 0.05466999326998234, tree_depth = 3, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.991738207345554, log_density = -112.95367985937521, hamiltonian_energy = 113.92529315483986, hamiltonian_energy_error = -0.005395757488841468, max_hamiltonian_energy_error = -0.042085776406864284, tree_depth = 3, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.7522628842738065, log_density = -113.23987739008999, hamiltonian_energy = 118.55413140042408, hamiltonian_energy_error = 0.04448497989716316, max_hamiltonian_energy_error = 0.403752078499096, tree_depth = 3, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 47, is_accept = true, acceptance_rate = 0.9505934263039489, log_density = -112.60687226583427, hamiltonian_energy = 115.43185709773078, hamiltonian_energy_error = 0.00513786946352468, max_hamiltonian_energy_error = 0.09047593572556423, tree_depth = 5, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false), (n_steps = 7, is_accept = true, acceptance_rate = 0.9488675859133326, log_density = -114.83645961119299, hamiltonian_energy = 115.18675130183698, hamiltonian_energy_error = 0.12763710418535368, max_hamiltonian_energy_error = 0.12763710418535368, tree_depth = 3, numerical_error = false, step_size = 0.45943941557288803, nom_step_size = 0.45943941557288803, is_adapt = false)])