This note is a stopgap demo of how to use a stanli backend with brms until the next version of brms (something newer than 2.23.0) comes out. (brms #1966 added an official stanli backend, and it was merged into the master branch of brms.)

Set up and install stanli:

install.packages("stanli", repos = c("https://seantalts.r-universe.dev"))
library(stanli)
stanli_install()

First, let’s fit a normal model with cmdstanr as a point of comparison for stanli’s performance gains.

The current demo of brms usage with stanli doesn’t feature priors or a custom model family, so we will use a contrived example with both:

library(stanli)
library(brms)
# stanli::stanli_install()

# Make a model from scratch
data <- lme4::sleepstudy
formula <- bf(
  Reaction ~ Days + (Days | Subject), 
  # Just to test a non-default family
  family = student()
)

# Loose basis for some priors
lme4::sleepstudy$Reaction |> mean()
#> [1] 298.5079
lme4::sleepstudy$Reaction |> sd()
#> [1] 56.32876
rexp(1000, .01) |> fivenum()
#> [1]   0.04885372  30.38196331  70.14801274 145.45496673 768.92542630
rexp(1000, .10) |> fivenum()
#> [1]  0.01731279  3.01678405  6.65988896 13.43290706 72.51916389
rgamma(1000, .2, .01) |> fivenum()
#> [1] 1.535653e-14 7.973998e-02 2.084129e+00 1.617510e+01 2.501939e+02

priors <- prior(normal(300, 100), class = Intercept) +
  prior(normal(0, 100), class = b) +
  prior(exponential(0.01), class = sigma) +
  prior(exponential(0.01), class = sd, group = Subject, coef = Intercept) +
  prior(exponential(0.10), class = sd, group = Subject, coef = Days) +
  prior(lkj(2), class = cor)

check <- validate_prior(priors, formula, data)

model_cmdstanr <- brm(
  formula = formula, 
  data = lme4::sleepstudy, 
  prior = priors,
  backend = "cmdstanr", 
  cores = 4, 
  file = "2026-11-17-model1",
  file_refit = "on_change",
)

model_cmdstanr
#> Loading required namespace: rstan
#>  Family: student 
#>   Links: mu = identity 
#> Formula: Reaction ~ Days + (Days | Subject) 
#>    Data: lme4::sleepstudy (Number of observations: 180) 
#>   Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#>          total post-warmup draws = 4000
#> 
#> Multilevel Hyperparameters:
#> ~Subject (Number of levels: 18) 
#>                     Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> sd(Intercept)          32.11      6.91    21.21    48.30 1.00     1663     2165
#> sd(Days)                7.08      1.43     4.80    10.46 1.00     1742     2107
#> cor(Intercept,Days)    -0.05      0.24    -0.50     0.43 1.00     1335     1932
#> 
#> Regression Coefficients:
#>           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> Intercept   250.94      8.22   234.50   267.77 1.00     1353     1955
#> Days         10.72      1.79     7.20    14.20 1.00     1352     1804
#> 
#> Further Distributional Parameters:
#>       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> sigma    15.05      1.84    11.65    18.87 1.00     2632     2397
#> nu        3.36      1.12     1.90     6.19 1.00     3171     2484
#> 
#> Draws were sampled using sample(hmc). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).

stanli is going to run a Stan program for us, so we need to prepare the things that a Stan program needs (code and data). Then we define a model and sample from it:

scode <- stancode(object = formula, data = data, prior = priors)
sdata <- standata(object = formula, data = data, prior = priors)
model <- stanli_model(code = scode, data = sdata)
fit <- sample_model(model, chains = 4, refresh = 0)

Note that there was no compilation.

Now, the current trick is to tell brms to fit the model using a "mock"1 backend.

model_stanli <- brm(
  formula = formula,
  data = data,
  prior = priors,
  backend = "mock",
  mock_fit = as_stanfit(fit)
)
model_stanli
#>  Family: student 
#>   Links: mu = identity 
#> Formula: Reaction ~ Days + (Days | Subject) 
#>    Data: data (Number of observations: 180) 
#>   Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#>          total post-warmup draws = 4000
#> 
#> Multilevel Hyperparameters:
#> ~Subject (Number of levels: 18) 
#>                     Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> sd(Intercept)          31.93      6.87    20.96    47.22 1.00     1533     2203
#> sd(Days)                7.09      1.45     4.81    10.40 1.00     1611     2224
#> cor(Intercept,Days)    -0.07      0.24    -0.51     0.40 1.01     1268     1664
#> 
#> Regression Coefficients:
#>           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> Intercept   251.10      8.18   235.19   267.16 1.00     1381     1934
#> Days         10.73      1.83     7.05    14.28 1.00     1346     1791
#> 
#> Further Distributional Parameters:
#>       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> sigma    15.06      1.86    11.67    18.86 1.00     2243     2574
#> nu        3.38      1.11     1.88     6.18 1.00     3051     2666
#> 
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).

This model should work like a regular brms model:

pp_check(model_stanli)
#> Using 10 posterior draws for ppc type 'dens_overlay' by default.
Posterior predictive density

Posterior predictive density

If we already have a brms-fitted version of the model around, we skip the data and code steps and use update.brmsfit() instead:

model_stanli2 <- update(
  model_cmdstanr,
  backend = "mock",
  mock_fit = as_stanfit(fit),
  recompile = TRUE
)
model_stanli2
#>  Family: student 
#>   Links: mu = identity 
#> Formula: Reaction ~ Days + (Days | Subject) 
#>    Data: lme4::sleepstudy (Number of observations: 180) 
#>   Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#>          total post-warmup draws = 4000
#> 
#> Multilevel Hyperparameters:
#> ~Subject (Number of levels: 18) 
#>                     Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> sd(Intercept)          31.93      6.87    20.96    47.22 1.00     1533     2203
#> sd(Days)                7.09      1.45     4.81    10.40 1.00     1611     2224
#> cor(Intercept,Days)    -0.07      0.24    -0.51     0.40 1.01     1268     1664
#> 
#> Regression Coefficients:
#>           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> Intercept   251.10      8.18   235.19   267.16 1.00     1381     1934
#> Days         10.73      1.83     7.05    14.28 1.00     1346     1791
#> 
#> Further Distributional Parameters:
#>       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> sigma    15.06      1.86    11.67    18.86 1.00     2243     2574
#> nu        3.38      1.11     1.88     6.18 1.00     3051     2666
#> 
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).

We can compare sampling performance. This is a pretty fast model to begin with, so the gains here might be more limited compared to a more complex model.

rstan::get_elapsed_time(model_cmdstanr$fit)
#>         warmup sample
#> chain:1  2.117  0.981
#> chain:2  2.009  1.047
#> chain:3  1.897  1.030
#> chain:4  2.232  1.031

rstan::get_elapsed_time(model_stanli$fit)
#>            warmup    sample
#> chain:1 0.6565742 0.2930656
#> chain:2 0.6128654 0.2871662
#> chain:3 0.6307627 0.2910862
#> chain:4 0.5246784 0.2682132

# If we want to say "n-times faster"
rstan::get_elapsed_time(model_cmdstanr$fit) / 
  rstan::get_elapsed_time(model_stanli$fit)
#>           warmup   sample
#> chain:1 3.224312 3.347373
#> chain:2 3.278044 3.645972
#> chain:3 3.007470 3.538471
#> chain:4 4.254034 3.843957

But don’t forget compile time:

stan_file <- tempfile(fileext = ".stan")
writeLines(scode, stan_file)

compile_time <- system.time({
  model <- cmdstanr::cmdstan_model(
    stan_file,
    force_recompile = TRUE
  )
})
compile_time
#>    user  system elapsed 
#>    0.09    0.06   23.87

cmdstanr_total <- model_cmdstanr$fit |> 
  rstan::get_elapsed_time() |> 
  rowSums() |> 
  max()

stanli_total <- model_stanli$fit |> 
  rstan::get_elapsed_time() |> 
  rowSums() |> 
  max()
  
(cmdstanr_total + unname(compile_time[3])) / stanli_total 
#> [1] 28.57189
  1. In software testing, we might define a “mock” version of a computation, like a file download, where we skip the actual and maybe unpredictable work and load up a prepared result. brms can test model preparation by skipping model compilation and skipping sampling with a mock backend and loading in some prepared MCMC samples. That’s how we can understand what this code is doing. ↩

Leave a comment