I'm a Chordata! Urochordata!

Exploring RTMB with Palmer Penguins: Linear Regression

Jarrett Byrnes — Sat, 07 Nov 2026 05:00:00 GMT

I was deeply intrigued to see Kristensen et al. (2026) in Ecology and Evolution the other day. With a title like RTMB: An R Package for Fitting Latent Variable Models With Template Model Builder how could I not be intrigued? Latent variables and possibility of SEM? Check! TMB? Well, I’m a regular user of glmmTMB and sdmTMB - the later of which has just moved over to using RTMB at its core. So, what indeed is RTMB and how does it work?

At its core, RTMB is a way to use TMB and it’s AD “machinery” without having to code in C++ and compile your model. Seems great! You make an objective function - kinda just like you did if you are using optim() or nlminb() and then use MakeADFun() to wrap it all up so that it can be fed to one of those optimizers to fit.

There are a few other things you want to do inside of the function. Use getAll() so you don’t have to do a lot of typing with a $. Flag the observed variable using OBS(). Use REPORT() to get an object you want to see after fitting without uncertainty. Use ADREPORT() to get a derived quantity for which you DO want to get some uncertainty. There are other bits and pieces, but, let’s look at this in context using palmerpenguins. Or, now that its in base R, penguins - haha!

A Simple Linear Regression model

Let’s start by loading in our data and, for the moment, getting rid of NAs.

library(RTMB)
library(dplyr) # to make our lives easier

penguins_clean <- na.omit(penguins) # get rid of NAs

Now, what’s our model? Let’s look at bill_len ~ body_mass as a linear regression. If we were just using lm, this would be

penguin_lm <- lm(bill_len ~ body_mass, data = penguins_clean)

This is always lovely, as we don’t really have to think about what is going on under the hood. Not so with RTMB. We actually define our model mathematically and estimate parameters. So, to remind you, a linear regression model looks like this:

$$ = _0 + _1 x_i$$ $$y_i (, )$$

To start, we have to actually create a list of parameters as well as some start values. Let’s not use betas and sigmas from above, but give them more meaningful names.

linreg_params <- list(intercept = 0,     # starting guesses
              body_mass_slope = 0,
              logsigma = log(sd(penguins_clean$bill_len))) #seems reasonable if all was flat!

Note we’re doing log sigma. That will make things easier to fit.

Now, we define our model with an objective function. We are going to actually have the linear model, the likelihood (the normal distribution for residuals), and then turn that into a negative log-likelihood. Well, ok, we’re going to use dnorm() so we’ll set log=TRUE to start out with the log-likelihood, as that’s good practice for rounding, etc. We’ll make a function with one argument - p. What is that? Well, p is a list of parameter values - the same one we just made! And we’ll use getAll() so we don’t need to keep typing p$ or penguins_clean$ throughout.

Supply one argument - p - which is a list of our parameter values. We’ll then use getAll() in order to not have to use p$ or penguins_clean$ in the rest of the function.
We will calculate sigmas as exp(logsigma) and then use ADREPORT(sigma) so that we can see it in our coefficient output. If we wanted to get simulations of it as well, we would include REPORT(sigma) for later use with the simulate() function
One funny thing is that we will use bill_len <- OBS(bill_len) - this allows us to define our observed variable, which will then be used in things like calculating residuals, simulations, and more.
Do you want fitted values? You have to define them and get them with ADREPORT() as well! Which is fine - and menas you can do all sorts of tricksy things - particularly if you want link and response values, etc.

linreg <- function(p){
  getAll(penguins_clean, p) # so we can stop using $
  
  # a transformed parameter
  sigma <- exp(logsigma)
  ADREPORT(sigma) #so we can see it later
  #REPORT(sigma) #for simulation
  
  # What are we observing for later sims
  bill_len <- OBS(bill_len)
  
  # model for DGP and output those fitted values
  bill_len_hat <- intercept + body_mass_slope * body_mass
  ADREPORT(bill_len_hat)
  
  # EGP
  pointwise_ll <- dnorm(bill_len, 
              mean = bill_len_hat, 
              sd = exp(logsigma), log = TRUE)

  # turn that into a negative log-likelihood
  nll <- -1 * sum(pointwise_ll)

  # return
  nll
}

Cool! Now we need to turn it into an TMB object. Note, we’ll use silent = TRUE to supress output from our fitting function.

linreg_obj <- MakeADFun(linreg, 
                        linreg_params, 
                        silent = TRUE)

Last, we fit it. Note, linreg_obj has a few parts - a list of parameters, an objective function, and a gradient function. We could call them individually in a call to optim or something, but, easier to use do.call().

linreg_out <- do.call("optim", linreg_obj)

Regression Output

OK, you wanna see the output? The big thing is that, it’s actually out object - not the fit from optim - that has all of the info in it. So, let’s see what we have! Note, I’m using some row notation, as otherwise, we get ALL of the fitted values and their SE as well.

# note this is obj and not out
linreg_coefs <- sdreport(linreg_obj) 
linreg_coefs

sdreport(.) result
                    Estimate   Std. Error
intercept       27.150733601 1.2877182107
body_mass_slope  0.004003285 0.0003006444
logsigma         1.484095254 0.0387491858
Maximum gradient component: 0.7064614

# If we wanted to see the fit values as well
# uncomment and run
#summary(linreg_coefs)

# to see the values including what comes out with ADREPORT
summary(linreg_coefs)[1:4,]

                    Estimate   Std. Error
intercept       27.150733601 1.2877182107
body_mass_slope  0.004003285 0.0003006444
logsigma         1.484095254 0.0387491858
sigma            4.410972798 0.1709216045

Nice! Look at the names of that coefs object - there’s a lot of info, like the VCOV matrix!

names(linreg_coefs)

[1] "value"          "sd"             "cov"            "par.fixed"     
[5] "cov.fixed"      "pdHess"         "gradient.fixed" "env"

linreg_coefs$cov.fixed

                    intercept body_mass_slope      logsigma
intercept        1.658218e+00   -3.802634e-04  3.484112e-08
body_mass_slope -3.802634e-04    9.038703e-08 -1.474645e-11
logsigma         3.484112e-08   -1.474645e-11  1.501499e-03

Model checking

First, did our model do a good job?

checkConsistency(linreg_obj)

Parameters used for simulation:
      intercept body_mass_slope        logsigma 
   27.150733601     0.004003285     1.484095254 

Test correct simulation (p.value):
[1] 0.9421078
Simulation appears to be correct

Estimated parameter bias:
      intercept body_mass_slope        logsigma 
   7.477649e-02   -1.519087e-05    7.237122e-04

What about model assumptions? To look at that, we want residuals, which we get via oneStepPredict().

res <- oneStepPredict(linreg_obj, method = "fullGaussian")

density(res$residual) |> plot()

qqnorm(res$residual)
qqline(res$residual)

Not bad, not great - but we know why! We’re not looking at species.

boxplot(res ~ species,
        data.frame(res = res$residual,
                   species = penguins_clean$species))

What about the fitted versus observed? For that, we need the fit values, which we have in our sdreport. If we look at the values in the object, we can see which of the derived parameters are coefficients and which are data. They go in order defined in the model.

names(linreg_coefs$value[1:5])

[1] "sigma"        "bill_len_hat" "bill_len_hat" "bill_len_hat" "bill_len_hat"

fitted <- linreg_coefs$value[2:length(linreg_coefs$value)]

So now we can do something like a fitted versus residuals plot.

plot(fitted, res$residual)

We could get fancy and throw this into ggplot2 and color by species, or all manner of other things. The world is our oyster.

Plot Model Fit and Fit Interval

What’s nice about this is that we can plot our data and our fit quite easily given what we have generated. We can do this using the outputs of fits from our ADREPORT() values of fitted values.

We know that summary(linreg_coefs) will give us our coefficients and fitted values. So, we can just remove the first four rows (intercept, slope, logsima, and sigma), and then use the estimate and SE to generate CIS.

fit_frame <- summary(linreg_coefs)[-c(1:4),] |>

  as_tibble() |>
  
  # clean those names!
  janitor::clean_names() |>

  # calculate CIs and make sure to add your X
  mutate(lower_ci = estimate - 1.96*std_error,
         upper_ci = estimate + 1.96*std_error,
         body_mass = penguins_clean$body_mass) |>
  
  # do a little arrange to make the ribbon look nice
  arrange(body_mass)


with(fit_frame,
     {
       plot(penguins_clean$body_mass,
             penguins_clean$bill_len,
            xlab = "body mass(g)", 
            ylab = "bill length (mm)")
       
       polygon(x = c(body_mass, rev(body_mass)),
             y = c(lower_ci, rev(upper_ci)),
             col = "lightgrey")
       
       lines(body_mass, estimate, col = "red")
       
       points(penguins_clean$body_mass, 
              penguins_clean$bill_len)
       
       }
)

Yes, you can do this in ggplot2, but, I’m feeling saucy and using base.

Plot Model Fit and Prediction Interval

What if we want to look at the interval for new predictions. This is harder, and can depend a lot on some details of the probability distributions we’re dealing with. So, what about simulation?

linreg_obj$simulate() will give us new simulated points for each pointwise prediction. This can be really useful if we replicate it, as it can give us means, CIs, and even let us do things like quantile residuals for DHARMA plots. Here, let’s use it as above to get means and PIs.

Fist, the simulated values. I’ll use replicate(). Note, we also have to draw out a variable. That’s right - we could have simulated out a LOT of things based on OBS() and REPORT().

sim_matrix <- replicate(1e3, linreg_obj$simulate()$bill_len)

dim(sim_matrix)

[1]  333 1000

To see that these really are what we want, let’s compare the means of the sims to our fitted values.

plot(fitted, rowMeans(sim_matrix))

Good for 1K sims, no? Now, let’s use this matrix to generate something like our prediction data frame above.

pis <- apply(sim_matrix, 1, quantile, c(0.025, 0.975))

sim_frame <- tibble(
  body_mass = penguins_clean$body_mass,
  bill_len_fit = fitted,
  bill_len_raw = penguins_clean$bill_len,
  lower_pi = pis[1,],
  upper_pi = pis[2,]
) |>
  arrange(body_mass)

head(sim_frame)

# A tibble: 6 × 5
  body_mass bill_len_fit bill_len_raw lower_pi upper_pi
                              
1      2700         38.0         46.9     29.7     46.2
2      2850         38.6         36.5     29.9     47.3
3      2850         38.6         36.4     30.5     47.8
4      2900         38.8         34.5     30.1     47.1
5      2900         38.8         33.1     30.5     47.1
6      2900         38.8         38.6     30.6     46.9

Fantastic. Now the plot

with(sim_frame,
     {
       plot(body_mass,
             bill_len_raw,
            xlab = "body mass(g)", 
            ylab = "bill length (mm)")
       
       polygon(x = c(body_mass, rev(body_mass)),
             y = c(lower_pi, rev(upper_pi)),
             col = "lightgrey")
       
       lines(body_mass, bill_len_fit, col = "red")
       
       points(body_mass, 
              bill_len_raw)
       
       }
)

BONUS Here’s how to get quantile resudials!

qres <- numeric(length = length(fitted))

for(i in 1:length(fitted)){
  qres[i] <- 
    sum(sim_matrix[i,] < penguins_clean$bill_len[i]) / ncol(sim_matrix)  
}

hist(qres)

gap::qqunif(qres, logscale = FALSE, col = "black")

Or, simpler

qres <- DHARMa::getQuantile(sim_matrix, penguins_clean$bill_len)

hist(qres)

gap::qqunif(qres, logscale = FALSE, col = "black")

Note, both of these have problems. Why? SPECIES!

Wait, but what if I wanted to get fit intervals via simulation?

You can! You would use exactly the code above. The difference is that, in your model, you would need to include a line that is REPORT(bill_len_hat) so that, when you simulate, it comes out as well. That’s it! Then just use bill_len_hat in the above code for prediction intervals instead of bill_len. Oh, and you should probably change any variable name with pi* in it to ci*!

Why might you want to do this other than simulation is cool? Well, **NOTE: The PIs above did not incorporate fit error - they do not incorporate the CI of sigma as well - but, if you REPORT(sigma) you can get uncertainty there, and then use make draws of noise to add to simulated hat values. Gelman and Hill cover this a bit in their classic book and it’s accompanying arm package with its sim() function. Worth looking up, and long one of my biggest kvetches about simulate() (I even got shirty about it and started to write a package sinterval that was meant to produce results much like tidybayes but, well, there are only so many hours in a day!)

Plot New Data (counterfactuals!)

What if we want to use our fit model to do wholly different predictions on new values? We’re used to fitting a model and then using a function with a new data frame of our choice to generate new values. It’s not quite as simple here - although you can write a function to do something like that.

The short version is

We’re gonna need to rewrite our function so that it takes in data - and some of that data will be new!
New rows will merely have NA values for the response variable - which we will then exclude at key points.
We can use the Hessian from the initial fit model for use with new values to avoid refitting.

A Function for new values

Let’s start by re-writing the objective function. We will now include a data argument. We will also check the data to see what values are new, what are observed, and use that information moving forward for our likelihood and for reporting. Note, I’m also going to introduce the %~% syntax for probability distributions. It matches other probailistic programing languages, and allows you to stop thinking about remembering log=TRUE and -1* and all that. We will also use it for random effects, latent variables, and other things - so, best to start working with it now.

linreg_p <- function(p, data){
  getAll(data, p) # so we can stop using $
  
  # a transformed parameter
  sigma <- exp(logsigma)
  ADREPORT(sigma) #so we can see it later
  REPORT(sigma) # for simulation

  # What are we observing for later sims
  bill_len <- OBS(bill_len)
  
  # model for DGP and output those fitted values
  bill_len_hat <- intercept + body_mass_slope * body_mass
  ADREPORT(bill_len_hat)
  REPORT(bill_len_hat) #for fit interval simulations
  
  # new predictions
  new_idx <- is.na(bill_len) # the indices of new values
  obs_idx <- !new_idx

  # Likelihood - but for obs only!
  bill_len[obs_idx] %~% dnorm(bill_len_hat[obs_idx],
                             sd = sigma)
}

Not super different, no?

Fitting the first time - now with data!

OK, we need to fit our model. We need to add the data to the call to MakeADFun() for fitting the model. Looking at the helpfile for MakeADFun() we see that the first argument is a function - our objective function! The second argument is the parameters, which is feeds to the function. So, we need a function to… make a function! In the docs for RTMB, they have a nice little one called cmb() which I implement here. But, I dunno, maybe the authors want to add a data argument to MakeADFun()?

cmb <- function(f, d) function(p) f(p, d)

linreg_obj_p <- cmb(linreg_p, penguins_clean) |>
  MakeADFun(linreg_params, 
                 silent = TRUE)

Yeah, I piped it, but, meh. Now let’s fit the thing. We’ll use nlminb() to show something different than the do.call("optim", mod) approach above, as who knows what optimizer you want to use! TO show you, let’s see what’s inside that we’ll need to use with names()

names(linreg_obj_p)

 [1] "par"          "fn"           "gr"           "he"           "hessian"     
 [6] "method"       "retape"       "env"          "report"       "simulate"    
[11] "force.update" "vectorize"

linreg_p_fit <- with(linreg_obj_p,
     nlminb(par, fn, gr))

# did it blend?
linreg_p_fit$message

[1] "relative convergence (4)"

Looks like it fit! And to convince you this has changed nothing…let’s compare it to our previous fit.

sdreport(linreg_obj)

sdreport(.) result
                    Estimate   Std. Error
intercept       27.150733601 1.2877182107
body_mass_slope  0.004003285 0.0003006444
logsigma         1.484095254 0.0387491858
Maximum gradient component: 0.7064614

sdreport(linreg_obj_p)

sdreport(.) result
                    Estimate   Std. Error
intercept       27.150726341 1.2877185869
body_mass_slope  0.004003289 0.0003006444
logsigma         1.484095547 0.0387491971
Maximum gradient component: 0.07012756

Predictions using our Hessian without no aggression

Sorry, I could not resist.

For those unfamiliar (and many applied scientists are), the Hessian matrix of a model fit is the square matrix of second-order partial derivatives of the log-likelihood with respect to the parameters. Inverting it gives us an estimate of the covariance matrix, and it is used for much more. Basically, we can think of it as the raw information about our fit model, and using it will allow us to generate the CIs and PIs on our new predictions.

To get new predictions, we need to 1. Make our new data frame for prediction with blank values for bill_len.

Get the fit parameters and fit hessian.
Use the new data, our model, our cmb function from above, and our fit parameters with MakeADFun().
Use sdreport() with fixed.hessian = fit_hessian to get the proper SE of our predictions. For ease of use, I’m going to supply some more processing workflow to make a nice new prediction data frame.

# new data to predict from
pred_data <- data.frame(
  body_mass = seq(min(penguins_clean$body_mass),
                   max(penguins_clean$body_mass),
                   length.out = 20),
  bill_len = NA
)

# extract parameters and Hessian
fit_params <- linreg_p_fit$par
fit_hessian <- linreg_obj_p$he()

# make a new ADFun
pred_adfun <- MakeADFun(cmb(linreg_p, pred_data),
                       as.list(fit_params), 
                       silent = TRUE)

# get the new predictions and their SEs
pred_values <- sdreport(pred_adfun,
                        hessian.fixed = fit_hessian) |>
  summary() |>
  as_tibble() |>
  janitor::clean_names() |>
  slice(-c(1:4)) |>
  bind_cols(pred_data) |>
  select(-bill_len) |>
  rename(bill_len = estimate) |>
  mutate(ci_low = bill_len - 1.96 * std_error,
         ci_high = bill_len + 1.96 * std_error) |>
  select(body_mass, everything()) |>
  arrange(body_mass)

# take a look!
head(pred_values)

# A tibble: 6 × 5
  body_mass bill_len std_error ci_low ci_high
                    
1     2700      38.0     0.513   37.0    39.0
2     2889.     38.7     0.464   37.8    39.6
3     3079.     39.5     0.416   38.7    40.3
4     3268.     40.2     0.371   39.5    41.0
5     3458.     41.0     0.330   40.3    41.6
6     3647.     41.8     0.294   41.2    42.3

Did it blend? Let’s visualize this with our fit line and CI to see where it falls in.

# First, our original fit - we already did this above.
with(fit_frame,
     {
       plot(penguins_clean$body_mass,
             penguins_clean$bill_len,
            xlab = "body mass(g)", 
            ylab = "bill length (mm)",
            type = "n") #blank plot
       
       polygon(x = c(body_mass, rev(body_mass)),
             y = c(lower_ci, rev(upper_ci)),
             col = "lightgrey",
             border = NA)
       
       lines(body_mass, estimate, col = "red")
       
       
       }
)


#Now, let's add our new points and their CIs!

with(pred_values,
     {
       segments(body_mass, ci_low, body_mass, ci_high)
       points(body_mass, bill_len)
       
     })

Yup, right on the prediction line, as they should be, with the correct CI. So, you could wrap the workflow above in a function, for example, and use it to generate a fit curve that is more complex for plotting. You could also use simualte() from the model if you wanted to get draws of new predicted values and knock yourself out there as well - and if you reported the hat values and sigma, you can even get the full prediction interval. The sky is the limit.

Well that was wild. Why did I do this?

That was A LOT. If you want to fit a simple linear regression, should you use this framework? NO! Come on. We have lives to live, dinner to cook, cats to pet.

BUT - what this does do is give you access to the ability to fit complex likelihood-based models, even with latent variables, easily. And quickly! Without leaving R! Working through more familiar models are a nice way to get into the guts of how to use this framework - which is SUPER powerful - before yo have to jump into something a bit more difficult.

It’s also not escaped my attention that this is a nice halfway house to going Bayesian. You can roll this into Stan with tmbstan - see here for how to do it. Or, the coding here is not too too different from rethinking which is a gateway into full blown Stan.

The biggest thing, you have a LOT of latitude. So don’t dream it. Be it.

References

Kristensen, Kasper, Anders Nielsen, Christoffer Moesgaard Albertsen, Casper Willestofte Berg, and Vanessa Trijoulet. 2026. “RTMB: An r Package for Fitting Latent Variable Models with Template Model Builder.” Ecology and Evolution 16 (11): e74572. https://doi.org/https://doi.org/10.1002/ece3.74572.

Modelbased for Quick and Beautiful Model Visualization

Jarrett Byrnes — Fri, 25 Jul 2025 04:00:00 GMT

A year ago, I found the modelbased package while looking for tools to visualize some mixed models. It was a minor revelation. Just like many of the packages from the easystats family.

I teach students to simulate the data they want for data viz (yay broom::augment() and modelr::data_grid()), and then plot that in combination with raw data (sometimes). This is great for really detailed nit-picky figures. But, particularly for students just learning, or for a quick-and-dirty data viz of a fit multivariate model, I always wished there was a simpler way.

modelbased and its estimate_relation() function is an amazing one-stop shop for just this (and more!)

To see it in action, let’s fit a model with the Palmer Penguins data set (now in base R!) and see what this function does. Here, we’ll have look at how mass influences bill length and let this relationship vary by species.

library(modelbased)
library(ggplot2)

mod <- lm(bill_len ~ species*body_mass,
          data = penguins)

estimate_relation(mod)

Model-based Predictions

species   | body_mass | Predicted |   SE |         95% CI
---------------------------------------------------------
Chinstrap |      2700 |     44.22 | 0.84 | [42.57, 45.88]
Adelie    |      3100 |     36.88 | 0.32 | [36.24, 37.51]
Chinstrap |      3100 |     46.01 | 0.56 | [44.90, 47.12]
Adelie    |      3500 |     38.15 | 0.21 | [37.73, 38.57]
Chinstrap |      3500 |     47.79 | 0.34 | [47.12, 48.46]
Gentoo    |      3500 |     41.06 | 0.71 | [39.66, 42.46]
Adelie    |      3900 |     39.43 | 0.21 | [39.01, 39.85]
Chinstrap |      3900 |     49.58 | 0.32 | [48.95, 50.20]
Gentoo    |      3900 |     42.69 | 0.55 | [41.61, 43.78]
Adelie    |      4300 |     40.70 | 0.32 | [40.07, 41.34]
Chinstrap |      4300 |     51.36 | 0.52 | [50.34, 52.39]
Gentoo    |      4300 |     44.33 | 0.40 | [43.55, 45.11]
Adelie    |      4700 |     41.98 | 0.47 | [41.05, 42.90]
Chinstrap |      4700 |     53.15 | 0.79 | [51.59, 54.71]
Gentoo    |      4700 |     45.97 | 0.27 | [45.44, 46.50]
Gentoo    |      5100 |     47.60 | 0.22 | [47.18, 48.03]
Gentoo    |      5500 |     49.24 | 0.28 | [48.68, 49.80]
Gentoo    |      5900 |     50.88 | 0.42 | [50.06, 51.69]
Gentoo    |      6300 |     52.51 | 0.57 | [51.39, 53.63]

Variable predicted: bill_len
Predictors modulated: species, body_mass

This is a really nice visualization of a data table of predicted values. Even better, though, is we can pipe this object to plot, and…

# basic plotting behavior
estimate_relation(mod) |> plot()

Interesting. We’re looking at both predictors and summary output by category for a range of different values of body mass. Neat, but not quite what I want. Fortunately there is the by argument for estimate_relation() which tells what predictors to use (if you have, say, 10), holding others at their mean value in order to marginalize. For example:

# show what's happening
estimate_relation(mod,
                  by = "species")

Model-based Predictions

species   | Predicted |   SE |         95% CI
---------------------------------------------
Adelie    |     40.39 | 0.29 | [39.82, 40.96]
Chinstrap |     50.93 | 0.46 | [50.02, 51.83]
Gentoo    |     43.93 | 0.43 | [43.07, 44.78]

Variable predicted: bill_len
Predictors modulated: species
Predictors controlled: body_mass (4.2e+03)

# plot it
estimate_relation(mod,
                  by = "species") |> 
  plot()

Nice, no? And the plot above is a ggplot, so you can scale and theme to your heart’s delight. More on that below.

But what if I want to show the relationship between mass and bill length - my goal after all - with different lines for different species? It all depends on the order you supply variables to the by argument.

I’ll also make an object so I can stop typing estimate_relation() in the future.

# so we can type less
mod_results <- estimate_relation(mod,
                                 by = c("body_mass", 
                                        "species"))


plot(mod_results)

This is great! Just what we want!

Well. Almost.

What if we want to customize? We can of course set scales and themes and just add them on - this is a ggplot2 object after all. But what about the data?

We can add an argument to plot() to show it - show_data = TRUE. Let’s do that, and add some more pizzaz. I’m going to do that using some themes from see, which is the viz package for many of the outputs from easystats packages (it powers all of those great visualizations from performance::check_model() for example).

# add data
# add a scale
plot(mod_results, 
     show_data = TRUE) +
  see::scale_color_metro() +
  see::scale_fill_metro()

Beautiful!

But what other arguments can I give to plot()?

This is one place where the documentation for modelbased currently falls a bit short. I’d love a vignette that goes through all of the ways to customize these plots. You can get it, however, if you search around and find the documentation for creating what is called a visualization_recipe() - particularly for the kinds of objects here

class(mod_results)

[1] "estimate_expectation"   "estimate_predicted"     "see_estimate_predicted"
[4] "data.frame"

So we can look at visualisation_recipe.estimate_predicted(). There, we have a number of arguments we can customize - point, line, pointrange, ribbon, facet, and grid - each using a list() to supply geoms and aesthetic arguments for each feature of the plot. We can also supply "none" for things we do not want.

For example, let’s make all lines black.

# make all lines black
plot(mod_results,
     line = list(color = "black"))

We see to line we supply a list where we give the aesthetic color as black. We can’t remap things here (at least, I haven’t been able to figure out how if we can), but we can give fixed aesthetics.

For example, let’s do three things

remove the CI ribbon
make the lines have a size of 1
add the data

# remove ribbon but add data
plot(mod_results,
     ribbon = "none",
     line = list(size = 1),
     show_data = TRUE)

Lovely!

Last - what underlies the predictions here? These prediction lines are great, but, how are they generated? What if I don’t want my lines limited to

In short, modelbased uses a insight::get_datagrid() to generate all combinations of relevant variables. We can use arguments for this function in estimate_relation() to futz with what data is used for predictions.

Let’s see what the default looks like by changing the line geom to a points.

# data grid
plot(mod_results,
       line = list(geom = "point"))

OK - we can see those lines are made up of 6 points each. We can change this with a length argument.

estimate_relation(mod,
                  by = c("body_mass", 
                         "species"),
                  length = 30) |>
  plot(line = list(geom = "point"))

You can see how this would be useful for wild nonlinear models - glms, gams, and more. More points will give a smoother curve.

We can do more using insight::data_grid() arguments. If we want to, for example, show the curves where each line is not limited to the range of the data for each species, but instead have one big common range, we can use preserve_range = FALSE.

estimate_relation(mod,
                  by = c("body_mass", 
                         "species"),
                  preserve_range = FALSE) |>
  plot(line = list(geom = "point"))

I’m sure there are other options I’m not covering here. And there’s even more fun to be had with mixed models. Let me know what else you would like to see, and I’ll try and explore the package a bit more!

Reflections on Two Weeks Using Mastodon

Jarrett Byrnes — Sat, 19 Nov 2022 05:00:00 GMT

Welp. It’s another social-media-pocalypse, folk. And I, for one, am kinda here for it?

Sometimes, I think my generation (Generation Oregon Trail!) really is the social network generation. We were here for the rise of social networking, the early networks that worked and failed (I mourn you, Orkut), and experienced the, oh, bi-annual need to move from one network to the next in the mid-late 2000s. Twitter has been one of the longest most stable social networks we’ve been on, and I could write reams about why I think it’s done a lot of good (and no small amount of evil).

As many of us seek a way out thanks to El Musko, one of the first options to spring up has been mastodon. I remember when it launched, I looked at it, and promptly ran away. But, now my urge to move over is much higher, so, I tried again, and found it much improved, and…. I think it could be the replacement? It’s different, but, well, here are my impressions/commentary after 2 weeks on the fossilized pachyderm. I’d do one ups and one downs, alternating to keep a balanced tone.

I’m also open to other folks’ comments (about things I might not notice given that I’ve lived about 20 years now on different social media platforms)

One Up

Social Media at a Slower Pace - Because of the lack of an algorithm pushing for virality, the pace of posts coming through on my home feed is, honestly, slower. There is more time between new posts. More time to read and think. Because my network has been pared down, there’s fewer voices. It’s…. kinda nice, actually. The conversation become more meaningful and intentional. The reposts from people in my network are, honestly, more interesting. With this slowness, I actually find the whole platform less anxiety inducing.

One Down

Oh god, you have to chose a server! This is the #1 complaint I see that makes people think Mastodon is super wonky tech-people only. But, well, yeah, you do need to choose. In the long run, it won’t matter, but it will shape your early experience. That’s because Mastodon has no one centralized server. Indeed, with loads of people joining some of the most obviously names servers (e.g., mastodon.social), we’re seeing some real fails as a small team of open source server admins are being tasked with moderating in a way they never anticipated before. But, if you choose a server before you start on your Mastodon journey, things will much more easily? How to find one? Well, there are some out of date themed lists, but, honestly, google whatever niche you want to find yourself in and “mastodon server” and you’re ready to go. Like, “ecology mastodon server” finds you ecoevo.social. “foodie mastodon server” finds you fearlessfoodies.org. “romance mastodon server” finds you romancelandia.club (gotta follow Olivia Waite somewhere!). And if you realize you don’t like your server, migrate - it’s not too hard! I actually chose the wrong server first, then migrated to ecoevo.social a few days later.

One Up

Tags for Search - A lot has been written about a lack of open text search. But, open text search can be a bit of a mess. Every time I try and do that on the bird site, I get a lot of material I don’t want. There’s something empowering about allowing folk to really control how and where their content can be searched for. It also ensures that when I search for a tag, I’m more likely to find content I want. It’s a win-win, and I’m so far enjoying it as a way of curating my search experience in a way I honestly had not anticipated.

One Down

How do I find people? I think we’ve settled into Twitter so comfortably, we’ve forgotten those initial days of joining when we had, like, 2 followers, and weren’t sure how to find more save search. Well, search is a bit janky, as it’s actually designed to not have open text searching as a harassment feature. Otherwise, you can search for a topic you hate, find the accounts, and pile on. Which…. I get. But also annoying. Fortunately, there are a few tools that make this easier. First, as your network moves over, if folk put their mastodon ID (e.g., I’m @jebyrnes@ecoevo.social) into their twitter bio/account name, you can use tools like fedifinder to get a CSV of folk you follow, and then directly upload them into mastodon. About 15% of my followers have done this, and, honestly, it was a game changer in terms of my experience. That, and if you have interests outside of your main ones, you can find the server that caters through them, browse through some of the users there, and then find new folk to follow. You’ll get a more random sample even, which can be fun!

One Up

Focus on What I Actually Want - Alongside the lack of an algorithm dredging up what it thinks I should see, the content I get is more… actually representative of the network of followers I’ve chosen to see. The ratio of science:internet id has changed dramatically (as much as I love checking out the id of the internet). Honestly, it feels much more like the days of science blogging/early science Twitter of yore. The conversations about problems in academia writ large are there. But so is so much pure science - conversations about ideas, new results, things that you might want to get to a broader audience. I’ve been seeing less and less of that on the bird site over the past few years. The fact that my feed is now just posts in sequential order of folk that I follow - it’s what I actually want, and who I want to see. I’m loving it.

One Down

But I want to browse the id of the internet! OK, the algorithm serves us a LOT of stuff that we don’t even follow, and gives a false sense that you are somehow browsing the collective gestalt of humanity on the internet. But. Eh. No. It’s just the id that the algorithm wants you to see. Frankly, choosing a server that more or less hits your sweet spot will do the same thing for you via its Federated timeline. It’s what? OK, so, on Mastodon, you can see your peeps in your Home timeline. You can see everyone on your server in your Local timeline (which is why it’s nice to pick a server that suits your main interests). Your Federated timeline shows everyone on your server, everyone they follow, and every boosted (think RT-ed) post they make. See here for a fabulous graphic that explains. So, choose a server, see its id. And not just what the algorithm thinks the id you should be seeing is.

Images from @cassoltl@queer.party that explains how mastodon timelines work.

One Up

Servers as Community - I really really like the fact that my server has a local timeline. I came to it for Ecology and Evolutionary Biology - it’s my niche! But, I don’t want to follow everyone on it! I want some timeline curation. Having the ability to hop to a local timeline and see posts of only users who have the same micro-niche as me - it’s nice! I see things to boost, people to follow, and just get bathed in the warm glowing warming glow of EEB. I know at bigger servers, this can be more chaotic, but at smaller niche-i-er servers, I’m guessing the experience is the same.

One Down

Following the followers of people I follow is needlessly hard One way we get new folk to follow is browsing through the follow lists of folk we already follow. In twitter, it’s just a click away. In Mastodon, you often have a popup, or link you have to click, if those people are on another server, and then you can follow them. It’s 1-2 more steps. And annoying as heck. I know, I know, again, an anti-harassment tool, but, is it? If you’re going to harass someone, you’ll make the extra clicks. So, thumbs down to the friction here. It’s needless.

One Up

CWs as “Subject Lines” - Much has been written about the heavy use of the Content Warning feature on Mastodon. It has good and bad sides. On the ‘bad’ side, I’ve seen a lot of - particularly older users - harrangue BIPOC and others about putting those people’s own personal experiences under a CW. This is obnoxious and bigoted behavior - an attempt to force people to hide their own lived experience on the platform. I’m hoping the pushback that is getting will but a stop to it.

On the ‘good’ and interesting use of CWs, aside from more recognized uses of putting potentially triggering content in a place where not everyone has to see, CWs allow a ‘subject line’ of sorts in order to tag content for people’s feed. I’ve been using ‘statistical mishegas’ a lot for fairly esoteric posts that, honestly, might not be interesting to everyone. It’s a ‘here be dragons’ opportunity of sorts. I really like it, and am much more likely to engage with smart CWs than I would have if it was just a random post scrolling by. I really enjoy it!

One Down

I can’t make lists! This is my complaint, actually. I liked making lists of people who I didn’t follow (so they weren’t in my home TL), but I could still look at via lists. For anti-harassment reasons, Mastodon doesn’t allow this, which I was miffed at and tooted about a lot. But….as I’ve read more, yeah, it makes sense. Someone can’t whip together a list of ‘targets’ that no one follows, but is an easy portal to go to in order to harrass others. So, instead, if you really want to have micro-timelines, just follow ’em all, and make better lists. You can use the advanced web interface in the desktop client or the lists on whatever app you use instead of your Home timeline firehose, and, it’s…. just fine. After getting over myself and my own annoyance, I’m now back to the same social media flow I had before.
I can’t quote-tweet Yeah, I kinda don’t like this one, as it makes crediting and commentary harder. Again, it’s for anti-harrassment reasons - you can’t add negative commentary to something and then direct your followers to go over and harrass the poster of the original toot. But. I even see a lot of folk from marginalized communities complaining about this in terms of giving credit and commentary. So, I’m 50:50 on this.

One Up

Server Bans - I lke the idea that, if there is a server hosting malicious actors with light to no moderation, the server itself can be defederated. Mmoderation is challenging and requires humans to undertake a thankless but essential task. But it happens, and bad actors in the Fediverse have the ability to pop up like a game of whack-a-mole until they find somewhere that no one cares. This option takes care of that. Heck, there’s almost been one or two high-profile instances until server moderators cleaned up their acts. I like this feature if used well.

One Down

The old guard on mastodon are jerks about ‘community norms’ and even harass BIPOC about it despite saying they’re trying to make a better social network No argument here. As the wave has come over, a lot of old school Mastodon folk have mourned what was for them. A small space - almost artisinal - in which they spent their time. A slow moving federated timeline. We’re just a big party that has crashed into their nice garden. I empathize. It must be jarring. But. This space wasn’t built as a walled garden - it was built as an alternative ot Twitter. So…. it’s becoming that? Community norms grow and change with the community. And many norms are inherently exclusionary instead of inclusive - something that often isn’t seen until the community size is large enough or someone brave speaks up. So. I empathized. But, I that’s gotta bo both ways.

And a lot of the complaints from ‘the old guard’ have been about people not using content warnings when talking about their own experiences of being targeted for being BIPOC, LGBTQIA+, or more. One of my favorite follows - Chanda Prescod-Weinstein - after being told how easy it is to block harassers, etc., was called all sorts of racial and religious epithets by the predominantly white techy old guard. Others folk are having their posts taken down for talking about real experiences of discrimination or making larger points about structural inequity with some (gasp) humor.

I honestly don’t know what to say about this one. I think some of my hope in moving somewhere that seems to, at its core, be anti-harassment, that the situation would start better than somewhere like Twitter, which is moving back to letting the worst wander its virtual halls. But the structural bigotry of the systems we live within - and the very nature of a network initially built by largely white OSS folk - mean that it’s there, alive and well. Telling people to block or complain just puts the onus on the BIPOC who are being harassed. And relying on mods is not 100% - they’re overwhelmed, and likely subject to the same biases that are causing problems. That said, I have hope at hope quickly this has been recognized as a problem, and it looks like good conversations are emerging. I’m hoping that the ‘old guard’ is listening and that, because of the decentralized nature of Mastodon, things can be dealt with more swiftly and better tools can be put in place to stop bad actors. I also know I’m a white man talking out of his own experience, so, let’s take my hope and believe it has to be better than how Twitter is devolving with a grain of salt.

Fin

I’m actually going to leave it on that last note. There are a lot of threads going around about the importance of moderation and human intervention. Moderation is, at the end of the day, a necessary thing. Computers cannot do it (as it is easy to trick them or make their moderation biased). Humans are needed. The continued success of Mastodon is going to hinge on the ability of the ability to create new initiatives to enable good moderation. As this is open source free software, it is where people actually throwing in a few sawbucks will be needed. And that’s because it’s a human and community problem. Not something for already overtaxed server admins to pickup as a new hobby.

Given the thoughtful discourse around moderation, the already excellent experience (I go to Twitter now girding myself for anxiety, and now that I’m playing with Post, I go there for a dose of self-importance in Times New Roman mixed with useful information - athough, early days), and the much warmer sense of community, I’m so far sold. Will I transition to 100% pachyderm and delete the bird app? I think eventually yes - particularly as voices I love are slowly being cut-off over there. I’ll likely keep a cross-poster on, though, both to hold on to my account, and because I know that this isn’t a universal decision at the moment for everyone.

But for now - two weeks in - I’m really enjoying Mastodon and finding it scratches my need for a social network and community itch!

Tidyeval/Dynamic Evaluation for Writing Functions in the Tidyverse

Jarrett Byrnes — Sun, 28 Nov 2021 05:00:00 GMT

Once we have become deeply immersed in the tidyverse, sometimes we want to write tidy functions. Functions where we can use ggplot() or mutate() but supply arguments to specific columns from a data set. This does not always work. For example, let’s have some fun with mtcars -

library(ggplot2)
library(dplyr)

my_fun <- function(x, y, col){
  ggplot(mtcars, 
         aes(x = x, y =y , color = col)) + 
    geom_point()
}

my_fun("mpg", "cyl", "disp")

Well that doesn’t look right. Not right at all. And if you do the following:

my_fun(mpg, cyl, disp)

That just throws an error.

Now, ggplot in particular has a nice little workaround - aes_string()

my_fun_string <- function(x, y, col){
  ggplot(mtcars, 
         aes_string(x = x, y =y , color = col)) + 
    geom_point()
}


my_fun_string("mpg", "cyl", "disp")

Warning: `aes_string()` was deprecated in ggplot2 3.0.0.
ℹ Please use tidy evaluation idioms with `aes()`.
ℹ See also `vignette("ggplot2-in-packages")` for more information.

But - that still doesn’t work with unquoted variable names. Nor is that a general solution to the problem, as dplyr and friends don’t have *_string functions.

There are a few solultions to this problem. The first is recognizing that tidyverse functions implicitly use . or .data, so we can use this to make a function that works for quoted column names.

my_fun_tidy <- function(x, y, col){
  
  ggplot(mtcars, 
         aes(x = .data[[x]],
             y = .data[[y]],
             color = .data[[col]])) + geom_point()
}


my_fun_tidy("mpg", "cyl", "disp")

Convenient. And usable in other contexts. However, you still have to quote your column names. Also, more annoyingly, you have to remember when and where to use . or .data which, I’ll be honest, I forget 90% of the time.

A second solution is to use glue syntax which is now baked into tidy evaluation.

my_fun_quotes <- function(x, y, col){

  ggplot(mtcars, 
         aes_string(x = {{ x }}, 
                    y = {{ y }} , 
                    color = {{ col }})) + 
    geom_point()
  
}

my_fun_quotes("mpg", "cyl", "disp")

Nice! It works! And all we have to remember is to wrap our variables in { } for it to work. This still doesn’t solve the unquoted column name problem, though. For that, we need a function from rlang where we tell R that a variable is quoted with enquo. We can then use it as before.

my_fun_enquo <- function(x, y, col){
  
  x <- enquo(x)
  y <- enquo(y)
  col <- enquo(col)
  
  ggplot(mtcars, 
         aes_string(x = {{ x }}, 
                    y = {{ y }} , 
                    color = {{ col }})) + geom_point()
  
}

my_fun_enquo(mpg, cyl, disp)

And there you go! This later will work really well for putting things into tidy workflows. And you don’t have to get deep into thinking about quosures and such, which, TBH, I could never follow/remember the exact sequence and logic of when I wasn’t working with them every single day.

R Tensorflow on an M1 Mac Without Crashing

Jarrett Byrnes — Fri, 07 May 2021 04:00:00 GMT

As my third semester of online teaching continued apace, something sad started to happen. My laptop started rebooting at irregular intervals. Then, the keyboard starting coming apart. This was a trusty laptop of 8 years - a delightful Macbook Air that has been my companion through several life transitions. And it would seem, parts for it were no longer made, unless I wanted to buy them aftermarket and fix it myself (still on my list - if I can find the parts!) I was truly sad.

But…. also secretly delighted? I was doing more and more deep learning and Bayesian work, and the computer was showing its age. In particular, the CNN work I was doing with the R Tensorflow wrapper was starting to heat up. What if, I reasoned to myself after seeing the speed benchmarks, I was able to harness the 6 CPUs and GPU of a Macbook pro for this work? I could do major testing at home before I sent big jobs off to a server. I was excited.

I also should have done a wee bit more research.

As I loaded up R and the requisite libraries the day I got my new hog, and ran the MNIST demo…. I got an immediate crash. Every time I tried any tensorflow code, CRASH!

I dug about, and found that it was due to architecture differences. R installs the x86 tensorflow, and….. well while there is a version for M1 computers, it’s on github and not part of the main release.

Heck, R itself is being run via Rosetta, as it’s not native to arm64 yet.

Thus began my quixotic quest to make all of this work. I’ve been posting and cobbling together different resources. I’ve learned a bit of python (and now know I need to learn more), and finally understand what all of this conda kerfuffle is all about. So. Here is what I have learned. Follow these steps, and it will run. And pretty well.

I am not 100% sure if it’s using my GPU…. if anyone has any comments on that, let me know. The guide below is likely not perfect, might not work for everyone, but, it worked for me? This is distilled from this thread. So, here you go.

Go through the instruction to install Tensorflow on an M1 Mac from here. Note, I called the environment I created tf_env.
Install one of the arm64 nightly builds at https://mac.r-project.org/ - I’m using 4.1. Oh hey, hi new pipe operator! Or, er, new_pipe_operator |> hello().
Install a daily build of Rstudio for OSX from https://dailies.rstudio.com/ so it can handle the new version of R.
To test, run Rstudio, and type version to make sure your arch is aarch64.
OK, in your terminal, run conda activate tf_env - or whatever you named your environment for tensorflow in for #1.
To open Rstudio from the environment, run open -na rstudio
Install the keras and tensorflow libraries. BUT - do not run install_keras() or install_tensorflow() - because you’ve got it!
At the head of your Rscript, include the following:

library(tensorflow)
library(keras)

use_python("~/miniforge3/bin/python")
use_condaenv("tf_env")

with the later two getting you all good and set.

And then - boom - it will run! I was never so happy as to see a successful run.

Look at tensorflow go!

Now, are there likely some problems here? Maybe! I’d love to know, and I will edit the post as any that are relevant updates come in!

I’m also still not certain it’s using the GPU. I tried running a CNN on it, watching GPU performance from Activity Monitor, and didn’t really see anything. Would be delighted at any thoughts/suggestions about this issue.

But, hey, it runs!

GF Pizza that Rocks

Jarrett Byrnes — Sat, 03 Oct 2020 04:00:00 GMT

Yes, these are GF pizzas

Since HD went gluten free, we’ve found a few decent gluten free pizza places around the ’ville and beyond. But, it’s only been OK. Never quite crispy enough for thin crust, and inconsistent from place to place. So, I decided to set out and make my own at home.

I culled through a bunch of recipes, but was always unsatisfied. Years ago, a friend of mine gave me an excellent recipe that is delightful for its inexactness and yet consistent excellent performance. Everything I saw involved tons of different ingredients, or really was just GF bread dough rolled out - which seemed OK, but, I still wasn’t sure it would get the right texture.

In reading around about GF baking in general, though, I kept running into this odd ingredient that appeared to improve texture of dough - or could even over-correct! Expandex - a modified tapioca startch. I had some in my pantry from my early days of starting the GF journey, but had never used it.

What if, I wondered, I used a GF flour that is ideal for pies and pastries, but added in some expandex to improve dough texture to give it better chew? Could I get back to something that works well for pizza.

The answer is - hell yes.

So, here’s my friend Jarrod’s recipe (his writing!), modified for GF use, with links to the products used, and a few small changes.

Gluten Free Pizza Dough

For the dough, if you’re feeling courageous:

Ingredients
About 1 cup of water, shower temperature
About 2 3/4 cups Cup4Cup GF flour (I suppose other pastry-oriented GF flours will work, too)
1/4 Cup of Expandex modified tapioca starch
1 teaspoon to 1 tablespoon of salt, your preference on dough saltiness
1 packet (2 1/4 tsp) of yeast
generous pinch of sugar or squirt of honey

Utensils
Big spoon Flat surface
Big bowl
2 cookie sheets (or one really huge one)

In a big bowl, mix the water, sugar or honey, and yeast. Cool your heels for five minutes.

Mix in the flour and salt, and maybe a glug of olive oil.

Stir and mix into in a good sticky ball that holds together. Let rise in a ball in an oiled bowl for 45 min - 1 hr.

Put some oil in the bowl so the dough doesn’t stick. Put the kneaded dough back in the bowl. Cover it up and put it in a not-cold (could be warm; doesn’t really matter) place for 20-30 minutes.

Divide into halves for big pans of thirds for rounds.

For halves, make sure the surface is floured with rice flour or somesuch and roll out the dough really really flat (1/8”?). Then lift it up, drape it over your cookie sheet, and either trim the edges or fold them down. For thirds, roll out rounds like a pie crust. Trim edges if you like, or fold for a bit more chewiness.

Put yummy things on top.

Bake in a 500-or-so-degree oven until it’s brown, but not dark brown. 10-12 minutes.

Bottom rack can be a-ok for GF dough. Not classic gluten dough. Why? Dunno! But you still might want to flip oven placement halfway if you’re doing two.

Simulating “Posterior” Predictions from Non-Bayesian Fits

Jarrett Byrnes — Thu, 24 Sep 2020 04:00:00 GMT

As I sat down to review my course materials for my grad Biostats and Computation class this summer - Biology 607 - I was settled in with a sense of calm. I had taught this for years. I knew what I was doing. I had reshaped the class twice, and the student outcomes were on track to be great. Just going to coast through this in Covid times so I could write some papers.

Then the mailman came. And delivered a brand spanking new copy of Gelman et al.’s new Regression and Other Stories.

After getting halfway through, and realizing that day had turned to night, it opened up…. a lot of things. I’ll probably cogitate on them in the future, and suffice to say I have re-written more than I want (and re-arranged - and realized my re-arrangement needs to be re-arranged next time).

But today, let’s talk simulating the outcomes of our model.

One thing I’ve always enjoyed about Bayesian models is the natural connection to simulation. And one way of model checking is to look at how your posterior predictions line up with the actual distribution of your data. For example, let’s look at a trivial example with the super-duper fun Palmer Penguins.

Comparing Posteriors from Bayesian Fits to Observed Distributions of Data

Let’s fit a simple model looking at flipper length as a function of body mass. It’s not going to be perfect, and that’s OK - I want to see some mismatch. We’ll do it with rstanarm (but use the approxmiation for speed).

library(rstanarm)
library(palmerpenguins)

#for some manipulations
library(dplyr)
library(tidyr)

#remove the NAs to make the rest smoother
penguins <- penguins %>% 
  filter(!is.na(flipper_length_mm))

mod_stan <- stan_glm(flipper_length_mm ~ body_mass_g, 
                     data = penguins,
                     algorithm = "optimizing"
                     )

So, one of the most usefu diagnostics here is to see how well the distribution of the predictions matches, or does not match, the data.

library(bayesplot)

yhat_stan <- posterior_linpred(mod_stan, draws = 100)
ppc_dens_overlay(penguins$flipper_length_mm,
                 yhat_stan)

Now, note, I used posterior_linpred() - so, this is for fitted values. But things look much the same with prediction intervals - if not a little worse

ypred_stan <- posterior_predict(mod_stan, draws = 100)
ppc_dens_overlay(penguins$flipper_length_mm,
                 ypred_stan)

What’s great, in both is that you can see the lack of bimodality, and that valley being missing. This is due to not including species. But, it’s SUPER SUPER SUPER useful in trying to diagnose where your model fails.

OK. This is great. I’m going to teach some Bayes. But. What about for those students who don’t want to go Bayes, for whatever reason, but still want access to this awesome machinery?

I was at a bit of a loss. In working with mixed models in the past, to get similar simulations, I’d turn to the merTools package and it’s predictInterval() function that can turn out raw simulations. But - it doesn’t work on lm() or glm() fit models.

Hrm. What to do.

ARM Yourself!

My first inclination was to go old school with the arm package from the original Gelman and Hill which is now being superseeded by this new book and whatever is to come next (which I am already excited for).

arm had a sim() function that could extract simulated coefficients, and then you could be on your merry way yourself. So, let’s fit a model, and then get those fitted simulated values

library(arm)

#let's fit a model from the palmer penguins ####
mod <- lm(flipper_length_mm ~ body_mass_g, 
          data = penguins)


#get some simulated coefficients using arm::sim() ####
sims <- sim(mod)

#now the model frame
X <- model.frame(mod)

#now get those simulated fitted values
yhat_arm <- coef(sims) %*% t(model.matrix(mod))

#and make it into a nice tidy data frame
yhat_arm <- as_tibble(t(yhat_arm)) %>%
  gather(sim, flipper_length_mm)

This is great! We can look at how this fits with the distribution of flipper lengths. Moreover, we can compare it to the Bayesian posteriors - we’ll see a difference due to priors, but, it’s informative!

library(ggplot2)
library(patchwork)
theme_set(theme_classic(base_size = 15))

arm_plot <- ggplot() +
  geom_density(data = yhat_arm, 
               aes(x = flipper_length_mm, group = sim),
               lwd = 0.1, color = "blue") +
    geom_density(data = penguins, 
                 aes(x = flipper_length_mm),
               lwd = 2) +
  labs(title = "Distribution of Flipper length (black) and arm Simulations")

#reshape posterior predictions and make a running DF
fit_sims <- 
  as_tibble(t(yhat_stan)) %>%
  gather(sim, flipper_length_mm) %>%
  mutate(type = "stan_fit") %>%
  rbind(yhat_arm %>% mutate(type = "arm_fit"))

compare_plot <- ggplot(data = fit_sims, 
               aes(x = flipper_length_mm, 
                   group = paste(sim, type), 
               color = type)) +
  geom_density(lwd = 0.1) +
  scale_color_brewer(type = "qual")


arm_plot + compare_plot

This is super cool as, 1) we can see that the same problem with the simulated versus observed distributions and 2) the simulations from our lm() and stan_glm() match really nicely.

But Can You Make it Simple?

OK. This is cool. But…. I had to extract coefficients, do the matrix multiplication myself. “Trivial!” you say. But - well, as I get more interesting models that aren’t just lm() fits, it gets harder. And, more importantly, for teaching, this is a path to code confusion for my students. So…. what else is there? I mean, sure, I can write a little function for them:

#can we make a nice little function? ####
predictInterval_lm <- function(mod, n = 100){
  sims <- arm::sim(mod, n = 100)
  model_mat <- t(model.matrix(mod))
  mod_coefs <- coef(sims)
  
  y_sim <- coef(sims) %*% t(model.matrix(mod))
}

But - well, that’s a one-shot. They need something more general. We all need something more general.

And so, I tested my google fu. And found…. stats::simulate() At which point I hit myself in the head. Simulated one or more responses from the distribution corresponding to a fitted model object indeed!

Let’s take a look.

# Wait, does stats::simulate work? ####
yhat_simulate <- simulate(mod, nsim = 100)

yhat_simulate <- yhat_simulate %>%
  gather(sim, flipper_length_mm) 

ggplot() +
  geom_density(data = yhat_simulate, 
               aes(x = flipper_length_mm, group = sim),
               lwd = 0.1, color = "red") +
    geom_density(data = penguins, 
                 aes(x = flipper_length_mm),
               lwd = 2) +
  labs(title = "Distribution of Flipper length (black) and simulate Simulations")

Huh. That does not look the same at all. Digging into it with getAnywhere(simulate.lm), it becomes clear that what simulate() is doing is taking fitted values and adding random draws from the error distribution - NOT taking draws from the coefficients to get simulated fitted values. sad trombone. OK, so this isn’t going to work.

Getting to the Core of Our Simulation Problem

So, I went back to the Goog. Poking around and around, I eventually found the excellent coreSim package, that let’s you generate simulated confidence intervals from fit models of many types. Huzzah!

The package has a function to build quantiles of interest based on models using simulation - qi_builder(). While it autimatically generates intervals from simulations and slims them down to a particular interval of interest, by giving a CI of one and specifying that we want the raw simulations with the argument slim = FALSE, we can generate the same simulations as above. We need to reshape and rename the output a bit, but it’ll do!

#what about coreSim? ####
library(coreSim)

#get those simulations, add simulation numbers, and rename
yhat_coresim <- qi_builder(mod, newdata = penguins, ci = 1,
                slim = FALSE, nsim = 100) %>%
  mutate(sim = rep(1:100, 342)) %>%
  rename(flipper_length_mm = qi_) %>%
  mutate(type = "coresim_fit",
         sim = as.character(sim)) %>%
  dplyr::select(-body_mass_g)

OK, so, first, how does this compare to the data? And then, how does it compare to other sim methods?

core_plot <- ggplot() +
  geom_density(data = yhat_coresim, 
               aes(x = flipper_length_mm, group = sim),
               lwd = 0.1, color = "orange") +
    geom_density(data = penguins, 
                 aes(x = flipper_length_mm),
               lwd = 2) +
  labs(title = "Distribution of Flipper length (black) and coreSim Simulations")

fit_sims <- fit_sims %>%
  bind_rows(yhat_coresim)

compare_plot_core <- ggplot(data = fit_sims, 
               aes(x = flipper_length_mm, 
                   group = paste(sim, type), 
               color = type)) +
  geom_density(lwd = 0.1) +
  scale_color_brewer(type = "qual")

OK! Nice! Everything lines up, and, while we had to do some manipulation of the qi_builder output (and, really, it needs some work on column names), this was way simpler than the arm solution. And we still see the problems with the mismatch between the model and the data.

But - still, I mean, I could go and re-write some of the coreSim functionality to have simulation numbers. But…. There had to be.

Finding my DHARMa

I’m a HUGE fan of Florian Hartig’s DHARMa package for assessing fit via simulated quantile residuals. Heck, eyeballing the data versus simulated distributions here is exactly what DHARMa is doing! I was working with a colleague on assessing fit in some glmmTMB models when it hit me.

DHARMa calculates simulated values. From which it derives residuals. Which means it’s got a pretty robust flexible simulation algorithm in there, which means…..it has a simulations method. I went spelunking around the help files, and found getSimulations() which works for a wide variety of objects. Would this be the key? It turns out a matrix, so I needed to do a bit of reshaping, but…

library(DHARMa)

# last, what about DHARMa?
yhat_dharma <- getSimulations(mod, nsim = 100 ) %>%
  as_tibble() %>%
  tidyr::gather(key = "sim", value = "flipper_length_mm") %>%
  mutate(type = "pred_sharma")

ggplot() +
  geom_density(data = yhat_dharma, 
               aes(x = flipper_length_mm, group = sim),
               lwd = 0.1, color = "purple") +
    geom_density(data = penguins, 
                 aes(x = flipper_length_mm),
               lwd = 2) +
  labs(title = "Distribution of Flipper length (black) and DHARMa Simulations")

Alas - DHARMa isn’t pulling simulated values of the coefficients. This looks the same as the simulate functions above. And with good reason! As I dug into the code for getSimulations() for lm() fit objects, it uses simulate!

Indeed, I would have noticed this from reading the helpfile (HA!) which says ” The purpose of this wrapper for for the simulate function is to return the simulations from a model in a standardized way”.

Other packages considered

In the course of putting this together, I looked a bit at simglm as well, but it’s for knowing parameters a prior, not calculating simulations from fit models.

Conclusion - coreSim for now?

So, for now, coreSim for non-mixed models seems to be the way to go, predictInterval for merMods, and…. I’m not sure for others. glmmTMB has the ability to pull simulated draws of random effects using simulate() as seen here. But, I find myself unsatisfied with the lack of a general case solution. A predict_interval, if you will, for any model types. And one that perhaps integrates variation not just in parameters (e.g. fitted simulations), but also error terms (e.g., prediction simulations). If anyone has a solution, I’d love to know it! Perhaps I am missing a package? Short of using rstanarm, I’m stumped for the moment!

Roaring Back into Mobility in the US Post-Covid-Lockdown

Jarrett Byrnes — Mon, 25 May 2020 04:00:00 GMT

As part of the Covid19R Project, I’ve been working on a new package, covid19mobility. It started as an effort to pull in the Apple Mobility Trends, and has since morphed as more mobility data comes out. Right now, I’m working through the Google Mobility Data, & I’m interested in the Descartes mobility data and any others you can suggest down below (link to source)! But the data itself - it’s FASCINATING!

As I finished getting the Apple data in (there are massive issues for all of the aforementioned data sets in terms of getting standardized location codes - I’m trying to use either the ISO 3166 2 standard for countries and states or the UN LOCODE for cities, and many of the states and cities have different names than those in the data standards. Watch me dplyr::mutate() and ifelse my way into oblivion)..

OK, that was a long rant, let me start over -

As I finished getting the Apple data in, I decided to check it out with a quick vignette. It’s Memorial Day, US states are starting to open up - I wonder what our traffic looks like? Now, note, Apple’s data is the relative volume of directions requests for each data type. So, let’s look at the US…

# remotes::install_github("covid19r/covid19mobility")
library(covid19mobility)
library(dplyr)
library(ggplot2)

#the data, filtered to the US
us_mobile <- #refresh_covid19mobility_apple_country() %>%
  covid19mobility_apple_country_demo %>%
  filter(location_code == "US")

max_date <- max(us_mobile$date)

#now, a plot!
baseplot <- ggplot(us_mobile,
       aes(x = date, y = value, color = data_type)) +
  scale_color_brewer(palette = "Dark2") +
  theme_bw(base_size=14) +
  labs(color = "Mobility Type",
       y = "Relative Value",
       x = "") 

baseplot +
  geom_line(size = 1) +
  ggtitle(paste0("Apple Mobility Trends for the US on ", max_date))

Wow. So, driving is basically back to where it was pre-lockdown. Walkign is coming back, but isn’t there yet. And transit is… nada. Looking at this using a GAM to smooth it all out shows the same thing (I know, I know, there’s much more complex and fun timeseries stuff to do here, but…)

baseplot +
  stat_smooth(method = "gam", fill = NA) +
  ggtitle("Apple Mobility Trends for the US from fitted GAMs")

The driving trend is really interesting. Is this really due to everywhere going up uniformly? To investigate, I pulled in the subregional data - for the US this just has driving stats. And then used the awesome geofacet package from Ryan Hafen to visualize. I’ll just use GAMs in order to facilitate visualization without the wigglies due to within-week variation.

library(geofacet)

state_mobile <- refresh_covid19mobility_apple_subregion() %>%
  filter(country=="United States") %>%
  filter(!(location %in% c("Guam", "Puerto Rico", 
                         "Virgin Islands")))

state_baseplot <- ggplot(state_mobile,
       aes(x = date, y = value)) +
  theme_bw(base_size=11) +
  labs(y = "Relative Value",
       x = "") +
  ggtitle(paste0("Apple Mobility Driving Trends for the US on ", max_date)) 
  theme(axis.text.x = element_text(angle = 90, hjust = 1, vjust = 0.5))

state_baseplot +
  facet_geo(~location) +
  stat_smooth(method = "gam", fill = NA, color = "#1B9E77") + 
  theme(axis.text.x = element_text(angle = 90, hjust = 1))

So, a few places have stayed - Hawaii, Mass, and a few others - and others have gone WAAAY up - ND, Wyoming, Montata. Looking at it another way - we can really see how the top 5 states drive the trend up - BUT - with some exceptions, driving is still back to normal most places

library(ggrepel) #for labels


#fit a bunch of GAMs so that labels connect to the right places
state_fits <- state_mobile %>%
  group_by(location) %>%
  tidyr::nest() %>%
  mutate(mods = purrr::map(data, ~mgcv::gam(value ~ s(as.numeric(date), bs = "cs"), data = .)),
         data = purrr::map2(data, mods, ~modelr::add_predictions(.x, .y))) %>%
  tidyr::unnest(data) %>%
  ungroup()

# make labels for the top 5 states
tops <- state_fits %>%
  filter(date == max(date)) %>%
  group_by(location) %>% slice(1L) %>% ungroup() %>%
  arrange(desc(pred)) %>%
  slice(1:5)

# make labels for the bottom five states
bottoms <- state_fits %>%
  filter(date == max(date)) %>%
  arrange(pred) %>%
  slice(1:5)

#merge in info about those states so we can highlight with colors
state_mobile <- state_mobile %>%
  mutate(top_state = ifelse(location %in% tops$location, "top", "middle"),
         top_state = ifelse(location %in% bottoms$location, "bottom", top_state),
         ) 

# the plot  - with labels! And colors!
# and a so-so job of using ggrepel!
set.seed(2020)
state_baseplot +
  geom_line(data = state_mobile, stat="smooth",
              method = "gam", 
              mapping = aes(group = location, 
                            color = top_state, alpha = top_state),
              size = 0.5) +
  scale_color_manual(values = c("darkred", "#1B9E77", "darkorange")) +
  scale_alpha_manual(values = c(0.6, 0.2, 0.6)) +
  guides(color = "none", alpha = "none") + 
  #labels those states - must be a better neater way.... hrm....
  geom_text_repel(data = tops,
                  mapping = aes(x = date, y = pred, label = location),
                  force = 5) +
  geom_text_repel(data = bottoms,
                  mapping = aes(x = date, y = pred, label = location),
                  force = 5,
                  hjust = 0.5) +
  theme_bw(base_size=14) +
  scale_y_continuous(limits = c(0,300))

Surprised Florida is at the bottom, currently, but, really interesting to see how most states are mostly recovered in terms of driving. Now I want to run this after memorial day and see how it looks!

Best GF Pancakes Yet

Jarrett Byrnes — Sun, 10 May 2020 04:00:00 GMT

I love pancakes. My wife loves pancakes. My daughter LOVES chocolate pancakes (and takes a picture with her face smeared in chocolate every time she she finishes eating chocolate chip pancakes). But… my daughter has celiac.

So, I’ve been working through a lot of GF pancake recipies, or using GF flour (Cup 4 Cup is hands down THE BEST) with regular recipes to make them. I kept being unsatisfied, but, eventually learned a basic formula to them. To thick, to thin, too cakey, to fluffy (yes, this is a thing). This morning, I made a few adjustments, and I think I hit on the right one, so I’m going to archive it here so I don’t forget.

Also, if you are unsure of how to cook pancakges, find the Good Eats Pancake Episode where Alton really teaches you how to do it right.

Best GF Pancakes Yet

Ingredients:

1 Cup Gluten Free Flour Blend (I prefer Cup 4 Cup, but others likely work) If no Xanthum gum in mix, add ~ 1/4 tsp
1/3 Cup Oat Flour (this was a huge innovation!)
1 1/2 tsp baking powder
1/4 tsp baking soda
1/2 tsp kosher salt
2 Tbs sugar
2 eggs
1 1/2 - 2 C milk (to get it to the thinness of your liking - play with it)
3 Tbs melted butter

Instructions:

Whisk dry ingredients.
Whisk wet ingedients.
Mix wet into dry until all dry is incorporated. Lumpy batter is just fine.
Heat griddle/cast iron/carbon steel pan to ~350F
Spoon on 1/4-1/3 Cup batter. Cook until bubbles form around edges.
Flip.
Cook until just cooked through.

The Covid19R Project

Jarrett Byrnes — Thu, 07 May 2020 04:00:00 GMT

We want our coronavirus data! We want it now! We want it TIDY!

tl;dr Go check out the Covid19R project and it’s documentation - a way for many people to bring their unique data harvesting efforts of Covid-19 data into a single tidy data format, and then be redistributed through a single low-dependency package, covid19R. It’s all one big work-in-progress, but check it out!

OK, the story!

Back in February (remember February), while teaching data science, I was trying to think of ways to spice up the class with more relevant data. We were in the midst of data tidying, when I saw a tweet from Rami Krispin about starting a Covid R data package (whose account as of 2022-11-27 is suspended?)

Cool! A coronavirus data package. I went on to make a neat data tidying lab - well, I think it’s neat - as well as weaving the data into other parts of the course. I also made a few pull requests on the repo to add features….as did Amanda Dobbyn. Rami contacted both of us, and thus a collaboration was born!

Tidy Covid-19 Data

We had been pulling from many different data sources, and found lots of difficulty in bringing them together. I had used rnaturalearth to get at unique place identifiers, but the proliferation of data sets and data aggregation efforts was bewildering. How to get a handle on the wide variety of data types, spatial location types, and more?

So, we began designing a very simple long format data specification and agreed to make our data packages create data in that format. I wrote up a package to harvest the updated New York Times data as an example.

date	location	location_type	location_code	location_code_type	data_type	value
2021-01-23	Alabama	state	01	fips_code	cases_total	439442
2021-01-23	Alabama	state	01	fips_code	deaths_total	6657
2021-01-23	Alaska	state	02	fips_code	cases_total	52820
2021-01-23	Alaska	state	02	fips_code	deaths_total	250
2021-01-23	Arizona	state	04	fips_code	cases_total	717593
2021-01-23	Arizona	state	04	fips_code	deaths_total	12170

One Package to Rule Them All

This was good - but, how would people get data?

We noticed that with nearly every different data set, we needed a different approach to data manipulation. If we threw all of our efforts into a single package, the dependency tree would rapidly spiral out of control. This seemed not good.

Many small packages, each specializing in unique data sets or collections of data sets seemed good, but…. how to make all of that searchable? Did we really want to make people have to install a bazillion different small packages? This seemed inefficient.

So, we took a cue from the many excellent packages by the good folk at ropensci and created a master package - covid19R. This package wouldn’t do anything, save redistribute data files. Moreover, so it didn’t have to depend on all of those little packages, and thus have all of those same problems as above, we built a separate covid19Rdata package/repo that updates once daily to refresh all of the data from any package that is part of the project.

So, you can get the information about what data is part of this Tidy Covid-19 data project, choose a relevant data set, and grab it, all without racking having to download tons of packages, satisfy weird dependencies, or otherwise pull your hair out!

Here’s what’s part of the project at time of writing

data_set_name	package_name	function_to_get_data	data_details	data_url	license_url	data_types	location_types	spatial_extent	has_geospatial_info	get_info_passing	refresh_status	last_refresh_update
covid19nytimes_states	covid19nytimes	refresh_covid19nytimes_states	Open Source data from the New York Times on distribution of confirmed Covid-19 cases and deaths in the US States. For more, see https://www.nytimes.com/article/coronavirus-county-data-us.html or the readme at https://github.com/nytimes/covid-19-data.	https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-counties.csv	https://github.com/nytimes/covid-19-data/blob/master/LICENSE	cases_total, deaths_total	state	country	FALSE	TRUE	Passed	2021-01-24 16:57:46
covid19nytimes_counties	covid19nytimes	refresh_covid19nytimes_counties	Open Source data from the New York Times on distribution of confirmed Covid-19 cases and deaths in the US by County. For more, see https://www.nytimes.com/article/coronavirus-county-data-us.html or the readme at https://github.com/nytimes/covid-19-data.	https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-counties.csv	https://github.com/nytimes/covid-19-data/blob/master/LICENSE	cases_total, deaths_total	state	country	FALSE	TRUE	Passed	2021-01-24 16:58:02
covid19france	covid19france	refresh_covid19france	Open Source data from opencovid19-fr on distribution of confirmed Covid-19 cases and deaths in the US States. For more, see https://github.com/opencovid19-fr/data.	https://raw.githubusercontent.com/opencovid19-fr/data/master/dist/chiffres-cles.csv	https://github.com/opencovid19-fr/data/blob/master/LICENSE	confirmed, dead, icu, hospitalized, recovered, discovered	county, region, country, overseas collectivity	country	FALSE	TRUE	Passed	2021-01-24 16:58:07
CanadaC19_cases	CanadaC19	refresh_CanadaC19_cases	Open Source data from multiple public reporting data throughout Canada. For more, see https://github.com/ishaberry/Covid19Canada.	https://raw.githubusercontent.com/ishaberry/Covid19Canada/master/cases.csv	https://github.com/debusklaneml/CanadaC19/blob/master/LICENSE	cases_new	state	state	FALSE	TRUE	Passed	2021-01-24 16:58:17
covid19us	covid19us	refresh_covid19us	Open Source data from COVID Tracking Project on the distribution of Covid-19 cases and deaths in the US. For more, see https://github.com/opencovid19-fr/data.	https://covidtracking.com/api	https://github.com/aedobbyn/covid19us/blob/master/LICENSE.md	positive, negative, pending, total_test_results_source, total_test_results, hospitalized_currently, hospitalized_cumulative, in_icu_currently, in_icu_cumulative, on_ventilator_currently, on_ventilator_cumulative, recovered, death, hospitalized, total_tests_viral, positive_tests_viral, negative_tests_viral, positive_cases_viral, death_confirmed, death_probable, total_test_encounters_viral, total_tests_people_viral, total_tests_antibody, positive_tests_antibody, negative_tests_antibody, total_tests_people_antibody, positive_tests_people_antibody, negative_tests_people_antibody, total_tests_people_antigen, positive_tests_people_antigen, total_tests_antigen, positive_tests_antigen, positive_increase, negative_increase, total, total_test_results_increase, death_increase, hospitalized_increase, negative_regular_score, negative_score, positive_score	state	country	FALSE	TRUE	Passed	2021-01-24 16:58:33
VirginiaC19	VirginiaC19	refresh_VirginiaC19	Open Source data from Virginia Department of Health. For more information, see https://www.vdh.virginia.gov/coronavirus/.	https://www.vdh.virginia.gov/content/uploads/sites/182/2020/03/VDH-COVID-19-PublicUseDataset-Cases.csv	https://github.com/debusklaneml/VirginiaC19/blob/master/LICENSE	cases_new,deaths_new,hosp_new	county	state	FALSE	TRUE	Passed	2021-01-24 16:58:33
covid19tunisia	covid19tunisia	refresh_covid19tunisia	Open Source data on distribution of confirmed Covid-19 cases, recovered ones and deaths in Tunisia. For more, https://github.com/MounaBelaid/covid19datatunisia	https://raw.githubusercontent.com/MounaBelaid/covid19datatunisia/master/dist/data.csv	https://github.com/MounaBelaid/covid19datatunisia/blob/master/LICENSE	cases_new, deaths_new, recovered_new	state	country	FALSE	TRUE	Passed	2021-01-24 16:58:33
covid19mobility_apple_country	covid19mobility	refresh_covid19mobility_apple_country	Data reflects relative volume of directions requests compared to a baseline volume on January 13th, 2020 for multiple transportation modes aggregated at the country level.	https://www.apple.com/covid19/mobility	https://www.apple.com/covid19/mobility	driving, walking, transit	country	global	FALSE	TRUE	Passed	2021-01-24 16:58:36
covid19mobility_apple_subregion	covid19mobility	refresh_covid19mobility_apple_subregion	Data reflects relative volume of directions requests compared to a baseline volume on January 13th, 2020 for multiple transportation modes aggregated at the subregion (state) level.	https://www.apple.com/covid19/mobility	https://www.apple.com/covid19/mobility	driving, walking, transit	state	global	FALSE	TRUE	Passed	2021-01-24 16:58:59
covid19mobility_apple_city	covid19mobility	refresh_covid19mobility_apple_city	Data reflects relative volume of directions requests compared to a baseline volume on January 13th, 2020 for multiple transportation modes aggregated at the city level.	https://www.apple.com/covid19/mobility	https://www.apple.com/covid19/mobility	driving, walking, transit	city	global	TRUE	TRUE	Passed	2021-01-24 16:59:11
covid19mobility_google_country	covid19mobility	refresh_covid19mobility_google_country	Changes for each day are compared to a baseline value for that day of the week as compared to the 5-week period Jan 3-Feb 6, 2020 for visits to places falling in to certain categories.	https://www.google.com/covid19/mobility/	https://www.google.com/covid19/mobility/	retail_and_recreation_percent_change_from_baseline,grocery_and_pharmacy_percent_change_from_baseline,parks_percent_change_from_baseline,transit_stations_percent_change_from_baseline,workplaces_percent_change_from_baseline,residential_percent_change_from_baseline	country	global	FALSE	TRUE	Passed	2021-01-24 16:59:31
covid19mobility_google_subregions	covid19mobility	refresh_covid19mobility_google_subregions	Changes for each day are compared to a baseline value for that day of the week as compared to the 5-week period Jan 3-Feb 6, 2020 for visits to places falling in to certain categories. Data is aggregated at the state or subdivision level.	https://www.google.com/covid19/mobility/	https://www.google.com/covid19/mobility/	retail_and_recreation_percent_change_from_baseline,grocery_and_pharmacy_percent_change_from_baseline,parks_percent_change_from_baseline,transit_stations_percent_change_from_baseline,workplaces_percent_change_from_baseline,residential_percent_change_from_baseline	state	global	FALSE	TRUE	Passed	2021-01-24 17:00:12
covid19mobility_google_us_counties	covid19mobility	refresh_covid19mobility_google_us_counties	Changes for each day are compared to a baseline value for that day of the week as compared to the 5-week period Jan 3-Feb 6, 2020 for visits to places falling in to certain categories. Data is aggregated at the county level for the USA only.	https://www.google.com/covid19/mobility/	https://www.google.com/covid19/mobility/	retail_and_recreation_percent_change_from_baseline,grocery_and_pharmacy_percent_change_from_baseline,parks_percent_change_from_baseline,transit_stations_percent_change_from_baseline,workplaces_percent_change_from_baseline,residential_percent_change_from_baseline	county	country	FALSE	TRUE	Passed	2021-01-24 17:02:45
coronavirus_jhu	coronavirus	refresh_coronavirus_jhu*	The 2019 Novel Coronavirus COVID-19 (2019-nCoV) Dataset from the Johns Hopkins University Center for Systems Science and Engineering	https://systems.jhu.edu/research/public-health/ncov/	https://github.com/CSSEGISandData/COVID-19/	cases_new, recovered_new, deaths_new	country, state	global	TRUE	TRUE	Passed	2021-01-24 17:02:48

What is a Covid19R Project Package?

We have a few requirements for a package to be added to our project. First, file an issue that you’re creating one! Then, take a gander at our documentation on building packages. We’ve provided a template for you to use, if you’re new to package building.

We then ask that you provide at least two functions in your package - a get_info_*() function, where * is the package name, that supplies standard information that we chonk into the table y’all see above. And for each data set a refresh_*() function, where * is the data set name, that pulls in tasty fresh data and reformats it into our tidy Covid-19 data standard using an ever-expanding standardized vocabulary - and you can file issues to keep expanding it.

Our request to the community

We’d love this to become a much larger community - have existing Covid-19 data package developers implement our two simple functions so we can add their data to the project, provide support for people interested in creating data distribution packages - maybe their first package ever - to use our format so that data is good and tidy. We’re very interested in conversation - start one up here for the moment!

Our Hope

What we hope is that, with many datasets going in using common best practices, we’ll be able to facilitate discovery. Data cleaning and joining is sometimes 80-90% of the effort of any data science project. With a standard format, a standardized vocabulary, and the usage of standard geospatial codes, we hope to seriously reduce that time in order to speed up other work.

Won’t you join us?

Dusting off the old blahg

Jarrett Byrnes — Wed, 06 May 2020 04:00:00 GMT

Welp. It’s been a spell. Time to bring ye olde blahg back.

As I moved talking about marine science more on to Deep Sea News, I realized that there’s a huge part of my identity as a scientist that didn’t work there.

Data nerdery.

As many of those posts were written in RMarkdown, I took the plunge and have remade this into a blogdown site. It’s my first venture here. I’m just glad I managed to port all of my old posts over!

There’s definitely some things still to do - a comments section, etc - but, in the meantime, if you find some dust around the edges or a poor transition, now you can file an issue on this blog’s github repo! HA!

But, we shall see how rebooting this goes. I’ve got a few things to write about.

Bayesian SEM with BRMS

Jarrett Byrnes — Thu, 21 Dec 2017 05:00:00 GMT

Note: for the most up to date version of this, see this post on rpubs

Background

So, it looks like brms version 2.0 implements multivariate responses - and hence piecewise Structural Equation Modeling in a Bayesian framework. After https://github.com/paul-buerkner/brms/issues/303, it became clear that while one cannot do tests of D-separation, WAIC, LOO, etc., are all available. Let’s take a peak with a familiar example.

As a note, blavaan also implements Bayesian SEM, but brms is so gorram flexible with model forms, that I think it’s an excellent way to approach the problem that gives a lot more breathing room. And it’s in STAN, not JAGS!

The Keeley Data

OK, the venerable Keeley et al. 2005 and extensions in Grace and Keeley 2006. The teaching example that a number of us use is to look at how fire severity mediates plan species richness via cover.

### Getting the data from piecewiseSEM
#install.packages("jslefche/piecewiseSEM@2.0") #for the keeley data set
library(piecewiseSEM)
data(keeley)

##Otherwise, on the web
#keeley <- read.csv("http://byrneslab.net/classes/lavaan_materials/Keeley_rawdata_select4.csv ")

Using covariance based modeling, we would fit the following:

library(lavaan)

k_mod <- "
  rich ~ firesev + cover
  cover ~ firesev"

k_fit_lavaan <- sem(k_mod, data = keeley)

We can fit this in piecewiseSEM as well

k_fit_psem <- psem(
  lm(rich ~ firesev + cover, data=keeley),
  lm(cover ~ firesev, data=keeley),
  data = keeley
)

Yep, the same.

brms and SEM

In the new brms you can build these models with mvbrmsformula or just adding multiple brmsformula objects together. We’ll use set_rescor(FALSE) to not model the correlation between response variables (but could to represent residual correlations, I think!) So, here we have

library(brms)

## Loading required package: Rcpp

## Loading required package: ggplot2

## Loading 'brms' package (version 2.0.0). Useful instructions
## can be found by typing help('brms'). A more detailed introduction
## to the package is available through vignette('brms_overview').
## Run theme_set(theme_default()) to use the default bayesplot theme.

rich_mod <- bf(rich ~ firesev + cover)
cover_mod <- bf(cover ~ firesev)

k_fit_brms <- brm(rich_mod +
                  cover_mod +
                  set_rescor(FALSE),
                data=keeley,
                cores=4, chains = 2)

## Compiling the C++ model

## Start sampling

Looks the same! How did the model do?

plot(k_fit_brms)

One thing we can do is look at the WAIC or LOO which we can see is the sum of its component pieces. Let’s start with fitting those pieces.

rich_fit <- brm(rich_mod,
                data=keeley,
                cores=2, chains = 2)

## Compiling the C++ model

## Start sampling

cover_fit <- brm(cover_mod,
                data=keeley,
                cores=2, chains = 2)

## Compiling the C++ model
## Start sampling

Now, let’s look at the whole and the parts

WAIC(k_fit_brms)

##    WAIC    SE
##  768.47 15.52

WAIC(rich_fit)

##    WAIC   SE
##  734.05 9.35

WAIC(cover_fit)

##   WAIC    SE
##  34.43 12.61

Nice!

Comparison across all three forms

Broadly, the figures above show simular results. But let’s dig deeper. In lavaan we get

summary(k_fit_lavaan)

## lavaan (0.6-1.1170) converged normally after  32 iterations
##
##   Number of observations                            90
##
##   Estimator                                         ML
##   Model Fit Test Statistic                       0.000
##   Degrees of freedom                                 0
##
## Parameter Estimates:
##
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                             Standard
##
## Regressions:
##                    Estimate  Std.Err  z-value  P(>|z|)
##   rich ~
##     firesev          -2.531    0.976   -2.593    0.010
##     cover             9.911    5.083    1.950    0.051
##   cover ~
##     firesev          -0.084    0.018   -4.611    0.000
##
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)
##    .rich            187.212   27.908    6.708    0.000
##    .cover             0.081    0.012    6.708    0.000

We can see the rich ~ cover path has 0=0.051. Which, you know, some people are OK with, some might wonder. piecewiseSEM produces broadly similar results.

summary(k_fit_psem, intercept=TRUE)

##
## Structural Equation Model of k_fit_psem
##
## Call:
##   rich ~ firesev + cover
##   cover ~ firesev
##
##     AIC      BIC
##  14.000   31.499
##
## ---
## Tests of directed separation:
##
##  data frame with 0 columns and 0 rows
##
## Global goodness-of-fit:
##
##   Fisher's C = 0 with P-value = 1 and on 0 degrees of freedom
##
## ---
## Coefficients:
##
##   Response   Predictor Estimate Std.Error DF Crit.Value P.Value
##       rich (Intercept)  53.9358    7.0437 87     7.6573  0.0000
##       rich     firesev  -2.5308    0.9926 87    -2.5496  0.0125
##       rich       cover   9.9105    5.1701 87     1.9169  0.0585
##      cover (Intercept)   1.0744    0.0893 88    12.0299  0.0000
##      cover     firesev  -0.0839    0.0184 88    -4.5594  0.0000
##   Std.Estimate Std.SE
##         0.0000 0.0000 ***
##        -0.2768 0.1086   *
##         0.2081 0.1086
##         0.0000 0.0000 ***
##        -0.4371 0.0959 ***
##
##   Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05
##
## Individual R-squared:
##
##   Response method R.squared
##       rich     NA      0.17
##      cover     NA      0.19

Note that p-value goes up! Huh. We also now have intercepts, which we can compare to…

summary(k_fit_brms)

##  Family: MV(gaussian, gaussian)
##   Links: mu = identity; sigma = identity
##          mu = identity; sigma = identity
## Formula: rich ~ firesev + cover
##          cover ~ firesev
##    Data: keeley (Number of observations: 90)
## Samples: 2 chains, each with iter = 2000; warmup = 1000; thin = 1;
##          total post-warmup samples = 2000
##     ICs: LOO = NA; WAIC = NA; R2 = NA
##
## Population-Level Effects:
##                 Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
## rich_Intercept     54.16      6.92    40.46    67.73       2000 1.00
## cover_Intercept     1.07      0.09     0.90     1.24       2000 1.00
## rich_firesev       -2.57      0.99    -4.52    -0.58       2000 1.00
## rich_cover          9.84      5.05    -0.30    19.59       2000 1.00
## cover_firesev      -0.08      0.02    -0.12    -0.05       2000 1.00
##
## Family Specific Parameters:
##             Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
## sigma_rich     14.02      1.07    12.04    16.27       2000 1.00
## sigma_cover     0.29      0.02     0.25     0.34       2000 1.00
##
## Samples were drawn using sampling(NUTS). For each parameter, Eff.Sample
## is a crude measure of effective sample size, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).

Nice effective sample sizes and convergence. Note the rich_cover credible interval is pretty big, now, and we can see how it overlaps 0. So, broadly, similar results again!

OK, what can we do with this?From an SEM framework, we want to compare this to an alternate model. I’ve seen suggested that we can calculate posterior p-values for the basis set and go from there for a traditional D-sep test, but that really does mix frequentist and Bayesian paradigms, so, I’m trying to ponder that - although evaluating each independence claim isn’t a bad idea. Might need to write some code to do that.

Model comparison is a snap, however.

Mediation in BRMS

Let’s say we have the following alternate model where fire severity’s effect is fully mediated via cover.

OK, let’s fit this model

rich_mod_fullmed <- bf(rich ~  cover)

fit_brms_fullmed <- brm(rich_mod_fullmed +
                  cover_mod +
                  set_rescor(FALSE),
                data=keeley,
                cores=4, chains = 2)

## Compiling the C++ model

## Start sampling

First, if we wanted to evaluate the one missing claim, we could have looked at the rich ~ firesev relationship in the full model.

fixef(rich_fit)

##            Estimate Est.Error    2.5%ile   97.5%ile
## Intercept 54.205538  7.197803 40.1750869 68.8146233
## firesev   -2.558515  1.021273 -4.6423633 -0.6419759
## cover      9.783821  5.276849 -0.8659768 19.9671382

We can see the 95% Credible Interfal doesn’t overlap 0. First clue that we need it.

Second, let’s look at WAIC and LOO

WAIC(k_fit_brms, fit_brms_fullmed)

##                                 WAIC    SE
## k_fit_brms                    768.47 15.52
## fit_brms_fullmed              773.11 16.89
## k_fit_brms - fit_brms_fullmed  -4.64  5.57

LOO(k_fit_brms, fit_brms_fullmed)

##                                LOOIC    SE
## k_fit_brms                    768.48 15.52
## fit_brms_fullmed              773.14 16.90
## k_fit_brms - fit_brms_fullmed  -4.66  5.57

So, either way, both say that the partial mediation model is better, but the difference between the two overlaps. This is subtly different from AIC results from frequentist approaches - not sure what’s going on here, but super interesting. I’d still say that credible interval from the basis set suggests that the partial mediation model is the way to go, however, if this was a confirmatory model analysis.

Prediction

This is one of the things I’m more excited about here. Let’s say we start with a fire severity value of 5. How does this predict plant species richness? How do we bring uncertainty to the picture? There’s no automation here, so we have to calculate paths out ourselved - something I’m hoping to handle in piecewiseSEM and cam maybe port over here.

We start simply by getting cover. Note, non-terminal endogenous variables need to have an NA as their values.

newdata = data.frame(firesev = 5, cover=NA)

cover_pred <- fitted(k_fit_brms, newdata=newdata,
             resp = "cover", nsamples = 1000,
             summary = FALSE)

median(cover_pred)

## [1] 0.6528961

posterior_interval(cover_pred)

##             5%       95%
## [1,] 0.5966115 0.7087388

Boom! A prediction and the CL given coefficient error. OK, let’s let this go to richness. Note, we’re going to expand.grid to incorporate uncertainty, and so when we get a matrix back, we need to group by initial exogenous variables. In this case, we only have 1, so we can just as.vector the whole thing, but, for more complex newdata, you’ll need to do some more processing.

OK, so, let’s get the terminal endogenous variable. Note, the as.matrix(diag()) bit is to keep the number of simulation consistent. Otherwise we’re just kicking up our number of sims quite a bit and we muck about with how we handle uncertainty.

newdata2 <- expand.grid(firesev = newdata$firesev, cover = as.vector(cover_pred))

rich_pred <- fitted(k_fit_brms, newdata=newdata2,
             resp = "rich", nsamples = 1000,
             summary = FALSE)

#to minimize excess uncertainty
rich_pred <- as.matrix(diag(rich_pred))

#visualize
plot(density(as.vector(rich_pred)))

median(as.vector(rich_pred))

## [1] 47.68215

posterior_interval(rich_pred)

##           5%      95%
## [1,] 45.2057 50.16215

And there we go!

What about propogating true prediction uncertainty (e.g., other forces influencing cover, and the residual error in richness)? To do so, instead of using fitted we would use predict and otherwise do exactly the same

Further directions

This is an amazing starting point. There’s a lot more to explore. Is an omnibus Dsep possible/advisable in a Bayesian framework, or are we really more down to just model comparison? Can brms fold in latent variables? Or composites? I’ve been working on multigroup analysis in a piecewise framework, and, the ideas track over here pretty well (lots of 2-way interactions and then some model comparison), so I’m not too worried there. Ditto categorical variables - they fold in here nicely.

What’s perhaps the greatest thing here is that brms is SUPER DUPER flexible in terms of model forms and other fun details - correlation structures, random effects, lots of error distributions, and more! It’s worth digging through their https://cran.r-project.org/web/packages/brms/vignettes/ to see all you can do or surf on over to their Google Group at https://groups.google.com/forum/#!forum/brms-users for more.

Introducing the Open Derby (guest post)

Jarrett Byrnes — Fri, 31 Mar 2017 04:00:00 GMT

Ensuring openness and reproducibility in ecology is an ongoing challenge, although open tools (e.g., R Studio, Github) are free to use, and user support is everywhere. To illustrate how ecology collaborations can fit within an open science framework, we had the idea of running an ‘Open Derby’. Similar to a research derby, where grad students and post-docs from different academic departments and universities collaborate together to solve a conservation problem, our idea extends the derby concept to working fully openly: learning version control in GitHub, writing and analysing data in RMarkdown, and publishing all project development in an open notebook. Our end goal is to examine a conservation issue using open access (OA) data, and publish our findings and code in an OA journal. Ultimately, we hope to share Open Derby with other graduate students and academics across the life sciences, with the hope that we can provide a model for switching entire lab groups into open and reproducible research machines.

If you are not familiar with open science tools, RMarkdown is a wonderful method of writing code into a manuscript that can be formatted into manuscripts, reports, and presentations. We use RMarkdown in tandem with GitHub, a service that allows version control of your data, analyses, and manuscripts, and is incredibly useful for managing input between collaborators. Learning these tools can be daunting but learning with friends makes it easier and more fun to build these skills. We are six PhD students from 4 institutions based in 3 countries, who are present or past members of Dr. Julia Baum’s marine ecology lab at the University of Victoria and have come together to demo the Open Derby concept:

James Robinson: PhD student in the Baum lab, studying macroecology of tropical Pacific coral reefs (defending April 2017) and leader of the Open Derby project

Danielle Claar: PhD student in the Baum Lab, studying the dynamics of coral symbioses in response to environmental stressors

Easton White: PhD student at the University of California, Davis. He uses mathematical and statistical techniques to address various ecological and conservation questions.

Jillian Dunic: PhD student at Simon Fraser University, studying the effect of multiple stressors on tipping points in eelgrass. (Recently graduated from Jarrett Byrnes’ lab)

Jamie McDevitt-Irwin: MSc student in the Baum lab evaluating how stressors, specifically human disturbance and bleaching stress from an El Niño event, influence the community structure of the coral microbiome (defending April 2017).

Geoffrey Osgood: PhD student in the Baum lab, using data and models to study shark populations in South Africa and on Pacific reefs

In January 2017, we met (virtually) to decide on a research question for Open Derby. Our aim was to address a question that would be accessible under an open science platform (e.g. open source data). And SPOILER, we failed in our first attempt. In the interests of openness, and publishing null results, here is what we did:

Our workflow

Tools used: Rmarkdown, Github, Slack, etherpad, Skype

As mentioned earlier, we used Rmarkdown and Github to manage our workflow. We met once every one or two weeks on Skype to discuss project goals and delegated tasks, taking notes on Skype calls in a publicly accessible etherpad. These meetings, combined with Github, allowed us to divide and conquer without duplicating each other’s work. The goal was to have everyone only spend 1-3 hours on the project per week (after all, we each have dissertations to write). Everyone contributed to coding in R—whether it was data cleaning, analysis, or visualizations—and writing the initial manuscript. We also used Slack to regularly chat about the project in between Skype meetings. Slack eliminates the need for endless email chains and also integrates with Github, allowing participants to be notified when a repository change has occurred. In the future, we might take notes on a markdown file that can be easily version controlled, and may make use of Github’s wiki and project planning features.

Open datasets - oil spills and fisheries catch

We extracted oil spill data from the Emergency Response Division of the National Oceanic and Atmospheric Administration (NOAA; https://incidentnews.noaa.gov/raw/index) which recorded all major spills that occurred between 1967-2016 in marine environments on the west coasts of U.S.A. and Canada (Figure 1). However, we were unable to find oil spill data from British Columbia (BC). After extensive web searches, emails and even a freedom of information request, we were told there is no existing electronic record of BC coastal oil spills.

oil_map_spillsize

Figure 1 Oil spill location and size across west coast of North America from 1967-2016 included in NOAA’s incident news dataset.

We extracted fisheries catch data from the Sea Around Us Project (SaU), which collates global, spatially-explicit estimates of annual fisheries catch at the national level (Pauly & Zeller 2015; Zeller et al. 2016). The SaU dataset uses international and national catch statistics to generate taxon-level catch estimates at a 0.5 degree resolution, spanning 1950-2013, and provided us with California, Oregon, Washington, BC, and Alaska fisheries catch. Yet, as promising as this data appeared at first, we were advised not use this data at spatial scales below the exclusive economic zone (EEZ) level, as data are aggregated. We also contacted Canadian and US fisheries departments to request port-level catch data.

Sadly, data resolution issues sunk our project. With fishing catch data, smaller spatial aggregations of the SaU dataset were simply EEZ level trends with catch estimates scaled downwards (Figure 2), and we did not reasonably expect to detect oil spill impacts at the EEZ level. We were also unable to locate consistent port-level data that spanned either BC, California, or Alaska. Similarly, the oil spill incidents were not consistently reported, and it was unclear how spill estimates were generated - a contact from NOAA warned us that there was no established methodology for estimating the size of an oil spill, and many reported incident volumes involve some amount of guesswork.

Figure 2 Catch estimates for USA west coast EEZ (top panel) and 3 randomly selected 0.5 degree resolution grid cells (bottom 3 panels, labelled with lat-lon centroid of grid cell) from 1970-2010.

Beyond data issues, the ecological impact of an oil spill is highly context-dependent, and will be affected by the environmental conditions, clean-up response, and type of oil contaminant. As a result, our understanding of oil spill impacts on marine ecosystems is, sadly, limited to large, catastrophic events. We urge more openness and data collection of spill characteristics, so that researchers may begin to quantify the cumulative impacts of smaller, more frequent spills on marine ecosystems.

Future Work

You can find our ideas, data, and progress in our public Github repository. We welcome any suggestions that might help us to resurrect this project, and hope to inspire others to write up their failed projects. Our project lead - James Robinson - is currently developing Open Derby for general consumption as a member of Mozilla’s Open Science Leadership training programme.

But, we are now brainstorming conservation questions for our next attempt. Ecologists, please, send us your unanswered conservation problems, unanalyzed datasets, and suggestions for advancing open ecology! This time we will each bring a dataset and question to the brainstorming session. We will let you know how we progress!

Which library to use to teach linear models & likelihood

Jarrett Byrnes — Mon, 09 Jan 2017 05:00:00 GMT

I’ve been teaching likelihood to grad students for the past few years using the wonderful bbmle package from Ben Bolker. I really love it’s flexibility and the way you can get students into fitting anything their imagination can countenance. But there’s a small cost to that. First, their imagination, when they first meet likelihood, is….not ready. I teach linear models using least squares, then likelihood, then Bayes, in order to show the full range of inferential possibilities. In the past, when I got to likelihood not only did I showcase the possibilities for fitting a straight line with a normal error distribution, but I also inserted the possibility of other error distributions and at least hinted at the possibility of other shapes of lines.

I also taught them the canned “formula” interface to the mle2 function as well as how to write a function for a model - thus introducing them to the wild world of optimizers.

In this year’s iteration, I’ve been trying to pair a lot back, and move from teaching loops and the like to teaching dplyr and a more functional programming style. We hadn’t even hit functions by the time we hit likelihood (and didn’t ever - but that’s partially due to this being v 2.0 of my intro class, and functions will be hit in v 2.1). So I only taught straight lines with a normal error distribution using the formula interface.

It…went OK. Given that we move next into rstanarm for Bayes, the lack of a common syntax with both least squares and our Bayesian efforts was a bit more jarring this year. Particularly given the need to supply start values, coefficient names, and the issues one quickly runs into when you need to put limits on parameters, like standard deviations.

So in thinking about what to do, I went back to glm as an easier way to introduce fitting linear models with likelihood - particularly as then it’s an easy slide to stan_glm. But bbmle provides some nice tools - it’s implementation of profile is so straightforward - although I make students do that by hand anyway. In glm, the resulting profile plots are kinda wonky and take too much explanation.

But…. then someone pointed out the offset argument that lets you fix. So, let’s say you have the following data frame.

library(dplyr)
adf %
  mutate(y=rnorm(100, 5*x, 90))

You can fit this with glm and check out the coefficients

summary(glm(y ~ x, data=adf, family=gaussian())

But, what if you want to get the profile likelihood for the intercept? Using offset, you can fix a coefficient value. Say, to 1 or 2, for the slope.

glm(y ~ 1, offset=2*x, data=adf)
glm(y ~ 1, offset=1*x, data=adf)

Vundebar!

Now, we can wrap this in with dplyr. I’m going to do this inefficiently (fit the model twice) as I’m trying to figure out how to do this without map. Which also was not a huge success - I need to give it more time. Functions and map. These are what needs to be hit next year a bit more solidly.

### Fit a bunch of slope coefficients to make a profile
### for the intercept
coefFrame %
  rowwise() %>%
  mutate(int = coef(glm(y ~ 1, offset=b1*x, data=adf)),
         LL = logLik(glm(y ~ 1, offset=b1*x, data=adf))[1])

plot(LL ~ b1, data=coefFrame)

rplot03

Works great!

Now we can get the upper and lower 95% CI.

critVal %
  filter(LL>critVal) %>%
  slice(c(1,n()))

Again, straightforward.

Hrm.

Given that I’m not using the full power of bbmle, and getting into the nitty gritty of algorithms, etc., slows things down - and that by jettisoning that weight I think I can fill it with a jump right into AIC rather than wait until we hit with MLR, this simple example makes me thing that next year… yeah, it’s time to introduce the GLM function sooner and leave bbmle for semester 2.

Note, you can do this with lm and an offset argument as well. But, by introducing, as I ponder this, I think that perhaps we can begin to think about non-normality earlier? Maybe? I need to ponder this.

Code in https://gist.github.com/jebyrnes/37f01b1c231c17e7dfff888ca5392825

Linear Model Power Analysis with Dplyr

Jarrett Byrnes — Tue, 18 Oct 2016 04:00:00 GMT

I really liked the power analysis example I gave in class today. I feel like I’m finally firing on all cylinders with the tidyverse. So I thought I would post it here, in case it was of use to anyone!

Power

OK, we can fit and evluate linear regression models, but what about their power? Let’s look at our seal fit again - maybe we want to know what would have happened with a lower sample size?

Note, I’m going to use a knitr function, kable to neaten up the table for markdown. Try it in your own homeworks!

knitr::kable(tidy(seal_lm))

term estimate std.error statistic p.value

(Intercept)	115.7667505	0.1763751	656.3668	0
age.days	0.0023706	0.0000447	53.0645	0

What about the residual SD?

knitr::kable(glance(seal_lm)[,1:3])

r.squared adj.r.squared sigma

0.2256493

0.2255691

5.6805

All right - there are our target values. Now, we can change a lof ot things. The effect size (slope), range of values, sigma, and more. But let’s try sample size.

1 Make a Table of Parameters, some of which vary

The first step of a power analysis is to setup a data frame with all of the different possibilities that you might want to assess for power. For linear regression, we have the following parameters, all of which might vary:

Slope
Intercept
Residual variance
Sample Size
Range of X values

To do a power analysis via simulation for a linear regression, we begin by building our simulation data frame with the parameters and information that varies. In this case, sample size.

library(dplyr)

set.seed(607)

simPopN <- data.frame(slope = 0.00237,
                      intercept=115.767,
                      sigma = 5.6805,
                      n=10:100)

Note, if we wanted to vary more than one parameter, we’d first create a data frame where only one parameter varied, then add a second that varried using crossing in tidyr, like so:

library(tidyr)

simPopN <- data.frame(slope = 0.00237,
                      intercept=115.767,
                      sigma = 2:6) %>%
  crossing(n=10:100)

2 Expand to have a number of rows for each simulated sample with each parameter combination

OK, once we have everything in place, including sample size, we need to expand this out to have some number of samples for each n. For that, we can use the function in tidyr (same library as crossings), expand().

library(tidyr)

simPopN <- simPopN %>%
  group_by(slope, intercept, sigma, n) %>%
  expand(reps = 1:n) %>%
  ungroup()

3 Expand to create repeated simulated data sets for each combination of parameters

Now, if we want to simulate each of these, say, 100 times, we need to assign unique sim numbers, so for each n and sim number we have a unique data set. We can use crossing to replicate each combination of variables above some number of times. Note - 100 is really low, but it doesn’t eat your processor! Use low numbers of simulation for development, then crank them up for final analysis.

simPopN <- simPopN  %>%
  crossing(sim = 1:100)

3 Simulate the data

Great - almost ready to go! Now we just need to add in fitted values. Fortunately, as rnorm() works with vectors, we can just use a mutate here. We’ll also need to simulate random draws of ages, but that’s just another random number.

simPopN <- simPopN %>%
  mutate(age.days = runif(n(), 1000, 8500)) %>%
  mutate(length.cm = rnorm(n(), intercept + slope*age.days, sigma))

Yatzee! Ready to run!

4 Fit models and extract coefficients

First, we now need to generate a lot of fit models. Dplyr doesn’t take too kindly to including fit things, so, we can use two powerful functions here - first, nest and unnest() allow us to collapse grouped data down into little pieces and re-expand it.

fits <- simPopN %>%
    group_by(slope, intercept, sigma, n, sim) %>%
    nest()

fits

## # A tibble: 9,100 × 6
##      slope intercept  sigma     n   sim              data
##                           
## 1  0.00237   115.767 5.6805    10     1 
## 2  0.00237   115.767 5.6805    10     2 
## 3  0.00237   115.767 5.6805    10     3 
## 4  0.00237   115.767 5.6805    10     4 
## 5  0.00237   115.767 5.6805    10     5 
## 6  0.00237   115.767 5.6805    10     6 
## 7  0.00237   115.767 5.6805    10     7 
## 8  0.00237   115.767 5.6805    10     8 
## 9  0.00237   115.767 5.6805    10     9 
## 10 0.00237   115.767 5.6805    10    10 
## # ... with 9,090 more rows

Second, the map function in the purrr library allows us to iterate over different levels or grouped data frames, and perform some function. In this case, we’ll fit a model, get it’s coefficients using broom. This is a new weird set of functions. What’s odd about map is that the first argument is a column. But that argument, for the rest of the arguments of the function, is now called . and we also use the ~ notation. What ~ does is says that, from this point forward, . refers to the first argument given to map.

In essence, what map does is iterate over each element of a list given to it. Once we nest, the data column is now a list, with each element of the list it’s own unique data frame. So, map works with lists to apply a function to each element. The output of the model fitting is another list - now called mod. A list of models. We then iterate, using map over that list to generate another list of data frames - this time of coefficients.

library(purrr)
library(broom)

fits <- fits %>%
    mutate(mod = map(data, ~lm(length.cm ~ age.days, data=.))) %>%
    mutate(coefs = map(mod, ~tidy(.)))

fits

## # A tibble: 9,100 × 8
##      slope intercept  sigma     n   sim              data      mod
##                              
## 1  0.00237   115.767 5.6805    10     1  
## 2  0.00237   115.767 5.6805    10     2  
## 3  0.00237   115.767 5.6805    10     3  
## 4  0.00237   115.767 5.6805    10     4  
## 5  0.00237   115.767 5.6805    10     5  
## 6  0.00237   115.767 5.6805    10     6  
## 7  0.00237   115.767 5.6805    10     7  
## 8  0.00237   115.767 5.6805    10     8  
## 9  0.00237   115.767 5.6805    10     9  
## 10 0.00237   115.767 5.6805    10    10  
## # ... with 9,090 more rows, and 1 more variables: coefs

Last, we cleanup - we unnest, which takes list-columns from above, and expands them out as into full data frames that get slotted back into our original data frame. Nice trick, no?

We’ll also filter for just the slope coefficient.

fits <- fits %>%
  unnest(coefs) %>%
  ungroup() %>%
  filter(term == "age.days")

fits

## # A tibble: 9,100 × 10
##      slope intercept  sigma     n   sim     term     estimate    std.error
##                                   
## 1  0.00237   115.767 5.6805    10     1 age.days 0.0004283944 0.0010318274
## 2  0.00237   115.767 5.6805    10     2 age.days 0.0021239104 0.0005736028
## 3  0.00237   115.767 5.6805    10     3 age.days 0.0033531879 0.0009399613
## 4  0.00237   115.767 5.6805    10     4 age.days 0.0032566962 0.0008342080
## 5  0.00237   115.767 5.6805    10     5 age.days 0.0028332982 0.0009062293
## 6  0.00237   115.767 5.6805    10     6 age.days 0.0028501936 0.0005176723
## 7  0.00237   115.767 5.6805    10     7 age.days 0.0021677718 0.0004457464
## 8  0.00237   115.767 5.6805    10     8 age.days 0.0034636535 0.0006116823
## 9  0.00237   115.767 5.6805    10     9 age.days 0.0012147324 0.0012945056
## 10 0.00237   115.767 5.6805    10    10 age.days 0.0039061730 0.0011307423
## # ... with 9,090 more rows, and 2 more variables: statistic ,
## #   p.value

5 Calculate your Power

Notice that we do indeed have p-values, so we can use these fits to get power for each sample size. We can now do our normal process - in this case grouping by sample size - to get power. And then we can plot the result!

pow <- fits %>%
    group_by(n) %>%
    summarise(power = 1-sum(p.value>0.05)/n()) %>%
    ungroup()

qplot(n, power,  data=pow, geom=c("point", "line")) +
  theme_bw(base_size=17) +
  geom_hline(yintercept=0.8, lty=2)

6. Examples

Let’s take a look at the whole workflow, this time trying a bunch of standard deviation values.

##setup parameters
simSD <- data.frame(slope = 0.00237,
                      intercept=115.767,
                      sigma = seq(2:6, lengthout=5),
                      n=100) %>%
  group_by(slope, intercept, sigma, n) %>%

  #add sample sizes
  expand(reps = 1:n) %>%

  #add sims
  crossing(sim = 1:100) %>%

  #add fake data
  mutate(age.days = runif(n(), 1000, 8500)) %>%
  mutate(length.cm = rnorm(n(), intercept + slope*age.days, sigma))

##make your fits
fitsSD <- simSD %>%
    group_by(slope, intercept, sigma, n, sim) %>%
    nest()%>%

  #mapping
  mutate(mod = map(data, ~lm(length.cm ~ age.days, data=.))) %>%
  mutate(coefs = map(mod, ~tidy(.)))%>%

  #unnest and filter
  unnest(coefs) %>%
  ungroup() %>%
  filter(term == "age.days")

##calculate and plot power
powSD <- fitsSD %>% group_by(n) %>%
    summarise(power = 1-sum(p.value>0.05)/n()) %>%
    ungroup()

qplot(n, power,  data=powSD, geom=c("point", "line")) +
  theme_bw(base_size=17) +
  geom_hline(yintercept=0.8, lty=2)

You Complete Me

Jarrett Byrnes — Fri, 05 Feb 2016 05:00:00 GMT

There’s a feature that is either new or I haven’t noticed before in the new release of tidyr that is making me giddy with giddiness. The complete function.

This seems odd. Why would just one function make you giddy, Jarrett? Why not all of them?

It’s making me so happy as it solves a problem that just took a substantial degree of thought and waaaay to many lines of code to solve in a massive meta-analysis of kelp abundances I’m working on with colleagues. See, when folk go out and sample multiple species, they often behave in funny ways. Sometimes, when they don’t see any individuals of a species, they’ll mark it off as a 0. Sometimes they won’t, and will just not record even the name of the species.

The former case is great. The later case can be a pain in the touchus to fix, as all other identifying information for, say, a plot, site, year, etc. needs to be kept constant for the new blank value you are inserting. It took a few tries, but I ended up with a kludgy function that would fix this issue, but it was slow, particularly for data sets with hundreds of plots and 30-40 species (we were looking at other algae, too).

But along comes complete to make it all quick, painless, and only take a line or so of code.

So, let’s see an example of a ragged dataset where zeroes are just not included.

kelpdf <- data.frame(
  Year = c(1999, 2000, 2004, 1999, 2004),
  Taxon = c("Saccharina", "Saccharina", "Saccharina", "Agarum", "Agarum"),
  Abundance = c(4,5,2,1,8)

)

kelpdf

##   Year      Taxon Abundance
## 1 1999 Saccharina         4
## 2 2000 Saccharina         5
## 3 2004 Saccharina         2
## 4 1999     Agarum         1
## 5 2004     Agarum         8


So, Agarum wasn’t recorded in 2000. Maybe it was NA, maybe it was 0 - ask your data provider! Assuming it was 0, how do we fill it in?
library(tidyr)

kelpdf %>% complete(Year, Taxon)

## Source: local data frame [6 x 3]
##
##    Year      Taxon Abundance
##   (dbl)     (fctr)     (dbl)
## 1  1999     Agarum         1
## 2  1999 Saccharina         4
## 3  2000     Agarum        NA
## 4  2000 Saccharina         5
## 5  2004     Agarum         8
## 6  2004 Saccharina         2
OK, what happened there? Well, complete looked at all possible combinations of Year and Taxon. When there was a missing combination, the remaining columns got an NA.
Cool!
But, as I said before, we wanted to have zeroes, not NAs. Well, for that, we have a fill argument.
kelpdf %>% complete(Year, Taxon, fill = list(Abundance = 0))

## Source: local data frame [6 x 3]
##
##    Year      Taxon Abundance
##   (dbl)     (fctr)     (dbl)
## 1  1999     Agarum         1
## 2  1999 Saccharina         4
## 3  2000     Agarum         0
## 4  2000 Saccharina         5
## 5  2004     Agarum         8
## 6  2004 Saccharina         2
Great! We can have different kinds of fills for different columns. This is perfect, as we often had 2-4 measurements of abundance, but only some should be zero. Others should have been NA, as they were never recorded.
But, there’s another problem complete solves. Sometimes, we deal with subsets of data - say we’ve only pulled out certain taxa - but we know that there is more data there. Let’s say in 2001-2003, these sites were sampled, but there was no Saccharina. Yes, in this subset of a larger data frame we have no rows - but we know we should!
For that, we can use the full_seq function in combination with complete. full_seq takes a vector of numbers, sorts them, and then based on the period you specify, fills out the sequence. In our case, this is 1 year. So, to fill in those zeroes…
kelpdf %>% complete(Year = full_seq(Year, period = 1),
                   Taxon,
                   fill = list(Abundance = 0))

## Source: local data frame [12 x 3]
##
##     Year      Taxon Abundance
##    (dbl)     (fctr)     (dbl)
## 1   1999     Agarum         1
## 2   1999 Saccharina         4
## 3   2000     Agarum         0
## 4   2000 Saccharina         5
## 5   2001     Agarum         0
## 6   2001 Saccharina         0
## 7   2002     Agarum         0
## 8   2002 Saccharina         0
## 9   2003     Agarum         0
## 10  2003 Saccharina         0
## 11  2004     Agarum         8
## 12  2004 Saccharina         2
Note we had to define (or overwrite, really) our Year column, but no worries there. Now look at all of those lovely zeroes. Ah, so nice. I wish this had been around for the last two years… so much head bashing saved.



Other Writings about Parenting & Climate Change by Scientists
Jarrett Byrnes — Thu, 21 Jan 2016 05:00:00 GMT
 

  
    I'm a Chordata! Urochordata!
    Marine Data Science for All, Regardless of Backbone!
    
      About
      Categories
      Twitter
      Lab Site
    
  






OK, so, I spoke too soon. There are at least a few stabs by scientists writing about being a parent who studies climate/environmental change. As folk send me more, I’ll try and keep this list updated. While there’s some great stuff here, I feel like this is an empty niche for some great writing. Maybe you want to contribute, too? Towards that end, anyone who has a yen, I will be happy to post, repost, or link your own writings here under my science parent category.
OK, the list:
A World for my Daughter by Alejandro Frid (hat tip to Jenn Burt)
A Rational Optimist’s Reluctant Case for Panic by Tom Webb
Scaring our children with the future by CJA Bradshaw
Should Climate Change Stop Us from Having Babies? by Eric Holthaus



 


Parenting as a Scientist in the Age of Climate Change
Jarrett Byrnes — Thu, 21 Jan 2016 05:00:00 GMT
 

  
    I'm a Chordata! Urochordata!
    Marine Data Science for All, Regardless of Backbone!
    
      About
      Categories
      Twitter
      Lab Site
    
  






An old friend with a new child asked me if I know any climate change scientists or global change biologists who are parents and have written cogently about being parents in the era of climate change. Off of the top of my head, I couldn’t think of any (prove me wrong, internet), but, heck, I’m one myself. Here I am, staring down the barrel of hitting one year since HD’s arrival.
I’ve actually had a lot of conversations about climate change and our daughter with my wife - ethically, what does it mean to have children knowing what we know about the future of their earth? How many children is it fair (to them!) to have? And within myself, I ask, what should I be doing? Both what should I do to try to make the future not suck as much as it might for her and how do I help her prepare for the world that will look so very different than the one my own parents handed me.
I’ll admit, I don’t have many good answers. Particularly not as the parent of a kiddo just shy of the end of her first year. Why is that?
One reason that climate science communication is oft said to fail is the huge scale of imagination needed to grapple with the implications of climate change in the future. Climate scientists often possess some key personality traits that help them with this understanding - moreso than the general public[1]. Add the daily insanity and of managing a tiny human and what it does to your brain, and my brain-box is just not always up to wrestling with these issues until the moment that they will become pressing. Granted, this may be due to being the parent of a babe (with the power) and scientist parents of older children might have some better answers. I hope so, as this is definitely a looming challenge that I have not yet been able to navigate with forethought.
But that kind of punting things down the road is not fair - not fair to her.
So, what are the issues to consider? Here are the ones that always rise to the top of my mind.
First - is it ethical to be having children at all? For me, that one was straightforward. Yes. I do want to pass on the legacy of the family that gave rise to me. I think that there is some great culture and history that has brought me to where I am today. And I think some of that moxie, honestly, will be useful to the world in helping navigate the way forward. I mean, damn, that’s an egotistical thing to say, but, I want my genes to sally forth and help create someone who will help heal the world in some way. The alternative is to fold, and go home. As a species. Not an option. Or see this as someone else’s kid’s problem, not mine. That’s the kind of ethical stance that got us into this problem in the first place. So, also, nope.
Second - what can I do to make a better future? Welp, as someone who studies global change biology, I hope I’m doing it. I hope that I’m building the ship of knowledge that we need to sail forward as a species. I hope that the choices I make - personally and politically - all help improve the world and our path into the future. I don’t see myself doing anything differently or with more or less urgency than I did before. But that’s because of the values I hold about the importance of my actions for how the impact the whole world and future generations.
And finally - how do I prepare my daughter for the the world she is inheriting? Well here’s where my I feel completely inadequate to the task. I want to imbue her with the same love of the sea (and nature in general) that I have so that she appreciates the possible. I want to help her embrace the philosophy of helping to heal the world that was such a strong part of my philosophical upbringing. But, again, I would have done these things climate change or no. When do we need to talk about why this problem exists in the first place? How do I teach her about making proactive choices that lessen her own risk to the dangers of climate change? How is this different than how I would have raised her before? What are the lurking issues that I am missing, that I know my mind slips around, avoiding them because they are so large that I cannot see them when it comes to my own child? To her future, once I am gone? Should I be helping her learn more skills for self-reliance that I might not have otherwise? Should I be giving her more cynicism and skepticism of the systems around her that have failed us all, or the pragmatism to know that they can be the solution? Are there actual tangible skills or ideas I should make sure to teach her?
Or should I just muddle along as a normal parent, confronting these issues as they arise with the best judgement and sense of ethical responsibility that I can teach as I would have anyway?
As you can see, I have nothing but questions.
But in writing this, I do feel comforted that a lot of my own answers come down to just raising a pragmatic ethical kid who loves the natural world and embraces the responsibility of being part of a species that constantly seeks to better itself.
Is that enough?
Some Interesting Literature Cited [1] Weiler, C.S., Keller, J.K., Olex, C., 2012. Personality type differences between Ph. D. climate researchers and the general public: implications for effective communication. Clim. Change. doi:10.1007/s10584-011-0205-7



 


Rafe Sagarin and Passion in The Life Scientific
Jarrett Byrnes — Mon, 01 Jun 2015 04:00:00 GMT
 

  
    I'm a Chordata! Urochordata!
    Marine Data Science for All, Regardless of Backbone!
    
      About
      Categories
      Twitter
      Lab Site
    
  






How to process something like Rafe Sagarin’s senseless passing? For those who never encountered him, Rafe was a wonderful marine ecologist whose body of work was as varied and eclectic as can be, from classic papers on long-term effects of climate change in Science and Ecological Monograph to books on how the natural world can help us deal with risk, like terrorist attacks, to working at Biosphere2. He’s even got a wonderful youtube channel that works, as he always did, outside of the box.
Rafe was one of those vivacious dynamic iconoclastic people that you are always delighted to encounter in life. I was fortunate to know him through the Western Society of Naturalists where his charm and vim earned him the annual slot of auctioneer to raise funds for students. I really only knew him as colleague at meetings, and yet he is someone who has served as an example for myself and so many other early career scientists.
It was his passion. Every talk I heard him give, every conversation we ever had - scientific, personal, or with others about the difficulties of being early career - every interaction I saw him have with anyone around him, all of it was infused with an infection passion. His passion for the natural world was infectious. It infused everything. It was in his voice. It was in his worldview. It was in how he shaped his talks and storytelling. It was in his damned silly shirt he wore at every WSN.
Rafe’s uncompromising passion for science and nature was an inspiration. I looked forward to seeing what he thought of next because there is no joy in the world like watching someone follow their passion. And now I will not be able to again.
But that passion. That is what I will always hold onto. Because the life scientific can get you down as it grinds forward. But holding onto our passion, our vim, our vigor, our joie de vivre, rooted in our love of the natural world - that is what can provide an endless wellspring of joy in every moment. It’s something that has helped me forge onwards in my own career and in my life in general.
Rafe will always serve as an example of that passion and the myriad of directions it can lead. I will hold onto that. I hope that his passion will long serve as an example for our whole community. It will for me.



 


Spicier Analyses with Interactive R Leaflet Maps
Jarrett Byrnes — Tue, 31 Mar 2015 04:00:00 GMT
 

  
    I'm a Chordata! Urochordata!
    Marine Data Science for All, Regardless of Backbone!
    
      About
      Categories
      Twitter
      Lab Site
    
  






Who wants to make a kickass, public-friendly, dynamic, online appendix with a map for their papers? ME! (and you, right?) Let’s talk about a cool way to make your data sing to a much broader audience than a static image.
Last time, I mentioned that I had also been playing the Rstudio’s Leaflet package. For those who don’t know, Leaflet is a javascript library for generating interactive maps. I don’t know javascript (really), but I do know R, so this library is incredibly powerful for someone like me - particularly when paired with Rstudio’s HTMLwidgets or Shiny for publishing things.
So, what is leaflet? How does it work? Let’s start with a simple example. First, install the thing!
if (!require('devtools')) install.packages('devtools')
devtools::install_github('rstudio/leaflet')
Now, at its core, leaflet makes maps. And then puts data on them. It uses the magrittr for pipes as syntax, which may be a bit daunting at first, but I think you’ll see that it’s all prety straightforward.
Let’s start with a super-basic simple map.
library(leaflet)
leaflet() %>% addTiles()
Oh! Map! Scrollable! Easy to use! And you can zoom down to streets. What’s going on here?
First, leaflet() starts the whole process. You can feed it arguments of data frames and such for later use. Second, addTiles says “OK, go to a map server that feeds up image tiles, and grab ’em!” It defaults to OpenStreetMap, but there are others. For example, ESRI has a tile server with satellite imagery.
leaflet() %>%
  addTiles(urlTemplate="http://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}")
There are a number of arguments and functions you could string after addTiles to zoom around, start in a certain area, manipulate the basic map widget in different ways, but I’d like to focus on two common applications for folk like us. First, site maps! Let’s say you had some awesome data from over 100 sites around the world, and you wanted to show it off. Maybe you wanted people to really be able to zoom in and see what the environment was like around your data (i.e., why I went and found the ESRI tile server). How would you do this?
Well, leaflet provides a few different ways to put this stuff on a map. You can add circles, markers, circlemarkers, and much more. Let’s give an example on a basic map. Note the formula interface for supplying values to latitude and longitude.
#fake some data
set.seed(100)
myData <- data.frame(Latitude = runif(100, -80,80),
                     Longitude = runif(100, -180,180),
                     measurement1 = rnorm(100, -3, 2),
                     Year = round(runif(100, 1975,2015)))
#map it!
leaflet() %>%
  addTiles() %>%
  addCircles(data=myData, lat = ~Latitude, lng = ~Longitude)
Awesome! But we know there’s information in that thar data. There’s a Year and a Measurement. How can we communicate this information via our map? Well, there are a few ways. Note those circles had a color and a size? We can use that. Circles and markers are also clickable. So, we can add some additional information to our data and feed that into the map. Let’s start with a popup message.
#round for readability
myData$info <- paste0("Year: ", myData$Year,
                      "
 Value: ", round(myData$measurement1, 3))
Note the use of some HTML in there. That’s because leaflet displays things in a web browser, so, we need to use HTML tags for text formatting and line breaks.
OK, great. Now what about a color? This can get a bit trickier with continuous values, as you’ll need to roll your own gradients. Kind of a pain in the ass. If you just wanted one or two colors (say, positive or negative), you can just do something simple, or even feed a single value to the color argument - say, “red” - but let’s get a little fancy. The classInt library is perfect for gradient construction. Here’s an example. We’ll start with grabbing a palette and making it a color ramp.
#grab a palette
library(RColorBrewer)

pal <- brewer.pal(11, "Spectral")

#now make it more continuous
#as a colorRamp
pal <- colorRampPalette(pal)
Great! Now we can use classInt to map the palette to our measurement values.
#now, map it to your values
library(classInt)
palData <- classIntervals(myData$measurement1, style="quantile")

#note, we use pal(100) for a smooth palette here
#but you can play with this
myData$colors <- findColours(palData, pal(100))
Now that we have info, colors, and, well, let’s say we want to make the circles bigger - let’s put it all together!
#Put it on a map!
newMap <- leaflet() %>%
  addTiles() %>%
  addCircleMarkers(data=myData, lat = ~Latitude, lng = ~Longitude,
             color = ~colors, popup = ~info, radius = 8)

newMap
Very cool! With different properties for zooming, filling, sizing points, different styles of markers, and more, I think the possibilities for online apendicies for papers - or outreach objects - are pretty endless here. Go forth, ye, and play!
Also, check out the code in this gist!