14-05-2025 07:08
Status: 01-drafting
Tags: bayes
Prior Selection
For people new to Bayesian Modelling, the question of how to select a prior is challenging. For people with more experience building Bayesian models, questions along the lines of “how do you know your priors are correct/good?” are common. Practical justifications do not work in practice. When a model is being deployed into a production environment, potentially influencing large-scale decisions, your stakeholders will seek more certainty than just your assurance that this is a “sensible” choice. Prior and posterior predictive checks are good ways to get started in gaining the trust of your stakeholders. However, disappointingly, I have always found writings on these checks to be quite hand-wavy.
I think the process of selecting prior can be broken down into three steps
- Determine your model
- Propose priors
- Validate model
If 3) is well-framed, then we can often end up invalidating our model, thus forcing us to return to 1) and repeat the process until we are happy. Such a process is commonly termed Box’s loop [@bleiBuildComputeCritique2014]
Determine Your Model
The first step is to build a model that describes how the observations you are working with could have been generated. This is the data generating process (DGP). The DGP is more formally a likelihood , and this allows you to reason about how plausible your data is, given some unknown parameters .
To concretise this, imagine that your data is a collection of 0s and 1s. In this instance, it may be sensible to specify your likelihood to be a Bernoulli distribution. Alternatively, your data may be customer basket values from the checkout of an online store. The data here is continuous and non-negative, so a sensible choice of likelihood may be the gamma or weibull distribution.
Propose Priors
Whilst we described the process of selection the likelihood distribution in Determine Your Model, we conveniently avoided any discussion around how to identify the value of . This is where