Background
One of the most important skills in occupational hygiene that’s not often talked about is being able to identify what the problem is and choosing an appropriate strategy that will answer the problem. It is often easy to fall into routines when it comes to assessing a risk - we’ll do some monitoring, do some observations of work, make some recommendations on controls, and go home. We cannot address the core problem if we do not dig deeper than surface level. This example is taken from Bayesian Workflow (2026) - Chapter 4 by Gelman et al. and is not an occupational hygiene example but does provide an interesting exercise in problem identification, while also practicing statistics and probability which is always an added bonus!
The Problem
You are running an experiment to test people’s honesty. Each person is given a fair coin. One-by-one everyone flips their coin in complete privacy before reporting their face to you. If they report flipping a heads, they are given $10. Everyone knows all this before starting.
Suppose 171 participants play this game, and 111 of them claim the prize. Calculate the proportion of participants who are dishonest. By dishonest, we mean an individual would say heads no matter what they flipped.
Imagine that you have 10 new participants. How many should you predict will claim the $10?
Initial thoughts
Before considering what the question is asking, a few things are already immediately obvious: 1. If everyone is honest, we should expect about half of the participants to flip a heads and therefore half the people to claim the $10 prize. 2. If everyone claims the $10 prize, then there are probably a lot of dishonest people. Though not necessarily everyone is being dishonest. There may be some honest people who get a heads and rightfully claim the prize.
The expected split of heads and tails for a coin is 50% but we know that there is going to be noise. We can calculate the range of the expected number of heads by using a binomial distribution B(n, p) where n is the total number of flips and p is the probability of getting a heads (for a fair coin, p = 50%). Using B(171, 0.5) we would expect to see 85.5 (SD±6.5) heads on average, with the full binomial distribution below:
The probability of getting the observed 111 (or more) heads given that everyone was honest is 0.006%. So yes, we probably have some dishonest people! However, the problem was not to identify whether dishonest people existed, but to estimate how many there were. This raises the second obvious point that identifying dishonest people might be tricky because some people who were planning to be dishonest didn’t have to lie. They flipped a heads naturally…
Pause
I implore you to stop and think about how you would go about solving this problem. Don’t worry about doing calculations. Just think about what would make sense before reading ahead. On the face of it, the problem might seem easy, but finding the right strategy and getting to the core of the problem is trickier than you might first think as you go.
The Traps
If we’re too quick to react, there’s lots of ways we can go wrong. One way might be to think that because we don’t know any information about the individual flips it’s therefore impossible to know how many people are being dishonest. And yes, it’s true: we can’t tell if any individual is being dishonest, but we have already shown that we know that some people are being dishonest. We just need to think of a way of estimating how many.
Another trap might be to try and simply take the expected number of heads from the total number of heads, and assume that the additional heads must be from people who are being dishonest. But this doesn’t work because some dishonest people would have gotten heads anyway, and we are defining dishonest as people who would report heads no matter what. Therefore, this approach would have given us a biased estimate.
The Logic
In our group of 171 people, we have some number of honest people (\(H\)) and some number of dishonest people (\(D\)). The proportion of dishonest people will be \(D/(D+H)\), which we will call \(\theta\) (theta). This is the variable we need to answer our first question. Remember that we are classifying dishonest people as those that would lie, and not just those that did lie. But of course, we don’t know who’s who. All we know is how many heads were reported from the 171 people, and that these reported heads either came from an honest person who flipped a heads or a dishonest person (what they flipped doesn’t matter).
So all we can ask is: what is the probability of seeing a heads given some unknown probability of the person being dishonest? Writing out the above statement into an equation we get:
\[ P(\text{reported heads}) = (1 - \theta)(0.5) + \theta(1) \tag{1} \] If \(\theta\) is the dishonest people, then 1 - \(\theta\) are the honest people who report heads 50% of the time (due to random chance), while the \(\theta\) dishonest people report heads 100% of the time.
We can simplify the above equation to: \[ P(\text{reported heads}) = \frac{1 + \theta}{2} \tag{2} \] We can also calculate the reported tails to get the following. I’ll leave that to you to see why it’s true.
\[ P(\text{reported tails}) = \frac{1 - \theta}{2} \tag{3} \]
Ultimately we don’t care about the probability of reported heads (or tails), we care about the number of dishonest people and will want to solve for \(\theta\). Then getting \(\theta\) on its own from equation (2):
\[ \theta = 2\ * P(\text{reported heads}) - 1 \tag{4} \] Now, given a proportion of reported heads, we have an estimate of the proportion of dishonest people. Let’s test it. If everyone always reported heads, then \[\theta = 2\ * P(\text{reported heads}) - 1\] \[= 2\ * 1 - 1\] \[= 1\] meaning everyone is dishonest. If we always had half the people reporting heads, then \[\theta = 2\ * P(\text{reported heads}) - 1\] \[= 2\ * 0.5 - 1\] \[= 0\] meaning no one is dishonest. You might also notice that equation (4) can return a negative number. If we had seen 80 heads instead of 111, it would give us \(\theta\) = -0.06. A negative proportion? That makes no sense. So \(\theta\) must be constrained between 0 and 1, and in this case the probability mass would pile up near zero - “probably noone is dishonest” but there's always uncertainty…
However, this equation only gives us a single value and does not capture the uncertainty inherent in flipping coins. Instead, we can fully model the uncertainty using Bayes’ rule:
\[ posterior = \frac{likelihood * prior}{normalising\ constant} \tag{5} \] or more specifically: \[ p(\theta \mid y) = \frac{p(y \mid \theta)\,p(\theta)}{p(y)} \tag{6} \]
Prior Knowledge
Every unobserved variable in a Bayesian model, in this case \(\theta\), needs a prior. But we are not given any information about our cohort. As far as we know, they could all be liars or they could all be saints. Therefore, it seems reasonable that the prior should be equal for all possible values of \(\theta\) (from 0 to 1), which will look like the following:
Assuming no prior knowledge in this case is quite convenient for us because in the Bayes rule equation (6), p(θ) will equal 1 and can therefore be effectively ignored.
Likelihood
The likelihood is the probability of observing our data (reported heads and tails) given the possible values of \(\theta\). In other words, for any given value of \(\theta\) how surprised would we be to see our data of 111 reported heads from 171 people? Suppose \(\theta\) = 0 (everyone is honest). From the first graph, you can see that we would expect to see ~85 heads and would be extremely surprised to see 111 heads. On the other extreme, if \(\theta\) = 0.9, we would expect to see ~162 and also be extremely surprised to see 111 heads. While these two extremes are possible, the least surprising \(\theta\) (given 111 heads) is somewhere in between.
For our case, the likelihood can be written as: \[ p(y \mid \theta) = \binom{n}{y}\left(\frac{1+\theta}{2}\right)^{y}\left(\frac{1-\theta}{2}\right)^{n-y} \tag{7} \] \[ p(111 \mid \theta) = \binom{171}{111}\left(\frac{1+\theta}{2}\right)^{111}\left(\frac{1-\theta}{2}\right)^{171-111} \tag{8} \] Shown graphically:
Normalising Constant
The normalising constant (or marginal likelihood) is the least important part of Bayes rule conceptually. It just takes the likelihood and prior and makes sure it results in a posterior which is a valid probability distribution (everything sums to 1). For those interested, it is the integral of \(p(y \mid \theta)\,p(\theta)\)
\[ p(y) = \int_0^1 \binom{171}{111}\left(\frac{1+\theta}{2}\right)^{111}\left(\frac{1-\theta}{2}\right)^{60} \ *1\, d\theta = 0.0116 \tag{9} \] Don’t stress about understanding the integral above. In many other cases, it’s not directly solvable and a Markov chain Monte Carlo (MCMC) is used instead.
Posterior
Now we have everything we need: a prior, a likelihood and the normalising constant. Combining these using Bayes’ rule and solving algebraically or using an MCMC, we get our posterior. Remember, our posterior represents the plausibility of every proportion of dishonest people given our data. Our posterior is:
You’ll probably notice that the posterior graph and the likelihood graph look exactly the same in shape. The only difference is the y-scale because the posterior has been rescaled to be a proper probability distribution.
Answer to Q1
The answer to question one is the posterior above. As a summary, the estimated proportion of dishonest people is 30% with a 90% credible interval of [17%, 41%]. If you’re curious how many people actually lied rather than just being a dishonest person, remember that dishonest people still have a 50-50 chance of getting a heads or tails. Therefore, we would expect half this number to have actually lied.
Answer to Q2
We also have all the information we need to answer question two. It asks: if we have ten new people, how many of them should we expect to claim the prize (i.e., report heads)? We can re-use equation (1) below, and then substitute in our posterior for \(\theta\). We want to estimate how many heads to expect, so we can use the binomial distribution just like we did before. We also just calculated the possible ‘p’ values for a binomial. And our sample size for this question is 10. Creating a binomial distribution graph, just like we did before, we get the following:
Our best single guess would be that 7 out of 10 people will report heads. But again the awnser to the question is the full posterior predictive distribution. After all, there’s still only a 25% chance that 7 is the correct number. Only reporting the point estimate would oversimplify the matter.
Conclusion
Although this is obviously not a hygiene problem I do still think there are parallels to how we approach OH problems. It’s very easy to use standard and repeatable methods to attack a problem or simply do what a client, boss, or senior asks us to do. The point is to first think about what is the problem that actually needs to be solved and what approach would get us the answer to that question; which might not always be the first thing that you think of.
AI was used to create the HTML code for this post. The text and charts are mine. The problem comes from Bayesian Workflow as referenced.
