Virginia is very much a middle-of-the-road state. We’ve had reasonable and sensible guidance from our governor (who is himself an MD) throughout the pandemic. The Commonwealth made disaster preparations early on, converting a dorm at one of the big state schools into a standby hospital facility. And we always seem to be in the middle of the pack in terms of infection rate, vaccination rate, mask use, and so on.
Not coincidentally, the Commonwealth of Virginia has consistently produced (IMHO) the best publicly-available data on the COVID-19 pandemic. In particular, they recently began summarizing “breakthrough” infections, that is, COVID-19 infections among vaccinated individuals.
They now allow the public easily to look at the data week-by-week, allowing us to isolate breakthrough infections in the Delta wave from Virginia’s prior experience. The data can be seen at this URL. (In theory, every state should be capable of doing this, because it only requires merging two files: vaccinated individuals and individuals with a positive COVID-19 test).
Breakthrough infections are an important topic these days, for a lot of reasons. If you are vaccinated, you need to understand them to know your risks. And as importantly, the forces of ignorance consistently cherry-pick and misinterpret data on breakthrough infections to generate anti-vaccination disinformation.
So it’s great that my home state has provided this nice, clean cut at breakthrough infections. And all I’m going to do in this post is take a look at it.
Spoiler alert: COVID-19 vaccines work pretty well.
Background 1: Clinical trial data show the Pfizer vaccine is 91% effective against the Delta variant.
Based on clinical trials data, for the first six months following vaccination, the Pfizer vaccine is 91% effective against symptomatic infection with Delta variant. That is, if you are vaccinated, all other things equal, you are only 9% as likely to get infected as an unvaccinated person. I summarized this in Post #1192, but it bears repeating:
Source: Six Month Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine,
The vaccine wears off over time. Once you develop full immunity, the efficacy of the vaccine in preventing a symptomatic COVID-19 infection is:
- 96% effective in months 1 and 2.
- 90% effective in months 3, and 4.
- 84% effective in months 5 and 6.
But, on the up-side, the vaccine is much better at preventing severe illness than at preventing any illness. Where it’s 91% effective against symptomatic illness, it’s 97% effectiveness in preventing severe disease. (I should note that “severe disease” is not the same thing as hospitalization, but it’s clearly correlated with that.)
There is an expectation that the Moderna vaccine is roughly that effective, and that the J and J vaccine is substantially less effective. (J and J accounts for a small fraction of total vaccine doses to date).
That said, the real world isn’t a clinical trial. People are not randomly assigned to be vaccinated or not. Instead, they choose. From the statistician’s point of view, it’s “observational data”. It’s what you observe to happen, based not just on the fact of vaccination, but also on how people behave.
And so, the observed rate of breakthrough infections will depend on the effectiveness of the vaccine, plus all of the other differences between the populations choosing to get vaccinated, and those choosing not to get vaccinated.
I guess that’s pretty obvious. This is the reason they do randomized clinical trials in the first place, because that’s the only reliable way to get rid of all those other confounding factors.
I won’t belabor that. Just consider the fact that a far larger portion of the elderly are vaccinated, compared to younger people. Even now, any infection in a person under 12 is going to be, by definition, not a breakthrough infection. The point is that any simple comparison of infections among vaccinated and unvaccinated may or may not yield the same results as the true (clinical trials) comparison.
Asymptomatic infections are another factor often overlooked in comparing real-world data to clinical trail data. The 91% effectiveness figure was against symptomatic infection. Nobody has a hard (randomized trial) number for prevention of any infection, including asymptomatic infection. But the observed data is a mix of symptomatic and asymptomatic infections. (Presumably, the asymptomatic ones were mostly found via contact tracing.)
The bottom line is that you hope and expect to see an observed breakthrough infection rate that’s somewhere around what you’d expect from the clinical trials. And that you don’t cry “wolf” unless you see a very large deviation.
Background 3: Vaccines give your immune system army an advantage. They aren’t an excuse for dropping all your other weaponry.
I went through this in Post #1200. The “immune system army” metaphor is not mine. Vaccines shift the odds, they don’t provide absolute protection. Give yourself a high-enough dose of COVID-19, and you’re going to get sick, vaccine or no vaccine.
Just to be clear, scientists routinely kill lab animals with viruses that are normally not lethal. They just make sure that the animals get a very large dose of the virus. That way, the virus is able to replicate out-of-control before the animal’s immune system “army” can mount an effective response.
The simple lesson is that there is a good reason to wear a mask in high-risk situations, even after you are vaccinated. The reduction of exposure, due to the mask, might prevent an infection where vaccination alone would not.
Background 4: Don’t forget the simple math.
If a higher proportion of the population is vaccinated, then a higher proportion of infections will be breakthrough infections, all other things equal. You can’t take a raw rate of breakthrough infections and know what it means unless you know the rate of vaccination in the underlying population.
This lesson was taught clearly in the Provincetown, MA outbreak, where it took me only a tiny bit of searching to find that that virtually all the residents (>95%) of Provincetown were vaccinated.
The popular press reported the high proportion of breakthrough infections there as if it were some sort of evidence of the lack of effectiveness of vaccines. When, in fact, it was primarily a result of nearly everybody being vaccinated.
That’s summarized in Post #1198.
The Virginia data.
The remaining graphs are taken direction from the Virginia Department of Health website, at this URL. Here’s the graph for the most recent week available:
Source: Virginia Department of Health, URL given above.
First, note that this is infections per 100K per week, not per day. So you’d have to divide the data by seven to get something comparable to the daily new cases data you are used to seeing. If you do that, you can see that in the week ending 8/14/2021, the rates boil down to:
- Entire population: ….. 21 new cases / 100K / day (per NY Times data)
- Fully Vaccinated: ……… 5 “
- Partially vaccinated: … 18 “
- Unvaccinated: …………… 43 “
You will find no finer illustration of the notion that this is now a pandemic of the unvaccinated. If we’d all get vaccinated, the situation would look a whole lot better.
But, to the point, Virginia does the per-capita math for you, double-circle in red above. For this one-week period, the unvaccinated had an infection rate that was 8.5 times higher, per capita, than the vaccinated. Assuming I did the algebra right, the observed effectiveness of the vaccine in that week was 88%.
That said, just by looking at the graph, you can see that something has changed over time.
First, there has been a slow but substantial decline in that ratio over time. If you look back to (say) the last week of the month, you would have found these estimate:
- January ….. 60 times higher rate in unvaccinated compared to vaccinated.
- February … 29 “
- March …… 16 “
- April ……. 18 “
- May ………. 15 “
- June ……. 7 “
- July ….. 5 “
To me, this is a classic example of how results from observational data will reflect not just the impact of vaccination, but also the risk and behavior of the vaccinated and unvaccinated individuals. Early in the year, the vaccinated consisted mostly of front-line health care workers, nursing home resident, and the oldest old. That population was vastly different from the Virginia average.
As time progressed, a much larger proportion of the population got vaccinated, and presumably the vaccinated and unvaccinated were not so hugely different in terms of (say) demographics and employment.
A second factor is also occurring, in that the effectiveness of the vaccines is wearing off over time. But the clinical trials data assures us that is a minor factor.
Finally, we made the switch from the Alpha variant to the Delta variant over this period, and the effectiveness of the principal vaccines is somewhat lower against the Delta variant.
(Aside: The only other place I’ve seen this type of huge apparent trend over time was in the data from Israel. The Israeli data was interpreted as showing a huge decline in vaccine effectiveness over time, and that’s what prompted Israel to start offering booster shots. As I said in my writeup of that (Post #1189), I think they were mostly seeing differences between the groups of individuals vaccinated early and late. I’m pretty sure that’s what we’re seeing here in this large secular decline.)
That said, if you take the four weeks prior to this final data reporting week shown above, the mean is only about 5.4. That is, apparent observed effectiveness of the vaccine in the prior four weeks was only 81%.
Source: Virginia Department of Health, URL given above.
To summarize: The Prizer vaccine is supposed to be something like 91% effective against the Delta variant, based on clinical trail data. The past week’s observational data from Virginia give an apparent observed effectiveness of 88%. The four weeks prior showed something like 81%.
Does any of that observational data make me think that the clinical trials data is materially wrong? Do I think, based on this evidence, that the clinical trials overstated vaccine effectviness against the Delta variant?
Nope. I spent my career analyzing health care data, and that sort of difference is commonplace, between an un-adjusted observed rate, and a “true” rate. Your best estimate for the effectiveness of the vaccine, right here and now, is still the estimate from the clinical trials data.
Finally, a note on publication bias: You’re never going to see that nice, normal Virginia finding from last week reported anywhere. It doesn’t make people frightened, or angry. It doesn’t contradict the accepted estimate of the efficacy of the vaccine. And so, it has no news value.
All you are ever going to see in the news is the bad outliers. For example, I am sure I could find some week in the Virginia data to cherry-pick, if I wanted to mislead people into thinking that the vaccines were failing. I’m sure I could take that time series — in bullet points above — and mis-represent it to try to make the same point of disinformation.
So, just have a care for what you read. Figures don’t lie, but liars can figure. That’s trite, but it sums up a lot of what you see circulating about vaccines and vaccine effectiveness for COVID-19.
