Background: Why breakthrough infections suddenly matter.
Two posts ago (Post #1189), I went into the new findings from Israel regarding breakthrough infections of the Delta variant. Their data suggest that by six months after the time of vaccination, the Pfizer vaccine has almost completely lost its ability to prevent infections with Delta. It still does a good job of preventing hospitalization and death, just not infections.
I’m not sure if that’s a real result, or just an artifact of the way in which Israel went about vaccinating people. Their sample size was small, and their results were odd in that younger people appeared to lose immunity at a much higher rate than the elderly.
The Israeli results aren’t from a clinical trial. They come from comparing the current infection rates of cohorts of Israelis who were vaccinated in January, February, and so on. The members of those cohorts aren’t randomly selected, but differ systematically. The earliest cohorts (the persons vaccinated first) focused on high-risk individuals, the elderly, and health care workers.
The upshot is that by contrasting cohorts of individuals based on month of vaccination, you aren’t looking solely at the effect of time-since-vaccination. You are also looking at the effect of being elderly, being at high risk, and working in the health care system. Plausibly, some of those other factors would influence your odds of being exposed to Delta and picking up a breakthrough infection.
Some aspects of their results suggest that at least some of what they observe is an artifact of who was selected for those cohorts. In particular, they found that immunity fades to a much greater degree among the non-elderly, which is the exact opposite of what you would expect, given the generally weaker immune response of the elderly. (That weaker response is why there are annual flu shots specifically formulated for the elderly, with an enhanced dose designed to stimulate those aging immune systems).
That said, the finding is out there. And if it’s true — if what the Israelis are seeing is in fact an indication that the vaccine’s protective effects fade profoundly within half a year — that has major implications for individuals and for our public health strategy.
But is it true, or just an artifact of their methods?
Excellence in public data: Virginia
Faced with something like this — some hazy finding, showing a huge and important effect, from a small number of cases, in a distant land, that nobody has seen before, using non-randomized data — you get the drift — my first reaction is to see if anybody else says anything even remotely similar. And I want to see that based on data that I understand and trust.
It seems to me that tracking breakthrough infections ought to be a piece of cake for U.S. states. As I understand it, state health departments know which individuals have been vaccinated. (There’s a caveat here for vaccines that flowed through various Federal programs, including the armed forces, Veterans’ Administration, and the U.S. Indian Health Service. But states distributed the vast majority.) For sure, state health departments know which individuals have had a positive test. I’d be shocked if both lists didn’t contain the Social Security Number (SSN). And even if not, name/gender/age/address is good enough to match up 99+% of those entries absent a unique identifier such as SSN. (I speak from experience there, because figuring out how to make such “soft” merges between data files used to be part of my job.)
In short, all a state needs to do is match up the list of the vaccinated and the list of the infected. The people who are on both lists constitute your breakthrough infections. You’ll miss a few — individuals who moved into or out of state, individuals whose cases were dealt with by Federal rather than state systems — but in most states, those exceptions should be a trivial fraction of the population.
And so, for months now, I’ve wondered why states haven’t done that.
Turns out, Virginia has. Virginia now has a web page devoted to tracking breakthrough infections. It’s titled “Cases by Vaccination Status”, but that’s breakthrough infections. And I sure wish other states would follow suit.
I’m going to take one paragraph to put in a plug for the Virginia Department of Health. I’ve been using Federal, state, and sometimes local government data sources for more than a year now, tracking the pandemic. Virginia’s public-facing data is head and shoulders above the rest. A lot of times when I’ve wanted to discuss a national issue, I “illustrate” it with data from Virginia. That’s because Virginia was the only place I could find the data files, publicly available, that would allow me to do it. When you see that — when the data meet the analytical needs — you know that the people creating the data are almost certainly the same as the people who are using the data. That’s how they end up providing usable data files.
In light of the Israeli findings, I would love to see Virginia’s data tabulated by month of vaccination. Even though those monthly cohorts were not randomly selected, I’d at least like to see whether or not the crude finding that appears in the Israeli data — that breakthrough infections become common by six months after vaccination — before considering the Israeli results further.
But let me try to do the next best thing. Let me at least look to see of those breakthrough infections are rising, as they plausibly should as the vaccinations age. If the Israeli findings are true and not spurious.
In any case, by looking at the Virginia data for the past couple of weeks, we can be reasonably sure that, so far, as of about a week ago, breakthrough infections were uncommon here in Virginia. This, despite a reasonably high fraction of the population being vaccinated.
Below, the breakthrough cases would be 1 minus the percentage shown. So, in this case, (1 – 98.54% = ) ~ 1.5% of infections were breakthrough cases for fully-vaccinated individuals. The remainder (98.54%) were among the un-vaccinated.
To interpret that, you need to realize that there’s considerable uncertainty around these numbers. It’s not “statistical uncertainty”, because this is a census of cases, not a sample. It’s more like “natural variation”, when small numbers of infections occur within a very large population pool. Each number is a bit shaky, so to speak, but not because we’ve drawn a sample. They are shaky just because there are so few of them and they may fluctuate from day to day.
In addition, you need to know that there is a strong age-related correlation in vaccination rate, hospitalization rate, and mortality rate. So you can’t just take this raw count and infer that (e.g.) the vaccines are better at preventing infection than they are at preventing hospitalization. That’s not true. Arguably, the reason you’re seeing breakthrough cases as a higher fraction of hospitalizations than of infections is that hospitalization is strongly concentrated in the elderly, who have a very high rate of vaccination. I’d have to age-adjust the infection and hospitalization numbers separately if I wanted to get a true apples-to-apples comparison of impact on infection versus impact on hospitalization.
What’s at issue with the Israeli findings is the infection rate. So let me just state this plainly, and do a bit of math. Almost all these infections are in adults, so let me focus on the adult population.
As of this most recent two-week period available, the fully-vaccinated population accounted for:
- 64% of the adult population.
- 1.5% of the infections.
Doing the math, that means that the observed effectiveness of the vaccines, against COVID-19, in Virginia, over this period, is:
(1.5/64) / (98.5/36) = <1%
(Ah, well, what I really mean to say is that the effectiveness is >99%. The chance of getting infected is <1% of the chance for a non-vaccinated individual.)
In Virginia, during this most recent time period, if you were vaccinated, your chance of having a COVID-19 infection was less than one percent of the chance faced by an un-vaccinated person.
That’s substantially better than the clinical trials found. So, no doubt there’s a behavioral aspect to number. The vaccinated aren’t chosen at random, but instead are drawn from the rational population possessed of common sense. The unvaccinated, by contrast, are largely a mix of the irrational and those ideologically-driven to reject the vaccine. Almost without a doubt, the unvaccinated are also the ones who reject COVID-19 hygiene.
And so, this is probably best interpreted as saying that if you’re vaccinated and adopt common-sense COVID-19 hygiene measures, your risk of getting infected is less than one percent of the risk faced by those who can’t be bothered to do either.
In Virginia. As of a couple of weeks back.
And so, whatever is driving those Israeli findings does not appear to have started happening here yet.
Now I need to ask a couple of more questions.
First, does this reflect the Delta variant? I’d say yes. I can’t find any direct measure of that, because CDC didn’t sequence enough samples to provide a state-level estimate for Virginia. But I can infer it from the fact that this period is squarely in the middle of the current upsurge in cases in Virginia.
The pale blue lines mark the start and end dates used in the breakthrough calculation above. Those increases didn’t really get going until Delta dominated, and Virginia is right in line with the rest of the South Atlantic states. The CDC shows that, during this period, about two-thirds of cases in this region were the Delta variant. Between those two pieces of evidence, I’m fairly confident in saying that the breakthrough rate above is largely reflective of Delta infections.
The next question to ask is, has this changed over time? That’s easy enough to answer. Let me set the dates to span an equivalent period on the downslope of the curve above, and see what the Commonwealth says the breakthrough infection rate was.
And the short answer is that breakthrough infection, as a percent of total, was actually higher at the start of June than it is now. That’s a time period when the Alpha variant was still dominant. Those accounted for 4.5 percent of total infections — in line with the clinical trials data — compared to less than one percent in the most recent period.
Here’s the kicker: If you download yet another one of Virginia’s data files, you can readily calculate that 24% of Virginia’s vaccine doses were administered before March 1, 2021.
In other words, somewhere around one-quarter of Virginia’s vaccinated individuals fall into the categories that should be suffering a massive loss of immunity to COVID-19 now, if the Israeli results are true. And yet, we are seeing no uptick in breakthrough infections. To the contrary, based on the two time periods I looked at, those breakthrough infections actually fell.
My conclusion, based on publicly-available data from Virginia, is that whatever is happening in Israel surely does not seem to be happening in Virginia. The Israeli findings shows a massive reduction in immunity for those whose immunizations were several months old. If true, given that almost a quarter of Virginia COVID-19 immunizations are five months old or older, given the estimated effect from Israel, that really should have started boosting the rate of breakthrough infections by now. And no such thing has happened.
This analysis could be done more cleanly by tabulating by date of vaccination, but that would require the person-level data that only the Commonwealth possesses. I hope they’ll take a quick cut at that and make the results now. Otherwise, these Israeli results would seem to through a monkeywrench into any planning for the pandemic.
Afterthought
I want to be clear that I think Israel’s Ministry of Health did the right thing in releasing their statistical analysis. In fact, I’d say they were ethically compelled to do so. And, based on the news reporting, the accompanying text (in Hebrew, which I cannot read) did mention all the relevant caveats, in that the monthly cohorts of the vaccinated were not randomly chosen.
It’s a tough call.
On the one hand, Israel was a couple of months ahead of most other countries. If this result were real, they’d be the first to have it show up in their national data. And if it were real, they really would be compelled to offer a warning to other countries that might be subject to the same loss of immunity within a couple of months.
On the other hand, you don’t want to make health care policy based on spurious results. (Though this would hardly be the first time that happened). Consider the the expense and hassle of providing booster shots on a semi-yearly basis to the entire vaccine-accepting population. Now consider the risk of doing that for no reason, if the Israeli result are spurious. (And I note that Israel itself has not yet decided to do that, based on their own results.)
So it’s a tough call. Alerting other public health agencies to this possibility seems like the right thing to do. The US CDC and FDA aren’t going to make any sort of snap decision on booster shots. They are going to gather the evidence, and make up their own minds. And, based on what I can see in Virginia, they are going to find that the Israeli results are not replicated in other places. That will tell them that the correlation observed there is an artifact of something about the Israeli experience and not a failure of vaccine-generated immunity. And the scientific method will have done the right thing in filtering out fact from fiction.
