William and Mary has started another round of “census” testing, administering COVID-19 tests to all students on the campus. Yesterday, an email from the W&M administration said that test results should start appearing on the W&M COVID-19 dashboard. And they have.
Yesterday, W&M reported test results for 1448 students. They found no (zero) positives. Zero new COVID-19 cases, out of 1448 tested.
Any normal person would say, that’s fantastic. And if that puts you in a happy place, that’s great. Stop here and move on.
But my inner statistician started grumbling the instant I saw it. So I went back to make sure I hadn’t made a typo, and I started to calculate just how likely it was to find a true 0-for-1448, given recent history.
Bottom line: I did my little skeptical inquiry, and the 0-for-1448 result is plausible, but only just so. It’s certainly at the low end of what one could reasonably expect, based on recent history. But it might also reflect the incredibly rapid vaccination of the W&M student body over the past few weeks. As of April 13, 75% of the student body had received at least one dose of vaccine. (That’s a self-reported number, of a subset of 3900 students, and so should be taken with that uncertainty in mind.)
Plausibly, this 0-for-1448 is true, and reflects W&M coming close to or exceeding “herd immunity” against COVID-19.
Let me put it this way: If it comes up as zero, again, today, you’ve got two choices. Either 1) I doubt it, or 2) looks like mass vaccination actually works, and works fast. For now, I’ll just say, my inner statistician continues to grumble, but, properly calculated, a single day of 0-for-1448 remains plausibly true, based on the recent historical rate of infections.
Analysis follows.
A bit of skeptical inquiry.
This is a mostly-straightforward math problem. Given the rate of positives we’ve seen in the days leading up to this most recent result, how likely is it that we’d see a true 0-for-1448 out of this new census testing?
If you look at the table above, W&M had ten new cases in the eight days leading up to yesterday’s results. That was 10 cases out of a small number of tests. So the test positivity for the period was well over 1 percent.
But that’s not a measure of the incidence of disease. My guess is, what you’re probably seeing there, in those 10 new cases, is people who were tested “for cause”. That is, people who were symptomatic or had some other reason to seek testing. So that isn’t 10, out of some small number of tests. More likely, that’s 10, out of the entire student body.
Let me run with that. For the moment, let me assume that all ten cases that were caught, in the eight days leading up to yesterday, were all the symptomatic cases that popped up on the William and Mary campus over that period. That’s 10 symptomatic cases out of the entire on-campus enrollment, over that period.
If that’s true, then a) what’s likely rate of actual new COVID-19 cases per student per day, and b) how many cases would you then expect out of a random sample of 1448 students?
Cases with symptoms are only a fraction of all cases. So I need to inflate that count of 10 cases by something reasonable. Let me make a conservative assumption about the ratio of asymptomatic cases to symptomatic cases, and assume two asymptomatic cases for every symptomatic case. That’s higher than the U.S. average (about 40% of cases are estimated to be asymptomatic), but this is a younger population, and asymptomatic infections are much more common in that population.
From that, I can calculate that the daily incidence (new cases) of COVID-19 in the William and Mary student body was about (10 positive tests x 3 total cases for every positive test) / 8 days / 6610 total students on campus = .0006 cases per student per day.
But, when you do census testing like this, you aren’t just getting the students who happen to be newly symptomatic on that day. You are catching the entire cross-section of students who a) have been infected and b) are still able to trigger a positive on the PCR test. Accounting for the false-negative rate of the test, you really ought to catch about seven day’s worth of students, by testing them all on one day. That is, on average, from time of infection to time of recovery, the average infected student has a roughly seven-day window during which they will test positive, if tested. (N.B., that’s not literally true, that’s weighting the days by the probability of being able to trigger a test, so it’s more like 21 days, with a decreasing ability to trigger a test with each passing day. The math works out the same.)
And so, in 1448 cases, if that were solely census testing (just a random sample of students, and no students who reported in with symptoms on that day), you would have expected to see, in one day, 1448*.0006*7= ~ 6 positives.
In addition, there’s a non-negligible false-positive rate for these PCR tests. The lowest I can found quoted, based on real-world experience, is 0.2 percent (given in this reference). This won’t matter if W&M re-tests all positives before reporting them as positives. So this may or may not factor in. But at the average false-positive rate, if W&M is merely reporting the raw test results, that should have added a further 1448*.002 = ~ 3 additional positive tests.
Finally, there is the ongoing for-cause testing. From that, you would have expected to see slightly under 1 additional case, out of the remainder of the student body whose test results were not reported yesterday.
In short, I’d have expected to see closer to 10 positive tests, rather than zero. That’s six true positives, based on the rate for the prior eight days. Maybe three false positives. And maybe one more, tested for cause (symptoms), yesterday.
One possibility is that there has been a fundamental change in the incidence of COVID-19 on campus. Possibly, the true rate of infections has plummeted. In which case, calculations based on the days leading up to the census testing are obsolete.
The obvious source of such a change would be the high self-reported rate of COVID-19 vaccination among the student body. As noted above, 75% of William and Mary students have been vaccinated against COVID-19, as of a about a week ago. So there may well have been a complete change, and at 75%, it’s plausible that William and Mary has hit “herd immunity”. So this could well be a true change in the rate of incidence of the virus.
Another possibility is just the luck of the draw. Assuming I’ve done the math right, even if I’m expecting to see 10 new cases, there’s more than a 10% chance that I’ll actually see zero, on any one day. (For the stat nerd out there, that does not include finite-sample correction factor, despite the fact that 1448 is a large fraction of the total student body).
Based on that, you really can’t rule out a true 0-for-1448. Given the underlying rates, you might see that more than 10% of the time, even if the expected (average) value should be more like 10-for-1448.
A third possibility is that something about the testing or reporting is in error. That’s not as crazy as it sounds, because there was one prior episode of testing error. At one point last semester, W&M received a string of positive tests. W&M staff questioned the reasonableness of that, based on part on no uptick in findings from their monitoring of the sewer system. And those did, in fact, turn out to be false positives. So lab error is a possibility.
