I tried to do a quick comparison of the COVID-19 testing rate on the William and Mary campus relative to the testing rate in Virginia. The ultimate goal of this is to (gu)esstimate the effect that more intensive testing of the W&M student body has had on the count of COVID-19 cases discovered.
Turns out, that’s a very hard thing to do, for a lot of different reasons. That’s not going to stop me from giving an estimate. But it does stop me from giving good estimate.
Best guess, the mandatory testing regimen at W&M ought to identify about three times as many cases as the voluntary, symptom-driven testing found in the community. The fact that it does not — that we don’t see a case count that is three times the community rate — probably reflects the self-selection of the W&M student body, and the fact that their pre-campus COVID-19 infection risk is (probably) much lower than that of the average 21-to-30-year-old Virginia resident.
For me, the bottom line remains the same: I want to see W&M improve, relative to that community benchmark, as the semester progresses. That’s how I’ll feel comfortable that W&M is controlling the spread of COVID-19 on campus.
Background
Source: Calculated from William and Mary COVID-19 dashboard, various dates, and Virginia Department of Health data on COVID-19 cases by age and date.
Recall that, both last semester and this semester, W&M students seem to have more-or-less the same rate of new COVID-19 infections as the average 21-to-30 year old Virginia resident. Last semester, the observed rate of infections on-campus fell relative to the Virginia average for that age group as the semester progressed. And it looks like the same thing is happening this semester (see chart above, data through 2/25/2021).
Neither situation — community (home) or campus — bears zero risk. But the implication of this is that sending your kid off to W&M wasn’t/isn’t particularly risky relative to keeping them home. At least, if you accept the heroic assumption that W&M students look just like the average Virginia 21-to-30-year-old.
But there is something obviously wrong with this simple analysis: W&M students get tested far more thoroughly, and far more often, than residents in the community. Testing is mandatory and appears to happen quite frequently. All other things equal, you ought to expect to see higher rate of diagnosed cases in the W&M group, owing to the large fraction of COVID-19 cases in the community that are never diagnosed.
And that surely is an error, but it’s mostly harmless in this context. If the question is “Is it safe to return to campus”, and I can say “yes” by using what has to be an over-estimate of the on-campus infection rate, then that’s OK. And then, in addition, if the true apples-to-apples comparison is actually better than I portrayed it, so much the better.
In other words, even ignoring the impact of more intensive testing, the on-campus infection rate looks O.K. And at some level, that’s a good-enough answer.
A crude estimate is often better than no estimate
But now, in addition, I’d like to take a hack at estimating the impact of all that additional testing. And I use the word “hack” advisedly here, because as near as I can tell, there is no good way to do this. But all I want is some plausible, reasonably sound number. If nothing else, some way to bracket the effect.
First, for the U.S. population as a whole, how big is the gap between diagnosed COVID-19 infections and all COVID-19 infections? Only a fraction of COVID-19 cases are formally diagnosed under a normal testing regimen. I’m going to use the all-US number for that fraction because a) I don’t think there’s a reliable Virginia number and b) I’m pretty sure there’s no number at all available by age group.
A recently-published analysis by CDC staff, using prevalence of COVID-19 antibodies in blood samples, produced an estimate of total infections that works out to be about 4.2 total actual infections per diagnosed infection (by my calculation, Post #933). But you need to be aware that there’s a lot of uncertainty there, for a lot of reasons, some of which are due to the numerous caveats associated with seroprevalence studies (Post #940). Other CDC staff estimates were as high as eight-to-one (See Post #933), using completely different methods.
Just for the sake of round numbers, I’m going to assume there are five actual COVID-19 infections for every one that is formally diagnosed, in the U.S., on average, under “normal” testing. Those additional infections will mostly be mild or even asymptomatic infections. So there would be four undetected infections for every detected infection. For better or worse, that’s the number I’m going to assume for my 21-to-30 Virginia resident comparison group.
How many of those undiagnosed cases would be found by more intensive testing? I’ll make this short: Nobody can tell you that, based on observational data. And that’s because, by and large, people get tested when they’re feeling sick.
Total tests and total diagnosed infections do, in fact, move in tandem, but the causality is from infection to testing, not the other way around. That’s because, by and large, individuals get tested when they have symptoms, and in fact testing guidelines in Virginia say that unless you have a good reason for it (e.g., symptoms or known exposure to an infected person), you shouldn’t get tested. Ergo, the more people with symptoms (because they are sick), the more people will get tested, if we all adhere to the guidelines.
It is true that many of the crazier Republican governors used the observed correlation of test rates and diagnosis rates to deny that increased case counts in their states were real. One after another, they blamed increased infection rates in their states on higher testing rates. See Post #864, The South Does Not Have a Monopoly on Goobers. The fact that one after another of them was proven wrong — that (e.g.) test positivity rate, ER visits, hospitalizations, and deaths all corroborated the fact of higher true infections — never stopped the next one from parroting that same line. But this was just a way to turn a blind eye to the pandemic. There never was a factual basis for saying it. It was and is a straight-up use of post-hoc-propter-hoc to sow uncertainty, and I doubt that any of them were actually dumb enough not to understand actual case counts were rising, despite their rhetoric to the contrary.
Well, can we start by determining how much more frequently W&M students get tested, compared to the Virginia average? The answer is yes, kind of. Even that’s not completely obvious.
For William and Mary, they’ve had 13,725 tests as of 2/25/2021. (Per their COVID-19 dashboard.)
Pre-arrival testing appears to be included in that count. It’s not crystal clear, but that’s my conclusion after looking at the data week-by-week.
The inclusion of pre-arrival testing complicates things, for two or three separate reasons. One, if the testing includes pre-arrival testing, then I would need to look at all positive cases, including pre-arrival positives. But in fact, up to now, I’ve only been interested in new cases that have popped up on campus. Two, if I include the pre-arrival screening, then it’s not clear what the relevant time period is, and by inference, the “person-days” of exposure. The testing period covers not just the time on campus, but some portion of the period just prior to arrival on campus.
So I’m making a command decision here and just chucking out an estimate of pre-arrival testing. That way I have a figure for tests and for person-days of exposure that match. So I’m netting out one test for each of the 6392 students that have been tested, leaving an estimated 7333 tests performed on-campus so far.
I estimate that, as of that date, they’ve had about 183,000 person-days of on-campus exposure (accounting for the fact that only 70% of total enrollment is on or around the Williamsburg campus, and accounting for the phased move-in this semester.) Or (100*7333/183000) = 4 tests / 100 students / day. Or, in round numbers, everybody gets tested once a month.
I don’t actually need to know the Virginia testing rate, for the rest of this, but I was curious to see how it would compare to the W&M on-campus rate, there is no age-specific testing data available. All I can do is calculate the analogous rate using tests for the entire population of Virginia, then hazard a guess at an age adjustment. (There’s also a minor issue in that Virginia counts “testing encounters”. Technically, that ignores multiple tests of the same type for the same person on the same day. I don’t think that matters.) Taking the last 28 days, there were about 944,000 test encounters in Virginia, excluding blood antibody tests. And so, the analogous calculation for Virginia’s 8.5M residents yields (100*944,000/(28*8,500,000) = ) 0.40 tests / 100 residents / day.
Even though I can’t find any test counts by age, I would guess that the all-Virginia rate modestly understates the actual testing rate in the age 21-to-30 demographic that I am using as my comparison group. The reason is that most testing is done in response to symptoms, and so testing rates go hand-in-hand with actual COVID-19 infection rates. In Virginia, as of today, the 21-to-30 age group has about 1.6 times as many diagnosed COVID-19 cases per capita as the Virginia population as a whole. If I were to adjust the test rate in proportion, I’d get an “age-adjusted” test rate of (1.6 * .40 = ) 0.64 tests / 100 residents age 21-30 / day. Or, in round numbers, everybody gets tested once every half-year or so.
The upshot is that the William and Mary on-campus testing rate is maybe 6 times higher than the estimated Virginia community testing rate for the 21-to-30 year olds.
Now for the true guesswork: How many more COVID-19 cases would be diagnosed under mandatory once-a-month testing, compared to voluntary testing of (almost exclusively) people who are symptomatic?
And so, return to the estimate of five true infections for every one diagnosed under ordinary testing. The question is, what fraction of the other four would be caught by extraordinary once-per-month mandatory testing on the William and Mary campus?
To guess at that, I’m revisiting the high false negative rates of COVID-19 PCR tests (Post #859). In particular, this graph:
Source: “Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction–Based SARS-CoV-2 Tests by Time Since Exposure:, Kucirka, Lauren M, Lauer, Stephen A, Laeyendecker, Oliver, Boon, Denali, Lessler, Justin doi: 10.7326/M20-1495 Annals of Internal Medicine, May 13 2020, https://doi.org/10.7326/M20-1495
What the top graph is telling you is that in the first few days of infection, the standard nasal-swab PCR test does you no good at all. You won’t find the COVID-19 on the nasal surfaces yet. And so you fail to diagnose anyone who was infected that recently. Then, about a week into the infection, your odds of having a positive PCR test get pretty good. And then, as time passes, and the viral load falls, the test gets worse and worse at identifying those who are infected.
If I punch in the numbers and take an average, I find that if you pick a random day out of a 30-day month, you’ve got almost exactly a 50% chance of diagnosing an infected individual. In other words, for the cohort that’s depicted above, mandatory once-a-month testing would have caught half of the missing cases.
Yet another caveat is that this ignores everything past day 30. Some individuals will continue to test positive for an extended period, up to 90 days, even with zero symptoms of continued infection, per the nearly-unreable U.S. CDC guidance on isolation and quarantine. This, plus an assumption of immunity following infection, is why individuals are often exempted from re-testing for an extended period once they have tested positive. Individuals who do that will get flagged by mandatory testing even after 30 days have elapsed since infection.
For a younger population, with a higher fraction asymptomatic or mildly symptomatic, you’d probably miss more cases than for the symptomatic cohort depicted above. That is, best guess, W&M actual once-per-month testing is going to catch less than 50% of those missing cases. But I have no way to quantify how much. So that just remains a caveat.
Bottom bottom line: With all the assumptions made here, you’d expect mandatory once-per-month testing to yield three times as many positives as you would get from voluntary, symptom-driven testing in the community. That’s because you’ll catch the one severely symptomatic case under either approach. And the mandatory once-a-month testing will also catch half of the four missing mildly-symptomatic or asymptomatic cases not found under voluntary testing.
And, while the W&M count isn’t three times as high as the community rate, note that it’s not three times as high, from the start of the semester. In other words, this cohort of students appears to arrive with a below-average infection rate. And that infection rate remains below average, while they live on the W&M campus.
