The issue for this post is whether widespread use of home COVID-19 tests is materially reducing the official counts of COVID-19 cases. My wife has been asking me this question for weeks. Yesterday I got an email from a reader with the same question. This post is about scraping together whatever I can, to try to put some bounds on the extent to which home testing is perturbing the numbers.
Bottom line: Between September and October 2021, the gap between the official count of cases and estimates of the “true” count (based on evidence of prior infection) grew considerably. At that time, home COVID-19 tests were probably being sold at the rate of 100M per month. Those sales actually exceed the roughly 45M per month PCR tests being reported to the U.S. CDC. But test sales are not the same as test use, and nobody knows the extent to which those 100M at-home tests have been used, or are merely being held by consumers. For sure, this issue will become more important as the Federal government aims to get at-home COVID test distribution up to 300M per month by February 2022.
Lots of details follow.
There’s already a lot of slack in the official counts of COVID-19 cases.
First, you have to realize that the official COVID-19 numbers grossly under-count total cases anyway. I’ll demonstrate that below. So, from the get-go, you have to take the official case counts with a large grain of salt.
You know that this degree of under-count may well vary across states, over time, and across countries. It will vary based on both the availability of tests and population’s willingness and ability to get tested. If there is a shortage of tests — as there was in the U.S., early in the pandemic, due to the failure of the CDC’s first test for COVID-19 — then the under-count will be higher.
Further, the degree of under-count of the true number of infections might vary across COVID-19 variants. Plausibly, most of the under-count is persons with asymptomatic cases. If different COVID variants generate different proportions of asymptomatic-to-symptomatic cases, that should affect the under-count. And if COVID becomes concentrated in a population with a high proportion of asymptomatic cases (e.g., children), that will also affect the under-count.
The best you can hope for is that, for some reasonable periods of time, the degree of under-count remains relatively constant, for whatever you are trying to look at. That way, even if it’s off, as long as it is consistently off, you can make some reasonably valid comparisons over time.
The issue with the growth of home testing is whether or not it has introduce an increasing “wedge” between the true count of infections and the official count. If so — if the gap between actual infections and the official numbers is growing rapidly due to unreported results of home tests — that will distort metrics based on the official count of cases. That includes not just total infections, but also measures of severity such as case hospitalization rate and case mortality rate.
In short, we know the official counts are an under-count. The question is whether the size of that under-count is rapidly increasing due to widespread available of home COVID-19 testing.
How do we know there’s an under-count?
We know the official COVID-19 case counts are an under-count based on the CDC’s national lab seroprevalence survey. There, they use blood drawn for other purposes (e.g., routine blood panels) and test it for antibodies to COVID-19. Presence of antibodies demonstrates a prior or current COVID-19 infection. You can then compare the estimated fraction of the population with antibodies, to the reported official number of COVID-19 cases, to determine some measure of the under-count in the official numbers.
Last time I checked — roughly August 2021 — there was just shy of a 50% under-count, based on the seroprevalence survey data. At that time, there appeared to be about 1.9 actual COVID-19 infections for every one shown in the official case counts.
Seroprevalence surveys are not perfect. Putting aside the issue of whether or not the blood samples are representative of the population, those immunoassays have limited sensitivity, and immunity fades over time. Both of those factors suggest that, if anything, the true number of COVID-19 infections has been even higher than the seroprevalence surveys suggest.
Unfortunately, CDC changed methods in September 2021. Per their website:
Note that in response to recent data, 23 jurisdictions in the nationwide antibody seroprevalence survey switched to an assay with increased sensitivity to detect past infection in September 2021, which could impact trends.
The upshot of that is that in comparing data prior to September 2021 to the present, I can’t tell what part of that increase is real — a true increase in the undercount of infections — and what part is due to the use of a more sensitive test in the seroprevalence survey. That leaves me with exactly two datapoints, September and October 2021, as shown below:
Source: CDC seroprevalence survey web page accessed 1/12/2022
I’ll point out that this was a fairly stable time for COVID-19 in the U.S. All the cases were the Delta variant, throughout the period. The Delta wave peaked around 9/1/2021, and the winter wave had just started prior to Thanksgiving 2021.
It’s always a risk to make a lot out of two data points. That said, per the CDC’s analysis, the purely statistical uncertainty in their estimates is quite small. The “95% confidence interval” for October looks to be plus-or-minus one percent of the estimate. There may be structural (non-statistical) errors — e.g., maybe their new, more-sensitive test was not fully phased-on at the start of September. There’s no way for the outside observer to know that.
All I can say is, taken at face value, as of October of last year, there was an increasing gap between the number of infections estimated from blood antibodies in a sample of persons, and the official count of persons who had tested positive for COVID-19.
The increasing gap between actual infections and the official count could arise from any number of sources. We can’t rule out the rise of home testing as one of them. Yet.
The number of OTC COVID-19 test sales is larger than the number of PCR tests reported to CDC.
Fully realizing that a test sold is not the same as a test used, I have to start with some credible estimate of the rate at which at-home COVID-19 tests are being produced and sold. The bottom line — below — is that, right now, those home tests are probably being produced and shipped at a rate in excess of 100M tests per month.
As a benchmark, there are about 45M COVID-19 PCR tests reported to the U.S. CDC per month. (That’s my calculation, based on data from the CDC COVID data tracker). So that’s test that were actually performed, but would not include any results from antigen tests that might be reported to the CDC.
What I want to know, for starters, is how many cheap, quick (no lab involved), at-home, no-prescription (over-the-counter or OTC) tests get sold every month. Fully realizing that “sold” is not the same as “used”, so that’s not strictly comparable to the number of PCR tests reported to the CDC each month.
This assumes that any test that has to be sent to a lab, or is done by a medical provider, or is done at some official testing site, should have been counted in the official statistics. This also assumes that expensive home tests would see relatively little use.
The information is piecemeal.
As of this 1/9/2022 reporting, there were 11 at-home OTC test kits approved by the FDA. (This somewhat older reporting also lists 11 tests, but it’s a slightly different list of 11). The list keeps expanding. Many were only approved lately for at-home (OTC) use.
Of those, per the same reporting, the following tests do not require you to mail your sample in to a lab, and are cheap:
- Binax Now, $20 for 2.
- Quickvue, $23 for 2.
- Flowflex, $10 each.
- Ihealth, $23 for 2.
- Intelliswab, $23 for 2.
- On/Go, $25 for 2.
- BD Veritor, $34 for 2.
There are in addition, a few new ones that are not yet on the market, I think:
- Celltrion Diatrust, 10/21/2021 OTC authorization.
- SD Biosensor, 1/5/2022 OTC authorization
- Siemens Clinitest, 12/29/2021 OTC authorization
(The last three are from various DHHS press releases).
That may not catch them all, but it’s enough to get started. Abbott Binax Now and Quidel Quickvue were the first two rapid test approved for OTC sales, per this 12/21/2021 reporting from Vox.
The Abbott Binax Now test was first shipped to retail outlets in April 2021, per this press release from Abbott. Their plan at that point was to produce “tens of millions per month”. I believe that was among the first approved for OTC use. The Abbott test was reported to account for three-quarters of all OTC test sales, per this 12/21/2021 reporting from Vox.
The Quickvue test from Quidel test sold at a rate of greater than 20M tests per month in the 4th quarter of 2021 (Reference).
Putting those two together, the two market leaders probably sold in excess of 80M tests per month in the final quarter of 2021. (Assuming that the 75% Abbott market share estimate is correct).
This estimate is a good match for a Federal press release from October 2021. At that time, the Federal government was expecting to increase U.S. OTC rapid test capacity from a current 100M a month to 200M test per month by the end of 2021, rising to 300M per month by February 2022. (Source).
That increase is part of a concerted push by Federal authorities to get rapid home test to market quickly. At least, that’s according to those same authorities. At the minimum, they threw $70M into a fund to streamline the authorization process. Separately, the Federal government is issuing hundreds of millions of dollars of contracts for purchase of home tests.
In summary:
- Each month, the CDC records the result of about 45M PCR tests. Those are tests that were actually used.
- Each month, recently, U.S consumers have bought on-order-of 100M OTC COVID-19 tests.
So there are certainly enough OTC tests out there to be able to perturb the official data.
The big unknowns are the rate at which consumers are using those tests, and the rate at which they are coming back with positive results. That is, how many positives are we missing due to the presence of cheap and widely available home tests.
Is there any information on actual use of home COVID-19 tests.
There are a couple of issues here, but the bottom line is no: As far as I can tell, nobody can tell you the fraction of those home test kits that has been used.
The only way you are going to be able to estimate this is with some sort of large-scale survey. How else are you going to know whether or not they used the purchased tests. The question is, has such a survey been done, and have the results been made public? No. Or, if so, I can’t find it.
Beyond that, I don’t think it’s worth flogging the nuances of the question. The nuances being that your behavior toward cheap at-home testing might differ from your behavior toward more formal testing approaches.
If anything, that’s more-or-less the point of the current Federal initiative pushing at-home testing. They want people to test more often.
Summary:
For sure, Americans are buying a lot of OTC home COVID tests. Best guess, right now, sales of those test are two to three times the volume of PCR (DNA) tests that are being reported to CDC monthly.
For sure, the shortfall between the official counts of infections and the estimate of the true number of infections via seroprevalence testing grew in October 2021. It’s plausible, but premature, to say that that increasing gap is due to home testing.
The big unknown is the rate at which consumers are actually using those tests and finding positive results. Near as I can tell, the only way to know that is to ask them, via a large-scale survey. So far, I haven’t found any evidence that anyone has conducted such a survey.
The bottom line is that it’s plausible, but not proven, that the widespread availability of cheap OTC tests is suppressing the official count of COVID-19 cases.
