Author: chogan@directresearch.com

Mea Culpa.  I’ve been tracking the pace of vaccination by looking at what is posted on the CDC website over time.  Just taking a snapshot every couple of days, and comparing the new snapshot to an older one.

Like so.  I like these snapshots because they provide a lot of detail that you can’t get elsewhere, over time, from CDC, such the age breakout.

Unfortunately, there’s a real potential for error there.  That’s because that count, as posted by CDC, is substantially incomplete.  And you only get a consistent time series if that lack-of-completeness stays stable over time.

The reason is that the CDC not only puts out new counts each day, it revises the old counts as well.  It adds in counts that took a while to get reported.  So what you see, day-by-day, is the number that have been reported up to that time.  But in fact, CDC will go back and add to those counts, especially for the last five reporting days.  There is not even any guarantee that older data will remain unchanged.

And so, while I can say that, as of today, at least 75% of the elderly have had some vaccine, a) I can’t accurately compare that to the snapshot two days ago because b) both the current and two-day-old numbers may have counts added to them as time passes.

If you look at the little bit of trend data that CDC will show you, like so, you can see that the figure total doses administered through 4/3/20201 and 4/2/2021 exactly matches the 4/3/2021 “snapshot” above.  And that the figure for 4/1/2021 is just slightly less.  And then the numbers begin to change.

And if you look at the detailed table underneath that (which is difficult to deal with), you can see that, sure enough, there are huge amounts of vaccine missing from the last reported day, by date of administration.

And I can quantify the amount of missing vaccine in any snapshot by comparing (say) my March 28 “snapshot” to the final, aged March 28 count.  The actual, final count of doses administered through March 28, 2021 (as of today) was 149.8 million.  The March 28 snapshot — the table that appeared on the CDC website on March 28 — showed 143.5 million doses.  That gap may not seem large until you realize that’s easily two days worth of growth in the numbers.  So the incompleteness of the “snapshot” number is on-order-of two days’ worth of growth.  So there is potential to mis-state the day-to-day changes if the level of incompleteness changes significantly over time.

I’m not sure that this has any material effect on what has been reported on the growth rate in vaccination, as long as nobody cherry-picks the numbers (see below).  As long as that error is constant, you can in fact just compare the current (and incomplete) daily snapshot to an older (and, you hope, equally incomplete) daily snapshot.  As long as you do that consistently, there shouldn’t be a problem.

Source:  CDC, same cite as noted directly above.

If I look at the detailed data, I can see that they tabulate it not only by date of administration, but by date on which the vaccination was reported.  Presumably, those numbers will not change.

But I have to say, those numbers look a lot lower than what I recall being reported by the news media.  In fact, the seven-day moving average only just now hit 3M doses per day.  But I am sure I heard some national commentators crowing about a 3.5M dose-per-day pace.  Which is something you’d get only if you looked at the successive snapshots.

In any event, I now fully understand what I’m looking at with those daily snapshots (first images above).  And I suspect that much of the apparent fluctuation in the daily increases was due to transient changes in the completeness of the snapshot data over time.

In fact, the CDC graphs the data above, when you ask for daily change.  And as you can see below, the reported data show a regular weekly fluctuation.  So if you were a news reporter, and wanted to present a sunny picture, you’d take the “by report date” series and just pick off the couple of days that always seem to have a lot of vaccines reported.

Source:  CDC, already cited above.

Which, in hindsight, is probably how I’ve been hearing all this incredibly cheerful, upbeat, big-numbers news about the rate of vaccination.  And, to be sure, it is going up.  But every (say) 3.5M vaccine day above is routinely and regularly offset by some 2M vaccine days in the same week.  Week in, week out.

In which case, the only figure that makes sense is the seven-day moving average.  And that just now managed to break 3M doses per day.

This is all readily-available public information from CDC.  The only reason I post this is that you can’t just go to the CDC website and see (e.g.) the a finely-detailed day-to-day history of the vaccination rate.  E.g., there’s no simple over-65 rate.  Hence the series of screen shots.  That may or may not give a good estimate of the day-to-day changes in the vaccination rate.

While I’m at it, I might as well point out one more thing. The CDC does show a graph of the vaccination rate by age, over time.  (I’m not sure that has always been there, but it’s there now).  But because of the vaccine doses that are missing from the most recent days, that will always look as if the vaccination rate is tailing off.  When, in fact, the flattening of the line at the very end is due, in part, to the fact that the last days of data are substantially incomplete, while the earlier data are not.

Source:  CDC, same cite as above.

My whole approach here needs a re-think.  And I’m going to keep an eye out for what gets reported.  If reporters are cherry-picking the high days, in something that has a regular weekly cycle, that’s unconscionable.