Post #815: What if this is as good as it gets?

Source:  Immunogenicity and protective efficacy of influenza vaccination
Claude Hannouna, Francoise Megas,  James Piercy,  Virus Research 103 (2004) 133–138.

The importance of this graph will be clear about five paragraphs down.

At this point, with the Phase III trials of coronavirus vaccines well underway, even if they don’t have enough “statistical power” to do the formal statistical test, our public health bureaucracy ought to have a fairly good indication of how things are shaping up.

I’ve been waiting for any US public health leader to start leaking information on the likely effectiveness of the coronavirus vaccines.   Informally tossing some numbers out there, to get us prepped for the eventual formal announcement.

We just got our first indication today.  And, although the CDC Director broke the news gently, and indirectly, and with spin, if you paid attention, the news was clearly not good. Continue reading Post #815: What if this is as good as it gets?

Post #807: A vastly simpler mask liner using Filtrete ™

Source for base data: 3M, ASHRAE.  See Post #593 for writeup of these and other filtration standards.

I’ve done several posts about making masks out of Filtrete ™ home air-filter material.  Each time I return to that task, I find simpler ways to work with Filtrete ™.

In this post, I’m going to show just how easy it is to take a Filtrete ™ home air filter and make some simple, flat, easy-to-handle, fiber-free pieces for use inside cloth masks.  In a nutshell, extract the Filtrete ™ fabric from the air filter and hot-glue it between two layers of the thinnest synthetic fabric you can find.

Then end-user can then cut it to size, for use as a liner for a cloth mask.  That seems to work just fine, and nothing more labor-intensive is needed.  The materials run well under $0.25 per mask liner, depending on what size filter you buy, and how large you cut your mask liners.

A few tips and tricks for doing that are given below.  Of which, the only one that might not occur to you is to use kitchen “parchment paper” as a non-stick surface as you are gluing.

Continue reading Post #807: A vastly simpler mask liner using Filtrete ™

Post #798: Dr. Fauci and statistical power

See a caveat about the very short trials at the end of this posting.  They exaggerate exactly how short this trials could be, because this table is based on “normal approximation” to the actual probability distribution.

Yesterday, Dr. Anthony Fauci correctly stated the one and only way that vaccine clinical trials can end early.  (Assuming that that the FDA was serious when it said that any vaccine approved for US use must pass all phases of its clinical trials.)

If the results are extremely good (or extremely bad), they could legitimately end the clinical trail before it was scheduled to end.  Because, if that happens, they would legitimately and accurately be able to pass judgment on the vaccine, given the available data.  And so, if the results are extremely good, they could approve the vaccine ahead of schedule.

That’s totally legit.  And there is significant precedent for it.  Clinical trials have been stopped before, and drugs given approval, if they are shown to be clearly effective against some life-threatening disease.  In some sense, that’s every drug manufacturer’s dream.  Google the phrase drug trial end early and you’ll see an entire scholarly literature on this topic.

I’m putting a marker down, in the form of this posting, for several reasons.

  1. It was refreshing to see a US public official get the math right.  (Note that he said very good or very bad, which is the fully accurate statement.)
  2. Ditto, making a clear and unambiguous statement of fact.
  3. It will be interesting to see what actually happens, as election day approaches.
  4. I get the sense that we’re finally coming around to the Russian view of vaccine implementation (Post #773), only our bureaucracy can’t admit to that.
  5. I like math, and figured I’d use this as an opportunity to do some back-of-the-envelope statistical power calculations that I had been meaning to do anyway.

In other words, with point 5, I’d like to make some reasonable guess as to just how good the vaccine would have to be, to allow trials to end appreciably earlier than scheduled.  Just so I have some sense of whether they are making things up or not, if (when?) they cut the Moderna (US) vaccine trial short later this year.

And I’ve now done that, shown at the top of this post.  Fauci is absolutely right.  If these vaccines are effective, it won’t take very long to demonstrate that.  That’s what I find, calculating it from the ground up.

And so, if the vaccines are effective, and they cut the trials short, that’s not really a shortcut.  That’s legit.  And that’s not and excuse for not getting vaccinated.

And as an odd side note, the need for a statistical test effectively bars marginally-effective vaccines from being marketed.  (Assuming they do their statistical tests legitimately.)  If a vaccine is only 60% effective, you can eventually show that it’s “statistically significantly different” from the FDA 50% threshold.  But with any luck the pandemic will be over by the time you do that.

Details follow.

Challenge trials are the joker in the deck

You may have missed this little news item that came out a couple of weeks ago.  It said that the Federal government was brewing up batches of COVID-19 virus to use in “possible” human challenge trials.

Challenge trails:  When you absolutely, positively need to know right now.  If you’re really in a hurry to see if something works or not, you vaccinate, dose your subjects with the infectious agent, and see what happens.  You “challenge” them with heavy exposure to the disease, typically one that would otherwise guarantee infection in an un-vaccinated individual.

With that approach, there’s none of this waiting around for nature to take its course.  That’s more of a slam-bang, count the bodies and be done with it approach.  At some significant risk to the participants. (Although, the writeup above suggests that it would still take months to set up and run, which I find hard to believe if it were done on an emergency basis.)

Challenge trials are not a new idea.  They’re not even a new idea for testing COVID-19 vaccines.  Even some of Our Statesmen in Congress figured this out, months ago.

But, interestingly, Fauci appears to dismiss that possibility completely.  Meanwhile, “government scientists” (not otherwise identified in the news reporting) are creating the batches of COVID-19 that will only be needed if there are challenge trials.  I wonder which one is right.  And I wonder who the unnamed “government scientists” are.

But for now, I’m dismissing that possibility.  If they were doing to do challenge trials, they should have done them months ago.

Approximate statistical power calculation

In Post #774, I showed some of basic arithmetic that explains why it is so hard to test a COVID-19 vaccine.

  • It’s a deadly disease, so you can’t just take a few hundred people, give half a placebo, and expose them to COVID-19.  (That’s a challenge trial, as discussed above.)
  • Instead, you have to take tens of thousands of people, give half of them a placebo, and wait to see who gets infected in the normal course of business.
  • Given the low rate of new infections, it takes considerable person-months of time to accumulate enough infections to provide usable data.

I did the arithmetic in that prior post.  The US vaccine is aiming for a 30,000 person clinical trial.   For the sake of argument, let’s assume that they instantly enrolled all 30,000 people.  If they split that 50/50 (vaccine/placebo, good enough to make this point), at Virginia’s current infection rate (12 new infections per 100,000 population per day), you’d only accumulate about 50 infections per month in the placebo group. 

But that might vary quite a bit, just by chance.  Might be 20, might be 100.  And if the vaccine works, you’d accumulate fewer than 50/month, on average, for the vaccine group.  But that also would vary quite a bit, just by chance.  And then, based on the small difference between those small counts, you’d have to decide how well you think the vaccine is working.

That is why the people conducting vaccine clinical trials:

  • Chase down COVID-19 hotspots and recruit people there.  That gives them more infections per person-month of exposure.
  • Use huge samples (30,000 persons).  That gives them more stable numbers (more “statistical power”) for a given infection rate.

The final thing you need to know is that the FDA set a floor of 50% effectiveness, which they defined as either preventing infection or reducing severity of infection in half of cases.  A vaccine must be proven to do at least that well before they’ll approve it.

As an aside, that “or reducing severity of infection” may end up being weasel-wording of the highest order.  I know how to count infected versus not.  That’s black and white.  But as far as I can tell,  there’s literally no one legitimate way to calculate that second part about having reduced the severity of infection in X% of infected cases.   Manufacturers will have to establish an arbitrary scale of severity (e.g., death = 1, hospitalized on vent = 2, hospitalized no vent = 3, symptoms but not hospitalized = 4, asymptomatic case = 5).  And then perform some arithmetic on the counts of cases in each category, and from that conclude that they reduced the severity of infection in X% of cases.  Near as I can figure, that last part is going to be some form of hand-waving.  It will be some arbitrary method, applied to that arbitrary scale.  That opens the opportunities for presenting your data in the most favorable light possible.  To put it nicely.

In any event, the FDA 50% floor gives us the “target” for any statistical test.  The clinical trial is going to come up with an estimate of effectiveness, and an estimate of how uncertain your are about that number, the so-called “95% confidence interval”.  In everyday press, you’d see that expressed as a plus-or-minus range.  (E.g., the estimate is 60% effective, plus or minus 3%.  That would mean that, based on the data, you’re 95% sure it lies between 57% and 63%.  And if you repeated that clinical trial many times, only one time in 20 would you see numbers like that, but the true (real) effectiveness of the vaccine was somewhere outside of that range.)

There’s also a little statistical cheat you can try to use here, called a “one tailed test”.  It would be entirely inappropriate in this case (because a priori, you don’t know if you’re going to beat that 50% threshold or not), and I assume that legitimate scientists at FDA would not accept that.  So I’m basing this on a standard two-tailed test.

With all that, and putting aside the part about reducing severity, it’s easy enough to mock up the resulting statistical test.  To see just how much data you would need, in terms of person-days of exposure, to be able to exclude 50% effectiveness from the 95% confidence interval.  Assuming you were able to perform you test in a place with, say 20 infections / 100K persons/ day.

So that’s my rough back-of-the-envelope word problem.  Suppose you have a clinical trial with:

  • 15,000 enrolled
  • in an area with 20 infections/100K/day
  • split 50/50 into vaccine and placebo groups.  (The 50/50 split isn’t optimal, but this is just a rough calculation).

For a given true vaccine effectiveness of Y%, roughly how many days does that trial have to go on before you can exclude 50% effectiveness from the 95% confidence interval around your estimate of Y?  In other words, how long until the vaccine passes the test?

Details that nobody cares about but me:  I’m also going to use normal theory approximations here, because I know how to set up the problem in a spreadsheet that way.  I don’t think that matters, as long as I have well over 30 infections in both placebo and vaccination groups. And I’m modeling the FDA threshold as a known number, when in fact, it’s going to be half the placebo group number, which is itself uncertain.   I’m also ignoring the fact that the test is expressed as a ratio, which can complicate the statistics when both numerator and denominator are estimates (not known numbers). Finally, I’m ignoring that the trial doesn’t really start until the second month, because the US vaccine requires two doses, one month apart.

I built up this calculation using the standard formula for the variance of a binomial (yes/no) variable.  Then I use the normal approximation and, in effect, calculate a set of standard t-tests.  As mentioned above, that’s one of many things that’s not precisely correct about this calculation.  But it should be close enough.

Results are given below.  It’s no surprise that the better the vaccine, the less time it takes to prove it.  (To exclude 50% effectiveness from the 95% confidence interval).  What is surprising, to me, is how much overkill a 30,000 person trial is, if you expect the vaccine to be at all effective.  I think this may explain why other vaccines, such as the British (Jenner Institute/Astrazeneca) vaccine are using more like 10,000 in their clinical trials.

As an extras for experts, some of those very short intervals would provide too few infections to allow legitimate use of the “normal approximation” that I used here.  So those almost certainly overstate how short the trials would have to be, at the very bottom of the table.  The rule of thumb is, you’d want to see at least 30 infections in at least one of the groups.  At the placebo rate, then, that sets a floor of about three weeks to satisfy that additional constraint.  But the gist of this is still correct.  You only need a half-year of clinical trials if your vaccine doesn’t do much.

Post #792: Mask use and asymptomatic cases

I just stumbled across this article (or, same thing, here) and thought it was well worth repeating.  If for no other reason that a) it’s about COVID-19, b) it’s incredibly logical and fact-based, and c) it explains something fairly important.

The gist of this is that:

  1. When you are exposed to COVID-19, the amount of COVID-19 that you breathe in is a strong determinant of how sick you’ll get.  The higher the initial dose, all things equal, the sicker you’ll get.  The lower the dose, the less sick.
  2. Masks greatly reduce the amount of virus you inhale, when you are exposed.
  3. The least sick you can get is an “asymptomatic case”, that is, a person who was infected but fought back the virus without having any symptoms.
  4. A high rate of mask use greatly increases the fraction of infections that are asymptomatic infections.

Continue reading Post #792: Mask use and asymptomatic cases

Post #790: Cigarette smoke does not work as a test for mask filtration ability

Way back in Post #750, 7/9/2020, I had the notion to use cigarette smoke to test the ability of masks to filter out aerosol-sized particles.  There is a need for some sort of home test, because it’s next-to-impossible to tell how well or poorly any off-the-shelf mask works.  That’s particularly true for the “KN95” masks now being sold in everywhere (Post #747).

The theory seemed sound.  Cigarette smoke particles are about the right size, and in the past, some people did in fact use N95 masks to try to avoid second-hand smoke.

Now, having executed this test on my back porch this afternoon, I can attest that it doesn’t work at all. I can smell cigarette smoke strongly right through a genuine (but quite old) 3M N95 respirator (upper left, above).  And I could not tell that the smell of smoke was any stronger when I used a worn-out 3M N95 dust mask (next), or a dust/surgical mask (blue) with no aerosol filtration capability. Continue reading Post #790: Cigarette smoke does not work as a test for mask filtration ability

Post #780: Making an aerosol-filtering respirator from a dust mask and Filtrete (r) 1900

This post shows one way to upgrade a common dust mask so that it filters aerosol-sized particles.  The basic idea is not exactly rocket science:  Cover it with something that filters aerosol-sized particles. Continue reading Post #780: Making an aerosol-filtering respirator from a dust mask and Filtrete (r) 1900

Post #750: Science alert: Proposed cigarette smoke test of “KN95” and other masks

Edit:  This did not work at all.  Not even a little bit.  See Post #790 for details.

In this post, I propose to rate masks on their ability to filter out cigarette smoke.  The particles in cigarette smoke are roughly the same size as virus/aerosol particles.  (Although their chemistry is quite different.)  I am going to use those second-hand smoke particles as my best proxy for human aerosol emissions. The basic idea being that if a particle-filtering mask is good at removing something as small as cigarette smoke particles, it probably does a pretty good job at filtering aerosols in general.

I’m announcing the full scope of the test ahead of time because that’s good science.  This way, I can’t just bury the results if they turn out unfavorable.  (AKA, toss them in the circular file.)  In some sense, it’s as important to know this doesn’t work, as it is to know that it does work, as a way to test masks.

The test is pretty straightforward:  Light a cigarette, hold it under your face, and rate how strongly you can smell cigarette smoke while breathing through the mask.  Less is better.

A cigarette is ideal for several reasons.  One, it provides a consistent burn, meaning, a consistent concentration of smoke particles from one mask to the next.  Two, it’s a readily-available and easily-repeatable standard.  Three, the smoke has been well-characterized.  Four, there is no risk (as with incense) that the material has been doused with molecules meant to volatilize as the base product is burned.  If you smell tobacco smoke, you are smelling smoke particles.  if you smell incense, it’s not clear what you are smelling.

I’m going to start with these five scenarios, hoping to establish some sort of scale:

  1. 3M N95 respirator (new old stock, should be true N95 filtration)
  2. Dust mask with two layers of Filtrete 1900 fabric (~ N85)
  3. Dust Mask with one layer of Filtrete 1900 fabric (~N60)
  4. Plain dust/surgical mask (N low).
  5. No mask (N00)

The first problem is, I still have to manufacture items 2 and 3.  I did a bunch of #3 masks for friends, early in on this pandemic.  I need to make some more, document that, and then go on to make #2 masks.  These should provide known filtration standards below N95.

Then I’ll rate the following six masks on that scale.  These are the masks whose performance I am trying to judge.  Starting with the beat-to-heck 3M dust mask that I have been wearing since the start of the pandemic, that I hope is still working well.  And then some other alternatives that are readily available to the US public.

  1. 3M N95 dust mask (extremely well-used)
  2. “KN95” mask #1, from Twins Ace Hardware in Fairfax
  3. “KN95” Mask  #2, from Twins Ace Hardware in Fairfax
  4. Generic single-use “surgical-style” mask #1.
  5. Generic single-use “surgical-style” mask #2.
  6. Plain-vanilla single layer cloth mask.

The point is to say whether or not you would materially improve your protection from aerosol-sized particles by swapping a plain-vanilla cloth mask for a typical  generic, non-certified “KN95” mask offered as an impulse item at our local Ace Hardware. Continue reading Post #750: Science alert: Proposed cigarette smoke test of “KN95” and other masks

Post #747: Can Kents clarify KN95 chaos? Updated


Update 2:  This didn’t work, at all.  Not even a little bit.  See Post #790 for details.  You can’t use the odor of cigarette smoke to test mask filtration.

Update:  See postscript at bottom.  The ability of genuine N95 masks to filter smoke particles is well known and well documented.  In that light, my proposed “sniff test” for KN95 masks looks fairly promising.  To the extent that a mask reduces the odor of cigarette smoke, then it is filtering out virus-sized particles.

In Post #740, I noted that my local convenience store had “KN95” masks for sale.    I’ve heard a rumor that one of the local hardware stores is also selling such masks.  (I plan to check that out soon.)  And I exchanged emails with  neighbor who is in the process of purchasing some KN95s, from a couple of different sources, for daily wear at work.

In theory, wearing a KN95 gives you the same protection as an N95 respirator.  So, in theory, upgrading from a cloth mask or similar to a KN95 is a smart thing to do.

In practice, not so fast.  I’ve started looking into the “KN95” mask market, and it is complete chaos.  I guess that’s no surprise.  That’s more-or-less of a piece with the entire Federal response to COVID-19. Continue reading Post #747: Can Kents clarify KN95 chaos? Updated

Post #740: Of Condoms and KN95s

Source:  Wikipedia.

Stores such as 7-11s are an endoscope into the soul of the common man.  Name a little vice, or a venial sin, and they’ve got it covered.  At an affordable price.

Nicotine fit?  Pack of Marlboros is calling you from just behind the counter.  How ’bout a Slim Jim and a six of Bud to go with that?  Feeling lucky?  Scratchers and Lotto will handle that itch.  After a sugar rush?  Try the Slurpees.  Or maybe a doughnut, or some potato chips, if you’re jonesing for some carbs.  Plus coffee for the caffeine addicts.

And for that special alcoholic in your life, What’s the word, Thunderbird?  A wine with a taste described as ” … goes especially well with being face-down in the gutter.”

Condoms?  They’ve got you covered, and they’ll deliver.

But no porn.  Not since 1986.

There’s a point to this.  In every convenience store, there’s a special sin area.  It’s always behind the counter, line-of-sight for adult patrons standing at the register,  showcasing the goods for the client in need.  You’ve got your Trojans, your Zig-Zags.  The stuff that’s not illegal, but not exactly fit for high society either.  The stuff that, say, a teenage boy of good upbringing might want, but might not want to ask out loud where it’s kept.

And now, you’ve got your KN95s.  Those seductive little masks that the Government says you should leave alone.  Cloth masks are good enough for you, Citizen.  That’s what our Federal government tells you.  And you comply.  Yet you secretly lust for a real mask anyway, don’t you?  Sometimes a person needs a little, eh, protection, am I right?

Anyway, I happened to find myself in a local convenience store this afternoon.  (None of your business).  And as I was casually eyeballing the sin area (again, none of your business), lo and behold, along with the rubbers and the rolling papers, there was a discreet blue box of KN95 masks.  No price listed, but clearly for sale.  Guess if you really want one, you’ll ask.

Putting the KN95s next to the Trojans seems logical to me.  Same line of business, different orifice.  At least in terms of preventing the spread of disease.  Buying “protection” just takes on a different meaning.

In any case, I was heartened to see that KN95s are going that mainstream, despite the best efforts of the Federal government to ignore them and to ignore our changed situation.  I see a lot of people wearing what look like KN95s these days.  If you have an N95 or KN95, wear it when you’re in an enclosed public space.  It’s the best thing you can do for yourself and for your neighbors.

Don’t know where to buy one?  Maybe a little ashamed to order from some sleazy Ebay seller?  Just ask the clerk at your local convenience store.  They’ll set you up.

Keep one on hand, just in case.  Don’t think they’ll fit in your wallet, though.

Source:  Farm and