How Test Engineers Decide What Counts as Good Enough

The statistical reasoning is sound, even if the ingestion test’s verisimilitude is a little hand-wavy. Bird populations change; migratory patterns, or what experts call “waves of biomass,” are fickle. In almost half of the bird-strike incidents investigated by the Federal Aviation Administration, the turbine’s charnel house is so messy that the culprit’s species remains anybody’s guess. (Davies, who has a wonderful ear for niche jargon, is pleased to report that bird slurry is called “snarge.”) We can be fairly certain that no snarge can be traced to chickens, which do not fly. From 1990 to 2019, Davies writes, “the FAA recorded more bearded seal strikes (one) than chicken-on-plane encounters (zero).” Testers resort to chicken artillery in the absence of a robust supply chain for osprey.

The use of blameless chickens reflects a compromise between workability and similarity. Experimental results are credible only if the procedure is practical, repeatable, and standardized. We don’t fly in planes because someone pulled it off once. At the same time, the results are informative only if the procedure is a serviceable approximation of the actual circumstances we expect to face. Often, some basic gimmickry will suffice. Manufacturers assess the usability of medical equipment by putting it in the hands of its intended users, sometimes in fake hospital rooms filled with real beeping, where nurses might be interrupted by some nudnik asking if they own “the blue Chevy whose alarm is going off.”

Other instances are more challenging. John Stapp, of the rocket sled, had a detached view of his body—he once noted that “to the human engineer, man is a thin, flexible leather sack filled with thirteen gallons of fibrous and gelatinous material, inadequately supported by an articulated bony framework”—but most of us regard the flesh with greater sentimentality. The safety features on cars, for example, have often been evaluated with cadavers, which aren’t typically at risk of further harm. Nevertheless, Davies observes, the dead “have their weaknesses.” For one thing, they do not brace for impact, and they frequently suffer from preëxisting conditions. They’re also expensive and tricky to source ethically. Crash-test dummies are more readily produced, and their uniformity allows for better experimental control, but they’re imperfect substitutes. Manufactured in the image of the “average” male of a skinnier era, they are intended to be one-size-fits-all. They are poor proxies for women, children, and the “average” male of today.

The testing industry, Davies emphasizes, consists of trade-offs all the way down. Although some products that made it to market were needlessly dangerous—there was never a reason for the Little Lady Hot Stove, a postwar toy, to be able to reach six hundred degrees, or for contemporaneous science kits to include actual uranium—other products aren’t useful if they can’t hurt you. The ideal balance isn’t always straightforward. In the nineteen-sixties and seventies, sixty thousand people a year reached into their lawnmower chutes to unclog them and ended up in the emergency room. In 1974, technocrats at the newly formed Consumer Product Safety Commission campaigned on behalf of the nation’s yard workers, eventually advocating for a dead man’s switch on every push-mower handle. In four studies, hundreds of volunteers moved their hands from a mower handle to the discharge chute; the results clustered at around three seconds. Once engineers confirmed that an affordable mechanism could stop the blade within that interval, three seconds became the standard. Industry lobbyists thought the rule was too stringent; safety advocates thought the blade should stop faster. The C.P.S.C. made a middle-of-the-road call, weighing minor additional costs against modest safety gains, and it prevailed.

Table saws went the other way. Davies recounts the story of an amateur woodworker named Steve Gass, who invented an electrical method to cut into America’s thirty thousand annual shop mishaps. (In need of a stand-in for his own meat digits, he settled on hot dogs.) The contraption worked beautifully, but, because it added hundreds of dollars to the price of the appliance, it was soundly defeated by Big Saw.

The medicine-cabinet poisoning of children was an analogous problem with an ingenious and effective solution. Just as lawnmowers and saws can’t function with rubber blades, drugs can’t be replaced with placebos. They could be stored in bottles that were challenging to open, but, if the bottles were too challenging, people would just leave the caps off, making things worse. The remedy appeared in the form of the Palm N Turn container, which charted a path between the Scylla of a child’s ability to access pills and the Charybdis of an adult’s inability to access them. Such containers are punctiliously calibrated. They must stump at least eighty per cent of a panel of fifty little kids in the course of ten minutes, even after the children are explicitly shown the winning technique. Then, within five minutes, they must submit to the will of at least ninety per cent of a panel of a hundred adults between the ages of fifty and seventy. Since the early nineteen-seventies, deaths from unintentional childhood poisonings have fallen by four-fifths.