There is an anecdote about four blind men describing an elephant (with no prior knowledge of this animal). Each blind man observes a different body part of the elephant (his tail, leg, and trunk, for example). Each blind man declares authoritatively what an elephant is, and each one is totally wrong.
This is a perfect example of how not to do statistics. Statistics is not the science that allows us to gather data haphazardly.
But this also describes how we currently “do” aviation safety–when I say “we” I am more inclined to mean organizationally. For example…
The National Transportation Safety Board has an office of Aviation Safety. It has a Most Wanted list that “represents the NTSB’s advocacy priorities”–general aviation safety is on this list.
But what is safety? More importantly, what is general aviation safety? How will we know when we’ve “captured” or found General Aviation safety (to extend the Most Wanted analogy)?
It’s hard to buy into the vision of a leader if you can’t see what he sees.
You probably won’t want to get on the bus if you can’t figure out where it’s going.
So, where is the NTSB headed? What are their vision and mission?
This is a screenshot of the About page of the NTSB. It’s mission is to promote transportation safety. (Still no definition of safety.)
If you read more about the General Aviation safety issue, and you’ll find this quote:
“But the best aircraft in the world will not prevent a crash if the pilot is not appropriately trained and prepared for conditions.”
I agree, but the following paragraphs suggest more training is the solution. Safety is NOT more training. All the training in the world won’t help any pilot if there is no test at the end of the course. Let me illustrate with an example that more of us can understand–drunk driving. People who drive drunk do so, not because they have not been trained about it but because they made a foolish choice to do so. In some cases, bystanders contribute to the foolish decision making by not doing something when they could.
Reading the NTSB “brochure” might lead you to believe that safety is “less deaths.” If we triple the number of flights and the number of fatal accidents remains the same, then “less deaths” might not be a good definition.
On the other hand, “no deaths” is certainly a descriptor of safety, and perhaps it should be the goal. ”Decreasing the number of fatal accidents” doesn’t seem like enough effort, as far as aviation safety is concerned.
Part of the NTSB charter is accident reporting. This is a very important piece, but we still don’t know much (or anything as far as the NTSB is concerned) about the P-51 crash at the Reno Air Races. We can’t learn from our mistakes if we cannot even remember them when the report comes out.
Scientists know that the “scientific process” follows a cycle:
Prediction — having a theory based on observations of the world
Test — rigorous controlled experiments designed to confirm or refute specific predictions that arise from theory
Validation — reasonable interpretation of results and update of theory.
Data sciences, like statistics, work best in this same scientific framework. Our safety efforts need the same kind of control and specificity.
My theory is that people make stupid decisions–sometimes intentionally and sometimes unintentionally. We must hold ourselves and fellow airman accountable for the intentional stupid decisions–we aren’t doing that.
And we must do everything we can to reduce the causes of the unintentional erroneous decisions–many times we aren’t doing that either, because of the litigious society we live in.
I have documented this rant for two reasons. First, I want to encourage fellow airman to lead–to be accountable for stupid decisions–to hold one’s self accountable–to challenge others to do the same. That’s my vision of safety, and this column will help you do that.
Second, I want us to use data and information related to safety to help us in our accountability, in our hypothesizing, in our application of our knowledge to the safety process. This column will help you do that too. Don’t become a statistic–understand them.
The Q & A in Launch Your Flight Test Career has generated some great discussions on Linked In.
The flight test community needs to learn Bayesian methods, and here is a free text. Though I confess to not having read the book in its entirety, the generosity of the author to share it 
The FAA has provided a veritable library of aviation knowledge virtually, including the Pilot’s Handbook of Aeronautical Knowledge. It is an excellent reference for that concept you know is true, but for which you cannot find a citation because it is so fundamental.
