You may be familiar with the musical / movie Chicago – a pair of women murder their philandering husbands and get a hot shot lawyer to defend them. The advice he gives them is that the truth doesn’t matter – it’s all about the show you put on.
“Give ’em the old razzle dazzle, Razzle Dazzle ’em, Give ’em an act with lots of flash in it, And the reaction will be passionate, Give ’em the old hocus pocus, Bead and feather ’em, How can they see with sequins in their eyes?”
Although it may not seem like a musical about twisting the truth and putting on a show would have much in common with statistics, it can be tempting to make use of the same tactic used by the lawyer in Chicago- putting on a good show and distracting people so that they don’t look closely at what the results actually are.
One way to do this is to place emphasis on the significant results – even though those results are peripheral to what the original purpose of the research was. Let’s say the research was originally interested in whether an intervention made dogs more intelligent. The results were non-significant (darn), but the report gets filled with all of the significant changes that they did find – dogs are more likely to get along better with other pets after the intervention; less likely to make a mess of the house while you’re out; more likely to live longer and be healthier. These things sound wonderful, but they have nothing to do with what the intervention was originally intended to do – make your dog smarter.
Another way to Razzle Dazzle the data is to ‘zoom in’ on graphs to make the changes between pre- and post-assessment scores look larger, without labelling the graph properly. On the new graph the results look much more impressive, but people aren’t given the information they need to interpret what they are seeing correctly.
At Unmind we have made a commitment to report our statistics ethically. This means that we report all of our results, and won’t attempt to Razzle Dazzle you with other results and sequins.