The danger of appealing stories: anecdata, expectations, and skepticism
This lovely image was taken by Russell Watkins in Sindh, Pakistan, and I was directed to it by a brief article in New Scientist. Reporter Seil Collins:
Covered in spiders’ webs, these cocooned trees in Sindh, Pakistan, are an unexpected result of floods that hit the region in 2010.
To escape from the rising waters, millions of spiders crawled up into trees. The scale of the flooding and the slow rate at which the waters receded, have left many trees completely enveloped in spiders’ webs.
Although slowly killing the trees, the phenomenon is seemingly helping the local population. People in Sindh have reported fewer mosquitos than they would have expected given the amount of stagnant water in the area. It is thought the mosquitoes are getting caught in the spiders’ webs, reducing their numbers and the associated risk of malaria.
I love this idea. My friends have been passing it around google reader, and it seems to make a lot of sense: More spiders, more webs, more dead mosquitoes in webs, fewer mosquitoes, less malaria. Bing bam boom.
The problem with it: It’s entirely based on anecdata. Anecdata has been my favorite word for about six months now, as well as a topic of fixation for me. It describes information from compiled from a number of agreeing anecdotes, stories, or items of hearsay — “psuedo-data [sic] produced from anecdotes” in the words of urban dictionary. Hearing that multiple people have made similar observations or had similar experiences can clue you into a trend, but hearing a lot of stories doesn’t prove anything. Storytelling is subjective and malleable, not the qualities of good data.
I haven’t been able to find a single scientific source for this mosquito/spider/flood story, though I’d give you a double high-five if you could find one for me.
So what’s the deal with data? We humans look around at our world, make observations, connect them, and use our rationality to draw reasonable conclusions. The story behind the photo makes sense: A number of separate correlations fit together. Who am I to say that we need SCIENTIFIC DATA, free from bias, to speak any kind of truth? After all, the chance to collect baseline data about mosquito and spider populations, average web coverage, mosquitoes per inch of web, etc. has passed and now we can only look back and try to remember what it was like before.
The problem with memory: It changes based on new information. And the problem with stories: They are borne from preconceived expectations.
I recently spoke with Timothy Mousseau, a biologist at the University of South Carolina, for The Scientist about the ecology of Chernobyl. If you scroll through media coverage of this topic, you’ll find many references to the Chernobyl site, which has received the highest level of radiation to date, as a wildlife preserve, one that has finally been able to thrive now that humans have left the area.
Mousseau, however, says this is a perfect example of anecdata. (Well, actually, I was the one who used the word. He thought it was very funny and I was the happiest.) Visitors go to Chernobyl expecting a wasteland, and instead they see the plants and animals that have returned in the past two decades. These people aren’t liars — they just can’t help but exaggerate. The visitors went in with expectations about what they were going to see, and when the reality was so drastically different, they went back home and told stories about the booming wildlife, even if it hadn’t actually returned to pre-radiation standards.
But many scientists, Mousseau included, have done a great deal of ecological research at Chernobyl and have found decreases in the number and diversity of many taxa, decreased sperm counts and brain size, and physical mutations, particularly in Mousseau’s specialty species, the barn swallow. (I’m going to write up more detail on this over the weekend, do not fear!)
This story of the thriving of wildlife in the absence of humans at Chernobyl, despite the nuclear fallout, is so appealing. It’s got the perfect ingredients: It’s a bit counterintuitive, but after a moment of thought, the pieces fit. “Ohhh, people were worse for the wildlife than radiation! Thank god this nuclear disaster happened and got rid of all the people so the animals can live in peace!” And as an added bonus, it lets us feel a little better about a terrible incident. It’s really no wonder the media clung to this story — but it’s probably not true. It’s just an instance of storytelling being interpreted as data, despite its contamination with human inference and expectation.
That’s what makes me nervous about this flood/spider/mosquito story. It has very similar appeal: The bit of surprise that a flood could decrease malaria, the “ohh” moment when the patterns seem to stack up, and, once again, a bit of good-feeling about a situation that was disastrous for many people.
Oh yeah, and that the reports are totally anecdotal, made by “people in Sindh.”
I want to believe it! It’s beautiful and makes me feel good inside! But the stories that are the most appealing are probably the stories that we should be most skeptical about. And they are also the most dangerous because they are the ones that will be retold over and over.