Culturing Science – biology as relevant to us earthly beings

The danger of appealing stories: anecdata, expectations, and skepticism

Image: DFID/Russell Watkins

This lovely image was taken by Russell Watkins in Sindh, Pakistan, and I was directed to it by a brief article in New Scientist. Reporter Seil Collins:

Covered in spiders’ webs, these cocooned trees in Sindh, Pakistan, are an unexpected result of floods that hit the region in 2010.

To escape from the rising waters, millions of spiders crawled up into trees. The scale of the flooding and the slow rate at which the waters receded, have left many trees completely enveloped in spiders’ webs.

Although slowly killing the trees, the phenomenon is seemingly helping the local population. People in Sindh have reported fewer mosquitos than they would have expected given the amount of stagnant water in the area. It is thought the mosquitoes are getting caught in the spiders’ webs, reducing their numbers and the associated risk of malaria.

I love this idea. My friends have been passing it around google reader, and it seems to make a lot of sense: More spiders, more webs, more dead mosquitoes in webs, fewer mosquitoes, less malaria. Bing bam boom.

The problem with it: It’s entirely based on anecdata. Anecdata has been my favorite word for about six months now, as well as a topic of fixation for me. It describes information from compiled from a number of agreeing anecdotes, stories, or items of hearsay — “psuedo-data [sic] produced from anecdotes” in the words of urban dictionary. Hearing that multiple people have made similar observations or had similar experiences can clue you into a trend, but hearing a lot of stories doesn’t prove anything. Storytelling is subjective and malleable, not the qualities of good data.

I haven’t been able to find a single scientific source for this mosquito/spider/flood story, though I’d give you a double high-five if you could find one for me.

So what’s the deal with data? We humans look around at our world, make observations, connect them, and use our rationality to draw reasonable conclusions. The story behind the photo makes sense: A number of separate correlations fit together. Who am I to say that we need SCIENTIFIC DATA, free from bias, to speak any kind of truth? After all, the chance to collect baseline data about mosquito and spider populations, average web coverage, mosquitoes per inch of web, etc. has passed and now we can only look back and try to remember what it was like before.

The problem with memory: It changes based on new information. And the problem with stories: They are borne from preconceived expectations.

I recently spoke with Timothy Mousseau, a biologist at the University of South Carolina, for The Scientist about the ecology of Chernobyl. If you scroll through media coverage of this topic, you’ll find many references to the Chernobyl site, which has received the highest level of radiation to date, as a wildlife preserve, one that has finally been able to thrive now that humans have left the area.

Mousseau, however, says this is a perfect example of anecdata. (Well, actually, I was the one who used the word. He thought it was very funny and I was the happiest.) Visitors go to Chernobyl expecting a wasteland, and instead they see the plants and animals that have returned in the past two decades.  These people aren’t liars — they just can’t help but exaggerate. The visitors went in with expectations about what they were going to see, and when the reality was so drastically different, they went back home and told stories about the booming wildlife, even if it hadn’t actually returned to pre-radiation standards.

But many scientists, Mousseau included, have done a great deal of ecological research at Chernobyl and have found decreases in the number and diversity of many taxa, decreased sperm counts and brain size, and physical mutations, particularly in Mousseau’s specialty species, the barn swallow. (I’m going to write up more detail on this over the weekend, do not fear!)

This story of the thriving of wildlife in the absence of humans at Chernobyl, despite the nuclear fallout, is so appealing. It’s got the perfect ingredients: It’s a bit counterintuitive, but after a moment of thought, the pieces fit. “Ohhh, people were worse for the wildlife than radiation! Thank god this nuclear disaster happened and got rid of all the people so the animals can live in peace!” And as an added bonus, it lets us feel a little better about a terrible incident. It’s really no wonder the media clung to this story — but it’s probably not true. It’s just an instance of storytelling being interpreted as data, despite its contamination with human inference and expectation.

That’s what makes me nervous about this flood/spider/mosquito story. It has very similar appeal: The bit of surprise that a flood could decrease malaria, the “ohh” moment when the patterns seem to stack up, and, once again, a bit of good-feeling about a situation that was disastrous for many people.

Oh yeah, and that the reports are totally anecdotal, made by “people in Sindh.”

I want to believe it! It’s beautiful and makes me feel good inside! But the stories that are the most appealing are probably the stories that we should be most skeptical about. And they are also the most dangerous because they are the ones that will be retold over and over.

Written by Hanner

March 25, 2011 at 12:57 pm

16 Responses

Subscribe to comments with RSS.

  1. Great write-up Hannah. Yes, people don’t understand that ecosystems as a whole are disrupted and just see one small facet of the change. Also, humans get used to their present circumstances and believe that what they see about them is normal. look at Scotland’s deforested hills. Most people don’t understand that Scotland was covered in forests. :o)

    Rob Mutch

    March 25, 2011 at 1:12 pm

    • I love the example of Italy too — people think it’s just flat rolling hills, when it was heavily forested back in the day. That day was just so long ago that we’ve all forgotten… Thanks, Rob

      Hannah Waters

      March 25, 2011 at 1:38 pm

  2. Are there more webs? Wouldn’t there be the same number but in a more concentrated area … or likely, there’s actually fewer (I doubt every spider managed to find a tree and survive). And being concentrated into a smaller area, wouldn’t that decrease the likelihood of a mosquito encountering a web? Or is it that the spiders are normally ground dwelling and the mosquitoes generally fly above the level of the webs (in which case it seems the spiders are missing a trick). Or am I just anecdating up the wrong spider tree …

    Rachel

    March 25, 2011 at 1:18 pm

    • PERFECT. That there is why we need to collect data before drawing conclusions — you took the same story and drew completely different conclusions, led it in another direction that also makes sense.

      I mean, we don’t even have information on what kind of spiders these are! As Rob said above, it’s extrapolating on one tiny piece of the puzzle.

      Their actual effects (in terms of your train of thought) would depend on the spiders’ abundance, distribution, etc. If they’re pretty localized, a thicker web probably wouldn’t help and might even be more avoidable. (Though I don’t know much about mosquito sensory systems.) Covering a large swathe of land could have more of an effect, but then who knows what will happen to those trees… etc.

      Let’s just go to Pakistan and collect data

      Hannah Waters

      March 25, 2011 at 1:45 pm

    • Are they even spiders’ webs?
      Those trees look like they are under attack from ermine moth caterpillars. The caterpillars spin webs around the leaves of the trees they are feeding on in order to protect themselves from birds and other predators. Heavily-infested trees look like they’ve been “shrinkwrapped” in web…

      Prufrax

      August 15, 2013 at 2:09 am

  3. Awesome post! You might consider checking out a book called “The Invisible Gorilla” by @profsimons. About all the mistakes we make thanks to our (often mistaken) intuitions about the world.

    I reviewed it on my blog: http://scienceblogs.com/thoughtfulanimal/2010/08/book_review_the_invisible_gori.php

    Jason Goldman

    March 25, 2011 at 4:19 pm

  4. I completely agree with you Hannah – we can learn about our world through data than through stories. However, data is often collected with the intention of telling a certain story. This is not a bad thing per se – but it is a good thing to keep in mind.

    Mosseau’s study in PLoS ONE is a good example of this.. The story here is that birds are affected by radiation. Specifically, their brain has been affected. There are multiple caveats in the study however, most of which have not been addressed in the media coverage. The bird’s brains have not directly been measured for example – only their head size. Potentially more confounding is that nobody really knows how much radiation these birds received during their lifetime, we only know the amount of radiation in the area in which they’ve been caught. Multiple migratory birds were measured in this study, how does that affect the conclusions by the authors? How much radiation would such a bird acquire (and what effect would that have on their brain size), if these birds only spent a small part of its life in an area with high radiation? In other words, how well does the area of capture correlate to the area in which the birds spend most of their lives? These questions have not been addressed in this study, but they should be addressed before any definitive conclusions are drawn.

    I’m not doubting that there’s a potential story here. I’m not doubting that radiation is bad for birds. What I’m trying to say is, that the data that is presented here, favours a certain conclusion (radiation is bad). The data might have even been collected with this conclusion in mind. As I said, this is not a bad thing – but only more data can reveal whether the conclusions in this study really hold up.

    Lucas

    March 25, 2011 at 5:44 pm

  5. Great post!…I just discovered your blog and will be visiting very often.

    I wanted to share this quote by Jonas Salk:

    “If all the insects were to disappear from the Earth, within fifty years all life on Earth would end. If all human beings disappeared from the Earth, within fifty years all forms of life would flourish.”

    Gian Toyos

    March 26, 2011 at 7:24 am

  6. I like the term ‘anecdata!’ It strikes me as a wise way of eliding the false distinction being drawn nowadays between experience and ‘data.’

    I’ve always thought that quote about the plural of anecdotes not being ‘data’ was terribly patronizing, discounting the experience of ordinary people as automatically insignificant, even about topics (like drug effectiveness) in which the data we want are the experiences of ordinary people!

    I think some scientists are trying to establish a special technical meaning for the word ‘data’ that does not exist in everyday life, just the way we are trying to get the rest of the world to accept our definitions of the word ‘theory’ in the evolution debate. I further think it is an effort doomed to failure. It’s hard enough to explain science to people, without demanding (in an insulting manner) that they change the way they use common words. Your new term ‘anecdata’ is just what we need!

    pat bowne

    March 26, 2011 at 11:05 am

  7. [...] Post at Culturing Science reflecting on a recent news story about an increase in spider webs in trees leading to decreased mosquitoes: I love this idea. My friends have been passing it around google reader, and it seems to make a lot of sense: More spiders, more webs, more dead mosquitoes in webs, fewer mosquitoes, less malaria. Bing bam boom. [...]

  8. I read this right before bed last night, and then proceeded to dream about leaving a comment. In my dream comment I complained viciously about how “anecdata” ruins the “The plural of anecdote is…” meme. :-)

    Also: this is an excellent post.

    Michael Meadon

    March 28, 2011 at 4:57 am

  9. [...] Culturing Science – biology as relevant to us earthly beings: The danger of appealing stories: anecdata, expectations, and skepticism [...]

  10. Ah, yes, good story with relevant truth. Kind of sums up the whole biblical story, doesnt it? Theist, take heed!

    sfhc2181

    March 28, 2011 at 8:12 am

  11. Even the original blurb accompanying the photo says “it is thought…” You probably can’t find any actual data because there just isn’t any yet, although if the description of the photo is accurate then I’m sure somebody will look into it.

    Personally I find it interesting when something reported as anecdote gets transformed into fact in a retelling. In this case, there is no reason to believe the hypothesized cause of the perceived drop in malaria instances, and not uncommon for such a remarkable phenomenon as these spider clusters (again, assuming the description of the photo is accurate) to trigger humans to create just-so stories about them. In fact, a slight drop in malaria is pretty mundane – I wouldn’t be surprised if there is some loony cultist trying to claim that this is a sign from Petra P’rkara, the Spider God, that we must turn all of our possessions over to his church. The impressiveness of the photo alone should be cause to be skeptical of any explanation, and the blurb is appropriately weaselly in its description. Yet I have no doubt that many people read that blurb and even though it says in black and white that there is no real evidence of any of this, they will report it as fact the next time they tell a friend. And from then on, it will just be “common sense.”

    lofgren

    March 28, 2011 at 10:12 am

  12. Perhaps the spider population increase is a less direct result of the floods. The floods did indeed cause stagnant bodies of water to form which certainly allowed for a large rise in the number of mosquitoes. However, this rise in mosquitoes meant an increase in the number of prey for the spiders which consequently led to a population boom. Perhaps the spiders were not running from the water they were just binging on mosquitoes. Speculation, speculation.

    Robert Bedford

    April 11, 2011 at 6:26 pm

  13. [...] One is backed up by data, the other by “anecdata“. [...]


Comments are closed.

Follow

Get every new post delivered to your Inbox.

Join 55 other followers

%d bloggers like this: