When I started an e-learning subject in an adult education masters, I hoped that the course would bring together the two aspects of what I believed e-learning is all about - content (from the Ad Ed subjects) and process (from the e-learning unit).
The effect has been quite unexpected. I have now discovered that to get the message out there is a presentation skills requirement - beyond use of the tools, and now I am coming to realise that the biggest concern is a psychological one.
Early in the course I was attracted to an on line discussion group, TR Dev at Yahoo groups. This is a closed group, but you can apply for membership through a button on the sidebar of this weblog.
The first discussion I encountered was about the mythical statistic that we recall 10% of what we hear, 20% of what we see, 30% of what we see and hear together, and an ever increasing proportion until you reach 90% for the type of communication medium that the salesman is trying to push.
Like most trainers I first heard those numbers early in my training career, but as a former engineer I asked questions like how long after the episode was the recall measured? My mentor didn’t know, but promised to get back to me. I am still waiting.
The discussion on the list caused me to reflect on the significance I give to numbers as a learning aid. Because I am from a numerate first life I start analysing the numbers as they are presented. For many it seems, the number is enough. Perhaps that is why 67.42% of statistics are made up on the spot. (That was meant to be a joke.)
But an article in this morning’s Sydney Morning Herald that addressed the numbers issue caused me to reflect on how much information I deliberately avoid looking for.
The article was an extract from the book Risk: the science of politics and fear, by Dan Gardner.
The headline attracted me because one of the areas that I train in is Risk Management. The first paragraph nearly turned me off, because it started “‘Recent figures suggest some 50,000 pedophiles are prowling the Internet at any one time,” says the website of Innocence in Danger, a non-government organisation based in Switzerland.”
The subject was an area that I prefer to remain blissfully ignorant of. I am not saying it is not important, just that my mind is made up that I find the subject repulsive. There is too much good stuff that I don’t have time to find out about. I choose not to spend my weekends contemplating an abhorrent topic.
It was the next paragraph that got me in: “No source is cited for the claim, which appears under the headline ‘Some terryfying [sic] statistics’.”
So for the moment, forget about the topic, I won’t mention it again until the end, when I will discuss the effect that the topic had in almost making me avoid the article.
The number 50,000 is just too round to be believable. like the 10%, 20%, 30% or 90% recall.
So rule number 1: Suspect numbers that are too round
But there are other concerns with the statement, that are common to misuse of numbers.
How about all 50,000 being on line at any one time. If each spends 12 hours per day every day on the net then the number of deviates has just doubled. They must work in shifts to keep the number at 50,000 at all times. If six hours of prowling is all that some can mange because of firewalls and footsteps, the number is now up to 200,000. If some are only a little bit deviant, so only prowl for six hours a week, then the number may be as high as 1.4 million. this is getting close to the results of the survey I saw that said 50% of respondents admitted to viewing X-rated material on the Internet. The other 50% were women.
Rule number 2: Look for specific details of the source of the study.
No doubt you can still remember the number quoted in the study, but which of the following is cited as the source:
- A study conducted at Harvard university
- FBI statistics
- The Swedish Government
- A non government organisation based in Switzerland
- Recent figures
Correct. It was “recent figures”. They were quoted on a Swiss web site, but the non-government organisation that hosted the site dose not claim to have done the counting.
According to the article, however, it is not just this one web site that displays them.
It has been cited in Britain, Canada, the US, and points beyond. Like a new strain of the flu, it has spread from newspaper articles to TV reports to public speakers, websites, blogs, and countless conversations of frightened parents. It even infected Alberto Gonzales, the former US attorney-general.
Rule Number 3: The more often the number is quoted, the more important it is to seek out the source.
Like most university students, I hate academic referencing. But the requirement to justify statements with references to authoritative sources is something that should be taught long before university.
Sure it is time consuming. Both in seeking out the source and appending it to each comment in a conversation. But this particular statistic is, according to Gardner, finding its way into all of he sources that we look to for information - ”from newspaper articles to TV reports to public speakers, websites, blogs, and countless conversations of frightened parents.”
Rule Number 4: if a celebrity said it Google for their exact words.
Gardner claims that “It even infected Alberto Gonzales, the former US attorney-general.”
I was unable to confirm through a Google search:
- Gonzales (as attorney general) had sought a random selection of 50,000 Google searches as part of a defense against a supreme court challenge to a child protection law;
- A suspect was found to have 50,000 illegal photos on his hard drive, and
- Yes, Gonzales did use the figure in the context quoted in a CNN interview. When asked for the source, he cited a Dateline program on another TV channel. When the Dateline reporter was asked for the source in a radio interview for a media watch program, he conceded he referred to an unnamed expert, but has not used the number in follow up programs.
Another favourite celebrity myth of mine is that we could all be as smart as Einstein if we would stop using just 10% of our brain. It has all the ingredients of a doubtful citation - 10%, nice round number, celebrity source, oft quoted by motivational speakers.
A source who does cite his sources, Dr. Karl Kruszelnicki had this to say:
The myth that we use only 10% of our brain is finally being proved untrue, because over the last few decades, we have invented new technologies (such as Positron Emission Tomography and Functional Magnetic Resonance Imaging) that can show the metabolism of the brain. In any one single activity (talking, reading, walking, laughing, eating, looking, hearing, etc) we use only a few per cent of our brain - but over a 24-hour day, all the brain will light up on the scan.
Notice how he maintains a readable style while revealing his authorities. While his source in this case is not specific, it can readily be confirmed by reference to the technology he nominates. We went to school together, although at the time I was probably only using 10% of my brain. Ooops!
Rule Number 5: Watch for qualifiers
The 50,000 cited in the article has been expressed as
- as many as 50,000
- at least 50,000
as well as the definitive number.
The place where they prowl has been identified as
- on the Internet
- in social networking sites, and
- on MySpace.
Quite a difference, I would imagine.
The number has also been sourced as coming from
- the FBI - who decline to confirm the number and say that an accurate estimate is impossible to offer;
- a conversation with British police, although details of time and context were not forthcoming. They were able to advise, however, that only 1 abduction and murder by a stranger had occurred in Great Britain in the previous two years, the fear inspired by the 50,000 prowlers.
- the attorney general, who, as we saw, got it from Dateline, who got it from”an expert.”
Rule number 6: Beware of internal biases
This is where the psychological insight joins the process, design and content concerns that i believe makes credible e-learning content.
I was pleased to find a citation from Dr. Karl, because I like his style. I have a remote 40 year old connection with him and no doubt this adds to his credibility in my eyes. The article was in my favourite paper. I would not have seen it, or taken it as seriously if it had been in an alternate paper.
I almost ignored the article on a subject dear to my heart because of it was set in a context that I find repulsive.
My preference is to write at level that is understood by year 6 students, because that is often the literacy level of my target audiences in my work. One university lecturer described one assignment as quite racy, but insisted I put in some speed bumps, as I saw the citations.
We also look for information that reinforces what we already know. That is a principle of adult learning. Paradigm shifts are painful.
Remember these 6 rules the next time you see an article that seems to be too good to be true.
Now I must go and reply to one of the e-mails that offer me $46 million dollars, the proceeds of a Nigerian coup. At least it is isn’t $50,000,000. That would be too round a number and would arouse my suspicions.
Tags: credibility, numbers, statistics
Entries (RSS)