This question of trying to figure out whether a book is good or bad by looking at it carefully or by taking the reports of a lot of people who looked at it carelessly is like this famous old problem: Nobody was permitted to see the Emperor of China, and the question was, What is the length of the Emperor of China’s nose? To find out, you go all over the country asking people what they think the length of the Emperor of China’s nose is, and you average it. And that would be very “accurate” because you averaged so many people. But it’s no way to find anything out; when you have a very wide range of people who contribute without looking carefully at it, you don’t improve your knowledge of the situation by averaging.
Richard Feynman, the Nobel Prize winning physicist and generally admired clever-man, wrote this paragraph in a fascinating essay about the problems he had found with both mathematics schoolbooks and the process by which they are assessed in California in the mid 1960s.
Ok, so, that doesn’t sound terrifically interesting but, trust me, this is a man who could make almost anything brilliantly interesting while not dumbing it down so much as to be insulting. Seriously, take a read of it then tell me it’s boring.
The reason I picked out this particular segment was because it struck me as an excellent way of looking at the much discussed communications problem of the development world – do we condemn widely disseminated but flawed development related productions like Kony2012 or the Failed State Index or do we embrace the fact that lots of people are now engaged in a topic they otherwise would not have?
The latter. Misinforming lots of people doesn’t lead to a better outcome, it simply legitimizes ineffective techniques.
Another interesting thing about the essay is that Feynman identifies the roots of the problem:
- Most of the people doing the assessment work are too busy to actually read all of the information
- Before or during the assessment period, the people producing the assessed materials (in this case textbooks) meet with the assessors and explain the material to them. This is not a process that will be repeated with end users (in this case, schools and teachers)
- The materials often reflect a flawed understanding of the topic
- The assessment process was de-centralised and segmented, taken up by many people only a few of which were directly responsible for the actual findings
This is getting weirdly similar to pretty much all development now isn’t it?
What you have here covers most of the big narratives about problems in the global development industry: the unfairness of the ménage à trois, the lack of suitable structure to deal with both implementation and assessment at the same time, the lack of consultation with end user stakeholders, and, most importantly, the way corruption hinders all systems lacking counter-measures.
Now it is important, at this point, to point out ‘the problem’ as I called it a few paragraphs ago has not yet been explicitly mentioned in this article. Back to Mr Feynman,
Many [Americans] thought we were behind the Russians after Sputnik, and some mathematicians were asked to give advice on how to teach math by using some of the rather interesting modern concepts of mathematics. The purpose was to enhance mathematics for the children who found it dull.
The problem was that the overall knowledge of mathematics in the USA was perceived as having dropped below a certain standard. The ‘solution’ was to sex up mathematics. Obviously, there are some critical issues here already. Not only is the ‘problem’ based on non-testable perceptions, the solution is simplistic and narrow. Again, this could be someone writing about any number of development issues today.
As Feynman saw it, the central problem was corruption (just look at the title of the essay). The other issues were either caused or compounded by it. Many development professional feel the same way, of course, but it remains something of a wicked problem – one with no simple answer and no simple definition. He wrote of his solution for this particular case, or, at least, what his solution could have been,
It seems obvious now, but I didn’t know what was happening the time I got a package of dried fruit and whatnot delivered by Western Union with a message that read, “From our family to yours, Happy Thanksgiving — The Pamilios.”
It was from a family I had never heard of in Long Beach, obviously someone wanting to send this to his friend’s family who got the name and address wrong, so I thought I’d better straighten it out. I called up Western Union, got the telephone number of the people who sent the stuff, and I called them.
“Hello, my name is Mr. Feynman. I received a package . . .”
“Oh, hello, Mr. Feynman, this is Pete Pamilio” and he says it in such a friendly way that I think I’m supposed to know who he is! I’m normally such a dunce that I can’t remember who anyone is.
So I said, “I’m sorry, Mr. Pamilio, but I don’t quite remember who you are . . .”
It turned out he was a representative of one of the publishers whose books I had to judge on the curriculum commission.
“I see. But this could be misunderstood.”
“It’s only family to family.”
“Yes, but I’m judging a book that you’re publishing, and maybe someone might misinterpret your kindness!” I knew what was happening, but I made it sound like I was a complete idiot.
Another thing like this happened when one of the publishers sent me a leather briefcase with my name nicely written in gold on it. I gave them the same stuff: “I can’t accept it; I’m judging some of the books you’re publishing. I don’t think you understand that!”
One commissioner, who had been there for the greatest length of time, said, “I never accept the stuff; it makes me very upset. But it just goes on.”
But I really missed one opportunity. If I had only thought fast enough, I could have had a very good time on that commission. I got to the hotel in San Francisco in the evening to attend my very first meeting the next day, and I decided to go out to wander in the town and eat something. I came out of the elevator, and sitting on a bench in the hotel lobby were two guys who jumped up and said, “Good evening, Mr. Feynman. Where are you going? Is there something we can show you in San Francisco?” They were from a publishing company, and I didn’t want to have anything to do with them.
“I’m going out to eat.”
“We can take you out to dinner.”
“No, I want to be alone.”
“Well, whatever you want, we can help you.”
I couldn’t resist. I said, “Well, I’m going out to get myself in trouble.”
“I think we can help you in that, too.”
“No, I think I’ll take care of that myself.” Then I thought, “What an error! I should have let all that stuff operate and [kept] a diary, so the people of the state of California could find out how far the publishers will go!”. . . .
Seeing as this comes from ‘the last true genius‘ of the 20th Century it’s a pretty ringing endorsement for whistleblowers. Now we just have to work out a way to protect them effectively and we’ll be golden…