Are infographics taking over the world?

infographic-us-facebook-statistics-demopgrahic-data

KPI -key performance indicators – a set of quantifiable (my emphasis) measures that a company or industry uses to gauge or compare performance in terms of meeting their strategic and operational goals. KPIs vary between companies and industries, depending on their priorities or performance criteria. Also referred to as “key success indicators (KSI)”.

From Investopedia.com

 performance:

  1. The act of performing; carrying into execution or action; execution; achievement; accomplishment; representation by action; as, the performance of an undertaking of a duty.
  2. That which is performed or accomplished; a thing done or carried through; an achievement; a deed; an act; a feat; especially, an action of an elaborate or public character.

    From wikipedia

When I was at secondary school, my friend, John, would often receive 50 or 60 christmas cards from friends every year. John was often invited to parties, he was well liked and was never without a girl at his side (very often a different one each month). John was funny, but something of a class clown and was often at trouble at school. He liked sport more than study, he would hang out with his friends after school rather than do his homework. Sometimes John would fail to turn up, have other engagements, he would drop friends just as quickly as he would make them. And the girls were interchangeable. My other friend, let’s call him Jack was not a popular guy at school he had few friends and only one girlfriend all through secondary school. He was quiet, not much fun at parties, in fact hardly ever invited. He was conscientious and loyal, though, and we’d often have long conversations about books, films and music. The friends he made at school remained his friends well into his adult life.

Now lets talk about KPIs. A KPI is a key performance indicator, a way of measuring success, and there are two main types – quantitive and qualitative. If we measure John’s success at relationships on a purely quantitive basis, then he is obviously more successful than Jack – he has more friends. This simple kind of mental arithmetic is what Daniel Kahnemann calls system 1 thinking, it’s quick, surface thinking and it’s the kind of thinking we do most of the time. It’s also a convenient shortcut to the concept of success, e.g. If your business makes more money at the end of the year than you had projected at the beginning of the year, your business is successful.

System 2 thinking, however, is deeper and slower. Lets take an example from Daniel Kahnemann’s book “Thinking Fast, and Slow”. Let’s do some mental arithmetic:

2+2 = ?

How long did it take you to come up with the correct answer? Hopefully not too long. Alright, a more difficult one now:

7×5=?

Still not causing you too much of a strain, is it?

How about:

36 x 312?

Can you do it in your head? Do you need a pen and paper or a calculator? If, like me, you were not able to do this in your head immediately, you are using a different kind of thinking – what Kahnemann calls system 2 thinking, or slow thinking.

Let’s take another example – when you are walking down the street and you see someone coming the other way – he has his hood up, he looks pretty big, and he’s swaying a little bit, he might be carrying something in his hand, could be a beer bottle. What do you do? Probably you put your head down and walk a little more quickly, you might cross the street to avoid him. You might scan the environment for exit routes or other people. What you don’t do is sit down to think about it, you probably don’t waste time wondering if he is a university professor down on his luck. You use your fast and intuitive system 1 thinking to stay out of trouble. Using system 2 thinking here would slow you down and may result in injury or even death.

What about a job interview? Which system of thinking should you use and which do you actually use? If you want the right person for the job you should probably weigh up the pros and cons of each applicant carefully, you should go through their applications with a fine toothed comb, interview them rigorously to find out what kind of people they are, are they right for the job, right for your organization? But you have 2000 applicants, so you go through the applications and immediately throw out all the handwritten ones – you figure, if someone couldn’t be bothered to type it why should I bother to read it. You’re now down to 1500. You then throw out any that don’t meet the exact specifications of the job, maybe they don’t have the requisite number of years experience, have a bachelors degree rather than a masters, etc. You might read a few in more detail, but your more likely to disregard people whose schools you haven’t heard of, or who studied subjects you don’t have much interest in. You might value team sports above solitary pursuits such as reading, and so on until you have your shortlist. So far so easy – all we’re really doing is matching candidates to our pre-defined perfect candidate – so far, so system 1.

Surely the recruiter will take more time and thought over the interview. According to many studies on heuristics and biases, however, most successful candidates are chosen within the first few minutes of an interview, regardless of their qualifications or experience. A lot of people are chosen because of their socio-economic background, their race or gender, the way they talk, how confident they are, how much they fidget, or even how tall they are (over 60 percent of CEOs are well above average height, apparently, not to mention, white, middle-class and male). Even when we should use system 2, we use system 1, and even when we think we are using system 2, we are, in fact using system 1. This is because system 2 thinking is slow and lazy (and works in the background) – it’s our fallback position and we only use it when we absolutely must, whereas system 1 is our default position, it’s what gets us through the day, it’s faster and more convenient.

Back to John and Jack. Who is the better friend? Who would you like to be your friend? Do you base your decision purely on the statistical data? If lots of other people like John, then surely he’s the safer bet? And in fact, John will have a lot of friends, purely because he has a lot of friends, irregardless of his personal qualities. And here is where it gets difficult and where we have to start using our system 2 thinking. What is a friend? What qualities are important to us when choosing friends? Is it always consistent? Do we want a fun-loving friend? Do we want our friend to introduce us to lots of other people? Or do we choose someone who is loyal but who might be moody, someone who probably doesn’t really want or need us as a friend? These are difficult questions – questions which require a deeper level of thinking – but how many of us actually choose our friends – Jack and John are both friends of mine, but when you ask me why, I can’t really say. I’d probably have to sit down and think about it.

The point I’m trying to make here is that quantitive data, such as how many friends I have, how much income I generate, how many students I get in my class on a regular basis, or how many likes I get on Facebook, are all easy to measure and appeal to our quick and convenient system 1 way of thinking. Quantitive data, the kind that is appearing more and more these days in metrics, infographics, on spreadsheets and colourful graphs, provide us with a convenient shortcut to a success story, a shared definition of success. Anybody, and I mean, anybody, can look at a graph and see immediately what success looks like, even if they don’t exactly understand what the numbers mean. More often than not, the presenter of this information has carefully selected the data to present a clear narrative of success with obvious causes and effects.

Qualitative KPIs are much more difficult to analyze and present – they cannot be turned so easily into colourful visual aids (although many companies try and turn this information into something quantitive – this is the purpose of scorecards), and their causes and effects are not so readily available or explicable. They require system 2 thinking both to collate and present the information and it requires system 2 thinking to process it (and ask questions about it).

Obviously we need to look at both types of KPI to judge whether something is successful or not, especially when we are talking about abstract things such as education.

Let’s take an example from our context and test it out. Let’s say that we run a large scale teacher training program – we have 2,000 teachers on the program and they all get something that they probably haven’t received before, some training in learner-centred teaching, for example. The Ministry of Education pays about 2,000 pounds per teacher. When we measure the success of this project we can look at the reach, we can look at the return on investment, we can look at income generated, in other words, the numbers, the quantitive data. Purely in numerical terms, this is a successful project, especially if the contract is renewed.

Let’s take another example. A teacher has a difficult student. Instead of giving up on this student, the teacher takes extra time to counsel the student, s/he gives the student extra homework, extra support, etc. S/he listens attentively to this student, finds out that they are having problems at home and offers some guidance for how to turn their life around. The student starts to perform better at school and the student is very happy and buys her teacher a gift at the end of term. In purely qualitative terms, this is a success.

Now let’s go forward in time, let’s say 10 years. Let’s revisit the teachers on our large scale teacher training project to see if they transformed the way they teach. Now let’s be honest here, I’ve been training teachers on projects like these for the past 8 years and I can say that I may have helped a few teachers transform the way that they teach (I hope), but let’s give it a number to help our system 1 thinking, let’s give an optimistic estimate of 30%. That’s 70% of the teachers that I have trained on such a project who probably didn’t transform the way that they teach. That’s a pretty poor return on investment, isn’t it? It won’t look very good on a graph.

Let’s revisit our problem student. Instead of dropping out of school, she went to university and completed a PhD, she became a Minister of Education, and helped transform the education system in her country. The return on investment here is massive, but how much of it is actually due to the teacher’s involvement? We don’t know how much of this success was due to other factors, there’s too much “noise” surrounding the data. This kind of success is too difficult to measure, and most of the time we don’t bother, especially as it takes a long time for this success to manifest itself. The teacher and the teacher’s manager will not even be aware of this long-term success.

So the next time you read an article about success, ask yourself how they are measuring this success. It’s highly likely that, in order to appeal to your system 1 thinking and to appeal to your sense of narrative, they will “show you the money”. They will talk about numbers, they may talk about transformation but not in qualitative terms, only in numerical terms, and you should be very skeptical of this.

You should be skeptical when your manager only talks to you about quantity and not about quality, especially when talking about your performance. Ask your manager to watch you teach, to assess the quality of your materials, to get feedback from your students, but don’t just rely on your manager to do this – remember that he will probably be using his system 1 way of thinking (he might be busy managing lots of other people, for example). The onus is on you to swing the conversation back to quality – and in order to do that you need to use your system 2 way of thinking and you’ll also need evidence to support your claims, because our system 2 is lazy and slow and often in need of a convincing argument.

Advertisements

2 thoughts on “Are infographics taking over the world?

  1. I agree that we should be considering how to measure the quality of projects rather than simply using useless things such as scorecards and feedback forms. How do we do it though? Quality is not as tangible as quantity. On a less serious note the first CEO that came to my mind was Alan Sugar. A very short man indeed!

    1. good question – we usually want to quantify information in some way with scorecards (hence the name) e.g. how many people liked our course, how many people gave us 8 or above (pretty useless measures, if you ask me). Feedback forms, on the other hand give us more information about the quality of our courses, as do questionnaires, interviews, counselling, etc. We need to get this kind of feedback not just at the end of courses, but also during it so that we can tinker with our courses to make it better. In terms of performance, qualitative feedback that relates to our behaviours is always going to be more useful than information regarding how many of our students are satisfied (it tells us nothing about how we satisfied them).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s