Andreas Schleicher, OECD director of education and skills, wrote an article for the BBC website claiming PISA answers seven big myths about education in the world. Andreas, apart from having an impressive job title, is the man in charge of PISA tests and therefore may not have a completely unbiased opinion. Here is my response to each of his myth busting claims.
1: Disadvantaged pupils are doomed to do badly in school
Andreas busts this myth by referring to Shanghai. He says “the 10% most disadvantaged 15-year-olds in Shanghai have better maths skills than the 10% most privileged students in the United States and several European countries”.
But, China still cheat the education system by entering their two most affluent cities instead of their country. Looking at population, this is probably equivalent to the UK entering Eton and Harrow into PISA! Time magazine ran this article explaining how China is cheating the PISA ranking system. 24% of Chinese students go to college but 84% of students from Shanghai.
One final point that even the BBC acknowledge is that Shanghai doesn’t enter all of its pupils. In fact, only about a third of school age pupils can afford to attend education in Shanghai. With an estimated 300,000 15 year olds eligible for the test, only 100,000 were included. Doing the maths (84% of a third) that means that around 27% of Shanghai students go on to college.
Myth busting busted: Even just looking at Shanghai, it is one of the lower attainers in the PISA ranking.
2: Immigrants lower results
Andreas tells us that there is “no relationship between the share of students with an immigrant background in a country and the overall performance of student in that country”. We would all love this to be true but unless the country concerned has 100% school attendance then Andreas’ statistics tell us nothing. Statistically if there is a cost for attending school then children of recent immigrants are likely not to be in school.
Myth busting busted: PISA data is too incomplete to answer this question.
3: It’s all about money
Andreas says “Success in education systems is no longer about how much money is spent, but about how money is spent.” He uses South Korea as an example South Korea did well in the PISA rankings but is not one of the biggest spenders on education. But, there is a cultural difference here that is not being included. Much of Asia see school as only a small part of the education a child receives. Children leave school and go immediately into expensive private classes. Often in small groups or individually children are tutored from school leaving time (around 4pm) for up to an additional six hours. This is a cultural difference. Most parents in the UK or America for example expect school to educate their child.
Myth busting busted: What we are actually measuring is not the total spent on education but in fact in the case of many countries we are seeing only about half of their educational spend. The other half is being made by parents after school.
4: Smaller class sizes raise standards.
Andreas finds that smaller class sizes do not, according to PISA, raise standards. Andreas, stop ony looking at PISA. Hattie has already discovered this by taking into account results from much more than just one study. (Visible Learning for Teachers: Maximizing Impact on Learning).
Mythbusted…but not by PISA: This myth that Andreas busted is correct but the proof is found with Hattie and later Alistair Smith (High Performers: The Secrets of Successful Schools) and PISA alone is insufficient to draw this conclusion.
5: Comprehensive systems for fairness, academic selection for higher results
The same logic that has countered some of Andreas’ mythbusting skills in earlier points applies here. PISA does not have complete information. Shanghai appears to be a non-selective education system and yet two thirds of its 15 year old pupils are invisible to PISA.
Myth busting busted: Unless PISA includes all pupils from all schools and takes into account the pupils not in school then PISA does not have sufficient evidence to discuss whether selective or non-selective education is more effective.
6: The digital world needs new subjects and a wider curriculum
At the risk of repeating myself, PISA does not have the data to answer this myth. Because the PISA data is woefully incomplete it just shouldn’t be seen as useful for making generalisations about curriculum.
Myth busting busted: Innsufficient evidence again!
7: Success is about being born talented
Andreas wheels out his only gun here and uses PISA to bust this myth. Again, PISA doesn’t have sufficient evidence to comment on this but others do.
Myth busting busted: Malcolm Gladwell (Outliers: The Story of Success) suggested there is no such thing as ‘Talent’ and that to be an expert at anything took 10,000 hours of practice. Carol Dweck has used research to say something very similar in her work on Mindset. (Mindset: How You Can Fulfill Your Potential).
And now time for the one truth…
Truth: PISA is riding high on the fact we love an international competition and is making claims that it just cannot back up by statistics. Until PISA collects data from all students in a country and compares like with like (country with country) then many of the claims that are being made from its data will be easily debunked.