Can we make those pigs fatter?

Again twitter has got me thinking again, I know a dangerous proposition. It began again with a tweet from Justine Hughes.
Then the following week the #edchatnz chat was based around assessment and the common mantra of you can not make a pig fatter by measuring it more. The chat was full of interesting discussions which had me up at night as use of data is one of the major 21st century trends and is becoming mainstream.

This idea of big data becoming an integral part of the modern world including in education was further reinforced by a Microsoft innovation expert meetup which introduced Power BI, Microsoft's attempt to enter the big data analytics market and apply this to education.

Having listened to all these voices about assessment and big data, my mind wandered to a Mark Twain quote.


As we grapple with the assessment and student data we are now obtaining as 'evidence' I think we as educators need to consider the questions we want to ask of the data, rather than let the data drive the questions.

Recently, “big data” has increasingly become a part of the education. Big data is data so large that they can only be analysed by computers. Policy makers have embraced the idea that information can help educators make systemic improvements in student outcomes.

Every year in New Zealand, league tables of schools which typically reflect standardized test results, are printed in National newspapers. For serious reformers, this is the type of transparency that reveals more data about schools and is seen as part of the solution to how to conduct effective school improvement. This information, however, does not provide any insight about teaching and learning in classrooms; they are based on statistics, not on the relationships that drive learning in schools. They also report outputs and outcomes, not the impacts of learning on the lives and minds of learners.

In New Zealand, the Ministry of Education introduced national standards for primary school students. As a result, various teacher evaluation procedures emerged in response to this mandate. Yet for all of these good intentions, there is now more data available than can be analysed and yet there has been no significant improvement in outcomes for learners.


If you are principal or senior manager in a school, you probably care a lot about collecting, analyzing, storing, and communicating massive amounts of information about your schools, teachers, and students based on this data. This information is “big data,” which refers to data sets that are so large and complex that processing them by conventional data processing applications isn’t possible. A decade ago, the type of data education management systems processed were input factors of education system, such as student enrolments, or education expenditures. Today, however, big data covers a range of indicators about teaching and learning processes, and increasingly reports on student achievement trends over time.

Among the best known today is the OECD’s Program for International Student Assessment (PISA), which measures reading, mathematical, and scientific literacy of 15-year-olds around the world. On this measure, New Zealand is beginning to lag against other similar nations despite the introduction of National Standards.

Despite all this new information and benefits that come with it, there are clear handicaps in how big data has been used in our education system. Policymakers often forget that Big data, at best, only reveals correlations between variables in education, not causality. As any introduction to statistics course will tell you, correlation does not imply causation.

Data from PISA suggests that the “highest performing education systems are those that combine quality with equity.”  This statement expresses that student achievement (quality) and equity (strength of the relationship between student achievement and family background) happens at the same time.  It doesn’t mean that student achievement (defined in narrow terms) is caused by the family background of the student. Correlation is important in education policy-making but to make the inference that a variable is causative must only be made once other possible causative relationships are carefully explored.

A major problem is that educational policymakers in New Zealand and around the world are now reforming their education systems through correlations based on big data from their own national student assessments systems and international education data bases without adequately considering what actually makes a difference in schools. New Zealand which takes part in the PISA survey has made changes in their education policies based primarily on PISA data in order to improve their performance in future PISA tests. But are changes based on big data really well suited for improving teaching and learning in schools and classrooms in New Zealand?'

It is becoming evident that big data alone won’t be able to improve student outcomes. Policy makers need to gain a better understanding of what good teaching is and how it leads to better learning in schools. This is where information about details, relationships and narratives in schools become important. These are what Martin Lindstrom calls “small data”: small clues that uncover huge trends. In education, these small clues are often hidden in the narrative of schools. Understanding this narrative must become a priority for improving student outcomes.


As each school and learner is different, there is no right way to gather small data. Perhaps the most important next step is to realize the limitations of big data in education. Relying too much on externally collected data can lead to false conclusions. Their are many advantages to focusing on small data:

    1. It reduces national standardised summative assessments to a minimum and transfers resources to enhance the quality of formative assessments in schools and allows for other alternative assessment methods. Evidence shows that formative school-based assessments are much more likely to improve quality of education than conventional standardized summative tests.

    2. It strengthens teacher  and school autonomy with more independence from external beauracracy and investing in collaboration in and between schools. This would allow for more relevant assessment and enhance student learning.

    3. It empowers students by involving them in assessing and reflecting their own learning and then incorporating that information into developing judgment about teaching and learning. Because there are different ways students can be succeed in schools, no one way of measuring student achievement will reveal success. Students’ voices about their own growth may be those tiny clues that can uncover important trends of improving learning.
Automation based on continuous data is now changing our daily lives. You don’t need to know how to use maps anymore when you can use smart phones that find the best routes. Similar trends are happening in education systems with countless reformers trying to “disrupt” schools.

Big data has proved a useful tool in education by letting us view connections that have occurred in the past. However, to improve teaching and learning, policy makers need to pay more attention to small - data - to the diversity and beauty that exists in every classroom and how the relationships in that classroom reveal the causation of the present. If we don't stop and start using small data and stop relying on big data we might be lead down the wrong path.

Comments

Popular posts from this blog

Branching Out in Physics Lessons with Decision Trees

How to Assess Learning in the Age of AI

ISTE23 - The Power of Connections