Recently, I was asked to write an article about Skólapúlsinn for a local Icelandic technology magazine. The theme of the issue was Education and Technology, and given Skólapúlsinn sits between both of those disciplines, we contributed. You can download a PDF version of the article that was printed. What follows is what I submitted with a few links, annotations and minor changes.
Streamlining School Measurements
School measurements used to mean getting all the students to sit in a classroom all day long and take a long boring standardized paper survey. We see this done over and over again with services like the national standard tests and the PISA survey. All the 10th graders are tested on academic achievement as well as surveyed for attitude and behavior at the end of the school year for an entire day. Other examples in Iceland of such broad sweeping surveys are WHO’s survey on Health Behavior in School-aged Children (HBSC) and the Ungt fólk survey by Rannsóknir og greining. Students sit through page after page of questions ranging from self-esteem to study behavior to risk behavior to health to computer use and many other psychometric tests. The outcome is tallied, ranked and sent back to the schools months and up to 2 years later in a paper report. Only then do problems within the school life appear, long after the students have left not only for summer vacation, but many have moved on to another school.
What if we look into taking many of the key aspects that make start-up culture so fertile, the ideas of rapid development, tight feedback loops, and always adjusting agile processes and apply them to a school environment? That’s exactly what our small company, Skólapúlsinn does.
Traditionally, it has been assumed that if you want to know what is going on in your school, organization or group you needed to ask everyone. With some basic statistical methods, it’s possible to gather data from a small sample group. Instead of surveying an entire school once a year, wouldn’t it be better to continually evaluate the school though out the school year, a few students at a time? In software and industry, we don’t go and build, manufacture and ship objects, only to test them later on. Instead we have unit tests, regression tests, integration tests, quality checks and other practices to minimize bad things from shipping. That is the ideal model we want to bring into education. A continual monitoring of just a sample of the population each time, as a representative part of the whole school, looking for any issues and quickly allowing people to respond and administer a change immediately, not after several months or years!
Skólapúlsinn is an online software survey system introduced in Iceland in 2008. We are a team of three coming from different backgrounds, psychological measurement, computer science and educational research. The fact that our skill-sets are so unique and distinct are what make this project move forward so well. By bringing some Computer Science into Social Science, we’ve managed to merge the ideas of these two disciplines in this new product, bringing about a fundamental change in the way school administrations works with their school development.
Since the beginning, Skólapúlsinn has surveyed over 12,000 students anonymously in over 70 compulsory schools, age 11-15, getting their opinions about various topics in the classroom, about their well-being, interactions with other students, engagement with their studies and staff relations. Instead of testing an entire school once, the system randomly selects a small representative sample of the school, balanced for gender and age. These 40 students are like a micro-version of the school and their answers, within statistical certainty, represent the school as a whole. This is done every month of the 9 month school year, giving the school a definitive pulse.
A school is a living organism, growing and changing. It needs care, attention and sometimes to be brought back inline when problems arise. The idea behind a rapid, short, monthly testing cycle is that if there is a problem in October, it can be quickly addressed even before the second semester starts. In subsequent months, as interventions are introduced, the change in target attitudes or behavior is observed to evaluate long-lasting effects and programs for improvement are re-evaluated, making sure that the programs stay effective.
Having a pulse monitor allows schools to quickly and easily compare themselves to all the other, anonymously, in near real-time. At the end of each test month, the results are calculated both for each school and for all schools participating as a whole. These results are released within days of the end deadline, not months or years later. This allows schools to get in and instantly compare themselves to the nation as well as seeing themselves improve. For instance, bullying, a school certainly has a general feeling if there are problems or not, but unless they have spent time at other schools, they have no idea what is normal. They might assume that a few kids getting teased here and there is what it’s like in all schools. Our system can quickly alert them and let them know that, perhaps no, that it is actually above the average rate. The principal can step in and setup stricter guidelines and continue to follow it up and see if things are getting better.
Our system is a custom web-based survey tool, written in PHP, backed by a mySQL database. By using open-source tools, we have been able to achieve massive jumps that the previous generation wasn’t able too. By skipping paper tests, we get much quicker results (the only reason it isn’t real-time is to protect anonymity). We can also look for much more advanced answering patterns than is available on paper. What if a student completes a page of the survey in 2 seconds? It is obvious to us they didn’t even read the questions, but using paper, no one would know and these faulty results are tabulated, skewing the results. By time-stamping everything we not only avoid these problems, but find new underlying trends.
Using the the web, building on free open-source software, borrowing new business practices like Agile, thinking of a school not as an institution, but a living organism and putting a laser focus on helping schools improve their environment, nothing like this has been built or could have been built before. Standing on the shoulders of the years of expertise put into statistical methods, hardened open source software and the web wasn’t an option 20 years ago. Now, a small team can set about making a difference by introducing something new and innovating into academics.
Skólapúlsinn isn’t a tool for some top-down management of our national education system. This is a tool for schools to improve the school life of our nations students. Today’s students are the next generation of Iceland’s workforce, brainpower and future. Giving them the best possible education, guidance and track today improves the opportunities we all have in the future. As the famous saying goes “A rising tide lifts all boats“. Spending time now improving the lives of today’s children, improves everyone’s life in the future. Much like the tide coming in, education assessment is used to monitor and improve the opportunities for our children. “Lifting the boats”, by improving our children’s education and the environment in which they learn, everyone comes along for the ride making society better.
Today we our survey software tests psychometric balance, but in the future technology will play and ever increasing role in how our children are taught and learn.