What Do Tests Test?

Over the past couple of weeks at work, I’ve been working on revising some of the exams for our elementary school curriculum. This has been an interesting task full of challenges. One thing I’m constantly working on is putting myself in the headspace of a bright, but still young, elementary school student. What wording can I allow in problems? How long can a problem be before we’re testing their reading comprehension instead of their math? How many problems should there be? How many problems of a certain level of difficulty? There are so many questions to discuss, but one is a bit more fundamental than all others, and can help inform the answers to each subsequent question. What do we want our test to test?

Behind any successful test needs to be a certain ethos, a motive, and information you want it to glean. Do you want your test to check for memorization? Understanding? Talent? Even within a specific subject, how much should a students’ talents in other fields affect how they can do in the given test? This is particularly true in a math test, where the quantity of information (and the level of critical thinking necessary at all levels) is the focus, and other difficulties with context or wording can obscure the results you want.

At my company, we focus on problem solving. It is literally in our name. As such, we try to focus on writing problems that don’t necessarily require an incredible breadth of knowledge, but definitely require deep understanding and a level of persistence not found in every student. In the elementary school curriculum, where one expects a lot of knowledge checks and routine, plentiful practice that will be mimicked on the exam, we instead have multiplication puzzles, tricky problems that pull from their understanding of geometry and arithmetic, and a handful or problems we expect at most 10 percent of students to get.

Even a company with such a strong ethic for working hard on problems, and focusing on attacking problems, over and over, until they yield to us and our improved understanding, does not do everything perfectly. When the curriculum for our physical center was launched, not everybody was pleased with the initial state of assessment. So much at the elementary school level seemed to be focused on giving a final answer, and were perhaps a bit more difficult than even we wanted. Especially at a young age, getting a 50 percent on a test in the class you’re supposed to be in is a bit demoralizing. As a result, a number of conversations were had about how we can change our tests to reflect our philosophy better, and also know what the data we get means.

We test for deep understanding, and so we want high scores (say, above 90 percent) to be rather rare cases. But if everybody gets near a 50 percent, that does not provide any useful information. We want to be able to compare between students, teachers, and years as we move forward. How you design a test, word the problems, and organize the problems says a lot about what the test is for, and you want students to believe the test is for more than just a grade, but an actual reflection of their ability and the growth they can have. It should represent something they can use — not just for meta-gaming the system as many of us learned to do throughout college, figuring out how to pass the test instead of learn the material — but to improve their understanding of the material and of how they learn.

Of course, this is a lofty goal for elementary school kids. Their capacity for self-refection is a bit limited. But instilling in them a sense of ownership for their scores, a desire for doing better, and increased persistence, is important. I went through a lot of school doing well on exams, but knowing that it wasn’t necessarily because I was great at the material, it was because I knew how to take a test successfully. Some exams in college — where professors really knew what they were doing when writing a difficult test to assess your understanding — confirmed that impression.

It is difficult as a student to be faced with these challenges, but growing up with that struggle as much as possible leads to greater success further on. The company I am with has shown this to be true, and I’m working on changing my mindset to follow it similarly. It is far more about the journey, and most tests don’t reflect that viewpoint.

A Little More Music

Now that I’m working full-time, I’m getting used to spending significantly more time focusing consistently than when I was in school. Back in college, I could break up my work as I saw fit, take rests and roam around, or just slack off a bit any given day. That does not go over particularly well in an actual working environment.

For my entire life I have not listened to music while working. In college, during particularly long stretches of time while working on take-home exams, I was known to lock myself away and play some thunder sounds or other atmospheric noise. This had a calming effect without distracting my brain. The issue has always been a musical background. Any music that has any pattern to it (as most music does) is immediately locked onto by my brain. I cannot zone it out and focus on something else. Music with lyrics is significantly worse. Something like podcasts is unimaginable. So, I went through most of school completely baffled by seeing people with earbuds on while working intensely on their projects.

Then, I went into the adult world, where I have to focus eight hours a day, five days a week, at a mostly pre-determined timeframe. This itself was not the cause of me finally listening to music while working. Rather, it was the nature of the work. When I was in school, I don’t recall ever having work that was mindless to the point of being almost administrative or programmable. When doing math homework, there was significant thought put in during the entire process, including how to write it up in the best way. Now, while my work has these heavily thoughtful aspects, often times I am writing a collection of problems (say, a few sets of subtraction problems for 2nd graders), and once the numbers have been selected to emphasize the skill they are working on, I am left with creating and copying a rather formulaic problem statement and solution. It takes a good chunk of time to input these; but other than making sure I’m not making typos or forgetting to change any relevant metadata in a new problem, I’m going on autopilot. And finally, I found this space in which I can listen to music.

It goes beyond the music that I often have at the ready when driving with other people. I select that music because I enjoy it, but also because a majority of people also do. Now I’ve been loading up my phone with various pieces of percussion music, and artists that I hadn’t really explored but knew about. One of the best albums I’ve had on repeat for quite a while is The Hands That Thieve by Streetlight Manifesto. This was a band a friend of mine introduced me to in high school, but that just never took off because I didn’t listen to music. I gave it a chance, and it really pumps me through the day.

I’m sure to many people reading this, listening to music while working is nothing new. But it is a piece of excitement I’ve had that lets me enjoy my day a bit more, explore something new in my life, and have a bit of fun along with it all. It has also been interesting to me to think about how my musical taste has changed (and how it hasn’t!) over the years. What do I still listen to, what do I sing along to, and now what can I play while working that still keeps me on task? Beyond that, it is interesting to hear what other people listen to during similar situations. While I’m not necessarily looking for suggestions, I think musical taste says a fair amount about most people, or at least provides a bit of insight into their routine.

Of course, as I mentioned before, this is still a very compartmentalized aspect of my life. I am certainly not listening to anything as I write this — that would be awful. In fact, I have been trying to reduce the number of inputs in my life. I’m not going as far as C.G.P. Grey has recently, but I understand what he says about losing focus, and it resonates with me. That will be another post at some point.

I’m just excited to be back with music. My new adult life has allowed me to play music at a church, join a community band, and have some time to practice on my own. It has been a lot of fun, and implementing it into a few more parts of my life has been very enjoyable.