What Do Tests Test?

Over the past couple of weeks at work, I’ve been working on revising some of the exams for our elementary school curriculum. This has been an interesting task full of challenges. One thing I’m constantly working on is putting myself in the headspace of a bright, but still young, elementary school student. What wording can I allow in problems? How long can a problem be before we’re testing their reading comprehension instead of their math? How many problems should there be? How many problems of a certain level of difficulty? There are so many questions to discuss, but one is a bit more fundamental than all others, and can help inform the answers to each subsequent question. What do we want our test to test?

Behind any successful test needs to be a certain ethos, a motive, and information you want it to glean. Do you want your test to check for memorization? Understanding? Talent? Even within a specific subject, how much should a students’ talents in other fields affect how they can do in the given test? This is particularly true in a math test, where the quantity of information (and the level of critical thinking necessary at all levels) is the focus, and other difficulties with context or wording can obscure the results you want.

At my company, we focus on problem solving. It is literally in our name. As such, we try to focus on writing problems that don’t necessarily require an incredible breadth of knowledge, but definitely require deep understanding and a level of persistence not found in every student. In the elementary school curriculum, where one expects a lot of knowledge checks and routine, plentiful practice that will be mimicked on the exam, we instead have multiplication puzzles, tricky problems that pull from their understanding of geometry and arithmetic, and a handful or problems we expect at most 10 percent of students to get.

Even a company with such a strong ethic for working hard on problems, and focusing on attacking problems, over and over, until they yield to us and our improved understanding, does not do everything perfectly. When the curriculum for our physical center was launched, not everybody was pleased with the initial state of assessment. So much at the elementary school level seemed to be focused on giving a final answer, and were perhaps a bit more difficult than even we wanted. Especially at a young age, getting a 50 percent on a test in the class you’re supposed to be in is a bit demoralizing. As a result, a number of conversations were had about how we can change our tests to reflect our philosophy better, and also know what the data we get means.

We test for deep understanding, and so we want high scores (say, above 90 percent) to be rather rare cases. But if everybody gets near a 50 percent, that does not provide any useful information. We want to be able to compare between students, teachers, and years as we move forward. How you design a test, word the problems, and organize the problems says a lot about what the test is for, and you want students to believe the test is for more than just a grade, but an actual reflection of their ability and the growth they can have. It should represent something they can use — not just for meta-gaming the system as many of us learned to do throughout college, figuring out how to pass the test instead of learn the material — but to improve their understanding of the material and of how they learn.

Of course, this is a lofty goal for elementary school kids. Their capacity for self-refection is a bit limited. But instilling in them a sense of ownership for their scores, a desire for doing better, and increased persistence, is important. I went through a lot of school doing well on exams, but knowing that it wasn’t necessarily because I was great at the material, it was because I knew how to take a test successfully. Some exams in college — where professors really knew what they were doing when writing a difficult test to assess your understanding — confirmed that impression.

It is difficult as a student to be faced with these challenges, but growing up with that struggle as much as possible leads to greater success further on. The company I am with has shown this to be true, and I’m working on changing my mindset to follow it similarly. It is far more about the journey, and most tests don’t reflect that viewpoint.

A Little More Music

Now that I’m working full-time, I’m getting used to spending significantly more time focusing consistently than when I was in school. Back in college, I could break up my work as I saw fit, take rests and roam around, or just slack off a bit any given day. That does not go over particularly well in an actual working environment.

For my entire life I have not listened to music while working. In college, during particularly long stretches of time while working on take-home exams, I was known to lock myself away and play some thunder sounds or other atmospheric noise. This had a calming effect without distracting my brain. The issue has always been a musical background. Any music that has any pattern to it (as most music does) is immediately locked onto by my brain. I cannot zone it out and focus on something else. Music with lyrics is significantly worse. Something like podcasts is unimaginable. So, I went through most of school completely baffled by seeing people with earbuds on while working intensely on their projects.

Then, I went into the adult world, where I have to focus eight hours a day, five days a week, at a mostly pre-determined timeframe. This itself was not the cause of me finally listening to music while working. Rather, it was the nature of the work. When I was in school, I don’t recall ever having work that was mindless to the point of being almost administrative or programmable. When doing math homework, there was significant thought put in during the entire process, including how to write it up in the best way. Now, while my work has these heavily thoughtful aspects, often times I am writing a collection of problems (say, a few sets of subtraction problems for 2nd graders), and once the numbers have been selected to emphasize the skill they are working on, I am left with creating and copying a rather formulaic problem statement and solution. It takes a good chunk of time to input these; but other than making sure I’m not making typos or forgetting to change any relevant metadata in a new problem, I’m going on autopilot. And finally, I found this space in which I can listen to music.

It goes beyond the music that I often have at the ready when driving with other people. I select that music because I enjoy it, but also because a majority of people also do. Now I’ve been loading up my phone with various pieces of percussion music, and artists that I hadn’t really explored but knew about. One of the best albums I’ve had on repeat for quite a while is The Hands That Thieve by Streetlight Manifesto. This was a band a friend of mine introduced me to in high school, but that just never took off because I didn’t listen to music. I gave it a chance, and it really pumps me through the day.

I’m sure to many people reading this, listening to music while working is nothing new. But it is a piece of excitement I’ve had that lets me enjoy my day a bit more, explore something new in my life, and have a bit of fun along with it all. It has also been interesting to me to think about how my musical taste has changed (and how it hasn’t!) over the years. What do I still listen to, what do I sing along to, and now what can I play while working that still keeps me on task? Beyond that, it is interesting to hear what other people listen to during similar situations. While I’m not necessarily looking for suggestions, I think musical taste says a fair amount about most people, or at least provides a bit of insight into their routine.

Of course, as I mentioned before, this is still a very compartmentalized aspect of my life. I am certainly not listening to anything as I write this — that would be awful. In fact, I have been trying to reduce the number of inputs in my life. I’m not going as far as C.G.P. Grey has recently, but I understand what he says about losing focus, and it resonates with me. That will be another post at some point.

I’m just excited to be back with music. My new adult life has allowed me to play music at a church, join a community band, and have some time to practice on my own. It has been a lot of fun, and implementing it into a few more parts of my life has been very enjoyable.

Trevor Project Donations

I recently put a short story I wrote on Amazon. It’s called When You Come Back.

You can find a link to it here.

When I originally wrote it, mental illness was not necessarily on my mind. But, the majority of its readers have told me it resonates with them to a fairly strong degree. So, I’ve put it up for $1 and any proceeds I receive will go towards The Trevor Project, in support of mental illness assistance.

If you’re not interested in the short story, or in providing Amazon with some of the overhead that comes with buying a Kindle book, you can donate to them directly.

If you want the story in PDF form, you can contact me.

New Chapter

I started my first post-graduation job at Art of Problem Solving this past week. I was an intern here last summer, and I was lucky enough for that to lead to a job. My official role is “Curriculum Developer”. I work on developing their elementary school math curriculum, as that is their current focus. It’s an incredibly good job, with fantastically intelligent and caring people.

While it is a great company whose mission I am deeply invested in, and San Diego is a beautiful place to be, these first few days have been very tough. It was weird driving across the country, having a good time, then suddenly getting to work. I’m living on my own, in a room I’m renting (technically an AirBnB) from a nice lady. I know the area from last summer, but I’m still getting over a mental hurdle of actually going out and doing things. Although at the time I write this I’ll have only been here a few days, I’m already feeling antsy. It’s strange.

Perhaps the biggest reason for my feelings is that I spent my entire life in Minnesota. I am very rooted there, and despite many friends leaving who have also graduated, there are many people I’ve left behind. I’m leaving the comfortable world of academia to work at a place where I have no true connections. It’s a rather isolating feeling that I am working through.

Yet, this is something I correctly anticipated. I have been solidifying Operation: Have a Conversation and Comical Start as ways to keep in contact with people. I’ve reached out to people (or luckily have had them reach out to me) to stay in contact via phone calls or letters. And I also committed to myself that I would write weekly on this blog, and not worry about people reading it. It’s just a good thing to have on my schedule, both for the purpose of self-reflection, and to stop myself from falling idle after I do a day of work.

I already reached out to the San Diego Concert Band, a local community band that has fairly open policies for joining. I’ll be rehearsing with them regularly starting next week, which I am incredibly thrilled for. Although my percussion chops are not what they once were, they will improve and I will be better off for having the experience. I also plan on finding a group (or maybe just a person or two) to try and play tennis with. It’s an easy sport to play as long as you have another person, and I definitely enjoy playing it. Ideally I’d find a softball league as well, and I also have a long book list to get through.

Despite a touch of melancholy and some misgivings on traveling so far, I am excited for what is to come. Knowing that my time here is rather indefinite, I can feel more comfortable finding my place and joining new things. I have more opportunity to be involved and help myself as I go along. It’s an interesting time.

Abducted: A 24-Hour Musical

This past weekend I had the fantastic experience of playing drumset in a musical put together in only 24 hours. My friend Tim, along with his friend Adam, wrote the entirety of the show. We showed up at Friday on 7pm, with nobody having seen the script or music except the writers. We then performed the musical — lines memorized, music rehearsed, choreography and blocking complete — at 7pm (and 9pm) Saturday evening.

I had an extremely good time. The music was engaging and written with some interpretation allowable, as all the members of the pit were experienced in this musical scenario. We had a lot of fun putting things together quickly, and were quite successful in performing our parts within a few hours.

The show was broadly a satirical take on the characters from Scooby-Doo. In addition to the normal gang (whose names are never explicitly stated at any point in the show), there is the scapegoat Brian, who is Daphne’s current boyfriend. He is verbally abused throughout the show, with some light slapping. In addition, Scooby-Doo is just a man in a Scooby-Doo outfit (naturally), although an old Hermit we meet at the beginning addresses this fact:

Velma: Oh, that’s our anthropomorphic dog. He loves food, and hates ghosts. So, we keep him on a leash and force him to solve crimes!

Hermit (Cooper): That’s not a dog! That is clearly a man in constant pain!

Fred: (Firmly, maintaining eye contact.) It’s a dog.

The fact that the gang keeps a man in a suit on a leash, and is either indifferent to this fact or somehow unaware of its humanity, is repeated throughout the show. Every time a character (normally Brian) goes to take his leash, Scooby screams in terror. At one point, a completely silent scene opens with Scooby alone on stage. Scared by the audience, he slowly stands on both legs, moves forward, and proceeds to intently say Help me! to various members of the audience.

What I hope to show with these descriptions, as well as the plot description to follow, is the creativity and fun that Tim and Adam bring to the shows they write, as well as some interest in watching their other shows online.

Now for the plot. The aforementioned hermit opens the show, describing to Fred and Velma how he was abducted and probed by aliens. Once all the characters were quickly introduced, they were subsequently abducted. During the musical sequence, a small creature, reminiscent of the chest-bursting alien from Alien, kills Fred during the “probing” procedure. We then meet the head of the ship, Marvin.

Shaggy accuses him of working too hard, so he takes Marvin away to “relax”. We later learn their natural high is Captain Crunch.

Daphne falls into a motherly love, with a bit of sexual tension, with the alien in Fred’s chest, much to Brian’s dismay. Velma proceeds to look for clues, until she meets another female alien with the exact same disposition and actions. Some innuendos occur.

Brian sings his heart out to the audience, beautifully I may add.

We finally learn the Hermit is on the ship. His plan was to run everybody out of town with talk about aliens, so he could have the oil deposit he discovered entirely to himself. Of course, he is now abducted by aliens so there is not much to do. Brian and Scooby were within earshot, and attempt to tell the rest of the gang.

Daphne appears, crazed, yielding a gun, rounding up the aliens and, as Marvin put it, “Yelling lines from Alien 3.” Of course, the alien in Fred’s chest talks her down, at which point another chest-bursting alien emerges from Daphne. Those two aliens go to town.

Brian explains to Velma that he solved the crime, and attempts to remove the mask (as is customary). Of course, being Brian, he takes some other alien and nearly chokes it. At this point, Scooby goes up to the Hermit, who is wearing a very obvious mask, and removes it. It is revealed the culprit is none other than D.B. Cooper.

At this point, we have “A case with no loose ends”, as Shaggy puts it. Of course we then realize the characters are still on an alien spaceship. Brian asks Marvin if they can be dropped off on Earth. Marvin declines, but makes an allowance that they can stay another day, until they are eaten.

Due to the inevitability of their demise, Brian removes Scooby’s collar. Scooby rises to his foot, removes the dog head from his head, and embraces Brian. The show ends on this scene.


It was an incredible show. Things weren’t perfect, but they were as imperfect as the musicals I did pit for that had weeks of rehearsal. It was a great experience, and while Tim and Adam have just graduated college, making another musical unlikely in the near future, I am excited to hear about whatever creative endeavors they have moving forward.

Dull Edge

The cutting edge of technology is particularly awesome these days. Cars are doing more on their own, phones are surpassing some current computers in their performance, and VR is coming into its own finally. I listen to a lot of tech podcasts, and love messing around with technology, but due to my status as a recent college graduate, I am definitely not maintaining a collection of cutting-edge devices. And that’s okay.

First, let’s talk about cars. I recently purchased a post-lease 2015 Honda Civic, LX trim (in other words, the only model more basic comes with a manual instead of a CVT.) The disparity between that car and other higher-end cars from the same year is rather shocking. Sitting in it, I feel super cool. It’s a big upgrade from the 1998 EX-L Honda Accord I had been driving. There’s a “cockpit” feeling to it, good Bluetooth connectivity, and a back-up camera. It’s relatively zippy for a cheap car, and the gas mileage gained by the CVT cannot be beat.

Then, I read a Golf R Review by Casey Liss. He is one of three car enthusiasts on the Accidental Tech Podcast. Not surprisingly, the only one I can identify with is John Siracusa, who to my knowledge has mostly driven manual Civics and Accords for his entire life. Casey though, he complained about the lack of assisted driving and automatic parking. The car needs to be fast, it needs a sunroof, and of course Carplay! This was absolutely baffling to me. I just cannot get my head wrapped around why some of these things are important. These three men have attempted to address it on their podcast, but it still does not click with me. Cars can be purchased only so often to be at all reasonable, and so one cannot even stay on the cutting edge.

The other issue is that he recently began working from home exclusively, yet sounded very hesitant to become a one-car family. Oh well, to each their own.

Now, obviously cars are a very special case of not staying on the cutting edge. I’ve only owned my own car for a few months, and it’ll be a number of years before I can even begin to think about getting another. But phones: now that’s another matter.

I’ve had my Galaxy S7 for two years now. Previously, I had a Galaxy S4 for two years, and then some random LG (I think) phone for 4 years throughout high school. In my mind, until I can afford the “every year” phone upgrade, two years is a reasonable time in these days of mostly non-replaceable phone batteries. So, with my S7 really slowing down and the battery life starting to tank, it was time to figure out what to do. Was the newest Galaxy S9 worth the incredible price tag? Did I want to save some money and get a Pixel XL or a new LG phone, each containing last year’s processors, despite them being the newest in the lineup?

I ended up choosing a Galaxy S8+. Due to the release of the S9, I got a very good deal on it, and the processing difference (and battery life) between the S7 and S8+ is much larger than the S8+ to a comparable newest generation phone. Once again, I opted to stay on the duller edge of technology. And I am happy with that decision. I tried the S9 in stores, and it truly did not impress me anymore than the S8 does. The S8 was the revolutionary phone (just like the iPhone X, and whatever comes out next will not be quite as lauded).

This is what is interesting about technology. So many people are excited to get the newest and best thing. The hype is always there, but the price-to-performance normally isn’t. I spent all of last year in school working on a 4-year old Ideapad and a 5-year old refurbished Thinkpad. They performed admirably for me, because like most people, I’m not doing much heavy-lifting.

Being on the dull edge, and looking out at what is available and what others have, can be fun. I don’t think there is anything wrong with living life on some sort of delay with technology. Perhaps as I grow older and make a bit more money, that will change. But for now, I am happy with scouring the internet for good deals, and getting what I actually need for the best price.

Side Projects (Part 1?)

I think it is important to have a variety of projects capturing one’s attention. The breadth and depth of these will vary by individual, but they should be there nonetheless. Someone who is incredibly invested in one particular field or interest will be more aware of the branching-off points, and can thus develop projects related to the disparate branches of that field. Others may be interested in many topics, and have projects related to each.

I fall into the latter category, as do a good chunk of my friends. I have become widely interested in many topics throughout college, and this was one of the main reasons I did not immediately pursue higher education. While I love mathematics, I cannot see myself devoting a majority of my life to only studying it for the next five years, and wanted the opportunity to do many things I did not do in college, or double-down on some of the projects I started then.

In addition to this blog that I am trying to keep up with better, there are the podcasts Operation: Have a Conversation and Comical Start. There was the joke-blog I announced a couple of weeks ago. I’m trying to read more, and still keep up with doing some math so that I can be more effective at my new job I am starting soon. I have been playing tennis more, and joined a softball league while I’m still in Minnesota.

These projects keep me busy, and keep me happy. I like to have a variety of things to work on, because I’ve always loved each subject I’ve been introduced to. My passion for them may diminish at times — I’ll always¬† be more invested in math than in biology — but being able to have conversations or read a few articles about new ideas is exciting. Writing this blog is exciting, and talking with my friends and editing podcasts is invigorating. Playing newer and older sports to me is always a good time, because I like to stretch the muscles I’ve worked all my life, but also pick up new skills. The internal growth I want to achieve is being reflected in the growth in new activities I’m participating in. There will be more to come about that last sentence.

Shameless Plug

An extension cord walks in on its son, a vacuum (three-pronged cord of course), plugging itself into an electrical outlet. Aghast at what it sees, the extension cord can only cry out: “You shameless plug, you’re grounded!”


I’m going on vacation this upcoming week. This original joke is the best I could do. The actual shameless plug is my friend Brandon’s review blog, as well as my competing review of his blog, where I verbally abuse and critique his writing even if I have not experienced whatever he is reviewing.

It’s all in good fun.