What Do Tests Test?

Over the past couple of weeks at work, I’ve been working on revising some of the exams for our elementary school curriculum. This has been an interesting task full of challenges. One thing I’m constantly working on is putting myself in the headspace of a bright, but still young, elementary school student. What wording can I allow in problems? How long can a problem be before we’re testing their reading comprehension instead of their math? How many problems should there be? How many problems of a certain level of difficulty? There are so many questions to discuss, but one is a bit more fundamental than all others, and can help inform the answers to each subsequent question. What do we want our test to test?

Behind any successful test needs to be a certain ethos, a motive, and information you want it to glean. Do you want your test to check for memorization? Understanding? Talent? Even within a specific subject, how much should a students’ talents in other fields affect how they can do in the given test? This is particularly true in a math test, where the quantity of information (and the level of critical thinking necessary at all levels) is the focus, and other difficulties with context or wording can obscure the results you want.

At my company, we focus on problem solving. It is literally in our name. As such, we try to focus on writing problems that don’t necessarily require an incredible breadth of knowledge, but definitely require deep understanding and a level of persistence not found in every student. In the elementary school curriculum, where one expects a lot of knowledge checks and routine, plentiful practice that will be mimicked on the exam, we instead have multiplication puzzles, tricky problems that pull from their understanding of geometry and arithmetic, and a handful or problems we expect at most 10 percent of students to get.

Even a company with such a strong ethic for working hard on problems, and focusing on attacking problems, over and over, until they yield to us and our improved understanding, does not do everything perfectly. When the curriculum for our physical center was launched, not everybody was pleased with the initial state of assessment. So much at the elementary school level seemed to be focused on giving a final answer, and were perhaps a bit more difficult than even we wanted. Especially at a young age, getting a 50 percent on a test in the class you’re supposed to be in is a bit demoralizing. As a result, a number of conversations were had about how we can change our tests to reflect our philosophy better, and also know what the data we get means.

We test for deep understanding, and so we want high scores (say, above 90 percent) to be rather rare cases. But if everybody gets near a 50 percent, that does not provide any useful information. We want to be able to compare between students, teachers, and years as we move forward. How you design a test, word the problems, and organize the problems says a lot about what the test is for, and you want students to believe the test is for more than just a grade, but an actual reflection of their ability and the growth they can have. It should represent something they can use — not just for meta-gaming the system as many of us learned to do throughout college, figuring out how to pass the test instead of learn the material — but to improve their understanding of the material and of how they learn.

Of course, this is a lofty goal for elementary school kids. Their capacity for self-refection is a bit limited. But instilling in them a sense of ownership for their scores, a desire for doing better, and increased persistence, is important. I went through a lot of school doing well on exams, but knowing that it wasn’t necessarily because I was great at the material, it was because I knew how to take a test successfully. Some exams in college — where professors really knew what they were doing when writing a difficult test to assess your understanding — confirmed that impression.

It is difficult as a student to be faced with these challenges, but growing up with that struggle as much as possible leads to greater success further on. The company I am with has shown this to be true, and I’m working on changing my mindset to follow it similarly. It is far more about the journey, and most tests don’t reflect that viewpoint.

A Little More Music

Now that I’m working full-time, I’m getting used to spending significantly more time focusing consistently than when I was in school. Back in college, I could break up my work as I saw fit, take rests and roam around, or just slack off a bit any given day. That does not go over particularly well in an actual working environment.

For my entire life I have not listened to music while working. In college, during particularly long stretches of time while working on take-home exams, I was known to lock myself away and play some thunder sounds or other atmospheric noise. This had a calming effect without distracting my brain. The issue has always been a musical background. Any music that has any pattern to it (as most music does) is immediately locked onto by my brain. I cannot zone it out and focus on something else. Music with lyrics is significantly worse. Something like podcasts is unimaginable. So, I went through most of school completely baffled by seeing people with earbuds on while working intensely on their projects.

Then, I went into the adult world, where I have to focus eight hours a day, five days a week, at a mostly pre-determined timeframe. This itself was not the cause of me finally listening to music while working. Rather, it was the nature of the work. When I was in school, I don’t recall ever having work that was mindless to the point of being almost administrative or programmable. When doing math homework, there was significant thought put in during the entire process, including how to write it up in the best way. Now, while my work has these heavily thoughtful aspects, often times I am writing a collection of problems (say, a few sets of subtraction problems for 2nd graders), and once the numbers have been selected to emphasize the skill they are working on, I am left with creating and copying a rather formulaic problem statement and solution. It takes a good chunk of time to input these; but other than making sure I’m not making typos or forgetting to change any relevant metadata in a new problem, I’m going on autopilot. And finally, I found this space in which I can listen to music.

It goes beyond the music that I often have at the ready when driving with other people. I select that music because I enjoy it, but also because a majority of people also do. Now I’ve been loading up my phone with various pieces of percussion music, and artists that I hadn’t really explored but knew about. One of the best albums I’ve had on repeat for quite a while is The Hands That Thieve by Streetlight Manifesto. This was a band a friend of mine introduced me to in high school, but that just never took off because I didn’t listen to music. I gave it a chance, and it really pumps me through the day.

I’m sure to many people reading this, listening to music while working is nothing new. But it is a piece of excitement I’ve had that lets me enjoy my day a bit more, explore something new in my life, and have a bit of fun along with it all. It has also been interesting to me to think about how my musical taste has changed (and how it hasn’t!) over the years. What do I still listen to, what do I sing along to, and now what can I play while working that still keeps me on task? Beyond that, it is interesting to hear what other people listen to during similar situations. While I’m not necessarily looking for suggestions, I think musical taste says a fair amount about most people, or at least provides a bit of insight into their routine.

Of course, as I mentioned before, this is still a very compartmentalized aspect of my life. I am certainly not listening to anything as I write this — that would be awful. In fact, I have been trying to reduce the number of inputs in my life. I’m not going as far as C.G.P. Grey has recently, but I understand what he says about losing focus, and it resonates with me. That will be another post at some point.

I’m just excited to be back with music. My new adult life has allowed me to play music at a church, join a community band, and have some time to practice on my own. It has been a lot of fun, and implementing it into a few more parts of my life has been very enjoyable.

Movie Review: BlackKklansman

I saw this movie a little over a week ago, and have been trying to work through how to write a review of it. It is the type of movie that is funny, but speaks to something much broader; as such, I want to make sure the comedic elements do not overshadow the intention of the movie, but also want to commend the way the movie inserts comedy in such a way that goes against modern comedic sensibilities.

The other issue in my mind when writing this review is that of language. This movie deals most directly with racism and antisemitism, and does not shy away from the language that would be used 50 years ago in a town that leaned toward bigoted behavior. Naturally I am no person to be employing the terms used in the movie, despite their usage being vital to the message and impact of the movie. So, I will be rather careful when writing this, and focus mostly on the impact. You can always read a plot summary on Wikipedia.


This movie is important, and I don’t think that popular culture will view it in this light. It takes a difficult issue — race and bigotry — and turns it on its head by using a true story that is ultimately amusing at its core. The idea of a black cop and white (Jewish) cop teaming up 40 to 50 years ago to infiltrate the KKK is patently absurd on the surface. Yet, once you see it play out, you can understand how it occurred so successfully. The premise being accepted, I could then focus on what the movie was really saying. What it was saying is that to some degree, our society has regressed over the past 50 years. We though we had made, and were actively making, progress towards the equality of every citizen in America, yet the current political climate and the actions that have been enabled by our current President have shown us quite the opposite is true.

There are certain statements made throughout the movie that effectively break the fourth wall, and speak directly to the viewer of the movie. When our main character is scoffing at the idea of someone like David Duke being elected president, the other officers set him straight, letting him know that once the face of racism has changed to something more palatable, it can be put in front of the American people as something worth voting for. It can be overlooked if the other talking points of a candidate are sufficiently engaging.

This movie is also a plea for peaceful progress and resolution to the race issue in our country. The Black Student Union plays a large role in the movie, and by extension so does the Black Power movement, notorious in its day for their intimidating techniques. Our main character, being a black police officer, keeps emphasizing his belief that systems can be changed from the outside without resorting to more drastic measures. Even in the face of a potentially violent demonstration on both sides, he sticks to these values and shows success, even if it is localized, can occur.

The end of the movie is much more explicit. It shows video of the violent and lethal protests in Charleston from 2017, in which a car drove at high speed into a group of demonstrators multiple times. President Trump refused to condemn any particular “side” of the altercation. Subsequently we are shown video of David Duke (who played a prominent role in the movie) giving a speech, saying that the President’s remarks are affirmation of their beliefs of white supremacy.

While this movie will not necessarily change the tide of society — the echo chamber that our world is in makes any one (or thousand) of individuals able to do so — it draws parallels to an earlier time. It points out that we have not made the progress we believe we have in this country, and there is significant work to do. It is not hopeless, but it is dangerous and frustrating.

I highly recommend everybody watch this movie. It will not change people’s minds, because our society has become too combative for that. But, it tells a wonderful story about persistence and change, when we live in a world where it is difficult to keep the former and obtain the latter.


Trevor Project Donations

I recently put a short story I wrote on Amazon. It’s called When You Come Back.

You can find a link to it here.

When I originally wrote it, mental illness was not necessarily on my mind. But, the majority of its readers have told me it resonates with them to a fairly strong degree. So, I’ve put it up for $1 and any proceeds I receive will go towards The Trevor Project, in support of mental illness assistance.

If you’re not interested in the short story, or in providing Amazon with some of the overhead that comes with buying a Kindle book, you can donate to them directly.

If you want the story in PDF form, you can contact me.

Mathbook: Introduction to Sets

Quite a while ago, an endeavoring individual tried to start an open-source repository of mathematical information called Mathbook. I contributed an article, which I’ll put down in two parts on this blog.

It seems that the project has died, although the website is still available. While this is a bit of a shame, I would like to give some of my own little lessons here. The creator’s idea behind Mathbook was to focus on giving people an understanding of why we do math in a certain way. This is missing from mathematical curriculum today, but it is vital to understand that when math was developed, decisions were made for specific reasons. Moving forward, I’ll occasionally add a new post here to that effect. The people in my life don’t always understand the math I learned, so this is part of my effort in showing it.

Continue reading “Mathbook: Introduction to Sets”

New Chapter

I started my first post-graduation job at Art of Problem Solving this past week. I was an intern here last summer, and I was lucky enough for that to lead to a job. My official role is “Curriculum Developer”. I work on developing their elementary school math curriculum, as that is their current focus. It’s an incredibly good job, with fantastically intelligent and caring people.

While it is a great company whose mission I am deeply invested in, and San Diego is a beautiful place to be, these first few days have been very tough. It was weird driving across the country, having a good time, then suddenly getting to work. I’m living on my own, in a room I’m renting (technically an AirBnB) from a nice lady. I know the area from last summer, but I’m still getting over a mental hurdle of actually going out and doing things. Although at the time I write this I’ll have only been here a few days, I’m already feeling antsy. It’s strange.

Perhaps the biggest reason for my feelings is that I spent my entire life in Minnesota. I am very rooted there, and despite many friends leaving who have also graduated, there are many people I’ve left behind. I’m leaving the comfortable world of academia to work at a place where I have no true connections. It’s a rather isolating feeling that I am working through.

Yet, this is something I correctly anticipated. I have been solidifying Operation: Have a Conversation and Comical Start as ways to keep in contact with people. I’ve reached out to people (or luckily have had them reach out to me) to stay in contact via phone calls or letters. And I also committed to myself that I would write weekly on this blog, and not worry about people reading it. It’s just a good thing to have on my schedule, both for the purpose of self-reflection, and to stop myself from falling idle after I do a day of work.

I already reached out to the San Diego Concert Band, a local community band that has fairly open policies for joining. I’ll be rehearsing with them regularly starting next week, which I am incredibly thrilled for. Although my percussion chops are not what they once were, they will improve and I will be better off for having the experience. I also plan on finding a group (or maybe just a person or two) to try and play tennis with. It’s an easy sport to play as long as you have another person, and I definitely enjoy playing it. Ideally I’d find a softball league as well, and I also have a long book list to get through.

Despite a touch of melancholy and some misgivings on traveling so far, I am excited for what is to come. Knowing that my time here is rather indefinite, I can feel more comfortable finding my place and joining new things. I have more opportunity to be involved and help myself as I go along. It’s an interesting time.

Movie Review: “Christopher Robin”

The other day my girlfriend and I went to see the film Christopher Robin, all about the titular character outgrowing his friends in the 100 acre wood, and slowly finding his way back to childhood. After leaving the movie close to tears (my girlfriend did cry multiple times), here was my one line review:

The entire movie was super predictable, but it was so well-done and moving that I didn’t even care.

Continue reading “Movie Review: “Christopher Robin””

Gabriel’s Horn

Here is one of my favorite “paradoxes” in mathematics, that many people will learn in a first year calculus course. It is called “Gabriel’s Horn.”

First, we take the function f(x) = \dfrac{1}{x}. This is a curve that we can imagine beginning at the point (1,1), then quickly sloping down towards the x-axis at which point it becomes nearly horizontal. A plot is shown below.


Now imagine taking that curve and rotating it around the x-axis, forming an infinitely long shape that looks like the bell of a trumpet, or horn. After doing so, we get a shape that looks roughly like this:


There are two things we wish to determine about this new three-dimensional figure: its volume, and its surface area. We can think of the volume simply as how much stuff we need to fill it completely, while the surface area is how much paint we need to coat the surface. As we will show, this figure has finite volume but infinite surface area. What this means is that we would need an infinite amount of paint in order to just cover the inside surface with a layer of paint. However, if we really wanted to paint the inside, we could take our finite amount of paint and just “pour it in” the horn. We could fill up this infinite horn with a finite amount of paint, and thus end up painting the inside.

The reason this becomes fun is because this is an infinite object. The skinny part of the bell goes on forever, toward +\infty. But this is no obstacle for some tools in calculus. The main idea of calculus is to split your object into arbitrarily thin or short pieces. In our case, we will take an arbitrary vertical slice out of our horn at some horizontal point we’ll call x_1. The disk we get as a result will be circular, and have some radius r. Consider the figure below.


The radius is the distance from the x-axis to the curve f(x) = \dfrac{1}{x}. But by definition, this is just the y-coordinate, f(x_1) = \dfrac{1}{x_1}. So, for any arbitrary vertical slice taken out of our horn, we can find the radius. This allows us to easily find the circumference (2\pi r) and area (\pi r^2) of each disk.

At this point, we’ll have to assume some knowledge of calculus. Consider taking every possible arbitrarily thin disk in the horn, infinitely many of them! If we add their areas over and over again, \pi r^2 = \pi \left(\dfrac{1}{x}\right)^2, multiplied by their tiny piece of width we call dx, we will obtain the volume of the horn overall. We can do similarly with the circumference, multiplying each tiny bit of circumference, 2 \pi r = 2\pi \dfrac{1}{x}, with that same bit of width dx. This will give us our surface area.

We do this formally by using integration. In particular, we use an “improper” integral, and technically have to take a limit (of course, once I finished calculus I got too lazy to write out those steps normally, but I will now for completeness.)

For volume, our integral is

\displaystyle \text{Volume} =  \pi \int\limits_1^\infty \dfrac{1}{x^2}dx = \pi \lim\limits_{b\to\infty} \int\limits_1^b \dfrac{1}{x^2}dx

We can evaluate this integral directly, getting

\displaystyle \text{Volume} = \pi \lim\limits_{b\to\infty}\left. -\dfrac{1}{x} \right|_1^b = \pi \lim\limits_{b\to\infty} \left( -\dfrac{1}{b} + 1\right) = \boxed{\pi}.

That -\dfrac{1}{b} goes to 0, so that term disappears. As we stated earlier, the volume is finite. Choose your units, and the volume of that infinite horn above is just \pi. Neat!

We can do similarly for the surface area, but our integral will diverge.

\displaystyle \text{Surface Area} = 2\pi\int\limits_1^\infty \dfrac{1}{x}dx = 2\pi \lim\limits_{b\to\infty} \int_1^b \dfrac{1}{x}dx = 2\pi \lim\limits_{b\to\infty} \left(\ln b - \ln 1\right) = \boxed{\infty}.

Since \ln 1= 0, we just get \ln b going off to infinity, hence the surface area is infinite.

There are many cool problems like this, some with significantly less background necessary to formulate. I’m hoping to mix these in with some other types of posts.