Review of The Math Academy Way, by Justin Skycak of Math Academy

A bit over a year ago, I reviewed How Children Learn, one of the foundational books of the unschooling movement. In my review, I expressed a certain amount of doubt that unschooling was the right approach for teaching math:

In fact, I think there are some things (math being an excellent example) which we should expect to require years of effort before they become natural and useful, but which are ultimately too valuable to pass over. With enough skill, it is certainly possible to non-coercively guide children to learn an appropriate amount of math (which I’ll define as math up to calculus and linear algebra) under their own motivation; I have seen it happen. But, even as someone who mostly hangs out with engineers, I know perhaps three people who do math for fun (maybe closer to ten if you count relatively small amounts of math done incidentally for personal projects). I think the heuristic of being non-coercive can get you a long way, but it also leaves too much value on the table to be the whole story.

This is supported by the fact that in the ACX survey, unschooled participants had worse math SAT scores than traditionally schooled participants. In some sense this is completely expected: of course people raised in an environment that completely de-emphasizes standardized test performance will perform worse on standardized tests.

…. An alternative explanation of the observations in How Children Learn, fueled by a synthesis of these books, might be that children can’t learn well when they’re not in emotional regulation. This suggests that children might have greater tolerance for coerced learning activities (like math) if you give them plenty of opportunities to feel smart, capable, and in charge - before, during, and after the work they’re not as enthusiastic about.

I generally try to steelman books in my reviews, because books can’t debate you back and it’s easy to dismiss a book as useless after you identify one or two bad ideas. But I recently read The Math Academy Way by Justin Skycak and it’s substantially changed my views on education (particularly, but not exclusively, math education). The Math Academy Way is many things: a manifesto, a giant piece of marketing material, and, above all, a Google Doc. You can find it here. I converted this to epub to read it on my iPad in December 2024, so this review won’t reflect changes after that date.

There are a few companies out there that promise to supercharge your child’s learning, sometimes in those exact words. I have experienced what I assume are the two most common types of private supplementary education: remedial tutoring and test preparation classes. But the most exciting types of private supplementary education don’t aim to help you eke out admission to that Ivy League school; they aim to create a generation of extraordinarily capable mathematicians, writers, and intellectuals of all stripes. One of these programs is Mentava, which focuses on teaching children to read when they’re very young (they say you should be able to use their program to have your kid reading at a basic level by age 4; kids adhering to the typical public school track reach this level closer to age 6-7). A two year head start by the time you’re four years old is seriously impressive. Math Academy is way more ambitious than this.

Math Academy was founded by Jason Roberts, a dad who was coaching an after-school program. He discovered that he could teach a small group of talented fifth graders math up to calculus and they would understand it and perform well. Understandably, this was exciting to him; if most people ever do calculus, they do it in college or perhaps the end of high school. If every fifth grader in his after-school program could learn it, the obvious implication is that school is extremely bad at teaching math. The theory in The Math Academy Way is that we allow this state of affairs to persist for a number of reasons, but mostly because we don’t think about education properly.

Talent Development

As a foil to modern math education, consider modern athletic training: though reasonable people may gripe about everyone getting a trophy nowadays, the truth is that there are abundant opportunities for talented and motivated kids to compete and train in sports at high levels. As of my childhood, travel leagues for the most popular sports started as early as 8-10 years old, and there were also all-star leagues and, I assume, even more advanced opportunities. When we train children for these leagues, we optimize in a very clearheaded way for talent development, which Skycak presents in contrast to “traditional schooling.” If a coach is lots of fun but doesn’t help our kids get better at basketball, we understand that we have to make a choice between raising excellent athletes and giving our kids a fun experience. Probably because of the frequent, public competitions involved in athletics (a backstop), we don’t often see people fooling themselves into thinking that actually inferior training techniques are better than they are (admittedly, there’s also a selection effect here: people who do this are eventually out-competed and lose visibility). We observe similar dynamics in musicianship.

These differences are particularly stark in the domain of math. One reason for this is that math has uniquely long dependency chains as compared to other subjects. It’s perfectly possible, for example, to learn all about American history between the end of WWII and the beginning of the Cold War with a bare minimum of knowledge of even causally related topics, but it’s impossible to learn calculus without knowledge of trigonometry. Long dependency chains create perverse incentives for teachers, who are evaluated according to students’ standardized test scores and punished (by complaints from parents and students) for teaching beyond those tests. Students are thus systematically underprepared for each successive math class, leading them to fall further behind unless they take their education into their own hands.

One interesting demonstration of this, which isn’t discussed in the book as far as I remember, comes from Bloom’s classic “two-sigma problem” paper. The most famous result is that the average student who receives one-on-one tutoring performs at the 98th percentile relative to the distribution of students who only receive conventional education in a 30-person class. But these curves also differ in another property: the tutored students exhibited a much lower standard deviation in their test scores. It looks to me like this may be because a substantial portion of the tutored students saturated the tests used for evaluation, but even based only on the left tail of student performance, there is a noticeable difference. This seems to validate the theory that student performance spreads out over time under conventional schooling based in part on the ability of the students to correct the gaps in their own knowledge, whereas tutoring largely eliminates this axis of variation.

You can also think about what happens at the right end of the bell curve. In math, excellent students are, at best, excused from class or accelerated by a grade. Imagine the equivalent of this for competitive sports: a coach identifies a talented athlete and either promotes him to play with older children (?) or excuses him from practice because he is already good enough (??!!). The promotion to play with older children is probably better than doing nothing, but it ignores an opportunity to nurture the player’s skill by paying extra attention to his training and form - it basically treats the player’s skill as a problem which causes boredom, and treats the boredom by introducing extra challenges. The most common outcome in sports and musicianship is to for coaches to nominate the child for private lessons and in some cases provide extra individual instruction on the side. This never happens in math class.

One reasonable question you might ask is: what if that’s just because math isn’t like sports or music? I feel like the Math Academy people (who are otherwise extremely thorough in rebutting possible counterarguments) overlook this a bit and maybe overuse the sports metaphors. Sports and musicianship seem to me to have a small set of extremely deep skills which require extensive practice to reach the point of perfection, whereas math seems like it has the opposite structure: a huge set of skills which are individually pretty easy to master but build on each other. It would make sense if math and sports benefited from different pedagogies. But from a first-principles perspective, it seems like math should benefit more from individualized instruction. If prerequisites are such an important part of math, it follows that the best thing to do when a math student gets a problem wrong is to figure out why and retrain them on the prerequisite knowledge they were missing: personalized instruction allows this, while even small group instruction makes it impossible. Meanwhile, athletic instruction can be relatively more straightforward (though I’m sure it’s highly personalized at elite levels). Having a problem dribbling? Practice dribbling. Or so I imagine, I was never an elite athlete.

One last point for this section is that colleges have a lot of leeway to do pretty much whatever they want (MIT has a pirate certificate, Brown lets you make up your own major, etc), but don’t seem to take education seriously. This is probably unsurprising to most people who attended large universities, but there are horror stories in the book about professors who try to implement a self-paced learning strategy called PSI (Personalized System of Instruction) in university classrooms:

… Avoiding a frontal attack, the chairman of the Psychology Department at Georgetown declared by fiat that something on the order of 50% of class time must be devoted to lecturing. By reducing the possibility of self-pacing to zero, this effectively eliminated PSI courses.

He issued this order on the grounds that in the context of lecturing ‘it is the dash of intellects in the classroom that informs the student.’ No data were presented on this point! The spectacle of purporting to defend scholarship while deciding the merits of instructional methods by assertion is silly.

There are also some good stories, including about MIT, which implemented an active learning system they called TEAL (Technology-Enhanced Active Learning):

“A typical [TEAL] class is comprised of mini lectures scattered throughout the class, separated by periods in which students are engaged in hands-on desktop experiments, visualizations, problem solving, and peer discussion.”

Unfortunately, despite the fact that TEAL got extremely good results in an experimental trial, it doesn’t look like any TEAL-related content has been published in the last 10 years, so it’s hard for me to assess its current use. At least, it doesn’t seem to have completely revolutionized how classes are taught at MIT.

Techniques

Having established that we would like math education to be a lot better and that there’s probably a lot of low-hanging fruit given that no one seems to have taken it seriously in recent history (I’ve previously written a little bit about aristocratic tutoring), the natural question is, how can we do better? Fortunately, there’s lots of research on this that has mostly been ignored. Here are a few techniques which empirically improve student performance:

The Math Academy book goes into a lot of detail about the research supporting these techniques and why they think they are underutilized. Thinking of higher education, an industry dominated by private institutions which, at first glance, you might imagine are incentivized to provide a high-quality education, it’s pretty incredible to see students passively sitting in lecture halls absorbing material that they won’t have to recall until the final exam. It’s about as close to the opposite of high-quality education as it’s possible to get.

Side note: you can get closer to the opposite of this list, by doing unschooling! I’d like to say that unschooling really appeals to me on a number of levels, and unschooling advocates would probably have a lot of questions about the methodology of studies that show improvements for these techniques - maybe improved test performance isn’t everything. But I’m increasingly convinced by the interpretation of unschooling that I introduced at the beginning of this post: probably most of the benefits are a combination of one-on-one tutoring with appropriate attention to the child’s emotional state

Math Academy

The book is Math Academy marketing material, and it worked on me! I feel pretty confident in my knowledge of all the math they currently teach, but will be signing up for abstract algebra once it’s up. The secret sauce of Math Academy is a manually constructed knowledge graph that covers all of math up through calculus. This graph has typed edges, so there are some edges representing:

This seems like an extremely cool approach to me, and I believe that they can get fantastic results from it. They also have an XP system that’s supposed to gamify the experience a bit and presumably make it easier to set goals. I really can’t wait to try out the system for myself!

Earlier Wordcells vs Better Shape Rotators

Math Academy is mostly for kids, and mostly marketed towards parents. Speaking as a non-parent, it seems like it would be important as a parent to carefully decide how you’re going to allocate your time, energy, and money. In this sense, Math Academy directly competes with Mentava in the space of ’things you could spend both money and interpersonal capital forcing your child to do.’ I think Mentava is an extremely cool project with lots of value to offer the world, but I’m not sure I would force my child to do it, whereas I do think I will force my child to do Math Academy. The main reason for this is that I think getting your child to read earlier isn’t particularly important.

Part of the reason for this is that I think it’s extremely realistic for kids to master reading early enough in a standard public education that they don’t lose out on very much. If you aren’t reading for fun until 3rd grade, that’s fine! Most people, including early readers, aren’t going to be reading challenging books until their teens anyway, and challenging books contain 90% of the value it’s possible to get from reading. 85% of native-born Americans are literate, and I suspect almost all of them could read at least some “classic literature” if they wanted to (think Hemmingway). Only a very small percentage of people ever master math through calculus in a comparable way (such that the basic operations feel effortless and the symbolic manipulation is a transparent representation of the underlying concept, such that they can understand important works in the field). Some of this is probably that math is more difficult than reading, and some of it is probably that math education has more room for improvement. At the end of the day, I think it’s likely that Math Academy will produce better mathematicians, whereas it seems like Mentava will produce earlier readers (but not necessarily better-read or more educated people).

I reached out to Niels Hoven on Twitter to see if he was aware of any evidence that learning to read early produced better outcomes for children. He said that he wasn’t aware of any and suspected that any correlation was probably not causal, so I asked him about the argument for using Mentava. I thought that maybe they were planning to build a larger suite of courses for comprehensive acceleration of early childhood education and reading was a prerequisite. But according to Niels, “the argument is just that helping children learn quickly is good. If someone doesn’t believe that without evidence they’re likely not our target demographic.” I think this is consistent with my comments above: like Math Academy, Mentava solves a real problem in education with a solution that teaches children much more efficiently than public school (I’m not aware of formal experiments supporting this, but purely based on priors I would bet on “app I heard about on Twitter” over public school). The problem Mentava solves happens to be a much smaller one, as measured by real-world impact, than the problem Math Academy is solving, but I think they’re both great examples of how much you can improve education if you take it seriously and I really appreciate Niels taking some time to answer my questions about it. Thank you, Niels!

Closing Thoughts

The Math Academy doc was, for me, the final nail in the coffin of unschooling. I still think that children benefit from substantially more autonomy than they typically get and that they can learn a lot of stuff in a self-directed manner, especially if the self-directed time is interspersed with challenges in a sort of Socratic style (I think history, especially, can benefit from a structure like this: you can just ask your kids to go research the answer to a question, and when they come back it’s almost always easy to challenge their answer and give them another avenue of exploration). I believe that giving a child opportunities to exercise autonomy early will help them develop into a curious, disciplined, connected adult. But it’s also clear that you can choose a few areas that you care about to practice diligently and effectively and produce outcomes way beyond anything a child would be able accomplish on their own. I think math should be one of those things, and Math Academy seems like they offer a great product for people looking for a one-stop solution to learn it.

You can find more information on Math Academy here.