T-assessment: a bold suggestion modestly advanced

I sometimes use this column to float an idea I think deserves attention. Not on a whim, but after considerable thought and discussion with others expert in the relevant domain(s). This is one of those times. I already set the scene with last month’s post. Here is a nuanced, bullet-point summary of what I wrote then: 

1. The heart of learning mathematics is mastering a particular way of thinking – what I (and some others) call “mathematical thinking,” sometimes also described as “thinking like a mathematician.”
2. You can master mathematical thinking by focusing on any one branch of mathematics – arithmetic, geometry, algebra, trigonometry, calculus, etc. – and going fairly deep.
3. Once you have mastered mathematical thinking, you can fairly quickly acquire an equivalent mastery of any branch of mathematics with relatively shallow coverage. (There are limits to this. There is a complexity and abstraction hierarchy of branches of mathematics. But for K-12 mathematics, that is not an issue.)
4. Thus, to learn mathematics effectively, it suffices to (i) master mathematical thinking by the study of one branch of the subject, and (ii) acquire some breadth by branching out to a few other areas.
5. For the same reason, it meets society’s need for assessment of mathematics learning if we (i) assess mathematical thinking restricted to one branch, and (ii) measure the individual’s knowledge in a number of other branches.

If you buy this picture – and if you keep reading, I am going to try to sell it to you – then it means we can make use of modern technologies to be far more efficient, in terms of teacher and student time and of money spent, for both learning and assessment. I propose calling this approach T-learning and T-assessment, on account of the visualization of the model shown below.

The vertical of the T denotes the topic that is studied in depth to build mathematical thinking capacity. In last month’s post, I discussed a suggestion by mathematics learning expert Liping Ma that school arithmetic is the best subject for doing that, so I have put that down for the vertical. Read last month’s post to see my summary of her argument, and follow the link I gave there if you want to know more. But school arithmetic is not the only choice. Euclidean geometry could also work. In both cases (or with any other choice), it would be important to teach the T-vertical the right way, so as to bring out the general mathematical thinking patterns.

The horizontal bar of the T represents the collection of topics chosen to provide breadth. Each of the topics on the bar can be covered relatively quickly, once mathematical thinking has been mastered.

In my Introduction to Mathematical Thinking MOOC on Coursera, which has been running regularly since 2013, I used the structure of everyday language as the T-vertical, and some topics in elementary number theory for the bar of the T. I only needed one branch of mathematics on the bar, since the goal was to teach mathematical thinking itself, and for that, one application domain was enough. (I chose number theory since you need nothing more than arithmathetic to get into the early parts of the subject.) If the goal is to cover everything in the Common Core State Standards, you’d need a number of branches of mathematics (though only a handful).

So much for the diagram. Before I launch into my efficiencies sales pitch, let me make a few remarks about the itemized list above.

Item 1. I fear many readers will not really understand what I mean by this. Mathematicians surely will. But a sad consequence of the way mathematics has typically been taught, as a smorgasbord of definitions and facts to learn, tricks to remember, and procedures to practice, is that relatively few people survive their math education long enough to realize that the entire discipline revolves around a very small collection of thought patterns. I discussed this tragedy at some length in my last month’s post, citing some research results of my Stanford colleague Prof. Jo Boaler that show just how great a tragedy it can be.

Dr. Boaler is a former school teacher, education system administrator, and more recently a world-renowned mathematics education scholar of many years standing. She is one of a number of mathematics pedagogy experts I work and/or consult with. I mention that because my primary expertise is in mathematics, a discipline I have worked in for half a century. I do have a fair amount of knowledge of mathematics pedagogy, but purely as a result of studying the subject fairly extensively. I have not and do not engage in original research into mathematics pedagogy. I cannot, therefore, claim to be an expert in that domain. The suggestions I make here are, as always, in my capacity as a mathematician.

Item 2. As Boaler described, many students come away from the math class in despair, complaining that there are way too many things to remember. Yet, if you think back to your school days, there were probably one or two kids in the class for whom it seemed effortless. What was their secret? Were they simply math geniuses? Were they, as some parents like to say, “gifted”? I’ll answer those three questions in reverse order.

Very rarely, in various areas of human endeavors, exceptional individuals come along. That’s simply a feature of distributions with an element of randomization. But for the most part, children classified as “gifted” are simply the offspring of relatively well-off, educated parents who provide their children with excellent early role models and an educationally stimulating start in life. That is their “gift.” And indeed it is a gift; they were given it – by their parents. I point this out because, as research by Boaler and others has shown, labeling a child as “gifted” frequently turns out to have crippling consequences for that child. Having been told they were “gifted,” the child assumes that whatever level of effort leads to initial success in the math class will continue to do so – they rely on their “gift.” But in mathematics, as in many walks of life, the further you get into something, the harder it gets. While a child who accepts struggle and failure in math as part of the learning process will often keep trying, the “gifted” child may well (and often does) give up when they are no longer acing all the tests, perhaps claiming that they simply no longer found it interesting, in order to mask (from themselves as much as anyone else) the devastating consequence to their self-esteem that results from their having built up no tolerance of failure. (We learn when we get something wrong and figure out why. Getting something right simply gives us reassurance and maybe makes us feel good for a while.)

The “math geniuses” question. Unlike some of my colleagues, I don’t mind that term being used, as long as it is understood to refer to an individual who (a) was born with a brain particularly well suited to mathematical thinking, (b) found mathematics totally fascinating (for whatever reason, perhaps a desire to escape a miserable childhood environment by retreating into the mental world of mathematics), and (c) devoted thousands of childhood hours working on mathematics. For those are the three ingredients it takes to produce an individual who could merit being called a “genius.” There are very few such math geniuses in the world. In contrast, I suspect (on numerical grounds) there are a great many children born with a brain suited to mathematical thinking, who never pursue, or show prowess in, mathematics. The term “born genius,” which you sometimes come across, strikes me as idiotic.

The “What was their secret?” question. Those kids in your class who seemed to find math easy were the ones who, for whatever reason, managed to recognize that, for all that math was presented to them as a jumble of tricks and techniques, there was method to the seeming madness. Not just method, but a fairly simple method. Mathematics, they realized, was a theme-and-variations affair. There was no need to memorize anything beyond the basic multiplicative number bonds (the “times tables” as they were called when I was a lad growing up in the UK) – which even in today’s computation-rich environment are extremely useful to have at your mental fingertips.

ASIDE: Fortunately, the multiplicative number bonds can be committed to memory by using numbers often enough in meaningful contexts. But to my mind, since they can be mastered by rote (or even better, by playing one of several cheap, first-person-shooter, multiplication video games), you might as well get them out of the way as quickly as possible by a repetitive memorization process. Moreover, there is mathematical thinking mileage to be gained by this approach, when kids discover that there are various patterns that can be used to avoid actively memorizing most of the multiplication facts (x5, x10, and commutativity are three such time-saving patterns), leaving only a handful that have to be actually learned (6 x 7 and 7 x 8 are two such – though to this day I don’t have instant recall of 6 x 7, but rely on commutativity and instant recall of 7 x 6 – don’t ask!).

But I digress. The point is, that kid on the front row who annoyingly seemed to remember everything almost certainly remembered almost nothing; they worked out most of their answers on the fly. As I did with my answers for 6 x 7 and 8 x 9. (Okay, I guess I was one of those annoying kids.)

The point is, the crucial importance of approaching math learning as a process of acquiring a particular way of thinking does not just apply in the elementary grades – where many kids do manage to get by with pure memorization. The same is true all the way up into the more advanced parts of the subject, where memorization becomes impossible. Yet there is no need to memorize much of anything. Ever. Just a few key concepts, facts, and procedural details in each new branch of mathematics. In fact, if you continue in mathematics, after a while you realize that all the different branches of mathematics share what is essentially the same structure.

And the really nice thing is, mastering mathematical thinking to an adequate degree is like learning to ride a bike or to swim. Once you have it, you never lose it.

So much for the first two items on my initial list. But in elaborating on those, I’ve essentially covered Item 3, Item 4, and Item 5 as well. So we are done with that.

In fact, we’ve got so much useful stuff on the table now, it’s pretty straightforward to make that efficiencies sales pitch I promised you.

Current systemic assessments rely to a very high degree on digital technology, where students take a test presented and answered on a computer, which automatically grades their answers. To fit that format, questions are restricted to multiple-choice questions, questions that require a entry of single number as an answer, or some minor variant of one of these question types. (Earlier assessments used multiple-choice tests printed on paper that the student filled in with a pencil, with the completed test-paper then optically scanned into a computer.) This is fine for assessing what a student has learned on the horizontal (breadth) bar of the T. But on its own, the results of such a test are essentially useless. They measure either facts memorized or shallow (and often brittle) procedural manipulations based on memorized facts. They say nothing about an individual’s ability to think mathematically.

That is why the better systemic assessment systems on the market also present students with open-ended questions where the student has to solve a problem using paper and pencil, with the solutions for a whole class, school, or district being sent out for grading by trained human evaluators, who follow an evaluation rubric. Though this process does bring in an element of graders’ subjectivity – even with a well-thought-out and clearly expressed rubric, the graders are still faced with an often formidable interpretation task – it works remarkably well. But it is both time-consuming and expensive. It tends to be used only for major, summative assessments at the end of a unit or a school year. The time-delay alone makes it unsuitable for formative assessments intended to provide feedback to students about their progress and to alert teachers to the need for individual-student interventions or changes in the rest of the course.

With the T-model, only the core subject chosen to constitute the T-vertical needs to be assessed this way, of course. Even with existing assessment methods, making that restriction could lead to some cost reduction. But substantial savings, in personnel, time, and money, would be obtained if the subject in the T-vertical could be assessed automatically. With today’s technologies, it can.

To obtain good assessments of mathematical thinking, educators typically present students with what are known as “complex performance tasks” (or “rich performance tasks”), requiring multi-step reasoning. CPTs often (though not always) have more than one “correct” answer, with some answers being better than others. Even when there is a unique answer, there is frequently more than one solution (= sequence of reasoning steps) that gets to that answer. CPTs can range from very basic tasks, perhaps requiring only one or two individual steps (though with a period of reflective thought required in order to start) to the fiendishly difficult.

Some kinds of CPT (particularly in subjects such as arithmetic, geometry, and algebra) can be implemented as digital puzzles, where the student has to manipulate objects or symbols on a computer screen in order to find the solution. When deployed in this format, such CPTs can be used as systemic assessment tools. Not all mathematical subjects or topics lend themselves to this kind of presentation, so it is not a feasible approach for systemic assessment as currently conceived. But for T-assessment, it can work just fine. Simply specify the T-vertical to consist of mathematical topics that can be assessed using digitally-implemented CPTs. Because the assessment is conducted on the computer, the student’s entire solution to the CPT is captured and can be analyzed by an algorithm. In real-time. At no incremental cost. At whatever scale is required.

Of course, the key requirement here is to have a mathematical topic, or set of topics, and a set of CPTs in that area, that is collectively sufficient to demonstrate mathematical thinking ability. For that, remember, is what the T-vertical is all about.

Such digital assessment tools already exist. (Full disclosure: I am a member of one team developing and testing such tools.) So far, they have been subjected to limited testing on a small scale. The results have been encouraging. Conducting large-scale trials is clearly a necessary first step before they can be deployed in the manner I am suggesting. Moreover, to be useful, mathematics education has to be configured according to the T-model, where an in-depth study of one part of mathematics is used to develop the key capacity of mathematical thinking, coupled with much more shallow experiences in a number of other parts of math to achieve breadth.

That’s my suggestion. In putting it out, I might hear back that others have thought about, or advocated, something very much along the same lines. (In fact, Liping Ma essentially did just that in the article I discussed in my last post, albeit not in terms of the use of digital puzzles to provide automated assessments.)

I may also hear from psychometricians who will instantly recognize difficulties that would need to be overcome to put my proposal into practice. In fact, having talked with psychometricians, I am already aware of some issues that would need to be taken into account. Psychometrics is another of those disciplines of which I have some superficial knowledge but in which I have no expertise. But I have not yet encountered any reason why my suggestion cannot be made to work. (If I had, you would not be reading this article.)

To my mind, the really challenging obstacle is for the mathematics education establishment to accept, and then adopt, the T-model. Fifty years experience as a professional mathematician (the first fifteen or so in abstract pure mathematics, the remainder in various applied fields) has left me in no doubt that the T-model is not only perfectly viable, it is far superior to the “broad curriculum” approach we currently use, often referred to (derisively, but justifiably) as “a mile wide and an inch deep” education. But I am not in a position to mandate educational change. Nor, frankly, have I ever wanted to work my way into a position where I could have such influence. I like doing and teaching math too much! Instead, I am using what platform I have to put this suggestion out there in the hope that those who do have influence might take up the idea and run with it.

Of course, I can keep repeating my message. In fact, you can count on me doing that. :)