Many math instructors use clickers in their larger lecture classes, and can cite numerous studies
to show that they lead to more student attention and better learning. A recent research paper
on clicker use devotes a page-long introductory section to a review of some of that literature.
(Shapiro et al, Computers & Education 111 (2017), 44–59) But the paper—by clicker
aficionadas, I should stress—is not all good news. In fact, its main new finding is that when
clickers are used in what may be the most common way, they actually have a negative effect on
student learning. This finding was sufficiently startling that EdSurge put out a feature article on
the paper on May 25, which is how I learned of the result.
The most common (I believe) use of clickers is to provide students with frequent quiz questions
to check that they are retaining important facts. (The early MOOCs, including my own, used
simple, machine-graded quizzes embedded in the video lectures to achieve the same result.)
And a lot of that research I just alluded to showed that the clickers achieve that goal.
So too does the latest study. All of which is fine and dandy if the main goal of the course is
retention of facts. Where things get messy is when it comes to conceptual understanding of the
material—a goal that almost all mathematicians agree is crucial.
In the new study, the researchers looked at two versions of a course (physics, not
mathematics), one fact-focused, the other more conceptual and problem solving. In each
course, they gave one group fact-based clicker questions and a second group clicker questions
that concentrated on conceptual understanding in addition to retention of basic facts.
As the researchers expected, both kinds of questions resulted in improved performance in fact-
based questions on a test administered at the end.
Neither kind of question led to improved performance in a problem-based test questions that
required conceptual understanding.
The researchers expressed surprise that the students who were given the conceptual clicker
questions did not show improvement in conceptual questions performance. But that was not
the big surprise. That was, wait for it: students who were given only fact-based clicker questions
actually performed worse on conceptual, problem solving questions.
To those of us who are by nature heavy on the conceptual understanding, not showing
improvement as a result of enforced fact-retention comes as no big surprise. But a negative
effect! That’s news.
By way of explanation, the researchers suggest that the fact-based clicker questions focus the
student’s attention on retention of what are, of course, surface features, and do so to the
detriment of acquiring the deeper understanding required to solve problems.
If this conclusion is correct—and is certainly seems eminently reasonable—the message is clear.
Use clickers, but do so with questions that focus on conceptual understanding, not retention of
basic facts.
The authors also recommend class discussions of the concepts being tested by the clicker
questions, again something that comes natural to we concepts matter folks.
I would expect the new finding to have implications for game-based math learning, which
regular readers will know is something I have been working on for some years now. The games I
have been developing are entirely problem-solving challenges that require deep understanding,
and university studies have shown they achieve the goal of better problem-solving skills. (See
the December 4, 2015 Devlin’s Angle post.) The majority of math learning games, in contrast,
focus on retention of basic facts. Based on the new clickers study, I would hypothesize that,
even if a game were built on math concepts (many are not), unless the gameplay involves
active, problem-solving engagement with those concepts, the result could be, not just no
conceptual learning, but a drop in performance on a problem solving test.
Both clickers and video games set up a feedback cycle that can quickly become addictive. With
both technologies, regular positive feedback leads to improvement in what the clicker-
questions or game-challenges ask for. Potentially more pernicious, however, that positive
feedback will result in the students thinking they are doing just fine overall—and hence have no
need to wrestle more deeply with the material. And that sets them up for failure once they
have to go beneath the surface fact they have retained. Thinking you are winning all the time
seduces you to ease off, and as a result is the path to eventual failure. If you want success, the
best diet is a series of challenges— that is to say, challenges in coming to grips with the essence
of the material to be learned—where you experience some successes, some failures from which
you can recover, and the occasional crash-and- burn to prevent over-confidence.
That’s not just the secret to learning math. It’s the secret to success in almost any walk of life.
