​Grades – they don’t mean what you think they mean

A sequel to “The Reason Engineering School Let You Down”

The last article I wrote for this newsletter elicited a lot of responses. It is the most read and commented upon article in the series thus far. That tells me that many of us have a deep interest in the education and training of engineers. It also revealed something else – many seemed to assume that if you do well on the exam, then you understand the respective theory.
I am going to explore that assumption more in this article: Do good exam marks really mean good understanding?
It’s all on the surface.
When I was an academic and involved in education research, I was introduced to a phenomenon called surface learning. It is where students study to pass the exam as opposed to studying for understanding. We all probably have some experience doing this, where we drilled questions (maybe even used Schaum’s) or we remembered things. Or we know a fellow student who would get good marks, but never seemed to actually understand anything. That’s all surface learning.
You can get away with this when the exam is set such to allow for this.
And many exams are like that.
They don’t assess understanding, just your ability to drill problems to pick up on the procedure and repeat it at speed.
As an example of how pervasive this is, take the time to watch the video below. It features Eric Mazur, a physics lecturer, talking about his students and how shocked he was when he assessed conceptual (read “actual”) understanding after the first year of physics.

Some key points from the video:

• He never asked himself how he would teach.
• He still got high satisfaction ratings.
• He thought the students did well.
• He thought he was a great teacher.
• Exams went well – based on marks.
• He used typical textbook problems.
• When he found that many students at other universities did not learn well, he was convinced his Harvard students were learning.
• He was wrong – it is impressive that he chose to be so scientific about this.
• There was a difference between how students think in daily life and how they think for exams.
• The students used what he called “recipes” (read “surface learning”).
• Once he came up with a better teaching method, one where students actually learned, he does not say anything about how his satisfaction ratings went.

You can infer from this, that his students were engaged in surface learning until he corrected both his teaching style and assessment method.
Surface learning isn’t just about the student it is encouraged by most exams in engineering courses around the world.
As I mentioned in the previous article: many exam questions will list only the variables needed to find the answer. In such a scenario, students only need to recognise the pattern (or the recipe).
This is why grades don’t always (and often don’t) reflect understanding. The exam format can encourage procedural fluency at the cost not conceptual understanding.
But what to do about it now?
If you would like to improve your conceptual understanding of first principles, and you should, then one of the best sites you can go to is Arbor Scientific. They offer numerous teaching resources that you can sign up for, but they also have a great conceptual questions page - https://www.arborsci.com/pages/next-time-questions. Go check them out and get the resources once you are done. I liked the double boiler question and the bikes and bee question.  See which ones get you thinking or reveal your lack of understanding so you can improve it.
Before I finish though, I’d like to ask: what conceptual understanding tools do you know of? I am always keen for more and others here can benefit from them too.