Student co-creation of knowledge with ResponseWare

I recently took over the ResponseWare service at the University of Warwick, following the retirement of my colleague Chris Coe. Since we began introducing the hardware-based Turning Point personal response system (PRS) about eight years ago, I had been curious about its fit with Warwick’s research-based learning and student as producer ethos. At first I encountered resistance to the idea of using PRS in core academic teaching. As is often the case with innovations, it edged its way into peripheral applications. But more recently I have seen some convincing examples that prove its compatibility with a research intensive institution’s notion of student engagement.

Turning Point’s ResponseWare replaces the clickers (expensive, labour intensive) with an app (either an install for mobile devices or a browser based web application). Interactive presentations are still authored using the same PowerPoint plugin. Response data also exists in the same format, and is managed through the Turning Point 5 desktop application. So it is familiar to existing users of the clicker system, and can run alongside clickers in a mixed mode. However, the purely virtual alternative, allowing 2500 concurrent student connections, opens it up to many more teachers across the whole University. In technical terms, this should enable the innovation to spread more easily, to more people and into more contexts.

But what if it turns out that ResponseWare’s core design patterns fail to fit with a research intensive University? I don’t think that there is an incompatibility. And I have found academics and students who have:

  • a strong sense of what they want to achieve in learning and teaching design – developing a deeper level of understanding (beyond the repetition of facts and procedures), and being part of the creation of knowledge (often through applying/testing existing theory in application to new circumstances, but in some cases refining theory or creating new theories);
  • an imaginative adaptation of the technological facility to help achieve those ambitious goals.

When we first encounter ResponseWare (and other PRS systems) we should be forgiven for consigning its use to “checking knowledge of facts” (perhaps as a revision exercise) or “testing procedural knowledge”. It is, after all, based upon various types of multiple choice question (MCQ). And good sense immediately tells us just how hard it is to write MCQs that test deeper understanding. Perhaps that should be left to dialogical interaction in traditional seminars instead? As one ResponseWare user in the business school pointed out, MCQs may actually have a negative effect – narrowing-down student thinking and dialogue.

But PRS need not be used in that way. And our business school have led the way in discovering new ways of using it.

Here are five much more sophisticated design patterns, inspired by real cases at Warwick, with the aim of stimulating deeper student thinking and the co-creation of knowledge:

1. Crowdsourcing research

There doesn’t need to be a correct answer! In fact when creating interactive questions in Turning Point (for clickers or ResponseWare) they are by default “polls”. So why not use PRS to poll responses to questions, for example about attitudes, preferences, behaviours etc? The anonymous nature of PRS voting (on by default) means that people can be more honest as well. But crowdsourcing can go even further than that. What if we crowdsource the questions as well? In one example, groups of academic, professional and lay participants worked together in the morning of a conference (led by undergraduate students) to propose elements of a taxonomy to describe a set of medical conditions. The proposals were then incorporated into a Turning Point session for all of the participants (back together in a large room) to vote on.

2. Crowdsourcing questions written by students

Writing meaningful multiple choice questions is a big challenge in itself, especially when we want to test conceptual understanding or complex procedural knowledge. In order to pose a good question we have to gain a clearer understanding. It is a learning experience, with testable outputs. It should also make us think about points of confusion and the need for clear explanation. So, why not get the students to write the questions and test them out on each other?

3. Demographic mappings

In most cases we use PRS in an anonymous mode (we can match students to answers, but it is usually too complicated for most people to bother with). Turning Point (for ResponseWare and clickers) provides a mechanism through which we can do selective demographic analysis of anonymous answers. We start by posing the question upon which the demographic analysis will be based (e.g. male/female?, what continent are you from?, how old are you?, so you prefer Kant or Nietzsche?). Responses to subsequent questions are then mapped to the demographic question, so we might for example analyse responses to the question “do you like fast cars?” by philosophical preference or some other demographic. Again there is some value in getting the students to write the questions.

4. The rhetorical question

This is really just an extension of the ancient technique. Be like Socrates, but with a whole lecture theatre full of students! Start with a question that has a seemingly obvious answer, or perhaps one that will divide opinion. Then unpick the assumptions, or play one group against the other in a debate. Finally, ask the same question again to see how opinions have shifted. This could be achieved in the age old fashion with a show of hands, but the anonymity of PRS may help with honesty (increasing engagement), and the visual presentation of voting responses in a graph makes it clearer and more exciting.

5. Thinking, fast and slow

The name of this pattern comes from the book by Daniel Kahneman1, who’s aim was to reveal the unconscious biases that often work against our best interests and good judgement. That’s important for everyone, but perhaps even more so for professionals working in fields where small mistakes can have critical consequences. I first came across this use of PRS when looking into teaching techniques in medical education. Doctors need to think fast, but not too fast. They need to be aware of the limitations of information and knowledge, and the impact of unconscious bias. The need to think fast and slow, often at the same time. To encourage a heightened self-reflexivity, without becoming too obtrusive, medical students are sometimes asked a series of questions and are immediately asked to rate the confidence that they have in their answers. A combination of wrong answer and over-confidence is a very bad thing. Another technique, suited to the task of getting students to understand the value of thinking more carefully and deeply, involves asking them to answer a series of questions very quickly, and then repeating the sequence at a more leisurely pace, perhaps with an opportunity to discuss with a partner or a small group. By appreciating mistakes made in haste they get a sense of the value of more sophisticated theories and knowledge.


There we have five sophisticated and powerful learning design patterns using PRS. I’m sure there are more, and will add to this list as they are uncovered.


1Kahneman, D. (2011) Thinking, Fast and Slow. Penguin, Kindle edition.