Our events generate huge amounts of interesting data, which we know contains all sorts of valuable insights. When there’s so much of it, it can take a while to work out how to make best use of it. One example of this is thematic analysis of the questions students post in ASK. Over the years of running the event we’ve developed a pretty good sense of the types of things students ask about and will always showcase great examples of questions. Converting that into a systematised, reportable analysis is more difficult, but can be done. So, we have developed a system for coding questions based on a set of themes that students often ask about, and after some trials in March, we’ve applied it to all zones in the June event (read the latest reports here).

At their broadest level, most student questions fall into one of three areas: questions about the science; questions about the scientist; questions about the I’m a… event. But we can look at this in more detail, considering elements of science capital (look out for a separate blog on this coming soon), and “how science works” (although this term has now been re-framed as “working scientifically” in the latest UK curriculum, we think it is still descriptive and useful for us). We’ve identified five top-level themes that we will report on. Each of which has one or more sub-themes which we are using to code the questions.

 

Science topics

covers facts, theories and knowledge about science. These can be related to the scientists work, or more general questions about science topics. Examples include…

When was the first epidemic discovered?

What animal has the worst venom and what does it do?

Can you give a robot human emotions?

How science works

looks at the process, motivations and ethics behind science. This could be finding out more about how decisions are made, why people chose science, or looking at how science fits into wider society. Examples include…

Is your work accurate and can be depended on?

Why do you think that a lot of people believe that psychology isn’t a science?

How did you come up with the idea of making MRI scans faster?

Career and education

includes the experiences of the scientists and aspirations of the students. Questions in this category build a greater understanding of real-life careers and education, including previous experience, current situation and future plans. Examples include…

What criteria did you use to choose your university?

Do you think you’ll ever leave the science job route for something different?

What is a PhD and how do you get one?

Personal

encompasses questions about scientists lives, knowledge and opinions outside of their work. Examples include….

Do you still have a social life?

From studying human relationships have you altered how you interact with people in day to day life?

What’s your opinion on the election?

Event/other

includes questions about the event and prize money, as well as a handful of questions that really don’t fit anywhere else. Examples include…

Do you think you can win I’m a Scientist?

Method

Using moderators and the event team each zone is coded, and then checked, so each set of questions have at least two run throughs. We use a coding guide as reference for consistency, and we are always building on our bank of reference questions for each category.

Students can ask anything they want, so it’s not always a clear-cut decision. Some questions overlap the different categories, but in general fit best within one of the reportable themes. There are some trickier questions that a few people will make a decision on together, and when we see a lot of similar questions we can agree how to code these. Although this won’t eliminate the ambiguity, it will help us to be consistent and transparent in how we code.

 

Results

Here’s how the questions distribute across all I’m a Scientist zones for the June 2017 events.

 

You can see the outcomes for individual zones in the latest I’m a Scientist reports and I’m an Engineer reports. We’re going to keep refining this coding to make it more consistent and the best reflection of our events it can be.

If you have any comments, thoughts, or would like to know more, please get in touch.