Back in November 2012 we ran I’m a Scientist in Ireland for the first time. In short, it was a real success.
The enthusiasm shown by teachers, students and scientists was magnificent and the students really got into the spirit of the event.
The full evaluation report is up on the Irish site (imascientist.ie/2013/05/15/evaluating-im-a-scientist-in-ireland/) alongside a short summary of our findings.
In summary, we would love to run more events in Ireland.
Summary:Gallomanor was approached by the Wellcome Trust to run a live version of I’m a Scientist as part of the Wellcome Trust / Barbican Wonder Season in conjunction with the BNA Festival of the Brain in March and April 2013.
The deliverables were:
Gallomanor delivered the events but a lack of questions from the public outside of the events meant we did not need the IAS Zone.
The events were a great success. The Weekender heats had capacity audiences; the final was sold out. We succeeded in recruiting excellent scientists, but it was harder than expected. We had to do more cajoling and encouraging than originally anticipated. This surprised us as we expected that more scientists would be keen to join in a Wellcome Trust event.
Of those there, 9.5% were attending the BNA Conference, and 65% were not scientists;
the audience enjoyed the final;
learnt a lot;
and now have more questions about brains.
These charts should be treated with some caution. 35% of respondents also claimed to have no brain.
The budget for the project did not include a formal evaluation, but anecdotally the scientists enjoyed the experience:
Let’s do it again.
in Evaluation, Evaluation Reports, I'm a Scientist - Live |
Comments Off on I’m a Neuroscientist Live – Project Report
In November 2012 we ran 3 zones in I’m a Scientist. We used the event to make a few changes to how we ran it. We opened the event only to teachers who had taken part before. We used a new online registration system for teachers and students. And we ran 3 themed zones which tied into what students were being taught at the time – Cancer, Cells & Genes.
This short report looks at how the November event went. Offering the event just to teachers who had taken part before worked well. Using the new online registration system had a few hiccups, but despite the teething problems online registration is the way forward. Tying the themed zones into what students are learning at the time was a success. As one teacher said “I used it as an extension activity within the course they were already doing which was great. A really good idea to do this”.
modangela in Evaluation, Evaluation Reports, News |
Comments Off on November 2012 – Evaluation Report
In November 2012 we ran 3 zones in I’m a Scientist. We used the event to make a few changes to how we ran it. We opened the event only to teachers who had taken part before. We used a new online registration system for teachers and students. And we ran 3 themed zones which tied into what students were being taught at the time – Cancer, Cells & Genes.
This short report looks at how the November event went. Offering the event just to teachers who had taken part before worked well. Using the new online registration system had a few hiccups, but despite the teething problems online registration is the way forward. Tying the themed zones into what students are learning at the time was a success. As one teacher said “I used it as an extension activity within the course they were already doing which was great. A really good idea to do this”.
in Evaluation, Evaluation Reports |
Comments Off on November 2012 event evaluation report
GM Food Zone
For some time we’ve been wanting to expand the I’m a Scientist format and allow the general public to participate. The hysteria generated over the Rothamsted GM Wheat trials in May 2012 presented us with the opportunity to run a GM Food Zone.
We’ve written up our evaluation of the project. There are 4 sections in addition to this page:
You might find this incomplete. Please ask more questions in the comments.
For those who are too impatient to read the entire evaluation here is
Driven by some hysterical writing in the national press we decided to run a special I’m a Scientist GM Food Zone and open it to the general public. We wanted to create a neutral space where a constructive and civil discussion could take place.
We knew that we were placing ourselves in the line of fire. We knew we had to be neutral and seen to be neutral and balanced. We recruited an advisory panel and asked for nominations for the experts who would answer the questions. We had 70 nominations and despite an enormous amount of work to ensure we had the right range and balance of experts we were still accused of being Pro-GM. It’s possible that assumptions were made because of the I’m a Scientist name or the 3:2 ratio of experts in favour of GM.
As the questions came in they were fairly balanced. Slightly more were anti-GM than Pro-GM. Almost half the questions were from teachers and school students. A different set of visitors then started leaving comments. Three determined anti-GM visitors left just over half the comments we received. There was very little cross-over between the group of people asking questions and those leaving comments.
The site has so far been visited 3,977 times by 2,370 unique visitors. They averaged 3.4 pages per visit, but most pleasing is the stat that on average visitors spent 3:19 min on each of the 5 most popular questions. Visitors were spending a reasonable amount of time reading through the answers and comments.
UPDATE 09/01/15
7,900 visits from 5,900 users.
Yes. We did create a space where discussion could take place. The discussion was polite and mostly respectful. Many people read the discussions and a small group contributed towards it. Whilst it would seem that comments tended to be left by people who had already made up their minds a significant proportion of the questions were from people genuinely seeking information.
I’m not sure. The debate over GM Food has been going on for a long time and many people’s view are firmly entrenched. That said we think many visitors will have come away better informed about the wider issues associated with GM Foods.
The format worked. The technology worked. It would be good to run a zone on a topic where the battlelines are not already drawn. To find a topic where experts are still working out the issues, where they don’t yet know how best to articulate the issues, where feedback from interested parties and the public has some real value.
ModShane in Evaluation Reports, News |
Comments Off on GM Food Zone Evaluation
In June 2011 we came to the end of a grant from the Wellcome Trust to run I’m a Scientist in 2010 and 2011. We’ve evaluated throughout the project and the final evaluation report is available to download below, as well as a shorter executive summary and summary sheet.
Summary sheet of the event in 2010-2011 – this highlights some of the key numbers and quotes from evaluations, and reasons why the event works so well.
Evaluation is absolutely key to understanding what students and scientists have got out of the event, and it helps us continuously improve the event. There are two types of evaluation that we’ve done on I’m a Scientist, right from the first pilot event back in 2008.
Formative: We have done formative evaluation throughout the project – asking students, scientists, teachers and stakeholders for their views and acting upon them.
Summative: This is the more formal type of evaluation where we ask participants for their views before the events starts, observe the event in action in the classroom and survey views after the event has finished.
One of the benefits of running an online event is that we collect an enormous amount of data about how students and scientists engage with the event. We know how many students asked questions, how many questions they asked, how much they participated in live chats. We can tell the scientists how many students they’ve reached, how many questions they’ve answered and whether they’ve changed the attitudes of those students.
We’re always looking for ways to improve our evaluation, but what’s the point of evaluation is no-one reads and learns from what you’ve done. We’re genuinely interested in hearing what measures you’d like to see. What is it you want to see in an evaluation? Send us an email with your suggestions.
You can find the evaluation report summary event here
You can find the pilot evaluation report here
You can find a summary of the 2008 pilot event evaluation report here
modangela in Evaluation, Evaluation Reports, News |
Comments Off on 2011 Evaluation
In March 2012 we ran a Sports Science Zone, funded by The Physiological Society. It was a big success.
Read the full evaluation here.
As one sports scientist said “this has been a blast”.
in Evaluation Reports |
Comments Off on Sports Science Zone evaluation report
In June 2011 we came to the end of a grant from the Wellcome Trust to run I’m a Scientist in 2010 and 2011. We’ve evaluated throughout the project and the final evaluation report is available to download below, as well as a shorter executive summary and summary sheet.
Summary sheet of the event in 2010-2011 – this highlights some of the key numbers and quotes from evaluations, and reasons why the event works so well.
Evaluation is absolutely key to understanding what students and scientists have got out of the event, and it helps us continuously improve the event. There are two types of evaluation that we’ve done on I’m a Scientist, right from the first pilot event back in 2008.
Formative: We have done formative evaluation throughout the project – asking students, scientists, teachers and stakeholders for their views and acting upon them.
Summative: This is the more formal type of evaluation where we ask participants for their views before the events starts, observe the event in action in the classroom and survey views after the event has finished.
One of the benefits of running an online event is that we collect an enormous amount of data about how students and scientists engage with the event. We know how many students asked questions, how many questions they asked, how much they participated in live chats. We can tell the scientists how many students they’ve reached, how many questions they’ve answered and whether they’ve changed the attitudes of those students.
We’re always looking for ways to improve our evaluation, but what’s the point of evaluation is no-one reads and learns from what you’ve done. We’re genuinely interested in hearing what measures you’d like to see. What is it you want to see in an evaluation? Send us an email with your suggestions.
You can find the evaluation report summary event here
You can find the pilot evaluation report here
You can find a summary of the 2008 pilot event evaluation report here
In our pilot event, we did nearly all of the evaluation in-house, to save money. We tried to be as objective as possible, but of course it’s much better practice to have an independent evaluator. This year, we’ve been able to do that, and we appointed the lovely Kate Pontin to evaluate the project for us.
We are publishing the evaluation report on 2010 today (and also presenting it at the Science Council). You can also read the team’s early thoughts on this year in our write up of our Science Online London session, or see our evaluation report on the pilot.
Download full I’m a Scientist Evaluation Report 2010 (pdf)
This report summarises the findings of the first year of formal evaluation of I’m a Scientist, Get me out of Here! and thus acts as an interim report.
I’m a Scientist creates opportunities for scientists to answer questions from students via the internet, opening dialogue, while developing skills and understanding of the science process. It also gives students an idea of what it is like to be a scientist and the types of careers available.
Evaluation data was collected from scientists, teachers and pupils via surveys, and interviews. Observations of a number of sessions in school were also undertaken.
Data suggested that the IAS programme is very successful, providing a unique approach to the communication of science to students. It shows that:
Minor improvements suggested are:
Evaluation will in 2011 focus on finding out more about how to attract scientists and about the impact their involvement has on their organisations. Longitudinal studies will look at the impact of students in the longer term (for example in take up of science or exam results). It is also hoped to develop baseline and plenary activities to find out more about change in knowledge and attitude to science.
I’m a Scientist enabled students using an innovative approach to inspire and enthuse. It worked with pupils from high achieving classes but also those across the ability range and from a wide range of backgrounds including groups from BME families.
Key figures for 2010
About the evaluator
Kate Pontin is a former geologist who has been working in museums and evaluation for over 20 years. She specialises in the experiences of young people in informal learning environments.
modangela in Evaluation, Evaluation Reports, News |
Comments Off on So How *Was* IAS2010?
It’s important, in anything, to check that you’re doing a good job. When we’re running I’m a Scientist, we think we’re doing a good job and that the event is fun and exciting, and that students (and scientists!) learn things. But we aren’t the most impartial people to check!
We’re spending lots of money on this project (given to us by a charity called the Wellcome Trust), and we’re also taking up lots of lesson time at hundreds of schools around the country. So it’s important we check that the event is doing what it’s supposed to.
So a researcher called Kate Pontin has been checking for us. She visited some schools when the event was happening (maybe she came to your school?). She watched lessons and talked to students. She also talked to some of the scientists and teachers. And she looked at the surveys some of you filled in at the end.
What she found out was:-
Facts and figures for 2010
Students
Teachers
Scientists
She suggested we should make some improvements for next year though
If you want to take part next year (and you aren’t already signed up) then use these links
Teachers – to find out more about taking part, and sign up
Scientists – to find out more about taking part, and sign up
in Evaluation Reports |
Comments Off on So how did I’m a Scientist go in 2010?
Here is is, all 126 glorious pages of it. The I’m a Scientist Evaluation Report [pdf 1mb].
If you’re interested in how the event went and what people got out of it then look at Section 1 (findings) and Section 3 (case studies). If you’re a fellow science communication professional, and want to know more about how we did things and learn from our experience, then look at Section 2 (development) and Appendix 3 (formative evaluation findings).
The key messages were:-
Students said: “i learnt loads and feel much more confident to put my hand up and ask questions and know that ok sometimes i will get it wrong”
Teachers said: “The pupils were looking forward to their science lessons, asking about them in the corridor etc.”
Scientists said: “It engaged the kids in a way I’ve never seen before.”
Every scientist and teacher surveyed said they would recommend it to a colleague.
Key Outcomes
Key reasons it worked
The key lessons for others were
I hope this is useful to people.
in Evaluation, Evaluation Reports, IAS Event |
Comments Off on The long awaited IAS evaluation report is published!
I’m a Scientist, Get me out of here is an online activity where school students get to meet and interact with scientists.
It’s a competition between scientists, where the students are the judges.
Students challenge the scientists over intense, fast-paced, online live Chats. Then they Ask the scientists all the questions they want to, and Vote for their favourite scientist to win a prize of £500 to communicate their work with the public.