Category Archives: Evaluation Reports

Sports Science Zone evaluation report

In March 2012 we ran a Sports Science Zone, funded by The Physiological Society. It was a big success.

  1. It was the most popular zone requested by teachers
  2. More than 2 sports scientists applied for each place
  3. The students asked lots of questions on sports and the human body. Keywords that poopped up a lot include Body, Blood, Brain, Cardiovascular, Cells, Exercise, Heart, Human Body, Lung, Oxygen, Performance and Running. Discussions ranged from how extra organs would effect the human body to the best way to improve personal health.
  4. Heart researcher Fiona Hatch won the zone and spent her prize money giving a sheep’s heart dissection lesson to a local school in Hull.

Read the full evaluation here.

As one sports scientist said “this has been a blast”.

Posted on February 13, 2012 in Evaluation Reports | Comments Off on Sports Science Zone evaluation report

Evaluation

Evaluation in 2011

In June 2011 we came to the end of a grant from the Wellcome Trust to run I’m a Scientist in 2010 and 2011. We’ve evaluated throughout the project and the final evaluation report is available to download below, as well as a shorter executive summary and summary sheet.

2010-2011 Evaluation Report – at 75 pages long it contains a lot information including the methodology, aims, key figures and data, findings and recommendations.

 

Executive Summary – this outlines the main findings from the 2010-2011 Evaluation Report and looks at whether the objectives set at the start of the project have been met.

 

Summary sheet of the event in 2010-2011 – this highlights some of the key numbers and quotes from evaluations, and reasons why the event works so well.

 

How do we evaluate?

Evaluation is absolutely key to understanding what students and scientists have got out of the event, and it helps us continuously improve the event. There are two types of evaluation that we’ve done on I’m a Scientist, right from the first pilot event back in 2008.

Formative: We have done formative evaluation throughout the project – asking students, scientists, teachers and stakeholders for their views and acting upon them.

Summative: This is the more formal type of evaluation where we ask participants for their views before the events starts, observe the event in action in the classroom and survey views after the event has finished.

One of the benefits of running an online event is that we collect an enormous amount of data about how students and scientists engage with the event. We know how many students asked questions, how many questions they asked, how much they participated in live chats. We can tell the scientists how many students they’ve reached, how many questions they’ve answered and whether they’ve changed the attitudes of those students.

We’re always looking for ways to improve our evaluation, but what’s the point of evaluation is no-one reads and learns from what you’ve done. We’re genuinely interested in hearing what measures you’d like to see. What is it you want to see in an evaluation? Send us an email with your suggestions.

Previous evaluation reports

You can find the evaluation report summary event here

You can find the pilot evaluation report here

You can find the complete evaluation report on the 2008 pilot event here (this is the full, 127 page long baby)

You can find a summary of the 2008 pilot event evaluation report here

Posted on August 8, 2011 in Evaluation, Evaluation Reports | Comments Off on Evaluation

So How *Was* IAS2010?

In our pilot event, we did nearly all of the evaluation in-house, to save money. We tried to be as objective as possible, but of course it’s much better practice to have an independent evaluator. This year, we’ve been able to do that, and we appointed the lovely Kate Pontin to evaluate the project for us.

A student looks at a log in card for I'm a Scientist

We are publishing the evaluation report on 2010 today (and also presenting it at the Science Council). You can also read the team’s early thoughts on this year in our write up of our Science Online London session, or see our evaluation report on the pilot.

Download full I’m a Scientist Evaluation Report 2010 (pdf)

Summary of I’m a Scientist 2010 evaluation

This report summarises the findings of the first year of formal evaluation of I’m a Scientist, Get me out of Here! and thus acts as an interim report.

I’m a Scientist creates opportunities for scientists to answer questions from students via the internet, opening dialogue, while developing skills and understanding of the science process. It also gives students an idea of what it is like to be a scientist and the types of careers available.

Evaluation data was collected from scientists, teachers and pupils via surveys, and interviews. Observations of a number of sessions in school were also undertaken.

Data suggested that the IAS programme is very successful, providing a unique approach to the communication of science to students. It shows that:

  • Students gained a lot from the experience. Their feedback shows they thoroughly enjoyed the event, especially the live chat sessions. They developed key skills including focused questioning and gained confidence in scientific discussion. They found it interesting to discover more about what it is like to be a scientist and were surprised that scientists were not as stereotypical as they originally thought.
  • Teachers found that IAS supported their need to develop different approaches in the classroom and also to encourage pupils to think about science and who it is actually undertaken by. They found the resources very useful, in particular the debate kit and were also pleased with the support they received from the team during the running of the event.
  • Scientists also gained or developed skills in communicating with the public, explaining their work (often complex science) clearly to pupils aged 13 onwards. This in some also helped inspire and revive their enthusiasm for their own studies.

Minor improvements suggested are:

  • Helping teachers to prepare pupils especially in their development of clearly focused questions
  • Promote and signpost other debate lesson packs like the IVF debate, and to help conclude the event, with perhaps an assessment element.
  • Further initial guidance for scientists giving guidance on the time it might take, but also for those less familiar with students of this age some general information on KS3/4 and the diverse knowledge base.

Evaluation will in 2011 focus on finding out more about how to attract scientists and about the impact their involvement has on their organisations. Longitudinal studies will look at the impact of students in the longer term (for example in take up of science or exam results). It is also hoped to develop baseline and plenary activities to find out more about change in knowledge and attitude to science.

I’m a Scientist enabled students using an innovative approach to inspire and enthuse. It worked with pupils from high achieving classes but also those across the ability range and from a wide range of backgrounds including groups from BME families.

Key figures for 2010

  • Two events – March and June
  • 7,459 questions asked
  • 125 scientists were involved, in 25 zones
  • 94,909 visits to the site
  • 6397 students took part
  • 648,563 page views

About the evaluator

Kate Pontin is a former geologist who has been working in museums and evaluation for over 20 years. She specialises in the experiences of young people in informal learning environments.

Posted on March 4, 2011 modangela in Evaluation, Evaluation Reports, News | Comments Off on So How *Was* IAS2010?

So how did I’m a Scientist go in 2010?

It’s important, in anything, to check that you’re doing a good job. When we’re running I’m a Scientist, we think we’re doing a good job and that the event is fun and exciting, and that students (and scientists!) learn things. But we aren’t the most impartial people to check!

We’re spending lots of money on this project (given to us by a charity called the Wellcome Trust), and we’re also taking up lots of lesson time at hundreds of schools around the country. So it’s important we check that the event is doing what it’s supposed to.

A student looks at an I'm a Scientist login card

So a researcher called Kate Pontin has been checking for us. She visited some schools when the event was happening (maybe she came to your school?). She watched lessons and talked to students. She also talked to some of the scientists and teachers. And she looked at the surveys some of you filled in at the end.

What she found out was:-
Facts and figures for 2010

  • Two events – March and June
  • 7,459 questions asked
  • 125 scientists were involved, in 25 zones
  • 94,909 visits to the site
  • 6397 students took part
  • 648,563 page views

Students

  • Enjoyed taking part
  • Got better at asking good questions
  • Got more confident talking about science
  • Found it interesting to talk to real scientists and learn about real science
  • Learnt that scientists weren’t like the stereotypes

Teachers

  • Found it helped them try out new ways of teaching
  • Helped them teach How Science Works
  • Helped them teach students about science and scientists
  • Liked the classroom resources
  • Thought the I’m a Scientist team were helpful (*beams*)

Scientists

  • Got better at explaining their work
  • Some of them found it made them more into their work and remember why they liked it.

She suggested we should make some improvements for next year though

  • Add more classroom resources for teachers
  • Make it easier for teachers to use the ones we’ve got already – like make them more obvious
  • Explain more to scientists what level GCSE students are at, so they’ve got a better idea what to expect in some of the questions.

If you want to take part next year (and you aren’t already signed up) then use these links

Teachers – to find out more about taking part, and sign up

Scientists – to find out more about taking part, and sign up

Posted on November 11, 2010 in Evaluation Reports | Comments Off on So how did I’m a Scientist go in 2010?

The long awaited IAS evaluation report is published!

Here is is, all 126 glorious pages of it. The I’m a Scientist Evaluation Report [pdf 1mb].

If you’re interested in how the event went and what people got out of it then look at Section 1 (findings) and Section 3 (case studies). If you’re a fellow science communication professional, and want to know more about how we did things and learn from our experience, then look at Section 2 (development) and Appendix 3 (formative evaluation findings).

The key messages were:-

Students said: “i learnt loads and feel much more confident to put my hand up and ask questions and know that ok sometimes i will get it wrong”

Teachers said: “The pupils were looking forward to their science lessons, asking about them in the corridor etc.”

Scientists said: “It engaged the kids in a way I’ve never seen before.”

Every scientist and teacher surveyed said they would recommend it to a colleague.

Key Outcomes

  • Students realised scientists are real, interesting, fun people
  • Students were inspired and enthused
  • Developed debate and discussion and How Science Works (HSW) skills

Key reasons it worked

  • Giving some power to young people gives them a reason to engage and shows that they are trusted
  • The fact that it’s real – real scientists, real science, real prize money – makes it far more vivid
  • The intimacy of the medium makes it easier to break down barriers and make connections

The key lessons for others were

  • Teachers want to use more debate and discussion for the new GCSE but feel they and/or the students don’t know how to do it. So resources which help are welcome.
  • Many existing resources to support the new GCSE feel ‘rushed out’ and don’t always ‘get’ what How Science Works is.
  • A teacher panel is a really useful way to include teachers in development of your project.
  • A project blog is a great way to be transparent and keep everyone informed.

I hope this is useful to people.

Posted on December 12, 2008 in Evaluation, Evaluation Reports, IAS Event | Comments Off on The long awaited IAS evaluation report is published!