Category Archives: Evaluation Reports

Royal Society of Chemistry funded zones 2016

In 2014, the Royal Society of Chemistry (RSC) agreed to support I’m a Scientist over 3 years. Each year, they fund 9 zones across the UK and Ireland. Part of the arrangement is that we include five RSC members in general zones across the year to show school students the full breadth of science.

Below is a summary of our main findings after evaluating the 9 RSC funded zones of I’m a Scientist, Get me out of here 2016.

  1. Scientists improved their science communication skills and enjoyed an invigorating experience:
    “Taking part in this competition has reminded me why I love science, and I shall return to my research with renewed enthusiasm!” – Steve Street, chemist, Drug Discovery Zone
  1. The zones saw a great number of conversations around chemistry themes with keywords like “climate change”, “toxic”, “enzymes”, “battery”, “drug”, “energy”, “bacteria” being prominent in questions and live chats.
  1. 30 RSC members took part in I’m a Scientist thanks to this funding agreement. 25 took part in the 8 RSC funded themed zones and another 5 were spread across 2 general science zones.
  1. 8 RSC members were voted winners in the 9 RSC funded zones in 2016 and will continue to promote their research in chemistry with their £500 prize money.
  1. Widening participation in schools: 125 schools got a place to take part in I’m a Scientist RSC funded zones in 2016 the UK and Ireland. Of these, 31 (25%) were rural or Widening Participation schools, an improvement from last year, where 17% fulfilled this condition.

What students and teachers said about taking part:

Participating in this program, was a useful way to enhance my knowledge in my areas of interest…The scientists have all helped me to answer questions that I have been asking other people for a very long time. However, the major difference was that on this platform I received answers that clearly and fully answered all of my questions. Overall, this has been exceptionally useful to me and I look forward to more activities like this…” – Student, Antibiotics Zone

I’ve learnt how the children enjoy engaging with scientists and finding out about the world around them. An enjoyable, memorable learning experience”– Teacher, Climate Change Zone

Despite the success of the zones in 2016, due to changes in strategy decided by the Council of the RSC, the Society will not be funding the event in 2017 as planned. However, there are 94 Royal Society of Chemistry members registered on our scientist lists, waiting to take part in I’m a Scientist, and chemistry topics are the most popular among teachers. This change in strategy from the Royal Society of Chemistry means that the strong demand for online engagement with chemistry will be left unanswered this year.

Read more in the full report here

Posted on February 20, 2017 modantony in 2016, Evaluation, Evaluation Reports | Leave a comment

STFC Large Award Evaluation 2013 -2016

In 2013, we received an STFC Large Award to engage students with STFC research in 27 I’m a Scientist and I’m an Engineer zones over three years. During this time, we’ve measured every activity we’ve run, and we have stopped to look more closely at aspects like widening participation, or the impact of the events in researchers and students.

The STFC has been one of our most important funders, after the Wellcome Trust, in the last three years. This is a summary of how the collaboration has impacted I’m a Scientist and I’m an Engineer:

  • 135 scientists and engineers have engaged with 10,602 students from all around the UK between June 2013 and March 2016.
  • STFC zones reached a diverse set of students. 276 teachers, from 242 different schools geographically spread around the UK took part. Of these schools, 15% of them are Widening Participation schools.
  • Students and teachers leave positive comments on I’m a Scientist and I’m an Engineer. But as fans of evaluation and data, we have used pre and post event questionnaires for the students to measure how taking part affects their attitudes towards STEM. We have actually seen how the more active students are on the site, the more positive the change in attitude observed.

I have learnt how fascinating science can be and the wonders of space.(…) I now know science can be cheesy but you can discover things that you have never know before (…). I am very happy that I was able to take part in this chat” – student, Gravity Zone March 2016

  • I’m a Scientist and I’m an Engineer are good starting points for science communication. The scientists who did very little public engagement before taking part increased their number of outreach activities from an average of 1.6 activities per year, to 3.7 activities after the event. Scientists like Clara Nellist, who took part in 2014, are now putting a great emphasis on the communication side of their science careers:

After seeing how much you can do online, I was motivated to find more ways I could use the internet to communicate the research at CERN. Now I even manage all of the social media for my experiment, ATLAS.” – Clara Nellist, scientist

  • We have distributed £13,000 to 26 zone winners, 24 of which were STFC related scientists, who use the prize money to communicate STFC related research.
STFCscis

I’m a Scientist and I’m an Engineer Winners in STFC zones. Click here to download the full report.

And there have been challenges too:

  • Recruiting a diverse range of researchers, whose work is related to STFC, has not always been easy. However, towards the end of the award period we were able to work more closely with the STFC outreach team and use their input to choose zone themes and recruit scientists and engineers.
  • We are trying to reach new audiences. We created a Harwell Zone to allow Harwell Open Day visitors to text in questions to exhibitors. Visitors were able to text a question, get a response to manage their expectations and a notification when someone answered the question. Even though, there was not enough publicity to reach the public in significant numbers, we started the development of future zones open to non-school sectors of the public.

If you want to explore our findings in more detail, download the full report PDF here.

Have you been awarded with an STFC Award too? Have you faced similar challenges? How do you reach new audiences? Hop in, join the conversation and let us know what you think.

Posted on July 26, 2016 modangela in 2013-2016, Evaluation, Evaluation Reports | Leave a comment

Wellcome Trust Funded zones in November 2015 and March 2016

Screenshot_062816_041849_PM

Click here to download the report.

The Wellcome Trust provided funding to support the I’m a Scientist events in November and March of this academic year. We’ve written a report on what this funding allowed us to do, read it here.

Below are the key outcomes from the report:

  • 6 I’m a Scientist zones were run with Wellcome Trust funding in November 2015 and March 2016, including 2 themed zones, Fat Zone and Heart Zone, and 4 general science zones, two of which were for primary schools only.
  • 30 scientists were able to engage with school students, including 22 biomedical-related researchers. Of these, 45% were doing research supported by the Wellcome Trust
  • 2,826 school students logged into the zones, an average of 455 per zone.
  • 85 schools got a place in I’m a Scientist Wellcome Trust funded zones. 86% of these schools took part in the event.
  • £3,000 to be used for further public engagement was distributed among the winners of the zones.
  • Widening participation in schools – We developed criteria to identify schools that would benefit from taking part and 18% of the schools who took part in the zones met one of our criteria for schools underserved by STEM engagement.
  • Feedback from teachers and scientists was positive:

My school is absolutely buzzing about ‘I’m a Scientist’. Children are spending hours at home looking at questions. They don’t want to go out for lunch, so they can spend time on it!” – Teacher, Gold Zone

It’s been fun and manic all in one… Overall it’s been a great lesson in science comms! – Scientist, Rhenium Zone

Read more about the zones in the full report here

Posted on June 28, 2016 modantony in 2016, Evaluation, Evaluation Reports, News | Leave a comment

RSC funded zones 2015

RSC cover

Click to download the complete report (PDF)

The Royal Society of Chemistry (RSC) agreed to support I’m a Scientist over 3 years through to the end of 2017. Each year, they fund 9 zones across the UK and Ireland. Part of the arrangement is that we include RSC members in five of our general zones to show school students the full breadth of science.

Here is a summary of our main findings after evaluating the 9 RSC funded zones of I’m a Scientist, Get me out of here 2015.

  1. Scientists improved their science communication skills: They learned about students’ interests and how to communicate with them in a more efficient way.

… there is no better way to test how much you know about something than trying to explain it to someone with a more limited background on the topic.” – Jesús Calvo-Castro, Spectroscopy Zone

  1. The zones saw a great number of conversations around chemistry themes with keywords like “chemistry”, “light”, “materials”, “drugs”, “polymers” being prominent in questions and live chats.
  1. 37 RSC members took part in I’m a Scientist thanks to this funding agreement. 31 took part in the 8 RSC funded themed zones and another 6 were spread in 6 general zones.
  1. 6 RSC members were voted winners in the 9 RSC funded zones in 2015 and will continue to promote their research on chemistry with the £500 prize money.
  1. Widening participation in schools: 118 schools got a place to take part in I’m a Scientist RSC funded zones in 2015 the UK and Ireland. 47 were in Ireland and Scotland and 71 were in England and Wales and the remaining. Of these, 12 (17% of 71) were rural or Widening Participation schools.

And there were challenges too. In 2016 we want to:

  • Get more scientists working at private companies taking part in RSC funded zones. Only 12% of scientists who took part in RSC funded zones in 2015 came from the private sector. We would like to work with RSC to increase the proportion of scientists from private companies.
  • Get more Chemnet students involved in I’m a Scientist zones. We will keep working with ChemNet Team to find out better ways to promote I’m a Scientist to their community of students.
  • Improve the site mobile design. One of the scientists interviewed for this evaluation told us that she expected that it had been easier to take part from mobile devices. We have been improving the site design to adapt it to mobile, but we still need to make it easier for scientists to answer questions and take part in live chats form their mobiles.

You can download the whole report here (PDF).

Posted on January 26, 2016 modangela in 2015, Evaluation, Evaluation Reports, News | Leave a comment

Learning Zone – Evaluation Report

CLICK TO DOWNLOAD

CLICK TO DOWNLOAD

In February 2015 we partnered with the Wellcome Trust Education to create the I’m a Scientist Learning Zone.

In the Learning Zone teachers could send questions to and have conversations (thought text based live-chats) with neuroscientists and psychologists involved in research on a broad range of topics around the science of learning.

 

The Learning Zone was successful, both teachers and scientists found it a valuable experience.

  1. We created a conversation between teachers and experts on the science of learning: 40 scientists and almost 300 teachers registered to take part.
  2. The Learning Zone site was visited by 7,000 users during the event. Pages in the Zone were viewed over 33,000 times.
  3. Scientists and teachers valued the event. 89% of teachers and 94% of scientists would participate again.
  4. Teachers improved their knowledge about the brain and learning. 66% of the teachers agreed that “their understanding about the brain and/or learning had improved”.
  5. Improving knowledge may not lead to more confidence or enable more informed decision-making. Before taking part 73% of teachers strongly agreed that “Understanding more about the brain & learning will allow you to make more informed decisions about teaching” – yet afterwards only 36% agreed that they could take more informed decisions.
  6. The Learning Zone bridged the gap between scientists research and teachers. 77% of scientists agreed that they have a better understanding of teachers’ needs after taking part.

This being our first I’m a Scientist event specifically for teachers, we think we still can improve certain aspects of it:

  1. Increase the variety of live chat the schedule. There were 2 live chats per week: Monday 8pm and Wednesday 4pm. This schedule remained constant to create familiarity with the timings, however it meant that teachers and scientists who were available at other times were restricted from taking part in the live chats.
  2. Add themes to the Learning Zone. Consider theming certain weeks around specific topics, while leaving other weeks with a general “science learning” theme.

You can download the full report here.

Posted on July 23, 2015 modangela in Evaluation, Evaluation Reports | Leave a comment

Evaluation Report: 2012 – 2015

We were awarded a Wellcome Trust Society Award to run I’m a Scientist from March 2012 until June 2014. We later received a grant extension for November 2014 to June 2015. This report is all about our learnings in these last three years: from March 2012 until March 2015.

Our main learning points are:

WT report cover

Click here to download the full report

1. I’m a Scientist has gone from 30 zones per year in 2012, to 54 zones scheduled in the 2014/2015 school year.
2. Expanding zones to different audiences: primary school students and general public shows.
3. I’m a Scientist is a public engagement boost for scientists.
4. I’m a Scientist gets students enthused about science.
5. I’m a Scientist reaches a diverse set of students.
6. Teachers come back, but tricky to track.
7. Students ASK about cancer, animals, and life and CHAT about science, scientists and work.
8. Moving forwards, further adaptation to new technologies (such as tablets and smartphones) is important.

Click here to download the full report.

Posted on April 25, 2015 modangela in Evaluation, Evaluation Reports | Leave a comment

March 2012 – March 2015 Evaluation Report

We were awarded a Wellcome Trust Society Award to run I’m a Scientist from March 2012 until June 2014. We later received a grant extension for November 2014 to June 2015. This report is all about our learnings in these last three years: from March 2012 until March 2015.

Our main learning points are:

WT report cover

Click here to download the full report

1. I’m a Scientist has gone from 30 zones per year in 2012, to 54 zones scheduled in the 2014/2015 school year.
2. Expanding zones to different audiences: primary school students and general public shows.
3. I’m a Scientist is a public engagement boost for scientists.
4. I’m a Scientist gets students enthused about science.
5. I’m a Scientist reaches a diverse set of students.
6. Teachers come back, but tricky to track.
7. Students ASK about cancer, animals, and life and CHAT about science, scientists and work.
8. Moving forwards, further adaptation to new technologies (such as tablets and smartphones) is important.

Click here to download the full report.

Posted on April 4, 2015 modangela in 2012 - 2015, Evaluation, Evaluation Reports, News | Leave a comment

Evaluation Report – Body Zone report

body zone pic

Download the full report here.

The Body Zone was designed to contribute to the 2014 Physiological Society focus on Understanding Obesity, and help to reach secondary school students, at an age where we’ve all become more aware of our appearance and lifestyle.

The Body Zone in I’m a Scientist, Get me out of here! 10th – 21st November 2014 was successful:

  1. I’m a Scientist November 2014 has been our busiest event so far, and the Body Zone has been one of the busiest zones within the event.
  2. The Body Zone was particularly popular with teachers, it featured a steady influx of ASK questions that lead to more than 1,150 questions asked, of which more than 700 were approved and sent to the scientists.
  3. The students really got into the zone theme. Students asked lots of questions on the function, purpose and statistics of parts of the anatomy. There were also a lot of questions on diet, nutrition, exercise, obesity, childhood develpment, and diabetes.
  4. The scientists were challenged within and beyond their areas of research. All engaged well and were keen to answer questions and take part in the live chats.
  5. The drop out rate for schools was very low in the Body Zone – 14 of the 16 schools given places turned up with their students.

You can read the full report here.

Posted on December 17, 2014 modangela in 2014, Evaluation, Evaluation Reports | Leave a comment

Big Data Zones Report

click to download the full report

click to download the full report

In June 2014, teamed up with The Genome Analysis Centre (TGAC) to run two Big Data themed zones: The Genomics Zone and the Bioinformatics Zone, each featuring at least two TGAC scientists. And these were our main findings:

  • The students were interested in the scientists: Students read the scientist profiles and asked about the research of each individual scientist. They were also very keen to learn more about the scientists; daily lives and hobbies, as well about their opinions on topical issues like animal testing.
  • Students were interested in Genomics and Bioinformatics: Two difficult subjects which many of the students might not have been familiar with before the event. In fact, one teacher in the Genomics zone said she was pleasantly surprised with the level of the students’ questions and how the scientists explained what genomics is.
  • The Bioinformatics and Genomics Zones exposed students to the new trends in biology: Students learnt how biology is much more than identifying plants and animals, especially since the radical change that computers and Big Data have generated in the field. As one teacher said:

The genomics zone attracted us as it was a chance to expose kids to seeing biology as a difficult subject (which normally they don’t) and also speak to people in the research side of the subject. We felt it would link to a lot of topics in our current themes like microbes and reproduction.”

  • Scientists improved their science communication skills: For some scientists, I’m a Scientist, Get me out of here! was their first public engagement activity, and it served as a good pilot experience. They learnt about students’ interests and how to communicate with them in a more efficient way.

If you want to read the full report, you can download a PDF version of it here.

Posted on July 30, 2014 modangela in Evaluation Reports | Leave a comment

CHRISTMAS LECTURES Zone 2013/2014 – Evaluation Report

xmas eval picThe CHRISTMAS LECTURES are a series of science talks, which have been held at the Royal Institution (Ri) since 1825. In December 2013, the Royal Institution teamed up with us to extend the discussion around the CHRISTMAS LECTURES, and give everyone the opportunity to ask questions to real scientists online.

This was the first time we ran an I’m a Scientist zone alongside the CHRISTMAS LECTURES, so evaluating it was particularly important: we wanted to look at the strengths and weaknesses of the event and suggest improvements for future editions.

These were our main findings:

  • Almost 600 people participated in the CHRISTMAS LECTURES Zone, of whom 78% were students, 8% were teachers, and 14% were members of the general public.
  • All of the teachers that answered our feedback survey agreed that we should run the CHRISTMAS LECTURES Zone next year.
  • The two scientists who we interviewed after the event said that they now feel much more confident in science communication and they would recommend taking part in I’m a Scientist to their colleagues.
  • There is room for improvement, particularly regarding the participation of the general public, which we would like to see increase in future events.

If you are curious to know more about the event, you can access the full evaluation report here (PDF).

Posted on April 25, 2014 modangela in Evaluation Reports | Leave a comment

November 2013 Ireland Event Evaluation Report

IASIE report imageIn November 2013 we run I’m a Scientist in Ireland for the second time. We run 4 zones: two themed on Nanotechnology and Space and two general zones named Helium and Lithium with a mix of 5 scientists.

This report looks at how the event went. One of the main points of the report is our analysis of the impact that the event had on students’ perception of science. We found out that students’ interest in science and science related careers is clearly increased after taking part in I’m a Scientist. We also report on teacher and scientist feedback, which was generally very positive. As one teacher said: “This event offers the opportunity to do something different; an activity that brings out new strengths and abilities”. Sandra Byrne. Teacher

Download the report here.

Posted on December 19, 2013 modangela in Evaluation Reports | Comments Off on November 2013 Ireland Event Evaluation Report

How did the June 2013 event go? A summary for teachers

teacher summaryNearly 7,000 students received close to 10,000 answers to their questions in the June 2013 event.

But what did students ask? What were the most popular topics in live chats? What did your students teach the scientists?

Have a read of the summary for teachers to find out…

Posted on July 5, 2013 in Evaluation, Evaluation Reports, News, Teachers | Leave a comment

Evaluating I’m a Scientist – Ireland

IAS Ireland Logo May 2012Back in November 2012 we ran I’m a Scientist in Ireland for the first time. In short, it was a real success.

The enthusiasm shown by teachers, students and scientists was magnificent and the students really got into the spirit of the event.

The full evaluation report is up on the Irish site (imascientist.ie/2013/05/15/evaluating-im-a-scientist-in-ireland/) alongside a short summary of our findings.

In summary, we would love to run more events in Ireland.

Posted on May 15, 2013 in Evaluation Reports | Comments Off on Evaluating I’m a Scientist – Ireland

I’m a Neuroscientist Live – Project Report

IANSstageSummary:

Gallomanor was approached by the Wellcome Trust to run a live version of I’m a Scientist as part of the Wellcome Trust / Barbican Wonder Season in conjunction with the BNA Festival of the Brain in March and April 2013.

The deliverables were:

  • 2 Weekender heats
  • Final in Barbican Cinema One with a paying audience
  • IAS Zone to accommodate questions submitted by public during weekender.

Gallomanor delivered the events but a lack of questions from the public outside of the events meant we did not need the IAS Zone.

The events were a great success. The Weekender heats had capacity audiences; the final was sold out. We succeeded in recruiting excellent scientists, but it was harder than expected. We had to do more cajoling and encouraging than originally anticipated. This surprised us as we expected that more scientists would be keen to join in a Wellcome Trust event.

Key learning points:

  • The live version of the event works really well
  • Helen Arney was a brilliant MC
  • Keeping the format simple works best and causes less stress
  • Using coloured voting cards and getting the scientists to co-ordinate was the best idea this year
  • Electronic voting cards work well too
  • The public are not likely to submit questions in advance of an event. They need the context and a scientist to react to.

Key figures:

  • Weekender events estimated audience: 150 at each
  • Wide mix of ages from 1 – 80+, no bias towards men or women
  • The final was sold out. 260 tickets were sold and 20 comps allocated to the Wellcome Trust and guests. On the night however, only 218 people turned up. Evidence based on responses from voting cards.

IANSaudience1

IANSaudience2

 

 

 

 

 

Of those there, 9.5% were attending the BNA Conference, and 65% were not scientists;

IANSwho

the audience enjoyed the final;

IANSenjoy

learnt a lot;

IANSlearn

and now have more questions about brains.

IANSbrains

These charts should be treated with some caution. 35% of respondents also claimed to have no brain.

IANSbrain

The budget for the project did not include a formal evaluation, but anecdotally the scientists enjoyed the experience:

IANSpascaltweet

IANSjarvistweetIANSbirchtweetLet’s do it again.

Posted on April 18, 2013 in Evaluation, Evaluation Reports, I'm a Scientist - Live | Leave a comment

November 2012 – Evaluation Report

Nov reportIn November 2012 we ran 3 zones in I’m a Scientist. We used the event to make a few changes to how we ran it. We opened the event only to teachers who had taken part before. We used a new online registration system for teachers and students. And we ran 3 themed zones which tied into what students were being taught at the time – Cancer, Cells & Genes.

This short report looks at how the November event went. Offering the event just to teachers who had taken part before worked well. Using the new online registration system had a few hiccups, but despite the teething problems online registration is the way forward.  Tying the themed zones into what students are learning at the time was a success. As one teacher said “I used it as an extension activity within the course they were already doing which was great. A really good idea to do this”.

Download the report here.

Posted on March 4, 2013 modangela in 2012, Evaluation, Evaluation Reports, News | Leave a comment

November 2012 event evaluation report

Nov reportIn November 2012 we ran 3 zones in I’m a Scientist. We used the event to make a few changes to how we ran it. We opened the event only to teachers who had taken part before. We used a new online registration system for teachers and students. And we ran 3 themed zones which tied into what students were being taught at the time – Cancer, Cells & Genes.

This short report looks at how the November event went. Offering the event just to teachers who had taken part before worked well. Using the new online registration system had a few hiccups, but despite the teething problems online registration is the way forward.  Tying the themed zones into what students are learning at the time was a success. As one teacher said “I used it as an extension activity within the course they were already doing which was great. A really good idea to do this”.

Download the report here.

Posted on December 11, 2012 in Evaluation, Evaluation Reports | Comments Off on November 2012 event evaluation report

GM Food Zone Evaluation

GM Food Zone

For some time we’ve been wanting to expand the I’m a Scientist format and allow the general public to participate. The hysteria generated over the Rothamsted GM Wheat trials in May 2012 presented us with the opportunity to run a GM Food Zone.

We’ve written up our evaluation of the project. There are 4 sections in addition to this page:

You might find this incomplete. Please ask more questions in the comments.

For those who are too impatient to read the entire evaluation here is

The short version…

Driven by some hysterical writing in the national press we decided to run a special I’m a Scientist GM Food Zone and open it to the general public. We wanted to create a neutral space where a constructive and civil discussion could take place.

We knew that we were placing ourselves in the line of fire. We knew we had to be neutral and seen to be neutral and balanced. We recruited an advisory panel and asked for nominations for the experts who would answer the questions. We had 70 nominations and despite an enormous amount of work to ensure we had the right range and balance of experts we were still accused of being Pro-GM. It’s possible that assumptions were made because of the I’m a Scientist name or the 3:2 ratio of experts in favour of GM.

As the questions came in they were fairly balanced. Slightly more were anti-GM than Pro-GM. Almost half the questions were from teachers and school students. A different set of visitors then started leaving comments. Three determined anti-GM visitors left just over half the comments we received. There was very little cross-over between the group of people asking questions and those leaving comments.

The site has so far been visited 3,977 times by 2,370 unique visitors. They averaged 3.4 pages per visit, but most pleasing is the stat that on average visitors spent 3:19 min on each of the 5 most popular questions. Visitors were spending a reasonable amount of time reading through the answers and comments.

UPDATE 09/01/15

7,900 visits from 5,900 users.

So was the GM Food Zone successful?

Yes. We did create a space where discussion could take place. The discussion was polite and mostly respectful. Many people read the discussions and a small group contributed towards it. Whilst it would seem that comments tended to be left by people who had already made up their minds a significant proportion of the questions were from people genuinely seeking information.

Did we change minds? Move the debate forward?

I’m not sure. The debate over GM Food has been going on for a long time and many people’s view are firmly entrenched. That said we think many visitors will have come away better informed about the wider issues associated with GM Foods.

The future?

The format worked. The technology worked. It would be good to run a zone on a topic where the battlelines are not already drawn. To find a topic where experts are still working out the issues, where they don’t yet know how best to articulate the issues, where feedback from interested parties and the public has some real value.

Posted on September 25, 2012 ModShane in Evaluation Reports, GM Food, News | Comments Off on GM Food Zone Evaluation

2011 Evaluation

In June 2011 we came to the end of a grant from the Wellcome Trust to run I’m a Scientist in 2010 and 2011. We’ve evaluated throughout the project and the final evaluation report is available to download below, as well as a shorter executive summary and summary sheet.

2010-2011 Evaluation Report – at 75 pages long it contains a lot information including the methodology, aims, key figures and data, findings and recommendations.

 

Executive Summary – this outlines the main findings from the 2010-2011 Evaluation Report and looks at whether the objectives set at the start of the project have been met.

 

Summary sheet of the event in 2010-2011 – this highlights some of the key numbers and quotes from evaluations, and reasons why the event works so well.

 

How do we evaluate?

Evaluation is absolutely key to understanding what students and scientists have got out of the event, and it helps us continuously improve the event. There are two types of evaluation that we’ve done on I’m a Scientist, right from the first pilot event back in 2008.

Formative: We have done formative evaluation throughout the project – asking students, scientists, teachers and stakeholders for their views and acting upon them.

Summative: This is the more formal type of evaluation where we ask participants for their views before the events starts, observe the event in action in the classroom and survey views after the event has finished.

One of the benefits of running an online event is that we collect an enormous amount of data about how students and scientists engage with the event. We know how many students asked questions, how many questions they asked, how much they participated in live chats. We can tell the scientists how many students they’ve reached, how many questions they’ve answered and whether they’ve changed the attitudes of those students.

We’re always looking for ways to improve our evaluation, but what’s the point of evaluation is no-one reads and learns from what you’ve done. We’re genuinely interested in hearing what measures you’d like to see. What is it you want to see in an evaluation? Send us an email with your suggestions.

Previous evaluation reports

You can find the evaluation report summary event here

You can find the pilot evaluation report here

You can find the complete evaluation report on the 2008 pilot event here (this is the full, 127 page long baby)

You can find a summary of the 2008 pilot event evaluation report here

Posted on March 4, 2012 modangela in 2011, Evaluation, Evaluation Reports, News | Leave a comment

Sports Science Zone evaluation report

In March 2012 we ran a Sports Science Zone, funded by The Physiological Society. It was a big success.

  1. It was the most popular zone requested by teachers
  2. More than 2 sports scientists applied for each place
  3. The students asked lots of questions on sports and the human body. Keywords that poopped up a lot include Body, Blood, Brain, Cardiovascular, Cells, Exercise, Heart, Human Body, Lung, Oxygen, Performance and Running. Discussions ranged from how extra organs would effect the human body to the best way to improve personal health.
  4. Heart researcher Fiona Hatch won the zone and spent her prize money giving a sheep’s heart dissection lesson to a local school in Hull.

Read the full evaluation here.

As one sports scientist said “this has been a blast”.

Posted on February 13, 2012 in Evaluation Reports | Leave a comment

Evaluation

Evaluation in 2011

In June 2011 we came to the end of a grant from the Wellcome Trust to run I’m a Scientist in 2010 and 2011. We’ve evaluated throughout the project and the final evaluation report is available to download below, as well as a shorter executive summary and summary sheet.

2010-2011 Evaluation Report – at 75 pages long it contains a lot information including the methodology, aims, key figures and data, findings and recommendations.

 

Executive Summary – this outlines the main findings from the 2010-2011 Evaluation Report and looks at whether the objectives set at the start of the project have been met.

 

Summary sheet of the event in 2010-2011 – this highlights some of the key numbers and quotes from evaluations, and reasons why the event works so well.

 

How do we evaluate?

Evaluation is absolutely key to understanding what students and scientists have got out of the event, and it helps us continuously improve the event. There are two types of evaluation that we’ve done on I’m a Scientist, right from the first pilot event back in 2008.

Formative: We have done formative evaluation throughout the project – asking students, scientists, teachers and stakeholders for their views and acting upon them.

Summative: This is the more formal type of evaluation where we ask participants for their views before the events starts, observe the event in action in the classroom and survey views after the event has finished.

One of the benefits of running an online event is that we collect an enormous amount of data about how students and scientists engage with the event. We know how many students asked questions, how many questions they asked, how much they participated in live chats. We can tell the scientists how many students they’ve reached, how many questions they’ve answered and whether they’ve changed the attitudes of those students.

We’re always looking for ways to improve our evaluation, but what’s the point of evaluation is no-one reads and learns from what you’ve done. We’re genuinely interested in hearing what measures you’d like to see. What is it you want to see in an evaluation? Send us an email with your suggestions.

Previous evaluation reports

You can find the evaluation report summary event here

You can find the pilot evaluation report here

You can find the complete evaluation report on the 2008 pilot event here (this is the full, 127 page long baby)

You can find a summary of the 2008 pilot event evaluation report here

Posted on August 8, 2011 modemily in Evaluation, Evaluation Reports | Comments Off on Evaluation

So How *Was* IAS2010?

In our pilot event, we did nearly all of the evaluation in-house, to save money. We tried to be as objective as possible, but of course it’s much better practice to have an independent evaluator. This year, we’ve been able to do that, and we appointed the lovely Kate Pontin to evaluate the project for us.

A student looks at a log in card for I'm a Scientist

We are publishing the evaluation report on 2010 today (and also presenting it at the Science Council). You can also read the team’s early thoughts on this year in our write up of our Science Online London session, or see our evaluation report on the pilot.

Download full I’m a Scientist Evaluation Report 2010 (pdf)

Summary of I’m a Scientist 2010 evaluation

This report summarises the findings of the first year of formal evaluation of I’m a Scientist, Get me out of Here! and thus acts as an interim report.

I’m a Scientist creates opportunities for scientists to answer questions from students via the internet, opening dialogue, while developing skills and understanding of the science process. It also gives students an idea of what it is like to be a scientist and the types of careers available.

Evaluation data was collected from scientists, teachers and pupils via surveys, and interviews. Observations of a number of sessions in school were also undertaken.

Data suggested that the IAS programme is very successful, providing a unique approach to the communication of science to students. It shows that:

  • Students gained a lot from the experience. Their feedback shows they thoroughly enjoyed the event, especially the live chat sessions. They developed key skills including focused questioning and gained confidence in scientific discussion. They found it interesting to discover more about what it is like to be a scientist and were surprised that scientists were not as stereotypical as they originally thought.
  • Teachers found that IAS supported their need to develop different approaches in the classroom and also to encourage pupils to think about science and who it is actually undertaken by. They found the resources very useful, in particular the debate kit and were also pleased with the support they received from the team during the running of the event.
  • Scientists also gained or developed skills in communicating with the public, explaining their work (often complex science) clearly to pupils aged 13 onwards. This in some also helped inspire and revive their enthusiasm for their own studies.

Minor improvements suggested are:

  • Helping teachers to prepare pupils especially in their development of clearly focused questions
  • Promote and signpost other debate lesson packs like the IVF debate, and to help conclude the event, with perhaps an assessment element.
  • Further initial guidance for scientists giving guidance on the time it might take, but also for those less familiar with students of this age some general information on KS3/4 and the diverse knowledge base.

Evaluation will in 2011 focus on finding out more about how to attract scientists and about the impact their involvement has on their organisations. Longitudinal studies will look at the impact of students in the longer term (for example in take up of science or exam results). It is also hoped to develop baseline and plenary activities to find out more about change in knowledge and attitude to science.

I’m a Scientist enabled students using an innovative approach to inspire and enthuse. It worked with pupils from high achieving classes but also those across the ability range and from a wide range of backgrounds including groups from BME families.

Key figures for 2010

  • Two events – March and June
  • 7,459 questions asked
  • 125 scientists were involved, in 25 zones
  • 94,909 visits to the site
  • 6397 students took part
  • 648,563 page views

About the evaluator

Kate Pontin is a former geologist who has been working in museums and evaluation for over 20 years. She specialises in the experiences of young people in informal learning environments.

Posted on March 4, 2011 modangela in 2010, Evaluation, Evaluation Reports, News | Leave a comment

So how did I’m a Scientist go in 2010?

It’s important, in anything, to check that you’re doing a good job. When we’re running I’m a Scientist, we think we’re doing a good job and that the event is fun and exciting, and that students (and scientists!) learn things. But we aren’t the most impartial people to check!

We’re spending lots of money on this project (given to us by a charity called the Wellcome Trust), and we’re also taking up lots of lesson time at hundreds of schools around the country. So it’s important we check that the event is doing what it’s supposed to.

A student looks at an I'm a Scientist login card

So a researcher called Kate Pontin has been checking for us. She visited some schools when the event was happening (maybe she came to your school?). She watched lessons and talked to students. She also talked to some of the scientists and teachers. And she looked at the surveys some of you filled in at the end.

What she found out was:-
Facts and figures for 2010

  • Two events – March and June
  • 7,459 questions asked
  • 125 scientists were involved, in 25 zones
  • 94,909 visits to the site
  • 6397 students took part
  • 648,563 page views

Students

  • Enjoyed taking part
  • Got better at asking good questions
  • Got more confident talking about science
  • Found it interesting to talk to real scientists and learn about real science
  • Learnt that scientists weren’t like the stereotypes

Teachers

  • Found it helped them try out new ways of teaching
  • Helped them teach How Science Works
  • Helped them teach students about science and scientists
  • Liked the classroom resources
  • Thought the I’m a Scientist team were helpful (*beams*)

Scientists

  • Got better at explaining their work
  • Some of them found it made them more into their work and remember why they liked it.

She suggested we should make some improvements for next year though

  • Add more classroom resources for teachers
  • Make it easier for teachers to use the ones we’ve got already – like make them more obvious
  • Explain more to scientists what level GCSE students are at, so they’ve got a better idea what to expect in some of the questions.

If you want to take part next year (and you aren’t already signed up) then use these links

Teachers – to find out more about taking part, and sign up

Scientists – to find out more about taking part, and sign up

Posted on November 11, 2010 modemily in Evaluation Reports | Leave a comment

The long awaited IAS evaluation report is published!

Here is is, all 126 glorious pages of it. The I’m a Scientist Evaluation Report [pdf 1mb].

If you’re interested in how the event went and what people got out of it then look at Section 1 (findings) and Section 3 (case studies). If you’re a fellow science communication professional, and want to know more about how we did things and learn from our experience, then look at Section 2 (development) and Appendix 3 (formative evaluation findings).

The key messages were:-

Students said: “i learnt loads and feel much more confident to put my hand up and ask questions and know that ok sometimes i will get it wrong”

Teachers said: “The pupils were looking forward to their science lessons, asking about them in the corridor etc.”

Scientists said: “It engaged the kids in a way I’ve never seen before.”

Every scientist and teacher surveyed said they would recommend it to a colleague.

Key Outcomes

  • Students realised scientists are real, interesting, fun people
  • Students were inspired and enthused
  • Developed debate and discussion and How Science Works (HSW) skills

Key reasons it worked

  • Giving some power to young people gives them a reason to engage and shows that they are trusted
  • The fact that it’s real – real scientists, real science, real prize money – makes it far more vivid
  • The intimacy of the medium makes it easier to break down barriers and make connections

The key lessons for others were

  • Teachers want to use more debate and discussion for the new GCSE but feel they and/or the students don’t know how to do it. So resources which help are welcome.
  • Many existing resources to support the new GCSE feel ‘rushed out’ and don’t always ‘get’ what How Science Works is.
  • A teacher panel is a really useful way to include teachers in development of your project.
  • A project blog is a great way to be transparent and keep everyone informed.

I hope this is useful to people.

Posted on December 12, 2008 modshane in Evaluation, Evaluation Reports, IAS Event | Leave a comment

I’m a Scientist

‘I’m a Scientist, Get me out of here’ is a free online event where school students get to meet and interact with scientists. It’s a free X Factor-style competition between scientists, where the students are the judges. Students challenge the scientists over intense, fast-paced, online live CHATs. Then they ASK the scientists all the questions they want to, and VOTE for their favourite scientist to win a prize of £500 to communicate their work with the public.

Keep Me Updated

Subscribe to our mailing list

* indicates required
I'd like information for
Tell me about the following events

News Categories