Evaluation

Careers Zone

The Careers Zone is a new zone where students attending a careers fair at their school can talk online with a range of scientists and engineers, based all over the UK. A lot of the time, school careers fairs are only able to accommodate businesses and organisations in the local area. We want to give students an opportunity to ask their questions to experts working in a multitude of areas, letting them see how diverse STEM careers can be. In turn, STEM professionals can pass on honest advice and information, showing students what it’s like to be a real engineer or scientist. We selected a range of experts to take part, including a deep-sea researcher, a scientist for the NHS and a Jaguar Land Rover engineer. There were also engineers who had gotten into their professions through an apprenticeship. It was important for us to let students see the different … Continue reading

Posted on October 6, 2016 by in Evaluation, News, Project News | Comments Off on Careers Zone

Osmium Zone – I’m a Scientist, not just for scientists?

UPDATE: Read about the Institute of Physics’ research into gender differences in online engagement here. In all the zones we run, we aim to include a group of scientists that show how diverse STEM careers can be. At the start of the last school year our long-time collaborators, the Institute of Physics, asked us to take this idea a step further in the November 2015 Osmium Zone. A zone usually comprises five people working at the cutting edge of scientific research. This time the IOP wanted to fund a zone with four people who had studied Physics at A-level or higher, and now worked outside of academia and research, plus one physics researcher. The kicker? Ideally, those four people would also be in jobs that made use of their Physics education. Why? Studies show that children are not aware that studying science gives you transferable skills valuable in fields outside of traditional research. Making children more … Continue reading

Posted on September 1, 2016 by in Evaluation, Event News, News, Science Capital | Comments Off on Osmium Zone – I’m a Scientist, not just for scientists?

Demand vs. Capacity — An update for June 2016

Any avid readers of our project blog — there must be at least one of you — will have noticed we’ve written a lot recently about our increasing demand for classes, and our over-subscription rates. In January, we published some numbers. Since then we have run the March 2016 event, and finalised class places in the June 2016 event. Here are some updated numbers: In January, we wrote about the decreasing popularity of the June events, with more teachers moving to November and March. With the spectacular growth in November 2015, we speculated that teachers may be moving from March and June to earlier in the academic year. If that was true, then we would have expected a lower demand in March and June this year. What we see is a steady increase in demand in the June events. The growth in March 2016 was perhaps less than may have been … Continue reading

Posted on May 23, 2016 by modjosh in Capacity, Evaluation, IAS Event, News, Project News | Comments Off on Demand vs. Capacity — An update for June 2016

On student activity and simplified registrations

TL;DR: We made the site easier for students to access and the proportion of our audience engaging increased. Like any company, we have a set of Key Performance Indicators which we use to keep an eye on how well we’re doing. One of those KPIs (professionals use abbreviations) is the percentage of active students during an event, what we will call, %AS. %AS shows the number of students who log in to the site, and go on to — at the very least — ask a question, write a line of text in a live chat, cast a vote, or leave a comment. Basically, it shows the proportion of our audience who are actively engaging with the activity. In July 2014 Rosie posted a message on our project management app of choice, pointing out that the %AS for the previous events had been falling to the level it was at during the … Continue reading

Posted on May 23, 2016 by modjosh in Evaluation, Event News, IAS Event, News, Project News | Comments Off on On student activity and simplified registrations

Demand for classes

When we started I’m a Scientist we used our March events to allow teachers to test the event and for them to come back in June with more classes. That time after school exams when teachers looked for something to inspire kids was the ideal time. Times have changed since then. March remains popular as teachers look for activities to coincide with British Science Week, but June is no longer the most popular time of year. We think this is due to increasing numbers of schools starting the next year’s timetable and curriculum after exams and that fallow period is no longer fallow. November is where we saw some spectacular growth in 2015. We’re not sure what has driven this – we’ll investigate and we’ll rebalance when in the year we run zones. The other clear narrative from this graph is that excluding the lower than expected level of requests in … Continue reading

Posted on January 12, 2016 by ModShane in Capacity, Evaluation | Comments Off on Demand for classes

November 2015 – Bigger than ever

We’ve just published the latest batch of zone reports for the recent November 2015 round of I’m a Scientist. You can find them here, or at the bottom of this post, but before you dive into the pretty pie charts and wicked wordles, we felt it would be useful to provide some context to keep in mind as you read: We had an inkling that November’s eight zones were going to be big. As we reported beforehand, due to extremely high demand from teachers, it was the first time ever that we had to turn schools away from I’m a Scientist. We also had to limit the number of classes of those taking part to one or two per zone to fit as many schools in as possible. Without doing this we would have had to run double the number of zones to cover the demand. So, after the hype, exactly how big were the zones in November? Here … Continue reading

Posted on December 17, 2015 by in Capacity, Evaluation, Event News, News | Comments Off on November 2015 – Bigger than ever

3 words to describe a scientist

In March 2014 we started running zones for primary students only, increasing the number of primary school students who take part in I’m a Scientist. And we wanted to know more about how the event affects them.

This time, we drifted away from our usual online environment and sent printed surveys to teachers taking part in the Colour Zone and the Thulium Zone, the two primary-only zones we ran in June 2015.

We asked students which 3 words they would use to describe a scientist, before and after taking part. We got responses from over 300 students from 9 different schools. The students used more that 230 different words to describe scientists. They used 188 different words before taking part and 127 after, maybe indicating that they are clearer about what a scientist is after taking part. Awesome, cooperative, inquisitive, life-saving… are just some examples of the words they used Continue reading

Posted on November 2, 2015 by modangela in Evaluation | Comments Off on 3 words to describe a scientist

Which browsers and devices are schools using?

Every so often we take all our data, and look at how schools use the site; looking at how visitors access the site and how much that has changed in the past couple of years.  This post is for data up to 2015. See also data for 2014, 2018 and 2019. We’ve made a lot of changes in recent events, especially when it comes to registering accounts and using the site on mobile devices. We now pre-register all teachers and students, giving them usernames and passwords, so they can start asking questions immediately. We can assume that the student visitors give a fair reflection of the general school IT facilities and system capabilities. The graphs show the student data represented by dashed lines. We saw last year, that visits from mobile and tablet devices were increasing. and no surprise they continue to grow, while desktop usage drops slightly. For student tablet users … Continue reading

Posted on October 30, 2015 by modemily in Evaluation, School | Comments Off on Which browsers and devices are schools using?

Research vs Evaluation

Science Learning+ is a significant funding scheme provided jointly between the Wellcome Trust and the National Science Foundation. Learning can happen anywhere and at any time. Science Learning+ is an international initiative that aims to understand the power of informal learning experiences inside and outside of school. The second aim of the scheme is to “bridge the practice and research gap” At a seminar in July aimed at providing an update on the Phase I project an interesting conversation developed about that gap between Science Communication practitioners and researchers. I heard one speaker talk about practitioners wanting to know if a hypothetical red headline would give a 3% uplift in visitors. I responded on twitter: Disagree that practitioners want efficacy. I use eval. for that. I want research to tell me if the activity provides good outcomes #slplus — Shane McCracken (@ShaneMcC) July 27, 2015 Not all practioners agreed with … Continue reading

Posted on August 27, 2015 by ModShane in Evaluation, Science Engagement | Comments Off on Research vs Evaluation

Harwell Open Day Zone Report

On Saturday July 11th the Harwell Research Campus opened its doors to the public. 10,000 people were expected to come and see some of the most complicated and cutting edge science equipment in the world. Hundreds of volunteers working with that equipment were on hand to speak with the public and show them what happens. We created a Harwell Zone to allow visitors to text in questions. It was promoted using posters as above. The logic was that with 10,000 visitors some would not get to ask their questions, or may think of it later, or possibly be too shy to ask in person. It was a family day. Working with the over-worked (understatement klaxon just sounded) outreach team at Diamond and Harwell we set the zone up and promoted it to exhibitors. It wasn’t a great success. A mere 14 questions were sent. However in the interest of continual … Continue reading

Posted on August 24, 2015 by ModShane in Evaluation, News | Comments Off on Harwell Open Day Zone Report

Engaging under-served audiences – talk at Bristol Zoo

Last week I was asked to speak about our experience of engaging under-served audiences. Here are my notes from which I spoke: IAS is free online activity that connects school children and scientists. Kids go online, read scientist profiles, ask questions, take part in live chats and vote for the one they want to win. Split into zones of 5 scientists and about 350 students. Mixture of general and themed zones. Been running since 2008 and reached nearly 70,000 students and over 1,000 scientists and engineers have taken part. Scientists hear about it through colleagues, organisational emails and social media. They apply for the chance to take part. Tough selection process 1:4 for general zones. Students hear about it through their teachers who sign up to take part. Over subscribed for I’m a Scientist and many teachers don’t get as many classes as they would like. One way we do … Continue reading

Posted on June 8, 2015 by admin in Evaluation, Science Engagement, Widening Participation | Comments Off on Engaging under-served audiences – talk at Bristol Zoo

March 2012 – March 2015 Evaluation Report

We were awarded a Wellcome Trust Society Award to run I’m a Scientist from March 2012 until June 2014. We later received a grant extension for November 2014 to June 2015. This report is all about our learnings in these last three years: from March 2012 until March 2015. Our main learning points are: 1. I’m a Scientist has gone from 30 zones per year in 2012, to 54 zones scheduled in the 2014/2015 school year. 2. Expanding zones to different audiences: primary school students and general public shows. 3. I’m a Scientist is a public engagement boost for scientists. 4. I’m a Scientist gets students enthused about science. 5. I’m a Scientist reaches a diverse set of students. 6. Teachers come back, but tricky to track. 7. Students ASK about cancer, animals, and life and CHAT about science, scientists and work. 8. Moving forwards, further adaptation to new technologies (such as tablets and smartphones) is important. Click here to … Continue reading

Posted on May 26, 2015 by modangela in Evaluation | Comments Off on March 2012 – March 2015 Evaluation Report

Understanding Animal Research – Google Hangout

Before our most recent June 2014 I’m a Scientist event we partnered with Understanding Animal Research to run a Google Hangout for scientists who work with animals. Students often ask the scientists for their views on animal testing, and we want to help give scientists the confidence to be open and talk about working with animals in their research. John Meredith, Education Manager at UAR, ran this session with six scientists who were taking part in I’m a Scientist the following week: The A word: how to talk about animals in medical research  This informal webinar presented by Understanding Animal Research will look at how and why we should talk about animals in medical research. It will cover current public attitudes to animal research, the facts and figures, the value of openness and how to answer the tricky questions or deal with confrontation. There will be plenty of time for questions … Continue reading

Posted on August 6, 2014 by modjosh in Evaluation, IAS Event, News | Comments Off on Understanding Animal Research – Google Hangout

Wellcome Trust engaging science day

How to share and disseminate the learning from your project and evaluation. On 14th May the Wellcome Trust held one of their twice-yearly Engaging Science days. They’re an opportunity for grantholders to meet with each other and Wellcome staff, to share learning and ideas. And to sample Wellcome’s renowned lunch offerings (side salad complete with edible flowers and quail eggs, anyone?). Gallomanor were invited to speak on the panel of the session convened by Ben Johnson of Graphic Science, with the blurb “Once you have completed your project how do you widen its impact by ensuring others learn from what did (and didn’t) go right? An effective evaluation can also help with securing further funding and a wider roll-out of your project.” Other panelists were Manisha Lalloo from the Royal Academy of Engineering giving a funder’s perspective, and Becky Parker & Dave Colhurst from Simon Langton Grammar School. Evaluating the … Continue reading

Posted on May 19, 2014 by admin in Evaluation, News | Comments Off on Wellcome Trust engaging science day

I’m a Scientist zones for primary students

We’ve had plenty of primary schools take part in I’m a Scientist zones in the past, nestled in among secondary schools. Feedback from primary teachers and students shows they get a lot out of taking part. Primary schools are looking for science enrichment activities too, and talking with real scientists is exciting at primary school as well as secondary. Scientists often wouldn’t know the students were Year 5 & 6, not Year 7 & 8, if we didn’t tell them. This March we decided to run 2 primary school only zones, for Year 5 & 6 students. In part to avoid possible situations of primary students reading questions on non-primary-friendly topics asked by secondary students in their zone (think sex, drugs, rock’n’roll). Partly to open up I’m a Scientist to a wider group of UK schools and students. And importantly because one of the conclusions from the ASPIRES project about young people’s … Continue reading

Posted on April 10, 2014 by modjosh in Evaluation, News | Comments Off on I’m a Scientist zones for primary students

How does I’m a Scientist affect students’ attitudes to science?

We’re always evaluating the online STEM enrichment activity, I’m a Scientist, to explore its impact and how it can be improved. Check out latest evaluation posts. We know anecdotally that participating in the I’m a Scientist event has a positive effect on students’ attitudes to science. Teachers tell us that their students were buzzing, and that they understood more about science. Students thank scientists in live chats for an interesting lesson. They tell us that it is “better than Facebook”. But being data geeks that isn’t good enough. We wanted to know just how much we were affecting the students. Were we affecting them all by the same amount? Girls, boys, year groups? Did being more active in the event mean a greater change in attitude? Questions we ask students Thanks in part to the pilot work by Robin Longdin, then a SciComm masters student at UWE, we ask every student registering for … Continue reading

Posted on April 9, 2014 by ModShane in Evaluation, News | Comments Off on How does I’m a Scientist affect students’ attitudes to science?

March, June or November?

When teachers ask me what aged students they should bring on I’m a Scientist, my answer is often something along the lines of “well, it’s developed for Year 9 students but all ages get something different out of taking part, so take your pick”. The good news is that our data supports this. When students register we ask them what year group they’re in. There are Year 5s. There are also Year 13s. Around two thirds of the 30,000+ students registered so far are KS3 (Years 7, 8 & 9). The bad news? It’s not really bad news as such, but it looks like I should also be advising teachers on the time of year they should take part with different year groups. March and June are great for pre-GCSE students. Teachers tell us they’re looking for something fun to do with students at the end of the year. They’re less good … Continue reading

Posted on March 14, 2014 by modjosh in Evaluation, News | Comments Off on March, June or November?

Girls vs Boys

I was asked recently what the gender split was for students taking part in I’m a Scientist. I replied that we’d never looked on the assumption that since teachers took their students online it would reflect the school population. I’m never one to turn down the chance to crunch some numbers so I took a look. And was surprised. At registration we ask if the person registering is a Boy, Girl or A group of students (where more than one pupil is sharing a log-in). We have data going back to June 2012 and nearly 15,000 respondents to the question. Surprisingly it wasn’t a straight 50:50 split. So I then decided to look at it by year group. What we saw was that year 9 & 10 were significantly skewed towards girls. And that bias continued through to 6th form. Is that common for science outreach events?

Posted on March 11, 2014 by ModShane in Evaluation | Comments Off on Girls vs Boys

I’m a Scientist so far in numbers

How many zones have we run? How many students engaging with how many scientists? Asking how many questions? What year group are they? Where in the UK are they? These are some of the questions we get asked a lot, and until now haven’t had all the answers in one place to roll off the tip of our tongues. We’re pulling together some slides summarising I’m a Scientist to date and once they’re done we’ll post them up here. In the mean time, I’ll be posting snippets that show just how far we’ve come since our first big Wellcome Trust grant in 2010. To start us off (and to set the scene for the graphs and maps to come in future posts) here are some of the numbers so far, up to February 2014: Zones: 118 51 general science zones, 67 themed zones Spread across 10 events, in March, June … Continue reading

Posted on February 14, 2014 by in Evaluation, News | Comments Off on I’m a Scientist so far in numbers

Visits to IAS by technology type

In 2014, as part of a review of what we need to do over the next 5 years with I’m a Scientist, we looked at how visitors accessed the site and how that has changed year by year over the last 4 years. We also looked at how visitors registered as students differed from All visitors. The Student visitors probably give a fair reflection of school IT capabilities. In the graphs the student data is represented by dashed lines. We only have student data from 2011 when we started tracking registered users separately. View more data up to 2019, 2018, 2015. It was no surprise to see that visits from mobile and tablet devices were increasing but having 23.6% of visits from mobile devices in 2013 was a shock. Delving deeper it is clear that it is mostly questions that mobile users visit. Tablets tell a different story. For student tablet users … Continue reading

Posted on January 22, 2014 by ModShane in Evaluation | Comments Off on Visits to IAS by technology type