Category Archives: Evaluation

Researcher Development Framework and I’m a Scientist

It’s well established that scientists doing I’m a Scientist gain just as many positive outcomes as the students they talk to. Using the Vitae Researcher Development Framework to frame these benefits allows us to articulate them in a way that resonates with universities.

Recently, we surveyed event alumni from centres of doctoral training (CDTs) to quantify the effect of taking part on relevant RDF descriptors. This is part of our ongoing strategy to build relationships with these institutions. In total, 37 alumni who had taken part in I’m a Scientist or I’m an Engineer at least 3 months prior responded.

Pleasingly, we saw positive effects across the board. Here we breakdown the key results by relevant RDF domain (don’t worry, we weren’t interested in all 60+ descriptors).

RDF Domain B – Personal effectiveness

81% of respondents considered taking part as contributing to their Continuing Professional Development (B3 Continuing professional development B3)

73% say that taking part increased their enthusiasm and passion for their research (B1 Enthusiasm, B1 Self-confidence, B1 Self-reflection)

  • Getting to reflect on their personal motivations and the interest that students showed in their research were the biggest contributing factor to this effect.

68% had seen benefits to either their professional networks, the number of opportunities offered to them, and their profile and reputation as a researcher. (B3 Continuing professional development B3, Responsiveness to opportunities, B3  Networking )

“It seems to have been a big positive on my CV, which I have to admit I didn’t expect. I was asked about it specifically both at interview, and later by my line manager during the induction process.” – CDT student

“It establish me as an enthusiastic researcher willing to engage with researchers. if future employers google me they find something positive straight away.” – CDT student

RDF Domain D –  Engagement, influence and impact

94% said taking part improved their ability to communicate research with public audiences

  • Being able to ‘adapt language for different ability levels’ was the most improved skill among respondents

92% said they adapted their approach to communicating with students as the event went on

91% continue to use phrases they developed during the event to explain what they do in other contexts (D2 Communication methods, D2 Communication media, D3 Public engagement, also A3 Argument construction)

69% said taking part had improved their understanding of the impact of their research on society in some way (D3 Society and culture, D3 Global citizenship, D3 Public engagement, also C1 Ethics, principles and sustainability)

  • The impact of their research on people’s everyday lives was the area most had improved this understanding. Understanding of ethical issues and the economic value of their research were other improved areas.

“Writing an answer, it’s much easier to stop using jargon, it was a good lesson in recalibrating yourself… Now when you go to do face to face outreach you feel more equipped.” – CDT student

‘”Engineers typically do not have a lot of time to give back to the society in the form of public discussion, but this experience highlighted that it is very important to have those conversations with new generations coming through and as a result I have participated in more public engagement activities than I previously would have.” – CDT student

Posted on May 15, 2018 modantony in Evaluation | Comments Off on Researcher Development Framework and I’m a Scientist

Increasing students’ confidence with I’m a Scientist

“Many of our students lack confidence in their academic ability so they were buzzing when they realised they can hold their own in a conversation with intelligent, educated people and this helped them realise they are all scientists too!” – Julia Anderson, FE College Biology Lecturer

This general further education college is split across 3 sites in a large, post-industrial, non-university town. Students at the college took part in the I’m a Scientist Immune System and Genes Zones in March 2018. Julia tells us how meaningful engagement with the scientists increased her students’ confidence.

Why apply for I’m a Scientist?

One of the key things Julia is addressing with her students is their confidence issues: “Our students come to us lacking confidence. They’re doing A Level qualifications, but because they’re at FE college rather than a Sixth Form, they think they’re ‘2nd class’ and they really aren’t. I wanted to show them how worthy they are.”

Julia also wanted to get her students more involved in science so being able to connect them directly with scientists through the activity was another key incentive to apply: “They get unbridled access to scientists they wouldn’t normally get to talk to.”

What did the students do?

Students prepared by logging in to look around site and read scientists’ profiles. “We had a class discussion about how each scientist related to the A Level curriculum and debated the sort of questions students might want to ask ready for their live chat with the scientists.” Having access to the site for the full two weeks was helpful for Julia’s students to prepare and follow up either side of their live chat. “We finished with a de-brief lesson, talking about what was useful, what interested the students and their thoughts about the scientists as people.”

Growing students’ confidence

For Julia, the biggest benefit of the activity was improving her students’ confidence. “It was so good for them to see they can hold their own in a conversation with intelligent and educated people who have studied these topics for 10 years. They were buzzing and so excited to be talking to scientists and not sounding like idiots!” Taking part in I’m a Scientist not only helped Julia’s students gain confidence in their abilities, it helped them “realise they are all scientists too!”

Engaging students in a 2-way conversation

“Students and scientists were on fire, sending questions back and forth throughout the chats.” Julia explains how this 2-way interaction differs from a recent face-to-face where her students wouldn’t speak up; “we went to a university event and I was trying to get students to talk to the scientists and find out more about their work. They were too shy, saying things like ‘what if they don’t want to talk to me?’ There was none of that in the I’m a Scientist live chat. No fear from students that scientists would be too busy to talk to them, they were really comfortable.”

Improving social mobility

One of Julia’s students, from a deprived working class town he wishes to leave, was particularly interested in the background of the scientists; “the first thing he wanted to know was where the scientists went to school, was it state or private?” Participating scientists are selected to ensure a range of backgrounds and routes into science are represented. Julia’s student initially switched off from one scientist who attended private school but soon changed his mind, “when the scientist explained that the majority of students on his undergraduate course were from state schools, my student could relate to this academic route.” Julia managed to talk this student out of leaving college part way through his qualifications “I felt if he did, he would never leave the town he wants to leave. The scientists in this activity had varied backgrounds and he was content with that, it helped him see you can grow up working class, on a low income and get ahead.”

To help your students gain confidence in their abilities through I’m a Scientist activities, register your interest at imascientist.org.uk/teachers or contact admin@imascientist.org.uk for more information.

Already registered? Don’t forget to apply for the next event – we email registered teachers when applications open (about 2 months before the event starts).

Posted on May 14, 2018 modkatie in Case Study, Evaluation, News, School, Teachers, Widening Participation | Comments Off on Increasing students’ confidence with I’m a Scientist

Using I’m a Scientist to increase participation in higher education

“Traditionally, not many of our students go on to university. I’m a Scientist helps by allowing students to relate to scientists and helping them see the value of studying at a higher level.” – Mark McNally, Science Teacher

 

A mixed 2-19 academy, where over two thirds of the school population are students from disadvantaged backgrounds, took part in I’m a Scientist in March 2018. Mark tells us how the activity helped interest his students in science careers and consider higher education.

Why apply for I’m a Scientist?

Despite a good attitude to learning among the students, not many go on to higher education; something the school is working to improve. Mark explains how he wanted to challenge his students’ preconceptions of scientists and help them consider studying science at a higher level; “I wanted to show our students science is not just for ‘weird people with crazy hair and lab coats’ and help them find interest in things going on now in science to increase the chance of them pursuing STEM subjects in the future.”

What did the students do?

The activity was covered in three lessons led by the class teacher across a two week period. Students started by considering how to judge the competing scientists, then got to know them using their profiles and asking questions on the site. The final lesson involved an online chat where students typed their questions and responses to scientists in real time before voting for their favourite scientist.

Did it work?

Mark agrees that I’m a Scientist helps raise students’ science capital, increasing the likelihood of them studying STEM subjects or using science in their future professions. “Raising awareness of a variety of careers they can go into and getting them in contact with scientists helps students see it’s something they could do.”

Opening students’ minds to higher education

I’m a Scientist provided an opportunity for Mark’s students to engage with science professionals who have studied at a high level, helping to open students’ minds to options they may not have otherwise considered. “I think one of the main barriers for our students is that not many come from families with an academic background so they don’t often consider academic routes,” explains Mark, “through this activity, my students connected with academic people and found out about things that interest them in terms of a future career, so they are more likely to pursue an academic route.”

”Even if students don’t want a career in STEM, they can now see the value of studying at a higher level and if they do want a career in STEM, this activity helped cement that for them.”

Satisfying quieter students’ curiosity

Mark’s students developed knowledge and understanding on current scientific topics, “I’m a Scientist allowed the students to engage with the kind of science that’s going on right now in the world.” Mark also told us of the importance for his students to be able to ask whatever they liked throughout the activity, “for a lot of students, I’m a Scientist was about satisfying their curiosity. It’s important because they have so many questions and they don’t always ask, especially the quieter students, but using this platform allows them to get their questions addressed.”

To support your students in considering higher education through I’m a Scientist activities, register your interest here: imascientist.org.uk/teachers or contact admin@imascientist.org.uk for more information.

Already registered? Don’t forget to apply for the next event – we email registered teachers when applications open (about 2 months before the event starts).

Posted on May 2, 2018 modkatie in Case Study, Evaluation, News, Teachers, Widening Participation | Comments Off on Using I’m a Scientist to increase participation in higher education

Providing STEM opportunities for distant schools

“As we are a remote rural community we do not have a huge variety of careers on our doorstep but these events help to bring them closer to pupils. More students should be getting these funded opportunities across the UK.” – Emily Tulloch, Science teacher on the island of Unst.

The most northerly school in the UK is located in one of our most distant areas in the Shetland Isles. Emily tells us how I’m a Scientist allowed her remote students to explore a range of STEM careers and increased motivation to learn science.

Why apply for I’m a Scientist?

“For me, it’s all about increasing Science Capital.” Emily often tries to provide STEM opportunities in school, particularly to help students broaden their understanding of careers, but it can be a challenge. “The STEM ambassador programme is great and we have a number of ambassadors based in Shetland. However there are significant barriers for them to reach the school – it can take almost a day to visit for a 1 hour talk and of course there’s the cost implication too.”

The school often rely on parents to demonstrate different careers to students “we’re lucky that our community is so supportive and parents working in STEM are very willing to visit the school, although the variety of careers is still limited on our island”. Emily wanted to use I’m a Scientist to increase the range of STEM roles her students find out about.

What did the students do?

The 17 students in S1-3 (Year 7-9) took part in the Molecule Zone this March. They read scientists’ profiles and posted questions to scientists on the site throughout the 2 weeks. Students also took part in a 30 minute live chat session where they typed their questions and responses to scientists online in real time.

Emily used the activity to connect to curriculum areas and the careers work going on in school. “It’s a broad activity so easy to link to something we’re doing or have just covered in class. I also tie I’m a Scientist into My World of Work, which our students access to find out where they could study the course or the career steps required for their preferred role.”

Did it work?

Being online, I’m a Scientist provided a chance for Emily’s students to connect with scientists in a large variety of roles, generating interest in STEM careers without the need to travel. “They see different careers we don’t have in the local area by chatting with scientists all across the globe…There are lots of young people who live in remote areas compared to the rest of UK and they should all get same opportunities; this activity allowed me to provide this at no cost to the school.”

Emily also told us how involvement in the activity has improved science learning in her classes “It’s engaging and stimulating for students to actually speak to live scientists doing real-life science and discuss what they think might happen in the future. When students are engaged and start to see the real life impact of a subject, they’re more enthusiastic about it – I noticed an increased motivation to learn following I’m a Scientist.”

What else did students gain?

Asking questions to the scientists allowed Emily’s students to “develop essential literacy and communication skills, particularly in the live chat where they have to consider how to engage in a group conversation” whilst “researching the scientists helped them learn how to find information online.”

Emily also commented on how the activity is good for students’ health and wellbeing, by using the internet chats as a relevant educational tool “students like communicating in this way online and this activity helps promote positive online communication and staying safe online, rather than fighting against social media and similar technology in school.”

What would you say to a teacher who is hesitant to take part?

“Give it a go – it’s amazing and you don’t realise the full impact before you take part but afterwards, you realise how hugely beneficial it is, especially for students in rural communities. The whole process has knock on effects for your science teaching as students are more engaged. It’s very stimulating for science and every pupil should have access to it.”

Distant (under-served) schools are given priority places in I’m a Scientist. To support your students in exploring a range of STEM careers with I’m a Scientist activities, register your interest here: imascientist.org.uk/teachers or contact katie@mangorol.la for more information.

Already registered? Don’t forget to apply for the next event – we email registered teachers when applications open (about 2 months before the event starts).

 

Posted on April 6, 2018 modkatie in Case Study, Evaluation, News, Teachers, Widening Participation | Comments Off on Providing STEM opportunities for distant schools

Broadening horizons for students in a deprived area

“It was a really easy project to engage with for both children and teachers; it only took me about 30 minutes to prepare for all 3 lessons. I was pleasantly surprised with just how excited students were to get responses from serious adults about their work and the adults’ interests.” – Vicky Heslop, Year 6 teacher

A junior school that meets our widening participation criteria took part in the Climate Zone of I’m a Scientist for the first time in March 2018 with their three Year 6 classes. The activity broadened student aspirations, improved enquiry skills and challenged their perceptions of scientists.

Why apply for I’m a Scientist?

The school is in a small town with low levels of aspiration and social mobility where very few young people go on to higher education. 47.5% of students at the school qualify for Pupil Premium funding and the proportion of free school meals eligible students is over twice the national average.

Teachers were finding it difficult to promote working scientifically skills and to support students in developing an enquiring attitude within the curriculum. “We wanted to provide a broader experience of what science is and to increase student aspirations,” says Vicky.

What did students do?

The activity was spread across 3 lessons on different days which Vicky felt was “particularly good for the anxious students in Year 6, allowing them to ‘take a break’ from SATs preparations with something that was still a valuable use of time.” Students started by considering how to judge the scientists, then got to know them using their profiles and asking questions on the site. The final lesson involved an online chat where students typed their questions and responses to scientists online in real time. “Connecting with the scientists online provided an opportunity to have positive social contact with adults in roles they wouldn’t normally have contact with,” observed Vicky.

Did it work?

Vicky told us science has become more relevant and attainable for students and how their aspirations have broadened to include science – “After the live chat, students were telling me how they’d like to become scientists.”

Students’ perceptions of scientists have changed as a result of taking part – “I thought scientists were boring but now I think they’re AWESOME!” – Year 6 student

What else did students gain?

“Students developed oracy skills,” says Vicky “and the ability to ask appropriate questions.” Vicky also explained how the school has had issues with inappropriate use of instant messaging and how I’m a Scientist was “a great way to demonstrate a positive use of this technology,” helping students learn appropriate online etiquette.

When asked if she’ll take part next year, Vicky says “I hope to and I’ll be telling the other teachers about the activity as it’s such a good one for our students.”

If you’d like to broaden your students’ aspirations with I’m a Scientist activities, register your interest here: imascientist.org.uk/teachers

Already registered? Don’t forget to apply for the next event – we email registered teachers when applications open (about 2 months before the event starts).

 

Posted on March 27, 2018 modkatie in Case Study, Evaluation, Teachers, Widening Participation | Comments Off on Broadening horizons for students in a deprived area

School engagement in STEM enrichment: Effect of school location

In recent years funders of public engagement and outreach activities have made a priority of reaching underserved audiences.

Wherever we looked we found anecdotal evidence that while, as a sector we were becoming increasingly effective at reaching schools in deprived parts of our metropolitan areas, rural communities continued to miss out.

But anecdotal data only gets you so far. We wanted to find out just how much the more remote schools were missing out. We also wanted to know what constitutes a remote school in this context.

First we looked to see what information already existed. There are some organisations who hold vast quantities of data about scientists and engineers visiting schools. However that data was not easily available for analysis. So we turned to a source we could access. The teachers who have signed up for our projects.

We wanted to find out:

  • Whether some schools access more STEM enrichment activities than others
  • And if so, is the location of the school a limiting factor
  • Whether there’s a difference between visits to the school, and visits from the school

Method:

We surveyed teachers who have registered for any of our UK based projects, including I’m a Scientist, I’m an Engineer and Debate Kits. The teachers were predominantly subject teachers. We therefore worked on the assumption that teachers can best talk about the classes they teach, and it would be unreliable for them to make assumptions about the wider school.

Who answered the survey?

Survey respondents: Subject taught (left), type of school taught at (centre), and location of schools (right).

What did we find?

Distance really matters.

Access to STEM engagement is not universal. Schools within 15 minutes drive of a major research HEI are twice as likely to get a visit from a university scientist than those over a 30 minute drive.

Interestingly, there doesn’t seem to be a difference when arranging to take students off-site for visits, but with teachers citing costs and time restraints as barriers to offering these activities, it’s clear to us that we need to be improving the offer to more distant schools.

If you have any comments, thoughts, or would like to know more, please get in touch with shane@mangorol.la.

Posted on November 30, 2017 modnaomi in Evaluation, News, Widening Participation | Comments Off on School engagement in STEM enrichment: Effect of school location

Thinking about Science Capital

We’re thinking increasingly about Science Capital and how we apply it to our projects. It is a powerful concept that resonates strongly with what we aim to do and we want to make sure our projects make as much of a positive contribution to young people’s Science Capital as they can. We are looking for ways to evaluate in relation to Science Capital to show whether we are achieving this.

What is Science Capital?

If you’ve read the 2013 ASPIRES report you’ll be familiar with this striking graph showing that although nearly 80% of UK students value science, less than 20% aspire to be scientists. Why is this?

Graph courtesy of ASPIRES/ASPIRES2: ucl.ac.uk/ioe/departments-centres/departments/education-practice-and-society/aspires

Based on their research, the ASPIRES team have developed the concept of Science Capital; the combination of experiences, personal connections, knowledge and attitudes that contribute to how much a young person identifies as a “science person”. The researchers found that young people with high Science Capital are more likely to have science-related career aspirations than those with lower Science Capital.

Dimensions of Science Capital

The ASPIRES researchers identified eight dimensions of Science Capital. We’ve summarised them below. For more detail follow the link to the Enterprising Science site.

  1. Scientific Literacy
    A young person’s knowledge and understanding about science and how science works. This also includes their confidence in feeling they know about science.
  1. Science-related attitudes, values and dispositions
    The extent to which a young person sees science as relevant to everyday life.
  1. Knowledge about the transferability of science
    Understanding that science qualifications, knowledge and skills have broad applications and are useful for a wide range of jobs beyond and not just in science.
  1. Science media consumption
    Engaging with science content in the media (books, television, or on the internet).
  1. Participation in out-of-school science learning contexts
    Participation in, e.g. science museums, science clubs, fairs, etc.
  1. Family science skills, knowledge and qualifications
    Having family members with science qualifications, skills, and interests.
  1. Knowing people in science-related roles
    Knowing people in their community who work in science-related roles.
  1. Talking about science in everyday life
    How often a young person talks about science out of school and the extent to which they are encouraged to continue with science.

Science Capital and I’m a Scientist

Our data and experience suggest that taking part in our I’m a … events has a measurable impact on students’ attitudes towards science and make a positive contribution to young people’s Science Capital.

We’ve looked at the 12 dimensions of Science Capital listed above and thought about whether and how I’m a Scientist contributes to each. There are three dimensions where we think our contribution is – or should be – most significant: knowing people in science-related jobs, scientific literacy, and knowledge about the transferability of science.

Knowing people in science-related jobs

Description: The people a young person knows (in a meaningful way) in their family, friends, peers and community circles who work in science-related roles.

We had a lot of discussion about whether the interactions between young people and scientists during I’m a Scientist, qualify as knowing someone “in a meaningful way”. Ultimately, we felt that while our events don’t fully fit the description, they fit the spirit, and – we think – make a small, though convincing contribution.

In fact, this is the area where we think our projects are most distinctive among STEM interventions. Young people who take part, engage in sustained and enthusiastic interactions with a group of real STEM professionals. Some students even form opposing teams to support their favourite scientist!

How we contribute:

  • Through voting, young people…
    • Take time to consider what is important in making a good scientist
    • Learn about scientists by reading their profiles and deciding who to vote for
    • Make personal judgements based on their direct interactions with the scientists and choose favourites
  • Through ASK and CHAT, young people…
    • Hear about scientists’ motivations and achievements
    • Learn about scientists’ interests, likes and dislikes, and find areas in common
    • Hear about life as a scientist from scientists’ points of view
  • Through the whole process, young people learn that many different types of people become scientists

Scientific literacy

Description: A young person’s knowledge and understanding about science and how science works. This also includes their confidence in feeling that they know about science.

We see plenty of evidence that suggests we contribute to this. Students ask scientists and engineers a huge variety of questions about science and how science works. In addition, we hope that the student-led approach and the willingness of scientists and engineers to answer pretty much any question gives students more confidence that they know about science.

How we contribute:

  • Scientists answer questions on scientific topics and students learn from their answers
  • Students learn about scientific process, ethics, and science in society
  • Young people develop their confidence in feeling that they know about science as they ask questions

Knowledge about the transferability of science

Description: Understanding the utility and broad application of science qualifications, knowledge and skills used in science (e.g. that these can lead to a wide range of jobs beyond, not just in, science fields).

Knowledge about the transferability of science is a really important dimension, and one where we should be able to make a contribution. At the moment, we are concerned we’re not getting it quite right.

We have run the occasional zone with candidates who have a science background but work in other roles, but in general zones tend to comprise only practicing scientists or engineers. We’re concerned this reinforces the “science = scientist” idea rather than helping students see that science skills are transferrable.

We’re working on evaluating whether our concerns are justified, and how we could adjust the composition of future zones to improve things.

Next steps

We’re just at the start of this process of evaluating projects in relation to Science Capital. As mentioned above, the process has already made us think about how our I’m a… projects contribute to young people’s knowledge about the transferability of science.

We’d really welcome discussion about this to help us develop our thinking so please get in touch and share your thoughts, either in the comments below, or elsewhere.

As a final note, there are also a few Science Capital dimensions where we don’t have much influence, and one – science-related attitudes, values and dispositions – where we think we do contribute, but would find it hard to evidence.

Posted on October 6, 2017 modbecky in Evaluation, News | Comments Off on Thinking about Science Capital

How do students from different schools engage with IAS?

Thousands of school students meet scientists through I’m a Scientist every year, and they ask thousands of questions.

In June 2017, over 3,000 students took part, asking scientists more than 2,500 questions in the ASK section alone. This is also the event that we implemented our question coding system across all the zones to see what students are asking about.

This all got us thinking:

Do students from different types of schools ask more or less of certain question types?

We’ve identified two groups we want to look at:

Under-served: Schools more than 30 minutes travel time from a major research HEI

Widening Participation: Schools with an above average number of students eligible for free school meals

Taking the questions from the I’m a Scientist zones in June 2017 it appears that:

  • Overall, the split of questions is similar across all groups of students
  • Under-served students ask more “science topics” and “personal” questions, getting to know the scientists outside work
  • WP students ask slightly more questions around “careers and education”

 

 

 

 

 

 

For all groups, questions about careers and education are the most common, and questions about the event or completely random and unrelated are the least (phew).

But why the differences?

Are they indicative of how students at different schools view scientists and STEM?

Do under-served students ask slightly more “personal”, and science topic questions to compensate for fewer opportunities to meet scientists in person?

Does the slightly higher percentage of WP students asking about “careers and education” demonstrate their greater interest in understanding future opportunities in STEM?

 

We don’t have the answers, but it’s certainly interesting to us that there are small differences in the types of questions students’ ask.

If you have thoughts on why this might be, we’d love to hear them, or if you want to talk about how you can support more under-served and WP students to have this opportunity get in touch: shane@mangorol.la | 01225 326892

Posted on September 14, 2017 modnaomi in Evaluation, News, Widening Participation | Comments Off on How do students from different schools engage with IAS?

“I’m a Scientist is great, but wouldn’t it be better if students could see and hear the scientists too?”

This is a question we get asked from time to time. Here we explain why we’re confident that text interaction remains the best format for effective online engagement: it makes students and scientists more comfortable, levels the playing field between adults and children, and makes the events accessible to a wider audience.

Students are more familiar with text-based chats.

There is growing evidence¹ that young people communicate most via text and less and less through phone or video and we’re hearing that anecdotally too. Feedback from teachers has pointed out that students are not only more familiar with a text format but also more comfortable with it.

All parties feel more confident about not being visible.

As an ex-teacher myself, the thought of making a class visible online to an unknown person via a webcam makes me uncomfortable. I’d also be concerned about scientists inadvertently displaying confidential or inappropriate material in the background. Text-based chats make it much easier to protect student identity and safeguard young people online.

One teacher told us his students didn’t want to be involved in I’m a Scientist at first because they assumed it would involve webcams and audio. Once he showed the class what the text-based chat looked like, they were much more comfortable and keen to get involved. Consequently these 16 students, who wouldn’t have participated in a video chat, asked 89 questions in their text-based live chat.

Both students and scientists feel less exposed through text-based chats in comparison to audio-visual. For students, this has a huge impact because the fact they are “hidden” gives them the confidence to fully engage with the event and students who are often too shy to speak up in class are able to do so in this environment.

A big part of what we do is break down stereotypes and whilst scientists have a profile picture on the site, the lack of video and sound means students focus less on what the scientists look and sound like and more on what they have to say.

Being text-based also provides an opportunity for scientists to communicate clearly with the students and provide meaningful responses:

Logistically, it’s easier.

A number of scientists have commented on the convenience of text-based chats. With our current model, scientists can take part from their office or usual work environment; having audio-visual chats would make these kind of outreach activities inaccessible to some scientists.

In fact, our current model is so easy to access, scientists have taken part in public places when out of the office, including a motorway service station and Glastonbury festival!

Online outreach activities, both text-based and audio-visual, allow schools to connect with scientists despite being in very different locations; this provides a school with access to hundreds of STEM professionals across a range of roles. Online outreach is especially important for rural schools, but it only works if the schools have the infrastructure to facilitate the projects. A number of our rural schools have previously expressed concerns about bandwidth. Whilst their internet connections are able to cope with text interactions, a video chat would be problematic in these locations.

Our chats are fast-paced but everyone gets their say.

Whilst it is possible to create a video chat with multiple scientists, turn taking would need to take place, losing the fast pace that we currently have in our text-based chats. This means fewer questions answered in the same time frame and less impactful engagement.

A similar challenge occurs when considering how students ask their questions. Undoubtedly the teacher would need to facilitate the session to decide who speaks next and, potentially, which questions should be asked. This intervention would diminish the excitement element of I’m a Scientist. It would also dissolve the direct link between student and scientist in our events, taking the ownership away from students.

Through our text-based chats, we tackle the ‘those who shout loudest get heard’ issue with chat features and moderators creating a situation where all students get attention. The number of replies a student gets is displayed to scientists and moderators who can then focus on students with fewer replies.

Our events are about more than the live chat.

Through our text-based chats and the two week long events, students not only have time to fully engage and ask questions to the scientists, they also have the chance to build a rapport with the scientists and get a true picture of what their lives entail. Students have access to five experts, providing more breadth and an opportunity to interest more students in the class.

The competition element of our events generates excitement as the students are in charge of who wins and the scientists are fighting for student votes. Much of our feedback from teachers explains how this creates longer-term student engagement than a one off chat. Students continue to talk about I’m a Scientist long after the event finishes – an indicator of the popularity and success of our events.

We could go on but the main point is…

To get the most from outreach opportunities and maximise engagement of all parties, it is essential that students and scientists feel equally comfortable and the format is as accessible as possible. We’re confident that our current text-based chat model is the best way to do this.

If you would like to talk more about our reasons, leave a comment or email me at katie@mangorol.la, we’d love to hear what you think.

¹A 2014 Ofcom report found that children aged 12-15 spend over 50% of their communication time on text messages, compared to just 6% on phone calls and video calls combined.

Back to top

Posted on August 4, 2017 modkatie in Evaluation, IAS Event, News, Science Engagement | Comments Off on “I’m a Scientist is great, but wouldn’t it be better if students could see and hear the scientists too?”

What do students ASK about?


Our events generate huge amounts of interesting data, which we know contains all sorts of valuable insights. When there’s so much of it, it can take a while to work out how to make best use of it. One example of this is thematic analysis of the questions students post in ASK. Over the years of running the event we’ve developed a pretty good sense of the types of things students ask about and will always showcase great examples of questions. Converting that into a systematised, reportable analysis is more difficult, but can be done. So, we have developed a system for coding questions based on a set of themes that students often ask about, and after some trials in March, we’ve applied it to all zones in the June event (read the latest reports here).

At their broadest level, most student questions fall into one of three areas: questions about the science; questions about the scientist; questions about the I’m a… event. But we can look at this in more detail, considering elements of science capital (look out for a separate blog on this coming soon), and “how science works” (although this term has now been re-framed as “working scientifically” in the latest UK curriculum, we think it is still descriptive and useful for us). We’ve identified five top-level themes that we will report on. Each of which has one or more sub-themes which we are using to code the questions.

 

Science topics

covers facts, theories and knowledge about science. These can be related to the scientists work, or more general questions about science topics. Examples include…

When was the first epidemic discovered?

What animal has the worst venom and what does it do?

Can you give a robot human emotions?


How science works

looks at the process, motivations and ethics behind science. This could be finding out more about how decisions are made, why people chose science, or looking at how science fits into wider society. Examples include…

Is your work accurate and can be depended on?

Why do you think that a lot of people believe that psychology isn’t a science?

How did you come up with the idea of making MRI scans faster?


Career and education

includes the experiences of the scientists and aspirations of the students. Questions in this category build a greater understanding of real-life careers and education, including previous experience, current situation and future plans. Examples include…

What criteria did you use to choose your university?

Do you think you’ll ever leave the science job route for something different?

What is a PhD and how do you get one?


Personal

encompasses questions about scientists lives, knowledge and opinions outside of their work. Examples include….

Do you still have a social life?

From studying human relationships have you altered how you interact with people in day to day life?

What’s your opinion on the election?


Event/other

includes questions about the event and prize money, as well as a handful of questions that really don’t fit anywhere else. Examples include…

Do you think you can win I’m a Scientist?


Method

Using moderators and the event team each zone is coded, and then checked, so each set of questions have at least two run throughs. We use a coding guide as reference for consistency, and we are always building on our bank of reference questions for each category.

Students can ask anything they want, so it’s not always a clear-cut decision. Some questions overlap the different categories, but in general fit best within one of the reportable themes. There are some trickier questions that a few people will make a decision on together, and when we see a lot of similar questions we can agree how to code these. Although this won’t eliminate the ambiguity, it will help us to be consistent and transparent in how we code.

 

Results

Here’s how the questions distribute across all I’m a Scientist zones for the June 2017 events.

 

You can see the outcomes for individual zones in the latest I’m a Scientist reports and I’m an Engineer reports. We’re going to keep refining this coding to make it more consistent and the best reflection of our events it can be.

If you have any comments, thoughts, or would like to know more, please get in touch.

Posted on July 17, 2017 modbecky in Evaluation, News | 1 Comment

Gender differences in online engagement

“I thought scientists just looked like they do in the film Flubber and experimented on aliens or weird stuff but when I found out you liked Taylor Swift I realised you are more down to earth and not like mad scientist :)” – Student, November 2015

I’m a Scientist is about connecting pupils with real scientists. Something we want to find out is what effect this interaction has on different groups of students. Recent research by the Institute of Physics has done just that.

The IOP has carried out an independent research project to gain an insight into the behaviour and attitudes of boys and girls who do I’m a Scientist. As part of their Improving Gender Balance project, the IOP funded two zones: Terbium Zone and Osmium Zone.

The researchers anonymously surveyed the students’ attitudes to science and scientists before and after the event. We also provided them with all the student’s interaction data from the sites. Every line of live chat, every question in ASK. Specifically the researchers wanted to look for gender differences in two areas:

  1. How taking part in the event changed students’ attitudes to science and scientists
  2. The types of questions that students asked the scientists.

The first results from analysis of this treasure trove are now online at the IOP blog.

IOP gender difference Q1

Physicist, know thyself: After the event, girls in particular reduce the negative words they use to describe you.

Check out the post for their full findings on the effects of taking part in I’m a Scientist, including:

  • positive changes in word choice to describe scientists
  • an increase in girls’ awareness of careers where Physics A-Level is useful
  • girls’ improved confidence in talking to physicists

Girls want to see the scientists as real people

The report also dissects what girls and boys want to know from the scientists. Interestingly, as well as asking more questions overall, girls seem to ask many more personal questions, asking more than boys about job satisfaction, the career goals and achievements of the scientists, and their motivation to work in science.

For us, the identified trends make sense. For example, a boisterous classroom culture can be a barrier for quieter children to engage with visitors. As Natasha at the IOP notes, ‘the online, anonymous nature of the live chats gives students more freedom to ask questions than a traditional careers talk or even a speed-networking-style careers event.’ Girls, as a group, seem to relish this freedom.

And we believe that for science to be appealing there’s no need to glam it up with explosions and ‘wonder’. The more students hear about the reality of science from people like them, the more positive they feel about it. This research backs up that view.

Research with us

If girls are a group that benefits from this increased exposure to scientists as real people, it’s likely other underrepresented groups do too.

I’m a Scientist provides a unique way to study the ways children interact with scientists. We’d love to see more research being done using our data. If you think you, or someone you know, might be interested in analysing what hundreds of children want to know about science, just drop me a line at antony@mangorol.la

Read more on the findings in the full blog from the IOP: Online event gives insight into gender imbalances

And find out more about how Osmium Zone was set up: I’m a Scientist, not just for scientists

Posted on October 21, 2016 modantony in Evaluation, News | Comments Off on Gender differences in online engagement

Careers Zone

Report Cover

Careers Zone report, click to download

The Careers Zone is a new zone where students attending a careers fair at their school can talk online with a range of scientists and engineers, based all over the UK.

A lot of the time, school careers fairs are only able to accommodate businesses and organisations in the local area. We want to give students an opportunity to ask their questions to experts working in a multitude of areas, letting them see how diverse STEM careers can be. In turn, STEM professionals can pass on honest advice and information, showing students what it’s like to be a real engineer or scientist.

We selected a range of experts to take part, including a deep-sea researcher, a scientist for the NHS and a Jaguar Land Rover engineer. There were also engineers who had gotten into their professions through an apprenticeship. It was important for us to let students see the different options they have as they make decisions about their future careers. Experts could log in to the site from wherever they were for their scheduled Live Chat sessions, and then answer questions sent to them as they had time throughout the day (we even had an engineer take part in a chat from Mexico).

We set up on a long table with iPads and laptops for students to sit down at and use. We had two banners which showed some example questions, but often students came with a specific area of interest and asked us who the best expert was for them to talk with.

After two pilots, we’ve learnt it’s important for us to be on the same level as all the other stands at the fair, making approachable for students to ask who we are, and whether they can join in. We were also told that there was an announcement in the school newsletter about us being at the fair, and students had been preparing questions during tutorial times. This meant they came to the stand interested and prepared, making the chats lively and focussed. We’ve left the zone open so students can continue to log in, ask questions and leave comments.

Careers ZoneThe project clearly works at some level, but we are keen to pilot it in different schools. There are things we’ll have to change for next time, such as shortening the log in process, which can take valuable time away from students who only have 30 minutes or less at the fair. We’d also like to try a new way of showing students how to use the site. We’ve written up a report about our second pilot, including some examples of conversations that happened in the chat and ideas for next time, which you can read here.

Posted on October 6, 2016 modmichaela in Evaluation, News, Project News | Comments Off on Careers Zone

Osmium Zone – I’m a Scientist, not just for scientists?

UPDATE: Read about the Institute of Physics’ research into gender differences in online engagement here.

In all the zones we run, we aim to include a group of scientists that show how diverse STEM careers can be. At the start of the last school year our long-time collaborators, the Institute of Physics, asked us to take this idea a step further in the November 2015 Osmium Zone.

A zone usually comprises five people working at the cutting edge of scientific research. This time the IOP wanted to fund a zone with four people who had studied Physics at A-level or higher, and now worked outside of academia and research, plus one physics researcher. The kicker? Ideally, those four people would also be in jobs that made use of their Physics education.

Why? Studies show that children are not aware that studying science gives you transferable skills valuable in fields outside of traditional research. Making children more aware of this fact is a key recent recommendation from the ASPIRES project into children’s aspirations.

...And reply they did!

…And reply they did!

The first unknown: Would people outside the usual research audience be interested? Answer? Yes. A call went out on twitter, and very quickly we had a range of people interested in the zone. The final selection of the Osmium Zone consisted of a communications officer for the Royal Academy of Engineering with an undergraduate degree in physics, a diplomat at the British Embassy in Tokyo working in nuclear disposal,  a biomedical engineer, and a data analyst for a solar company. The ‘token scientist’ was a  postdoctoral researcher studying lasers.

So what happened? We were confident that both competitors and children would still find the event as engaging as ever, away from the usual science focus. This is illustrated every year in the sister I’m an Engineer project that covers the diverse world of engineering. The metrics for activity in the zone point to to this holding true, showing busy live chats and especially high numbers of page views for the final two contestants, Aaron and Natalie.

As usual, the students were keen to understand the choices and motivations of the different experts, asking questions like “Why did you choose this job?”. This allowed the group to talk about how studying science had led them to where they were, and how that knowledge was valuable to them now.

By letting the children discover for themselves the specific details of each person’s job, questions about these careers naturally followed:  Do you enjoy helping and working with Japanese companies and cities?; What did you take (subjects wise) to be workin with solar power?.

Furthermore, students who completed a survey before and after the event indicated a slight increase in wanting a job that uses science skills and knowledge, although the sample size is too small to draw conclusions. We will complete a proper analysis on this, and other Science Capital related outcomes, after analysing data from multiple zones and events.

Safe to say, the zone saw heated competition.

So a zone including non-scientists works. The strength of the I’m a… format is that it harnesses the power of connecting students with real people, regardless of background. What’s next? We’d like to run more zones in the future that demonstrate the diversity of science-related careers to school students, and maybe even zones completely unrelated to science. Why not I’m a Poet, Get me out of here?

For now we’re trialing a Careers Zone with alumni from past events and it’s already been fascinating seeing some of the places former researchers now work. Since November, even the token scientist in Osmium Zone has moved to a non-academic role. Natalie now works for the Met Office, coordinating efforts to maximise the impact of research into climate change, and proving further that studying science can take you to interesting places. 

Read the Osmium Zone Report for more information about the zone

Read an interview with Keith Franklin about his experience as part of the zone

Posted on September 1, 2016 modantony in Evaluation, Event News, News | Comments Off on Osmium Zone – I’m a Scientist, not just for scientists?

Demand vs. Capacity — An update for June 2016

Any avid readers of our project blog — there must be at least one of you — will have noticed we’ve written a lot recently about our increasing demand for classes, and our over-subscription rates.

In January, we published some numbers. Since then we have run the March 2016 event, and finalised class places in the June 2016 event. Here are some updated numbers:

Graph of I'm a Scientist UK class requests by event in academic year — March 2013 to June 2016

I’m a Scientist UK class requests by event in academic year — March 2013 to June 2016

In January, we wrote about the decreasing popularity of the June events, with more teachers moving to November and March.

With the spectacular growth in November 2015, we speculated that teachers may be moving from March and June to earlier in the academic year. If that was true, then we would have expected a lower demand in March and June this year.

What we see is a steady increase in demand in the June events. The growth in March 2016 was perhaps less than may have been expected given that of previous years. This could support the idea that teachers are opting to take part in November instead of March. The narrative is not entirely clear though and we need to look into this a little more.

As was true in January, what is clear is the decreasing capacity for classes in all of the events. Though this does though coincide with an increased capacity in our other projects; with the demand for classes increasing we need to increase the capacity. We need more funding.

For the past year or so we have been charging international schools wishing to take part, generating a few hundred pounds. This June we began asking the same charge of independent schools. Overall, the response has been positive — which to be honest has been a pleasant surprise. There’s a separate post to come on that, but the fact that teachers are willing to pay shows they value the activity, and that this could be a valuable funding stream in the future.

Posted on May 23, 2016 Moderator - Josh in Capacity, Evaluation, IAS Event, News, Project News | Comments Off on Demand vs. Capacity — An update for June 2016

On student activity and simplified registrations

TL;DR: We made the site easier for students to access and the proportion of our audience engaging increased.

Like any company, we have a set of Key Performance Indicators which we use to keep an eye on how well we’re doing. One of those KPIs (professionals use abbreviations) is the percentage of active students during an event, what we will call, %AS.

%AS shows the number of students who log in to the site, and go on to — at the very least — ask a question, write a line of text in a live chat, cast a vote, or leave a comment. Basically, it shows the proportion of our audience who are actively engaging with the activity.

In July 2014 Rosie posted a message on our project management app of choice, pointing out that the %AS for the previous events had been falling to the level it was at during the project pilot.

Graph of percentage of active students per IAS UK event. June 2011 to March 2016

Graph of percentage of active students per IAS UK event. June 2011 to March 2016

So, what did we do about it?

If you looked at the graph, you’ll see that we’ve already given the game away (but this is a one-graph-blog-post, and we’re not about to pad this out with multiple views of the same graph).

We started pre-registering students.

By visiting schools to observe students taking part, we saw that asking students to create their own accounts was taking way too long, was way too complicated, and largely, unnecessary.

We completely stripped down the process students go through to first get access to the site.

Previous and updated student access process

Previous and updated student access process

Previously, students would use an “access code” to get to a registration page, where they create a username and password, give us an email address, answer some other questions including some evaluation questions on their views of STEM. Now, students are given a generic username and password which gives them instant access to the site. From there, they can choose to go in and answer the evaluation questions, create a display name, and fill in their profile. But if they choose, they can get instant access to the live chats, to the question page, to scientists’ profiles.

The moral of the story then… By observing students use the site, we learnt that the registration process was too complicated. Pre-registering accounts for students does add a little more time and admin to the running of the event than not; but effort that pays off by making the site simpler to use and access for the students taking part.

Posted on May 23, 2016 Moderator - Josh in Evaluation, Event News, IAS Event, News, Project News | Comments Off on On student activity and simplified registrations

Demand for classes

Column chart showing demand for classes outstripping supply

When we started I’m a Scientist we used our March events to allow teachers to test the event and for them to come back in June with more classes. That time after school exams when teachers looked for something to inspire kids was the ideal time.

Times have changed since then. March remains popular as teachers look for activities to coincide with British Science Week, but June is no longer the most popular time of year. We think this is due to increasing numbers of schools starting the next year’s timetable and curriculum after exams and that fallow period is no longer fallow.

November is where we saw some spectacular growth in 2015. We’re not sure what has driven this – we’ll investigate and we’ll rebalance when in the year we run zones.

The other clear narrative from this graph is that excluding the lower than expected level of requests in June 2014, demand for I’m a Scientist is increasing and outstripping demand. The past four events have been oversubscribed and March looks like going the same way as November 2015. There are some advantages in terms of zones being busy, but we need a balance.

Posted on January 12, 2016 ModShane in Capacity, Evaluation | Comments Off on Demand for classes

November 2015 – Bigger than ever

We’ve just published the latest batch of zone reports for the recent November 2015 round of I’m a Scientist. You can find them here, or at the bottom of this post, but before you dive into the pretty pie charts and wicked wordles, we felt it would be useful to provide some context to keep in mind as you read:

We had an inkling that November’s eight zones were going to be big. As we reported beforehand, due to extremely high demand from teachers, it was the first time ever that we had to turn schools away from I’m a Scientist. We also had to limit the number of classes of those taking part to one or two per zone to fit as many schools in as possible. Without doing this we would have had to run double the number of zones to cover the demand.

So, after the hype, exactly how big were the zones in November? Here are some headline figures:

  • 481 students, on average, logged in to each zone. Compare this to the historical average of 353, and a target of at least 300 per zone.
  • Students submitted an average of 956 questions per zone and half of the zones saw well over 1000. The event average is 717 questions.
  • 20 live chats per zone was the average number in November. This is also usually the maximum number we allow.
The averages for the November 2015 event and the averages over three years of I'm a Scientist UK. '% of students active' is percentage of students who logged in and used ASK, CHAT or VOTE. Asterisks denote new record averages for an event.

The averages for the November 2015 event and the averages over three years of I’m a Scientist UK. ‘% of students active’ is percentage of students who logged in and used ASK, CHAT or VOTE. Asterisks denote new record averages for an event.

While the figures above are incredible in terms of numbers of students interacting with the site, high zone numbers aren’t everything. In fact, what’s best for both students and scientists is running smaller zones and more of them.

We still had great feedback from the students, teachers and scientists in November, but the bigger a zone gets, there are potential consequences. Scientists can only do so much in two weeks and with so many live chats happening, we did see slightly less scientists per chat in some zones. There’s also less chance for students to come up with their own ASK questions that haven’t been asked before, and then receive answers made just for them. The lower proportion of approved questions to submitted questions reflects this.

We don’t want these consequences. We want every student to have their own high quality, personalised interaction with a scientist. An increased number of smaller zones would make this easier for everyone.

The situation we’re in also means we have to choose which schools take part, and in an effort to reach those who lack regular STEM engagement, we’ve started prioritising places for Widening Participation and rural schools. Ideally, however, we want all schools who register for an event to get a place in a zone.

Thanks to our current funders, and the sterling efforts of the scientists, November was a success. But if we’re near the limit of zone size now, what’s next? Luckily, the high demand for I’m a Scientist represents a huge opportunity for organisations to reach the public. Being oversubscribed means we can show there are hundreds of schools ready to engage with hundreds of keen scientists. All that’s needed is the funding for the zones to let them get on with it, and we’re going to try our best to make that happen.

November 2015 Reports:

Posted on December 17, 2015 modantony in Capacity, Evaluation, Event News, News | Comments Off on November 2015 – Bigger than ever

3 words to describe a scientist

Survey results from a Year 6 student - Swanmead Community School

Survey results from a Year 6 student – Swanmead Community School

In March 2014 we started running zones for primary students only, increasing the number of primary school students who take part in I’m a Scientist. And we wanted to know more about how the event affects them.

This time, we drifted away from our usual online environment and sent printed surveys to teachers taking part in the Colour Zone and the Thulium Zone, the two primary only zones we ran in June 2015.

We also wanted to check if the response rate to printed surveys was higher than the response rate to digital ones, and it is. We got pre and post event results from roughly 40% of the students who took part in the two primary zones. Usually around 10% of students who take part in the events fill in both the pre and post event digital surveys. Analysis and interpretation of online surveys is quicker and more efficient that printed ones, but getting teachers involved in the evaluation process definitely made a difference.

We asked students which 3 words they would use to describe a scientist, before and after taking part. We got responses from over 300 students from 9 different schools.

  • The students used more that 230 different words to describe scientists. They used 188 different words before taking part and 127 after, maybe indicating that they are clearer about what a scientist is after taking part. Awesome, cooperative, inquisitive, life-saving… are just some examples of the words they used.
  • ‘Intelligent’, ‘clever’ and ‘smart’ were the most popular describers, although students used them less often after taking part – 46% of students used at least one of these words pre-event compared with 40% post-event.
  • Students described scientists as ‘awesome’, ‘epic’, and ‘cool’ twice as many times after taking part in I’m a Scientist– 9.7% of students used these words post-event compared with 4.4% pre-event.

We grouped words in synonym clusters, and we represented the relative pre to post-event difference on word use by gender:

new graphOnly words that were used at least 10 times are represented.

From the graph you can see that girls and boys equally use less negative stereotype words like ‘crazy’ after taking part.  Emotional descriptors – like ‘fun’ or ‘exciting’ – are mentioned more frequently after the event. You can also see that boys are more likely to use the word ‘fun’ after the event than girls. However, girls were more likely to use the word ‘awesome’ after taking part.

What do you think? Is there anything that particularly calls your attention? We asked a primary schools teacher and she told us:

I’m not surprised by the ‘awesome’ leap – I don’t think children really understand what is involved in the daily work of scientists until they interact on I’m a Scientist. My own students have been blown away by talk of live sheep spines, looking at volcanoes in space, etc.” – Tracy Tyrrell

If you are a primary school teacher and you want to evaluate your students’ attitudes to science before and after doing I’m a Scientist, you can download PDF version of them here:

If you are an academic (or, really anyone!) and you are keen to see and analyse the raw data of these printed surveys, please let us know in the comments and we’ll share them with you.

Please leave a comment, let’s continue the dialogue.

Special thanks to Dr Jessica Hamer, for her advice on the the printed surveys design.

Posted on November 2, 2015 modangela in Evaluation | Comments Off on 3 words to describe a scientist

Which browsers and devices are schools using?

Every year (see 2014) we take all our data, and look at how schools use the site; looking at how visitors access the site and how much that has changed in the past couple of years.

We’ve made a lot of changes in recent events, especially when it comes to registering accounts and using the site on mobile devices. We now pre-register all teachers and students, giving them usernames and passwords, so they can start asking questions immediately.

We can assume that the student visitors give a fair reflection of the general school IT facilities and system capabilities.

The graphs show the student data represented by dashed lines.

Device

We saw last year, that visits from mobile and tablet devices were increasing. and no surprise they continue to grow, while desktop usage drops slightly. For student tablet users there is little difference in content visited when compared to student desktop users.

We’re currently right in the middle of an overhaul to make the site more responsive and mobile friendly- which should make the user experience a lot smoother for all users. Allowing students who use tablets and mobiles the same experience as desktop users.

Browser

If desktop usage is going down somewhat, it’s unsurprising that Internet Explorer is also sinking,  with Chrome taking a the lead and Safari jumping up (a lot of tablets and mobile devices being Apple products, where the default browser is Safari). The rise in other browsers will be down to mobile and tablet browsers using Android, Blackberry and Opera.

IE versions

And finally the bane of web designers and developers lives.. The old versions of Internet Explorer. A collective sigh of relief, IE6 has finally drifted out of favour (available since 2003!!!!), IE7 is on the out, IE8 is dropping, as is IE9, even IE10 is dropping.. But IE11 has taken a big leap. Understandably when a new version is released, it will lead to older versions not being used, but the jump in the graph is still quite dramatic, compared to last years.

Posted on October 30, 2015 modemily in Evaluation, School | Comments Off on Which browsers and devices are schools using?

Research vs Evaluation

Science Learning+ is a significant funding scheme provided jointly between the Wellcome Trust and the National Science Foundation.

Learning can happen anywhere and at any time. Science Learning+ is an international initiative that aims to understand the power of informal learning experiences inside and outside of school.

The second aim of the scheme is to

“bridge the practice and research gap”

At a seminar in July aimed at providing an update on the Phase I project an interesting conversation developed about that gap between Science Communication practitioners and researchers.

I heard one speaker talk about practitioners wanting to know if a hypothetical red headline would give a 3% uplift in visitors. I responded on twitter:

Not all practioners agreed with me. Some felt each project would be unique enough to warrent a rewriting of expectations

Others simply disagreed and place efficacy as something for researchers:

In the end 140 characters felt underpowered.

For me research and evaluation are different, but very related.

I expect research to tell me if an approach to science communication works and how it works. I expect evaluation to tell me how well a project is working and how it can be improved. I would like evaluation to draw upon the research to extrapolate that particular  activities will lead to particular outcomes.

For example using I’m a Scientist:

The feedback we get from our participants is that connecting online with scientists improves their attitude to science, and to jobs in science. We seem to find the changes in attitude among girls is greater than it is for boys.

I want some research to tell me why those conversations are improving attitudes and if those changes are persistent. I want the research to be telling me how online activity compares to offline activity and why.

I want research to tell me what characteristics of engagement deliver the best and most persistent improvements in attitude and achievement.

Then I want my evaluation to examine our work against those characteristics and to suggest ways to improve them.

Research  = why something works

Evaluation = how well something works

What do you think?

Posted on August 27, 2015 ModShane in Evaluation, Science Engagement | Comments Off on Research vs Evaluation

I’m a Scientist

‘I’m a Scientist, Get me out of here’ is an online event where school students get to meet and interact with scientists. It’s an X Factor-style competition between scientists, where the students are the judges. Students challenge the scientists over intense, fast-paced, online live CHATs. Then they ASK the scientists all the questions they want to, and VOTE for their favourite scientist to win a prize of £500 to communicate their work with the public.

Keep Me Updated

Subscribe to our mailing list

* indicates required
I'd like information for
How we use your information* Subscribing to this list means we will email you with news about our projects. You can unsubscribe at any time by clicking the Unsubscribe link at the bottom of our emails. For more information, take a look at our Privacy Policy.

News Categories