How we used your feedback to improve the SELFIE tool

From the outset, school leaders, teachers and students from all over Europe have been involved in creating and testing SELFIE. 5,000 users provided their input on a very early design of the tool. 650 primary, secondary and vocational schools (from 11 EU Member States, Serbia, Georgia and Russia) then tested a beta version of SELFIE.  

Through feedback questions (‘What worked well?’, ‘What worked less well?’, ‘What was confusing?’, ‘Was the tool easy to use?’), we collected 12,600 comments in 12 different languages from 67,714 school leaders, teachers and students. The SELFIE team then translated these comments into English and analysed them using a special piece of software for text analysis.

The results show that school leaders, teachers and students value the tool and feel that it is useful, interesting, easy to use and informative. Students also appreciate the tool, which allows them to express, in an anonymous way, their opinion on how digital technologies are used at their school. The test schools gave valuable feedback on how we could further improve the tool before launching the full version. 

"I like this tool because it is easy to use and very practical and I can express my opinion easily and without being judged".

Based on feedback and suggestions provided by users, the following changes were made prior to the full launch in October 2018:

  • Shorter registration process: makes it easier and faster for school coordinators to register their schools;
  • Simplified and standardised answer scale: almost all questions in SELFIE can be answered using the same 5-point scale; 
  • Explanatory text for each question: to make it easier for respondents to understand the answer scale, we added a description of what the higher score means;
  • ´Not Applicable´ and ´Prefer not to say´ answer options added: to be used when users are not sure how to answer, the question doesn't apply to them, or in case they simply don't want to answer the question!
  • Alert when questions are skipped: Users can answer the questions in the order they like, but they are informed which questions have not yet been answered;
  • Clearer wording of the questions: Many users told us that the SELFIE tool was easy to use, but that they found some questions difficult to understand. We carefully reviewed the questionnaire and removed some expressions which users told us were too hard to understand or too abstract. For example, in the students' questionnaire, we replaced the question "I learn how to make correct references to the digital resources I use in my assignments" with "I learn how to use others' work I have found online”.
  • Simplified questions for primary schools: Many teachers and school leaders felt that the student questionnaire was quite advanced for primary school students. We asked a team member who has experience in designing questionnaires for different age groups to test it - read more about the test here.  As a result, we reduced the number of primary student questions from 29 to 17. We now also recommend to schools that only students aged nine or over should take part in the self-reflection exercise. 

"Perhaps it would be advisable to use simpler and more general terms for people with less computer knowledge".

These are just a few examples of how we have improved the SELFIE tool by taking user feedback into account. We analyse user feedback on a continuous basis, so stay tuned for new features and improvements to SELFIE!

About the author: Arianna Sala has a PhD in Psychology and is a researcher in the field of education and gender. Arianna helps to improve SELFIE by analysing feedback provided by users.