In an average year, Dubit’s research team visits scores of homes, welcomes hundreds of children to our Leeds PlayLab, and meets with families in research facilities worldwide. We sit on rugs, at tables and sometimes in tiny chairs to talk directly with young people. Sometimes we’re testing a new game or toy, sometimes we watch video, and often we ask kids to show us their favorite things, to gain insight into their lives and passions. Obviously, since the spread of COVID-19, those face-to-face interactions aren’t possible.
Like so many other offices around the world, Dubit’s development studio and research teams have transitioned to home-working. Still, we’d already contracted for multiple qualitative research projects to take place over Spring and Summer of 2020. So, we needed to revisit and adapt our proposed methodologies to carry out each project from the safety of our homes, whilst still meeting the objectives.
Pandemic or no pandemic, there is huge potential in thoughtful use of online qualitative research – we can engage diverse participants anywhere in the world without the time and expense of travel, and the children are most comfortable participating from their own homes. Moderators can frame the product or service being studied in the context of participants’ own lives, something that is difficult or outweighed by other factors when operating at a research facility.
Different qualitative studies require different methodologies, however, and all come with unique requirements and obstacles to consider. Ensuring validity of findings, recommendations and implications depends largely on the use of platforms that meet those particular requirements. Over the past few weeks, we’ve spent time considering which platforms are best suited to help us reach our goals across different types of research.
While we have been adapting to the new ‘normal’, we thought we would share with you the solutions we explored, predominantly centered around content testing and user interviews, that could not only act as a comparable alternative for face-to-face research during this unprecedented time, but also offer new methodological options for the long term, in a world where connecting virtually has never been so familiar to our audiences.
Lookback is a web-based platform allowing researchers to observe users navigating through a site or app in real time, with options for conducting interviews live or sending tasks for participants to complete independently, in their own time.
- The video interview captures and records participants’ screens, including a track of where they touch;
- Self-testing allows time-efficient simultaneous collection of information from multiple participants at once;
- During live testing, you can jot down time-stamped notes within the platform, useful during content testing, for example, to note points where participants respond positively, negatively or become distracted;
- Easy video editing, making it simple to add clips to reports.
- Lookback’s basic dashboard doesn’t offer the option to create a calendar or schedule for larger projects with multiple groups, or multiple projects simultaneously;
- The platform has limited capacity for adding stimulus, and allowing participants to edit or interact with the stimulus;
- There is no chat function, useful if the participant is struggling with the technology (e.g., webcam or microphone) - reverting to email to solve these problems is at best tricky, and impossible if your contact with the participant is only through an external recruiter;
- Lookback is used with one person at a time, so not suitable for conducting focus groups.
We think Lookback is best suited to usability testing of apps, websites or prototypes, and not to focus groups, content testing or co-creation.
itracks is a purpose-built online platform for qualitative research. It offers phone and video IDIs, chats, discussion and community boards for carrying out a range of different research methodologies.
- The platform is comprehensive, offering multiple different features and tools including scheduling of groups, uploading of stimulus, polls and screen-sharing;
- Participants can review images, video or website stimuli on the ‘whiteboard’, increasing the amount and format you can collect data in;
- The platform is built for collaboration before and after the session, great for digging deeper into what has been shared - a step further than standard interview methodologies;
- The range of tools makes it adaptable and suitable for multiple different methodologies. If you’re looking for a solution for the long-term, this type of platform offers flexibility when considering the range of requirements that come with different research projects.
- The platform is complex and described as being suitable for participants who are ‘tech savvy.’
iTracks would be best suited to focus groups and in depth interviews with participants who are already comfortable and confident when using technology or participating in online research, to make use of all their tools and features.
VisionsLive is another purpose built, online qual platform with options for individual video IDIs, focus groups, online bulletin boards, each of which are built with a range of tools to help engage participants and gather rich data.
- The platform is reliable and stable, without technology and admin hurdles, permitting focus on engaging the participants
- Connections can be tested in advance of the session;
- Virtual whiteboards filled with stimuli (video, images and audio) are created in advance, and so instantly ready to share with the subjects;
- The virtual whiteboard comes with a toolkit enabling participants to use pens and stamps to show – not just tell – what they think;
- Dual chat features – one for the participant(s) and moderator(s) and one for the moderator(s) and observer(s) – allow observers to contribute to the session in real time;
- The platform comes with ongoing support, including during sessions, in case of issues with the technology.
We found this platform useful for content testing and can also see potential for conducting focus groups, ethnographic research or co-creation.
During the pandemic, Zoom has become the go-to conferencing platform for meetings, webinars and more.
- The platform is now familiar to a wide range of people of all ages, minimizing set-up or training time and troubleshooting.
- Unlike face to face research, where a child can chat with others in the room while watching a video, Zoom mutes audio streams when it detects simultaneous activity;
- The platform lacks additional, useful features like whiteboards and interactive functions (e.g., polls).
Zoom is best suited to straightforward methodologies, such as in-depth interviews with adults to gather insights around more directed and structured research objectives, or to accompany discussion guides.
The Clickroom is an interactive 3D focus group room where your participants are represented as avatars. It allows you to chat in real time with your participants, one on one or as a group.
- With no webcam or microphone, the Clickroom feels less intrusive, making it perfect for talking with groups who may be self-conscious or who are discussing a sensitive topic;
- Avatars give each participant a personal, physical presence in the room, whereas participants in other chat-based focus groups appear as just lines of text;
- The Clickroom offers anonymity, which encourages all participants to engage openly and safely;
- Clients can watch and contribute to the direction of the session, in real time;
- The Clickroom automatically transcribes the session, producing a spreadsheet ready to download instantly afterward;
- Customization of rooms and characters give the platform a personal, fun touch.
- As it relies on text chat, the platform is less suitable for younger children than a video chat or face-to-face research.
The Clickroom is most suited to focus groups, especially with teenage or adult participants discussing sensitive topics.
There are situations in which face-to-face and online qualitative procedures cannot be compared like-for-like, or where there is irreplaceable strength in conducting research in person. There are challenges and bumps in the road, some we could forecast and some we didn’t see coming.
Whichever methodology you choose, it is critical to trial, test, and throw all possible problems at a platform to see how well it can cope, not just before the session but before proposing your ideas to clients.
It’s unlikely that there is a “one platform fits all” solution. Above, we’ve reviewed methodologies we’ve implemented because they suit our current research projects. We also conduct frequent UX and usability testing, however, which may demand different measures for a platform’s suitability.
We also need to consider why we usually see our participants in person, and the adaptations we need to make as moderators when not face-to-face while conducting research. Participants and moderators have different perspectives on what is important or interesting, and most online qual methodologies and platforms naturally pass control to the subject.
Right now, safety and continuity are the primary reasons for conducting “virtual” qualitative research, but there are other, undeniable cost and time benefits that will last beyond 2020 and the pandemic. Necessity being the mother of invention, we will learn to adapt our techniques to get the best of both worlds.