Remote qualitative data collection: Lessons from a multi-country qualitative evaluation

By Mehjabeen Jagmag *

Like most researchers who had planned to begin their research projects earlier this year, our research team found our data collection plans upended by the pandemic. We had designed our research guides, received ethical clearance and completed training our research teams for a multi-country endline evaluation of an education programme in Ghana, Kenya and Nigeria much before we heard of the COVID-19 pandemic.

A few days before our teams travelled to their respective data collection sites, phone calls started pouring in – schools were closed indefinitely, travel between cities was restricted, and we were beginning to understand how much the COVID-19 pandemic would change our lives. After a few weeks of waiting and watching, it became apparent that we could not continue in-person data collection.

We revised our research guides and prepared ourselves for conducting remote phone-interviews with our research participants. Given that this was the third and last round of data collection in our multi-year panel research, we had previously collected phone numbers of our research participants and acquired permission to be able to contact them on the phone for further research. We set up remote research desks for the team and began preparation for data collection.

What we were unsure about was whether our research plans would be successful. Accounts of fraudulent callers promising medical remedies and peddling fake health insurance packages had made people wary of responding to unknown phone numbers. We were not sure how many of the phone numbers we had collected in the previous year would still be working, and most importantly, we were not sure how our research participants were faring under the lockdown and whether they would want to speak with us. Finally, our research participants included primary school students, who were an essential voice in our study. We were keen to conduct interviews but were not sure if this would be feasible – would parents trust us enough to speak to us and consent to their children speaking to us? Once we secured consent from parents, would children provide assent? As trust was the key element to completing our research successfully, we devised a data collection plan that included the following elements, that are likely to be necessary for future remote data collection.

Training and retraining for remote data collection

We spent time discussing as a team what the potential challenges may be and how we plan to respond to them. We drew up a collective list of answers that we could draw on to communicate clearly and effectively about the evaluation, respond to any queries and alleviate any concerns that our participants had. This list and knowledge grew, and we collected data, and debrief meetings with the teams at the end of each data helped ensure this was a live document.

Seek feedback from key informants

We contacted community leaders and headteachers to enquire about how we should approach data collection with school and community participants. They provided important contextual information that was occasionally specific to each community. We used this information to improve our introductory messages, the time and dates we called and how we approached research participants.

Seek introductions from trusted leaders

We also asked community leaders and headteachers to support our recruitment process by sending messages to the participants about our research before it began. Doing so helped minimise any uncertainty of the veracity of our calls. Where relevant, we compensated them for their airtime.

Give participants time to prepare for the interview

We shared information about our organisation and the research objective over text messages or calls, which gave research participants enough time to decide whether they wanted to participate. It also helped plan to speak at a time would suit them best for a discussion, and also consult with their family and children if they wanted to participate in the research.

Ensure continuity of research teams

As this was an endline evaluation, we had research team members who participated in previous rounds of data collection calling the participants they were likely to have visited in the past. Where this was possible, it increased trust and facilitated easy conversations.

Prepare case-history notes

We prepared short case history notes about the programme and school and what we had learned from previous research rounds for each school to build confidence that our intentions and credentials were genuine. These notes helped remind research participants of our last conversation, helped us focus on what has changed since that last conversation, which in turn helped keep conversations short and in general proved to be a useful conversation starter.

Save time at the beginning and end for questions

We ensure research participants had enough time to ask us about the programme, our motivations, go over the consent form, understand why we wanted to speak with the children or for children to ask parents for their permission before we began our interviews. To ensure that that the conversation did not feel rushed, we designed shorter research guides.

Plan for breaks or changes when interviewing with young participants

When speaking with students, we anticipated time to break and distractions during the call, which helped maintain a relaxed pace during the interview. If students were uncomfortable with phone interviews, we, eased the conversation to a close to minimise any distress caused to the participant.

Summary and Conclusion

We completed data collection in all three countries, albeit with a less ambitious research plan that we originally intended for an in-person research study. The key objective of our research was to collect the optimal amount of data that would inform the programme evaluation while making the interview process convenient and comfortable for the research participants involved. To do so, we have learned that it is vital for participants to have confidence in the researchers and the motive for collecting data. Planning before we began data collection and updating our knowledge as the research progressed proved invaluable to our experience.

* Mehjabeen Jagmag is a Senior Consultant with Oxford Policy Management.

Teaching online research methods online with asynchronous international distance learning students during Covid-19

By Elizabeth Hidson and Vikki Wynn

Challenges in asynchronous international distance learning pre-Covid

Working on an international distance learning teacher training programme brings multiple challenges, the biggest of which had previously been the asynchronous pattern of teaching and learning for the academic elements. Teaching is based on a systematic instructional design approach adopted by our university and broken down into weekly thematic units to support acquisition, discussion, investigation, collaboration, practice and production to meet learning outcomes. Recorded micro-lectures, learning activities and discussion boards are accessed asynchronously, with face-to-face online group sessions for further consolidation. The assessed teaching practice element of the programme had always been carried out in the host international schools, facilitated by school-based mentors and in-country professional practice tutors.

Developing research-informed practitioners

The importance of developing research capacity in trainee teachers stems from the expectation that they will become research-informed practitioners who can use evidence to inform decision-making (Siddiqui and Wardle, 2020). Being consumers of research is not enough, however: teachers need to also develop the tools to carry out their own research in school settings. The first MA-level module that our trainees encounter requires a case study approach to explore specific interventions that their schools implement to address targeted pupils’ learning needs. Typically, our trainee teachers undertake observations, conduct interviews and collect a range of data in their settings to understand how and why this additional support is provided and discuss it in relation to ‘what works’ in education, using initial sources such as the Education Endowment Foundation and the What Works Clearinghouse portals.

Establishing the heritage of research methods and methodology

Good teaching is good teaching, and it follows therefore that good research practice is still good research practice, irrespective of a global pandemic. Early rapid evidence assessments concluded that teaching quality was more important for remote teaching and learning than how it was delivered (Education Endowment Foundation, 2020), which had also been our starting point when considering our own research methods pedagogy. The initial teaching of research methods starts on our programme with key concepts and expectations: conceptualisation, literature, developing research questions, justification of research methods, consideration of ethics, all designed to ensure that the student teacher can apply theory to practice. We start with a formative proposal assignment to ensure early engagement with methodology and methods.

Our face-to-face online group sessions, themed as weekly ‘coffee shop’ meetings, provide a collaborative forum for knowledge exchange and trouble-shooting. Trainee teachers join to listen, to share ideas, to pose questions and problems and the module leaders respond with a dialogic teaching approach, helping to contextualise research methods in school settings and develop knowledge and understanding in a supportive online space.

Elizabeth Hidson promoting the weekly ‘coffee shop’ meeting

The ‘hybrid’ assignment and hybrid research methods

As teaching practice became hybrid for trainee teachers, so did research and assessment. Schooling around the world moved in and out of face-to-face, hybrid and fully online modes over the course of 2019, with the realities of the pandemic hitting earliest in the far east, where half of our students are based. As physical access to schools and participants fluctuated with local restrictions and impacted on students’ research plans, our alternative assignment pathways opened out to include hybrid and hypothetical assignments designed to act as a safety net for completion.

A key feature of the hybrid assignment was the shift to online and alternative research methods, building on the core research methods pedagogy we had established. Where face-to-face interviews were not an option, we promoted video calling and desktop-sharing (Hidson, 2020), but maintaining the spirit of semi-structured or artefact-based interviewing. Where classroom observations were no longer possible, we promoted fieldnotes captured from hybrid or online teaching sessions, urging a re-think of ethics and collection of additional secondary data in various forms to attempt triangulation.

The outcomes in terms of the final case studies produced have been pleasing: creative and thoughtful academic discussions that responded to the unique challenges of each setting. We regularly quoted Hamilton and Corbett-Whittier (2013) to our trainees, where they advised thinking of a case study as a living thing and ensuring that it made “as much sense to the reader as it did to the researcher” (p.179). The act of thinking in detail about the research methods seemed to have been beneficial to the understanding of research methods and real-world research.

Developing resilient research capability as a factor of resilient teaching

Although our programme continues to respond to the global challenges of Covid-19, we are keen to retain what has worked into the future. The ability for trainee teachers to embrace the need for resilience in teaching as well as in research is a benefit. Their capacity to see research as a live and responsive part of their practice has always been our intention; we believe that the response to research during Covid will itself be a case study for future cohorts.

References

Education Endowment Foundation (2020). Remote Learning, Rapid Evidence Assessment. London: Education Endowment Foundation.

Hamilton, L., and Corbett-Whittier, C. (2013). Using Case Study in Education Research. London: Sage.

Hidson, E (2020) Internet Video Calling and Desktop Sharing (VCDS)as an Emerging Research Method for Exploring Pedagogical Reasoning in Lesson Planning. Video Journal of Education and Pedagogy, 5 (1). pp. 1-14. https://doi.org/10.1163/23644583-00501001.

Siddiqui, N. and Wardle, L (2020). Can users judge what is ‘promising’ evidence in education? Research Intelligence 144 (Autumn 2020). London: BERA.

Qualitative health research beyond and alongside COVID-19

By Sue Chowdhry, Emily Ross and Julia Swallow

As qualitative researchers in academia, like many others our practice has been transformed in light of the global Coronavirus pandemic. The ‘lockdowns’ enforced across the world have introduced greater awareness of our proximity to others in everyday life, and of the need to maintain a prescribed distance between bodies. This has implications for our work as researchers in the field of health and illness, whose tools include face-to-face methods such as focus groups, interviews and ethnography.

In this blog post, we reflect on the meaning and implications of doing qualitative health research beyond and alongside COVID-19. Drawing on examples from our individual research projects, we first focus on who and what might be excluded or silenced through the changes to our research environments and practices prompted by the pandemic. We then reflect on several implications of the ruptures caused by the pandemic for qualitative research in health more widely.

Exclusions and silences

Research interactions

As researchers in medical sociology and science and technology studies, we had been undertaking separate projects at the time of the pandemic. Sue’s research concerned pregnant women with experience of pre-term birth, and Emily and Julia’s considered patient and practitioner engagement with novel cancer treatments (genomic techniques and immunotherapies respectively). All three of our projects thus involved individuals classified as especially ‘vulnerable’ to COVID-19 by the UK Government (see Ganguli-Mitra’s opinion piece for a wider discussion of the classification of ‘vulnerability’ as related to COVID-19). As a result, in addition to the restrictions imposed by Institutional and NHS bodies on research practice, we were particularly mindful of the potential consequences of face-to-face methods for our participants.

The prospect of continuing our research in the absence of physical proximity to our participants was daunting. Viewing interactions between researcher and participant as sites for the active co-creation of qualitative data, we were concerned that the inability to conduct research encounters in person, and loss of the intersubjective encounter, could be detrimental to our practice. As Alondra Nelson’s blog post on this issue points out, valuable research insights can be gained from being close enough to observe gestures such as “toes tapping and nervous hands”. Sue interpreted her physical presence as key to the success of focus groups she had conducted prior to COVID-19 restrictions. For example, her occupation of the physical space was performed so as to signal to participants that they controlled the discussion. Equally, participants orientated their bodies to each other in ways that indicated interest and support, through spontaneous shared laughter, eye contact and sometimes fleeting touches of hands at emotional junctures.

We have also been reflecting on the implications of a move away from face-to-face methods for relations of power within qualitative research practice. Our research projects have often focused on life-events that can be distressing and emotional for participants, and throughout we have all maintained a commitment to democratising the research process. We have endeavoured to forge reciprocal relationships with participants, and adopted forms of practice that more equitably distribute control whilst qualitative interviewing. Reflection on the issue of power in research is significant for those turning to online methods in light of the pandemic, particularly where online material pre-exists the research encounter. Here the intimacy of face-to-face methods, which feminist scholars have claimed better allow for the involvement of participants in the production of knowledge (Ramazanoğlu and Holland, 2002), is absent. Having used pre-existing online material in previous projects, Emily felt that the creativeness of qualitative research practice as a shared project between researcher and participant was not as achievable in online research, nor was the closeness that comes from being a key participant in the creation of qualitative data. As such, those adopting online methods in light of the pandemic may try to re-craft participant involvement and reciprocity in other ways. This may be through initiating contact with authors of online material, sharing information about the research with them, and if appropriate seeking consent from authors to use online posts in research.

Research landscapes and spaces

The concerns discussed above are further situated within the landscapes and spaces in which qualitative research takes place. In the example of focus groups, the ‘affective atmospheres’ (Anderson, 2009) shaping the research encounter provided the backdrop for Sue and her participants’ responses to the research and each other. Sue offered refreshments to her focus group participants, and vividly recalls the smells, tastes and sounds of this shared experience. The atmosphere was carefully fashioned for respondents to feel valued and at ease, and to allow for the exchange of intimate reflections on experience.

The arrangements of care provision, and situated contexts in which care is given and received, shape patient, clinician and researcher accounts of disease and treatment. In the time of COVID-19, the research spaces with which we as health researchers had been familiar are being re-shaped, with this particularly visible in cancer care. Before the pandemic, the settings for Julia’s ethnographic research were already stratified and fragmented, with consequences for the practice of healthcare and patients’ biosocial experiences of cancer. Novel immunotherapies could not be accessed by all, raising questions around the inclusions, exclusions and silences provoked by these therapies – who has access and who benefits? COVID-19 is potentially (re)producing or exacerbating existing inequities. As researchers, who we are able to observe and engage in our projects is a key concern, as we ask what and whose realities, experiences and practices might be privileged over others in the context of contemporary cancer care, and in relation to the healthcare worlds (re)shaped by COVID-19.

Responding and intervening

Although the current situation has prompted us to halt or reformulate our ongoing research, in our experience the need to reflect on and adapt our methodologies has also provided opportunities. Importantly, recognising and responding to the methodological restrictions prompted by the pandemic has encouraged us to think about the inclusions, exclusions and silences that already exist in healthcare worlds, which have been exacerbated or magnified by COVID-19. Attention to these issues through an alternative lens has prompted us to question how we can use method to respond and intervene. Method as practice is a means of understandingrather than organising for complexity and uncertainty, and a way to respond to the disruptions, inclusions/exclusions and silences which are rendered visible and exacerbated by COVID-19. Method produces particular realities and as such, drawing on feminist STS scholars, we have the opportunity to intervene, and to do what Alondra Nelson describes as ‘creating knowledge pathways to a better world’. If methods shape how and what we know and are always political (Annemarie Mol (1999) would describe this as ‘ontological politics’) – what kinds of social realities do we want to create or bring into being?

Online methods afford possibilities for responding to the contemporary challenges we face as researchers. Qualitative analysis of pre-existing blog posts, solicited online diaries and other methods helpfully detailed by Deborah Lupton and colleagues allows us to continue research projects disrupted by the pandemic. Further, online spaces present opportunities to intervene; to engage with those typically excluded from qualitative research due to geographical location or accessibility – with this even more pronounced in a time of ‘shielding’ those deemed most vulnerable. Online approaches can capture forms of networking and support-seeking around experiences of ill health which have been obscured by the pandemic, but which continue to be shaped by inequalities in access and survival.

As another approach, the benefit of doing ethnography, however limited this might be and whatever this might look like in the future, is that it is about opening space for complexity and uncertainty. It allows us to acknowledge and respond to the messiness of practice as an attempt to understand, rather than organise, the uneven and unpredictable ways in which knowledge is produced in research (Law, 2004). It is about taking the world as it is,whilst also keeping in mind the importance of doing what Donna Haraway would describe as critical, political, partial and situated work which is always on-going.

Reflecting

In an academic environment which emphasises activity and impact, COVID-19 has forced upon us ‘space to breathe’ (Will, 2020). The restrictions imposed by our governments and institutions have demanded an additional layer of reflexivity as we contemplate our research projects in light of the pandemic. In some cases, this has entailed the adaptation of research questions, as well as consideration of how alternative methods align with our wider research paradigms. With restrictions in our ability to engage in face-to-face research methods, we lose key aspects of the relational qualitative research encounter, and are pulled away from the research atmospheres, landscapes and spaces with which we are familiar. However, the loss of face-to-face methods has provided us with an unexpected opportunity to explore new approaches, encouraged tough reflection on our research questions and methodologies, and prompted deeper contemplation of the worth of our research itself.

The authors of this post are supported by the Wellcome Trust (grants 104831/Z/14/Z and 218145/Z/19/Z) and NIHR (grant 17/22/02).

Anderson, B. (2009) Affective atmospheres. Emotion, Space and Society. 2, pp.77-81. doi:10.1016/j.emospa.2009.08.005

Haraway, D. (1988) Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective. Feminist Studies, Vol. 14, No. 3, pp. 575-599.

Law, J. (2004) After Method: Mess in social science research. London: Routledge.

Mol, A. (1999), Ontological politics. A word and some questions. The Sociological Review, 47: 74-89. doi:10.1111/j.1467-954X.1999.tb03483.x

Ramazanoğlu, C and Holland, J (2002) Feminist methodology: challenges and choices. London: Sage.

Will, C.M. (2020), ‘And breathe…’? The sociology of health and illness in COVID ‐19 time. Sociology of Health & Illness, 42: 967-971. doi:10.1111/1467-9566.13110

Exploring unplanned data sites for observational research during the pandemic lockdown

By Areej Jamal (UCL Social Research Institute)

I was getting my PhD upgrade and preparing for my fieldwork soon after, when the Covid-19 outbreak struck. Due to the border closures, I was stuck in the country where I was undertaking my PhD and my fieldwork was due to start overseas which is also my home. I could not travel back home for months.

Not sure of when the borders would open again and anxious about starting my fieldwork, I had to adapt to the new circumstances and utilise my research time more efficiently. Since my research employs a mixed methods design using online surveys and qualitative interviews, I did not have to pivot much of the original design. However, I also had an observatory aspect in my proposed design. Being an insider to the community I am researching, I was looking forward to attending some social meetings to observe and gather the less apparent insights during my fieldwork time. But since the social distancing protocols of Covid-19 seemed long term, I had to reconsider the observation element. In this blog, I will reflect on how I took the changing circumstances in my stride and tried to gather insights from unplanned data collection sites.

Reframing the methods design

My research investigates lived experiences of long-term migrants and how they make sense of their identity and belonging in a country where they have no pathways to citizenship. Due to the Covid-19 crisis I had to reframe a few ways of collecting data ensuring to keep the essence and relevance of my research questions. Though I could carry out the online surveys as originally planned, I had to switch the in-depth interviews to online interviews. The ethical and methodological amendments arising out of online interviews were submitted to the ethics committee for review.

Being stranded abroad in lockdown, I started exploring and observing the online content as sites for data collection. The unobtrusive observation of online content became a very significant part of my revised research design. The two main online sources where I found salient insights were the online videos (vlogs) on YouTube where some migrants had been documenting almost every aspect of their lives and the other was online support groups where mostly distressed migrants were interacting. The latter was a chance discovery as I, myself was a part of these groups seeking guidance and information to return home.

Unobtrusive observation implies that a researcher observes and collects data from online sources such as websites; social media sites or discussion forums without necessarily interacting with the participants. I was passively observing the interactions taking place on these online platforms.

Making sense of the data and meanings emerging from the observation sites

Robinson (2016) and Seale et.al (2010) posit that unsolicited narratives provide richness of new knowledge that is often lacking in solicited accounts. Since the researched community is unaware of the ongoing observation, they tend to offer genuine and certain interpretations of life under specific circumstances. And so, the narrator controls the content without the researcher’s interference.

The migrant stories I had been observing from the selective YouTube accounts and the online support groups that I had joined on WhatsApp particularly offered me very significant source of information and insights. The self-reflections and opinions the observed groups expressed about their temporal migrant status and the impact it has had on some of their life decisions and which was further exacerbated by the pandemic gave me context and ideas relevant for my research questions. The narrator driven stories uncovered aspects of life, which probably would not have occurred in any of my other methods. Seale et.al (2010) suggests that since the online interactions occur in real time, they offer some sort of immediacy which is often lacking in methods where participants mostly reflect and reconstruct past occurrences.

The new knowledge emerging out of these online sources helped me draft some of the initial themes and areas to further investigate through the other methods.

Methodological and Ethical considerations

There are various ethical debates surrounding the unobtrusive research in existing literature and ways of examining personal narratives produced by individuals through online medium.  Some researchers (Eysenbach and Wyatt,2002, Seale et.al, 2010) argue that since most of the online content is publicly available aimed at general audience, the need for informed consent is ambiguous. However, in case of online support groups as Barker (2008) and O’Brien and Clark (2011) discuss the limitations of private content due to smaller number of groups members. The support groups I had been a member of, had admins and certain privacy protocols that all group members had to abide by.

The positionality of the researcher is a very important aspect throughout the research process. Salmons (2012) E-interviews Research Framework offers useful tips reflecting on some crucial questions of self-reflexivity when undertaking unobtrusive observations of the online content.

Matters of confidentiality and seeking consent for data generated from online resources needs much deliberation and largely depends on the objectives of the researcher and the ways they aim to present and report the findings. Although I am still exploring ways of representing information from these valuable sites of information, I find the idea of fabrication approach by Annette Markham (2012) quite useful, which implies ‘involving creative, bricolage-style transfiguration of original data into composite accounts or representational interactions’ (2012, p334) without divulging any specific details of the researched community.

Conclusions

Sometimes the most obvious data sites would not be as apparent unless faced with unexpected circumstances restraining our methodological choices. Agility is an intrinsic characteristic of most social research. At present, data collection is in constant flux responding to the unprecedented crisis. And now more than ever, the relentless pandemic situation offers a critical window for researchers to make every effort to explore creative and novel approaches of data collection and innovative ways to tap the potential of the existing methods.

Call for Contributions: Adapting Conventional Research Methods for the New Normal

In search of Novel Adaptations in Social Science Research Methods for the World of COVID

With a second wave of the global COVID-19 pandemic now sweeping across the West, it is becoming clear that returning to ‘normal‘ maybe a long way off yet. Though, the need for social research is now more pressing than ever. The International Journal of Social Research Methodology is inviting researchers, academics, and doctoral students to share blog contributions reflecting their experiences in adapting existing research methods to meet the needs of our new research paradigm. The goal is to help the social science research community find inspiration and learn from one other as we continue to adapt our methods to meet contemporary needs. Contributions reflecting all aspects of social science research, including research ethics as well as quantitative and qualitative methods, are welcome. Preference will be given to submissions that demonstrate novel adaptation, creativity, and/or innovation.

Your contributions should be emailed to tsrm-editor@tandf.co.uk in MS Word format. Accompanying images may be included. Contributions should not exceed 1,000 words.

Updated Deadline for submissions is 21 December 2020.