featured, Notebook

A virtual collection to celebrate 25 volumes of IJSRM 

The International Journal of Social Research Methodology is celebrating its 25th anniversary!!  In that period, we have become a leading methods journal publishing high quality contributions across the methodological field – both qualitative and quantitative, and including mixed, multi-media, comparative and simulation methods. 

To mark the occasion, we have gathered together a series of methods discussions that have been published in the journal, and our publisher, Routledge, is making them freely available as a collection.  

Choosing which articles to include in our anniversary virtual collection was a hard task.  We inevitably had to leave some important and favourite pieces aside.  The collection below includes contributions that we felt represented the range of methodological articles that we publish in IJSRM, a selection of early career prize winning articles, influential pieces and discussions that deserve more attention for their contributions, and individual editors’ personal choices.  

The methodological reach of our anniversary, then, ranges across survey non-response, behavioural experiments, quantitative pedagogy, the Delphi method, the problem-centred expert interview, the self-interview, narrative and computerised text analysis, qualitative methods transformations, anonymisation, triangulation of perspectives, indigenous data sovereignty, post-humanism, and researcher safety. 

We hope that you enjoy our selection. You can access it at:  

https://www.tandfonline.com/journals/tsrm20/collections/TSRM_25th_Anniversary_SI

Rosalind Edwards, Jason Lamprianou, Jane Pulkingham, Malcolm Williams 

IJSRM Co-Editors 

featured, Notebook

Seeking alternatives: A reflection on conducting online interviews with disabled young people during the COVID-19 pandemic

Angharad Butler-Rees and Stella Chatzitheochari

While scholars have increasingly documented and reflected on their approaches to conducting research during the pandemic, little is still known about the impact of social distancing measures on qualitative research with disabled young people.

Our new paper provides a methodological reflection on undertaking qualitative research with disabled young people as part of the Educational Pathways and Work Outcomes longitudinal study. Our study started in March 2021 during the third national lockdown in England. Due to social distancing measures in place at the time of commencing, it was necessary to revise our original plans to conduct face-to-face interviews with disabled young people and conduct online interviews instead. We conducted a total of 35 online interviews with autistic, dyslexic, and physically disabled young people aged 15-16 years old.

Ensuring Accessibility

The internet has long been deemed as a potentially empowering platform for disabled people, connecting isolated individuals and ensuring access to social, civic and community life. Our focus on young people was particularly useful, as this population tends to be very comfortable with the use of technology. The extensive periods of enforced home-learning during the covid-19 pandemic had further increased young people’s familiarity with online communication platforms, rendering the idea of online interviews a far less daunting prospect. However, it is worth noting that online tools can also present a number of accessibility barriers e.g., poor text layout, little colour contrast and limited keyboard functionality. These were important factors to consider when designing online interviews for our project.

Accessibility has to be incorporated into every part of the research process when working with disabled people. To put participants at ease prior to their interview, we sent them participant information packs as well as a short video of the interviewer introducing themselves and the study. Familiarity with the researcher was greatly valued for autistic young people, making them feel more at ease. Previous literature has suggested that autistic young people may be disconcerted or unresponsive in encounters with strangers, so building a degree of initial trust and rapport was of upmost importance for successful interviewing. In line with this, we also arranged online pre-interview meetings with participants and their parents to build rapport.

Pre-interview meetings also helped us ensure that any accessibility requirements were put into place. We asked participants to choose their preferred communication platform. Several participants opted to use assistive software during their interview e.g., enabling captioning, magnification or modifying volume. Other adjustments included allowing participants to sit off screen or to keep their cameras off while the interviewer remained visible. This made interview far less intrusive and anxiety provoking and was greatly valued by autistic participants. Other adjustments included the presence of a guardian that could provide practical assistance or emotional support, simplification of interview questions, as well as collection of data over several interviews as opposed to one. Overall, we felt that these adjustments made interviews considerably more accessible for disabled young people, ultimately giving voice to a population who may not always be amenable to conventional face-to-face interviewing methods that can be experienced as more restricting and demanding.  

Challenges during Interviewing

While some young people were very comfortable in engaging with the interview process and narrating their lived experiences, others were far more hesitant, requiring regular prompting and reassurance. The online medium made this slightly more challenging for the interviewer, with prompting and encouragement occasionally leading to cross-talking. It was also notably more difficult to interpret emotion and body language online, while the loss of internet connection at times affected the flow of the interview.

Another challenge was the difficulty in maintaining participants’ attention. We sometimes felt that the lack of physical presence meant that participants were far easier distracted by being in their homes, e.g., checking their mobiles, playing with family pet. However, we also recognise that this may be interpreted in a different manner: Indeed, it may be indicative of a greater share of power afforded to disabled young people in online settings. Overall, we did not feel that such distractions affected the quality of our data collection and think that the physical distance may have aided disclosure of personal experiences. A feedback survey confirmed that participants enjoyed the use of the online medium, with the vast majority requesting online interviews for the future waves of data collection.

A final note on accessibility

Our reflections may not speak to studies seeking to interview disabled young people with different accessibility needs such as speech or communication difficulties (e.g., stammering). These participants may find online communication more difficult due to possible misunderstanding and difficulties in lip reading and interpretation. Similarly, it is worth noting that interviews can be experienced as particularly exhausting by some disabled young people, whether face to face or online, preventing them from taking part. Researchers may consider offering alternatives such as email interviews alongside conventional online interviews.

Looking ahead

Our overall experience with online interviews was very positive. We were privileged to be able to access disabled young people’s lived experiences during an unprecedented period of global disruption. Notwithstanding the challenges mentioned above, we feel that online interviewing is a valuable tool that should not be viewed as second best to face-to-face conversational methods. We therefore encourage researchers to explore the use of online methods, especially with regards to young and disabled populations.

Read the full article here: Giving a Socially Distanced Voice to Disabled Young People: Insights from the Educational Pathways and Work Outcomes Longitudinal Study

Notebook

An image of China in Africa through the lens of mixed-methods

by Jarosław Jura & Kaja Kałużyńska

The increasing number of digital and digitized content sources (online versions of traditional media, news portals, various websites on myriads of topics, and, of course, social media) has started to influence empirical social research. Huge amounts of easily accessible and almost ready-to-analyze datasets seem to be a dream coming true for social researchers, especially those who prefer to work with unobtrusively-collected data.

Such large datasets ask for being analysed by mixed methods, to avoid wasting their potential by either choosing a sample or focusing on quantitatively obtained information only. Here come other tools that make the life of a contemporary researcher much more comfortable – software solutions. Of course, in the ideal situation, one could just ‘feed’ all the data to AI and wait for the results, but there are many limitations to such an approach, like usability in specific cases, its accessibility, and, of course, the researcher’s nightmare: a limited project’s budget. Moreover, in the case of smaller datasets, consisting of heterogeneous data, analysis’ results might prove unsatisfactory.

Our research project, an exploratory study on the image of China and the Chinese in Zambia and Angola, included also an analysis of textual media content, namely news articles published in these countries and mentioning China or the Chinese. We obtained a mid-sized dataset, consisting of 2477 articles; the material was very heterogeneous, because of the wide scope of topics covered by the texts and the fact that we analysed content from both English- and Portuguese-language media.

In the course of analysis, we realized that a new method would be needed to obtain the best possible results on the basis of the collected data. After a series of trial-and-error approaches, we managed to develop MIHA – Mixed Integrative Heuristic Approach. The application of this method allowed us to create an exhaustive, contextual and precise keyword dictionary for automated classification of text units as well as a set of sentiment indexes.

We have to admit, that even though we did our best to utilize all the possibilities of the software (Provalis QDA Miner and Wordstat), the dictionary creation process was a time-consuming task since it included reviewing each word of frequency higher or equal to 10 in the whole database.

Our classification, similar to the initial conceptualization of theoretical categories within the grounded theory approach, aimed to explore the most frequent contexts in which China was depicted in African e-media. Each examined word was either added to an exclusion list (words irrelevant from the point of view of the research) or assigned to a chosen – sometimes a newly created – category, together with other words of the same root and all the synonyms.

In the next step, we examined the already categorized keywords in their context to refine the categorization results, mainly by removing those keywords that appeared within the text in unexpected contexts. Most of the categories were re-coded, and some of the keywords were re-assigned in the next steps. This heuristic approach resulted in a set of categories, including ‘emotional’ ones, positive and negative, that later on were used to design sentiment indexes. Our indexes are based on a comparison of the results of quantitative and qualitative analysis and coding. They could be used as a tool for improving dictionary-based sentiment analysis by comparing the results of sentiment analysis performed on the basis of automated coding with manually-coded samples.

We believe that MIHA constitutes a conceptual approach applicable by researchers of various backgrounds in projects focused on investigating the general image presented in textual content, especially in case of mid-sized, heterogeneous data sets. We do not overlook the fact that soon, automated machine learning coding methods will constitute the main approach towards text analysis. However, since such procedures are still imperfect and context-sensitive, we presume that MIHA, consisting of a contextualized dictionary, manual coding of chosen parts of the database and index measurements, could be useful for analysis of data sets related to less common study areas (social groups, languages, geographical areas, subcultures, etc.), in which machine learning-based research would contain a low level of construct validity.

Both the dictionary-creation process and the indexes are described in detail in our paper.

Read the full article in the IJSRM here.

Announcements, featured

The winners of our ECR paper competition for 2020-21

We are pleased to announce the results of our 2020-21 IJSRM competition for papers written by early career researchers (ECRs) who, at the time of submission, were either doctoral students or in their first three years of post-doctoral employment. Our aim has been to encourage and recognise research and contributions from new scholars in current and emerging methodological debates and practice.

All entries were subject to the Journal’s usual refereeing processes and had to reach our normal publishing standard. The winners were selected by a sub-panel of members of the IJSRM Editorial Board and the Journal Editors. The panel were impressed with the very strong field of entries, and we are pleased to announce not only a winner of the ‘Best ECR Article’ but also three ‘highly commended’ runners up.

Our IJSRM Early Career Researcher Prize is awarded to Stefanie Döringer (Austrian Academy of Sciences and University of Vienna) for her article on ‘The problem-centred expert interview: Combining qualitative interviewing approaches for investigating implicit expert knowledge’. The panel of judges remarked on a ‘clearly written and illuminating account’, representing ‘a lightbulb moment that brings two previously disconnected traditions together’, and that will be ‘highly valuable as a reference for many researchers for years to come’. Stephanie’s article has already been viewed over 16,500 times.

Stephanie said, ‘It is a great honor for me that my paper is awarded with the IJSRM Early Career Researchers’ prize. The appreciative comments from the competition judges encourage me to follow my research interest further and to deepen my work with qualitative methods in social research’.

Our highly commended runners (in alphabetical order) are: Riccardo Ladini (University of Milan): ‘Assessing general attentiveness to online panel surveys: The use of instructional manipulation checks’ Órla Meadhbh Murray (Imperial College London): ‘Text, Process, Discourse: Doing Feminist Text Analysis in Institutional Ethnography’ Kate Summers (London School of Economic and Political Science): ‘For the greater good? Ethical reflections on interviewing the ‘rich’ and ‘poor’ in qualitative research

Many congratulations to Stephanie, and also to Kate, Órla and Riccardo.

Announcements

Changes to our Journal metrics – a more rounded picture?

Journal Impact Factors – based on citations of articles published in the journal concerned, have been used as a proxy for the prestige of a journal in comparison with others in its field. And promotion committee considerations can be based on whether an academic has had articles published in high impact factor outlets. But this means of assessing the value of journals, the quality of articles published in them, and by extension the standing of the authors of published pieces, has been subject to criticism. These concerns run from questioning the reliability of the measurement and ranking, through encouragement to editors and authors to game the system, to condemnation of a neo-liberalised audit culture in academia. Some publishers and platforms, such as PLOS, have decided not to display Impact Factors.

Our Journal’s publisher, Routledge/Taylor & Francis, is now starting to shift away from the Impact Factor as a key indicator of quality, replacing it with a ‘basket’ of metrics in an effort to provide a more rounded views of the various ways in which a journal and articles published within it may have scholarly, policy and social ‘impact’.

The metrics being posted on the publishers’ IJSRM page for 2020 are:

  • an Impact Factor of 3.061 for the year, and one of 4.508 over a 5-year period
  • a new Journal Citation Indicator of 2.15, ranking 13/254 in the category of Social Sciences, Interdisciplinary, and
  • a CiteScore of 5.0, ranking 16/260 in all Social

Whether or not such moves deal with the concerns about measurement reliability, gaming the system, and evaluating of academics, is a moot point.