Earlier this year, the student-run Quantitative Methods Group at the Faculty of Education, University of Cambridge began discussing the implications of power, positionality and bias in quantitative research. While qualitative researchers have undertaken important work over the past decades to address these implications, we felt that work by quantitative researchers hasn’t benefitted from that kind of critical reflexivity at the same scale. These initial conversations grew into a 4-part discussion series which allowed postgraduate students, researchers, and faculty members to critically engage with the challenges and opportunities that accompany questioning the unstated assumptions and the power in their own and in the reading of others’ quantitative research.
Here are some of the lessons we learned:
- We must move beyond questioning only our own power, positionality and bias in our quantitative research
This series helped us consider how we came to our research topic, and how our life experiences shape where we locate injustice and hope within our datasets. However, as quantitative researchers, we can’t stop there. Lenna Cumberbatch, a Diversity and Inclusion strategist undertaking her PhD at the University of St. Andrew’s, raised this as a panellist during our Power dynamics and quantitative research: real experiences session. Lenna challenged the audience to unpack and account for the (often unstated) positionality and biases of other researchers involved in the development of the tools or data they use. For instance, when using widely-accepted categorisations like ‘indigenous’ and ‘non-indigenous’, or ‘LGBTQ+’ and ‘heterosexual cisgender’, we may need to reflect on how these categorisations have been constructed (e.g. by whom and when?), how they are measured (using what tools?) and the consequences of these constructions for our research and findings. Equally, when engaging with existing quantitative research, it is important to question the biases we bring to our reading and understanding of the existing body of knowledge – not just those we bring to the development of our work.
- We exercise power through our inclusion and exclusion of certain knowledge and voices in our work
These sessions led us to question seemingly little decisions made as part of our research – particularly in early stages. When we group participants into categories, what important heterogeneity is elided — and conversely, who benefits when we make so many different categories that we lose statistical power? When we focus on one vulnerable group, what knowledge do we miss in our exclusion of others? While our PhDs cannot encompass everything we find interesting, spending time considering and acknowledging both who and what knowledge we include and exclude is critical for our work. Another panellist, Laura Budzyna of MIT D-Lab, shared insights from her work co-designing monitoring and evaluating frameworks that centre research participants and their priorities, thereby improving the utility of the research process and findings for participant communities. The underlying educational issue being researched (for example, the “global learning crisis”) is likely to be defined differently by funders, employers, researchers, school leaders, teachers, and students. It is critical that we understand the implications of these differences in definition, and how the definition(s) we emphasise informs our research priorities, design, methods, findings, and any policy recommendations.
- There is power in how we present our research and the language we use
We spent time unpacking how sloppiness in our language may imply sloppiness (or worse) in our conceptualisation — like describing patterns of marginalisation as caused by, rather than associated with, ethnic origin. Such misinterpretations, even when unintended, can be dangerous. Maggie Walter, who presented a seminar on Indigenous Quantitative Methodologies and the Indigenous Data Sovereignty movement as part of the discussion series, summarised this for the group when she outlined a history of such research situating Aboriginal and Torres Strait Islander Peoples in Australia as “Hapless, Hopeless and Helpless”. This is not only relevant for our papers, conference presentations and theses but also in early stages of research planning when forming research aims and questions.
Moreover, the way we describe our research to participants is important. Connecting to our previous point, this could impact in terms of who is and isn’t included in the research. Describing a project as focusing upon LGBTQ+ people’s experiences might, for example, exclude those who experience same sex or same gender attraction but do not see themselves within that umbrella. Additionally, we need to be aware of our participants’ potential biases in light of existing dominant discourses and power structures, when we design research tools and measures. For example, when asked demographic questions on a questionnaire, many participants might, for example, assume (even unconsciously) that categories of ‘white’ ‘straight’ or ‘male’ will appear first in a list of options.
Questioning these issues in our own work and others’ is complex and challenging. While the discussion series has not provided all the answers (and, in some cases, led to more questions!), applying these lessons serves as an important starting point. More importantly, these sessions – especially our collaborative workshop Addressing and accounting for power, positionality, and bias in your quantitative project – taught us that we cannot do this alone. Working alongside other researchers who are committed to engaging in sometimes uncomfortable conversations helped each of us feel less alone as we traverse this necessary discomfort. An essential part of questioning our power, positionality and bias is reaching out to others who can support, and be supported by, you in this journey. As Chancellor’s Fellow at the Department of Social Anthropology at the University of Edinburgh Chisomo Kalinga recently tweeted (@MissChisomo) “decolonial work is not compiling a list, it is nurturing a community”.
