Provoking conversations around power, positionality, and bias in quantitative research

Poster with slogan "Keep Calm and Decolonize"
“Keep Calm and Decolonize” by pasa47 is licensed under CC BY 2.0

Earlier this year, the student-run Quantitative Methods Group at the Faculty of Education, University of Cambridge began discussing the implications of power, positionality and bias in quantitative research. While qualitative researchers have undertaken important work over the past decades to address these implications, we felt that work by quantitative researchers hasn’t benefitted from that kind of critical reflexivity at the same scale. These initial conversations grew into a 4-part discussion series which allowed postgraduate students, researchers, and faculty members to critically engage with the challenges and opportunities that accompany questioning the unstated assumptions and the power in their own and in the reading of others’ quantitative research.

Here are some of the lessons we learned:

  1. We must move beyond questioning only our own power, positionality and bias in our quantitative research

This series helped us consider how we came to our research topic, and how our life experiences shape where we locate injustice and hope within our datasets. However, as quantitative researchers, we can’t stop there. Lenna Cumberbatch, a Diversity and Inclusion strategist undertaking her PhD at the University of St. Andrew’s, raised this as a panellist during our Power dynamics and quantitative research: real experiences session. Lenna challenged the audience to unpack and account for the (often unstated) positionality and biases of other researchers involved in the development of the tools or data they use. For instance, when using widely-accepted categorisations like ‘indigenous’ and ‘non-indigenous’, or ‘LGBTQ+’ and ‘heterosexual cisgender’, we may need to reflect on how these categorisations have been constructed (e.g. by whom and when?), how they are measured (using what tools?) and the consequences of these constructions for our research and findings. Equally, when engaging with existing quantitative research, it is important to question the biases we bring to our reading and understanding of the existing body of knowledge – not just those we bring to the development of our work.

  1. We exercise power through our inclusion and exclusion of certain knowledge and voices in our work

These sessions led us to question seemingly little decisions made as part of our research – particularly in early stages. When we group participants into categories, what important heterogeneity is elided — and conversely, who benefits when we make so many different categories that we lose statistical power? When we focus on one vulnerable group, what knowledge do we miss in our exclusion of others? While our PhDs cannot encompass everything we find interesting, spending time considering and acknowledging both who and what knowledge we include and exclude is critical for our work. Another panellist, Laura Budzyna of MIT D-Lab, shared insights from her work co-designing monitoring and evaluating frameworks that centre research participants and their priorities, thereby improving the utility of the research process and findings for participant communities. The underlying educational issue being researched (for example, the “global learning crisis”) is likely to be defined differently by funders, employers, researchers, school leaders, teachers, and students. It is critical that we understand the implications of these differences in definition, and how the definition(s) we emphasise informs our research priorities, design, methods, findings, and any policy recommendations.

  1. There is power in how we present our research and the language we use

We spent time unpacking how sloppiness in our language may imply sloppiness (or worse) in our conceptualisation — like describing patterns of marginalisation as caused by, rather than associated with, ethnic origin. Such misinterpretations, even when unintended, can be dangerous. Maggie Walter, who presented a seminar on Indigenous Quantitative Methodologies and the Indigenous Data Sovereignty movement as part of the discussion series, summarised this for the group when she outlined a history of such research situating Aboriginal and Torres Strait Islander Peoples in Australia as “Hapless, Hopeless and Helpless”. This is not only relevant for our papers, conference presentations and theses but also in early stages of research planning when forming research aims and questions.

Moreover, the way we describe our research to participants is important. Connecting to our previous point, this could impact in terms of who is and isn’t included in the research. Describing a project as focusing upon LGBTQ+ people’s experiences might, for example, exclude those who experience same sex or same gender attraction but do not see themselves within that umbrella. Additionally, we need to be aware of our participants’ potential biases in light of existing dominant discourses and power structures, when we design research tools and measures. For example, when asked demographic questions on a questionnaire, many participants might, for example, assume (even unconsciously) that categories of ‘white’ ‘straight’ or ‘male’ will appear first in a list of options.

Questioning these issues in our own work and others’ is complex and challenging. While the discussion series has not provided all the answers (and, in some cases, led to more questions!), applying these lessons serves as an important starting point. More importantly, these sessions – especially our collaborative workshop Addressing and accounting for power, positionality, and bias in your quantitative project – taught us that we cannot do this alone. Working alongside other researchers who are committed to engaging in sometimes uncomfortable conversations helped each of us feel less alone as we traverse this necessary discomfort. An essential part of questioning our power, positionality and bias is reaching out to others who can support, and be supported by, you in this journey. As Chancellor’s Fellow at the Department of Social Anthropology at the University of Edinburgh Chisomo Kalinga recently tweeted (@MissChisomo) “decolonial work is not compiling a list, it is nurturing a community”.

Authors

  • Laura Cashman

    Laura is a doctoral student at the Research for Equitable Access and Learning (REAL) Centre at the Faculty of Education, University of Cambridge and a coordinator of the Faculty’s Quantitative Methods Group. Her doctoral research focuses on addressing inequality in education in the Global South through the engagement of stakeholders, particularly parents. Prior to her PhD, Laura worked in educational research and programme management in sub-Saharan Africa, Southern Asia and Europe.

  • Deborah Spindelman

    Deborah Spindelman is a fourth year PhD student in the University of Cambridge's Faculty of Education, where she is a member of the REAL Centre and co-leads the Whole Child in Development discussion group. A fellow of the Royal Geographical Society, her doctoral research examines the implications of early undernutrition for cognitive development and unmet learning needs among early primary students in rural and peri-urban government schools in northern Ghana. Originally from Southern California, Deborah worked in international development prior to her PhD and holds an MPA in Development Practice from Columbia University and a BA (honors) in Anthropology from California State University, Long Beach.

  • Charlotte Allen

    Charlotte Allen is a part-time PhD student at the Faculty of Education and Trinity Hall. Her research is a mixed methods project that focuses upon LGBTQ+ and heterosexual cisgender students’ experiences and resilience at school. Charlotte is a member of the Faculty of Education Wellbeing and Inclusion Special Interest Group and co-led the graduate student-run Quantitative Methods Group from 2018-2021.

BAICE is a charity, registered in the UK. The BAICE Media Hub supports BAICE's charitable objective of stimulating and disseminating knowledge and research in the field of international and comparative education. Views expressed in outputs hosted on the BAICE Media Hub are those of the contributors. They do not necessarily represent the views of the BAICE Executive Committee or the wider BAICE membership.