Essential Evaluators seeks to gather evaluators in a common space to dialogue, reflect, and support each other in a world upended by COVID-19 and the Black Lives Matter protest movement. This is a time of uncertain and unknown expectations in our professions, in our institutions, and in our communities. We invite you to join us as we rethink, revision, and ultimately redefine our roles as evaluators and our place in museums.
Other posts in this series:
As museum professionals delve into conversations about equity and racial justice, evaluators must scrutinize our practices as social scientists and bring to light the ways in which they perpetuate inequities and reinforce power imbalances.
This begins with examining the relational dynamics at play when we engage in a study: between the evaluator and the museum, the specific client within the museum and the researcher, and the researcher and the subjects. In these relationships, power is a central though often overlooked dynamic; however, it is key to examine when we commit to equity and justice, because the relationship between researcher and subject is not free from power imbalance (Samuels, 2004; Smith, 2006).
Skip over related stories to continue reading articleAs a field dominated by white, female, well-educated professionals, our biases and experiences influence all of our work, from designing our evaluations to determining the final presentation of the results. It is most often our practice to craft questions, analyze data, and present results in a vacuum with internal stakeholders (or even with just our own teams)—all without systematically engaging participants in the process. While there are many arguments for the distanced, “objective” researcher, we must also consider the ability of participants to be an authoritative voice in how their own thoughts, perspectives, experiences, or other data are gathered, interpreted, and disseminated. Acknowledging this as a power imbalance is a first step, but we must then examine how we—as professionals—participate in it, and seek opportunities to shift toward more equitable evaluation and research practices.
To advance that goal, this post will explore a data collection method that has little momentum in the museum evaluation world but much potential for creating studies which are rooted in the perspectives, voices, and knowledge of the diverse communities with whom we engage. The method, known as “collaborative knowledge production” (Van Auken, Frisvoll, & Stewart, 2010, p. 375), entails creating environments where participants feel that they are meaningful collaborators in the research project, dissolving the wall between researcher and subject. By reframing our work in this way, we can view ourselves as part of a collective, a team, all working together to create a deeper understanding of the phenomenon in question. In the process, we can not only shift how we view ourselves and our role, but in turn create a deeper sense of participation and ownership on the part of the participant. This type of research takes a different approach—it asks us to connect and be present with those being researched and engage in more deep and personal ways.
There are numerous techniques for disrupting the idea of an objective researcher in this way, but this post will focus on two that have application for museums—participatory action research and photo elicitation.
Participatory action research (PAR) employs the “subjects” of a study as co-researchers inquiring into issues that directly impact them and their communities. Participants take part in everything from identifying the issue to be examined, to collecting and analyzing the data, to determining the course of action. This is meant to disrupt the traditional power dynamic between a researcher and subject so far that the subjects become the researchers. PAR is seen as a new paradigm in science, differing greatly from positivist science by positing that the researcher impacts that which is being observed and that their inquiry—and their values—will change both the study and the situation (Wadsworth, 1998).
One example of a museum PAR project comes from the Denver Museum of Nature & Science. When the museum’s staff wanted to understand more about the underrepresentation of Latino/a/x people among free-choice visitors, researchers leveraged an existing relationship with a bilingual parent group, inviting those interested to join them in exploring the topic of museum visitation. These caregiver-researchers worked with the research staff to conduct autobiographical accounts of their own experiences with the institution, then identify themes from these accounts on which to build interview questions. These themes included: 1. The museum was perceived as being “quiet like a library,” signaling that it would be uninviting or an un-welcoming place for exuberant families; 2. Parents feared not being able to understand the information shared in the almost exclusively monolingual English halls, prohibiting them from engaging with or answering questions from their bilingual children; 3. There was a lack of familiarity with the institution itself.
The caregiver-researchers then reached out to their own community members to explore the questions that surfaced. Together, the team participated in collaborative data analysis and synthesis, and worked with museum researchers to propose solutions based on their research findings. These solutions included offering bilingual caregiver-child programming, which would allow caregivers to hear information in Spanish in preparation for visiting exhibitions; allowing previous program participants to work as ambassadors for the museum within their own communities to share their experiences and break down existing perceptions; and allowing participants to have designated hours, such as before or after museum hours, to explore the space with bilingual museum professionals who could answer questions and create a sense of comfort and familiarity with the space. The research team presented the study’s findings to museum program staff, and while the solutions were not ultimately adopted, the impact of their work resonated and shifted conversation and understanding throughout the museum.
Another intriguing avenue for exploration is known as photo elicitation. Photo elicitation is when the researcher utilizes photographs during an interview, either by presenting pre-selected photographs (“researcher-driven”) or allowing participants to choose or create them (“participant-driven”) (Banks, 2001; Collier, 1967; Collier & Collier, 1986; Harper, 1994; Prosser & Schwartz, 1998; Suchar, 1989). In such studies, photographs become the basis for an interview, and researchers have found that using imagery elicits deeper, more vivid descriptions of the phenomenon in question. Within the two types of photo elicitation approaches, researcher-driven photo elicitation studies tend to be more close-ended and deductive (Torre & Murphy, 2015; Van Aukenn, Frisvoll, & Stewart, 2010), while participant-driven photo elicitation can create an environment in which participants feel they are meaningful collaborators in the research project (Harper, 2002; Van Auken et al., 2010), thus “creating opportunities for citizens to be more meaningfully involved in data generation” (Van Auken, Frisvoll, & Stewart, 2010, p. 375). When participants are allowed to select the images that best represent the phenomena, or—a step further—create or collect the images, their contribution and agency in developing an understanding of the phenomena grows.
At the Denver Museum of Nature and Science, Andréa Giron Mathern was set to engage in a large participant-driven photo elicitation study of an underserved demographic in the museum’s free-choice visitor base. She wanted to understand, from the visitor’s perspective, where in the museum they felt that they belonged, where they felt that they were included, and where they felt a sense of exclusion. Participants were to explore the museum with their family or social group and take photographs of objects, signage, interactions, and spaces where they experienced the feelings being explored. She was then to interview the participants and ask them to select the images they wanted to use to discuss belonging, exclusion, and inclusion.
The study was ready to go, until the museum closed to the public for COVID-19 the day before the first interviews were scheduled. Upon reopening, the approach became more complex, in order to maintain social distancing and respect visitors’ varying comfort levels in returning to the institution. Suddenly, even the action of handing a visitor a data collection device had health implications. So the study has had to move online, and Giron Mathern has had to rethink how to create the depth of information, connection, and diversity of participants. Things like sample size, membership status (the museum is currently experiencing a disproportionally higher response rate from members than in pre-pandemic times), and the infeasibility of having people capture their experiences in the moment were among a few of the considerations she has had to adjust for during COVID times. She knows she is not alone in this, and the questions now are: how do we employ such techniques, which require deep relationship building, in the time of social distancing?
Do you practice research or evaluation methods which disrupt the power imbalance of traditional research, such as participatory action research, digital storytelling, participatory video, photovoice, theater for development, the reality check approach, or another kind of participant-centered research? How do we create opportunities for collaborative knowledge production when much of our data collection efforts have been pushed to the online arena? We’d love to hear from you; share your experiences in the comments section below!
Works Cited
- Banks, M. (2001). Visual Methods in Social Research. Thousand Oaks, CA: Sage.
- Collier, J.J. (1967). Visual Anthropology: Photography as a Research Method. New York, NY: Holt, Rinehart, & Winston.
- Collier, J.J., & Collier, M. (1986). Visual Anthropology: Photography as a Research Method. Albuquerque, NM: University of New Mexico Press.
- Harper, D. (1998). An argument for visual sociology. In Prosser (Ed.), Image-based Research: A Sourcebook for Qualitative Researchers, (pp. 24-41). London, UK: Falmer Press.
- Harper, D. (2002). Talking about pictures: A case for photo elicitation. Visual Studies, 17(1), 13-26.
- Jones, R, N Hussain, and M Spiewak (2020). The Critical Role Research and Evaluation Assume in the Post-Truth Era of Climate Change. Journal of Museum Education, 45(1), 64-73.
- Prosser, J., & Schwartz, D. (1998). Photographs within the sociological research process. In Prosser (Ed.), Image-based Research: A sourcebook for Qualitative Researchers (pp. 115-130). New York, NY: Routledge Falmer.
- Samuels, J. (2004). Breaking the ethnographer’s frames: Reflections on the use of photo elicitation in understanding Sri Lankan monastic culture. American Behavioral Scientist, 47(12), 1528-1550.
- Smith, G. A. (2007). Place‐based education: Breaking through the constraining regularities of public school. Environmental Education Research, 13(2), 189-207.
- Suchar, C.S. (1989). The sociological imagination and documentary still photography: The interrogatory stance. In Flaes (ED.), Eyes Across the Water: The Amsterdam Conference on Visual Anthropology and Sociology (pp. 51-63).
- Torre, D., & Murphy, J. (2015). A Different Lens: Using Photo-Elicitation Interviews in Education Research. Education Policy Analysis Archives, 23(11), 111.
- Van Auken, P.M., Frisvoll, S.J., & Stewart, S.J. (2010). Visualising community: Using participant-driven photo-elicitation for research and application. Local Environment,15(4), 373-378.
- Wadsworth Y. What is participatory action research? Action Research International. 1998
Comments