Skip to content

Essential Evaluators: A Brave New Dance

Category: Alliance Blog
A graphic with a teal background reading "Essential Evaluators."
A new series explores how museums can center evaluation at a time when COVID-19 and the Black Lives Matter protest movement are upending audience expectations.

Essential Evaluators seeks to gather evaluators in a common space to dialogue, reflect, and support each other in a world upended by COVID-19 and the Black Lives Matter protest movement. This is a time of uncertain and unknown expectations in our professions, in our institutions, and in our communities. We invite you to join us as we rethink, revision, and ultimately redefine our roles as evaluators and our place in museums. #EssentialEvaluators

Other posts in this series:


During the past month, a few brave voices have emerged from the corridors of the museum world to boldly and defiantly confront institutional racism. Some of these voices have recounted painful personal stories of being ignored or blatantly discounted when raising issues of social injustice. Others have joined forces anonymously to create a cacophonic narrative. Still others have taken a tongue-in-cheek approach to parody scripted responses from leadership.

One of the most striking, though largely overlooked, aspects of these collective narratives is the degree to which fear is used to silence and suppress our voices—fear of retaliation, of termination, of not being promoted, of not being able to find another job in the post-COVID crisis. It’s a pernicious fear that has long been used to preserve the status quo, even as most institutions would not claim to promote it. Yet museums have tacitly managed to sidestep real-world issues like racism, sexism, and classism through a carefully scripted choreographed dance. They have moved in accordance to a well-rehearsed pattern of rhythmic steps—shallow listening, hushed conversations, rhetorical musings, and token appointments—that essentially serve to keep us all in line, stepping to the same beat and moving in the same pattern.

Skip over related stories to continue reading article

In the evaluation field specifically, this fear can manifest in looking for “good numbers” instead of those that reveal uncomfortable truths, avoiding transparency in data collection, and reaching out only to those respondents who want to talk with us.

The pressure for museums to diversify audiences can compel evaluators to “prove” that institutions are already making strides, rather than pointing to where they need to improve. The dance between remaining relevant and needed in our institutions and providing the “right” answers can drive evaluators to solutions that may reflect well on an institution or program but require the evaluator to manipulate data in ways that challenge our rigor and professionalism.

This pattern can look like an example witnessed by our co-author Andréa:

“While visiting with an evaluation team at a science museum, they walked me through their visitor demographic information. I was really interested in how they had managed to serve Latino/a/x visitors in much higher proportions than their city and surrounding area. They explained that members of their team would stand on a bridge above the entrance and ‘eyeball’ Latino/a/x visitors as they came in on special event days. I was shocked; it was a well-respected museum and evaluation department. While this is an extreme example of unethical and unprofessional evaluation practices, it is not the only experience I have had over the years related to data collection efforts that seek to bolster diversity, access, and inclusion efforts—efforts that keep the dominant narrative in-line and alive.”

In another example, a children’s museum gathered visitor demographics on select days throughout the year, including all of their free days and nights, which had a higher proportion of diverse visitors. The result of their sampling strategy was that free days and nights were overrepresented in their data, thus providing a picture that their overall audiences were more diverse.


If real change is to finally and fully emerge in museums, it is critical that the field join forces to disrupt this rhythm. We need a new dance, one where fear and silence are no longer part of the beat, where data is data—neither “good” nor “bad,” just more information and opportunity to help reflect, learn, pivot, and plan. To spur evaluators on in this, we want to highlight some examples where our field has pushed for—and normalized—inclusion, equity, and accessibility:

Language—From the Collaboration for Ongoing Visitor Experience Studies

“From its inception, COVES has encouraged collective growth and change among our participating institutions; sometimes survey questions pushed our participating institutions to ask things they would not have otherwise considered, and sometimes our participating institutions pushed us to change our questions to make them more inclusive. As a collaborative project that includes over 30 museums, we know that we have the power to shape the trajectory of visitor experience research, and the questions we ask matter.

One example from our time as a collaborative is when our partners at the Exploratorium translated the bulk of the core COVES survey into Spanish, and others at the Science Museum of Minnesota and the Museum of Science, Boston updated those translations as we made changes to core questions. This ensured we did not systematically exclude our Spanish-speaking visitors from our research, and we continue to work to incorporate more languages.

More recently, we changed a scale question where the high point was the word “Superior” to use the word “Outstanding” instead. Members of our survey review team pointed out that word “superior” is associated with pseudoscientific claims about racial hierarchies, and a partner institution shared that they had indeed encountered visitors who felt uncomfortable with the word.”

Inclusion—From the Denver Museum of Nature & Science

“After years of community partnership, the Museum was interested in gathering information about if respondents identified as part of the LGBTQ+ community. We worked with LGBTQ+ community members, whom we had developed strong relationships with, to help us examine whether gathering visitor demographics on these communities would be helpful or not. Through this partnership of trust and conversation, both internal and external stakeholders felt that a question on the visitor experience survey would help identify any gains in community visitation as well as ensure LGBTQ+ community members felt welcomed, important, and seen.

The Museum’s LGBTQ Core Team worked with community members to craft a question and responses that felt inclusive and comfortable. These stakeholders also helped change the language of our gender-based question, which initially included man, woman, transgender, and gender non-conforming. Based on their feedback, we changed gender non-conforming to non-binary and added ‘prefer to self-describe.’ The community then challenged us to think critically about why we needed to collect data on gender at all. By staying open to that critical reflection, we were able to examine how we use that data and have since removed the gender question from our standard list of demographic questions on our visitor experience survey all together.”

Accessibility—From the Denver Museum of Nature & Science

“To help focus the work of our Accessibility & Relationships initiative team, the evaluation team decided to ask the question: Would accommodations in any of the following categories have benefited anyone in your group? We worked with community members to provide us with guidance and feedback about appropriate language for the options. Together we decided upon these options: physical/mobility, hearing, developmental, intellectual, autism spectrum, dementia/memory loss, and none of the above. Moving into data collection in the time of COVID-19, we added “Compromised Immune System” as another choice.

This example showed the evaluation team that survey-writers impact not only the types of data collected but ideas of welcoming, representation, and belonging; important factors in creating spaces that are truly diverse and inclusive. As we navigate the ‘new normal’ of COVID-19, which feels ever-changing, we must continue to be flexible, open, and reflective about the questions we are asking, how we ask them, who crafts the question and how these decisions impact the kind of data we receive.”

Staff—From a not-to-be-named “Public library with historical collections in an urban setting”

“As one example of using evaluation to correct a problem related to bias and racism, I looked at current staff and reviewed personnel and anecdotal records from the prior 10 years to confirm that, as a personnel director once said in a seminar, ‘there was a distinct pattern of staffing with people who look like me.’ In a community that was 60/40 African American/white, the staff was almost entirely white, and overwhelmingly female. I also recommended the development of policies around anti-bullying and anti-harassment (this was about 10 years ago) with enforceable penalties and also the development of hiring guidelines that called for broader posting of vacancy notices and of making hiring decisions based on qualifications including education and experience. Within 18 months, the changes in staffing and in behavior were apparent and well-received by the community; visitors and other staff alike. Staff who did not appreciate the changes, for the most part, self-selected out of the agencies.”


As these examples show, evaluators have instigated change; however, we need to do more. We must disrupt the pattern of fear and silence. We must be liaisons for both audience and community engagement, keepers of empirical and objective data, and stewards for institutional ethics and integrity—all absolutely essential for revisioning and rebuilding museum space to be inclusive, equitable, transparent, and accountable.

We believe that it is critical for all evaluators—employed, unemployed, part-time, internal, and external to join together and share their voices and experiences so that we can help build a robust and complete portrait of evaluation in museums. For many of us this is a new role and will most likely be an uncomfortable one, but we need to take a look at our place in this new landscape—what has changed? What questions do we need to be asking? What should be asked of us? How do we use our position to advocate for our communities? How do we embrace the social justice directive to become visible and viable? How do we speak up, stand up, and demand a seat at the decision-making table?

As part of this call to action, we ask you to send us your stories and examples. When have evaluators challenged institutional norms and injustices? To what end? When have you seen evaluators bring the voice of diversity, equity, inclusion, and accessibility to the table? We want to shine a light on the work that has been done in the field to show others what can happen when we dance our new dance. We also want to show how much work needs to be done. Where have we not stepped up and challenged social injustice? Where do we need to do better? Tell us, either in the comments or at trainerevaluation@gmail.com!

AAM Member-Only Content

AAM Members get exclusive access to premium digital content including:

  • Featured articles from Museum magazine
  • Access to more than 1,500 resource listings from the Resource Center
  • Tools, reports, and templates for equipping your work in museums
Log In

We're Sorry

Your current membership level does not allow you to access this content.

Upgrade Your Membership

Comments

12 Comments

  1. Hi
    Very interesting and shows how important community in-put is and that even if it’s uncomfortable we need to do the work. I know that museum free days have long been very popular and bring in crowds, but how do museums change to make that kind of divers addence an everyday thing? I am not sure how espaly in this world when we all face so many chalenges. It may take creativity and time. I do think it’s about time we all have the uncomfortable chats that lead to change. The one thing I know for sure is if we don’t talk about nothing changes. I look forward to future topics and to continueing to learn.

    1. Rachel, this quote came to mind upon reading your comment and reflecting on the past: “Change will not come if we wait for some other person or some other time. We are the ones we’ve been waiting for. We are the change that we seek.” – Barack Obama
      There are so many challenges right now, which perhaps makes it the best time to dismantle systems and question norms. It is exciting to hear people talking about change and writing about it, I can’t wait to see more examples of it!

  2. Hi Laureen
    Yep that’s right. I can’t wait to see the change and be a part of it. Time waits for no one.

  3. Thanks to my colleague, Nick Visscher, for sharing @ChangeTheMuseum – Pressuring US museums to move beyond lip service proclamations by amplifying tales of unchecked racism. Share your story: forms.gle/9RKxBrTezd2oQwFC9

  4. From Patience Baach: This is a great initiative and I appreciate the stories that have already been shared. I’ll be sure to spend some time reflecting this weekend and will consider adding some stories of my own. Thank you Rose for shedding a light on this.

  5. From Molly Beyer: I don’t work in museum evaluation, but I do work in evaluation (as you know :)). This part really rang true everywhere I’ve been:

    “In the evaluation field specifically, this fear can manifest in looking for “good numbers” instead of those that reveal uncomfortable truths, avoiding transparency in data collection, and reaching out only to those respondents who want to talk with us.

    The pressure for museums to diversify audiences can compel evaluators to “prove” that institutions are already making strides, rather than pointing to where they need to improve. The dance between remaining relevant and needed in our institutions and providing the “right” answers can drive evaluators to solutions that may reflect well on an institution or program but require the evaluator to manipulate data in ways that challenge our rigor and professionalism.”

  6. From Brian Robertson: Good points, Molly Beyer. Truth and objectivity should always be the guiding principles for an evaluation, with the goals of showing what works, what doesn’t, and how to make it better. Unfortunately, truth and objectivity are hard pills to swallow for some. The path to improvement should be a welcomed challenge as it will yield the desired results far better than manipulated data.

  7. Hi
    I agree that the time for uncomfortable talks is now fallowed by action. I see so much poromice in museums to do so much to change themselves and society. To teach us all the uncomfortable truths we need to learn. I only wish I could do more. The future is what we make it, so lets make a better one.

  8. In my work, I worry about representing different groups without tokenizing them. I don’t think I’m always successful, but one good thing that we have done is to create a list of individuals with a range of disabilities who are willing to come help us test exhibit prototypes. This helps us better understand the accessibility of our offerings and is something we do as a part of every new exhibit project.

    1. Liz, thanks for sharing that! It would be interesting to hear more about that process and how you created the trust and connections necessary for that work.

  9. Also, be sure to check out the latest blog in the series: In the Fog of War. (https://www.aam-us.org/2020/08/12/essential-evaluators-in-the-fog-of-war/)
    It is our great pleasure to have Michelle Moon, a highly respected leader in museum education and community resilience, serve as guest blogger for this edition of Essential Evaluators. “In the Fog of War” focuses on the grassroots movement she has led to document the economic impact COVID-19 closures have had on museum employment.

  10. Kris McCall, Director of Cernan Earth and Space Center, reached out to us to comment in regards to the example from DMNS in the Accessibility Section. He expressed that he liked the question but was disappointed that “Vision” was not an answer option. I reached out to DMNS and received a response and it turns out we have a typo in the post!

    The answer choices on the survey are:
    Physical/Mobility
    Visual
    Hearing
    Developmental
    Intellectual
    Autism spectrum
    Dementia/Memory loss
    Compromised immune system
    Additional
    None of the above

    Thanks to Kris for his comment and for helping us set the record straight!

Leave a Reply to Laureen Trainer Cancel reply

Your email address will not be published. Required fields are marked *

Subscribe to Field Notes!

Packed with stories and insights for museum people, Field Notes is delivered to your inbox every Monday. Once you've completed the form below, confirm your subscription in the email sent to you.

If you are a current AAM member, please sign-up using the email address associated with your account.

Are you a museum professional?

Are you a current AAM member?

Success! Now check your email to confirm your subscription, and please add communications@aam-us.org to your safe sender list.