The Truth About Peer Selection in the Academy

by Todd Benson

 

As we near the end of our survey administration cycle, our team has begun working on institutional reports. COACHE’s reports are distinctive in that they allow our partners to select five peer institutions for more direct comparison. That kind of nuanced comparison can be both a blessing and a curse for our partners. The benefit of having near peer comparators comes from the ability to see beyond the national benchmarking. They provide context that helps explain why your institution may be over- or under-performs in the national landscape. The challenge that comes with this sort of data is that it can create the opportunity to dismiss the findings when the peers aren’t “perfect.” In truth, there are no perfect peers. There are choices with consequences that need to be considered and communicated throughout the rollout process. To that end, I wanted to share some thoughts about peer selection based on my experience with our partners.

 

The Context of Comparison is Key

Many institutions have an officially designated peer list that may be the sole source for choosing comparative data. Often, these peers are developed for many uses but the rationale for choosing those peers needs to be consistent with the purposes of the analysis. Using peer institutions that were developed with student metrics in mind might not make sense for a faculty study. We often see this in the case of regional peers. Institutions choose peers based on geographic location which can make sense with students. But in the case of faculty, I would argue that geography matters less.

 

For Faculty, Peers are Symbolic

In higher education, we have a tendency to say that rankings and comparisons aren’t important but then immediately click on the US News and World Rankings when they are released. For faculty, identifying peers says something about their own quality. As the arbiters of the rigor at their institution, a step down in comparators can seem indicative of a decline in their own quality. That is why, in my experience, the most common instinct is to be aspirational.

 

Being aspirational is fine as long as you are transparent about the decision. Quite often when institutions pick aspirational peers, they end up with less than stellar results – and they are surprised. Why would anyone compare themselves to aspirational peers and expect to outperform them? The answer is, they shouldn’t (but often do). Choosing aspirational peers is an opportunity to highlight where the biggest gaps are between where an institution is and where it hopes to be. It can improve focus but only if the institution can move past the idea that it won’t be at the head of the class.

 

Strategic Engagement of Stakeholders Pays Off

One way to reduce resistance to peer choices is to engage faculty in the peer selection process. This isn’t to say that an institution should post a list of eligible peers and have faculty vote. Instead, I encourage campuses to ask faculty what types of data would help them to compare one campus to another in terms of the faculty workplace. Is it about resources? What about selectivity? Does disciplinary make up matter? They are all choices that can inform the analytics that you use to select peers. The important thing here is to engage faculty in the process so that they can trust the results when they come out later. Between the time of peer selection and reporting out, make faculty aware of how those decisions were made. Share back with faculty the criteria that you used and how were they were weighted.

 

Imperfection is Unavoidable

Like all research, there are limitations. Yet we do not dismiss every research article because the authors are transparent about those limitations. In fact, when good research is forthright about limitations, it creates the opportunity to nest those findings within a context. The challenge is to understand and accept those limitations without using them as an excuse to dismiss the findings.

 

Peers Can be Used for More than Just Comparative Data

Often we hear of partners searching for best practices via google. The problem with that approach is that there is no data to substantiate the quality of the practice. But at COACHE we use our actual survey data to identify practices that work. To that end, something that our partners are engaging in more often is reaching out to their peers for advice and collaboration on the issues that matter most. COACHE has brokered discussions on sharing results with Deans, interdisciplinary work, and tenure and promotion policies. Each time, that we get to sit in on these discussions, we are impressed by the level of candor. In these discussions, our partners talk about what works but more importantly, they talk about what didn’t work. They point out the stumbling blocks so that others can avoid them.

 

When it comes to peer data sharing, higher education is an anomaly. Very few other industries share as much with competitors as we do. To leverage that industry advantage, we have to be thoughtful and transparent about how we choose and use peer data and our peer networks.

See also: Todd Benson