How Data Triangulation Can Convert Skeptics to Supporters

By Todd Benson

Data TilesWhen we ask our partners “What’s going to get in the way of working with your faculty around COACHE data?”, a frequent response is that faculty are questioning the results. But to ensure that faculty fully engage with data from the Collaborative on Academic Careers in Higher Education (COACHE), it is vital that everyone trusts the results — including any skeptics.

Skepticism, in my view, can be both a benefit and challenge. Faculty are trained to be skeptical about research, and rightly so. If your faculty were reviewing an article for publication or a dissertation for defense, you would absolutely expect them to ask tough questions about the methods of data collection, the analytic approach, and the subsequent findings. They would be derelict in their responsibilities if they failed to ask pointed questions in those settings. Do we really expect them to turn off that part of their brain when we share the COACHE results? I hope not.

In fact, I think you should be tapping into their skepticism. Leaning into those challenges will make your work better because in the long term, managing skepticism constructively will help faculty engage in the problem-solving phase of the work.

That is why, we have been pushing our partners to triangulate their COACHE results with other sources of institutional data and in doing so deepen understanding about the issues, cultivate more trust and credibility, and make a more compelling case for action.

Leveraging additional data sources to support the COACHE process

Colleges and universities have volumes of data about faculty and the environment where they work. Yet, we tend to compartmentalize the data. I’d argue that we would be much more effective at cultivating trust with faculty if we stripped away some of the artificial barriers and let the data crosspollinate.

Here are just two examples of how this might take shape:

  • If COACHE data suggests a lack of clarity in promotion policies, we encourage institutions to look at how many faculty have gone up for promotion in the past few years. Are some disciplines more successful at putting their faculty up for promotion than others? COACHE data is about faculty perceptions. Aligning those perceptions with behavioral data (number of applications for promotion), one might be able to make the case for a policy review or targeted interventions.
  • If the data suggests differences by gender in the support faculty receive for teaching, we can look at the number and types of courses taught by men and women.  Perhaps women are teaching more introductory courses and would like the stimulation that comes from teaching advanced students? You have the data to help you see if a pattern like that exists.

All of this is good in theory, but it is also labor intensive. Below are a few strategies that the team working with COACHE data might consider.

Strategies for Accessing Additional Data

  1. Start by befriending your Institutional Research staff. Work with them on developing a directory of readily available data to support your interests.
  2. Don’t stop at Institutional Research. Consider other offices that may be collecting data about faculty too. Centers for Teaching and Learning (CTL) might have some sense of who attends their workshops? Knowing who attends (and who does not attend) CTL events may lend some insights into who needs additional support. Your Chief Diversity Officer is most likely engaged in some form of data collection too. Reach out and ask them.
  3. Take the time to ask yourself “What types of data would help me understand this issue better?” Make a data wish list and consider whether to include that in your final COACHE recommendations.

An ancillary benefit that comes from reaching out to other offices is that you are showing them how the COACHE results can serve their interests, and they may find the results can inform their practice too. When other offices get excited by the COACHE results, it increases the number and quality of uses for the data. Distributing the COACHE results to other offices that work with faculty increases your impact with a smaller investment of your time.

The work of integrating disparate data sources requires time and thoughtfulness but, in my opinion, it will improve the quality of your analysis and demonstrate to your faculty that you are taking the COACHE results seriously.

If you are seeking opportunities to leverage data triangulation at your institution, please don’t hesitate to reach out to the COACHE team for ideas and support.