Using AI to Listen to Faculty at Scale

In a recent Collaborative on Academic Careers in Higher Education (COACHE) briefing, Jeannie Kim, interim director and interim principal investigator of COACHE at the Harvard Graduate School of Education, and Michele Hansen, associate vice president for institutional research and planning at The Ohio State University, discussed a new framework for using generative AI in qualitative analysis. The conversation focused on how Ohio State moved from skepticism to a strategic, "human-in-the-loop" model to analyze large datasets. 

Breaking the qualitative bottleneck  
Institutional researchers often face a "qualitative bottleneck" when processing thousands of open-ended survey comments. At Ohio State, this challenge involved analyzing a large, complex dataset of faculty comments. Traditionally, this task can take several weeks of coding due to the complexity and length of the faculty comments. By using generative AI as a high-powered research assistant, the university team generated actionable insights in just 48 hours. 

The "human-in-the-loop" framework The Ohio State approach does not use AI on autopilot. Instead, it follows three strict operational guardrails to ensure data integrity and institutional trust: 

• Secure environments: The university uses approved, secure versions of tools like Google Gemini and Enterprise Copilot. This prevents sensitive faculty data from being used to train public models.
• The "verbatim" mandate: To prevent AI "hallucinations," the team uses specific "guardrail prompts." They instruct the AI to identify themes using only verbatim quotes and to provide a table matching every theme to the specific comments that informed it.
• Human verification: The process is iterative. Researchers cross-validate AI results against multiple tools and benchmark them against traditional manual coding. Expert oversight remains the most critical component of the workflow to ensure the authentic faculty voice is preserved. 

Impact on institutional action  
The primary benefit of this streamlined process is increased responsiveness. When qualitative analysis takes months, the window for effective intervention often closes. By reducing the analysis phase from months to days, academic leaders can address faculty needs and retention challenges in real time. 

This case study demonstrates that when combined with human judgment and strict methodology, AI can help institutions honor every faculty voice at scale.