A Six-Metric Framework

By Tyler Small, AI Transformation Advisor

As artificial intelligence reshapes virtually every sector of society, universities face a defining question: how do we know if our institution is actually transforming, not just experimenting? Most existing frameworks focus narrowly on individual assignments or classroom-level AI usage policies. They address whether students are allowed to use AI tools on a given quiz or how much AI assistance is appropriate for a particular paper.

That is not what this framework measures.

This document presents six metrics designed to evaluate the depth and progress of AI transformation at the institutional level, measured by course, department, or program. These metrics were developed to fill a critical gap: no university has yet achieved a recognizably successful, measurable AI transformation. This framework is intended to help institutions become the first.

Strategic Context: Why This Framework Exists

The fundamental problem driving this framework is straightforward: universities have historically prepared students for entry-level work. That entry-level work is disappearing. AI can now perform much of the simpler, repetitive work that once served as the on-ramp to professional careers, and it can do so at a fraction of the cost of a human worker.

The trends indicate this will continue. As a result, students graduating today increasingly need something universities have not traditionally provided: real, paid, stakeholder-facing work experience. Not just papers and tests. Actual experience demonstrating they can deliver value to employers at a level that previously required two to three years of post-graduation work.

This is not the "new entry level." It is a mid-career skill set that students must begin building during their education. The path to that level is not through more exams; it is through genuine workforce experience supported by faculty, acting as coaches and mentors. To enable this strategic solution, we can use AI to free up the time and capacity needed to create these experiences, and then to measure whether the transformation is actually working.

The six metrics below are the measurement system for that transformation.

The Six Metrics

Metric 1: Faculty and Leader Beliefs

This metric measures the degree to which faculty members and institutional leaders believe that AI is a critical necessity, not merely a useful tool, for fundamentally transforming the model of education itself.

There is an important distinction between surface-level and transformational belief. Surface-level belief sounds like: "Our students need to be AI literate" It treats AI as a subject to learn about. Transformational belief sounds like: "We must fundamentally change our educational model because of AI." This metric targets the latter.

Both faculty and leadership are assessed separately, as their beliefs often differ, and both groups are essential drivers of institutional change. AI analysis of current syllabi can quantify the intensity of belief across courses, departments, programs, and universities, making it possible to identify where change management efforts are most needed. Still, scores will be descriptive and considered relative to the discipline. Computer science programs may transform faster than others, for example, because entry level jobs have already disappeared en masse.

Metric 2: Student Experience & Faculty Experience

This metric serves as both an anchor point and a quality control mechanism for the entire framework. As the institution automates more faculty tasks and redesigns course delivery using AI, student satisfaction ensures the experience does not degrade. At the same time, by making faculty experience an explicit focus, the framework ensures that transformation is something faculty genuinely embrace, not something imposed on them. When faculty feel empowered, supported, and confident in the changes they are making, they become advocates for the transformation rather than obstacles to it.

The measurement captures students' and faculty's overall satisfaction across the same categories used to track faculty automation in Metric 3. This alignment is intentional. If a particular task is automated and satisfaction drops in that same area, among students, faculty, or both, the data surfaces the problem quickly and precisely. Student satisfaction is the primary signal: automation should never come at the expense of the quality of what students receive. Faculty satisfaction is the secondary signal: transformation should never come at the expense of the people being asked to lead it.

Metric 3: Faculty Task Automation

This metric measures the extent to which faculty members are using AI to automate the tasks that make up their professional workload. The goal of this automation is not the reduction of faculty involvement; it is the liberation of faculty time for higher-value activities, particularly mentoring students on real-world, paid work. As a side benefit, faculty who demonstrate expertise automating less important tasks will be more competent, confident, and credible to teach AI to their students.

The categories of automation include course development and enhancement, assessment creation, delivery of instructional content, leading discussions, answering questions, student mentorship on projects, assessment proctoring, and grading. Each of these is an area where AI can take on partial or full responsibility, reducing the routine burden on faculty. Critically, these categories are designed to mirror those in Metric 2 (Student Satisfaction), enabling a direct comparison between what faculty are automating and how students and faculty are experiencing those specific changes.

Task automations that are completely successful will be those which maintain high satisfaction scores from both faculty and students. This may dictate that automation pause short of what is technically possible. It doesn’t mean we shouldn’t automate or that we should automate everything. It means we can use a complete feedback cycle to measure the effectiveness of each step we take in using AI to improve outcomes for all parties.

Metric 4: AI-Integrated Value-Creation Learning Experiences

This metric is based on the percent of coursework that is composed of doing work for an industry client. There are two forms of this. One is actual work done for a real organization, including a university department, a business, or a non-profit organization. The second form of workforce experience is a proxy or hypothetical; a simulation of a real work experience. This could be as simple as a story problem in a math class, or a strategic analysis done for a hypothetical company. The most advanced and valuable type of learning experience is one where a stakeholder is served and value is created for their customers.

For example, a class of students first creates three strategic options for statistically improving customer satisfaction, then deploys two of them in two different parts of a library, and measures the difference.

A simulation may be counted as 20% of the value of an experience working with a real client. Why? Connecting and working with real clients can be five times more difficult (or more) than an isolated, fictitious simulation that is completely within the professor's control.

The enabling factor for real client work is generative AI. AI agents can be configured with specialized information and instructions to act as a coach, advisor, source of information about best practices and processes, etc. In fact, we are beginning to believe that it is only with AI that this type of workforce alignment is possible to sustain. Otherwise, it tends to quickly burn out the instructor; especially when the instructor doesn’t have a strong background in consulting. But with properly designed AI agents, the tables turn and students can be infinitely more supported.

Metric 5: Student Earnings During Enrollment

This metric measures the tangible market value of the skills being developed, not just academic performance.

It measures how much money students earn during a semester as a direct result of work performed as part of their coursework. It is a relative metric, contextualized by the nature and level of the course. Certainly not every learning experience warrants or merits payment. However, as students gain valuable expertise through years of industry experience, there comes a point of ethical fairness where it no longer makes sense for them to work for free.

Even a modest example demonstrates the principle: a business pays $400 for an ethical analysis conducted by an ethics class of 20 students. Each student earns $20. That may seem small, but it represents a real transaction, a real stakeholder who valued the students' work enough to pay for it. The course became more meaningful than merely reading about what dead philosophers said. The course became applying what the philosophers said!

At higher levels, the numbers scale accordingly. A senior-level course, taught by a professor who has been building these students’ skill sets since the students' freshman year, might generate hundreds or even thousands of dollars per student. Over four years, a student could potentially earn back a meaningful portion of their tuition; making school pay for itself upon graduation.

Metric 6: Employer Satisfaction

This metric captures how satisfied employers and external stakeholders are with the quality of work delivered by students during their coursework. It is the external validation of the entire framework.

Employer satisfaction scores are collected from the businesses or organizations that receive and pay for student work products, whether that is one employer for an entire class project or multiple employers matched with individual students or groups. The average of those scores constitutes the metric.

Paired with Metric 5, employer satisfaction tells a complete story. High student earnings combined with high employer satisfaction confirm that real value is being created. High earnings combined with low employer satisfaction signal a mismatch: students are being paid, but the quality of their work did not meet expectations. This data is actionable for faculty, students, and institutional leadership alike. At its broadest level, it measures the university's effectiveness at preparing students for the workforce in an AI-transformed economy.

How the Six Metrics Work Together

These six metrics are not independent measurements. They form an integrated system, and each one depends on the others to be fully meaningful.

Metric 1 (Faculty and Leader Beliefs) establishes the foundation. Transformation cannot happen if faculty and leaders do not believe it is necessary. This metric identifies where cultural and change management work is most urgent.

Metrics 2 and 3 (Student Satisfaction and Faculty Task Automation) work together as a check-and-balance system. Faculty are given a clear mandate to automate lower-value tasks, while student satisfaction scores ensure that this automation does not degrade the quality of the experience.

Metric 4 (AI-Integrated Value-Creation Experiences) connects learning with economically and socially valuable action. It eliminates the entire “transfer” problem from higher education. This acts on a check for metric 1, builds on metrics 2 and 3, and sets up the rationale for metrics 5 and 6.

Metrics 5 and 6 (Student Earnings and Employer Satisfaction) measure whether the transformation is achieving its ultimate purpose: preparing students for an AI-transformed workforce by giving them real, paid, stakeholder-validated experience before they graduate.

Together, these six metrics enable a university to build a dashboard that answers the question no institution has yet been able to answer with rigour and specificity: How AI-transformed are we, really?

Conclusion

No university has yet achieved a recognisably successful, publicly documented AI transformation. This framework is designed to change that by giving institutions the measurement tools to know what transformation actually looks like, track progress toward it, and demonstrate results to students, employers, and the broader public.

The six metrics described here are the foundation of a dashboard that can serve as a model for universities across the country and beyond. The institutions have an opportunity to lead, not by experimenting with AI at the margins, but by measuring and driving transformation at its core.