Last year, President Barack Obama announced his administration would, by the beginning of the 2015 academic year, rate America’s colleges "on who’s offering the best value, so students and taxpayers get a bigger bang for their buck."

Then the president charged the Department of Education with figuring out how, exactly, to build a rating system so that schools that enroll low-income students and give them a good, affordable education would be rewarded and recognized while those that don’t would be penalized and shamed.

This has proved to be a complicated task.

Friday, after delays, controversy and much anticipation, the department released not the ratings themselves, but a broad description of what the ratings will look like, along with an invitation for the public to weigh in on the details.

The department’s struggle to build a workable rating system shows the difficulty of representing the complexity of an individual college’s contribution to society in a single number. It has to be wary of creating perverse incentives to game the system. The antagonism the proposal has provoked among traditional colleges shows why the plan is considered a worthy idea nonetheless.

The newly released rating framework identifies three main elements of institutional quality: affordability, access and outcomes. The data and methods used to define these elements will matter a great deal. Most people support the goal of helping needy students attend college. But who counts as needy? If the ratings use eligibility for the federal Pell Grant program as a proxy, as the Upshot’s College Access Index does, then colleges will be motivated to enroll students from the lowest income bracket. If the ratings look at students along the whole range of income, there will be more incentives to recruit from the middle class.

The department will publish separate ratings for community and four-year colleges and is "considering accounting for differences in institutional characteristics such as degree and program mix and selectivity." It also seeks to recognize "performance improvement over time." This make sense — can we really compare Columbia to CUNY when deciding which is high- and low-performing?

But there’s also a danger of contextualizing each college’s performance to the point of meaninglessness. Colleges will undoubtedly push for the creation of many rating subcategories and sub-subcategories, so that each institution is compared only with others that are almost identical. But what if every college in the subcategory is objectively doing poorly? Obama has promoted the ratings as a tool for students choosing colleges. Does it make sense to steer them toward one that has significantly improved from awful to barely mediocre, or is good only compared with its very bad peers?

The weight given to the three elements will also matter as they push and pull against one another in creating incentives for better college behavior. If too much emphasis is given to the price colleges charge low-income students, colleges might be tempted to give generous aid packages to a handful of token poor students while admitting mostly rich kids. (This is essentially the current business model for many elite schools.)

If, on the other hand, too much weight is given to the number of low-income students enrolled, colleges might be inclined to enroll many such students, cash their federal grant and loan checks and then let them drop out. (This is essentially the business model for many for-profit colleges.) And if the department tries to avoid this by emphasizing graduation rates, colleges could simply lower academic standards and hand out worthless degrees.

That’s why the keystone of any good college rating plan will be a measure of academic quality. Without it, the whole structure will collapse into a jumble of perverse incentives. But, as the department notes, it is difficult to accurately or consistently calculate a direct measure of student learning for campuses that teach everything from physics to Proust.

It’s likely, then, that the department will lean heavily on the value the labor market assigns to degrees from different colleges, by tracking whether graduates can get jobs, earn a living and pay back loans. These measures can easily be created by matching earnings data reported by employers to the Social Security Administration with student financial aid records held by the Department of Education. Graduate school enrollment rates are also in the mix.

Many public university leaders have supported the ratings plan. Other higher education leaders, many representing private institutions, have complained that the federal government’s measures of prices and graduation rates lack nuance — failing, for example, to account for students who drop out of one college and graduate from another. This is true, although they usually fail to note that the reason it’s true is because private college lobbyists had language inserted into the 2008 federal Higher Education Opportunity Act that prohibited the government from building a database to calculate exactly the measures that those same lobbyists are now complaining don’t exist.

In other words, having ensured that any rating system would have limitations, the private higher education lobby says there should be no rating system because of the limitations. Expect more of the same in the public comments the department asked for Friday.

Some college administrators have argued in good faith that the labor market doesn’t capture all of what a college education provides, a point the department concedes. But the proposed ratings will be only one of many sources of college information. With the nearly 70 percent of undergraduates who borrow to pay tuition now graduating with an average of loan balance of almost $30,000, the government wants to hold colleges accountable for their success — or failure — in preparing students to manage their debt.

Behind the technical and substantive critiques of various ratings scenarios is a deep unease in academia with the prospect of government officials making any kind of judgment about whether colleges are doing their jobs well. Traditionally, the federal government has limited its involvement in higher education to subsidizing research and providing financial aid, leaving quality issues up to state governments, accreditors and the free market. The terms of prestige that arose in that laissez-faire environment, in which colleges were rewarded for raising prices and enrolling more affluent students, are sharply at odds with the rating principles the department laid out Friday.

The colleges that will be rated poorly by those standards know who they are and would rather not be exposed. Their opposition is to be expected, and that’s why Obama appears prepared to forge ahead and release an actual set of ratings before the class of 2019 enrolls next fall.

© 2014 The New York Times Company