Can you really select a quality nursing home by reading Yelp reviews?

There’s a nursing home the next town over from me, for instance. Nine reviewers have given the place both laudatory five-star ratings (“She has a far greater quality of life than she would have living at home”) and outraged one-star complaints (“The nursing staff, RN’s and CNA’s are caring but overworked”).

Overall, the place gets three stars. It’s a highly flawed measurement, of course. Nine reviewers for a 300-plus-bed facility? Over four years, in an industry known for such sky-high turnover that many of the hands-on staff have undoubtedly left since the first online critic weighed in? Reviewers can deplore the food, but can they know how often residents fall?

Yet gerontologists at the University of Southern California have been looking into Yelp nursing home reviews and think they make a useful addition to the homework any prospective resident or family member needs to undertake.

It’s not that reviews posted on Yelp and other online platforms (Google, Facebook, Caring.com) are such reliable guides to nursing home quality, said Anna Rahman, senior author of a recent article in The Gerontologist. It’s that the supposed gold standard, the five-star ratings on the federal government’s own Nursing Home Compare website, remains so faulty.

“We had a growing sense of how disappointing those measures have been,” Rahman said of Nursing Home Compare. “After 20 years and all the money spent to create it, it’s become a marketing tool. But most people don’t realize how little it measures. It’s garbage in and garbage out.”

A primer: Nursing Home Compare made its online debut in 1998 and added the starred ranking system in 2009, after a senator complained at a congressional hearing that it was easier to shop for a washing machine than a nursing home.

Compare the data

The site contains a trove of information about nursing homes, for those willing to click through to findings like hours of staff time per resident per day, deficiencies discovered during the most recent inspections, and how many residents have bedsores and other indications of poor care. This year, the federal Centers for Medicare and Medicaid Services anticipates the tool will be used 2.4 million times.

Those on-site inspections, which Medicare requires every 12 to 15 months, constitute a key component of the ratings. But the two other categories contributing to the ratings — staffing and quality measures — are reported by the homes themselves. Critics have charged for years that the metrics are untrustworthy and prone to manipulation.

A New York Times investigation in 2014 reported that facilities had learned how to game the system, resulting in rising proportions of four- and five-star nursing homes. A Brookings Institution report two years later concurred that ratings had become inflated.

“I would never use it for quality measures because I don’t believe it,” said John Schnelle, director of the Center for Quality Aging at Vanderbilt University Medical Center. “I think they’re distorted.”

Schnelle, note, serves on the expert panel that advises the Centers for Medicare and Medicaid Services on its nursing home rating system.

Researchers also have discovered that even if the star ratings were accurate, they don’t bear much relationship to how residents and families feel about these facilities.

“Plenty of times residents really liked a facility with a low star rating, and vice versa,” said Robert Applebaum, a gerontologist at Miami University in Ohio.

He has found that families’ and residents’ satisfaction varied significantly at the poles — they did prefer five-star to one-star homes — but the differences in satisfaction between one- and two-star homes, or those with four versus five stars, were negligible.

Nursing Home Compare doesn’t include consumer feedback at all, although a Government Accountability Office report urged the Centers for Medicare and Medicaid Services to add this feature.

“It excludes the most important element,” Applebaum said. “The problem is, it’s really expensive to do. Collecting data from families and residents is no small task if you do it systematically.”

Yelp enters the picture

Enter Yelp, where the first nursing home review appeared a decade ago, and its online competitors, including Facebook. Rahman and her colleagues, looking at 51 Yelp-rated nursing homes across California, found that most reviewers commented on intangibles like staff attitude, caring and responsiveness; they rarely mentioned health care quality or safety concerns.

So perhaps it’s not surprising that in an earlier study, the team found that consumer rankings on Yelp correlated only weakly to the star ratings on Nursing Home Compare.

“They’re not measuring the same things,” Rahman said. In fact, they rather complement one another.

You wouldn’t want to rely too much on Yelp and other online platforms. Relatively few nursing homes, assisted-living facilities and retirement communities have Yelp reviews at all. The 51 California nursing homes in Rahman’s study had garnered just five reviews on average — a tiny, probably unrepresentative sample.

And the great majority were polarized — either five-star reviews or one star. “Nursing homes inspire a lot of emotion,” Rahman said.

That bimodal response isn’t typical in most Yelp categories, but “you’d expect that to normalize over time” as more users post reviews, said Luther Lowe, the company’s senior vice president for public policy.

For now, only 7 percent of Yelp’s roughly 150 million total reviews concern health care: doctors, dentists, hospitals, acupuncturists and reiki practitioners, along with assisted-living and nursing homes.

Yelp uses software to try to weed out fake reviews and sorts roughly one in five into its “not recommended” category, though users can still read them. Probably its most consumer-friendly move, though, was joining forces in 2015 with the nonprofit investigative news organization ProPublica, which created the tool Nursing Home Inspect with federal data.

There are limitations

The bottom line is that all sources have dismaying limitations. Others may be even worse: Nursing homes conduct their own consumer satisfaction surveys, but “they say everybody is as happy as can be,” Schnelle noted. Online placement services like A Place for Mom get paid by the nursing homes.

So experts advise starting your investigation online, using Yelp and other consumer reviews and Nursing Home Compare and Nursing Home Inspect, and talking to friends and relatives who’ve had recent experience with local facilities.

Then, inescapably, you’ve got to show up at the nursing home, walk around, talk to residents and family and staff, ask a zillion questions. Then, go back and do it again.