Insurance Coverage & Costs
Access to Care
Quality & Care Delivery
Quality Improvement & Measurement
Information Technology
Patient-Provider Relationships
Payment Policy
Chronic Conditions
Public Health
Health Care Markets
Issue Briefs
Data Bulletins
Research Briefs
Policy Analyses
Community Reports
Journal Articles
Other Publications
Surveys
Site Visits
Design and Methods
Data Files
Consumer Tolerance for Inaccuracy in Physician Performance Ratings Varies Widely
But Significant Minority of Americans has Low Threshold for Inaccurate Ratings, Especially When Choosing Their Own Doctor or for Pay for Performance
News Release
March 29, 2007
FURTHER INFORMATION, CONTACT:
Alwyn Cassil (202) 264-3484 or acassil@hschange.org
"As doctors and health plans spar over the accuracy of fledgling physician performance rating programs, consumer views about accuracy have been absent from the debate until now," said Paul B. Ginsburg, Ph.D., president of HSC, a nonpartisan policy research organization funded primarily by The Robert Wood Johnson Foundation.
"Consumers differ a huge amount on the level of inaccuracy they are willing to tolerate in physician performance ratings," said Matthew Davis, M.D., M.A.P.P., an HSC senior researcher and coauthor of the study with Judith H. Hibbard, Dr. P.H., of the University of Oregon; and Arnold Milstein, M.D., M.P.H., of Mercer Human Resource Consulting.
"Despite the variation, consumers’ willingness to tolerate inaccurate ratings likely is much higher than physicians’, possibly because consumers see flawed information as preferable to no information at all, while physicians are justifiably concerned that inaccurate ratings will jeopardize their reputations and livelihoods," Davis said.
Based on a nationally representative survey of 1,057 adults aged 18 years and older conducted Dec. 14-18, 2006, the study’s findings are detailed in a new HSC Issue Brief—Consumer Tolerance for Inaccuracy in Physician Performance Ratings: One Size Fits None—available here.
The survey measured consumer acceptance for measurement error in physician performance ratings for four applications: releasing ratings to the general public, using ratings to choose one’s own primary care physician, using ratings to vary physicians’ payment rates based on performance, and using ratings to encourage consumers to seek care from more highly rated physicians via tiered-benefit plans.
Regardless of the use of physician performance ratings, consumers reported a wide range of tolerance for rating inaccuracy. The most common response for each application of physician performance ratings was low tolerance for inaccuracy (5 percent or less), and for three of the four uses at least 40 percent of consumers had low tolerance for inaccuracy. On the other hand, more than 20 percent of consumers were comfortable with inaccuracy of 20 percent or more across all four uses.
Consumers were relatively tolerant of inaccuracy when ratings were used for public reporting and tiered networks. Consumers demonstrated the lowest tolerance for inaccuracy in two circumstances—using ratings to choose their own physicians and insurance plans paying physicians differently based on ratings.
"Major limitations in readily available clinical data sources and resulting performance measures make it likely that error in most individual physician ratings exceeds 5 percent," Davis said. "This raises a question: Should plans publicly report the level of rating inaccuracy in their performance measurement systems?"
However, communicating the accuracy of ratings, along with all the complexities
associated with interpreting the meaning of the ratings, will increase the complexity
for consumers, according to the study. If a substantial proportion of consumers
decline to respond to ratings, plans and provider leaders will need to improve
the accuracy of ratings or consider other means of trying to improve care.
| ### ### |
The Center for Studying Health System Change is a nonpartisan policy research organization committed to providing objective and timely research on the nation’s changing health system to help inform policy makers and contribute to better health care policy. HSC, based in Washington, D.C., is funded principally by the Robert Wood Johnson Foundation and is affiliated with Mathematica Policy Research, Inc.
