WHYY/NewsWorks health reporter Elana Gordon has been looking into the new Consumer Reports ratings of how hospitals do on elective surgeries.  In this Q and A with Health and Science Desk colleague Maiken Scott, Gordon explores the tricky question of how much stock to put in such ratings:

As your first story about the Consumer Reports ratings noted, the organization mined Medicare billing data to find out which hospitals have the best results for elective surgeries, such as knee or hip replacements. Several area hospitals were named in the report.  How did they react to the findings?

All of the hospitals I talked with said access to data is a good thing, but as you might expect there were mixed reactions to the report itself and its methodology. The report, by the way, found a lot of variation in elective surgery quality. So some of the more prestigious hospitals actually did worse, while some lesser known, rural places did better. Fox Chase Cancer Center, which did better, thought the report was a good indicator for where a hospital might be in terms of quality. I talked with some area hospitals who fared poorly, and they all warned that the data wasn't necessarily reflective of overall care. They cautioned against relying on any one report in making a decision about where to get care.

This looked at Medicare data – that means just older patients?

Yep. Medicare is the federal health program for seniors 65 and older. From what I understand, the billing data is some of the most comprehensive information that's out there. The challenge is how you analyze it, and it can get complicated.

So, is this report helpful to me, when I try to pick a hospital?

Great question – yes and no. What the people I spoke with for this story told me is you can glean some information from the report, but it's not the be-all-end-all. You can't just rely on one report to make a decision about care, though it might be a useful part of the conversation you have with a doctor. One challenge is there are a lot of reports out there. I talked with Jordan Rau, a reporter at Kaiser Health News, not related to Kaiser Permanente – he has done a lot of reporting on all kinds of health surveys and public data.  He told me that companies looking to rank the same hospital measure come out with contradictory results, or a hospital may rank really well one year in a report and then poorly the next, perhaps because the reporting group tweaked its methodology. Some rankings may rely on physician surveys, or hospitals have to pay to participate. So the bottom line from what people told me is a report like this may be a useful tool in the process of determining where to go for care, but shouldn't be seen as a stand-alone source.

As a patient trying to pick a hospital for a specific surgery- what kind of data CAN I get my hands on? I know hospitals track outcomes and complications and have discharge data, can I see that?

You can't see all of it. The state hospital association and others I spoke with recommended the Pennsylvania Health Care Cost Containment Council as a good state resource. This Consumer Reports study actually has a paywall, but you can find some highlights here. A federal Medicare website, Hospital Compare, also has information, like hospital mortality rates for heart attacks, heart failure, and pneumonia. Rau, the reporter with Kaiser, tells me it's conservative with what it labels as an outlier, for say an infection rate, so you have to be really off the charts to be flagged for something. The Centers for Disease Control also has information on infections, but that's self-reported information, so you have to hope hospitals are telling you everything.

Oh, wow. It sounds like this is all really complicated.

I know! One thing that many people see as promising, is with this move toward digitizing medical records, and increased focus on reporting an data, we may see a shift toward more public data that, with time, could become better and more useful, especially as these methodologies for analyzing it get tweaked. One would hope that might create opportunities for better answers to questions about hospital quality – questions that, when you think about it, are a lot easier to find out when you're trying to assess which car or refrigerator to buy.

You can listen to Elana Gordon's full conversation with Maiken Scott on the NewsWorks Tonight's Reporters Roundtable segment from Friday, Aug. 2.