I’m just kidding with the title; I’m actually a big Atul
Gawande fan. He’s a provocative thinker
with a compelling way of explaining complicated issues, and I suspect he is an
excellent physician. Among other things,
he’s helped elevate the issue of surgical checklists into broader awareness,
such as through his best-selling The Checklist Manifesto. Now, though, some Canadian researchers are
challenging the recent conventional wisdom that surgical checklists actually
improve care.
New research
by Dr. David Urbach and colleagues studied all Ontario hospitals on a variety
of measures, comparing them pre and post-adoption of a mandated surgical
checklist. They found no difference in
mortality or complications that could be attributed to the introduction of the
checklists.
That’s quite a surprise (or, as the accompanying editorial
in The New England Journal of Medicine
called it, a “conundrum”). After all,
the WHO had made surgical checklists a major priority,
going all the way back to 2007 and backed by their own research, which
found almost a 50% drop in mortality and a drop of over a third in inpatient
complications. Other research, such as
those by van Klei and
by Kwok, has confirmed the value of the checklist, although
the former did warn that their research did not find the effect as dramatic as
the original WHO research.
To further complicate things, other recent research,
by Krell, et. alia., raised cautions about even using outcome measures like
risk-adjusted morbidity or mortality to compare hospitals or to compare
surgical outcomes. In many cases, the
authors found, the low numbers of events reduced the reliability of the results. They warned this was especially true when
only using a sampling of cases and without using advanced statistical modeling
approaches.
In other words, we may not even know what we think we know,
which isn’t nearly as much as we’d like to know.
There could be lots of reasons why Dr. Urbach’s study didn’t
produce the same results as earlier studies.
As he
pointed out, the early results stating a 50% drop may have overstated the
effect. "That's a really big
effect," Urbach said. "Very few things reduce something by 50
percent."
Dr. Gawande doesn’t seem too worried about the new results,
and has already responded
to them. He worries that the data it is
based on was “underpowered,” with neither enough cases nor over a long enough
period of time, and he suspects that the major cultural changes required for
such a change to have an impact didn’t have enough time or organizational
backing to produce significant results.
Other experts have also cautioned
that the effect is not so much in just using a checklist as it is in changing
the culture, which was unlikely to have fully happened in the time frame of the
Ontario study. Becker’s Hospital Review
had a nice article
on the challenges of implementing such major cultural changes. In short, it’s not just about the checklist,
not even just about processes in general, but must involve organization-wide
changes. Those are very hard to
make.
We’re still a long way away from truly making patient safety
job #1.
After all, a study
released just last month found a number of failures in adherence to infection
control policies. One in ten hospitals
didn’t have checklists to prevent bloodstream infections, one in four didn’t
have them to prevent ventilator-associated pneumonia, and one in three had no policies
to prevent catheter-related urinary track infections. As Patricia Stone, the study’s lead author, said,
“Hospitals aren’t following the rules they put in place themselves to keep
patients safe. Rules don’t keep patients
from dying unless they’re enforced.”
Worse yet, they may not even have the right rules in place.
If this is progress, I don’t want to know what things used
to be like.
In psychology there a something called the “Hawthorne
effect,” which is a tendency for subjects to perform better on tasks when
being measured. Although the effect and
the original experiments on which it was based have subsequently been largely
dismissed on a factual basis, variations of it still seem to pop up – including
with checklists. Researchers at Tulane
recently released findings
that showed that when patients are informed of, and involved in, surgical
safety checklists, use of and compliance with the checklists improved. As the researchers said, these involved
patients not only feel safer but also likely are safer – a “win-win,” as they
put it.
We should be demanding this kind of involvement.
The Leapfrog Group
has been pushing measuring and publicizing patient safety measures since 2001,
with a nifty hospital comparison tool
and a hospital safety score,
but even they say
progress is too slow. Less than a third
of the hospitals that report on their measures – which is less than half of all
U.S. hospitals – scored an “A” on the safety score in the 2013 results. As Leah Binder, the president and CEO of The
Leapfrog Group lamented, “we are burying a population the size of Miami every
year from medical errors that can be prevented.”
Some hospitals pick
on the Leapfrog scores, especially for getting “penalized” for not
participating (!), but the U.S. News & World Report “Best Hospital” ratings have
more than their share of critics as well, yet those ratings show up in hospital
ads all the time. Anyone seen any
hospitals ads touting their Leapfrog patient safety score? If they’re not, it’s because we, as consumers,
don’t care enough, or understand enough, about it. Shame on us.
The real question is why hospital management everywhere
hasn’t made patient safety more of a focus.
After all, the 2013 HIMSS
Innovation Survey still found cost reduction as the top provider priority
and in terms of actual initiatives. It’s
not that they don’t care about improving patient safety or the quality of care
– those are high priorities as well -- it’s just that those aren’t seen to be
as important as costs. Six sigma/lean
manufacturing experts would argue that sustainable cost reduction comes from a
quality and process improvement focus, not simple cost-cutting.
As patient safety expert Ashish Jha, MD, MPH, recently
told Forbes, “I haven’t heard of
any hospital that went out of business because its care was unsafe. I also haven’t heard about any CEO who got
fired because the hospital’s infection rate was too high. It doesn’t happen, and that’s telling… It’s
not what keeps CEOs awake at night. And
until we get CEOs losing sleep about unsafe care, we’re not going to make a big
dent in the failures of our health care system.”
Several years ago I mentioned (Gambling on Health Care) a point former Senator Tom Daschle
made, that he could get more performance data on virtually any athlete than on
virtually any physician, which certainly says something about our
priorities. Hospital data is somewhat
more available, although definitely not comprehensive, and what data there is
does not tease out important characteristics like which physician(s) and care
teams were involved. Surgeon Smith may
do better overall than surgeon Jones, but in hospital A surgeon Jones may
perform the best…but only with anesthesiologist Sleepy. It’s a mess, but so is the uncoordinated care
system we use every day.
You can easily assemble fantasy sports teams on the
available data sets, but no such luck with trying to pick out your real life
health care team.
Look, I don’t really care about checklists in and of
themselves, or the very technical patient safety measures Leapfrog
collects. Checklists are a tool, not a
magic wand, and like all tools, what matters is how they are used. It sounds cynical, but maybe one reason why
airplane pilots have religiously relied on pre-flight checklists for decades
while physicians can’t quite seem to get into them is because the pilots know
their lives depend on them too.
We need to look at patient safety as though our own lives
are at stake, because, in the final analysis, they are.
I’ll let Dr. Gawande get the final word, from his book Better:
A Surgeon’s Notes on Performance:
“We always hope for the easy fix: the one simple change that will erase
a problem in a stroke. But few things in life work this way. Instead, success
requires making a hundred small steps go right - one after the other, no
slipups, no goofs, everyone pitching in.”
Apparently posting surgical quality measures, as done by Medicare, isn't having much impact on patient choice: http://onlinelibrary.wiley.com/doi/10.1111/1475-6773.12164/abstract
ReplyDelete