Practicing in an Age of Uncertainty

If you've ever had a hard time trying to decide what's best for your health (e.g., Sorry, There's Nothing Magical About Breakfast), perhaps you can take comfort in the fact that physicians often aren't so sure either.

Or perhaps not.

A new study in Annals of Surgery, and nicely reported on by Julia Belluz in Vox, focused on surgical uncertainty.  The researchers sent four detailed clinical vignettes to a national sample of surgeons, seeking to get their assessment on the risks/benefits of operative and non-operative treatment, as well as their recommendations. You'd like to think there was good consensus on what to do, but that was not the case.

In one of the vignettes, involving a 68 year-old patient with a small bowel blockage, there was fairly universal agreement -- 85% -- that surgery was the best option.  In the other three vignettes, though, the surgeons were fairly evenly split about whether to operate or not, even on something as common as appendicitis.

So, there may be a "right" answer but you might as well flip a coin in terms of getting it, or there may just not be a right answer.  Both options are troubling.

The authors believe that surgeons are less likely to want to operate as their perception of surgical risk increased and the benefits of non-operative treatment increased, and more likely to want to operate as their perception of surgical benefit increased and non-operative risk increased.  The problem is that surgeons vary dramatically -- literally from 0 to 100% -- on their perceptions of those risks.

Vox quoted Ashish Jha, a physician and professor of health policy at Harvard, as saying: "The truth is that most of the surgeons in their sample are quite experienced, and yet have wildly different assessments of risks and benefits among similar patients."  Dr. Jha called the findings "deeply disturbing," noting that most of us are bad at evaluating risk -- and surgeons are no exception.

Most surgeons based their estimates of risks/benefits on their experience, their training, and -- if you're lucky -- on whatever literature might be available, but it is doubtful that we can usually expect an objective, quantifiable assessment.

The American College of Surgeons has developed a "surgical risk calculator" to help surgeons better gauge these risks, using data from a large dataset of patients.  However, an earlier related study from the same team of researchers found that it doesn't make much difference.  The calculator did narrow the variability of surgeons' assessment of risk, but: "Interestingly, it did not alter their reported likelihood of recommending an operation."

Oh, well.

The lead author told Medscape that 45% of the physicians surveyed had not been previously aware of the calculator's existence, even though it has been available for several years.  And only 6% of those who had been aware of it reported using it routinely.  All in all, it would appear that risk assessment may not be the core of the problem.

It is not just surgeons who aren't always sure of the right course of action, of course.  A study in the American Journal of Managed Care found that 62% of physicians reported that they found the "uncertainty involved in providing patient care disconcerting."  The discomfort with uncertainty did not vary appreciably between type of specialty.

The study looked at how the Choosing Wisely initiative has helped physicians reduce use of unnecessary tests and procedures, and the answer appears to be, not very much.  Ninety-six percent of physicians who were aware of Choosing Wisely did agree that it was a legitimate source of guidance, but almost 60% of physicians weren't aware of it.  Moreover, even among physicians aware of Choosing Wisely, almost 30% (46% of surgeons) felt it hadn't "empowered" them to reduce such tests and procedures.

Physicians want to do the right things.  Ninety-seven percent agreed that doctors should limit unnecessary services, 98% believed they should be aware of/adhere to clinical guidelines, and 92% felt physicians have responsibility to control costs,  Only 31% felt there was too much emphasis on costs, and only 28% agreed that doctors were "too busy" to worry about cost.

On the other hand, only 37% felt they understood the costs of the tests and services they order, and only 21% thought "physicians across specialty are likeminded in their commitment to reducing unnecessary treatments."  Physicians reported also pressure from patients, both for ordering more tests and procedures (68%) and for referrals to consultants (52%).  Almost half (46%) admitted ordering tests/procedures out of malpractice concerns.   The problem isn't just with physicians.

Then there is the example of PSA tests.  In 2008 the US Preventive Services Task Force recommended routine PSA tests not be given to men over 75, and in 2012 broadened that recommendation to all ages.  Yet data suggest that the group least likely to need the tests -- men over 75 -- had the smallest declines in rates of testing.  Almost 40% of this age group are still getting the test, which is not far from the previous rates.

As one researcher told The New York Times,   "That’s just insanity...bad medicine, poor use of health care resources and poor decision-making.”

There's all too much of that in our health care system.

We all share in the problem.  As Dr. Jha said, humans are usually not good about evaluating risk.  We're especially bad when there isn't good -- and understandable -- information to help inform our decisions.  Health care is just not an industry that uses data well, nor one that is very transparent about the data that does exist.

Look at the recent controversy over CMS wanting to move to a simpler, star rating for hospitals, which the industry managed to block.   As noted health policy guru Michael Millenson recently wrote, "Individual hospitals and the industry remain quick to point out flaws in others' report cards to the press and policymakers while keeping mum about the wealth of comparative quality information they use internally."        

It's not so clear to me that there is such a "wealth" of information already available, especially not at the individual hospital/physician/condition levels, but there should be, and it should be better used.  Until then, we'll have to keep living -- and dying -- with the uncertainty.

Figuring Out the Code

Three stories about computer programming caught my attention this week.  Admittedly, this is out of my wheelhouse; while I like to think I could have learned to program, the fact is that I never did, which probably makes me like a lot of you.  We all use products and services that rely on such programs, but most of us probably don't give them much thought.

I promise I'll relate this to health care, but first the three stories.

The first was in Wired, about something called the Mover Kit.  The Mover Kit is made by an interesting organization called Technology Will Save Us, whose mission is "to spark the creative imagination of young people using hands on technology."  Their previous efforts let kids create their own electronic music or their own games, but they've just announced a Kickstarter campaign for the Mover Kit, which allows kids to make and code their own wearable devices.

As they describe it, "The Mover Kit is an intuitive way for kids ages 8 and over to learn the fundamentals of electronics, programming, and solve problems creatively.  It encourages kids to learn by doing what they do best -- being active and playing!"  TinyBop's Infinite Arcade uses a similar approach, letting kids design and build their own video games.

Pretty cool huh?

CIOs are wailing about the shortage of IT talent, calling it an "existential threat" to their companies' futures.  They are realizing that they need new strategies to attract this talent.  Getting kids to learn how to program while they think they are having fun sure seems like a good one.

The second article was an op-ed in TechCrunch by Basel Farag, Please Don't Learn to Code.  On first blush, it would seem to be giving the opposite message that the Mover Kit and Infinite Arcade are promoting.  However, Mr. Farag's message is more nuanced than that.

He concedes that programming skills are important, but not for everyone.  As he says, "I would no more urge everyone to lean to program than I would urge everyone to learn to plumb" (alas, yet another skill I never acquired).

His key point is that the crucial skill is understanding the problem, especially since programming methods and languages can change rapidly.  There may be a shortage of people who can program well, but I'm willing to bet that there is a far greater shortage of people with the analytical skills to truly get to the bottom of problems.

I've already written about how this may be especially true in health care.

The third story suggests that the fuss about programming may be short-lived.  Writing in Wired, Jason Tanz predicts that the rise of artificial intelligence and machine learning means that soon we won't program computers; we will "train" them.

He gives an example.

If you want to teach a neural network to recognize a cat, for instance, you don't tell it to look for whiskers, ears, fur, and eyes.  You simply show it thousands and thousands of photos of cats, and eventually it works things out.  If it keeps misclassifying foxes as cats, you don't rewrite the code.  You just keep coaching it. 

Gosh, and I thought debugging a program was hard; imagine trying to coach it!

This is not futuristic, pie-in-the-sky thinking; this is already happening.  Ask Google, Apple, or Facebook.  One consequence of computers, in essence, programming themselves is that their supposed masters -- i.e., us -- don't and won't always know exactly what they've done or how it works.  As Google's Andy Rubin told Mr.  Tanz: "After a neural network learns how to do speech recognition, a programmer can't go in and look at it and see how that happened."   The programs become, as Mr. Tanz says, a "black box."

It may sound scary to have our society relying on programs that we didn't write and can't understand, but Mr. Tanz views this optimistically.  He sees it as democratizing programming; it no longer will depend on nerds writing in arcane computer languages, but rather on people figuring out how to get programs to learn what we want them to do -- e.g.,  "It will be accessible to anyone who has ever taught a dog to roll over" (something I have, yet again, never done).
  
For me, the commonality of all these articles is the importance of figuring things out.  We tend to view programming as a hard thing we leave to other people, but it's easy to forget that programming isn't the problem itself, but merely a means towards solving the problem.

Health care is full of black boxes.  As much as we think we've learned about the human body over the last hundred years, we're still constantly reminded about how little we actually understand its working (e.g., the microbiome).   As much time and money we spend training physicians, much of how they diagnose and design treatments for patients remain a mystery.  And does anyone know why we always have to fill out so many damn forms?

The many organizations working on applying AI to health care are trying to figure out some of these black boxes, although their solutions may come at the price of new black boxes.  I hope, though, that we don't just turn things over to AI.  We still need people to figure out the problems.

That's why I love reading about initiatives like MakerHealth, about which I could have written an entire post.  It started out as MakerNurse, based on founder Anne Young's realization that front-line nurses could design tools to help improve patient care.  Launched out of the Little Devices lab at MIT and supported by the Robert Wood Johnson Foundation, it now has mobile spaces in numerous hospitals and a dedicated MakerHealth Space in the University of Texas Medical Branch in Galveston.  Their goal is to empower not just nurses but all staff.

They believe in "democratizing the tools health making around the world.  Whether it's an hospital bed, or a smart pill bottle, or a improved triage mobile phone app we believe that design should be transparent, hackable, and enabling for everyone to be the designers and makers of their own healthcare solutions."  The word that often seems to be associated with them is "MacGyver."

Health care could use fewer programmers and more MacGyvers.  

DIY Health to the Rescue

Experts often compare how slowly the health care system is "reforming" to how hard it is to turn a battleship.  They're so big that they can't turn on a dime (much less on $3 trillion!), and there is as much risk in trying to oversteer as in not turning at all.  Things are changing, we're assured, but it will take time to get on the desired new course.

Maybe.  But maybe it is time to jump off the obsolete battleship onto something more nimble.

Some call it Do-It-Yourself Health (there are both .org and .com sites devoted to the topic, among others).  PwC declared it to be one of the top ten trends of 2015.  Dave Chase believes that "DIY health reform is now leading the way for the highest performing reform" -- not Medicare, health insurers, not even employers.
 
Americans spent some $34b on "alternative and complimentary" medicine way back in 2009, even without firm proof that they work, so imagine what we might be willing to pay for "traditional" care that was more convenient, more self-directed, and faster.  

In case anyone still needs confirmation about how slowly our health care system is edging towards reform on its own, some cases-in-point:

• A new study has updated the famous IOM 1999 estimate of 100,000 annual deaths due to avoidable medical errors, believing that the number is more like 250,000 -- making medical errors the third leading cause of death in the U.S.
• The Senate Finance Committee has issued a report warning that physician with ownership interests in medical device distributors are -- what a surprise! -- much more likely to perform surgery using those devices, .  This is coming from a bunch of Republicans, mind you, so the problem must be bad.
• The U.S. has spent some $35b to move providers to EHRs, but as Vice President Joe Biden just said, "we didn’t realize five [EHR] companies would create their own silos. What the heck are we doing?"
• The move to value-based purchasing is supposed to help combat our perverse incentives, but early indicators are not so positive.  Two new studies looked at Medicare's Hospital VBP program, and found that (a) it ended up rewarding not just low spending but also low quality hospitals, and (b) it doesn't seem to actually have any positive impact on patient mortality.  

What the heck are we doing indeed.

I'm increasingly thinking that the solution will not be to reform the system, but to work from outside it.  I offer a few examples:

The Wall Street Journal wrote about one in Tech Savvy Families Use Home-Built Diabetes Device.  It profiles how, by WSJ's count, some 50 families have developed their own artificial pancreas device to help themselves or their children manage their diabetes.  Fifty families in a big country like the U.S. doesn't sound like many, but, hey, I'm amazed there are 50 families bold enough and capable enough to build such a device.  Put those specs on the Internet, and maybe an instructional YouTube video or two, and that number could easily skyrocket.  

It wasn't prescribed by their doctor, and it wasn't purchased from a medical device company, because such devices aren't yet on the market.  Several medical device companies are working on similar devices, and the FDA says approving them will be a priority, but nothing is through the regulatory process yet.  The FDA can't stop individual families from building their own, as long as they don't them sell or further distribute them.  

So, it is a device highly desired by the market, and the technology is simple enough that people can literally build them at home, yet it is not available within our health care system.  Something is wrong with that picture.  

Then there are hearing aids.  If you know anyone with a hearing aid, you're probably heard them complain about how expensive they were, on average about $5,000 for a pair.  The New York Times reports how those prices are under attack from the consumer electronics industry, which is selling products under the unwieldy name of "personal sound amplification products," or PSAPs.

To escape FDA regulation, PSAPs can't be marketed as hearing aids or even purport to be used for hearing loss (which would make it a medical condition), but canny consumers are looking at what PSAPs can do, weighing their much lower cost, and using them anyway.  The audiology profession warns about such self-diagnosis and the limits of PSAPs, but it's not like they're doing such a bang-up job already: it is estimated that less than a third of the people who could benefit from hearing aids actually use them.  

The competition seems to be working; The Street reports that hearing aid prices are starting to tumble.  

If only we had "competition" for other services from outside the traditional health care system.  Oh, wait, we do.  For example:

• You can now use your smartphone to monitor lung function in lieu of going to a doctor's office to use a spirometer.  
• You may soon be able to use your smartphone to monitor heart palpitations  instead of using existing event monitors.
• You can use your smartphone to monitor your blood pressure.  

I'll bet medical device manufacturers, the FDA, and even some clinicians are eager to wrap these in under the cloak of our existing health care system, which would no doubt end up making them both more expensive and harder for us to obtain.  Let's hope they are not successful (although we still need to demand proof of efficacy).    

Some see a future where we sequence our own DNA to identify genetic risks, have implantable health monitors, do at-home diagnostic tests, even 3D print our own prescriptions.   Throw in peer-to-peer financing for big bills (more on that in a future post) and more providers selling their services DTC (like direct primary care or concierge medicine), well, I'm not sure we'll even need to be on the stodgy old battleship anymore.  

You have to believe that Walmart, Walgreens, and CVS are preparing for a DIY future.  Other health care providers are going to have to decide if they will do so as well -- or risk going down with the battleship.

Health Care Is Better as a Game

So many counter-intuitive findings recently.  For example, a new study claims 7 of the 10 most profitable hospitals in the country are "non-profit."  Let me say that again, most profitable hospitals in the country are usually nonprofit.  Or, despite the drive to improve surgical quality by limiting surgeries at low volume hospitals, it appears that the relationship between volume and patient outcomes is not as clear as had been thought, once "more advanced statistical modeling" is used to analyze the data.  Wait, what?

Either one of these would be a good topic to write about, and many others have done so already (e.g., KHN and Modern Healthcare, respectively).  Instead, I'll talk about something equally counter-intuitive but more fun: how important games might be for health care.

Let's start with what's going on with tuberculous.  About one-third of the world's population is infected with TB (many of whom don't know it).  It is one of the leading infectious disease killers.  Surprisingly, though, it is not particularly easy to diagnose, and researchers at Stanford think a video game could help with that.

The game is Eterna Medicine.  The first version of Eterna was released five years ago, and it allows anyone to try to design bio-molecules.  Seriously.   Apparently learning to fold RNA molecules can be fun, a real-life kind of puzzle, Something like 100,000 players have tried it so far.  As one of them, Jeff Anderson-Lee, told NPR: "You can start out not knowing any of the science.  You start to get a sense for what things go together to make an RNA design fold the way you want.  While you're doing that, you're gaining points and rising in rank."

Fun or not, it has proved to be meaningful.  In February, Mr. Anderson-Lee and fellow gamers used their Eterna learnings to publish an article (check it out: "Principles for Predicting RNA Secondary Structural Design Difficulty") in Journal of Molecular Biology, an eminently legitimate, peer-reviewed scientific journal.  It is believed to be the first such "citizen scientist" publication, although it probably will not be the last.

As Peter Venkman might say, "no studying."

With Eterna Medicine, the goal is to try to figure out what particular configuration of RNA designs is most likely to mark the presence of TB.  Players "vote" on designs, and the winning designs will actually get synthesized to see if they work.  If they do, testing for TB could become as easy as using a home pregnancy test.

The Wall Street Journal profiled not just Eterna but also several other game approaches to health care research.  These include:

• EyeWire, which maps neural connections in a mouse's retina;
• Phylo, which cross-indexes disease-related DNA sequences;
• Project Discovery/Eve Online, which classifies human cell proteins.

If people got bored with those, there are games that allow users to play with quantum physics.   They've published too.  

I wrote about the use of games in health care a couple years ago, and it has continued to fascinate me.  After all, some 155 million Americans admit to playing video games, spending some $23b annually.  It is particularly common among -- but not unique to -- younger people.  Health care experts have seen ways to utilize games or gaming principles in health education, motivating better health habits, even training of medical professionals.  

I got a little more interested when Microsoft bought the popular game Minecraft for $2.5b in 2014.  I got even more interested when they announced they were expanding into education with Minecraft Education Edition.  As the announcement said: "We've seen that Minecraft transcends the differences in teaching and learning styles and education systems around the world.  It's an open space where people can come together and build a lesson around nearly everything."  The Education Edition is now in beta and will be released for free in an early access program this summer.  

And I got really interested after reading Isaac Kohane's great post What Minecraft can teach the health care system.  Dr. Kohane reports watching with awe as a group of 11 year-old children worked together in the game to get through the complex virtual environment-- and managed to have fun doing so.  As he says, "It was a remarkable demonstration of using technology to coordinate teams in complex tasks without prior training."   

He goes on to note, perhaps superfluously, "I can tell you with assurance that the use of team technology in the American health care system lags decades behind the seamless coordination that Minecraft players take for granted."

Dr. Kohane sees three substantial (but not insurmountable) obstacles to changing that:

• Unique systems: Health care thinks it is so unique that its systems need to be, making them "less secure, less versatile, and more expensive than they have to be."
• Billing gets in the way: this needs no explanation, but I'll say it anyway.  Our system is  more about making sure providers get paid than on achieving better patient outcomes or experiences.
• Clinicians only: Patients have not historically been involved in the decision-making, so no wonder communication flows are at best siloed.

He is optimistic that -- if we can overcome "institutional inertia -- "there will be no reason that a patient's history, diagnoses, and next steps can't be presented in a way that makes as much sense to adults as Minecraft intuitively does to 11-year-olds." 

I love his optimism, but that's a powerfully big "if" he qualifies with. 

By far the most viewed post I've written was one I did a couple years ago on Twitch, the hugely popular online service that doesn't even let people play games, just watch others play them.  I'm old enough that I rarely play video games but young enough to at least understand the appeal, but Twitch confirmed for me that there are younger generations that see them in entirely different ways than us older generations do.   My conclusion is that if the health care system doesn't recognize and adapt to gaming's technology and its users' preferences, it is going to become obsolete.  

Perhaps I'll go try to fold some RNA sequences...