Go Ahead, A.I. -- Surprise Us

 Last week I was on a fun podcast with a bunch of people who were, as usual, smarter than me, and, in particular, more knowledgeable about one of my favorite topics – artificial intelligence (A.I.), particularly for healthcare.  With the WHO releasing its “first global report” on A.I. -- Ethics & Governance of Artificial Intelligence for Health – and with no shortage of other experts weighing in recently, it seemed like a good time to revisit the topic. 

Credit: ITProToday

My prediction: it’s not going to work out quite like we expect, and it probably shouldn’t. 

“Like all new technology, artificial intelligence holds enormous potential for improving the health of millions of people around the world, but like all technology it can also be misused and cause harm,” Dr Tedros Adhanom Ghebreyesus, WHO Director-General, said in a statement.  He’s right on both counts.

WHO’s proposed six principles are:

• Protecting human autonomy
• Promoting human well-being and safety and the public interest
• Ensuring transparency, explainability and intelligibility 
• Fostering responsibility and accountability
• Ensuring inclusiveness and equity 
• Promoting AI that is responsive and sustainable

All valid points, but, as we’re already learning, easier to propose than to ensure.  Just ask Timnit Gebru.  When it comes to using new technologies, we’re not so good about thinking through their implications, much less ensuring that everyone benefits.  We’re more of a “let the genie out of the bottle and see what happens” kind of species, and I hope our future AI overlords don’t laugh too much about that. 

As Stacey Higginbotham asks in IEEE Spectrum, “how do we know if a new technology is serving a greater good or policy goal, or merely boosting a company’s profit margins?...we have no idea how to make it work for society’s goals, rather than a company’s, or an individual’s.”   She further notes that “we haven’t even established what those benefits should be.”

Ms. Higginbotham isn’t specifically talking about healthcare, but she could be.  We can’t really agree on what a healthcare system should and shouldn’t do, much less one augmented by A.I.  It’s no wonder that our first generations of A.I. in healthcare are confused.

The example that I’ve been using for years is that we can’t even agree on how human physicians seeing patients in other states via telehealth should be licensed/regulated, so how are we going to decide how a cloud-based healthcare A.I. should be?   

The FDA is paying attention

Carissa Véliz has an idea.  Writing in Harvard Business Review, she suggests that the FDA test AI like it does prescription drugs or medical devices, using randomized control trials to prove validity and efficacy.  I’d feel better about that if we didn’t already have a lot of history of that process taking too long, being swayed by non-data driven factors (e.g., Aduhelm), or being frequently circumvented. 

It gets worse.  Christopher Mims just wrote about how AI is moving from the cloud to edge devices (like your phone or home appliance).  Edge computing is going to be a big part of our future, including healthcare, but, as computer science professor Elisa Bertino pointed out to him, how can anyone certify/regulate AI that is evolving on its own, in the real world?  It won’t necessarily resemble the A.I. that it started out as; it’s going to depend on the data/inputs it receives. 

Mr. Mims also warns: “Modern AI, which is primarily used to recognize patterns, can have difficulty coping with inputs outside of the data it was trained on.”  Oh, boy -- it’s going to run into a lot of that with health care.  People are messy, so to speak, and a lot of that mess impacts their health.  A.I. better be ready to deal with it. 

AI is going to evolve much more rapidly than other healthcare technologies, and our existing regulatory practices may not be sufficient, especially in a global market (as we’ve seen with CRISPR).  Not to be facetious, but we may need AI regulators to oversee AI clinicians/clinical support, just as we may need AI lawyers to handle the inevitable AI-related malpractice suits.  Only another black box may be able to understand what a black box is doing.

I worry that we’re thinking about how we can use A.I. to make our healthcare system do more of the same, just better.  I think that’s the wrong approach.  We should be going to ground principles.  What do we want from our healthcare system?  And, then, how can A.I. help get us there? 

For example, we should want that everyone has access to affordable health care – when they need it, where they prefer it.  That health care should tailored to the individual, including genetics, environment, and socio-economic status, and should be based on solid evidence.  That all sounds like a list of the usual platitudes, but none of it is currently true.  How can A.I. help make it true, or, at least, truer?

If A.I. for healthcare is a better Siri or a new decision support tool in an EHR, we’ve failed.  If we’re setting the bar for A.I. to only support clinicians, or even to replicate physicians’ current functions, we’ve failed.  We should be expecting much more.

E.g., how can we use A.I. to democratize health care, to get advice and even treatment in people’s hands?  How can we use it to help health care be much more affordable?  How can A.I. help diagnose issues sooner and deliver recommendations faster and more accurately? 

In short, how can A.I. help us reorient our health care from the healthcare system that delivers it, and the people who work in it, to our health?  If that means making some of those irrelevant, or at least greatly redefining their roles, so be it. 

Right now, much A.I. work in healthcare seems to be focused primarily on granular problems, such as diagnosing specific diseases.  That’s understandable, as data is most comparable/available around granular tools (e.g., imaging) or conditions (e.g., breast cancer).  But our health is usually not confined within service lines.  We need more macro A.I. approaches. 

We might need A.I. to tell us how A.I. can not just improve our healthcare but also to “fix” our healthcare system.  And I’m OK with that.    

Better Broadband for Better Health Care

Here’s a question that we don’t often ask: which is the U.S. more likely to accomplish – getting everyone health insurance, or broadband?  Hint: it’s probably not what you think.

Credit: Cal Matters

The health insurance part of it is often debated.  We passed ACA, but the number of uninsured stubbornly remains at nearly 30 million, almost 10% of the population.  Still, except for residents of those 12 states that have refused to pass Medicaid expansion, everyone in the country has at least access to public or private health insurance, with subsidies available to many. 

Broadband hasn’t been around as long a health insurance, but it has become an integral part of our society, as the pandemic proved (ever try remote work or learning without broadband, much less telehealth?).   Unfortunately, some 20 million households lack broadband; assuming an average household size of about 2.5, that’s some 50 million people, which is way more than the number of uninsured. 

Welcome to the digital divide.   

Even Republicans support broadband spending

Everyone seems to agree increasing access to broadband is a good goal.  It’s part of President Biden’s proposed infrastructure plan, and even many Republicans support some funding towards the goal, as in a recent bipartisan proposal. 

We often think about the issue as being a rural problem, similar to the problem of electricity availability in rural areas before the Rural Electrification Act (1936).  It’s just hard, or at least expensive, to wire all those vast spaces, those farms and small communities that comprise much of America. 

The fact of the matter, though, is that of those 20 million households without broadband, some 15 million of them are urban households.  A higher percent of rural households may lack broadband, but, in terms of actual numbers of households lacking it, it is urban dwellers.  For the most part, broadband is available in their neighborhood; they just can’t afford it (or don’t see the need).  

But mostly we focus about the rural problem, for the wrong reasons.  Blair Levin, a Brookings fellow, told The New York Times:

From an economic and society perspective, the most important thing to do is to get online everybody who wants to be online.  From a political perspective, the biggest political capital is behind accelerating deployment where there is none, which means in rural areas.

The same article quoted testimony from Joi Chaney, senior vice president at The Urban League, before the House Appropriations Committee: “Our investments must not only solve for the deployment or availability gap.  They must also solve for the adoption gap, the utilization gap and the economic opportunity gap to truly achieve digital equity.”

As with health insurance, the problem is less access than it is affordability.  Josh Stager, senior counsel at the Open Technology Institute, emphasized:

Once the pandemic started, it became painfully obvious that internet connectivity is a utility, and it's not just necessary to get through the pandemic but to get through modern life in America.  And the reality -- that so many people are struggling with affording the service, not access -- became undeniable.

Digital redlining. 
Credit: Electronic Freedom Foundation

Brookings calls it “digital poverty.”  Others point to “digital redlining,” which means you’re more likely to get fiberoptic or other types of faster connections if you live in a wealthy suburb or a gentrified urban neighborhood. 

We don’t allow these kinds of disparities for electricity, telephone service, or water, but we do for broadband -- and for health care.  How very 1930’s of us. 

As Brookings put it:

Digital poverty is akin to an entire neighborhood with spotty electricity or unreliable water service. These are places where students struggle to engage with digital coursework and adults can’t check online job boards. Digital poverty is a tangible drag on economic prosperity.

There are some subsidies available for broadband, most notably the FCC’s Lifeline, which pays a maximum of $9.25 monthly. An Open Technology Institute analysis estimated the subsidies only cover 13% of the actual broadband costs, while even the FCC acknowledges that only 26% of eligible households participate. 

I guess we should stop complaining about ACA subsidies. 

Broadband reminds me of healthcare in another way: Americans pay way too much for way too little.  Our astronomical healthcare spending gets us only middling health care outcomes, but, by the same token, among OECD countries, only Mexico pays more for broadband.  Our broadband speeds rank us at best tenth in the world; it’s one thing to be behind urbanized countries like Singapore or Hong Kong, but France or Hungary? 

It’s all part of a pattern.  It is true that rural hospitals have been struggling, even closing, at alarming rates in recent years, but so-called safety net hospitals, usually in urban areas, have been hit hard as well.  It is true that many rural areas qualify as “food deserts,” but more urban residents live in them, and that affordability is as least as important as availability for food as well. 

Ezra Klein wrote recently:

This is the conversation about poverty that we don’t like to have: We discuss the poor as a pity or a blight, but we rarely admit that America’s high rate of poverty is a policy choice, and there are reasons we choose it over and over again.

Whether it is poverty, broadband, health care, food security, unemployment benefits, wage inequalities, or a number of other hot issues, the root of our problems usually lie not in poor personal choices but in policy choices – some with unintended consequences but many with foreseeable outcomes. 

ACA made a tactical choice to put more money into our existing structures, such as expanding Medicaid, and guaranteeing access to and subsidizing private insurance.  There simply weren’t the votes to make more dramatic changes (and still aren’t). 

We may be making a similar mistake with broadband.  We may choose to simply wire more areas that lack broadband, but without ensuring that more people have the ability, and see the need, to pay for it.  We should also be attacking our high cost of broadband and forcing improvements in our speeds.  As Brooking’s Tom Wheeler pointed out, “it is foolhardy for the government to spend public money for second class service.”  He urged that we “future-proof” broadband.

Even prior to the pandemic, some labeled broadband access as a public health issue.  Bauerly, et. Alia., called broadband access a “super-determinant” of health, and warned: “digitally isolated communities may risk worse health outcomes resulting from the effects of limited broadband access on educational and economic opportunities as well as access to high-quality health services.”

We can’t get the health care system, or health care outcomes, that we want unless we also “future-proof” broadband.   

Twins for Everyone!

I have lived my entire life as a twin, and, while it isn’t an unalloyed blessing, on balance I’d recommend it.  Most of you, though, probably aren’t twins and have missed the experience.  Don’t worry: you may still get a chance – with a digital twin. 

Credit: Institution of Mechanical Engineers

It could have profound implications for your health and for healthcare generally.

A digital twin, in case you are not familiar with the concept, is a virtual representation of a physical object.  It is created from data about that physical object, and is fed ongoing data (e.g., via IoT) about it to keep the model accurate. 

Credit: GE

The concept is not new, often attributed to Michael Grieves at the Florida Institute of Technology in 2002.  Dr. Grieves saw the value of the concept for manufacturing; for example, GE’s Aircraft Engines has been using them to make their engines safer and more efficient.  Other applications include building maintenance, data centers, and even creating a digital twin of the whole planet.

People have seen the potential of digital twins for healthcare for years.  Back in 2016, GE’s Digital CEO Bill Ruh predicted:

I believe we will have a digital twin at birth, and it will take data off of the sensors everybody is running, and that digital twin will predict things for us about disease and cancer and other things. I believe we will end up with health care being the ultimate digital twin. Without it, I believe we will have data but with no outcome, or value.

We’re not there yet, not nearly, but it’s coming.

Digital twins are, to some extent, still in early days.  The Digital Twin Consortium (DTC) was formed in 2020 with a goal “to drive consistency in vocabulary, architecture, security and interoperability of digital twin technology.” Last month it announced “an open-source collaboration community to accelerate the adoption of digital twin-enabling technologies and solutions.” 

Credit: Digital Twin Consortium

Dell’s Dr. Said Tabet, a DTC steering committee member, said: “Open-source collaboration will encourage innovation in digital twin solutions.  Our Open-Source Collaboration Community will also accelerate the adoption of digital twins and drive business transformation.”  Pieter van Schalkwyk, CEO, XMPro, agreed: "Building a library of open-source digital twin resources will help lower the barriers to entry for many companies who want to get started with digital twins.” 

But, as Gartner’s Peter Havart-Simkin told VentureBeat: “There is no multi-vendor open standard for a digital twin that can be used by third parties, and there is currently no such thing as an open multi-vendor digital twin integration framework.”  VentureBeat translates: “The industry is lacking a digital twin app store where enterprises could buy a digital twin template of an asset that they own.” 

Microsoft and IBM – are you paying attention to the “digital twin app store” idea?

We have a ways to go to get there.  VentureBeat noted that a survey by the Industrial Internet Consortium identified at least eight industry-wide efforts working on digital twin standards, with the report concluding:

The development of standards for Digital Twin is an imperative for setting the necessary foundation to ensure its successful adoption in the market. However, it is clear that an effort based solely on establishing open standards is not enough. In particular, …the demand for enabling software is increasing. Adequate, widely supported and widely adapted Digital Twin Open Source software can establish de facto standards for the underlying architecture of Digital Twins.

DTC has some work to do. 

Meanwhile, a new paper in Nature outlines “a probabilistic graphical model as a formal mathematical representation of a digital twin and its associated physical asset,” which sounds pretty dry until senior author Karen Willcox points out that, until now, “missing has been the foundational mathematical framework that would enable digital twins at scale.”  The press release promises: “the same mathematical model could be applied in situations as seemingly disparate as the human body, a space rocket or a building.” 

And we’re back to healthcare.

In The Hill, Mark Minevich argues that, with digital twins, “If implemented successfully we can say goodbye forever to human clinical trials,” since we “can test all possible vaccines and treatments on digital twins, save lives faster and never test potentially dangerous treatments on humans again.”  He also mentions patient monitoring that detect symptoms at an early stage, surgery simulation, diagnosis and treatment, and digital twins for hospital operations. 

Similarly, in a recent Forbes article, Sindhu Kutty wonders: “why aren’t more smart hospitals cropping up like smart factories?”  Good question. 

Another digital twin advocate, Stephen M. Levine, Ph.D., wrote in Fierce Healthcare:

Once it was considered impossible to build a twin of an entire commercial jet under all possible flight conditions. Now it is routine. The human body poses some unique challenges, but these projects demonstrate that if we collectively commit to developing them, virtual twins can deliver the holy grail for medicine; personalized, precise, successful medical treatments—not by themselves, but by providing, in parallel, a living, breathing medical record that combines the latest fundamental knowledge with the patients exact history and unique physiology.

Credit; Getty Images

My dream is that our infuriating, siloed, clunky EHRs transition into our digital twins.  They’d know us, our health history and what is happening with us in real-time.  More importantly, they’d be capable of identifying problems at early stages and modeling future risks/benefits – of lifestyle changes, treatments, procedures, or lack thereof.  Let our digital twins take the risks; we’d enjoy the benefits. 

Throw in some holograms and we’d really have something new.

It’s not going to be easy.  The aforementioned Dr. Willcox also contributed to an opinion piece in Nature Computational Science, along with three other digital twin experts.  Lead author Steven Niederer (King’s College London) warned: “We also need to further develop the mathematics of how we create digital twins from patient data, how we measure uncertainty in patient data, and how to account for uncertainty in the model in predictions.” 

Another author, Mark Girolami (The Alan Turing Institute), sees the promise of digital twins: “In healthcare for instance, the increasing power of computers and algorithms are enabling technologies to build a patient-specific digital twin, catering to our diversity as human beings and improving individual health outcomes.”  He warned, though, that “these promised advances are going to be hard won, requiring further concerted and sustained foundational research and development to fully realise the promise of the digital twin.”  

The advances will be hard won, but are worth winning.  I’m happy with my current twin, but I can’t wait for my digital twin. 

Hey, How About Starship Earth?

I missed the job announcement on the company website.  I missed it again when the company posted the job on Linkedin.  I missed it when Eric Ralph tweeted that the posting was “probably the coolest job posting I’ve read in years.”  Fortunately, though, I follow Isaac Kohne (MD, PhD), and I did see his tweet:

 Yes, I’m talking about SpaceX.  Yes, the job is for a “Starship Medical Engineer.”  Yes, it’s to help Space X’s mission to Mars, whenever that might be.  Who knows, the job might even entail going to Mars, although that’s not spelled out. 

I am not, of course, remotely qualified for such a job.  In fact, I don’t even know anyone who might be.  But I agree with Mr. Ralph that it’s probably the coolest job posting I’ve seen in years, maybe ever.  And I even more agree with Dr. Kohne: it could be an “opportunity to rethink a bigger broken system.”

Hint: I don’t think he’s talking about just the SpaceX mission. 

SpaceX is looking for a physician – M.D. or D.O. – who also has a Masters of Engineering and experience with aerospace medicine.  I imagine that substantially cuts down on the candidate pool.  The list of responsibilities are pretty daunting:

• Serve as a point of contact for customers with relevant SpaceX stakeholders for medical development initiatives
• Work across teams to design, integrate, and implement a medical system of the future
• Support research including health data collection before, during, and after human spaceflight missions focusing on effects of long-duration spaceflight within the context of a widening range of passenger health issues
• Serve as the aerospace medicine technical expert for human spaceflight activities
• Develop and coordinate space medicine flight operations with technical, operations, and programmatic parties
• Provide medical support during flight operations and development as a console operator

Elon Musk

In short, “As a Starship medical engineer, you will be responsible for developing the medical system for Starship.” 

In one way, SpaceX has it easy: it’s all going to be new.  The world has now had over sixty years of aerospace engineering, ever since we started trying to send people into space.  We’ve sent men (yes, only men) to the Moon, we’ve had people circle in earth for months at a time, and we’ve made sending astronauts into orbit almost routine, with multiple countries and even a few private companies doing so or preparing to do so (e.g., Blue Origin, Space X, Virgin Galactic). 

But no one has sent anyone to Mars.  It’s been just shy of fifty years since we’ve sent anyone to the Moon.  The moon is a little over 200,000 miles from earth; the closest Mars ever gets to earth is over 30 million miles, and it can be over 200 million miles away.  The trip will take over seven months just to get to Mars.  

The Mars trip is quantitatively and qualitatively much different than anything we’ve ever tried before. 

Credit: Dr. James O'Donoghue

As you can imagine, if someone gets sick or injured, there’s no calling 911.  There’s no local hospital.  There’s no corner drugstore.  You can’t airflight anyone home.  As the mission proceeds, it becomes far enough away that the speed of light limits the ability to even communicate with Earth in a timely way.  Whatever expertise and equipment the crew comes with is all they’ll have in order to deal with whatever happens to them. 

That’s why SpaceX wants a Starship medical engineer.  Not a job for the faint of heart.  The Starship medical engineer may build on existing solutions, but extending them to a Mars mission requires innovative thinking and de novo approaches.

Starship Earth has a similar problem.  UFO’s (or UAPs, as they are now referred to) aside, we’re all on our own here.  There are no comparable planets, and no source of expertise anywhere else.  If we screw things up here, we’re out of luck (which is one reason Elon Musk is keen to get to Mars).

And, let’s face it, we’re screwing things up here.  COVID-19 showed us that we’re still vulnerable to pathogens.  Climate change may make large parts of the world uninhabitable within a century.  Microplastics have infiltrated virtually everywhere, including inside us, with yet-to-be-determined impacts on our food chain and our health.  Bioweapons, nuclear weapons, and cyberweapons each could destroy us, each in their own way. 

Unlike SpaceX, we’ve got our medical expertise and facilities (well, many of us, anyway).  We have lots of trained health care professionals, lots of health care offices and facilities, and more prescription drugs and medical devices than we know what to do with.  Yet in many countries, the U.S. included, life expectancy was falling even before the pandemic.  We’re living more of our lives with chronic diseases, with too many of us spending our last years needing significant care.  We have plenty of health care, but not enough good health.

It’s as if we planned for a short trip to the Moon and unexpectedly found ourselves on the way to Mars.  We don’t have the resources, and particularly not the healthcare system, that we need to make the trip. 

Starship Earth needs a Starship Medical Engineer.

Credit: EASAC

Just look at how the pandemic forced us to discover anew the potential of telehealth, and how we’re still fumbling to incorporate it.  We’re pouring money into “digital health,” without really figuring out how “digital health” becomes just part of “health.”  Our healthcare system is largely separate from our public health system(s), and both are largely separate from our environmental health “system(s).”  We can’t even really integrate medical, dental, vision and hearing within the healthcare system. 

Who is inventing not just the medical system of the future but the health system of the future – not just care but lifestyle and environment?   Who’s doing the research, who’s coordinating with the various domain experts, who’s overseeing putting new practices into operation?  Who’s lobbying to not just get more funding for existing entities but for developing truly de novo ideas for the 21^st century? 

SpaceX will find someone for its Starship Medical Engineer, and he/she/they will do some cool stuff.  Starship Earth, I’m not so sure about. 

Make Some Microbe Friends

It’s the coolest story I’ve seen in the past few days: The New York Times reported how an Italian  museum cleaned its priceless Michelangelo sculptures with an army of bacteria.  As Jason Horowitz wrote, “restorers and scientists quietly unleashed microbes with good taste and an enormous appetite on the marbles, intentionally turning the chapel into a bacterial smorgasbord.”

And you just want to kill them all with your hand sanitizers and anti-bacterial soaps. 

The Medici Chapel in Florence had the good fortune to be blessed with an abundance of works by Michelangelo, but the bad fortune to have had centuries of various kinds of grime building up on them.  In particular, over time the corpse of one Medici “…seeped into Michelangelo’s marble, the chapel’s experts said, creating deep stains, button-shaped deformations…”

This is, I assume, why they tell you not to touch the art.

Scientists picked a bacteria -- Serratia ficaria SH7, in case you’re taking notes – that ate the undesired grime without also eating the underlying marble.   It wasn’t hazardous to humans either and didn’t create spores that might go elsewhere.  “It’s better for our health,” one of the art restorers told NYT.  “For the environment, and the works of art.”

Medici Chapel after bacteria. 
Credit: Gianni Cipriano for NYT

The technique was a success, allowing the sculptures to look like they did centuries ago. 

Using such bacteria to clean art has been around for at a decade, and not just for sculptures.  Perhaps more surprising is bacteria isn’t just cleaning art, it’s also creating it; the American Society for Microbiology hosts an annual Agar Art Contest. 

If you’re impressed by that, researchers are teaching bacteria to read, or at least to recognize letters.  That’s not all they might learn to do.  “For example, the framework and algorithm in our study can be used to facilitate the design of living therapeutics, such as targeted drug release systems based on engineered probiotic bacteria systems,” the researchers say.    

The thing is, we not only don’t know what microbes do, or could do, but we have only a vague understanding how they surround us.  That’s starting to change.  We’ve known for some time that each of us has a unique microbiome (including mycobiome!).  What we didn’t realize until recently was that each urban area has its own microbiome as well. 

Summary of study.  Credit: Danko, et. alia

A new study took samples from the subway systems in 60 cities around the world, and found thousands of previously unknown viruses and bacteria.  There was a “core urban microbiome” that almost all the cities shared, but each city had its unique microbiome. 

The authors conclude:

…these data suggest that urban microbiomes should be treated as ecologically distinct from both surrounding soil microbiomes and human commensal microbiomes. Though these microbiomes undoubtedly interact, they nonetheless represent distinct ecological niches with different genetic profiles.

"Every city has its own 'molecular echo' of the microbes that define it," said senior author Christopher Mason, a professor at Weill Cornell Medicine (WCM). "If you gave me your shoe, I could tell you with about 90% accuracy the city in the world from which you came."

There may be, Dr. Mason thinks, as much biodiversity on a subway railing as in rainforest (which, if you’ve ridden any U.S. subways, probably does not come as a surprise).  He marvels: “I think it’s a wonderful affirmation of how much left we have to discover about the world.”

“The amount of microbial diversity is just incomprehensibly vast,” Erica Hartmann, a microbiologist who was not involved in the study told The New York Times.  “There’s so much out there that we just don’t really understand, and there could be all kinds of nifty biotechnologies and all kinds of fun chemistries that we’re not aware of yet.”

The researchers were able to identify “antimicrobial resistance genes” that indicated resistance to antibiotics and other antimicrobial agents.  Lead author David Danko speculated:

Can we give some kind of heads-up about what to look for? Can we track the spread of bacteria or genes that will make bacteria resistant to antibiotics in the future? Can we use this as a way to inform public health departments in the use of antibiotics going forward?

The team is creating a “global metagenomic map” of the organisms, and plans to keep swabbing to collect more sample.  A companion paper looked at the “air microbiome” of the subways systems, finding a similar “geographic specificity.” 

They’re all around us.  They’re in us.  We live in a microbial world.  Some argue that our microbiome should be considered another organ, although it may be more accurate to view it as a colony that tries to tolerate us.  However you view microbes, they’re not going away; if they did, we would as well.

You remember

The pandemic has caused all of us to fear the coronavirus, and to take measures to kill it.  We all desperately sought Clorox wipes, stayed away from other people and their viruses, and tried things like UV sterilization.  Scientists worry all these efforts may have unintended consequences.  “We’re starting to realize that there’s collateral damage when we get rid of good microbes, and that has major consequences for our health,” said B. Brett Finlay, first author of a paper on the topic in PNAS.

As Dr. Finley told James Hamblin for The Atlantic. “The microbes we carry around are involved in many of the fundamental processes of Homo sapiens.”  Brendan Bohannan, a professor at the University of Oregon, agreed, telling The New York Times: “The more we learn about our relationships with the microbial world, the clearer it is that we are connected to them and to the rest of the natural world.” 

Mr. Hamblin concluded: “The ongoing challenge is to avoid binary thinking about microbes: They are not simply good or bad, any more than people are, and neither is Purell.”

Last year I argued that modern medicine was reaching the kind of limits that classical physics did at the beginning of the 21^st century, when quantum effects were starting to become known.  It required an entirely new approach to physics – quantum physics – to deal with them, and that ended up revolutionizing physics and our understanding of the world. 

Medicine needs that kind of “quantum” revolution, particularly in regards to understanding, accepting, and benefiting more from our coexistence with the microbial world.  If we can co-opt microbes to clean art, who knows what we can “convince” them to do for our health? 

Holograms to the Rescue

Google is getting much (deserved) publicity for its Project Starline, announced at last week’s I/O conference.  Project Starline is a new 3D video chat capability that promises to make your Zoom experience seem even more tedious.  That’s great, but I’m expecting much more from holograms – or even better technologies.  Fortunately, there are several such candidates.

Project Starline.  Credit: Google

For anyone who has been excited about advances in telehealth, you haven’t seen anything yet.

If you missed Google’s announcement, Project Starline was described thusly:

Imagine looking through a sort of magic window, and through that window, you see another person, life-size and in three dimensions. You can talk naturally, gesture and make eye contact.

Google says: “We believe this is where person-to-person communication technology can and should go,” because: “The effect is the feeling of a person sitting just across from you, like they are right there.” 

Sounds pretty cool.  The thing, though, is that you’re still looking at the images through a screen.  Google can call it a “magic window” if it wants, but there’s still a screen between you and what you’re seeing.

Not so with Optical Trap Displays (OTDs).  These were pioneered by the BYU holography research group three years ago, and, in their latest advance, they’ve created – what else? – floating lightsabers that emit actual beams:

Optical trap displays are not, strictly speaking, holograms.  They use a laser beam to trap a particle in the air and then push it around, leaving a luminated, floating path.  As the researchers describe it, it’s like “a 3D printer for light.”

The authors explain:

The particle moves through every point in the image several times a second, creating an image by persistence of vision.  The higher the resolution and the refresh rate of the system, the more convincing this effect can be made, where the user will not be able to perceive updates to the imagery displayed to them, and at sufficient resolution will have difficulty distinguishing display image points from real-world image points.

Lead researcher Dan Smalley notes:

Most 3D displays require you to look at a screen, but our technology allows us to create images floating in space — and they’re physical; not some mirage.  This technology can make it possible to create vibrant animated content that orbits around or crawls on or explodes out of every day physical objects.

Co-author Wesley Rogers adds: “We can play some fancy tricks with motion parallax and we can make the display look a lot bigger than it physically is.  This methodology would allow us to create the illusion of a much deeper display up to theoretically an infinite size display.”

Indeed, their paper in Nature speculates: “This result leads us to contemplate the possibility of immersive OTD environments that not only include real images capable of wrapping around physical objects (or the user themselves), but that also provide simulated virtual windows into expansive exterior spaces.”

I don’t know what all of that means, but it sounds awfully impressive.

The BYU researchers believe: "Unlike OTDs, holograms are extremely computationally intensive and their computational complexity scales rapidly with display size.  Neither is true for OTD displays.”  They need to meet Liang Shi, a Ph.D. student at MIT who is leading a team developing “tensor holography.” 

Before anyone with mathemaphobia freaks out about the “tensor,” let’s just say that it is a way to produce holograms almost instantly. 

The work was published in Nature last March.  The technique uses deep neural networks to generate 3D holograms in near real time. I’ll skip the technical details of how this all works, but you can watch their video:

Their approach doesn’t require supercomputers or long calculations, instead allowing neural networks to teach themselves how to generate the holograms. Amazingly, the “compact tensor network” requires less than 1 MB of memory.  The images can be calculated from a multi-camera setup or LiDAR sensor, which are becoming standard on smartphones.

“People previously thought that with existing consumer-grade hardware, it was impossible to do real-time 3D holography computations,” Mr. Shi says.

Joel Kollin, a Microsoft researcher who was not involved in the research, told MIT News that the research “shows that true 3D holographic displays are practical with only moderate computational requirements.” 

All of the efforts are already thinking about healthcare.  Google is currently testing Project Starline in a few of its offices, but is betting big on its future.  It has explicitly picked healthcare as one of the first industries it is working with, aiming for trial demos later this year.

The BYU researchers see medicine as a good use for OTDs, helping doctors plan complicated surgeries: “a high-resolution MRI with an optical-trap display could show, in three dimensions, the specific issues they are likely to encounter. Like a real-life game of Operation, surgical teams will be able to plan how to navigate delicate aspects of their upcoming procedures.”

The MIT researchers believe the approach offers much promise for VR, volumetric 3D printing, microscopy, visualization of medical data, and the design of surfaces with unique optical properties. 

If you don’t know what “volumetric 3D printing” is (and I didn’t), it’s been described as like an MRI in reverse: “the form of the object is projected to form the model instead of scanning the object.”  It could revolutionize 3D printing, and, for healthcare specifically, “Being able to 3D print from all spatial dimensions at the same time could be instrumental in producing complex organs…This would enable better and more functional vascularity and multi-cellular-material structures.”

OSU shoulder surgery. 
Credit: Wexner Medical Center

As for “visualization of medical data,” for example, surgeons at The Ohio State University Wexner Medical Center are already using “mixed reality 3D holograms” to assist in shoulder surgery.  Holograms have also been used for cardiac, liver, and spine surgeries, among others, as well as in imaging.    

-----------

2020 was, in essence, a coming out party for video conferencing in general and for telehealth in particular.  The capabilities had been around, but it wasn’t until we were locked down and reluctant to be around others that we started to experience its possibilities.  Still, though, we should be thinking of it as version 1.0.

Versions 2.0 and beyond are going to be more realistic, more interactive, and less constrained by screens.  They might be holograms, tensor holograms, optical trap displays, or other technologies I’ve not aware of.  I just hope it doesn’t take another pandemic for us to realize their potential.