Expect the Unexpected

Most people probably know about Alexander Fleming “discovering” penicillin by accident, thus inventing the field of antibiotics. Others may know that Viagra was originally developed to combat hypertension and angina pectoris. You may think of Botox for its cosmetic purposes, but it was originally intended to treat strabismus (cross eyes), and has a host of other medical purposes as well. All of those, and countless other examples, speak to the importance of serendipity in science.

Science often surprises. Credit: Microsoft Designer

A new study, by Aslan et. al., attempts to quantify how common such serendipity is. Bottom line: it is way more common than you think.

The team analyzed over 1.2 million publications, which had resulted from 90,000 NIH grants between 2008 and 2016. They used the NIH's Research, Condition, and Disease Categorization (RCDC) to categorize what the researchers said they were looking for in their grant application versus what their findings were when published.

The results:

We found that 70 % of the publications have at least one RCDC category not in its grant, which we termed ‘unexpected’ categories. On average, 40 % of categories assigned to a publication were unexpected. After adjusting for similarity across some of the RCDC categories by empirically clustering the categories, we found 58 % of the publications had at least one unexpected category and, on average, 33 % of publication categories were unexpected.

Larger grants tend to result in more unexpected results, as do results published longer after the grant was issued.

Fig. 1. Proportion of publications with (A) one or more unexpected categories or (B) one or more unexpected categories (clustered); distribution of (C) unexpected category proportions and (D) unexpected category proportions (clustered).

“The bottom line is that ‘unexpectedness’ is not rare — this came through loud and clear,” said Ohid Yaqub, a biochemist and social scientist at the University of Sussex in Brighton, UK, who led the work. According to Nature, this research is “part of a wider project to understand the role in research of serendipity, of which unexpectedness is just one aspect.”

The researchers do caution: “what we are calling “unexpected” on the basis of text in the grants and publications may not in fact be unexpected for the investigators, or others in the field. Future research could make progress on these issues by using interviews and surveys to validate text-based measures of unexpected spillovers.” Professor Yaqub admits: “What we’ve looked at is only just scratching the surface.”

Telmo Pievani, a philosopher of biological sciences at the University of Padua in Italy, told Nature that the results go “beyond the anecdotal view of serendipity in science” and “for the first time verifies it on a quantitative and statistical level.” It is, indeed, part and parcel of how science works.

Somewhat surprisingly, the researchers also found:

Our results suggest that disease-orientation and clinical research were less likely to be associated with spillovers. Grants resulting from targeted requests for applications were more likely to result in publications with unexpected categories, though the magnitude of the differences was relatively small.

E.g., one would think those targeted requested would result in more targeted results, but that was not the case.

Samatha Copeland, a philosopher at Delft University of Technology in the Netherlands, told Nature that she worries that funding pressures may inhibit such serendipitous findings: “What’s happening right now is, scientists have actually been working around funding processes in order to make room for unexpected discoveries,” with Ph.D. students in particular likely to feel “oppressed by the scientific method and model of producing exactly the results you said you would produce.” Getting unexpected results may not be good for tenure or for Ph.D. thesis approval.

The key is being open to the unexpected results. Professor Pievani believes: “It is okay to fund both basic research and applied research, as long as both are open to unexpected results and do not eliminate anomalies too hastily.” Indeed, in his recent book Serendipity: The Unexpected in Science, he writes: “The common thread is chance and observational sagacity: a mixture of skill, clairvoyance, an incisive and fortunate mind, and the ability to discover connections.”

As the saying goes (oft attributed to Louis Pasteur), chance favors the prepared mind.

Still, Professor Pievani worries: “In the dictatorship of an instantaneous, emotional, and overwhelming present, you can always look for something specific with a quick google and find it for sure. Serendipity instead blossoms in the bends and meanders, the dull moments and wanderings.” Scientists struggling to stay within their funding or to attract new funding may be more reluctant to follow those bends and meanders, wherever they might lead.

Innovation expert John Nosta, founder of Nostalab, believes that AI can serve as “serendipity engines” by using AI’s so-called hallucinations. “What if these hallucinations could be harnessed to create a serendipity engine?” he writes. “An AI that doesn’t merely predict the next word but facilitates unexpected connections, delightful surprises, and groundbreaking insights?”

If you doubt that, note that a 2024 Wharton study found that AI was more creative than Wharton students (which may say more about Wharton students than AI): “There was a significantly higher preference for the ideas created by AI than by the Wharton students…Of the 400 ideas generated, only five human-created ideas were among the 40 most desirable products in this experiment.” The researchers say: “ChatGPT-4 can generate ideas much faster and cheaper than students, the ideas are on average of higher quality (as measured by purchase-intent surveys) and exhibit higher variance in quality. More important, the vast majority of the best ideas in the pooled sample are generated by ChatGPT and not by the students.”

The authors of that study, Christian Terwiesch and Karl Ulrich, wrote in The Wall Street Journal:  “First, generative AI has brought a new source of ideas to the world. Not using this source would be a sin…Second, the bottleneck for the early phases of the innovation process in organizations now shifts from generating ideas to evaluating ideas.”

The question is whether organizations are open to unexpected results when evaluating those new ideas.

-----------

In an era of Big Science, with expensive projects staffed by dozens or hundreds of researchers, is there flexibility to pursue unexpected results? In an era of corporate innovation centers, can those workers really follow the “bends and meanders,” or are they constrained by budget and corporate goals? I.e., is there room for serendipity?

Science and innovation do not show linear progress. They take unexpected turns, they run into dead ends, and they take leaps no one could have predicted. If we’re not open to serendipity, if we’re not encouraging it and on the lookout for it, we’re going to miss those leaps.