Innovation is not linear

The question of how scientific advances flow through to practical applications is central to our understanding of progress, and guides research funding policy.

The tone on this issue for the second half of the 20th century was set in 1945 by Vannevar Bush’s landmark report “Science, the Endless Frontier”. One theme, already widely accepted in American industrial culture and given official voice in Bush’s report, was that basic research, although not guided by practical uses, is nonetheless necessary for long-term progress:

“Basic research is performed without thought of practical ends. … The scientist doing basic research may not be at all interested in the practical applications of his work, yet the further progress of industrial development would eventually stagnate if basic scientific research were long neglected.”

By the early 1950s, the newly created National Science Foundation was justifying its mission based on an explicit model of technological development as a flow from basic research through to development. Quoting from the NSF’s second annual report:

“The technological sequence consists of basic research, applied research, and development… each of the successive stages depends upon the preceding. Unlimited expansion of effort toward applied research and development, without corresponding support for basic research, will defeat the entire effort by limiting technological progress to minor improvements and refinements of obsolete processes and equipment.”

There is an important truth in this “linear model” of innovation. But there is also much more interaction between science and invention than a naive interpretation of the model would imply.

The truth in the model is that science is, in an important sense, the foundation for invention. The electric power industry would not have been possible before the science of electromagnetism: Michael Faraday’s 1831 discovery that a moving magnet creates a current in a nearby wire is the law that the electric generator, deployed widely decades later, depends on. Plastics were not created until chemistry was well-established: the first synthetic plastic, Bakelite, was created by a Belgian-American chemist, Leo Baekeland, after careful experimentation inspired by the potential of phenol-formaldehyde compounds established by previous researchers.

Technological development is possible through ad-hoc tinkering, but without a scientific base, such development eventually plateaus. Sanitation reformers made only limited progress against disease before the germ theory, because they were guided only by perceptible qualities such as the look, smell, and taste of the water supply. The understanding that microbes were the cause of disease refined their understanding of the goal and gave them new ways to measure the success of their efforts. It was after this, for example, that chlorination was added to water treatment, and pasteurization to food processing.

Practical applications often flow naturally from science: a deeper understanding of nature suggests possible uses. But science does not flow from applications, because it is not obvious what science might apply to a given problem. As Richard Nelson pointed out:

“It is seriously to be doubted whether X-ray analysis would ever have been discovered by any group of scientists who, at the turn of the century, decided to find a means for examining the inner organs of the body … It seems most unlikely that a group of scientists in the mid-nineteenth century, attempting to develop a better method of long-range communication, would have developed Maxwell’s equations and radio.”

The impact of science on invention is long-term and often impossible to foresee. There are some times, certainly, when scientific pursuits have obvious applications: when Robert Koch identified the bacteria responsible for tuberculosis, he must have known that this would someday help us prevent or cure the disease. But when Bohr peered into the structure of the atom, or when Rutherford and Curie investigated the nature of radiation, it is doubtful that they expected their work to lead to nuclear power or MRI scans.

Investments in science, then, if motivated by long-term progress, cannot be prioritized by immediate practical impact. It requires, in Bush’s words, “the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown.”

However, in the naive interpretation of this model, the stages are fully distinct, non-overlapping, and separated by a clean handoff. Science is pursued out of pure curiosity, with no thought of applications. It discovers elegant natural laws, which are crisply formulated, written down, and then thrown over the transom to engineering. Invention is then a straightforward process of deductively applying these laws to practical problems.

But the relationship between science and invention is not one-way. It is reciprocal.

Invention is rarely, if ever, a pure deduction from established physical laws. Inventors tinker, often right at the boundaries of scientific knowledge—where the best opportunities are. Edison famously tested thousands of materials as filaments for his light bulb to find one that would last long enough to make the bulb economical. Charles Goodyear experimented for many years to create an improved form of rubber that would not become soft in the heat or brittle in the cold, finally hitting on pressure-treating with sulfur as the key process to create what he would call “vulcanized” rubber. Even today, drugs are discovered by screening thousands of compounds against a target.

Science, of course, often guides this experimentation. Edison was specifically looking for a high-resistance material; he knew from his understanding of the physics of electric circuits that a high-resistance bulb would require less power. Drug compounds, including some early antibiotics, are discovered by modifying “side chains” of a known molecule. Without scientific background knowledge, pure tinkering often has too many possibilities to explore; science narrows the search.

Sometimes when an invention is produced by tinkering, science cannot explain its operating characteristics, or even why it works at all. Aspirin was first marketed in 1899, but its mechanism of action wasn’t explained until 1971. These cases can inspire new theories or even entire new fields. The field of thermodynamics was created in the 1800s in part to explain the working of heat engines, such as the steam engines that had been invented in the previous century. A better scientific understanding of technology then helps us optimize or expand it. The first vaccine, for smallpox, was invented in 1796, but no one was able to replicate this success for any other disease until almost ninety years later, after the advent of the germ theory.

When the right researchers are brought together in the right environment, this iterative process combining both science and engineering can even be driven forward by the same people, in the same lab, on a single project. A dramatic example is the invention of the transistor at Bell Labs. Researchers went back to the drawing board multiple times to improve the theory of semiconductor physics, in order to explain their experimental results. Each improvement to the theory let them make the next advance in engineering: from failed prototypes, to the first working “point-contact” transistor, to the improved “junction” transistor that became the standard design.

Some of the greatest researchers were those who were able to pivot rapidly from discovery to invention, or vice versa, as the opportunity arose. The master of this was Louis Pasteur, a pioneer of microbiology. Some of his greatest scientific discoveries came from projects with purely practical, commercial goals. His work to improve the processes of wine, beer, and vinegar makers led to the discovery of anaerobic metabolism and the role of microbes in fermentation; his assistance to the growers of silk worms gave him observations that ultimately led to the germ theory of disease. At other times, while engaged in scientific research, Pasteur was able to seize on an accidental discovery when he saw its practical applications. This is how a botched experiment with chicken cholera led to the first laboratory-created vaccines, including a vaccine for rabies.

So the “clean handoff” model is wrong. Instead, we should replace it with a “gap” model. Science gives inventors a conceptual framework: a toolbox of categories, principles, and equations that provide fertile ground for practical experimentation. But through experimentation, inventors can discover working solutions outside the bounds of what has been fully explained by science. They can’t get too far ahead of scientific understanding—Bell Labs would never even have created a semiconductor research group if physics hadn’t first identified semiconductors as a material.

But if the distance between known science and a desired invention is short enough, they can cross the gap. They can bridge the gap with their own experiments, extending scientific knowledge; or they can leap over it, ending up with an invention that works for reasons yet to be explained. Leaping the gap can actually pull science forward, giving it new phenomena to explore and new motivation to do so.

As a consequence, influence does not flow in one direction only, from science to invention. Invention feeds back into science, and rapid progress can come from a tight integration of the two.

Our assumptions about how research happens guide how we fund, organize, and manage it.

The great corporate laboratories of the early 20th century, which integrated basic research and product development under one roof, have gone into decline. Today the people, projects, and institutions who do science and invention are more separate than they were in those days. Startups and venture capitalists mostly don’t do long-term or high-risk research, focusing instead on product development based on proven technologies. Basic research is federally funded, performed in universities, and handed off to industry through a process of “tech transfer” with a reputation for being cumbersome and inefficient.

As we think about new models of funding and organization to address stagnation and revitalize progress, we should recognize that science and invention are tightly intertwined and mutually reinforcing, and seek models that integrate them more closely. Projects, programs, and institutions—and the funding that backs them—should not be siloed into one category or the other. And there should be a career path for researchers who want their work to encompass both discovery and practical applications, and who are ready, willing and able to nimbly hop from one to the other while following the scent of a breakthrough. Historically, at least, this appears to be the way to get more plastics, vaccines, and transistors.