Sam Arbesman is a complex systems scientist and writer with a PhD in computational biology from Cornell University and a BA in computer science and biology from Brandeis University. Sam’s most recent book, The Half-Life of Facts (Current/Penguin, 2012), explores how different fields of knowledge—medicine, physics, technology—change over time. Sam argues that knowledge in most fields evolves systematically and predictably, and that any given field’s change in knowledge can be measured like the half-life of a radioactive element. We can measure when facts will be rendered obsolete, the rate at which new facts are created, and even how facts spread. This measurable evolution of truth, fact, and reality can have a powerful impact on our lives.
You can learn more about Sam at his website, https://www.arbesman.net, follow him on Twitter at @arbesman, or read his blog, Social Dimension, at Wired.com.
This interview has been edited for length and clarity.
Joey Eschrich, Project Hieroglyph: Could you tell us a little about yourself and what you’re working on right now?
Sam Arbesman: I’m currently Senior Scholar at the Kauffman Foundation based in Kansas City, as well as an Associate of the Institute for Quantitative Social Science at Harvard University. I’m currently thinking about the models we build in science, as well as the systems we build in technology. Specifically, I’m considering the central role that the tension between elegance and complexity has played in our thinking about science, technology, and the world around us.
So on one hand, especially in science, we want single equations to explain the world. We want elegant solutions: the Pythagorean theorem, or E=mc2. We frequently want the same thing in our technology—think about the attractive simplicity of the original iPod. But often, I think, the world doesn’t adhere to our desires, so we need to create fairly complicated models or systems that are glitchy and that kind of work and kind of get the job done, but are not particularly clean or simple.
In some cases, not only are these systems more complicated than we might have wanted; they’re so complicated that even the experts don’t fully understand how they work anymore. A question I’ve been asking about this is: Should we be concerned about this seeming escalation of complexity? Should we say, “This is how it’s always been and this is not a problem?” Or should we be trying some sort of middle path between simplicity and complexity?
When we think about all the things that we won’t understand, or computers making discoveries that make no sense to us, or creating systems that we don’t understand anymore, we think that maybe these things are very far at the margins of experience, or maybe not even going to be happening for decades or even centuries. But it turns out they’re already happening and they’re happening more often than we might have realized. Especially within science, with the advent of Big Data, we have very powerful means of deriving meaning from data or saying, “All right, we can create a model that has predictive value.” But whether or not we fully understand how that model works is a completely separate question.
JE: Can you give us an example of a situation where even experts were confused about an important development in their own field?
SA: In the years leading up to Y2K there were some experts in information technology and computing who were saying that we were heading towards disaster, and others were people saying, “Oh, don’t worry about it.” And it turned out (and I think actually partly because of lot people actively worked to fix it) it wasn’t actually a big problem. But it was interesting to see leading up to that point, there were many people, even experts, who genuinely weren’t sure what was going to happen.
Another example of this is a system called TCAS (traffic collision avoidance system). It’s what airplanes use to make sure they don’t crash into each other in the middle of the air. It turns out that there are only a handful of people who really understand how this system works. And it’s not because it’s legacy code or some antiquated system, it’s because it’s so enormously complicated and there are a lot of conditional statements—Under this condition, this thing has to happen, unless this has already happened—and it’s such a weird system that often the experts who work with it on a daily basis are surprised by its behavior when they’re simulating it.
And so we’re beginning to see that we find elegance where we want to, but behind that there is a massive amount of complication and complexity that we don’t fully understand.
JE: That reminds me a little bit of this article I recently read about Netflix, about their recommendation algorithms. Their engineers are often surprised about the types of recommendations that come up and we, as consumers, are frequently surprised as well. But they don’t really know exactly how the algorithm works sometimes—it can act in ways that seem kind of capricious.
SA: Absolutely. I think we’re seeing this more and more and I’m trying to figure out how we should respond to that. And this has actually led me in some fun directions. I’ve been talking to philosophers of science and also historians as well, because a lot of things we’re thinking about aren’t particularly new questions—they’re just heightened because of the increasing power of our computers and our machines.
While I’m oversimplifying a great deal, with the advent of the Scientific Revolution, there was this idea that if we put our minds to it, everything was amenable to our querying of nature. That we could completely understand the universe. In the Middle Ages, with more of a focus on the infinite, there was a sense that there were only so many things that you could ask questions about and actually get answers. So in the 12th century the philosopher and physician Maimonides, in his book The Guide for the Perplexed, gives a nice little list of things we could know for sure, including the number of stars in the sky and whether that number is even or odd. Or the number of planets or the number of types of animals, things like that. And in fact if you look within science and we actually now know the number of stars visible to the naked eye, although not necessarily the total number of stars. We know that number and it’s around 9,000 and the number is, of course, even.
And in a way, we’ve had this kind of triumphalist attitude when it comes to science. But now we’re beginning to say, “Well, maybe there are actually some limits to the systems we can understand.” We should recognize that sometimes when we hit our limits that’s okay. It doesn’t have to make us worried.
JE: Do you think some of this growing complexity has to do with the massively collaborative nature of some of these enterprises and the ways that they’re institutionalized—instead of this historical model of a unitary great thinker who comprehends everything?
SA: I definitely think so, and it varies from system to system. But I think one of the reasons we see increasing complexity is because it’s a lot easier to add to something and to modify something than to take the entire thing and then re-examine it. You can actually see in the nature of law and regulation, which is one of these systems I think about as a technological system, although we don’t traditionally think of it that way. And it is a system that we ourselves have built. But it’s a lot easier to add new laws or new regulations rather than to scrap everything entirely. Now it’s even easier—we have these widely distributed digital systems through which you can collaborate very easily.
When people add things they often interact with the things that are already there in ways that we might not have anticipated: and in the old situations, laws have unintended consequences, or these really massive technological systems are buggy in ways that we don’t fully understand.
As science grows and grows, no single person can read all those scientific papers, or even the new papers within their own specialty. So you have a situation where there’s probably a lot of discoveries that can be made simply by connecting one paper to another. This dates back in the 1980s, when computer searching was in its infancy, when the information scientist Don Swanson said, “Imagine there’s a paper connecting concept A with concept B, and another paper that connects concept B with concept C. If you connect them all together, maybe concept A implies concept C.” But the literature is so large that no one would actually be able to connect these two things on their own. And we now actually have computer programs that can kind of step in and help build those links.
As we specialize, people still have expertise in certain areas, but it becomes more and more difficult to see the connections among a variety of fields.
JE: So you’re working, talking, and collaborating with these philosophers and historians and trying to see what perspectives they can add to your work on elegance and complexity in science. That’s very much what Project Hieroglyph is all about: finding the unexpected connections between different fields and figuring out how they might come together in a bigger picture, a bigger story.
How did those people—who don’t consider themselves “scientists” in a literal way—help you think more rigorously about science?
A: I learned that the topics that I’m thinking about and the questions I’m asking have been taken up by many people in many different ways. I’ve also been given lots and lots of interesting examples to think about.
As an example, there’s this Yiddish term naches, which is used when you have a certain vicarious pride and joy in the accomplishments of your kids. So you have naches or shep naches, when your kid gets married or they have their bar mitzvah or graduation. So even if we can’t understand the discoveries computers are making, that’s okay, because they’re our creations and so we can actually maybe have naches for our machines. And I think I wrote something about this and then someone on Twitter said, “Oh, actually there’s this great science fiction short story that discusses the same kind of thing.” They don’t use that Yiddish term, but it was just great to see that oftentimes when you interact more broadly with people from all different areas, you get to see how other people have approached these same ideas.
Having talked with a number of historians and philosophers, I have a much better sense of how these trends have ebbed and flowed over time. And it’s been remarkable.
JE: Do we need to tell better stories about complexity? Do we need narratives that make messiness and complexity seem as beautiful and enriching as simplicity and elegance?
SA: Some of this is just about expectations and mindset. If we constantly yearn for elegance, don’t find it and are disappointed, that’s bad, because we’re often not going to find it. But if we say, “the world can be simple and elegant and that’s really cool—but the world can also be really complicated and that’s great too,” then I think that’s even better.
Storytelling can be a powerful vehicle for inspiring people about both elegance and complexity. You can have a story that unifies everything and helps us think about the world as an elegant gestalt. But some of the best-loved stories revel in the exception or the weird situation—they’re not always tied up in a neat bow.
JE: One of Project Hieroglyph’s fundamental precepts is that we had a more optimistic, maybe even utopian discourse about science, technology and the future in the 1950s and 60s, and now we’re not telling those optimistic stories anymore. Does this ring true for you?
SA: Well, I think it’s more a question of what gets the publicity. It’s a lot easier to be successful when you’re pessimistic because that’s what gets the headlines. And it’s the same reason when you watch the evening news, it’s not going to be feel-good story after feel-good story. It’s going to be that something has gone bad because people’s attention is naturally drawn to disruption and danger. In fiction, and in television, people have to some extent moved toward these grittier stories that depict the seedy underbelly of the world.
But at the same time, we still see a lot of very optimistic stories. People are still publishing space operas. It isn’t only dystopian near futures in our fiction, although the darker stuff is perhaps more popular.
People have become a bit more embarrassed about enjoying optimistic stories, and maybe that’s the thing we need push back against. We might need a more unabashed willingness to say, “Yeah, the world is an awesome place, and it can get better.”
JE: What story has inspired you most? Or most profoundly shaped the way you think about the world?
SA: I’ll say the entire world building exercise of Star Trek, rather than just one of the series. When Next Generation came out, I was at a very tender age, kindergarten or first grade, so I watched those with my family. I’m certain I didn’t get out nearly as much as I could have at that age. But they inspired me to think about the world in a completely different way.
I remember watching one episode that focused on something about DNA and genetics. When I first saw the episode, I didn’t know anything about genetics. And I remember having a very long conversation with my father afterwards about how DNA works and RNA and proteins and enzymes. It was incredibly eye-opening and then I think that inspired me to get more involved in biology. And my PhD is in computational biology, so that was obviously a critical moment for me.
And so, at the risk of coming across as a hopeless fanboy, I think the Star Trek franchise ultimately provides a venue for discussing really interesting ideas about technology, but also a lot of important social and cultural ideas as well. And from a firmly optimistic perspective.
I think that’s really what good science fiction can do. Science fiction isn’t just an excuse to talk about transporters or spaceships. I mean, sometimes it is, because those things are cool. But it’s ultimately a great genre of ideas and I think many episodes of Star Trek embody that potential in its purest form.
JE: I love that answer because when I go and visit classes of high school kids and middle school kids to talk about Project Hieroglyph and our work with science fiction, I always talk about it as an ideal meeting place for ideas, because everybody feels like an expert about it, in one way or another. Everyone feels like they know enough to take part in the conversation. Whether you’re a person who loves to read, or an encyclopedic Star Trek fan, or whether you’re a rocket scientist, or an industrial designer, everybody has a claim to some sort of expertise, and Star Trek is a great example of that, because it’s an incredibly inclusive cultural touchstone.
One last question for you: one of our challenges with Project Hieroglyph is to try to tell optimistic and inspiring stories about the future. As someone who is deeply engaged with science and technology, how do you think we can tell optimistic and inspiring stories about the future that are still thoughtful and critical?
A: Well I’m very optimistic by nature—some people have told me that I’m too optimistic. I’m willing to wear that as a badge of pride. I feel that in general, you can look at every trend with an optimistic lens or a pessimistic lens. We’re constantly at the frontier of the unknown and there are always many things we don’t know. But science is always in a draft form, and the potential to be able to revise our understanding and incorporate new discoveries is really powerful and amazing.
For me, to be optimistic we don’t need to always strive for elegance and closure. We can revel in a universe of wonderful details that don’t necessarily add up to anything perfectly coherent, but are interesting in and of themselves.
In the face of this massively increasing technological and social complexity of our world, in the face of people’s fear of information overload, we need to realize that these are not bad things. Irreducible complexity isn’t bad. We can find an incredible diversity of cool phenomena in our world and respond by saying, “Wow, the world is overwhelmingly beautiful.” It can be overwhelming, but that complexity is also amazing, and we have the opportunity to constantly explore further and deeper and learn more about it.