Machine learning as an innovation accelerator

The speed of innovation increases when new knowledge or new technologies are themselves used to discover the next round of new technologies. A canonical example is in computer processors — where engineers use the latest processors to help them design and optimize the next generation of processors. This is essentially what enables “Moore’s law” — the observation that computer capability increases exponentially over time. (This is how today’s smartphones became a hundred times more powerful than desktop computers from 20 years ago.)

By contrast, if we were still using paper and pencil to design the latest processors (as was necessary before computers existed), we would expect computer capability to increase only linearly, as we worked out improvements at the same rate that was achievable by engineers back then.

Most of the recent press and hype about “AI” — which really means machine learning with deep neural networks — focuses on direct applications such as self-driving cars and workplace automation. But I think a much more profound possibility lies in the ability of deep learning to increase the speed of innovation itself.

This is not a vague notion about “intelligence” or even a discussion about the extent to which computers can replace humans. Rather, it’s a specific capability that’s well suited to at least some types of scientific research. As David Rotman describes one such application in Technology Review:

Human researchers can explore only a tiny slice of what is possible. It’s estimated that there are as many as 1060 potentially drug-like molecules—more than the number of atoms in the solar system. But traversing seemingly unlimited possibilities is what machine learning is good at. Trained on large databases of existing molecules and their properties, the programs can explore all possible related molecules.

This by itself is not a revolution in chemistry; it’s a tool like any other. But increases in the speed of innovation build on each other. An advance aided by machine learning could very well lead to faster computer processors which themselves support even more complex machine learning — and the cycle continues.

Rotman also makes a compelling point about the compounding effects of faster research in the context of business and academia:

It takes an average of 15 to 20 years to come up with a new material, says Tonio Buonassisi, a mechanical engineer at MIT who is working with a team of scientists in Singapore to speed up the process. That’s far too long for most businesses. It’s impractical even for many academic groups. Who wants to spend years on a material that may or may not work? This is why venture-backed startups, which have generated much of the innovation in software and even biotech, have long given up on clean tech: venture capitalists generally need a return within seven years or sooner.

“A 10x acceleration [in the speed of materials discovery] is not only possible, it is necessary,” says Buonassisi, who runs a photovoltaic research lab at MIT. His goal, and that of a loosely connected network of fellow scientists, is to use AI and machine learning to get that 15-to-20-year time frame down to around two to five years by attacking the various bottlenecks in the lab, automating as much of the process as possible.

In other words, if the time needed for materials discovery can be decreased below the roughly 5-year threshold, it would kick off an explosion in investment because the payoffs finally align with human time scales.

Futurists like Ray Kurzweil have been writing about this type of acceleration for many decades. But Rotman’s article resonated with me as an antidote to the more common narratives about “AI” as a vague long-term utopia/dystopia or a narrow short-term technological advance. Far more interesting to me is how it fits into the broader story of accelerating scientific advancement.

[Leave a Comment]

Frying pan of shame

“We carry [our] shame with us in hopes of preventing it from happening again, but that is not [necessary]. If someone clocks me in the head with a frying pan, that’s going to hurt like hell. In order to remember that it hurts, do I need to hit myself with a frying pan every day? I sure hope not. So let’s all put down the frying pan of shame and find a better path forward.” (Self-forgiveness.)

-Jackson MacKenzie, Whole Again (p. 190)

[Leave a Comment]

Forgiveness is internal

“You should not need to feel compelled to do anything as you work on forgiveness. This is an internal process, not one involving reconciliation or contact.

“Your love or understanding of [another] person will not prevent them from continuing to harm you, unless they are also doing the hard work to heal themselves. Wounded people may pretend to be healed so that you’ll let them back into your life, only to continue to harm you.

“If at any point your forgiveness process convinces you to invite an abuser back into your life (or even talk to them), this is not the kind of forgiveness we’re looking for. It will actually impede your own progress.”

-Jackson MacKenzie, Whole Again (p. 207-212)

[Leave a Comment]

Illusions of healing

“The way in which we approach healing (or healing exercises, like therapy or forgiveness or meditation) is [clouded by] our own protective self. [For example], perfectionists use [spiritual and healing practices] to become what they think an ideal spiritual person should look like, eternally seeking to be “good enough” for spiritual love. Codependents use it to dismiss their own needs and emotions, deciding they must rescue and help even more people in order to achieve selfless sainthood. Narcissists use it to start cults and show others how worldly and wise they are. Borderlines use it to seek sympathy and validation from a higher power for their poor decisions, and then feel betrayed when their decisions inevitably backfire. Avoidants use it to stay lost in their imagination, viewing their own healing through the lens of invented characters.

“The protective self convinces you that if you “do” this thing or if someone else “does” something, you will feel good. … Healing exercises like therapy or forgiveness or meditation [become yet another] external measure of worth. … In this book, I’m encouraging you to stop “doing” and instead sit with the deeply uncomfortable, frustrating sensations that arise when you don’t take action.”

Jackson MacKenzie, Whole Again (p. 23-24)

[Leave a Comment]

The paradox of acceptance

“The curious paradox is that when I accept myself just as I am, then I can change.”

-Carl Rogers (as quoted in Tara Brach, Radical Acceptance)

[Leave a Comment]

Nesting dolls

“We’re all kind of like Russian nesting dolls. As we get older, we keep putting on all of these costumes. For me, growing up, that’s what I thought I had to do — to mature, to age, to get wisdom — is to put on all these different costumes and see which one fit…. I realize [now] that the more you can actually take those costumes off and get down to that little, small, immobile Russian nesting doll — that is who you are, your true, true self. That is the humanity of all of us. We all are in there.”

-Abby Wambach (via On Being)

This reminded me of something I used to say: “everyone has an inner nut.”

[Leave a Comment]

Learning is non-linear

“Not long ago, in the 1960s, mathematicians and scientists began to notice a property of natural systems that had been overlooked since the dawn of science: that tiny changes of condition, even in stable systems, can have dramatic and often unpredictable effects. … For example, if one adds a reagent, one drop at a time, to a chemical solution, nothing may happen at all until, with the addition of a single drop, the whole mixture changes color.

“I have witnessed the same progression in dozens of students: a surprising leap forward, followed by a period where the student appears to have reached the limits of their abilities; then another tiny advance that precipitates another leap. … Lisa, who couldn’t count by twos in Grade 6, now teaches herself new material from a difficult Grade 9 text.

“The fact that mathematical ability appears spontaneously in a gifted child is cited as evidence that ability is determined by genetics. But if the mind, like other complex systems, is subject to chaotic and non-linear effects, even siblings with the same genetic features, and who are offered the same opportunities, might develop entirely different abilities. Some small event in early childhood or at school might start an avalanche of learning in one child but not another. The fact that an avalanche occurs on one mountain and not another… does not prove that one mountain is more prone to avalanches or that an avalanche could never be started.”

-John Mighton, The Myth of Ability (p.18-20)

[Leave a Comment]

Transparent Journalism

It’s common practice for journalists to disclose conflicts of interest such as “the author owns shares in Amazon” or “the newspaper is owned by Jeff Bezos”.

It’s also standard for opinion writers to explain their personal background or area of expertise, so that readers can assess the credibility of their proposals.

But there is another type of transparency I would like to see applied to regular news stories: transparency about the amount of effort that was put into research and seeking other points of view. Put another way, I want to know to what extent the story is based on a press release that only reflects one organization’s narrative.

For example, when a new computer chip for smart watches was recently announced, most news stories simply included a selection of talking points from the press release, focusing on improvements in battery life. However, one journalist did more of an investigation and discovered a relevant twist: the new chip uses 5-year-old fabrication technology and its speed has not improved in several years. The new chip merely has the ability to slow down even more to save energy. Meanwhile, a competitor’s chip has gotten 3x faster in the past two years alone without impacting battery life.

After reading the investigative article, I felt misled by earlier coverage. The takeaway from the press-release-based articles was: “New chip has better battery life.” The takeaway from the investigation was quite different: “New chip falls even further behind the competition despite battery life improvements.”

My goal here is not to scold journalists for failing to uncover all relevant details. After all, investigative journalism is expensive and time consuming and there is pressure to produce content as quickly and cheaply as possible. Instead, what I’m requesting is more transparency about the extent to which a given story has been researched — and thus, the likelihood that it may be missing relevant details and alternate points of view.

Providing a list of sources would be a helpful step. A list such as the following would provide an indication that the story is essentially a press release and thus likely to be one-sided:

  • Qualcomm press release (Sep 10, 2018)
  • Jim Berger, Qualcomm Public Relations

Additional sources and research would indicate deeper investigation:

  • Sharon Rutledge, chip design lecturer, Columbia University
  • Anonymous industry insider (based in Shenzhen)
  • 4 hours, independent lab testing
  • 2 hours, independent research on competing products

Investigative pieces often emphasize the substantial effort that has been undertaken to uncover and validate relevant information. I can imagine why news organizations may be hesitant to emphasize a lack of research in some stories. But ultimately, providing this information would make a news outlet more trustworthy to me — while still allowing journalists flexibility to balance the relative priorities of speed, cost, and depth.

[Leave a Comment]

Contextual Journalism

If journalists want to help us understand the world, they need to provide some context.

For example, an article about Brexit politics includes the sentence: “Vince Cable, leader of the Liberal Democrat party and an arch opponent of Brexit, [criticized] the announcements and their timing.”

Since many readers don’t know who Vince Cable is, the information about his position and views are necessary to help make sense of the quote. Without it, for most readers the sentence would translate to: “Someone criticized the announcements and their timing.” Which of course is not particularly useful or interesting.

However, news reports rarely provide this type of context for numbers. For example, many recent stories cover the news that “the president is demanding $5 billion in funding for a border wall” but very few of them include the context needed to understand that number. Is $5 billion a lot or a little in this scenario? For most readers, the sentence may as well be: “The president is demanding a number of dollars.”

For the same reason audiences need context for people and places, we need context for numbers. Here’s an attempt at the version I would like to see: “The president demanded $5 billion in funding for a border wall — triple the amount spent in 2018 on border walls and a 16% increase in overall federal spending on immigration and border security.”

I’m not asking for full-on data journalism here — just a small bit of additional information to put things in context. One or two points of comparison with basic arithmetic would be a major step forward.

I created a few more examples:

“The terrorist attacks of 9/11 killed 2996 people — comprising about one third of US deaths on that day and 2% of all mortal injuries that year. (By comparison, 42,443 US residents died in 2001 in car accidents.)” [Data from CDC]

“Researchers estimate that US involvement in the war in Iraq cost $750 billion from 2003-2010, or 3% of the federal budget during those years. (This was 200 times the amount spent on traffic safety.)”

In these examples, I tried to provide some balance by including both a larger and smaller point of reference. I also did my best to choose comparisons that most people would agree are relevant (perhaps slightly less so in the parentheticals). Of course, judgement and subjectivity is involved in choosing which comparisons to include — just like any other aspect of journalism, which always involves choices about which stories to report and which facts to highlight.

I had to spend significant time searching through documents to find the data I wanted for my comparisons — enough work to reinforce the idea that it’s unreasonable to expect casual news audiences to do this spontaneously, but not so difficult that it seems to present a major barrier to a professional newsroom. Much of what’s needed is basic government data on population statistics, economics, health, spending, etc., which can be reused across many stories. And experts already being interviewed for a story may know which metrics would provide useful context in their field.

Perhaps there’s resistance because context can make a story less sensational. “$5 billion” sounds more impressive than “a fraction of a percent of the federal budget”. But of course that’s part of why I want contextual journalism in the first place — I’m tired of hype. I’m looking for an accurate picture of what’s happening in the world.

[Leave a Comment]

Compassionate Journalism

“When we settle our attention on other people’s feelings and needs, we experience our common humanity. … I’ve learned that I enjoy human beings more if I [focus] on what’s going on in their hearts and [don’t get] caught up with the stuff in their heads.”

-Marshall Rosenberg (Nonviolent Communication, p.151)

Lately I’ve been wondering what it would look like to create a news source based on the practices of compassionate communication. What would it be like to report the news in a way that focuses attention on people’s feelings and needs rather than their judgements and opinions?

As an experiment, I started re-writing an existing news article and quickly realized that the effort would require going back to the source and performing new interviews — asking a different set of questions to try to uncover what the various parties were feeling and needing.

So for the next experiment, I took one of Marshall Rosenberg’s dialogues as the basis for a hypothetical news segment. Here is part of the interview:

Palestinian crowd: “Murderer! Assassin! Child-killer!”

American interviewer: “Are you angry because you would like my government to use its resources differently?”

Palestinian man: “Damn right I’m angry! You think we need tear gas? We need sewers, not your tear gas! We need housing! We need to have our own country!”

Interviewer: “So you’re furious and would appreciate some support in improving your living conditions and gaining political independence?”

Man: “Do you know what it’s like to live here for twenty-seven years the way I have with my family—children and all? Have you got the faintest idea what that’s been like for us?”

Interviewer: “Sounds like you’re feeling very desperate and you’re wondering whether I or anybody else can really understand what it’s like to be living under these conditions. Am I hearing you right?”

Man: “You want to understand? Tell me, do you have children? Do they go to school? Do they have playgrounds? My son is sick! He plays in open sewage! His classroom has no books! Have you seen a school that has no books?”

Interviewer: “I hear how painful it is for you to raise your children here; you want what all parents want for their children—a good education, opportunity to play and grow in a healthy environment…”

Man: “That’s right, the basics! Human rights—isn’t that what you Americans call it? …”

How might a journalist report on her experience in Palestine?

  • Palestinians View American Involvement as Child Murder
  • Palestinians Furious Over American Participation In Conflict
  • Palestinians Request Desperately Needed Supplies and Infrastructure

All of these versions are equally true — they just emphasize different aspects of the truth. The first reports on what people are thinking; the second reveals what people are feeling; and the third focuses on what people are needing and requesting.

I can feel the wide disparity in my own reactions to these headlines. The first puts me on the defensive (as an American), with tightening muscles and a desire to hurl my own accusations. The second feels more neutral; a description of an emotion. And reading the third, I soften as I imagine the difficult situation.

The news is not simply an impartial collection of facts. It reflects a series of choices about which facts are included and how they are framed. Compassionate communication demonstrates how to look for shared human experience rather than judgements and blame. It seems possible to me that by focusing on the human needs driving current events, compassionate journalism could actively promote empathy toward a wide range of people and situations.

[Leave a Comment]

The Cynic

“The [person] who looks least engaged may be the most committed member of the group. A cynic, after all, is a passionate person who does not want to be disappointed again.”

-Benjamin Zander, The Art of Possibility (p.39)

[Leave a Comment]

Blame

“People are used to hearing blame. Sometimes they agree with it and hate themselves—which doesn’t stop them from behaving the same way—and sometimes they hate us for calling them racists or whatever—which also doesn’t stop their behavior.”

-Marshall Rosenberg, Nonviolent Communication (p.152)

[Leave a Comment]

The paradox of loneliness

“One of the tragic ironies of modern life is that so many people feel isolated from each other by the very feelings they have in common.”

-Sir Ken Robinson
(as quoted in Brené Brown, Rising Strong)

[Leave a Comment]

Doing our best

“[I’ve found that] my life is better when I assume [that] people are doing the best they can with the tools they have. … It keeps me out of judgment and lets me focus on what is, and not what should or could be.”

-Brené Brown (Rising Strong p. 108-113,
quote attributed to her husband Steve)

[Leave a Comment]

The myth of closure

“There is no such thing as closure. … Once you’ve become attached to somebody, love them, care about them — when they’re lost, you still care about them. … I don’t like to use the word ‘acceptance’, but I think we can try to be comfortable with what we cannot solve.”

-Pauline Boss (via On Being)

[Leave a Comment]

Compassionate thinking

“While studying the factors that affect our ability to stay compassionate, I was struck by the crucial role of language and our use of words. [However], Nonviolent or Compassionate Communication (NVC) is more than a process or a language. On a deeper level, it is an ongoing reminder to keep our attention focused on a place where we are more likely to get what we are seeking…. The essence of NVC is in our consciousness of the four components, not in the actual words that are exchanged.”

-Marshall Rosenberg, Nonviolent Communication (p. 2-8)
(italics mine)

[Leave a Comment]

Burnout

“One sign that I am violating my own nature in the name of nobility is a condition called burnout. Though usually regarded as the result of trying to give too much, burnout in my experience results from trying to give what I do not possess — the ultimate in giving too little! Burnout is a state of emptiness, to be sure, but it does not result from giving all I have: it merely reveals the nothingness from which I was trying to give in the first place.”

-Parker Palmer (Let Your Life Speak, p.49)

[Leave a Comment]

Limits

“Americans… resist the very idea of limits, regarding limits of all sorts as temporary and regrettable impositions on our lives. Our national myth is about the endless defiance of limits: opening the western frontier, breaking the speed of sound, dropping people on the moon, discovering ‘cyberspace’ at the very moment when we have filled old-fashioned space with so much junk that we can barely move. We refuse to take no for an answer.

“Despite the American myth, I cannot be or do whatever I desire… there are some roles and relationships in which we thrive and others in which we wither and die.

“If I try to be or do something noble that has nothing to do with who I am, I may look good to others and to myself for a while. But the fact that I am exceeding my limits will eventually have consequences. I will distort myself, the other, and our relationship — and may end up doing more damage than if I had never set out to do this particular ‘good’.”

-Parker Palmer (Let Your Life Speak, p.42-46)

[Leave a Comment]

Abundance

“Authentic abundance does not lie in secured stockpiles of food or cash or influence or affection but in belonging to a community where we can give those goods to others who need them — and receive them from others when we are in need. … Abundance is a communal act… in which each part functions on behalf of the whole and, in return, is sustained by the whole.

“Community doesn’t just create abundance — community is abundance.”

-Parker Palmer (Let Your Life Speak, p.108)

[Leave a Comment]

The spiritual journey

“In the deeps are the violence and terror of which psychology has warned us. But if you ride these monsters down, if you drop with them farther over the world’s rim, you find what our sciences cannot locate or name, the substrate, the ocean, …the unified field: our complex and inexplicable caring for each other, and for our life together here. This is given. It is not learned.”

-Annie Dillard (as quoted in Palmer, Let Your Life Speak, p.80)

[Leave a Comment]

Adding a delay to the end of an animated gif

I converted a screen recording to a looping animated gif. It looked and played fine, but I also wanted it to pause for a few seconds at the end before starting the loop again. I couldn’t find an easy way to do this using online tools without increasing the file size of the result.

I eventually figured out how to add the delay using the open-source command-line tool gifsicle. Here is the command:

gifsicle -U original.gif "#0--2" -d200 "#-1" -O2 > with-delay.gif
  • The -U option unoptimizes the input gif so that we can operate on individual frames.
  • -d specifies the delay to use in hundredths of a second.
  • “#0” format specifies a frame number or range of frames. Negative numbers count backwards from the last frame, starting with #-1. So the range from the first to second-to-last frame is #0 to #-2 or “#0–2”.
  • -O2 (capital letter O) directs gifsicle to re-optimize the gif using recommended settings.

In summary, this command unoptimizes (-U) the original gif into its component frames; then says we want a new gif with the first frame through the second-to-last frame (#0–2) unchanged and the last frame (#-1) with a new delay of 2 seconds (200 hundredths of a second); and finally specifies that we want to re-optimize the result (-O2).

Thanks to the gifsicle manual and a reddit user for helpful hints.

[Leave a Comment]

Microwave cooking frozen fish

I don’t think it’s widely known that (1) you don’t need to defrost fish before cooking, and (2) microwave cooking is not only fast and easy but yields better-tasting, melt-in-your-mouth results (at least for hapless chefs like me).

Most “fresh” fish sold in grocery stores is actually defrosted frozen fish. So it seems to me that you may as well just buy it frozen and let it sit in your freezer until you’re ready to cook. At that point, defrosting is time-consuming and smelly, so you may as well just cook it from frozen.

After doing some internet research and experimenting with frozen salmon and cod fillets, I feel it is my civic duty to share with the world the easiest, least messy, and most delicious way to cook fish.

  1. Rinse the frozen fillet in warm water to remove its crust of ice, then pat dry with a paper towel.
    (Do not skip this step! I learned this the hard way. The fish interior will not cook if there is too much ice/water on the outside. Also: you need to pat-dry the fish quickly or else the paper towel will stick.)
  2. Place the fillet on a plate. Drizzle with oil, salt, pepper, and any spices you want.
    (I often just spread coconut oil with a little salt and pepper and maybe some dried sumac. The internet is full of ideas… pesto, sriracha, mayonnaise, lemon, parsley… just be aware that too much water content will affect the cooking.)
  3. Cover the plate lightly.
    (It does not need to be airtight. I just use a normal plastic microwave cover.)
  4. Microwave on full power for 4 minutes.
    (This works well for an 8oz fillet in a 1000-watt microwave. A large fillet needs an extra minute or two of cook time.)
  5. Wait 1-2 minutes after the microwave stops. Then check the interior with a fork for uncooked areas (they appear translucent or bright). If there are uncooked areas, microwave again for 1 minute and wait 1 minute.
    (The reason for waiting is that the heat inside the fish continues to cook for a while even after the microwave stops.)

Over time you get a feel for how long you need to cook different fillet sizes in your microwave. The goal of course is for it to come out fully cooked (but not overcooked) after the first microwave session. In my 1000-watt microwave, 4-5 minutes is usually enough (plus the 1-2 minutes of waiting).

[Leave a Comment]

Interactive blog post

I updated my javascript library and blog to play well together.

That means I can illustrate a point about interest rates with an interactive widget for you to play with. (Go ahead, try it!)



(For more details, see the Visual Loan Calculator.)

It’s all part of the vision to make it easier to communicate using the tools of explorable explanations.

[Leave a Comment]

Religion from the perspective of game theory

Religion serves many purposes, but I’ve come to believe that one of the most important is helping people cooperate with each other. “Do unto others as you would have others do unto you.”

Many religions label behaviors that serve the community as “good” and those that are overly self-serving or detrimental to others as “evil”. Individuals are often cast as having a “light side” and a “dark side” or dual influences from “god” and “satan”. Religion helps and encourages people to engage in the former type of behaviors.

Robert Wright realized that we can insightfully describe this using the language of game theory.

Specifically, some situations are “zero-sum” or “win-lose”, meaning that one person’s gain is the other’s loss. For example, if there is only one rabbit to eat, then if I eat it, you go hungry.

Other situations are “non-zero-sum” or “win-win” meaning that cooperation results in gains for both people. For example, if we can only catch rabbits by working together, then cooperation is what allows us both to eat — cooperation leads to abundance.

Humans are equipped to handle both types of situations. If we sense a zero-sum situation, hormones like cortisol flood our body and we become protective, aggressive, and narrow-minded. If we sense a non-zero-sum situation, hormones like oxytocin flood our body and we become generous, caring and creative.

I’ve come to believe that one of the key insights at the core of all successful religions is that nearly every situation can been seen as non-zero-sum. In other words, we can nearly always apply creativity to find win-win solutions (which generate abundance) even if a situation at first seems clearly win-lose. The key hurdle is that our ability to be creative and generous depends on our non-zero-sum hormones being activated. In other words, this is a self-fulfilling prophesy where believing that a situation is win-win is required to put us in the right state of mind and body to figure out what set of actions will make it win-win.

Viewed in this light, many (even most?) religious and spiritual practices are ways of maintaining that community-oriented, win-win mindset. For example, an emphasis on forgiveness helps us return to a cooperative stance with a person who has betrayed our trust. A practice of gratitude inspires us with past examples of generosity and abundance. A weekly gathering helps us stay connected and aware of the community which we all depend on.

It makes sense to me that successful religions became successful in part because they helped communities cooperate and create more win-win solutions leading to abundance, relative to religions which were less effective at promoting cooperation and whose communities thus missed out on valuable opportunities.

The Dalai Lama has often said, “My religion is kindness.”

I’ve come to believe that this has a sound basis in mathematics.

[Leave a Comment]

Tools

“A tool can give you something concrete that you can imagine using, but a tool is not going to give you persistence… it’s not going to tell you which path to take. It’s not going to give you mettle… when everything feels like it’s been destroyed around you, you can’t just pull out a tool… Where do we nourish and foster the creative imagination that permits you to bring into the world something that does not now exist?”

-John Paul Lederach (via On Being)

[Leave a Comment]