Robin Stewart: portfolio
icon

Moving Math

TeraFractal for Mac & iPad

Summary

TeraFractal combines real-time graphics rendering technology with an innovative direct manipulation interface that lets users intuitively experiment with fractals. After pioneering this concept on the desktop, I also created a touch-based version for iPad. The result is a tool that makes advanced math fun for 3-year-olds and PhDs alike.

Background

When I first came across fractals based on iterated function systems, the best software interfaces looked something like this:

Mockup of the legacy fractal rendering interface

You would type in some numbers, press “Render”, and get a fractal like this:

Fractal

Then you would go back to the fields of numbers, change some of them, and press "Render" again to see the result:

Fractal with some rotation

And after repeating this process many times, you might eventually converge on the fractal you were hoping for:

Fractal with many adjustments

This type of interface is the status quo for scientific computation software: set up parameters, render/simulate, view results. That workflow makes sense when it takes a relatively long time to perform the computation. But computers have gotten so much faster in recent years that in many cases the entire simulation can be run in a fraction of a second. The computer ends up spending 90% of its time sitting idle, waiting for the human to enter new numbers.

If results can be calculated and displayed within about a tenth of a second, then it is possible to achieve the experience of real-time control. Instead of the "render button" workflow, we can start using a direct manipulation interface where results are shown immediately.

This is exactly what I did for fractals.

Process

I had read about an efficient, randomized algorithm for rendering the types of fractals shown above, so I set about implementing it to see if I could get it to run fast enough. My first attempts were too slow; but by making full use of the graphics card (via OpenGL), I was able to get the rendering time below the tenth-of-a-second benchmark.

Next, instead of the arrays of text fields, I created an interface that represents fractal parameters as sets of colored boxes.

The colored-box interface for fractal parameters

The position, rotation, size, and shapes of the boxes all correspond to fractal parameters which can be easily changed by clicking and dragging the boxes and controls. Thanks to the earlier engineering work, all adjustments are reflected in the corresponding image in real time.

A fractal created with the new interface

In the video below, I use this interface to quickly create a lifelike fractal.

Iterations

When the iPad was released, I realized that it presented an opportunity to take this idea further. Instead of adjusting the control boxes, wouldn't it be interesting if you could directly manipulate the fractal itself with your fingers? I envisioned an even simpler user interface: an iPad as a window onto a fractal.

Vision of a fractal iPad app

First I needed to adapt the rendering algorithm to the less-powerful mobile processors in an iPad. Once I had gotten that to run within a tenth of a second, I turned my attention to the touch screen.

Instead of using the colored boxes from the desktop app, I mapped the parameter space onto the same area where the fractal was being drawn. For example, dragging with one finger adjusts the virtual box closest to where the drag started. But since you can't see the box, it feels like you're manipulating the fractal image itself.

I started experimenting with more gestures to adjust the rotation and scaling factors. What ended up working best was a combination pinch-and-rotate gesture that adjusts all of the parameters simultaneously, depending on the absolute and relative positions of your two fingers on the screen.

Next Steps

The invisible-box interface is fun because it feels like manipulating the fractal directly. However, it can also be frustrating because it's such a black box. It's hard to get to specific fractals because the interface doesn't provide enough clues about the current state of the underlying parameters.

I'm working on ways to achieve the best of both worlds. My favorite idea so far is to overlay the fractal with translucent touch controls:

Translucent touch controls for adjusting fractal parameters

The positions of the rings and the positions of the knobs relative to their rings concisely represent all of the fractal parameters. These controls can also be dismissed with a tap to return to an unimpeded view of the fractal image.