Earlier this week, I watched an astonishing, delightful, and thought-provoking talk by Bret Victor entitled “Inventing on Principle” (partial transcript).
At the beginning of the talk, Victor describes his principle:
“Creators need an immediate connection [to their work].”
and then presents five beautifully polished demonstrations of how this principle can be applied to the design and performance of drawings, game-play, algorithms, circuits, and animations.
Below are attempted summaries of three or so of the several cascades of thought that his talk has sent roaring through my mind so far…
As will undoubtedly surprise absolutely no one who knows me, the first thing I began to wonder after the demos began was something to the effect of “how the devil did he make that?!”.
Here’s what I’ve noticed (and inferred) so far:
The demos can all be thought of as simulations, typically attached to one or two stages (canvases) displaying posed costumed actors.
The simulations respond to user inputs. Additionally, at least one of the simulators can faithfully replay input events (meaning that input events are recorded and that the resulting records include event timing information).
Most of the simulations are parameterized by time and appear to have distinct “paused” and “running” modes. These “temporal” simulators typically expose a time-scrubber and a play/pause keybinding.
The statics and dynamics of most of the simulations appear to be defined via JavaScript programs. These programs appear to give initial conditions and a function for computing future states from past states and inputs.
The editor for the JavaScript programs appears to operate on syntax trees: it performs syntax-highlighting and language-sensitive indentation, it subtly displays parse errors when input directed to the syntax pane generates texts which fail to parse, and it exposes both tangible and syntactic costumes for the actors that represent numeric literals in the program’s syntax tree.
The simulator can be configured to superimpose the posed costumes from a segment of several actors’ trajectories. As a result, I hypothesize that the interpreter for the simulation’s dynamics provides a lens into the interpreter’s state.
Several of the simulators support omnidirectional picking, meaning that the user can identify costumes being worn simultaneously by an actor by pointing to (any of?) that actor’s costumes while holding a modifier key.
Thinking through the observations above, I think I see a second principle at work in addition to the fundamental principle that Victor identified at the start of the talk. My hypothetical second principle goes something like this:
Closed programs describe scripts in the theatrical sense.
by which I actually mean something like:
Every closed program can be interpreted as a simulatable specification of motions, utterances, and costume changes for costumed actors performing on a stage in front of a director who elaborates, edits, and explores the script in real-time via pantomime, textual edits, and time-scrubbing.
This potential principle interests me because it suggests three additional opportunities for “connection” in Victor’s sense:
If regular interpreters give us a kind of forward kinematics for programs, then “symbolic interpreters” like klee and s2e ought to give us an inverse kinematics for programs, including open programs.
Such interpreters should also give us a way to interact with alternate program executions via search, enumeration, and, manipulation of tangible representations of constraints on execution (or their corresponding envelopes of feasible trajectories).
Presenting programs as manipulable histories might be a good way to introduce inexperienced users to programming and, hence, to the construction of narratives (see also: video parts 1, 2, 3, 4, 5, and 6 and more recent work).
(One primitive but eye-opening version of this sort of interface can be seen in Feliam’s animation of klee solving a maze.)
Last but not least, there’s the synchronicity and resonance of the thing: you see, back in April of 2011, I was trying to think through how the XO-3 might amplify or weaken what I took to be the principles underlying Sugar. According to my notes, my thoughts went something like this:
I like to think of tablets as tesseracts or “magic mirrors”; they are holes in the world that bring us views of distant places, that help us to communicate with people who are far away, and that give us windows into the past and the future. Thus:
Can we create an awesome power-on/home-screen experience that is biased toward action, reflection, and communication?
(think of a group of friends opening a new boxed game, or running out to play after an unexpected snow storm)
Can we give people a way to “tune in” to another part of their environment?
(think of how often we peer over friends’ shoulders or eavesdrop for a few moments in order to decide whether we want to join an activity or a conversation. maybe we could build in support for displaying a feed of other folks’ low-res screenshots in a small overlay window?)
Can we choose to publish more about the people, places, documents, things, events, conditions, experiences, etc. that are relevant to our bearer so that everyone involved shares a more embodied, communicative, and reflective lifeworld?
(think about how reading the titles on a friend’s bookshelves or studying the clutter on their desk tends to inform you of their current interests and to stimulate further conversation)
Can we make sure that our opportunities for the bearer action are well illustrated (as in a storybook) and foreshadowed (e.g., with a small and ghostly animation)?
(think of how helpful signage and pantomimes can be for encouraging shy folks to try new things or for navigating complex and unfamiliar spaces like international airports)
(Isn’t it curious how, in retrospect, how all of the experiences and half the questions I mentioned involve developing “immediate connections” in the sense illustrated by Victor’s demos and site?)