Imagination vs. Creativity

I like to make a distinction between imagination and creativity that you may or may not agree with. Imagination is the ability to see known possibilities as being reachable from a situation. Creativity is the ability to manufacture new possibilities out of a situation. The two form a continuous spectrum of regimes in simple cases, but are disconnected in complex cases.

I’ve been playing with Legos in open-play mode lately to try and develop better intuitions about both. I’m limiting myself to a set of rectangular blocks on a base plate for now. I’m afraid so far the results are terrible.

I can follow fairly complex instructions to build models from a kit pretty easily, but faced with a pile of bricks and no plans or goals, I come up with dull designs to build, exhibiting very little imagination and near-zero creativity. Nothing in this collage gets even a passing grade on creativity. The most imaginative thing in the collage below is the model of a FinFET — a nano-scale feature of semiconductor chips — at the bottom left. I give it a D+ on imagination because it took a minor leap of imagination to recognize that Legos can be used to model things at scales besides the familiar range of scales covered by Lego models (typically coffee-cup scale to cityscape scale). I had to let go the “habit” of only seeing normal-scale-range design possibilities. But even that minor, barely passing-grade leap felt exciting. I plan to pull out my copy of Open Circuits and model more tiny electronics parts and features.

Just to give you a sense of how pedestrian these are, consider this dragon model with 6500-7000 parts by an expert Lego builder, Donny Chen (who also designed a playable grand piano that became an official kit).

This dragon, unlike the far simpler dragon kits sold by Lego itself, uses a 2×4 oval tile for scales and a set of other parts for creating the curving spine, all from mostly unrelated kits. It’s very hard to get Lego parts to do static curves, since the grammar has a strong orthonormal bias due to the mating technique. Chen managed to pull it off:

“The dragon I promised for the Year of the Dragon—maybe a bit bigger than a bunny, LOL! I kicked off this project about a year back, right after Brickvention2023, and I’ve been working at it on and off. Started building it about a month ago, and I’m pretty happy about how it turned out. No strings, no wires, not a drop of glue, not even a flexible tube, all solid connections. It stretches a solid 2 meters when fully spread out, around 1300 scales and made up of 6500-7000 pieces.” 

Chen’s design exhibits way more of both imagination and creativity than anything I’ve ever made up in any physical construction medium. He has clearly mastered Lego to the point where working forwards from the possibilities of a set of parts, and backwards from the constraints of a vision, are part of a near-unconscious fluency in the medium. But I can dimly see radically advanced versions of my own primitive pidgin Lego compositions in Chen’s process as described in the linked video. I’m at least at conscious incompetence in the Lego medium and language. I’m aware of my own decided lack of creativity and imagination. Chen is clearly at some advanced level of unconscious competence on the shuhari developmental curve that I’ll never come close to.

Keeping Lego in mind as a reference example, what can we say about imagination versus creativity? Here’s my theory.

Imagination is an aptitude based on analysis, and is a variety of reasoning forwards from a current state marked by freedom from habituated patterns of seeing. Creativity is an aptitude is based on synthesis, and is a variety of reasoning backwards from desired outcomes marked by closing of realizability gaps. To some extent, the two behaviors exist on the same continuous spectrum, and in most situations we alternate between forwards and backwards reasoning modes. But in complex situations, there is also a discontinuity between the two modes, which is the same as the general discontinuity and qualitative difference that separates analysis from synthesis.

Forward and backward are not symmetric. Synthesis, since it works backwards from a desired state, is strictly more expressive, since it can start from desired states that are not realizable or reachable from the current state using known techniques and patterns of behavior. It can also fail in more ways, since it might attempt impossibilities.

A leap — a creative leap — may be required to connect the forward and backward regimes. Sometimes this might just manifest as a textbook technical problem that is easy to solve once you pose it correctly. You could even outsource that to an appropriate sort of technician to actually execute. Craftsmanship and skill are useful for creativity up to the point where you can see the leap that is needed, but once seen, others can often do it. The most creative people in a medium are rarely the master technicians.

I like the definition of genius as “talent hits the target others can’t hit, genius hits the target others can’t see.” Creative genius likes in seeing what others don’t see. But once you’ve actually seen it, you might be able to simply point it out to others to hit. They might even be better at hitting it than you, once you point it out.

At other times creativity might manifest as an “invention gap,” as I’ve taken to calling it, or even a “discovery” gap — uncovering a new principle or phenomenon to harness in nature. A problem that nobody knows how to solve, or a behavior of nature that nobody has noticed, modeled, or figured out how to harness.

Both imagination and creativity in adult forms rest on a lot of pre-existing knowledge and learned patterns, which is what distinguishes the useful adult variety from the charming but generally useless variety exhibited naturally by children. In unimaginative and uncreative (UIUC) behaviors, knowledge is deployed in an unsurprising way, and patterns are applied via some sort of simple recognition-and-look-up mode.

One way to think of it is that UIUC behaviors are typically closed-loop. They rely on behaviors that have been executed before, up to parametric variation. You have lifted cups of coffee before, though you may be lifting a cup of coffee weighing exactly 102.8 grams for the first time. In computational terms, UIUC behaviors have known finite-time execution bounds. Any computations involved are embodied by relatively efficient low-level algorithms. They are protocolizable. They are solved behaviors. They are likely also to some degree efficient behaviors (they’ve been optimized somewhat according to some criteria) and thorough behaviors (they completely cover all concerns of potential interest within some scope). They can be made safely unconscious and reliable. You can expect them to operate within known bounds, and plan other things on the basis of such expectations, without uncertainty snowballing. In your brain, they can likely be stored in the lower layers, with little to no supervision from higher layers.

Case data can be feed to AI models and the parametrizable (by model weights) space of UIUC behaviors can be efficiently “colonized” in automated ways.

Behaviors that are either imaginative, creative, or both, are typically open-loop. They will rely on behaviors without known bounds on either execution time or performance quality. You cannot plan on the basis of expectations about solutions because uncertainty will snowball. That’s why non-UIUC behaviors cause so much trouble in organizational contexts. Organizations are all about bounded-uncertainty planning on the basis of bounded expectations about low-level behaviors.

A good example of a behavior that is typically imaginative but rarely creative is bug-fixing. As you are working through a tree of diagnostic possibilities, and inferring root causes from various clues, you have to imagine a lot of things, but you rarely have to solve a non-trivial problem or invent something along the way.

Usually, once you find a bug and verify a hypothesis about it, the fix is usually relatively trivial. My favorite personal example: I once spent hours failing to fix a bug in my code, then a partner on the project spent several more hours — and discovered that the bug was a missing apostrophe, the operator that takes a matrix complement in Matlab. Net ~5-6 hours to find the bug. 5 seconds to fix it.

The unpredictable and potentially unbounded execution time that makes bug-fixing an open-loop behavior is due to the imaginative difficulty of laying out and exploring an efficient troubleshooting tree. Do it badly and you could waste endless time going down unlikely pathways and possibly never even figure out the bug. Develop good instincts and a knowledge base of past cases, and you can zero in on the bug very quickly, almost all the time (but not 100% of the time — in that case it wouldn’t be bug-fixing but a UIUC closed-loop adaptive control behavior).

Imagination to some extent is relative to training data. What for you is a leap of imagination may be a straightforward inference for someone who has seen or experienced more cases. A sufficiently trained AI model may produce behaviors indistinguishable from highly imaginative human behaviors.

Creative behaviors require imagination, but also require more something more. Imagination is necessary but not sufficient for creativity.

Creative behaviors, I think, call for the equivalent of mutation or noise-injection into an evolutionary process. There is a non sequitur quality to creative leaps that strikes me as fundamentally non-analytical and serendipitous.

Humans typically tap into this quality with techniques like injecting literal randomness. Open a book at a random page and pick a random word with which to start your Great Novel. Or juxtaposing incongruous things: You’re stuck trying to make a dragon out of Lego parts — how about kit-bashing a jet-fighter kit and a flower-vase kit? Or to use the classic Boydian example from Destruction and Creation, if you mash-up a toy tank, a motorcycle, and skis, maybe you invent a snowmobile? (I’d like to see someone try this with appropriate Lego kits). A major source of injecting randomness is conflict. An adversary competing with you in some way is a source of mutations being injected into your behaviors that might potentially open up creative possibilities for you. Or kill you.

There is a connection here to strategy. The Clausewitzian notion of a coup d’oeil, or strike-of-the-eye, refers to the creative leap that lets the (necessary) analytical store of imaginative memory be transformed into a possibility newly recognized or constructed as realizable.

From what I can tell from the evolution of AIs, currently AIs can successfully fake imagination, but can’t yet do creativity. But there doesn’t seem to be any fundamental blocker to AI creativity. If you can inject randomness, incongruous juxtaposition, and adversaries deliberately into human behaviors, there’s no reason you can’t do that to AIs. Many of the computational processes of both GOFAI and modern deep-learning based AI already rhyme with phenomenology I’ve described above. There are forward and backward state-space models, there is injection of randomness, there are “adversarial” architectures. They just need to be leveled-up more. Making AIs more imaginative and creative is itself a problem that’s almost in the UIUC regime of software design now.

Can you improve your imagination and creativity? I think so, but it’s primarily a grind of time investment in a specific domain. To get more imaginative and creative at building Lego models, you have to spend a lot of hours building Lego models. Clever insights from other domains you’ve already achieved some mastery in may help a bit, but ultimately it’s about domain-specific grinding. There was a study somewhere that showed that Chess masters are no better at remembering random board positions than ordinary people, but have uncanny perfect memories for board states that could actually legally occur in a game. I’d like to think my more developed writing skills can help me get better at Lego faster, but so far, that has not proved to be the case. But maybe there’s a “refactored perception” or “constructions in magical thinking” approach to Legos that I’d be able to master faster than most people, and I just haven’t figured it out yet. Maybe porting creative-imagination techniques across domains is itself a challenge of creative imagination. It would be odd if such porting problems were themselves UIUC. We’d all become generalist geniuses in short order if that were the case, bootstrapping laterally from a single mastered domain to many.

Within a specific domain, how can you improve imagination and creativity? I think you have to alternate between imitation behaviors and open-play behaviors. In the case of Lego, you have to build existing kits to slowly build up a mental library of construction techniques, and internalize the grammar of various sorts of parts. And you have to spend time in open play to apply that learning to imagining and realizing new model possibilities from sets of parts, and working backwards from desired models by making the Lego language do things it has never done before. Which may involve going looking for specific parts in the Lego universe that are not in your kit yet. That behavior embodies the “creative leap” element perhaps. Yes, shopping for parts can be creative.

Lego is a fairly tightly circumscribed and low-complexity closed universe of parts and techniques, so the more you open up such domains, the harder it gets. So that’s a meta-behavior you have to overlay. For any complex domain (more complex, messy, and open than Lego), perhaps you have to start with some Lego-like subset and work to gradually broaden your command and use of the language. At least that’s what I suspect, with low-medium confidence. Or maybe a different strategy is needed for complex domains. I wouldn’t know because I don’t think I’ve truly mastered any such domain. I’m mostly a bumbling conscious-incompetent in every domain I touch, whether simple or complex.