Warrens, Plazas and the Edge of Legibility

Long-time reader and astute commenter, Xianhang (Hang) Zhang wrote a very interesting post a couple of weeks ago on his blog: The Evaporative Cooling Effect. It is part of a fascinating series he is doing on social software. The post explores a phenomenon that is very close to the status illegibility phenomenon I explored two weeks ago, and in fact draws inspiration from the same Groucho Marx/Lake Wobegon observations that I started with.

Evaporative cooling is basically the effect of the highest status people in a group leaving, lowering the average status of those left behind.

What I found fascinating though, was Hang’s suggestion for how to combat the effect (and thereby stabilize groups). In my post, I proposed that status illegibility helps create the stability. Hang brings in another dimension, which is illegibility in the group’s environment/context.

In particular, in social software (or physical environments for that matter), smarter-than-average early adopters often leave when the “unwashed masses” start to jump on the bandwagon, devaluing the social cachet. Hang proposes that one of the best ways to combat this is to build (or rather catalyze the evolution of) “warren” architectures instead of “plaza” architectures. Here are the pictures that pair of evocative terms produces in my head. You might imagine something else:

Warrens vs. Plazas

A warren is a social environment where no participant can see beyond their little corner of a larger maze. Warrens emerge through people personalizing and customizing their individual environments with some degree of emergent collaboration. A plaza is an environment where you can easily get to a global/big picture view of the whole thing. Plazas are created by central planners who believe they know what’s best for everyone. The terms are very evocative, and should remind you of the idea of legibility in physical environments that we talked about recently, in my post A Big Little Idea Called Legibility. In fact, it wouldn’t be a gross oversimplification to say that warrens and plazas differ primarily in their legibility. There are many subtleties of course.

The warren/plaza concept also sheds new light on one of my oldest posts, Harry Potter and the Cuaron Slam. In that (rather murky) post, I argued that not The Prisoner of Azkaban was the best Harry Potter book, and that Alfonso Cuaron’s movie version surpassed the book. I can now summarize that whole post very briefly with the warren/plaza concept. The entire Harry Potter series is of poor literary quality because most of the books are very plaza-like overly-legible books. There is none of the atmosphere of mystery you get with warren-like books (the Lord of the Rings for instance). The overarching central-planning map overwhelms narratives and character arcs. The 3rd book is the only warren-like book.

Two pictures from that old post get at a different aspect of warrens and plazas. I offered an analogy between the book/movie context and plot and robot path planning algorithms, which come in two basic varieties: bird’s-eye-view map navigation, and the more difficult worm’s eye view navigation where the robot only knows what it can see and remember from a ground view (the main algorithm for doing this is called SLAM, hence the bad pun in the title). The second picture illustrates how SLAM-style navigation works: you gradually build up a map of your environment by remembering what you see in your field of view as you navigate.

Again, the warren/plaza concept is useful.  Plaza navigation is easier, but less powerful, since it requires global information (which is only easily available with plazas). Warren navigation is much harder but much more powerful, since it does not require global information.  In warren navigation, learning is a necessary feature, since you cannot plot a shortest path  a priori. You need to explore and stumble and build up a map while groping towards the goal.

In the movie version, Cuaron’s camera work is literally warren like. Most of the shots are ‘follow the action’ ground-level shots. You feel like a mouse following the action rather than an eagle.

Hang offers Facebook and Quora as examples of highly “warren” like social sites. He also asserts that social sites that fail at a particular scale due to evaporative cooling are typically plazas.

The Edge of Legibility

I think there is a very fascinating line of thought here. I haven’t yet worked out the details, but here’s a potentially powerful conjecture that suggests where to go: social systems that thrive and grow are on the edge of legibility.

All kinds of legibility: status legibility, environmental legibility and probably a couple of other kinds. If they become too legible, they fail in the Seeing Like a State mode or through evaporative cooling.  If they become too illegible, they fail by ossifying, with highly ritualized and sacramental cultures, with a great deal of environmental irrationality and very few entries/exits.

The idea isn’t particularly original. In complex systems, an idea that was very popular in the early 90s (to the point of being a fad) was that rich, growing complex systems were on “the edge of chaos” (a state called self-organized criticality, SOC). Around the same time, in biology, Santa Fe theorist Stuart Kauffman proposed that biological systems self-organize in ways that keep them on the edge of criticality with respect to key chemical autocatalysis reactions (so for example, multicellular organisms evolved because the chemical soup inside a too-large single-celled creature would become supercritical and spiral out of control, while too-small cells would lack the critical concentrations of key molecules to get to autocatalysis at all).

This line of research isn’t dead; in fact it is steering towards the group dynamics stuff we are talking about. In the 2000s, a lot of complexity theorists got interested in what is known as the El Farol Bar Problem (and a related idea called the Minority Game). This line of work explores phenomena that are very strongly related to the Groucho Marx and Lake Wobegon effects. Surprisingly, the right approach to these problems seems to involve methods from statistical physics.

Resurrecting Self-Organized Criticality?

What’s new in what we are discussing here is the idea that complex systems that are also recursively self-aware in some ordinary sense (i.e. we are not talking “meaning of consciousness” here, but merely the ability to act on the basis of models of yourself, your group, and your environment; this can be programmed in simple AIs today) will differ from the old 90s style SOC systems in a critical way: their behavior will also be driven by the legibility of the situation to the various self-aware agents, and self-aware collectives of those agents, and so on recursively outward to the whole dimly self-aware global system.

So there may be an interesting model of social systems (both descriptive and prescriptive) lurking underneath this qualitative discussion: a combination of growth rates, “cell division” rates that keep a system on the edge of legibility that allow baby plazas to gradually morph into adult warrens, with the level of legibility for all actors itself driving the system’s evolution. Call it “Legibility-Modulated Ontogenic Evolution of Self-Organized Critical Systems.”

That sounds like the title of a paper I’d like to write. Pity I have no time any more to indulge in such things.

I am pulling ideas from a lot of different places here, so if you are interested in following this train of thought,  follow this Edge of Legibility trail starting with a review of the Self-Organized Criticality idea. Warning: this is a total geek-fest, and if you aren’t interested in the complexity theory side to many of the things I talk about on this blog, you should probably ignore this. But you should know that there is a radically different way of thinking about all the stuff I talk about in the Gervais Principle series and other posts on this blog, and that the other way is actually more natural for me. It just isn’t as much fun to read.

Another Personal Note

A couple of readers found the personal note I inserted into my last post interesting, so I’ll throw another one in here. Back before I went over to the dark side and actually did hands on technical work, the mathematical aspect of this kind of stuff was my main gig. During the early part of my grad school career, I was fascinated by the complexity theory work, which the mainstream of my field (systems and control theory) regarded as soft, and slightly disreputable interdisciplinary TV science (Santa Fe has a far bigger reputation in pop science than in mainstream academia).

Eventually the fascination led me to steer my PhD direction away from classical aerospace systems/control problems and towards complex systems. But I was uninspired by the main focus of the complexity theory world (things like sand piles, very simple agents like ants, traffic jams and so forth).  For both pragmatic reasons (you can’t manufacture a defensible PhD thesis in aerospace engineering by studying sand piles) and personal reasons, I studied things like formation flight and teaming algorithms for autonomous aircraft and spacecraft (very much the bandwagon research topic in the late 90s/early 00s). For my postdoc, between 2004 and 2006, I went even deeper into such stuff and did a lot of work on mental models and how individual decision makers cooperating or competing in a battlefield might act together given different views of the same world.

It was fascinating stuff, and I had a great deal of fun doing it. It was also stuff that was positioned perfectly to fall right through the cracks between the fields of AI, operations research, cognitive science and control theory. I even threw in bits and pieces of linguistics and the philosophy of language. And “falling through the cracks” is what happened. Interdisciplinary research sounds sexy to people outside the world of professional research, but it is an extremely risky thing to do, especially early in your career. This despite the fact that funding agencies clamor for interdisciplinary work. The publish or perish equation gets massively loaded on the perish side, and I perished.

The majority of researchers manage to dress up their work as “interdisciplinary” while still staying close to the lower-risk core of their disciplinary fields, while a minority take the risk of actually being interdisciplinary and do things so stunning that things work out despite the risk (these are the true pioneers who help reshape the academic landscape). Universities scramble to create positions for these pioneers, and they deserve it.

I wasn’t smart enough to pretend and dress up conservative work as interdiscplinary, and not the sort of genius who could actually make it work. I did good work, but I’ll be the first to admit it wasn’t powerful enough to reshape the landscape in ways that could create room for me.

My interests didn’t align with any established funding sources or publishing channels, so one fine day, after mulling over a pile of faculty position rejection letters and a few adjunct/second postdoc offers that would have had me in a holding pattern over the tenure landing track for a couple more years, I decided to call it quits. I nursed my academia sour grapes for a few weeks, got drunk, got over myself, and headed into industry. This was 2006.

It was probably the best decision of my life. When I look at my peers who are now in faculty positions I realize that I could never do what they do so well. They actually enjoy academic publishing and navigating the warrens of the funding and publishing ecosystems, rather than merely enduring the game as I did.  In a way, the free-form medium of blogging is my home territory, where I can basically do what I like without worrying about publish-or-perish pressures. I have no idea how I deluded myself for so long that I was actually cut out for the academic life.

I do regret that the blogging medium is not friendly to more mathematical exploration though (turning the sorts of ideas I explore in this blog into equivalent mathematical problems would take 100x more time and effort than just writing about them qualitatively).  Maybe someday I’ll be able to retire rich and early, scrape the rust off my math and programming skills, and get back in the game on my own terms.

Anyway, here endeth the geek fest. We’ll get back to your regularly scheduled programming next week.