It is always interesting to recognize a simple pattern in your own thinking. Recently, I was wondering why I am so attracted to thinking about the margins of civilization, ranging from life on the ocean (for example, my review of The Outlaw Sea) to garbage, graffiti, extreme poverty and marginal lifestyles that I would never want to live myself, like being in a motorcycle gang. Lately, for instance, I have gotten insatiably curious about the various ways one can be non-mainstream. In response to a question I asked on Quora about words that mean “non mainstream,” I got a bunch of interesting responses, which I turned into this Wordle graphic (click image for bigger view)
Then it struck me: even in my qualitative thinking, I merely follow the basic principles of mathematical modeling, my primary hands-on techie skill. This interest of mine in “non mainstream” is more than a romantic attraction to dramatic things far from everyday life. My broader, more clinical interest is simply a case of instinctively paying attention to what are known as “boundary conditions” in mathematical modeling.
Mathematical Thought
To build mathematical models, you start by observing and brain-dumping everything you know about the problem, including key unknowns, onto paper. This brain-dump is basically an unstructured take on what’s going on. There’s a big word for it: phenomenology. When I do a phenomenology-dumping brainstorm, I use a mix of qualitative notes, quotes, questions, little pictures, mind maps, fragments of equations, fragments of pseudo-code, made-up graphs, and so forth.
You then sort out three types of model building blocks in the phenomenology: dynamics, constraints and boundary conditions (technically all three are varieties of constraints, but never mind that).
Dynamics refers to how things change, and the laws govern those changes. Dynamics are front and center in mathematical thought. Insights come relatively easily when you are thinking about dynamics, and sudden changes in dynamics are usually very visible. Dynamics is about things like the swinging behavior of pendulums.
Constraints are a little harder. It takes some practice and technical peripheral vision to learn to work elegantly with constraints. When constraints are created, destroyed, loosened or tightened, the changes are usually harder to notice, and the effects are often delayed or obscured. If I were to suddenly pinch the middle of the string of a swinging string-and-weight pendulum, it would start oscillating faster. But if you are paying attention only to the swinging dynamics, you may not notice that the actual noteworthy event is the introduction of a new constraint. You might start thinking, “there must be a new force that is pushing things along faster” and go hunting for that mysterious force.
This is a trivial example, but in more complex cases, you can waste a lot of time thinking unproductively about dynamics (even building whole separate dynamic models) when you should just be watching for changes in the pattern of constraints.
Inexperienced modelers are often bored by constraints because they are usually painful and dull to deal with. Unlike dynamics, which dance around in exciting ways, constraints just sit there, usually messing up the dancing. Constraints involve and tedious-to-model facts like “if the pendulum swings too widely, it will bounce off that wall.” Constraints are ugly when you first start dealing with them, but you learn to appreciate their beauty as you build more complex models.
Boundary conditions though, are the hardest of all. Most of the raw, primitive, numerical data in a mathematical modeling problem lives in the description of boundary conditions. The initial kick you might give a pendulum is an example. The fact that the rim of a vibrating drum skin cannot move is a boundary condition. When boundary conditions change, the effects can be extremely weird, and hard to sort out, if you aren’t looking at the right boundaries.
The effects can also be very beautiful. I used to play the Tabla, and once you get past the basics, advanced skills involve manipulating the boundary conditions of the two drums. That’s where much of the beauty of Tabla drumming comes from. Beginners play in dull, metronomic ways. Virtuosos create their dizzy effects by messing with the boundary conditions.
In mathematical modeling, if you want to cheat and get to an illusion of understanding, you do so most often by simplifying the boundary conditions. A circular drum is easy to analyze; a drum with a rim shaped like lake Erie is a special kind of torture that takes computer modeling to analyze.
A little tangential kick to a pendulum, which makes it swing mildly in a plane, is a simple physics homework problem. An off-tangent kick that causes the pendulum bob to jump up, making the string slacken, before bungeeing to tautness again, and starting to swing in an unpleasant conic, is an unholy mess to analyze.
But boundary conditions are where actual (as opposed to textbook) behaviors are born. And the more complex the boundary of a system, the less insight you can get out of a dynamics-and-constraints model that simplifies the boundary too much. Often, if you simplify boundary conditions too much, the behaviors that got you interested in the first place will vanish.
Dynamics, Constraints and Boundaries in Qualitative Thinking
Without realizing it, many smart people without mathematical training also gravitate towards thinking in terms of these three basic building blocks of models. In fact, it is probably likely that the non-mathematical approach is the older one, with the mathematical kind being a codified and derivative kind of thinking.
Historians are a great example. The best historians tend to have an intuitive grasp of this approach to building models using these three building blocks. Here is how you can sort these three kinds of pieces out in your own thinking. It involves asking a set of questions when you begin to think about a complicated problem.
- What are the patterns of change here? What happens when I do various things? What’s the simplest explanation here? (dynamics)
- What can I not change, where are the limits? What can break if things get extreme? (constraints)
- What are the raw numbers and facts that I need to actually do some detective work to get at, and cannot simply infer from what I already know? (boundary conditions).
Besides historians, trend analysts and fashionistas also seem to think this way. Notice something? Most of the action is in the third question. That’s why historians spend so much time organizing their facts and numbers.
This is also why mathematicians are disappointed when they look at the dynamics and constraints in models built by historians. Toynbee’s monumental work seems, to a dynamics-focused mathematical thinker, much ado about an approximate 2nd order under-damped oscillator (the cycle of Golden and Dark ages typical in history). Hegel’s historicism and “End of History” model appears to be a dull observation about an asymptotic state.
How the World Works
In a way, the big problem that interests me, which I try to think about through this blog, is simply “how does the world work?”
At this kind of scale, the hardest part of building good models is actually in wrestling with the enormous amount of “boundary conditions” data. That’s where you either get up off the armchair, or turn to Google or Amazon. Thinking about boundary conditions — organizing the facts and numbers in elegant ways — becomes an art form in its own right, and you have to work with stories, metaphors and various other crutches to get at the right set of raw data to inform your problem. Only after you’ve done that do dynamics and constraints get both tractable and interesting.
Abstractions and generalizations, if they can be built at all, live in the middle. Stories live on the periphery.
This is part of the reason I don’t like traditional mathematical models at “how the world works” scale, like System Dynamics. They ignore or oversimplify what I think is the main raw material of interest: boundary conditions. A theory of unemployment, slum growth and housing development cycles in big cities that ignores distinctions among vandalism, beggary and back-alley crime is, in my opinion, not a theory worth much. If you could explain elegantly why some cities in decline turn to crime, while others turn to vandalism or beggary, then you’d have interesting, high-leverage insights to work with.
It’s not surprising therefore, that one of the most seductive ideas in abstract thinking about history, the deceptively simple “center periphery” idea (basically, the idea that change and new historical trends emerge on the peripheries and in the interstices of “centers”) is extremely hard to analyze mathematically, since it involves a weird switcheroo between boundary conditions and center conditions. Some day, I’ll blog about center-periphery stuff. I have a huge, unprocessed phenomenology brain-dump on the subject somewhere.
So in a way, thinking about things like the words in the graphic is my way of wrapping my mind around the boundary conditions of the problem, “how does the world work?” If I just made up a theory of the mainstream world based on mainstream dynamics, it would be very impoverished. It would offer an illusion of insight and zero predictive power. A theory of the middle that completely breaks down at the boundaries and doesn’t explain the most interesting stories around us, is deeply unsatisfying.
I have proof that this approach is useful. Some of my most popular posts have come out of boundary conditions thinking. The Gervais Principle series was initially inspired by the question, “how is Office funny different from Dilbert funny?” That led me to thinking about marginal slackers inside organizations, who always live on the brink of being laid off. My post from last week, The Gollum Effect, came from pondering extreme couponers and hoarders at the edge of the mainstream.
So I operate by the vague heuristic that if I pay attention to things on the edge of the mainstream, ranging from motorcycle gangs to extreme couponers and hoarders, perhaps I can make more credible progress on big and difficult problems.
Or at least, that’s the leap of faith I make in most of my thinking.
About Venkatesh Rao
Models and theories inherently seek to explain and predict the normal data set, the periphery periodically overturns the dominant theory to replace it with a new one and the cycle continues. The periphery houses the naughty, the mysterious, the uncomfortable and the unimportant. In some cases they tend to prefer to be on the edge, in other cases but they get pushed and contained there.
Creativity gurus, of course, highlight the importance of borders. Good scientists also recognize it, like VS Ramachandran and his role models.
Popularity of a fringe activity can suddenly redefine the boundaries such that erstwhile edgers find themselves part of the new mainstream (with dismay, mostly). Coolness quotient obviously dips as you move away from the edge. This means the periphery not only houses the unimportant but also the elite.
The absolute center has characteristics more similar to the periphery than to the general mainstream central region.
Generally, constraints introduce new boundaries or accentuate/attentuate existing ones. Contrary to popular notions, creativity loves constraints.
Great post.
Understanding the boundary conditions is a great way of understanding a problem because they a) let you know of the extents of the problem and b) help you to build an accurate mental model of the problem. When you have multiple variables interacting, it’s not always obvious the effect that changing one will have. Pushing one to it’s minimum or maximum gives a much clearer view of how that variable affects the overall problem.
I’m not super familiar with them, but it seems like a lot of problem-solving methodologies like Polya’s “How to Solve it” involve understanding/manipulating the boundary conditions.
I think, in programmer-speak, you’re talking about Edge Cases and Corner Cases (http://en.wikipedia.org/wiki/Corner_case)
Kinda, but not entirely. Edge/corner cases are subtly different from boundary conditions. Generally core dynamics segue smoothly into more trivial boundary dynamics in calculus. So a drum skin vibrates with decreasing amplitude towards the edge, where it is still (zero amplitude).
Corner conditions and edge cases are like division-by-zero exceptions in equations, or instantaneous events. That can happen in continuous-math problems both off and on the boundary. If you suddenly hit a swinging pendulum with an impulse force, you’ll do an instantaneous discontinuous switch in the equations. That impulse is a boundary condition between two regimes and requires “corner case” type thinking.
More generally, I am thinking of rich problems where there is very little interior and lots of boundary, and lots of corners in the boundary. Like a fractal boundary. More corner case code than regular. An example I like is the Quincunx (basically a simple pinball machine type device).
Here a ball drops through a pattern of nails. But the pattern of nails so dominates the Newtonian dynamics of the ball (through the uncertain path a ball can take when it hits a dead center… it is then randomness that determines whether it will continue right or left) that the best model actually ignores the dynamics altogether, and simply counts the instances of different trajectories the ball can take.
For a triangular pattern of nails, you get a normal distribution (an effect that’s often used as an illustration of the central limit theorem).
But think of the slippery slope. Suppose you started with no nails. You’d use Newton.
One nail?
3 Newtonian regimes, 1 random event?
Two nails?… Three? You get a combinatorial explosion in the number of cases.
At that point, your modeling focus should shift to where the richness is. So you start playing around with large numbers of nails, throw Newton out, and try to gain insight by trying different patterns of nails. At some point you think of counting the paths and going to statistical mechanics instead of Newton.
But to me the interesting part is the transition region where you don’t quite know what framework to use. The study of history is kinda like that. If you focus exclusively on the dynamics or some model of the boundary condition set (think of historical events as “nails” and different counterfactuals as “what if this nail wasn’t there?” type thinking), you get a simplistic and impoverished model.
This is hard because we like to simplify models down to the scope of one modeling paradigm. In this case, AI people would naturally pick a regime like the Quincunx that’s friendly to discrete math, while a control theory guy like me would naturally pick a regime where the Newtonian dynamics get interesting (one nail, but make the ball spin and nutate or something, and throw in aerodynamics).
Solving problems that necessarily involve both requires serious striving for elegance, and an ability to tastefully blend different modeling paradigms. Unfortunately most people either simplify down to one paradigm, or clumsily cobble two together to produce heavy 1+1=3 theories instead of elegant 1+1=1.5 theories.
It’s not often that a blogger makes me feel stupid. Kudos, sir.
You have a very interesting perspective on quantifying the qualitative with limits, controls, models, and patterns/sequences of correlation. I can stand to learn a lot from you.
Venkat,
The way I see, you’ve used this post to put into so many words, your “thought process”, i.e., what makes you (Venkat) tick. Indeed if there’s one thing that’s sacrosanct, it’s: There’s no “right or wrong” way about the human thought processes. And what is it that drives thoughts? Our personalities that we bring to this world. It is the Myers-Briggs Type Indicator (MBTI):
So, if the above approach in thinking makes you productive, “in-the-flow” and “slightly evil” (Ha! ;-)) then more power to you. Now I can appreciate the behind-the-scene-mechanics (or dynamics, whatever is appropriate) behind your popular posts, but only you will be able to truly ‘understand’ the ‘feeling’, since it is your march, and you march to a different drummer’s tune than I do. I couldn’t help notice your discussion with @qwe where he tried to place your thoughts into his map of understanding/cognition; turns out, his understanding was not the same as yours.
So, with that, I will leave you with two “thoughts” (pun intended):
——————————————————————————————-
I) No matter how well-exercised your boundary condition thinking might be ;-), you will do well to realise it is all but an aspect of the whole. Remember this old tale:
To drive home the point, a small quotation from Wade Davis’ speech on the differing natures of realities:
Subjective realities, brother. But we can see the consequences of this action all around us, in the pillaging of the world. Not sure if you are following it there in the USA, but that quote is particularly poignant for me, due to the exactly duplicate ongoing “Vedanta” tussle here in Orissa.
II) And to end with, I add another quote from your “favourite” (teasing you there ;-)) detective about looking at your own well-honed methods and their results with humility (this is not related to my earlier comment about Office Vs. Dilbert):
Ciao for now.
Surio.
—
P.S:- Loved that Tabla observation. I hope you are still playing it?
P.P.S:- Liked the “clarity of thought” that went into the “thought process”. (Ha! :-D)
I really enjoyed the follow-up post on Gollum effect on Quora by you. Even more punchy than the original, if I may say so ;-)
I can’t be bothered signing up, so wanted to say here, you’d have got 2 upvotes from me if I was there ;-)
It’s somewhat strange that two of the most obvious examples of peripheral figures which form the center are missing in the Outlaw map: the criminal and the terrorist. They are among the most aggressive vectors which challenge or directly attack the center, while the center responds in the way of creating a police state which yields yet another peripheral figure, which is also missing: the dissident.
@Kay,
It is always fair to keep in mind: one man’s terrorist is another man’s freedom fighter.
So I think that the omission is justified, even if the omission was a serendipitous one.
Surio.
The relationship between your modeler’s dynamics-constraints-boundaries mode and the dialectical thesis-antithesis-synthesis mode is working well in my mind; in problem space, the relationship is particularly interesting in the context of differing scales or over time.
I don’t know whether that makes sense, but you’ve sent me down a productive path and I appreciate it!
Venkat wrote:
> “how is Office funny different from Dilbert funny?”
It was a throw-away comment, but it had me thinking long and hard. I’ve blogged my views here in case anyone is interested.
I enjoy this but from an admittedly egotistical point of view: you’ve described my thought process, too. I’m a storyteller, mostly of fiction, but I loved math growing up and it certainly set the tone for how I approach things. In fact, this is exactly how I come up with a story.
First, I do the brain-dump. I write down every little bit of information I have in my notebook. I’ll even sketch something if it’s important and easier than writing it. Once I get that all down, I break it down into usable pieces and I rearrange them into something coherent. The building blocks actually correlate to the three types you suggest above.
1. Dynamics. These are the basic cause-and-effect rules of the world in which the story takes place. Usually, it’s character stuff: relationships (Bob and Jill are married, so they probably kiss when they get home), personality (Bob has a temper, so if you punch him, he’ll punch you back but it takes a lot to enrage Jill). If you’re writing sci-fi/fantasy it could encompass physics (Bob and Jill are superheroes so they hit harder).
2. Constraints. These are usually technical stuff you have to adhere to. It includes stuff like format (script, prose, serial, single story) and length. If you’re writing for someone else, they may also story points you can’t alter (like say you’re told Bob and Jill can’t break up).
3. Boundaries. This is where the drama comes from! You push things to the limit and see what happens. For instance: Jill punches Bob.
What could have made Jill act in such a violent way? Is Bob going to react as he normally would and punch his wife? If they’re two superheroes fighting in their apartment, is that going to cause serious property damage? And, since they have to stay together, how do you resolve this situation in a way that doesn’t lead to a them breaking up? If it’s got to be a 100 word prose story, what details do I choose to include and how do I structure it?
As usual, thanks for giving me something to think about!
That last item is very interesting, where you create dramatic tension by having people act out of character. It gives you the basic character-level tension to drive the story and make the character grow.
Your Jill punch would create a great storyline if Bob DIDN’T do something obviously extreme enough to explain it, so you now have the Jill mystery to solve.
I’d say an average story would come out of hidden external situational factors that the author gradually reveals (eg. Bob actually DID do something extreme that we don’t know about). But a great story would come out of an inner tension within Jill herself (for example, “her poise/soft temper is actually about a subconscious sense of elitism from having born rich and upper class, but something happened to make her recognize that she’s now lower class, and she can’t be soft-tempered any more.”)
This fits within the mathematical notion of boundary conditions I suspect, but I am not quite sure how.
Your technique there reminds me of some writing exercises in Jennifer von Bergen’s “Archetypes for Writers.”
That’s similar to your point regarding the “illusion of understanding.” If you simply the boundary condition (Bob did something stupid), you still have a true-to-life story but one that’s very simple to understanding. Like the rim of a round drum. If you complicated it (Jill had internal conflict that lead to the punch), you create something much richer, with depth and complexity. Like the surface of lake Eerie.
However, the big difference is that in the case of math there is a right answer that you are searching for. In the case of fiction (or most art) it’s subjective. There is no right solution to why Jill punched Bob, since it’s made up to begin with. There are just a bunch of different ones that the storyteller chooses from and builds the story around. Some maybe more successful at eliciting a good reaction from the audience than others but even that is subjective. I know I prefer the story you suggest but some would rather have the simple one.
Thanks again. I’ll be thinking about this for a while.
It would be great to see a brain-dump associated with a blog post.
The first thing I want to say is “well, yes”. The second is that I desperately want to see your core/periphery brain dump to see if it would fit with mine. I’m leaning towards a belief that it’s a fundamental flaw in most models of sociopolitical behavior, and generates boom / bust cycles. Distributed network models seem far more descriptive and predictive.
But maybe that’s just me.