Picasso once noted that “when art critics get together they talk about Form and Structure and Meaning. When artists get together they talk about where you can buy cheap turpentine.” When you practice a craft you become skilled and knowledgeable in two areas: the stuff the craft produces, and the processes used to create it. And the second kind of expertise accumulates much faster. I call this the turpentine effect. Under normal circumstances, the turpentine effect only has minor consequences. At best, you become a more thoughtful practitioner of your craft, and at worst, you procrastinate a little, shopping for turpentine rather than painting. But there are trades where tool-making and tool-use involve exactly the same skills, which has interesting consequences. Programming, teaching, writing and mechanical engineering are all such trades.
Self-Limiting and Runaway Turpentine Effects
Any sufficiently abstract craft seems to cause some convergence of tool-making and tool-use. Painters aren’t normally also chemists, so that’s actually not a great example. But I don’t doubt that some of Picasso’s forgotten technician contemporaries, who had more ability to say things with art than things to say, set up shop as turpentine sellers, paint-makers or art teachers. But in most fields the turpentine effect is self-limiting. As customers, pilots can only offer so many user insights to airplane designers. To actually become airplane designers, they’d have to learn aerospace engineering. But in domains where tool-making involves few or no new skills, you can get runaway turpentine effects.
As Paul Graham famously noted, hackers and painters are very similar creatures. But unlike painting or aircraft, programming is a domain where tool-use skills can easily be turned into tool-making skills. So it is no surprise that programmers are particularly susceptible to the runaway turpentine effect. Joel Spolsky struck me very forcefully as the runaway-turpentine-effect type, when I read his Guerrilla Guide to Interviewing (a process designed to allow technically brilliant programmers to clone themselves). It is no surprise that his company produces tools (really good ones, I am told) for programmers, not software for regular people. And their hiring process is guaranteed to weed out (with rather extreme disrespect and prejudice) anyone who could get them to see problems that are experienced by non-programmers. 37 Signals is another such company (project management software). If you see a tool-making company, chances are it was founded entirely by engineers. And the consequences aren’t always as pretty as these two examples suggest.
Linus Torvalds’ most famous accomplishment was an act of thoroughly unoriginal cloning (Unix to Linux, via Minix). But among programmers, he seems to be most admired for his invention of git, a version control system whose subtle and original design elements only programmers can appreciate. This discussion with a somewhat postal Torvalds comment (I hope it is authentic) is a revealing look at a master-tool-maker mind. Curiously, Torvalds is going postal over a C vs. C++ point, and it is interesting to read his comment alongside this interview with another programmer’s programmer, Bjarne Stroustroup, the inventor of C++.
Eric Raymond codified, legitimized and spiritualized this path for programmers, in The Cathedral and the Bazaar, when he noted that most open-source projects begin with a programmer trying to scratch a very personal itch, not “other-user” needs. The open source world, as a result, has produced far more original products for programmers than for end users. Off the top of my head, I actually can’t think of a single great end-user open source product that is not a clone of a commercial original (aside: there is a crying need for open-source market research).
When I was a mechanical engineering undergraduate, my computer science peers created a department t-shirt that said “I’d rather write programs to write programs than write programs.” That about sums it up. In my home territory of mechanical engineering, some engineers naturally like to build machines that do useful things. Others build machine tools, machines that build machines, and wouldn’t have a raison d’etre without the first category.
Next door to programming and engineering, the turpentine effect can occur in science as well. Stephen Wolfram is my favorite example of this. His prodigal talents in physics and mathematics are probably going to be forgotten in 50 years, because he never did anything worthy of them (according to his peers, neither his early work, nor A New Kind of Science, is as paradigm-shattering as he personally believes). But the paradigm-shifting tool he built, Mathematica, is going to be in the history books much longer.
Teaching is a very basic creative skill that seems to emerge through runaway turpentine effects. I knew a professor at Cornell who had, outside his door, a sign that said, “Those who can, do. Those who can do better, teach.” Methinks the professor doth protest too much. There is a reason the actual cliche is “those who can, do. Those who can’t, teach.” But the insinuation that teachers are somehow not good enough to “do” is too facile. That is often, but not always, the case.
What happens is that all talented people engage in deliberate practice (a very conscious and concentrated form of self-aware learning) in acquiring a skill. But if you can’t find (or get interested in) things to do that are worthy of your skill, you turn to the skill itself as an object of attention, and become better at improving the skill rather than applying it. Great coaches were rarely great players in their time. John Wright, a mostly forgettable cricket player, had a phenomenal second innings in his life, as the coach who turned the Indian cricket team around.
But this effect of producing great teachers has a dark side as well, especially in new fields, where there are more learners than teachers. Thankfully, despite being tempted several times, I never started a “how to blog” blog. More generally, this is the writing/speaking/teaching/consulting (W/S/T/C) syndrome that hits people who go “free agent.” We talked about before in my review of One Person, Multiple Careers (check out the comments as well).
This relation to teaching (via self-learning) has actually been studied in psychology. In Overachievement, John Eliot talks about a ‘training’ mindset and a ‘performance’ mindset. The former involves meta-cognition and continuously monitoring your own performance. The latter involves an ability to shut off the meta-cognition and just get lost in ‘doing’. Great teachers were probably great learners. Great doers may be slower learners, but are great at shutting off the meta-cognition.
Causes and Consequences
I think the turpentine effect is caused by — and I am treading on dangerous territory here — the lack of a truly artistic eye in the domain defined by a given tool (so it is ironic that it was Picasso who came up with the line). Interesting art arises out of a combination of refined skills and a peculiar, highly original way of looking at the world through that skill. If you have the eye without the skills, you become an idiosyncratic eccentric who is never taken seriously. If you have the skills without the eye, you become susceptible to the turpentine effect. The artistic eye is innate and requires no real refinement. In fact, the more you learn, the more the eye is blinded. The adult artistic eye is largely a matter of protecting a childlike way of seeing, but coupling it to an adult way of processing what you see. And to turn it into value, you need a second coupling to a skill that translates your unique way of seeing into unique ways of creating.
There is a feedback loop here. Sometimes acquiring a skill can make you see things you didn’t see before. When you have a hammer in your hand, everything looks like a nail. On the other hand, if you can’t see nails, all you see is opportunities to make better hammers.
The artistic eye is also what you need to make design decisions that are not constrained by the tools. A complete absence of artistic instincts leads to an extreme lack of judgment. In a Seinfeld episode, Jerry gets massively frustrated with a skilled but thoroughly inartistic carpenter whom he has hired to remodel his kitchen. The carpenter entirely lacks judgment and keeps referring every minor decision to Jerry. Finally Jerry screams in frustration and tells him to do whatever, and just stop bothering him. The result: the carpenter produces an absolute nightmare of a kitchen. In Wonderboys, (a movie based on a Michael Chabon novel) the writer/professor character played by Michael Douglas tells his students that a good writer must make decisions. But he himself completely fails to do so, and his book turns into an unreadable, technically-perfect, 1000-page monster. No artistic decisions usually means doing everything rather than doing nothing. Artists mainly decide what not to do.
What about consequences?
The most obvious and important one is a negative consequence: creative self-indulgence. Nikki Hilton designs expensive handbags (which is still, admittedly, a more admirable way of spending a life than the one her sister models). There is a reason most product and service ideas in the world are created for and by rich or middle-class people for their own classes. The turpentine effect is far more prevalent than its utility requires. There is a limit to how many people can be absorbed in safe and socially-useful turpentine-effect activities like tool-building or teaching. Let loose where a content-focus, artistic eyes and judgment are needed, it leads to over-engineered monstrosities, products nobody wants or needs, and a massive waste of resources. Focusing on the problems of others, rather than your own (or of your own class), requires even more effort.
The positive effects are harder to see, but they are important. The turpentine effect is how isolated creatives can get together and form creative communities that help refine and evolve a discipline, sometimes over centuries, and take it much further than any individual can. Socially, this emerges as the aesthetic of classicism in any field of craft.
About Venkatesh Rao
Very nice post. Ceramics and glass are fields where the artists are frequently also chemists (via glaze formulation), btw.
One thing worth considering in terms of the turpentine effect comes from strengths-based psych – the learning talent. At least combined with input, it tends to almost force one into the recursive tools to build tools rabbit hole. Either that, or I’m rationalizing from my own experience. :)
G
Doh! I should have spotted the ‘Learning’ strength connection :) Great build.
I’d like to point out that you linked Linus’ message twice instead of linking to Bjarne interview.
Fixed, thanks!
“I actually can’t think of a single great end-user open source product that is not a clone of a commercial original ”
Although I agree with this impression I think it is a misleading one, especially in the context of your broader point here. If “end-user” rules out “programmer” for the sake of this discussion, then yes, most great open-source software is of a kind which tends to be of interest to programmers, even if it’s not specific to programming — for example software for system administration, text editing, and (originally) ftp clients — basically the software that only “computer nerds” would be interested in, though this group has become somewhat more fuzzily defined in the age of ubiquitous computers. But I think this is as much demand-driven as supply-driven, provided that you measure demand in terms of what free software pays. Programmers who create stuff for free (and of course there’s plenty of “open source” software whose development is mostly funded by companies one way or another, including most of “Linux”) do it for social reasons — they want recognition. It makes perfect sense that the people from whom they most want recognition are their peers, who will often be programmers. Apart from that, it’s much easier to make software for programmers, not because of what I know but because of what other programmers know. To turn a “useful program” into a “product” appealing to an end user requires a lot of extra work, of a kind that is unlikely to be creatively interesting to a programmer.
As you probably know there is tons of great (for example) scientific software which is written largely by and for scientists, for free. Most of this stuff wouldn’t come to mind as a “great open-source product” because the individual contributions are not grand in scale (there’s not much need to combine individual packages into a larger “product” since no-one is trying to market or sell them) and are “niche” products (there’s only a few broad categories of mass-market software, and that’s where big companies spend and earn a lot of money).
Anyway, very interesting post as usual and I agree that the turpentine effect is a serious trap for many programmers, myself included.
Good points about demand-driven. That’s why I speculated that we need open source market research as well, to get programmers market data as well as recognition for non-programmer-oriented innovation.
Mmmmmm…
Yet another incredibly insightful post.
In a few (rares) cases I went so much “uber-turpentine” that the meta-programming paid off on on single application, i.e. the returns were higher than using off-the-shelf vanilla solutions.
But this is very, very frustrating because it goes unnoticed.
Being worried about not hearing from a customer two years after delivery of a software product the sales manager called them just to be told: “No, everything works fine, we don’t need you anymore!”
Definitely clueless…
“The open source world, as a result, has produced far more original products for programmers than for end users.” The computer skills of end-users are determined by the software forced on them in work evironments. They use Windows at work, so they want a window system that is like Windows; they use Microsoft Word, so they want word-processor that is like Microsoft Word, &c.
Look at the first generation of netbooks, running Linux, often with unusual, highly original, well-thought-out UIs. Microsoft blew them away with—Windows XP. Open-source products for end users resemble commercial models because nobody wants originality.
(Apple of course in a exception, but that’s a question of social engineering rather than software engineering.)
“Artists mainly decide what not to do.” I don’t understand this statement. Where does the novelist have the chance to “do everything”? To get to the next scene—to get to the next sentence—he has to choose what his characters do to get there. The same goes for brushstrokes, notes, &c. Ending a novel is rarely an artistic decision anyway—it’s a cliché among novelists that novels are “never finished, only abandoned.”
“The adult artistic eye is largely a matter of protecting a childlike way of seeing, but coupling it to an adult way of processing what you see.” This is unusually sound.
Artists deciding what not to do usually means having the courage to get rid of too many pointless subplots etc.
The originality point seems circular. We clearly adopt a lot of original stuff. We just never seem to do so from open source. WordPress is a great exception… leading innovator in blog technology, and all open source.
Not to sound like a broken-record, but without open-source marketing work, this will not change. Commercial s/w is driven by market research in most cases (though not necessarily traditional focus group type… just somebody being paid to think about people other than themselves), which isn’t cheap. That’s why they produce new stuff end users adopt.
In the age of open information, market data is the most jealously guarded last bastion of high-value information.
WordPress only got really popular because Movable Type went from gratis but not libre to paid-for, even for one user.
You are hereby awarded the 2010 Moniker Maniac award for provocatively pertinent phraseology :-)
Hat doff to Greg B, fellow Input-Learner-*-*-*, for his warning on ‘recursive rabbithole descent’.
Now I think I know one possible reason why I don’t write stuff.
The so-called fifth stage of ‘conscious competence of unconscious competence’ or ‘reflective competence’ seems to imply a teaching ability though all great doers may not necessarily want to or be good at imparting their mastery of the skill to others.
This could be looked at as those interested merely in the products (and improving their own process of creating those products) and those interested in the process (looking at it as a product). The latter probably craves for variety at a higher level of abstraction and is not excited as much by a profit-oriented utility. Doers are able to fulfil their desire for variety by more products and their impact. Great teachers have to soon deal with the itch to improve the methods and train teachers and then… suffer?
In general the process teacher category makes less money, derives deeper satisfaction and can leave longer lasting impact on the world. Big Consulting firms (those that are still surviving) and management guru authors seem to be an exception some of who peddle newer frameworks and models (4X4 or otherwise) profitably, but come back with newer replacements by the time the businessworld widely adopts the old one!
From The Silence of the Lambs (1991)
Hannibal Lecter: First principles, Clarice. Simplicity. Read Marcus Aurelius. Of each particular thing ask: what is it in itself? What is its nature? What does he do, this man you seek?
Clarice Starling: He kills women…
Hannibal Lecter: No. That is incidental. What is the first and principal thing he does? What needs does he serve by killing ?
Clarice Starling: Anger, um, social acceptance, and, huh, sexual frustrations, sir…
Hannibal Lecter: No! He covets. That is his nature. And how do we begin to covet, Clarice? Do we seek out things to covet? Make an effort to answer now.
Clarice Starling: No. We just…
Hannibal Lecter: No. We begin by coveting what we see every day. Don’t you feel eyes moving over your body, Clarice? And don’t your eyes seek out the things you want?
And a poor bastardised version
Hannibal Lecter: First principles, Clarice. Simplicity. Read Marcus Aurelius. Of each particular thing ask: what is it in itself? What is its nature? What does he do, this man you seek?
Clarice Starling: He programs…
Hannibal Lecter: No. That is incidental. What is the first and principal thing he does? What needs does he serve by programming?
Clarice Starling: Anger, um, social acceptance, and, huh, sexual frustrations, sir…
Hannibal Lecter: No! He automates. That is his nature. And how do we begin to automate, Clarice? Do we seek out things to automate? Make an effort to answer now.
Clarice Starling: No. We just…
Hannibal Lecter: No. We begin by automating what we struggle at every day.
If the turpentine is what the programmer struggles at, thats what the programmer focuses on, else he focuses on his understanding of the biggest problem which requires automation. The turpentine is incidental.
Note: I had to struggle between automate, solve and simplify. Some may choose to use either of the other two words
~
Note to Venkat: I accidentally entered my email address incorrectly in the earlier comment. Entered it correctly in this comment. This comment could be deleted. Apologies.
I’m not entirely sure this is the turpentine effect, but I have a lot of trouble with As Difficult as Possible:
It’s the same process that leads people to think being a manager is more important than a worker, because are responsible for more being done. I like to think there is worth in doing a job well, even though you’re not operating at the highest level – we can’t all be managers. A similar thing happens with self-improvement gurus – you can either change the world by doing something, or by changing other people so they are more able to do what they want. This can lead to recursive self-improvement to truly change the world you need to have followers changing more people, but somewhere along the line the goal changes from making the world better to making everyone a self-improvement guru.
My favourite example of the turpentine effect is
Venkat, excellent article. Your conclusions calling for the inclusion of an artistic eye to bring good judgement (I see this in the context of finding such eyes, and including them in working teams engaged in creative/engineering endeavour) makes a lot of sense. The way I see it, decisions bottom out in emotion, not rationality, though they maybe informed by vast amounts of rational input pushing either way. Emotion is trained by pattern recognition that can extend beyond the power of domain specific tools. Also, see this paper on handicapping, which has some bearing on how people self-select into these teaching and doing roles – I wonder how. http://bit.ly/dtgjkC
Dhananjay: Nice pastiche on Silence of the Lambs. On related note, some cancers arise because we can’t stop scratching everyday familiar itches. Repetitive trauma :)
James: I have to admit, I have enough of the turpentine effect in me that I sympathize with the ‘as difficult as possible’ thesis. I think you are right in making the connection. The guy is defending succumbing to the turpentine effect. But at some point, developers have to realize that just because they enjoy developing a sledgehammer to swat a fly, they cannot sell flyswatters at sledgehammer prices.
Amit: I mostly agree, but I am wary of using loaded terms like “emotion” in association with differently loaded terms like “rationality” (the former term has strong biological/neouroanatomical correlates, the latter does not… the impedance mismatch makes arguments tricky). Thanks for the paper ref, will check it out.
Venkat
Yes, but this is the very motivation which makes good developers (Spolsky’s type), how do you solve the conundrum?
This is at the core of how to manage software developers, and it’s probably applicable to any field, really.
Hyper-competent developers revel at solving complex problems. If you give them a simple problem to solve, they will turn it into a complex problem to solve so that they can remain happy. Less competent developers are happy to do simple rote work.
Ideally, you’d want a mix of both. Spolsky speaks of looking for candidates that are “smart and get things done” – the “gets things done” obviously refers to the latter. Allocating the correct tasks to the correct people is good management.
Leadership on the other hand in this case is the art of saying “no”. The leader must be able to keep a big-picture view and recognize that there is no need for a sledgehammer to swat flies. It’s his job to make sure that people stay on track and communicate the vision of what is needed in a way that the team can relate to.
Obviously this is a highly idealized view of things.
Maybe a future ribbonfarmesque post is hidden in the following:
The three domains chosen as examples, viz., painting, speaking/writing and software, differ from each other in fundamental ways.
W/T/S/C is a horizontal that you have covered and is applicable to any craft, including W/T/S/C as a craft. Building software tools that help build software involves the same activity/craft but painter-teachers don’t paint lessons. Those interested in the process of creating software do not always build tools but may get into W/T/S/C and stay there.
Painting involves and is dependent upon things such as canvas, brush and turpentine, but writing does not depend on pen or paper quality nor is hardware a major factor in the software development craft.
JLD: Spolsky represents one type of good developer, not all. I guess there are those who miraculously are able to focus on others’ itches, and those who find a good complementary partner with a non-programmer-itch. And there are those with a Steve Jobs on the staff. But most times, I’d guess the problem DOESN’T get solved, leading to crappy s/w or unserved/underserved needs.
Ganesh: heh, I see a runaway turpentine effect possibility right there in your sequel suggestion. I think I am personally more curious about “how to see the work, not the tools” than thinking more about the nuances of the turpentine effect :)
Venkat
I thought you were going to talk about how programmers often spend more time finding the right “turpentine” i.e., tool such as text editor and learning how to use it, than actually tackling whatever programming challenge they need to be solving.
This happens in music too, as noted by Jerry Garcia. When you practice an instrument for a long time, your playing becomes overly technical and less musical as you attempt to utilize all your new skills. To paraphrase: “It takes a lot of effort and awareness to resist playing the guitar and instead just make music.”
That reminded me of a somewhat related but different musical quote:
“Your hands are like dogs, going to the same places they’ve been. You have to be careful when playing is no longer in the mind but in the fingers, going to happy places. You have to break them of their habits or you don’t explore; you only play what is confident and pleasing. I’m learning to break those habits by playing instruments I know absolutely nothing about, like a bassoon or a waterphone.”
— Tom Waits
Or IOW, you can deliberately subvert the turpentine effect by giving up the tools you are comfortable with.
I think the distinction between actually doing something and making tools to do something is a valuable one (and explained to me why the 37 signals guys had been annoying me so much). My main problem with this post is that I don’t get this distinction from the story about Picasso. In fact, I get completely the opposite point. To me, what Picasso is saying is that the painters are getting on and painting, they don’t have time to *talk* about meta-topics like form and structure they’re producing these things in their painting. And because they’re painting, they need turpentine, and they’re broke. Whenever I meet artists I’m always pleasantly surprised by how practical they are. They want to get on and actually do things in the world rather than talk about it, and to do that, you need access to cheap turps!
Also, your characterisation of what constitutes and artistic eye is amazingly restrictive, if not naive: “The adult artistic eye is largely a matter of protecting a childlike way of seeing, but coupling it to an adult way of processing what you see.”
Picasso occasionally mentioned that painting was like tapping into childlike ways of seeing but is that really what’s going on in cubism? What about Vermeer? In what possible way could you describe that as childlike? Defining what constitutes a good artist (and therefore good art) is hardly something that you should expect to manage in a drive-by blog post.
The very core of your writing whilst appearing reasonable in the beginning, did not really sit properly with me personally after some time. Somewhere within the sentences you were able to make me a believer but just for a while. I however have got a problem with your jumps in assumptions and you would do well to fill in those breaks. When you can accomplish that, I would undoubtedly end up being fascinated.
There is no such thing as “too technical” – just “not artistic enough.”
“‘When artists get together they talk about where you can buy cheap turpentine.’ When you practice a craft you become skilled and knowledgeable in two areas: the stuff the craft produces, and the processes used to create it. … Under normal circumstances, the turpentine effect only has minor consequences.”
No. Not only does the “turpentine” have its own economy, which means its ages and becomes obsolescent, but such cycles are a hidden driver behind innovation and product design. Rather than designing products for end users and actual needs, designers, including artists and programmers, follow opportunities provided by “turpentine” merchants. When the market for a specific type of “turpentine” ends, the merchants suddenly find a new kind of “turpentine” which promises much better results. In programming, we this as the lifecycle of toolkits such as Qt. And this is why the desktop so clearly prototyped in the 1980s has had to be re-invented again and again, never achieving completion and stability.
Picasso was really pointing out how you can tell real artists from fake ones. Real artists talk about their craft, not how their product is marketed.