Discovery-Heavy Projects

I don’t like processing huge fire-hoses of complex, poorly structured and somewhat arbitrary information, but I am good enough at it, mainly because I’ve had a lot of practice. I call these discovery-heavy projects. They require three cognitive skills:

1. Triage Skill: Simple information, such as a shoe-box of family photographs, can usually be categorized rapidly within a simple taxonomic system. Complex information, such as a pile of paper documents that are part of a legal discovery process, tends to require much more thought to codify into usable form for processing. You have to triage the simple and complex, and resist the temptation to find a pigeonhole for everything. Some critical stuff will stay sui generis. 
2. High-touch processing: Poorly structured implies low automation potential. This means you will need to examine every bit of information that comes in via the firehose.
3. Data slumming: Arbitrariness of information means you cannot infer it from other information you’ve already processed. For example, the GDP of Great Britain in 1973 is something you just have to look up.

These behaviors can be very exhausting to those who are not naturally skilled at them, or energized by them. So what can you do?

The key, I have found, is to interleave these behaviors with ones that you do find energizing and are skilled at. For me, that’s often writing. If I cannot interleave writing with discovery, I lose motivation really fast. So I cannot stand waterfall plans that have a long, upfront discovery process with no opportunity to produce output.

But the other piece of it is that firehoses don’t just shut off these days. In pre-Internet days, you could go get everything you wanted for a research project from the library for instance, process it via a “literature survey” and then get on with the processing. Today, typical projects involve a continuous inflow of information that must be integrated into execution. So bringing an Inbox Zero mentality to the work means you’ll never get to doing anything more than discovery processing. 

The trick, I have found, is to manage the tradeoff between the discovery backlog (size of inbox) and output rate. Don’t let output rate fall to zero, and don’t let the inbox bloat to extreme levels that might require a declaration of bankruptcy due to “intention debt” (runaway levels of commitment to process inputs). Work throughput is key. If the work is highly coupled, and leads to a single finished output artifact, such as a book or completed building, this is complex. If the work is a lot of discrete, disconnected or loosely connected pieces of output, such as meals being served at a restaurant or the output of a manufacturing process, this is somewhat easier: the key there is to run “lean” (minimize Work-In-Progress or WIP).

Increasingly, this is the type of work humans have to become good at managing, if they don’t want to be replaced by machines.