Fwd: Slate Article: Decentralized Intelligence

[Owen Densmore]

Begin forwarded message:
Subject: Slate Article: Decentralized Intelligence

hey, wait a minute
Decentralized Intelligence
What Toyota can teach the 9/11 commission about intelligence gathering.
By Duncan Watts
Posted  Thursday, Aug. 5, 2004, at 11:30 AM PT

The most publicized recommendation of the 9/11 commission?and one
President Bush and Sen. John Kerry have raced to endorse?is that the
United States create a national director of intelligence. Centralizing
is an understandable response to the pre-9/11 intelligence fiasco. But
as organizational science and history show, it's also a misguided one.

When organizations fail, our first reaction is typically to fall into
"control mode": One person, or at most a small, coherent group of
people, should decide what the current goals of the organization are,
and everyone else should then efficiently and effectively execute those
goals. Intuitively, control mode sounds like nothing so much as common
sense. It fits perfectly with our deeply rooted notions of cause and
effect ("I order, you deliver"), so it feels good philosophically. It
also satisfies our desire to have someone made accountable for
everything that happens, so it feels good morally as well.

But when a failure is one of imagination, creativity, or
coordination?all major shortcomings of the various intelligence
branches in recent years?introducing additional control, whether by
tightening protocols or adding new layers of oversight, can serve only
to make the problem worse.

To understand this, we need to step outside government bureaucracy for
a moment and take a look at the world of industrial organization, which
has had some valuable experience in recovering from major mistakes.

In 1997, the Toyota group suffered what seemed like a catastrophic
failure in its production system when a key factory?the sole source of
a particular kind of valve essential to the braking systems of all
Toyota vehicles?burned to the ground overnight. Because of their
much-vaunted just-in-time inventory system, the company maintained only
three days of stock, while a new factory would take six months to
build. In the meantime Toyota's production of over 15,000 cars a day
would grind to an absolute halt. This was the kind of disaster with the
potential to wreck not just the company itself, but the entire Japanese
automotive industry. Clearly, then, Toyota, along with the more than
200 other companies that are members of the extended Toyota group, had
ample incentives to find a solution.

The big question was: How? How does one rapidly regenerate large
quantities of a complex component, in several different varieties,
without any specialized tools, gauges, and manufacturing lines (almost
all of which were lost), with barely any relevant experience (the
company that made them was highly specialized), with very little
direction from the original company (which was quickly overwhelmed),
and without compromising any of their other production tasks? Well,
actually it's not clear that one could do it at all, nor was it clear
at the time to any of the senior managers of the Toyota group. After
all, if this was the kind of disaster that their risk management
executives had considered, they would never have left themselves
vulnerable to it in the first place.

Nevertheless, they succeeded, but not in the way one might have
expected. Rather than relying on the guidance and coordination of an
inspired leader (control mode), the response was a bewildering display
of truly decentralized problem solving: More than 200 companies
reorganized themselves and each other to develop at least six entirely
different production processes, each using different tools, different
engineering approaches, and different organizational arrangements.
Virtually every aspect of the recovery effort had to be designed and
executed on the fly, with engineers and managers sharing their
successes and failures alike across departmental boundaries, and even
between firms that in normal times would be direct competitors.

Within three days, production of the critical valves was in full swing,
and within a week, production levels had regained their pre-disaster
levels. The kind of coordination this activity required had not been
consciously designed, nor could it have been developed in the
drastically short time frame required. The surprising fact was that it
was already there, lying dormant in the network of informal relations
that had been built up between the firms through years of cooperation
and information sharing over routine problem-solving tasks. No one
could have predicted precisely how this network would come in handy for
this particular problem, but they didn't need to?by giving individual
workers fast access to information and resources as they discovered
their need for them, the network did its job anyway.

Much the same kind of recovery happened in lower Manhattan in the days
after Sept. 11, 2001. With much of the World Trade Center in rubble and
several other nearby buildings closed indefinitely, nearly 100,000
workers had no place to go on Sept. 12. In addition to the
unprecedented human tragedy of lost friends and colleagues, dozens of
firms had to cope with the sudden disappearance of their offices along
with much of their hardware, data, and in some cases, critical members
of their leadership teams. Yet somehow they survived. Even more
dramatically, almost all of them were back in business within a week?an
achievement that even their own risk management executives viewed with
amazement.

Once again, the secret to their success was not so much that any
individual had anticipated the need to build up emergency
problem-solving capacities or was able to design and implement these
capacities in response to the particular disaster that struck. Rather,
the collective ability of firms and individuals alike to react quickly
and flexibly was a result of unintentional capabilities, based on
informal and often accidental networks that they had developed over
years of socializing together and collaborating on unrelated and
routine?even trivial?problems. When talking about their recovery
efforts, manager after manager referred, often with puzzlement and no
small sense of wonder, to the importance of informal relationships and
the personal knowledge and understanding that these relationships had
engendered.

Perhaps the most striking example of informal knowledge helping to
solve what would appear to be a purely technical problem occurred in a
particular company that lost all its personnel associated with
maintaining its data storage systems. The data itself had been
preserved in remote backup servers but could not be retrieved because
not one person who knew the passwords had survived. The solution to
this potentially devastating (and completely unforeseeable) combination
of circumstances was astonishing, not because it required any technical
wizardry or imposing leadership, but because it did not. To access the
database, a group of the remaining employees gathered together, and in
what must have been an unbearably wrenching session, recalled
everything they knew about their colleagues: the names of their
children; where they went on holidays; what foods they liked; even
their personal idiosyncrasies. And they managed to guess the passwords.
The knowledge of seemingly trivial factoids about a co-worker, gleaned
from company picnics or around the water cooler, is not the sort of
data one can feed into a risk-management algorithm, or even collate
into a database?in fact, it is so banal that no one would have thought
to record it, even if they could. Yet it turned out to be the most
critical component in that firm's stunning return to trading only three
days after the towers fell.

So, how does one make this kind of magic happen? Unfortunately, no one
is quite sure. Different organizations, from business firms to research
communities to the military, have tried to address their collective
problem-solving needs in a variety of ways. Some militaries make a
point of training their officers in joint-service academies and staff
colleges, both of which serve the purpose of building friendships and
professional relationships across otherwise frosty institutional
boundaries. Academic researchers, for their part, organize
interdisciplinary conferences and working groups that serve to
introduce disciplinary specialists who turn out to have complementary
knowledge or skill sets, but who otherwise would never have had
occasion to meet. And business firms from the automotive to high-tech
and finance industries deliberately cross-pollinate their intellectual
capital by fostering worker exchanges across divisions or even firms,
building problem-solving teams around tasks rather than departments,
emphasizing informal agreements and collaborations over formal
contracts, and organizing sophisticated team-building exercises for
geographically dispersed junior executives.

None of these methods, however, come with any guarantee of success, and
all of them come with their associated (and usually far more tangible)
costs. As a result, not everyone in either private or public sectors
explicitly encourages informal network building, nor is it always
effective when they do. Even from a purely theoretical perspective, no
one has figured out what is required to build organizations that are
not only efficient at performing routine, familiar tasks, but also good
at adapting to the exceptional and the unanticipated.

And even where we have apparently clear examples of success, it isn't
clear that what works, say, for a car manufacturer or finance firm is
going to work for the CIA and the FBI. In the world of intelligence,
the kind of information that, in less-clandestine businesses, tends to
flow along informal social networks must necessarily be subject to
greater constraints. So harnessing the power of social networks for
innovation, creativity, and rapid adaptation is a trickier business for
intelligence organizations than arguably for any other kind.

What should be clear, however, is that combining the many different
agencies involved in intelligence gathering and analysis at a single
point?that of the director of intelligence?is almost certain not to
succeed in delivering the kind of ambiguous yet essential functionality
that everyone wants. So, some other kind of connectivity, along with a
more creative approach, is required?one that incorporates not only the
sharing of information across agency boundaries (a recommendation of
the commission's that has received relatively little attention), but
active collaboration, joint training, and the development of long term
personal relationships between agencies as well. Creative intelligence
analysis has a lot in common with other kinds of problem-solving
activities: thinking outside the box, challenging deeply held
assumptions, and combining different, often seemingly unrelated, kinds
of expertise and knowledge. By understanding how innovative and
successful organizations have been able to solve large-scale, complex
problems, without anyone "at the top" having to micromanage the
process, the intelligence community could learn some valuable lessons
that might help it escape the mistakes of the past.

Duncan Watts is associate professor of sociology at Columbia University
and author of Six Degrees: The Science of a Connected Age.

How are things going?