"It formulates the CAP principle for the first time, yet the central theme is far broader: how to build services that degrade gradually under failure rather than failing altogether."
With that remark, CTO and System Design & Highload author Alexey Rybak opens his breakdown of Eric Brewer and Armando Fox's half-forgotten 1999 paper. In technical interviews, the CAP theorem has long been reduced to a dry "pick two out of three" rhyme, yet behind it lay a very different practical question: how do you design a high-load service so that when some servers go down, it keeps answering users instantly instead of collapsing over a single hung node?
When I reread the original paper, Harvest, Yield, and Scalable Tolerant Systems, presented at the 7th Workshop on Hot Topics in Operating Systems (HotOS VII) in Rio Rico, Arizona, in March 1999, I was struck by how much real engineering pain runs through the text. Rather than abstract math, it unpacks hardware economics, trading data completeness for availability, and an approach to splitting a monolith that is often forgotten today.
Behind the Scenes at Inktomi: Designing Services Before Microservices and the Cloud
In 1996, UC Berkeley professor Eric Brewer and his PhD student Paul Gauthier founded Inktomi, named after the trickster spider from Lakota legends. While corporate giants built systems around expensive fault-tolerant mainframes, Brewer and Gauthier assembled a search engine on a cluster of hundreds of inexpensive workstations.
Inktomi did not run its own consumer search site; instead, it licensed the technology to major portals—first HotBot, and later Yahoo and MSN. Its second flagship product was the caching proxy Traffic Server, which was donated to the Apache Software Foundation in 2009 and lives on as Apache Traffic Server inside large content delivery networks. At the peak of the dot-com boom in 2000, Inktomi's valuation reached $25 billion; after the market crash, Yahoo acquired the company in 2003 for $235 million.
In a cluster of hundreds of cheap machines serving millions of Yahoo and MSN queries around the clock, hardware failed daily: memory sticks threw errors, disks jammed, and rack switches dropped links. Trying to tie those nodes together with classic ACID transactions and two-phase commit would have brought the system down immediately. When a query waits for a synchronous reply from every server, one broken workstation stalls the entire cluster. You save money by skipping mainframes, yet a penny-ante node failure takes down search for the whole portal. Brewer and Fox needed a way to turn hardware failures from outages into routine statistical noise.
Real-World Metrics: Yield vs. Harvest
Availability is usually measured in "nines" of uptime, but a binary "up or down" scale ignores the quality of each response. Fox and Brewer replaced it with two metrics that let you weigh the cost and payoff of an architectural trade-off:
- Yield — the fraction of client requests successfully served without an error or timeout, out of all incoming requests.
- Harvest — the fraction of relevant data included in the response, relative to the complete target dataset in the system.
The difference is crystal clear in Inktomi's search cluster itself. Suppose the search index is sharded across a hundred servers, and suddenly 10% of the machines stop responding. If you insist on 100% Harvest, the central aggregator waits on the silent nodes, hits a timeout, and returns a 500 error. You stayed faithful to absolute data completeness, and drove Yield to 0%.
Inktomi took the opposite route and traded a slice of Harvest for 100% Yield. When 10% of the search nodes go quiet, the aggregator gathers results from the remaining 90% of the index and immediately returns an answer to Yahoo and MSN. If two out of a hundred nodes drop out, Yield stays at 100% while Harvest dips to 98%. The user on the other end never notices technicians swapping drives in the rack: the top links are all there, and temporarily losing a couple percent of rare pages pays for instant response times.
The Birth of the CAP Principle: Strong and Weak Formulations of the Reliability Trade-Off
That search trade-off led Fox and Brewer to realize a similar balance governs any distributed data store. The HotOS VII paper introduced the CAP principle—linking Consistency, Availability, and Partition tolerance—along with the CA, CP, and AP classification that later drew plenty of debate.
Three years before Seth Gilbert and Nancy Lynch proved Brewer's conjecture as a formal theorem over read-write registers in 2002, the authors themselves stated CAP in 1999 in two versions:
- Strong formulation: You cannot simultaneously achieve more than two of strong consistency, high availability, and partition tolerance.
- Weak formulation: The stronger the guarantees you demand on any two properties in that trio, the weaker your guarantees on the third.
The real engineering insight of the 1999 paper lives in that weak formulation. Theorists later flattened it into a binary toggle of ones and zeros, whereas Brewer was describing a continuous spectrum of trade-offs. Tightening requirements on data completeness and freshness (Harvest) inevitably costs you successful responses (Yield) during network partitions. Conversely, allowing slightly incomplete results buys you resilience when links between racks snap.
Lowering Harvest rescues search engines, recommendation feeds, news storefronts, and caching proxies. That trade-off does not apply to financial transactions, balance deductions, or ticket reservations: losing part of the data there is unacceptable by law and business logic, so 100% Harvest and strict consistency are mandatory.
BASE Architecture and Designing Services with Independent Failure Modes
How do you combine tasks that demand penny-accurate precision with storefronts that care most about instant response inside a single product? Back in 1997, the Berkeley research group proposed BASE (Basically Available, Soft state, Eventually consistent) as a counterweight to ACID transactions.
In the 1999 paper, that idea is expanded into a deliberate rejection of monolithic services—a full decade before the microservices boom. Fox and Brewer proposed splitting a system not by business domains or boxes-and-arrows diagrams, but by required reliability guarantees and independent failure modes:
- Isolated core of strict guarantees (ACID): Billing and payments are isolated into minimal components with rigid checks, where a temporary failure during a network split is acceptable to preserve 100% data accuracy.
- High-availability perimeter (BASE): Search, recommendations, and caches are built with soft intermediate state and eventual consistency, always answering the user and gracefully lowering Harvest under load spikes.
When subsystem boundaries follow degradation modes, losing 10% of the search cluster will not block payments, and a hiccup on one node will not bring down the entire portal. Later, amid debates over NoSQL, NewSQL, and PACELC, BASE was largely forgotten and narrowed down to Eventual Consistency. Yet Inktomi's lesson has not aged a bit, and Alexey Rybak sums it up best in his breakdown:
"Internet services do not need strict ACID guarantees everywhere; giving up those guarantees 'buys' availability and scalability."

