How Netflix queries a billion-edge graph in under 100 milliseconds

Netflix's team found that graph queries pull the system in two very different directions. A 'shallow and wide' query - something like 'which devices has this account streamed from in the last 30 days' - is only one hop away from the starting point, but that one hop can fan out into hundreds of edges to fetch, filter, and aggregate.

A 'deep and narrow' query is the opposite problem: something like 'across every profile on this account, show me the full Stranger Things viewing history' has to walk multiple hops in sequence - first find the profiles, then find what each profile watched. If each hop takes even 10 milliseconds of network time, four sequential hops alone burn 40ms before any real work happens.

Supporting both kinds of queries, at tens of thousands of queries per second, on a graph that never stops growing, is what shaped every design decision that follows.

The intuitive way to walk a graph is depth-first: pick a path, follow it all the way, backtrack, try the next one. Netflix's team explicitly avoided this, because in a distributed system, each hop means a network round trip, and depth-first would trace one profile's entire history before even starting on the next profile - wasting the chance to batch those lookups together.

Instead, the engine works breadth-first: fetch all profiles for an account in one round trip, then fetch the relevant edges for all of those profiles together in a second round trip, and so on. A query that chains two hops becomes two rounds of parallel work instead of one long sequential chain - which is a big part of how a multi-hop query stays under 100ms.

Most of the delay in answering a graph query isn't computation - it's waiting: waiting on the storage layer, waiting on a cache, waiting on an enrichment service. A traditional design would assign one thread per in-flight query, and with thousands of concurrent queries, most of those threads would just sit idle waiting for a network response to come back.

Netflix built the entire query engine around asynchronous composition instead. A small pool of 16 to 24 threads handles thousands of concurrent requests, because no single thread ever blocks waiting on I/O - while one storage call is in flight, that thread picks up other work and comes back for the result later. The team describes this as the foundational decision everything else in the system rests on.

Breadth-first traversal only helps if the work within each level actually runs in parallel. Netflix's team compares their design to a professional kitchen: separate stations for separate kinds of work - one pool of workers for fetching nodes, another for reading adjacency lists, another for enrichment calls - so a slowdown in one station doesn't stall the others.

On top of that, the system uses adaptive concurrency limiting: when things are healthy, it slowly raises how much work it allows in flight at once; when errors or timeouts start climbing, it backs off by a much larger step. That combination is what lets the engine run many lookups in parallel without accidentally overwhelming the storage layer underneath it.

A profile might have hundreds of viewing events on record, but a query usually only cares about a slice of that - the last 30 days, say. Rather than pulling everything back and trimming it afterward, Netflix streams adjacency data in small batches and applies filters as each batch arrives, stopping as soon as it has enough to satisfy the query.

The team also built a layered system of defaults and overrides - global defaults, per-query overrides, and per-edge-type limits - so different teams can ask for different lookback windows or edge limits without needing a code change every time. They describe this as having eliminated an entire category of one-off feature requests.

Not everything in the graph changes at the same speed. An account's plan type or a show's metadata barely changes; who-watched-what-and-when changes constantly. Netflix caches the slow-changing, frequently-accessed data - accounts, profiles, content - in a distributed cache (EVCache), which brought hit rates up to 70-80% on those lookups.

The subtler decision was knowing what not to cache: a node that's about to fall outside the graph's retention window isn't worth caching even briefly, so the system weighs a node's last-activity time against the retention period before deciding whether caching it is worthwhile at all.