How Uber cut search latency in half by adding gRPC to OpenSearch

Most of Uber's internal infrastructure already ran on gRPC and Protobuf - strongly typed contracts, efficient binary serialization, and transports built for streaming. OpenSearch, which Uber had standardized on for search and retrieval, only exposed REST and JSON APIs.

That mismatch meant every request between Uber's gRPC-native services and OpenSearch needed a translation layer converting Protobuf to JSON on the way in, and JSON back to Protobuf on the way out. Translation layers are the kind of thing that look like a minor implementation detail until you measure what they actually cost - added latency, added complexity, and an additional piece of infrastructure that can break.

Uber's search team could have kept patching the translation layer, or maintained a long-term fork of OpenSearch. Instead, they built native gRPC support directly into OpenSearch itself, designed to run alongside the existing REST transport rather than replace it - both protocols run on different ports, sharing the same underlying node-to-node logic, so only the outermost client-server layer actually differs.

That design choice mattered for adoption: teams already using REST didn't have to migrate all at once. Search and Bulk (ingestion) were prioritized first, since those were the most latency-sensitive APIs both for Uber and for OpenSearch users generally.

Adding a second protocol only works long-term if both protocols stay in sync as the API evolves - otherwise gRPC quietly drifts out of date every time someone changes the REST API. Uber's team built an automated conversion pipeline with three stages: preprocessing, core conversion, and postprocessing.

Preprocessing resolves the fact that REST and Protobuf don't think about APIs the same way - REST leans on paths, query parameters, and status codes, while Protobuf needs everything spelled out as explicit, strongly typed messages. Core conversion turns the cleaned-up API spec into actual Protobuf artifacts (extending an existing open-source tool, OpenAPI Generator, since it didn't already support this). Postprocessing is the safety net: Protobuf can't tolerate something as small as a field being renumbered without breaking every existing client, so this stage runs compatibility checks against previous versions before anything ships.

One concrete place this showed up was Uber's Search Gateway, the service that proxies every customer search and ingest request before it reaches an OpenSearch cluster, adding security, observability, and rate-limiting along the way. Before native gRPC support existed, the gateway ran an in-house adaptor that transpiled every Protobuf request into JSON, sent it to OpenSearch over REST, then transpiled the JSON response back into Protobuf - extra work on every single request.

Once OpenSearch could speak gRPC natively, that adaptor became unnecessary. The gateway could pass a client's Protobuf request straight through to OpenSearch's own gRPC endpoint, removing an entire translation hop instead of just optimizing it.

The performance difference wasn't marginal. On M3, Uber's in-house metrics system, switching to gRPC cut p99 write latency by roughly 60% (from 34.1ms down to 13.6ms) and p50 latency by about 34%. The M3 Indexer's maximum indexing delay - a metric that matters most during failovers, when the system is already under stress - dropped by 20-35% at higher request rates.

The biggest gains showed up in vector search, which makes sense once you know why: a large vector is expensive to represent as JSON text, but Protobuf can pack a repeated float array far more compactly. Uber Eats' delivery shopping-list recommendations saw p50 search latency drop about 53% (83ms to 38ms) and p95 drop about 43% (114ms to 64ms) after the switch. Batch ingestion jobs that indexed data using Apache Spark saw job runtimes drop 20-35% simply by switching their Bulk calls from REST to gRPC.

Uber's team states their own takeaway plainly: how a system represents its API - REST versus gRPC, JSON versus Protobuf - isn't just a stylistic preference between two ways of writing the same thing. At scale, it directly shapes performance ceilings, how a system can evolve, and how fast teams can move on top of it.

It's also worth noting what they didn't do: rip out REST and force every team onto gRPC overnight. Keeping both transports first-class let teams migrate incrementally, on their own schedule, rather than treating a protocol change as a company-wide flag day.