Livery - high-performance HTTP/1.1, HTTP/2, HTTP/3 server for Erlang/OTP 27+

Your Libraries & OS Mentoring Libraries
, , , , , , ,
1

Livery 0.2.0 is out. This is the first public release.

Livery is a native Erlang web framework, written very much in the spirit of Axum, Tower and Hyper. You write one router and one middleware stack, and it serves them over HTTP/1.1, HTTP/2 and HTTP/3 from a single runtime, sharing the same handlers across the three adapters and advertising Alt-Svc so clients can move up to H3.

Why it is shaped this way

A handler is a plain function, and a piece of middleware is a continuation over immutable values, call(Req, Next, State) -> Resp. It is the Tower/Axum model, not the old (req, res, next) where everyone mutates a shared request. Req and Resp are plain records, never a shared mutable handle, so they are easy to pass between processes and easy to test. Next is simply the rest of the pipeline, handed to you as a function, so wrapping it in a try/catch, a timeout or a span is just calling it inside your own code. The handler never has to know whether it sits on H1, H2 or H3: the request and response values are the same everywhere, and the protocol differences show up only through capabilities (trailers, extended_connect, datagrams). The nice consequence is that the very same model also runs outbound, which is exactly what the client is.

Features

  • One handler, three protocols: the same router and middleware serve H1, H2 and H3, and Alt-Svc upgrades clients to H3. A handler never has to know which protocol it is on.

  • Tower-style middleware: value-based call(Req, Next, State) -> Resp, composable globally or per route.

  • Routing and composition: a radix-trie router with path params, plus livery_router:nest/2,3 and merge/1,2 to build an area on its own and graft it in.

  • Shared state: livery:start_service/1 takes a config map that every handler and middleware reads through livery_req:config/1,2,3.

  • Streaming: chunked responses, Server-Sent Events, NDJSON, and file responses with byte ranges, over every adapter.

  • WebSocket and WebTransport: WebSocket over H1/H2/H3, WebTransport over H3.

  • Composable HTTP client (livery_client): the outbound twin of the middleware, with timeout, retry, concurrency-limit and circuit-breaker layers, streamed bodies, and load balancing across a pool of endpoints (power-of-two-choices or round-robin, with passive outlier ejection).

  • OpenAPI 3.1: generate the document from your routes, serve Redoc or Swagger UI, and validate request bodies.

  • MCP: serve the Model Context Protocol Streamable HTTP transport on the main listener.

  • Batteries included: CORS, security headers, gzip and deflate compression, multipart and form parsing, rate limiting and load shedding, ETag and conditional GET, static file serving, health and readiness endpoints, and a Prometheus /metrics handler.

  • Auth: signed session cookies, token introspection, and bearer middleware with OIDC discovery and JWKS.

  • Observability: OpenTelemetry-style traces and metrics, with trace context carried into your logs.

The dependencies moved with this release too. Livery keeps its adapters thin and protocols are handled by external libraries: quic 1.6.3, h2 0.8.0, erlang_h1 0.4.0, erlang_ws 0.3.0, webtransport 0.3.1, hackney 4.2.0 and instrument 1.1.3.

It is Apache-2.0, still early 0.x and under active development.

Feedback and questions are very welcome.

9 Likes
2

Do you have any performance benchmarks against cowboy and bandit?

3

@saleyn I made a quick benchmark on my MacBook Pro M4, for what it’s worth:

  • HTTP/1.1: livery ~130k · cowboy ~142k · bandit ~147k
  • HTTP/2 TLS: livery ~154k (fastest, 0 errors) · bandit ~139k · cowboy ~80k (stream resets under load)

The benchmark is available on GitHub and is part of the 0.2.2 release. Livery’s H2 leads the field. The remaining 8% H1 gap is the per-request worker-process model compared to cowboy_loop. There is some room to improve the HTTP/1.1 layer though?

HTTP/3 is slower than HTTP/2 on loopback due to its pure Erlang implementation. In real conditions it should perform better.

4

This benchmark was kind of too simpler. Livery has now a new benchmark testing different load (see blow the result). Livery seems to perform quite well on my busy machine.

##### Summary (req/s) #####

     [HTTP/1.1]
     server           tiny     bytes1k    bytes10k   bytes100k        echo
     livery      121662.16   119169.44   114507.74    60105.03   110865.24
     cowboy      136198.20   136136.63   131270.34    59101.07   123544.96
     bandit      145435.21   142888.02   136460.48    55840.80   135905.99

     [HTTP/2 over TLS]
     server           tiny     bytes1k    bytes10k   bytes100k        echo
     livery      118493.00   111122.25   104413.12    26829.38   100465.75
     cowboy       79940.62    79933.38    79932.12    79223.25    39926.25
     bandit      121088.00   108694.62   101074.38    51490.25    99564.50
     --- h3 ---
     === livery (h3, 8s) ===
     throughput  : 14758.5 req/s
     latency p50 : 4.112 ms

The benchmark is available in the sources.

5

**Profiling led to a 2.4× win on large HTTP/2 responses **

I’ve been benchmarking Livery against **cowboy ** and **bandit **, and HTTP/2 large-body throughput was lagging roughly 2× behind bandit at 100 KiB.

I started to play with the “flow control” , bumping the client’s window (h2load -w24 -W24), but it changed **nothing **. So it was not flow control.

eprof on the out-of-process server (driven by external h2load) told me the real story, per 100 KiB response:

  • ssl:send/2 : **8.0/req ** (one TLS write per DATA frame)
  • crypto:aead_cipher_nif : ~15/req
  • gen_statem:call : 12.6/req

ssl:send afaik *is * a gen_statem:call to the TLS connection process. So writing one frame at a time meant ~8 round-trips + 8 records + ~15 encryptions per response. The h2 library has the be fixed to coalesce a response’s frames into a **single ** ssl:send (one iolist) while still respecting flow control and max_frame_size, only the write are batched.

Before → after (Livery, HTTP/2 over TLS)

Body before (h2 0.8.0) after (h2 0.9.0)
tiny GET ~128k ~137k
1 KiB ~122k ~133k
10 KiB ~114k ~123k
100 KiB **~28k ** **~66k **
echo POST ~108k ~118k

Cross-server, HTTP/2 over TLS (req/s, after the fix)

Body livery cowboy bandit
tiny GET **137k ** 80k 125k
1 KiB **133k ** 80k 118k
10 KiB **123k ** 80k 107k
100 KiB **66k ** 80k 62k
echo POST **118k ** 40k 104k

With latest change the large-body gap is closed: at 100 KiB Livery went from 28k to 66k (2.4×), now ahead of bandit and approaching cowboy; smaller bodies gained ~7-9% from the same change. (cowboy’s flat ~80k is its own HTTP/2 behaviour under h2load’s stream churn.) . The remaining gap with Cowboy is mostly due to our current design and will be hard to eliminate without compromising resilience: every request is counted so we can protect against DDoS.

The benchmark stools are available on git.

1 Like
6

livery 0.2.4 is out with these changes. Enjoy :slight_smile:

7

Hi! This is a really big job!

1 Like