Concurrent mod_esi use - single ESI modules invoked sequentially?

Questions / Help
1

For various perverse reasons, I’m experimenting with httpd and mod_esi.

In my testing, it seems like a single ESI module is invoked sequentially – that is: if I make 5 GET /Alias/Mod/Fun requests, they’re serviced in order. If I throw a timer:sleep(500) in there, then no client can proceed until the previous sleep completes.

Is my understanding correct? When I implement the same thing as a cowboy loop handler (with timer:send_after), the clients (I’m using httpc, again … various perverse reasons) complete in parallel.

1 Like
2

I don’t have an idea of how handling ESI requests should be any different than handling any other requests, that would be a bottleneck. Can you save me some minutes and provide me a minimal setup that I can experiment on?

3

Hmmm. If I extract the code into a separate test app, I don’t see the problem. I’ll keep digging to see what the difference is.

(I did some digging; see the later posts)

4

It looks like it’s actually a problem with (how I’m using?) httpc. Here’s a repro, with instructions:

5

I’ve added httpc/cowboy and hackney/httpd tests to the example. It definitely looks like something’s going on inside httpc – if I make a single request, then multiple concurrent requests, they run sequentially. If I don’t make the initial request, then the concurrent requests run, well, concurrently.

I see this with both httpd+mod_esi and cowboy. Hackney doesn’t exhibit the problem.

6

It could be how you configure it to use pipelining/persistent connections. Needs configuring to be used efficiently. It could also be that looking for reusable connections is currently done in a dumb way, that could become a bottleneck.

7

Something really weird is happening with SessionID. When pause/1 is run, all the requests that should be “concurrent” get the same SessionID. When pause/1 is removed, they all get different SessionID’s. If you perform request with {sync, false} in pause/1 then it also works. If you wrap request in pause/1 into a new process it also works. I’ll investigate more when I find time.

1 Like