Is httpc considered bad? Why shouldn’t I use it?

Questions / Help
,
1

It’s been repeated so many times that it’s become an article of faith: “httpc is bad”.

Back in the day, ESL implemented lhttpc to replace it, and today there are numerous replacement HTTP clients for Erlang (hackney, gun, etc.) and Elixir (various hackney wrappers, finch, mint, etc.), and there’s been some discussion of dropping httpc from OTP entirely.

My question is this: why is httpc considered bad? Why shouldn’t I use it?

7 Likes
2

IIRC most of complaints about httpc were about interface. request/4,5 has that weird interface where your Request has to be either 2-tuple or 4-tuple, depending whether your request contains a body or not. If you want to wrap all your htttp calls into some function you always have to write something like:

case has_body(Method) of %% note, has_body is not exported anywhere in inets
    true ->
        {Url, Headers, Body, Mime};
    false ->
        {Url, Headers}
end

I’ve been there couple of times. Example when you do stuff like this is when you’re just forwarding HTTP requests.

There are also 2 different types of answers, of which 1.st is just the subset of the 2.nd:

%% 1st
    {StatusCode, HttpBodyResult}
%% 2nd
    {StatusLine, [HttpHeader], HttpBodyResult}

Its just a waste of effort to match both every time and I don’t remember if there is a reason when the first one is returned. IMO there should be just {StatusCode, [HttpHeader], HttpBodyResult}

@tsloughter mentioned couple more stuff here

Anyways, I’m still using httpc, but with a single-module wrapper around it that utilizes map interface for requests and responses.

EDIT: there is a small problem with standalone mode

3

httpc was a symptom of lacking routines back in the day to receive contributions.
So we added a client that was undocumented and almost untested and the kind of expectation that it is better than nothing and we can improve it later as it not official supported from start. This was not a good idea, and we had to fire fight for quite some while and legacy decisions made the API not the best. Current state is fairly ok (there are probably some problems left) but we like to clean it up more and make a better API (but it takes time and maybe next release if we managed to prioritize it). I think the aim should be light but good at what it does,
good for basic stuff but other can have all the bells and whistles.

3 Likes
4

Good to hear what you like and not like, it can help us improve what we got.
Standalone mode was a mistake, I think it should be removed in the long run.

2 Likes
5

It’s not bad, it’s just not getting updates and features. For example, there is no HTTP2 support (not even saying HTTP3). Existing software uses httpc extensively, even that running for billions of users.

6

Before it supported TLS or PATCH it was awkward, but we still used it. Now it’s great and working well for us, at load.

2 Likes
7

Since Erlang/OTP 26 (?), httpc always starts the ssl application and always checks the system certificates. It’s annoying.

8

when there is an opportunity to significantly improve httpc, it would be great if a lower level module eg gen_http (see Httpc/httpd improvements - #7 by josevalim) would be considered, a very low level wrapper around network socket without even processes and then build httpc on top.

On the Elixir side we have three libraries:

On one hand it may seem like too much but we have found it to be very practical, most users are very happy with Req but have an option to jump to lower levels. If OTP provides a very low level http socket implementation, I believe it will be a very welcome addition.

6 Likes
9

A great suggestion! I’m confident it would find a lot of use in the Gleam world too.

10

The main issue with httpc that caused me to not use it and to advise others to not use it many many years ago was it would occasionally hang when using HTTPS. I believe @ingela you fixed this :). But that was so long ago I don’t remember any details, but I think that is the sort of core of what became the fear of using httpc.

I definitely think the API could use a revamp (I created that forum thread about doing just that) but there isn’t an underlying fault in httpc I’m aware of that should make it be considered bad at all for HTTP 1.1. Obviously requirements vary so which client works best for a use will vary.

I think a starting point should be a Req like API for httpc and then, or in parallel, the addition of a low level module like gen_http.

One issue I’d like to see a gen_http resolve is that you can’t use {active, N} with http decode packet because you have to switch decoding from headers to body, so like Elli, the http server, is unable to use {active, N}.

5 Likes