Is_improper_list/1: why isn't there this builtin type query?

Questions / Help
1

Hi There,

I was playing around with the Diameter example and came across a new type: improper_list. I never knew of its existence and apparently it is not intended to be passed out of internal workings to external API callers.

But the message() type that is returned is documented as -type message() :: record() | maybe_improper_list().

My use case is such that I pass Erlang data structures out via JSON string to a frontend. Of course the json:encode(..) function failed when it encoded the improper_list type, there doesn’t seem to be a universally agreed upon representation of the Erlang improper list type - fair enough.

Having an unrecognised type and extending the json encoder isn’t a big deal, I’ve done it for Pid, Ref, Key-Value lists etc. What I find strange is that I can’t do this for improper_lists because there isn’t a reliable test for improper_list, i.e., is_improper_list(...)

My argument for such a builtin is that improper lists break code that works with lists while pretending to be a list:

1> is_list([1|1]).
true
2> length([1|1]).
** exception error: bad argument
     in function  length/1
        called as length([1|1])
        *** argument 1: not a list

Of course I could create my own is_improper_list:

is_improper_list(V) when not is_list(V) ->
  false;
is_improper_list(V) when is_list(V) ->
  try
    length(V),
    false
  catch 
    _ -> true
  end.

but this breaks the anti-pattern of using exceptions as control flow, which I don’t wish to be doing. Plus I don’t want to be raising - potentially - exceptions for each data structure that I might or might not be sending out over a wire via a web-socket.

My explicitly use case involves not knowing exact types, so tighter type checking isn’t a solution for me.

Is there a reason why is_improper_list doesn’t exist? Second is there a reason for it not existing?

Cheers!

EDIT: code fix - boolean logic on a sunday afternoon doesn’t work!

2 Likes
2

It looks like what you’ve run into is a long-standing quirk in Erlang. Improper lists are technically valid terms in the language, but there has never been a built-in predicate like is_improper_list/1. The only option today is to traverse the structure yourself, which is linear in complexity, or to rely on functions like length/1 that crash on an improper tail. That’s not elegant and it’s understandable you’d prefer something explicit and reliable.

The lack of such a check isn’t an oversight so much as a deliberate omission: improper lists are rarely encouraged outside of iodata use cases, and because you can’t determine “properness” in constant time, the designers never made it a guard BIF. Still, the absence of a clear runtime test becomes painful when you need to sanitize or serialize arbitrary Erlang terms into JSON or other external formats. In your case you’ve already adapted for refs, pids, and other opaque values, but improper lists leave you without a clean hook.

Since you’ve run into a real usability gap, I think the most constructive next step is to raise this as a GitHub issue in the official Erlang/OTP repository. Describe your exact situation - passing arbitrary Erlang terms over JSON, where improper lists surface in APIs like Diameter and break serialization - and point out that there’s currently no sanctioned way to detect them without catching exceptions. Framing it that way invites discussion, and even if the core team decides not to add a new BIF, you’ll at least get clarity on the reasoning and maybe spark agreement on providing a helper in stdlib.

2 Likes
3

Makes me think of new term: Hackalution:

  1. noun: An actively implemented solution that is known to be a hack but that won’t be fixed because the underly programming language doesn’t allow for a better solution.

Which implies the verb hackaluting.

@vkatsuba Thanks for the explanation but I won’t be pursuing the issue any further than leaving this here. Your explanation is fairly clear: it’s a known bug but because it shouldn’t be, it is.

1 Like
4

Just FYI, see the Elixir version of it elixir/lib/elixir/lib/list.ex at 9a8794183e99fc595c9e66c27f199000644f853a · elixir-lang/elixir · GitHub

3 Likes
5 
  def improper?(list) when is_list(list) and length(list) >= 0, do: false
  def improper?(list) when is_list(list), do: true

That would imply that the length guard does not throw an exception in Elixir? Is that a builtin or does that also do some try/catch flow control under the hood?

EDIT:

Ok, that’s the fix: using the guard length:

1> R = fun(V) when is_list(V), length(V) >= 0 -> true;
   (V) -> false
   end.
#Fun<erl_eval.42.81571850>
2> R([1,2]).
true
3> R([1|2]).
false
4>
3 Likes
6

A guard that throws an exception is caught as a guard failure.

I would say this thread has arrived at the right conclusion. There is no guard is_improper_list/1 because it would run in linear time. That length/1 exists as a guard was a mistake from the start, and we wouldn’t want more of them.

Improper lists are mostly frowned upon by type systems, because such in general likes lists to be homogenous, which an improper list is not. The Erlang language in itself has no problem with improper lists (cons cells). But a guard that has a not predictable running time (like length/1) is problematic.

6 Likes
7

“There doesn’t seem to be a universally agreed upon representation of the Erlang improper list type
”
There isn’t a universally agreed upon representation of integers either.
559341137632631922106918752387018466764798630491700586799866719699848428890023584320089622090651200528167004731897590718558233136622073615913210052132884741903843641574540751605795928590257949865390953245937462753179316419658966686262515385439526360795514282052629747942942913739553258875568360476666078672900
is a valid integer in Erlang (Lisp, Smalltalk, Python), but try to send it in JSON and if the receiver is in almost anything else,
you’ll get a floating-point overflow.

Indeed, almost no Erlang data type has a ‘universally agreed upon representation’.
It’s not clear to me how you can reconcile ‘NOT knowing exact types’ and ‘passing Erlang data in a JSON string’.
Smalltalk programmers faced a similar issue, and just gave up, devising STON GitHub - svenvc/ston: STON - Smalltalk Object Notation - A lightweight text-based, human-readable data interchange format for class-based object-oriented languages like Smalltalk. instead.

I’m not seeing a huge difference between “might include an improper list” and “might include a bit string”.

2 Likes