Eqwalizer doesn't like lists:foldl?

Questions / Help
1

Given the following code…

-module(example).
-export([example/3]).
-export_type([acc/0]).

-record(acc, {last}).
-opaque acc() :: #acc{}.
-spec example([binary()], map(), acc()) -> acc().
-spec accumulator(Elem :: binary(), Extra :: map(), acc()) -> acc().
accumulator(Elem, _Extra, Acc) ->
    Acc#acc{last = Elem}.

example(List, Extra, A0) ->
    lists:foldl(fun(Elem, A1) -> accumulator(Elem, Extra, A1) end, A0, List).

…eqwalizer spits out all of the following…

error: incompatible_types
   ┌─ src/example.erl:13:5
   │
13 │     lists:foldl(fun(Elem, A1) -> accumulator(Elem, Extra, A1) end, A0, List).
   │     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   │     │
   │     lists:foldl(fun, A0, List).
Expression has type:   term()
Context expected type: acc()

See https://fb.me/eqwalizer_errors#incompatible_types
   │

  term() is not compatible with acc()
  because
  term() is not compatible with #acc{}

error: incompatible_types
   ┌─ src/example.erl:13:46
   │
13 │     lists:foldl(fun(Elem, A1) -> accumulator(Elem, Extra, A1) end, A0, List).
   │                                              ^^^^ Elem.
Expression has type:   term()
Context expected type: binary()

See https://fb.me/eqwalizer_errors#incompatible_types

error: incompatible_types
   ┌─ src/example.erl:13:59
   │
13 │     lists:foldl(fun(Elem, A1) -> accumulator(Elem, Extra, A1) end, A0, List).
   │                                                           ^^
   │                                                           │
   │                                                           A1.
Expression has type:   term()
Context expected type: acc()

See https://fb.me/eqwalizer_errors#incompatible_types
   │

  term() is not compatible with acc()
  because
  term() is not compatible with #acc{}

3 ERRORS

As far as I can tell, it fixates on all of the Acc :: term() items in the spec for lists:foldl/3, and doesn’t realise that acc() is a term().

If I erase the types, by wrapping lists:foldl/3…

foldl(Fun, Acc0, List) ->
    lists:foldl(Fun, Acc0, List).

…then eqwalizer is happy. But that doesn’t seem particularly idiomatic.

Moreover, when I add a spec to my foldl, to make use of type variables…

-spec foldl(Fun :: fun((Elem, Acc) -> Acc), Acc, [Elem]) -> Acc.

…eqwalizer doesn’t like that either.

Am I doing something wrong? Is eqwalizer doing something wrong? Are the typespecs on lists:foldl/3 wrong?

1 Like
2

The problem is the term() type (I’ve no idea why).
Copying your example and pasting the lists:fold function in the example module and changing term() to dynamic() in the spec solves the issue:

-module(example).
-export([example/3]).
-export_type([acc/0]).

-record(acc, {last}).
-opaque acc() :: #acc{}.
-spec example([binary()], map(), acc()) -> acc().
-spec accumulator(Elem :: binary(), Extra :: map(), acc()) -> acc().
accumulator(Elem, _Extra, Acc) ->
    Acc#acc{last = Elem}.

example(List, Extra, A0) ->
    foldl(fun(Elem, A1) -> accumulator(Elem, Extra, A1) end, A0, List).

-spec foldl(Fun, Acc0, List) -> Acc1 when
      Fun :: fun((Elem :: T, AccIn) -> AccOut),
      Acc0 :: dynamic(), % <- it was term()
      Acc1 :: dynamic(), % <- it was term()
      AccIn :: dynamic(), % <- it was term()
      AccOut :: dynamic(), % <- it was term()
      List :: [T],
      T :: dynamic(). % <- it was term()

foldl(F, Accu, List) when is_function(F, 2) ->
    case List of
        [Hd | Tail] -> foldl_1(F, F(Hd, Accu), Tail);
        [] -> Accu
    end.

foldl_1(F, Accu, [Hd | Tail]) ->
    foldl_1(F, F(Hd, Accu), Tail);
foldl_1(_F, Accu, []) ->
    Accu.

What’s wrong with the term() type?

3

This spec silent eqwalizer and I also gets no warnings/errors in dialyzer:

-spec foldl(Fun, Init, List) -> Acc when
      Fun :: fun((Elem, AccIn) -> AccOut),
      Init :: AccIn,
      Acc :: AccOut,
      AccIn :: AccOut,
      List :: [Elem].

foldl(F, Accu, List) when is_function(F, 2) ->
    case List of
        [Hd | Tail] -> foldl_1(F, F(Hd, Accu), Tail);
        [] -> Accu
    end.

foldl_1(F, Accu, [Hd | Tail]) ->
    foldl_1(F, F(Hd, Accu), Tail);
foldl_1(_F, Accu, []) ->
    Accu.

Edit

This also works:

-spec foldl(Fun, Init, List) -> Acc when
      Fun :: fun((Elem, AccIn) -> AccOut),
      Acc :: Init | AccOut,
      AccIn :: Init | AccOut,
      List :: [Elem].

I’m unsure what spec is better/correct.

4

Upping this, actually eqwalizer bombards me with errors for just about almost all functions from the lists module :smiling_face_with_tear:

5

If you add eqwalizer_support dependency to your project (see the instructions in README - eqwalizer/README.md at main · WhatsApp/eqwalizer · GitHub) then your example is accepted.

eqwalizer_support ships a special module eqwalizer_specs (eqwalizer/eqwalizer_support/src/eqwalizer_specs.erl at main · WhatsApp/eqwalizer · GitHub) that provide more accurate specs for some stdlib functions (especially “generic” functions like lists:foldl).

Compare it to the OTP spec:

The original spec says that the result type is Acc1 :: term() - note that Acc1 is not connected to any other type variables, so it’s just term().

There is an EEP - Eep 0071 - Erlang/OTP - bringing some clarification how type variables should be considered for type checking and there is a PR to make the lists specs more accurate for type-checking - Update and uniformise lists specs by VLanvin · Pull Request #8736 · erlang/otp · GitHub.

There is also a task - add sanity check that eqwalizer_support is present · Issue #82 · WhatsApp/erlang-language-platform · GitHub - to add more signal about eqwalizer_specs in CLI/UI.

3 Likes
6

I think this is the big take away here. The eqwalizer_support dependency is easy to miss, so validating its presence would probably make transitioning to Eqwalizer easier. I am excited to see it happen!

2 Likes
7

Aha. I have eqwalizer_support, but it’s not in the default rebar profile – because of Fix conversion of git_subdir to git-subfolder by rlipscombe · Pull Request #1005 · ninenines/erlang.mk · GitHub.

If I set REBAR_PROFILE=eqwalizer before running elp eqwalize-all, it picks up eqwalizer_support and reduces the number of warnings in my real project from 39 to 2.

…which I will now fix properly.

2 Likes
8

Ohhhh, that explains everything! Indeed, my bazillion errors are impressively minimised, also a solution here!

On the PR to OTP (Update and uniformise lists specs by VLanvin · Pull Request #8736 · erlang/otp · GitHub), that seems like a sensible change. Is there anything we can do to move that forward? The PR was created quite a while ago, would be lovely to see it done.

2 Likes