What are the main reason for Erlang numbers not having infinity and nan

Questions / Help
21

I’m sorry, but all this “it isn’t useful” “there is no correct answer” “it makes no sense” stuff is not making a good impression on me. Floating-point arithmetic is not real number arithmetic and cannot be. Whatever we do, it’s going to behave differently from real arithmetic.

Let’s stipulate for the sake of argument that integer and rational arithmetic bounded only by the capacity of the memory is usually the right tool for the job and that quiet truncation, clamping, clipping etc are almost always about as bad a footgun as you can find. We recall that Dijkstra commented that

  • we program for an unbounded machine
  • if our program runs on a sufficiently large machine, we get the right answers
  • if our program runs on an insufficiently large machine, we get wrong answers
    (and we may get them fast, but what’s the point of that?)
  • if our program runs on a HOPEFULLY sufficiently large machine, we either
    get the right answers or we get an apology from the machine
    so we really want our machines to respond to integer overflow with exceptions.
    (I briefly had the pleasure of programming in C on a MIPS machine where integer overflow did raise an exception, and it was a real joy to be able to trust the answers.)

We must also recognise that there is no shortage of problems, from trigonometry to (probability) significance testing, where integers and rationals are NOT the right tool for the job; that while it may be possible to tackle them using fixed point arithmetic with manual scaling, it’s unspfxkably difficult to get right and to maintain. There have been many proposals for rivals to floating-point, of which I’d say that Gustafsson’s unums and posits (see the book “The End of Error”) are to me the most impressive, but in terms of what we can buy today, aut punctum volans aut nihil.

It’s also worth digging into the meaning of the word “infinite”. It doesn’t necessarily mean a specific value, it simply means “not bounded”. A bit pattern that means “whatever this value is, it is definitely too be for me to put a finite bound on it” is etymologically entitled to the term “infinite”. If I compute 1.0e200 * 1.0e200, the fact that the computer cannot represent it as an IEEE/IEC double means that “definitely too big for me to put a finite bound on it” is an entirely accurate description of the result. “isinfinite(X)” is unfortunately named, but once you realise that it just means “istoobig(X)” it’s quite unobjectionable.

Now the thing is that IEEE/IEC arithmetic is specified. It cannot be specified to be consistent with real number arithmetic. Even if a computer had infinitely many bits of storage ($\aleph_0$) that would still be infinitely too few for almost all real numbers. So there are two questions. Making sense: is IEEE/IEC arithmetic consistent in its own terms. Usefulness: can the IEEE/IEC operations give us a sufficiently good approximation of real number arithmetic for a sufficiently wide range of problems for it to be worth vendors providing it and some programmers learning how to use it carefully.

The answers are yes, and yes.

BUT butchering the standard without having a thorough understanding of why the standard is the way it is means that doing floating point calculations in Erlang means having to follow different rules from floating-point in modern hardware and popular modern programming languages. As I wrote earlier, it feels like having to program in the 1970s.

Of course it could be worse. Like a Lisp system I sometimes use, whether floating-point overflow triggers an exception or returns an infinity might depend on a remotely set flag (worse still, a flag set in the debugger!).

You may legitimately say that IEEE/IEC special values don’t make sense to you.
That’s a paraphrase for “I don’t understand them” or “I don’t like them.”
They make sense. I’m not sure that I care for them myself.
But they are in the standard, they were designed and approved by floating-point experts,
and they’ve been implemented over and over, so they clearly make sense to some people who ought to know.

There is a hardware cost to providing the special values. Xerox Lisp Machines punted:
they copied the IEEE formats but not the IEEE semantics. Early SPARCs didn’t implement the special values in hardware; when they came up the machines trapped to software emulation. I remember showing a student how to make his neural network code substantially faster by changing x += yy to if (y > 1.0e-16) x += yy. Modern hardware doesn’t have this performance bug.
So now consider some contrived code:
z = …;
y = 1/(zz);
x += y;
And let’s imagine that z could get very big.
WITH infinities, this code is fine.
WITHOUT infinities, this has to be written as
z = …;
if (z < THRESH) {
y = 1/(zz);
x += y;
}
And I have to look at EVERY SINGLE operation to see if it might overflow or underflow (yes, there are systems where underflow raises an exception, and yes, I’ve used them, and yes, it is no less of an error than overflow, but MY WORD it is painful).

Oh, let’s go back to ‘not continuing the computation’.
Suppose I have
sum += 1.0/ddot(N, xs, 1, xs, 1);
Or in Erlang terms,
dot(Xs, Ys) →
dot(Xs, Ys, 0.0).

dot([X|Xs], [Y|Ys], S) →
dot(Xs, Ys, S+X*Y;
dot(, , S) →
S.

…
Sum1 = Sum0 + 1.0/dot(Xs, Xs).

Suppose there is a floating-point overflow somewhere in the calculation of <Xs,Xs>..
I don’t care where exactly it is and I don’t care if the loop completes before
telling me. In fact, continuing the calculation past the overflow will give me the
right answer: 1/<Xs,Xs> will be too small to represent and won’t make a difference.
I call this useful. If I want to do this, my code will be simpler due to the IEEE/IEC rules.

3 Likes
22

@nzok thanks for responding so thoroughly, and I can now clearly see the performance penalty we’re encoring, and I myself prefer branchless code. I also don’t see how using the standard would introduce foot guns, as floating point will at best be an approximation, and there is nothing wrong with infinites or NaNs, as they do communicate what happen rather that was intentional or not. I actually thought these decisions were more fundamental to the BEAM and fault tolerance.

1 Like
23

These decisions where made before -97, none of us working in OTP where there then,
the only active person is @rvirding so maybe he knows/remember.

2 Likes
24

I would say you are both wrong, @Maria-12648430 by saying that infinity is unconditionally not a number, @nzok by saying that it unconditionally is. It depends on what context you use and what you mean by “number”, for which there seems to be no unconditional definition, either.

That aside, in practical terms regarding IT, infinity can IMO not be seen as a number. If you want to bring up extended reals, we are not even using “simple” reals but IEEE 754 floats, a mere approximation of reals. By extension, we can not use extended reals, respectively infinity, only “Extended floats”, where the IEEE 754 Inf is just an approximation (if you want to call it that) of mathematical infinity.

1 Like
25

There is everything wrong with that approach. It would be ok if IEEE 754 infinity was actual mathematical infinity, not a way to say “too big for me to handle”. IEEE 754 further allows to work with this approximation as if it was mathematical infinity.

For illustration, assume you’re summing up the money your company has, and at some point it turns out that you have a lot, too much to handle, so you get infinity. Looks good to shareholders and the like, but it’s not true. You don’t have infinite money, you only have a lot of it.
Now you go on a shopping spree, and with every purchase ask “can I afford it?”, like “is the price of this thing smaller than what I have?”: $price_of_thing < infinity –> true. Turns out you can afford it, so you buy it and subtract the price of the thing from what you have: infinity - $price_of_thing –> infinity. Then you repeat, and again, until the money runs out, which it doesn’t. Keep in mind that you really don’t have infinite money, only an amount that once was too big to handle. Your calculations happily chugg along as if you did have infinite money, though.

If NaN and infinity were valid numbers, you would rather have to check manually after each calculation (or also before, depends) what the result was, which is not too great for performance. You may want to check at the point where the result actually “matters” (like, when going spending), but then it’s hard to find out where this infinity (or NaN) happened. You may forget the check. Or you may use part of the same calculation in a new function, and not be aware that you have to check.

Compare this to how Erlang handles it: if it can’t handle it (like, too big to handle), it throws an error. You immediately know when and where it happened, and don’t have to remember manual checks to prevent basically invalid results from spreading around and spoiling the consistency of your system.

There may be scenarios where you could say "I don’t care if it is too large to handle, in that case I’ll just treat it $some_custom_way”. I would consider those rare edge cases, while cases where when things can’t be handled are actual errors are pretty common. Anyway, if you want/have to, you can still try…catch the error and handle it $some_custom_way.

1 Like
26

@nzok tl;dr, sorry. TBH, I’m still trying to figure out if I should take some of the things you said as ridicule and be offended, or not. ATM I tend to “why bother”.

27

Fault tolerance is not “everything is ok, there will never be errors, we gloss over everything”.

What one generally wants is a consistent system that is fault tolerant. If something happens that is not expected, it is dangerous to continue as if nothing happened because then it spreads and propagates. You can gloss over errors if you need, but the system won’t and should not do it for you. That is, if you expect some calculations ending in indefinite/invalid values one way or another and can handle it, you can try…catch the error.

2 Likes
28

I don’t know if early releases of Erlang ran on VAXes at all, but in 1997 VAXes were still around.
They were beautiful machines to program in assembly, and they were also the only machines I’ve ever used that had two different double precision floating point formats neither of which was IEEE/IEC. (If memory serves me correctly, the bit pattern that might have been interpreted as -0.0 was a trap value, not entirely unlike an sNaN.)
IEEE/IEC arithmetic was absolutely dominant in the personal computer (PC and Mac) and workstation (Sun, HP, …) world, but in the mainframe and supermini world, hardly known. Modern IBM z/Series machines support IEEE/IEC arithmetic, including decimal floats, but back then they didn’t.
The obvious question is what processors Ericsson were using back then, whether they all supported IEEE/IEC arithmetic, and whether there might have been an attempt to keep the possibility of porting Erlang to them open. (I do know that when someone decided to provide higher level services on an AXE-10 system, the Erlang code ran on a separate board with a separate processor; I believe this was due to a desire to make absolutely minimal changes to the EriPascal code running on the main processor(s).)

Quintus Prolog was ported to Xerox Lisp Machines (IEEE formats but not IEEE special values) and IBM mainframes (no IEEE support at that time), so back in the 1980s and 1990s it made sense not to require IEEE compatibility. The AWK implementation I still maintain had to struggle with irregularities in IEEE support amongst systems that claimed to have it.

But it’s 30 years later. DEC is deceased. Compaq is carked. Sun has set forever and the SPARC has gone from my life. MIPS is missing. Transputers have been transported to museums. Those lovely Burroughs machines survive, just, but only in emulation. Nobody is considering running Erlang on an emulated E-mode Burroughs machine; why would they when it can run fine on the associated Windows boxes? C and Fortran got official IEEE/IEC support years ago. Why should Erland be stuck in the '70s?

29

Where on earth did I say that infinity is “unconditionally” a number?
There are mathematical systems (like the ordinary integers) where it is not;
and there are mathematical systems (like the ordinal and cardinal numbers
of set theory) where there are infinities that count as numbers every bit
as much as 0 or 137. The fact that some mathematical systems include
infinities and some do not was pretty much the point that I was making.

As for computing, the IEEE infinities are members in good standing of
the category “IEEE numbers”. They are an essential part of IEEE floating-point
semantics, and refusing to call them “numbers” is petty quibbling over words
(lexicon) rather than meanings (semantics). NO floating-point system is or
can be anything other than a crude approximation to the mathematical real
number field. Get over it. The IEEE infinities are not peripheral or
optional; they are core elements of the IEEE system. It’s not useful
as an aid to practice or clear thinking to say that -0.0 is a number but
+infinity is not. If the result of a calculation using IEEE-compliant

operations is +infinity or -infinity, that actually gives you useful information
about what the numerical result would have been if possible: it would have
existed as a mathematical real AND its magnitude is very great. It’s not as
precise as other floating-point numbers, but floating-point numbers are not
all equally precise anyway, so once again, get over it.

Again, let’s not argue lexicon. Let’s argue semantics. +infinity is NOT an
approximation of an infinite number. It’s an approximation of an extremely
large number which is FINITE in the reals but outside the limits of finite floats.
It’s unfortunate that the word “infinity” was abused for this purpose;
“superbig” might have caused less confusion. (Where 1.0/0.0 comes in is that
0.0 doesn’t actually mean zero, it means “non-negative but too small to fit”.
Think of 0.0 as \epsilon, -0.0 as -\epsilon, +inf as 1/\epsilon, and
-inf as -1/\epsilon (except that each time you use \epsilon, it represents
a different teeny tiny number).)

Of course you are free to refuse the label “number” to anything you want to,
just as ancient Greek mathematicians refused to call 1 a number. (It’s not
a number, it’s the generator of numbers!) The question is whether the refusal
simplifies your mathematical life. Or in this case, your programming life.

30

Your illustration illustrates a problem with floating-point numbers,
but not a problem with floating-point superbigs.

The largest Zimbabwean banknote was $100,000,000,000,000 or 1.0e14.
Suppose I have a million of them, so that I have 1.0e20 Zimbabwean dollars.
1.0e20 - 1000.0 = 1.0e20.
So I could keep on spending $1000 indefinitely without ever diminishing my $1.0e20.
The problem is the limited precision of floats;
the infinities as such are in no way special here.

This is a very good illustration of why calculations with money need to be done using
integers or rational numbers, but since it fails to discriminate between superbigs and
very-large-but-representable floats, it doesn’t really belong in this thread.

31

That’s right, you didn’t. And neither did @Maria-12648430, though her phrasing had a stronger suggestion in that direction. So if we want to split hairs, you two made two complementary (not sure this is the right word?) but not contradictory statements, so calling either of them wrong would be wrong. But yeah, I get it.

I’m not refusing it (and you are equally free to assign the label “number” to anything you want to). I’m saying it depends on what you mean by it in what context.

Anyway, how ever that may be… I fail to see why we must introduce something that contains virtually dozens of trapdoors (or footguns, whatever) without any obvious gain (ie, not obvious to me) other than “those are standard trapdoors that everybody else has!”

You may want to bring this up again:

I would argue that having IEEE semantics in Erlang imposes extra checks, first of all on the users, to check the outputs for NaN/Inf, or otherwise risk propagation of (semantically) invalid data through the system.

Then again, maybe I just fail to see any useful application here, so maybe just give me a real world use case? :man_shrugging:

32

Well, I didn’t say anything about real-world money.

Assume the imaginary “money” in my illustration to be ridiculously big, 1.0e308 Bogus-Dollars if you want. I have two of them, so I get Inf. Now I spend one of them. I still have Inf.

Unless you bring in the restriction of the largest real world money that has existed at some point, I think my example does illustrate the point.

Apart from that, good example for another problem with floats (that I don’t dispute and am willing to accept).

33

I like where this discussion is going, but since the question was why the limitations exist, then I would prefer that we see examples of real world cases where this limitation is needed. I have not been able to find one my self, what @nzok is communicating is as far as I can see and know the closest so far. And also the most logical as the standard was in it’s early infancy and not at all as widespread in hardware today.

34

These decisions where made before -97, none of us working in OTP where there then

I remember a conversation over morning coffee I had with Joe when he worked at SICS. I had played around with L-systems and Joe said that floating point support was added to Erlang when he too played with L-systems.

If the initial floating point support was for a toy project, I’m not surprised that the semantics are what they are and not proper IEEE 754 as, as soon as they were in Erlang, fossilization and backwards compatibility hinder changes.

3 Likes
35

Laslo Hunhold’s Takum Arithmetic is another alternative (or a family of alternatives):

Takums adopt the essential properties proposed by Gustafson, while also satisfying some additional criteria. The goal for Takums is to be suitable as a general purpose real number approximation, while Gustafson’s designs are arguably lacking in this respect, compared with either IEEE-754 or Takums, when comparing dynamic range properties. On first glance, Takums seem the more principled design compared to Gustafson’s designs.

Nonetheless, the principal aim here is to establish a dynamic range that exhibits overall compatibility with general-purpose computational tasks. While augmenting the dynamic range through an elevation of the upper limit is plausible, it invariably entails a compromise on the coding efficiency for the encompassed numbers. Moreover, it is imperative to delineate between crafting a new format tailored for application-specific representations and devising a novel general-purpose arithmetic number format. The aspiration of this research is to foster the latter.

36

Great to hear that, although I do not agree with fossilization and breaking backwards compatibility, rather we can look at it as the current implementation is incomplete, and therefore it can be argued that it is a bug, it was only recently in OTP 27 that signed and unsigned zero equality was fixed.

37

Yes, originally we didn’t need floating point for the telecom systems we were looking at. From what I remember they were added because Joe wanted to do graphics. :grinning_face: So they became a bit of a hack and backwards compatibility has always been important.

IIRC one of the original issues we were discussing was whether you could mix integers and floats without using explicit conversion, e.g. do 1.5 + to_float(45). Fortunately we decided against that. :grinning_face:

2 Likes
38

I wanted to quote a public source so that it just didn’t get reduced to “trust me bro.” If you read the paywalled standard carefully, you will see that the only way to produce a non-finite from an operation (given only finite inputs) is to run into an exception of some sort. That is, the default exception handling produces an appropriate non-finite (and also sets a status flag indicating that the exception occurred).

Our alternate exception handling, which is fully supported by the standard, raises an Erlang exception whenever there is an IEEE 754 exception other than underflow and inexact. This is fine and is not a violation.

That the standard allows you to continue in the face of exceptions, in a sane manner, is an optional feature (which is on by default) that can be useful if you’re careful.

However, the standard is explicit about that expression of yours signaling a divideByZero exception. While “the default result of divideByZero shall be an ∞ correctly signed according to
the operation,” we are explicitly within our rights to raise an Erlang exception instead as described in section 8.3.

We are not butchering the standard. We are following it to the letter. In fact, we are even allowed to do this for underflow and inexact if we wish, but we do not for practical reasons.

Indeed, there are many good reasons. There are also many good reasons for immediate error reporting, which is, once again, supported by the standard.

The default behavior described in the standard is good for what it’s intended for. As idiomatic Erlang code crashes when encountering the unexpected rather than trucking on, the alternate exception behavior that Erlang adopted is good for what it’s intended for.

Yes, and this is pretty much my point. If you follow the default handling in IEEE 754, you have no way of distinguishing the “actual infinity” of 1 / 0 and “oh the result of this intermediate computation way over there was just too big tee hee let’s propagate it in absurdum and maybe hide it when we feel like it.”

Delayed error reporting and that Inf acts mostly like you would expect infinity to act is useful if you know what you’re doing. I am not arguing that it’s not, I’m arguing that it becomes easier to make mistakes because it conflates errors and regular return values.

Consider erlang:binary_to_term/1: if we had made it return error on errors instead of raising an exception, it would have been torn apart as an example of awful API design because error is also a perfectly valid return value.

We’re fully compliant with the standard (albeit perhaps by accident), and the signed/unsigned zero equality thing had nothing to do with IEEE 754 as such.

5 Likes
39

I think there still is an argument for incompleteness, as there is no flag provided to change mode. I think it would be great if we can position Erlang as a modern platform in all aspects, and especially for scientific compute. Furthermore language integration is limited at best when most other languages support infinites, and Erlang has to sort to hacks instead of proper first class citizens.

40

But why?

Erlang is unlikely to ever have good enough sequential performance for that, it’s better to leave orchestration to Erlang and the actual compute workload to something better suited.

Sure, but we cannot do anything about it now. If we were to leak NaN/Inf into terms we’d spend the next few decades fixing bugs arising from that. We assume finite floats just about everywhere.

5 Likes