Asn1ct support for non-ascii files? Attempting to compile some TCAP-MAP ASN.1 files resulting in error

I am brand new to Erlang, and wanted to test the ASN.1 environment provided through the compiler asn1ct. I am currently giving an attempt at compiling some files from the telecom TCAP-MAP ASN.1 specification, but am facing straight errors with what would look apparently as file encoding issues.

In short, here is an example of an error I am facing:

Erlang/OTP 25 [erts-13.0.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Eshell V13.0.2  (abort with ^G)
1> asn1ct:compile("MAP-CommonDataTypes.asn", [ber, verbose]).                                                        
Erlang ASN.1 compiler 5.0.19
Compiling: "MAP-CommonDataTypes.asn"
Options: [ber,verbose,{i,"."}]
MAP-CommonDataTypes.asn:300: syntax error before: 'Â'

Those ASN.1 files are visible here: They have nothing really special, are standard ASN.1 files, may contained some non-ASCII / UTF-8 characters in comments, but I expect this should not bring any issue to the compiler.

With the error provided by Erlang, I fail to understand if the error corresponds to a line or character number, neither can I spot any  character in the file from my terminal or any standard editor. Moreover, the corresponding hex codepoint 0xC382 does not exist in this file. I am using a recent Ubuntu system with a default terminal and environment config (LANG=en_US.UTF-8).

Any help of feedback will be really appreciated.

1 Like

For reference you could compare with this working project:


The ASN.1 compiler does not handle UTF-8 encoded files and since UTF-8 and ISO-latin are the same for ASCII characters the problem does not show up until line 300 which contains the bytes 0xC2 and 0xA0 which is U+00A (non breaking space). These byte codes are converted directly to tokens since they are not interpreted as white space which they would have been if the scanner handled UTF-8.

On line 300 you have
alertingLevel-0 AlertingPattern ::= '00000000’B

the C2 A0 bytes appear just before ::= '00000000’B so they are not in a comment

It might be a good idea to improve the handling of UTF-8 here but it has not been necessary so far.
The standard MAP-CommonDatatypes and friends that we have tested with does not contain UTF-8 in this way.

This is the first time I have noticed a problem like this so apparently all ASN.1 specifications we have tested until this one has stayed away from UTF-8.


Thanks a lot for your analysis. Those ASN.1 files are extracted from 3GPP docx documents converted to txt. I guess those unicode specific characters are coming from formatting oddities under Microsoft word.