Tell me more ×
Programming Puzzles & Code Golf Stack Exchange is a question and answer site for programming puzzle enthusiasts and code golfers. It's 100% free, no registration required.
up vote 3 down vote favorite

Your task is to golf a program that prints which character encoding, if any, the data read from the input represents. It should be able to detect the following encodings:

One of the above names of character encodings should be printed (in the same way) if it can be determined which is used. If the input is not ASCII, but it's ambiguous which of the other encodings is used, a ? should be printed. If none of the encodings can be valid, X must be printed.

Furthermore, the following should be taken into account:

  • If the input is valid ASCII, ASCII should be printed. Even though this means the other encodings are also valid. When the input is not ASCII, ? should printed in case of any ambiguity.
  • The UTF's are not required to start with a BOM.
  • UTF-16 can be either big or little endian. This means there are actually five different encodings to detect.
  • If UTF-8 or UTF-16 data encodes a number between 0 and 0x10FFFF you may consider it valid, even if it represents a Unicode code point that is technically not allowed in this encoding. Therefore you do not have to worry about unassigned code points and such.
  • ISO 8859-1 is considered invalid when any of its undefined bytes (positions 0-31 and 127-159, except 9, 10 and 13) are used.
  • This is code golf.

Happy golfing!

share|improve this question
4  
Needs test cases which give each possible output. In particular, I challenge you to provide a test case which is unambiguously UTF-8. You also need to specify how to handle a stream which could be ASCII or UTF-16. – Peter Taylor Jun 7 '12 at 22:39
2  
Presumably the idea is to identify it as the most restrictive encoding that it adheres to. Thus, If a stream contains only 7-bit chars, it's ASCII. Otherwise, if the stream consists entirely of valid UTF-8 sequences, it's UTF-8. Otherwise, if it's a valid UTF-16LE or UTF-16BE sequence, then identify it as such. ISO 8859-1 is then the final fallback. The question I have is, is the program expected to recognize unassigned Unicode code points as a way to invalidate a stream? Or is it enough to just recognize if they're in the valid range? I certainly hope it's the latter. – breadbox Jun 8 '12 at 1:31
@breadbox: you don't have to take unassigned code points into account, it should just be a valid range between U+000001 and 0x10FFFF. I disallowed NULL-characters to make it easier to distinguish between UTF-16 and the rest. – AardvarkSoup Jun 8 '12 at 11:34
There are 16 cases. You have specified the output for 12 of them. None of your conditions match a string which is valid ASCII and valid UTF-16. I believe that 2 of the 4 unspecified cases are actually possible. – Peter Taylor Jun 8 '12 at 11:44
2  
\n and \t aren't valid ISO 8859-1? I used them all the time! – user unknown Jun 8 '12 at 13:14
show 5 more comments

1 Answer

up vote 0 down vote

Tcl, 247

set i [read [open f rb]];foreach n\ d {ascii ASCII utf-8 UTF-8 unicode UTF-16 iso8859-1 ISO\ 8859-1} {if {[encoding convertt $n [encoding convertf $n $i]]eq$i&&($n!="iso8895-1"||[regexp {[\1-\10\13\14\16-\37\177-\237]} $i])} {puts $d;exit}};puts X

Reads the input from the file f in the current directory.

It works by decoding and re-encoding the input into the different charsets and check if the input and the re-encoded input is equal.

I believe that valid UTF-8 is also valid (but useless) ISO 8859-1, so I print UTF-8 then.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.