# Corrupting A Binary File By Interpreting It As Text

Using FTP's ASCII mode to transfer binary files is a bad idea. It is likely to corrupt your data. Badly. Let's simulate this corruption!

Your task is to write a program which takes a file as input and outputs the corrupted version of the file. Because how exactly your file gets corrupted is dependent on both the FTP client and server and their systems/implementations, for this challenge, we will use the following rules for corruption, in this order:

• Clear the high bit in each byte of the file. (Masking with 0x7F is an option here)
• When encountering escape (0x1b) followed by [, delete from the escape character through the next letter character (A-Z and a-z), inclusive.
• Escape followed by any other character should remove both the escape and the character following it.
• All control codes (characters in the range 0x00-0x1F, inclusive, and 0x7F) other than tab (0x09) and linefeed (0x0a) should be removed.

## Other rules

• Input and output can be in any convenient format. You do not need to use actual files.
• Shortest code wins, of course.
• Standard rules and loopholes apply.
• How about 0x9B? Should it be interpreted as 0x1B once the most significant bit has been cleared? (I think the answer is yes, according to the description.) – Arnauld Dec 26 '18 at 16:27
• @Arnauld yes, that would be correct. We're pretending to be a really dumb FTP server that just pretends the high bit doesn't exist. – Beefster Dec 26 '18 at 16:30
• Could we have a test-case? – wizzwizz4 Dec 26 '18 at 16:52
• @wizzwizz4 I was thinking you could use just about any binary file, though I suppose the escape sequences need test cases. – Beefster Dec 26 '18 at 16:56
• @Beefster You can use just about any binary file, but we need to know what the expected output should be. – wizzwizz4 Dec 26 '18 at 16:57

# Perl 5, 62 53 bytes

y/\x80-\xff/\0-\c?/;s/\e(\[.*?[a-z]|.)|[^\t\n -~]//gi


Try it online! No, I don't know how to input nonprinting characters, particularly illegal UTF-8 sequences, into TIO. Edit: Saved 10 bytes thanks to @NahuelFouilleul.

• \c[ could be changed by \e and \ck by \v, \x7f is missing also /g modifier – Nahuel Fouilleul Dec 28 '18 at 10:34
• sorry seems \v doesnt work in character range (indeed \v matches also \f, \r and \n), but [^\t\n -~] is shorter than [\0-\b\ck-\c_\c?] – Nahuel Fouilleul Dec 28 '18 at 10:48
• @NahuelFouilleul Ah, of course, that's much nicer, thanks! – Neil Dec 28 '18 at 10:51

## x86 assembly, 85 bytes

Bytecode:

8d0c244289deb00331dbcd803c0175418024247f8a042401f375203c1b0f94c374e23c09740c3c0a
74083c1f7ed83c7f74d4b00431db43cd80ebcb3c5b75014380e3fe0c203c617cbb3c7a7fb731dbeb
b3b001cd80


Source (with explanation in comments):

.section .text
.globl _start

_start:
leal (%esp), %ecx  # read to (%esp)
incl %edx          # read one byte

loop:
movl %ebx, %esi    # save previous %ebx
loopnosave:
movb $3, %al # read xorl %ebx, %ebx # stdin int$0x80

cmpb $1, %al # exit if we didn't get exactly one byte jne quit andb$0x7f, (%esp)
movb (%esp), %al   # moving byte into %al saves bytes later on

addl %esi, %ebx    # restore previous value of %ebx (currently 0)
# we use add instead of mov here to update flags
jnz ignore         # check if we're in the middle of an esc-sequence

cmpb $0x1b, %al # check for esc sete %bl # if it is esc, set the "ignore" flag... je loop # ... and skip the output cmpb$0x09, %al    # explicitly output tabs and newlines
je out
cmpb $0x0a, %al je out cmpb$0x1f, %al    # don't output control characters
jle loopnosave
cmpb $0x7f, %al je loopnosave out: movb$4, %al       # write
xorl %ebx, %ebx
incl %ebx          # stdout
int $0x80 jmp loopnosave # we clobbered %ebx, so we don't want to keep it ignore: cmpb$'[', %al     # check if it's [
jne ok             # skip next instruction if not
incl %ebx          # set %ebx to anything but 1
ok:
andb $0xfe, %bl # clear lowest bit on %ebx orb$0x20, %al     # uppercase to lowercase
cmpb $'a', %al jl loop cmpb$'z', %al
jg loop
xorl %ebx, %ebx    # clear ignore flag if letter found
jmp loop

quit:
movb $1, %al int$0x80


Try it online! (with testcases)

Overview / general explanation:

• The program reads one byte at a time. %al holds the input byte, %ebx holds a flag that gets set when an ESC character is encountered. Since we need %ebx to call sys_read and sys_write, %esi is used as a temporary storage space for its previous value.

• The %ebx flag is 0 ordinarily, 1 if the previous character was ESC, and 2 or greater if an ESC[ was found. On every loop, the program first tests if the flag is zero. If it is, we either set the flag (if the next character is ESC) or print the next character and move on.

If it's nonzero, we don't print the character, but instead unset the least significant bit of the flag. This has the effect of zeroing it if it's 1, but it will remain nonzero if it's any other positive integer (signifying a preceding ESC[). Then we set it to zero (by xoring it with itself) if the current character is a letter, terminating the ESC[ sequence.

• One trick I'd like to point out: it's a little miraculously convenient that the file descriptor for STDIN happens to be 0. This allows us to postpone moving %esi back to %ebx until the first time we have to check the flag, which happens to be checking whether it's zero. A movl instruction would not update flags, but addl does, and since %ebx was previously set to zero to read from STDIN, the add instruction happens to be equivalent.

• Uses many of the same tricks as in my cat program.

• providing a hexdump would be in the spirit of the question :) – qwr Dec 28 '18 at 20:37

## 16/32-bit x86 assembly code, 47 bytes

Bytes:

AC247F3C7F74253C0974203C0A741C3C1B7514497418AC3C5B7511497410AC24DF2C403C1A77F43C1F7601AAE2D2C3

Disassembly:

l1: lodsb ;fetch a byte
and   al, 7fh    ;clear bit 7
cmp   al, 7fh
je    l5         ;skip 0x7F
cmp   al, 9
je    l4         ;accept 0x09
cmp   al, 0ah
je    l4         ;and 0x0A
cmp   al, 1bh
jne   l3         ;branch if not ESC
dec   ecx
je    l6         ;exit if out of characters
lodsb            ;fetch a byte
cmp   al, '['
jne   l5         ;skip if not '['
l2: dec   ecx
je    l6         ;exit if out of characters
lodsb            ;fetch a byte
and   al, 0dfh   ;clear bit 6 so lowercase -> uppercase
sub   al, 40h    ;convert potential uppercase to 0x00-0x1A range
;(lesser values will wrap around)
cmp   al, 1ah    ;check for 'Z'
jnbe  l2         ;loop if not less or equal
l3: cmp   al, 1fh
jbe   l5         ;ignore all other values less or equal to 0x1F
l4: stosb            ;store surviving character
l5: loop  l1         ;loop while characters remain
l6: ret

Input: (E)SI is pointer to input text, (E)DI is pointer to output buffer, (E)CX is length of input.
Replace ECX with CX for 16-bit code. Code size is not affected.

# Clean, 164 bytes

Escape sequences counted as one byte each, since they're valid code but not valid UTF-8
(so TIO and SE don't like them)

import StdEnv
?[]=[]
?['\33':t]= ?(drop 1((if(t%(0,0)==['['])(dropWhile(not o isAlpha))id)t))
?[h:t]|h<' '&&h<>'	'&&h<>'
'||h>'\176'= ?t=[h: ?t]

?o map(\e|e<'\177'=e=e-'\200')


Try it online!

## Python 3, 186100 84 bytes

lambda s:re.sub(b"␛\[.*?[A-Za-z]|␛.|[^ -~\n␉]",b"",bytes(b&127for b in s))
import re


Applies each filter in turn, with filters 2, 3 and 4 being implemented by a regular expression. (Diagram not to scale.)

-15 bytes thanks to Erik the Outgolfer

Try it online!

## Test cases

I assume that my program's doing the right thing here.

1. f(b"ab\x1b[cde\x1b[;'3GH\x1bIJK\x84\xa2\r\n\t") => b'abdeHJK"\n\t'

• the TIO linked is 85 bytes, not 84? – FlipTack Dec 28 '18 at 23:31
• @FlipTack Whoops! Fixed. – wizzwizz4 Dec 29 '18 at 9:50
• Out of curiosity, why is the import at the end? – Solomon Ucko Dec 30 '18 at 2:29
• @SolomonUcko That was Erik's idea. Since I had an f= in the middle, and the policy is to allow programs to be preceded by f= if they're using a lambda, restructuring the source code so that that was possible saved 2 bytes. – wizzwizz4 Dec 30 '18 at 9:45
• I see, that makes sense. I thought you were allowed to use the last expression evaluated, but maybe not. – Solomon Ucko Dec 30 '18 at 14:29

# C (gcc), 116 112 bytes

This program assumes that a second ESC in a row matches as a non-[ (e.g. "ESC ESC [0a" -> "[0a".) So that I can send NUL characters to the function, I provide a counted string to it instead of a standard string.

Thanks to @ceilingcat for the suggestions.

f(t,c,i,s)char*t;{for(s=0;i=*t++&127,c--;)s?s=2*!(s-1?isalpha(i):i-91):i-27?i-9u<2|i>31&&i<127&&putchar(i):s++;}


Try it online!