Hexdump:
00000000: 0000 00ff ff00 0000 ffff 0000 00ff ff00 ................
00000010: 1400 ebff 0000 00ff ff00 0000 ffff 0000 ................
00000020: 00ff ff00 1400 ebff 4288 21c4 0000 1400 ........B.!.....
00000030: ebff 4288 21c4 0000 1400 ebff 4288 21c4 ..B.!.......B.!.
00000040: 0000 1400 ebff 4288 21c4 0000 1400 ebff ......B.!.......
00000050: 0000 00ff ff00 0000 ffff 0000 00ff ff03 ................
00000060: 1300 0000 0313 0000 00 .........
Try it online!
(You might want to verify it offline - since the input is hexdump and the output is raw.)
This relies on the fact that Bubbleugum tries to DEFLATE decode its input first:
...
o = zlib.decompress(code, -zlib.MAX_WBITS)
...
So if we can find a fixpoint in DEFLATE compression, such that x = zlib.decompress(x, -zlib.MAX_WBITS)
, we are done. But how to do this?
Part I: Generic Compression Quine
Say we have a compression programming 'language' that has two operations:
Pn
: Print the following n
tokens as literals, and skip interpreting them
Rn
: Print the last n
tokens printed
Let's write some simple programs in this to understand how it works.
Input | Output
P1 P0 | P0
Input | Output
P1 P0 | P0
P1 P1 | P1
Input | Output
P1 P0 | P0
R1 | P0
Input | Output
P4 P0 P0 P0 P0 | P0 P0 P0 P0
R4 | P0 P0 P0 P0
Now the question is: Just with these two instructions, can we create a quine? The answer is yes, thanks to Russ Cox:
Input | Output
P0 |
P0 |
P0 |
P4 P0 P0 P0 P4 | P0 P0 P0 P4
R4 | P0 P0 P0 P4
P4 R4 P4 R4 P4 | R4 P4 R4 P4
R4 | R4 P4 R4 P4
P4 P0 P0 P0 P0 | P0 P0 P0 P0
(The tokens are not on the same line, but you can check they're the same).
This gives us hope we might be able to write a DEFLATE quine. But we're not close to done yet, since we have to deal with actual file formats and not made up tokens. Read on!
Part II: Zlib and DEFLATE
Zlib usually appends a 2 byte header and a 4 byte checksum to everything it compresses. The 4 byte checksum would make the creation of a quine much more difficult. But luckily, Bubblegum is designed using to utilize the -zlib.MAX_WBITS
flag, which skips the header and the checksum! So we just have a raw DEFLATE stream. How does DEFLATE work? The full thing can be a bit complicated, but luckily we only need to pull out the bits that allow us to have our Pn
and Rn
building blocks.
Part III: The Pn
building block
A deflate stream is made up of a series of blocks. Each block starts with the following:
BFINAL
: 1 bit, set to 1 if it's the last block.
BTYPE
: 2 bits. All we need to know is that it's 00
for 'no compression' (ie Pn
) and 01
for 'fixed compression' (which turns out to map to Rn
).
If we have a 'no compression' block, the rest of the bits in the current byte are set to zero and the next bytes look like:
+---+---+---+---+================================+
| LEN | NLEN |... LEN bytes of literal data...|
+---+---+---+---+================================+
Where LEN
is a 2-byte little endian unsigned number of bytes in the literal data, NLEN
is the complement of LEN (also unsigned little endian) and we then have N
literal bytes. Keeping in mind the first byte is packed from LSB to MSB, this means we can encode the following:
P0 = 00 00 00 ff ff
00000 00 0 | 00000000 | 00000000 | 11111111 | 11111111
^ ^ ^ ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
| | | LEN = 0x0000 NLEN = ~LEN = 0xFFFF
| | |
| | \- BFINAL = 0 (not final block)
| \---- BTYPE = 00 (no compression)
\---------- 5 bits padding in block
P4 = 00 14 00 eb ff
00000 00 0 | 00010100 | 00000000 | 11101011 | 11111111
^ ^ ^ ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
| | | LEN = 0x0014 NLEN = ~LEN = 0xFFEB
| | |
| | \- BFINAL = 0 (not final block)
| \---- BTYPE = 00 (no compression)
\---------- 5 bits padding in block
Why is P4
printing 0x14 = 20
bytes, you ask, instead of 4? Well, the previous token 'quine' had the units of 1 byte ~ 1 token, but we don't have that luxury. So instead, we have a fixed length of 5 bytes per token, since this is the minimum size of a print token. So 4 tokens is 20 bytes.
Part IV: The Rn
building block
The BTYPE = 01
allows us to make queries of the form REPEAT(n, q)
:
Starting from q
bytes away in the output, print n
bytes.
It shouldn't be hard to see that REPEAT(n, n)
gives us Rn
. But there's a problem, since it turns out that R4 = REPEAT(20, 20)
only takes up 3 bytes instead of 5! Since we are assuming all our tokens take up 5 bytes for our quine to work, this is no good. However, we can introduce some redundancy - it turns out if we define R4 = REPEAT(10, 20), REPEAT(10, 20)
, then we do the same thing but now the instruction takes up 5 bytes total!
The way these blocks are actually encoded as bytes is a little complex. I'll annotate the block, and to fill in the gaps read the RFC. For compressed blocks, the data is turned from bits into bytes LSB to MSB with a couple of exceptions.
P4 = 42 88 21 c4 00
01000 01 0 | 1 00010 00 | 001000 01 | 11 00010 0 | 0000000 0
^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^
| | | [5] | | | | [8] | | padding [8]
[3] | \- [1] [4] [3] [6] [5] [7] [6]
\---- [2]
[1]: BFINAL: 0 (not end block)
[2]: BTYPE: 01 (fixed compression)
[3]: Literal code 264 (print 10 bytes...)
[4]: Distance code 8 (starting from 17 + ... )
[5]: Extra distance code bits ( ... 3 bytes back) (= 20 total)
[6]: Literal code 264 (print 10 bytes...)
[7]: Distance code 8 (starting from 17 + ... )
[8]: Extra distance code bits ( ... 3 bytes back) (= 20 total)
So we've got all our building blocks! P0
, P4
, R4
right? Are we done?
Part V: The final tweak
Well, not so fast. Remember we had a bit saying which block was the end block? It turns out, for Python at least, that we need to include this on the last block, else it messes up our program. And unfortunately, if we let P*0
be a P0
end block token, the following is NOT a quine:
Input | Output
P0 |
P0 |
P0 |
P4 P0 P0 P0 P4 | P0 P0 P0 P4
R4 | P0 P0 P0 P4
P4 R4 P4 R4 P4 | R4 P4 R4 P4
R4 | R4 P4 R4 P4 <-\
P*4 P0 P0 P0 P0 | P0 P0 P0 P0 |
^ |
\--------------+----------------+
|
Not the same!
However, if we introduce an R*1
, we can fix this quite easily:
Input | Output
P0 |
P0 |
P0 |
P4 P0 P0 P0 P4 | P0 P0 P0 P4
R4 | P0 P0 P0 P4
P4 R4 P4 R4 P4 | R4 P4 R4 P4
R4 | R4 P4 R4 P4
P*4 P0 P0 P0 R*1 | P0 P0 P0 R*1
R*1 | R*1
It turns out we can encode R*1 = 03 13 00 00 00
, so we are done. Use the following Python program to assemble and verify our DEFLATE quine:
import zlib
P0 = b'\x00\x00\x00\xff\xff'
P4 = b'\x00\x14\x00\xeb\xff'
R4 = b'B\x88!\xc4\x00'
R1_F = b'\x03\x13\x00\x00\x00'
comp = b''
comp += P0
comp += P0
comp += P0
comp += P4 + P0 + P0 + P0 + P4
comp += R4
comp += P4 + R4 + P4 + R4 + P4
comp += R4
comp += P4 + P0 + P0 + P0 + R1_F
comp += R1_F
print(zlib.decompress(comp, -zlib.MAX_WBITS) == comp)
Well done! You are now a certified deflate quine expert™.