Skip to main content
replaced https://tools.ietf.org/html/rfc with https://www.rfc-editor.org/rfc/rfc
Source Link

So if we can find a fixpoint in DEFLATEDEFLATE compression, such that x = zlib.decompress(x, -zlib.MAX_WBITS), we are done. But how to do this?

The way these blocks are actually encoded as bytes is a little complex. I'll annotate the block, and to fill in the gaps read the RFCRFC. For compressed blocks, the data is turned from bits into bytes LSB to MSB with a couple of exceptions.

So if we can find a fixpoint in DEFLATE compression, such that x = zlib.decompress(x, -zlib.MAX_WBITS), we are done. But how to do this?

The way these blocks are actually encoded as bytes is a little complex. I'll annotate the block, and to fill in the gaps read the RFC. For compressed blocks, the data is turned from bits into bytes LSB to MSB with a couple of exceptions.

So if we can find a fixpoint in DEFLATE compression, such that x = zlib.decompress(x, -zlib.MAX_WBITS), we are done. But how to do this?

The way these blocks are actually encoded as bytes is a little complex. I'll annotate the block, and to fill in the gaps read the RFC. For compressed blocks, the data is turned from bits into bytes LSB to MSB with a couple of exceptions.

edited body
Source Link
Sisyphus
  • 15k
  • 3
  • 45
  • 88
Input            | Output
P0               | 
P0               |
P0               | 
P4 P0 P0 P0 P4   | P0 P0 P0 P4
R4               | P0 P0 P0 P4
P4 R4 P4 R4 P4   | R4 P4 R4 P4
R4               | R4 P4 R4 P4
P*4P4 P0 P0 P0 R*1  | P0 P0 P0 R*1
R*1              | R*1
Input            | Output
P0               | 
P0               |
P0               | 
P4 P0 P0 P0 P4   | P0 P0 P0 P4
R4               | P0 P0 P0 P4
P4 R4 P4 R4 P4   | R4 P4 R4 P4
R4               | R4 P4 R4 P4
P*4 P0 P0 P0 R*1 | P0 P0 P0 R*1
R*1              | R*1
Input            | Output
P0               | 
P0               |
P0               | 
P4 P0 P0 P0 P4   | P0 P0 P0 P4
R4               | P0 P0 P0 P4
P4 R4 P4 R4 P4   | R4 P4 R4 P4
R4               | R4 P4 R4 P4
P4 P0 P0 P0 R*1  | P0 P0 P0 R*1
R*1              | R*1
Source Link
Sisyphus
  • 15k
  • 3
  • 45
  • 88

Bubblegum, 105 Bytes

Hexdump:

00000000: 0000 00ff ff00 0000 ffff 0000 00ff ff00  ................
00000010: 1400 ebff 0000 00ff ff00 0000 ffff 0000  ................
00000020: 00ff ff00 1400 ebff 4288 21c4 0000 1400  ........B.!.....
00000030: ebff 4288 21c4 0000 1400 ebff 4288 21c4  ..B.!.......B.!.
00000040: 0000 1400 ebff 4288 21c4 0000 1400 ebff  ......B.!.......
00000050: 0000 00ff ff00 0000 ffff 0000 00ff ff03  ................
00000060: 1300 0000 0313 0000 00                   .........

Try it online!

(You might want to verify it offline - since the input is hexdump and the output is raw.)


This relies on the fact that Bubbleugum tries to DEFLATE decode its input first:

...
o = zlib.decompress(code, -zlib.MAX_WBITS)
...

So if we can find a fixpoint in DEFLATE compression, such that x = zlib.decompress(x, -zlib.MAX_WBITS), we are done. But how to do this?

Part I: Generic Compression Quine

Say we have a compression programming 'language' that has two operations:

  • Pn: Print the following n tokens as literals, and skip interpreting them
  • Rn: Print the last n tokens printed

Let's write some simple programs in this to understand how it works.

Input   | Output
P1 P0   | P0
Input   | Output
P1 P0   | P0
P1 P1   | P1
Input   | Output
P1 P0   | P0
R1      | P0
Input          | Output
P4 P0 P0 P0 P0 | P0 P0 P0 P0
R4             | P0 P0 P0 P0

Now the question is: Just with these two instructions, can we create a quine? The answer is yes, thanks to Russ Cox:

Input          | Output
P0             | 
P0             |
P0             |
P4 P0 P0 P0 P4 | P0 P0 P0 P4
R4             | P0 P0 P0 P4
P4 R4 P4 R4 P4 | R4 P4 R4 P4
R4             | R4 P4 R4 P4
P4 P0 P0 P0 P0 | P0 P0 P0 P0

(The tokens are not on the same line, but you can check they're the same).

This gives us hope we might be able to write a DEFLATE quine. But we're not close to done yet, since we have to deal with actual file formats and not made up tokens. Read on!

Part II: Zlib and DEFLATE

Zlib usually appends a 2 byte header and a 4 byte checksum to everything it compresses. The 4 byte checksum would make the creation of a quine much more difficult. But luckily, Bubblegum is designed using to utilize the -zlib.MAX_WBITS flag, which skips the header and the checksum! So we just have a raw DEFLATE stream. How does DEFLATE work? The full thing can be a bit complicated, but luckily we only need to pull out the bits that allow us to have our Pn and Rn building blocks.

Part III: The Pn building block

A deflate stream is made up of a series of blocks. Each block starts with the following:

  • BFINAL: 1 bit, set to 1 if it's the last block.
  • BTYPE: 2 bits. All we need to know is that it's 00 for 'no compression' (ie Pn) and 01 for 'fixed compression' (which turns out to map to Rn).

If we have a 'no compression' block, the rest of the bits in the current byte are set to zero and the next bytes look like:

+---+---+---+---+================================+
|  LEN  | NLEN  |... LEN bytes of literal data...|
+---+---+---+---+================================+

Where LEN is a 2-byte little endian unsigned number of bytes in the literal data, NLEN is the complement of LEN (also unsigned little endian) and we then have N literal bytes. Keeping in mind the first byte is packed from LSB to MSB, this means we can encode the following:

P0 = 00 00 00 ff ff
00000 00 0 | 00000000 | 00000000 | 11111111 | 11111111
^     ^  ^   ^^^^^^^^^^^^^^^^^^^   ^^^^^^^^^^^^^^^^^^^
|     |  |   LEN = 0x0000          NLEN = ~LEN = 0xFFFF
|     |  |
|     |  \- BFINAL = 0 (not final block)
|     \---- BTYPE = 00 (no compression)
\---------- 5 bits padding in block
P4 = 00 14 00 eb ff
00000 00 0 | 00010100 | 00000000 | 11101011 | 11111111
^     ^  ^   ^^^^^^^^^^^^^^^^^^^   ^^^^^^^^^^^^^^^^^^^
|     |  |   LEN = 0x0014          NLEN = ~LEN = 0xFFEB
|     |  |
|     |  \- BFINAL = 0 (not final block)
|     \---- BTYPE = 00 (no compression)
\---------- 5 bits padding in block

Why is P4 printing 0x14 = 20 bytes, you ask, instead of 4? Well, the previous token 'quine' had the units of 1 byte ~ 1 token, but we don't have that luxury. So instead, we have a fixed length of 5 bytes per token, since this is the minimum size of a print token. So 4 tokens is 20 bytes.

Part IV: The Rn building block

The BTYPE = 01 allows us to make queries of the form REPEAT(n, q):

Starting from q bytes away in the output, print n bytes.

It shouldn't be hard to see that REPEAT(n, n) gives us Rn. But there's a problem, since it turns out that R4 = REPEAT(20, 20) only takes up 3 bytes instead of 5! Since we are assuming all our tokens take up 5 bytes for our quine to work, this is no good. However, we can introduce some redundancy - it turns out if we define R4 = REPEAT(10, 20), REPEAT(10, 20), then we do the same thing but now the instruction takes up 5 bytes total!

The way these blocks are actually encoded as bytes is a little complex. I'll annotate the block, and to fill in the gaps read the RFC. For compressed blocks, the data is turned from bits into bytes LSB to MSB with a couple of exceptions.

P4 = 42 88 21 c4 00
01000 01 0 | 1 00010 00 | 001000 01 | 11 00010 0 | 0000000 0
^     ^  ^   ^  ^    ^    ^      ^    ^   ^    ^   ^       ^
|     |  |  [5] |    |    |      |    [8] |    |   padding [8]
[3]   |  \- [1] [4] [3]  [6]    [5]      [7]  [6]
      \---- [2]

[1]: BFINAL: 0 (not end block)
[2]: BTYPE: 01 (fixed compression) 
[3]: Literal code 264 (print 10 bytes...)
[4]: Distance code 8 (starting from 17 + ... )
[5]: Extra distance code bits ( ... 3 bytes back) (= 20 total)
[6]: Literal code 264 (print 10 bytes...)
[7]: Distance code 8 (starting from 17 + ... )
[8]: Extra distance code bits ( ... 3 bytes back) (= 20 total)

So we've got all our building blocks! P0, P4, R4 right? Are we done?

Part V: The final tweak

Well, not so fast. Remember we had a bit saying which block was the end block? It turns out, for Python at least, that we need to include this on the last block, else it messes up our program. And unfortunately, if we let P*0 be a P0 end block token, the following is NOT a quine:

Input           | Output
P0              | 
P0              |
P0              | 
P4 P0 P0 P0 P4  | P0 P0 P0 P4
R4              | P0 P0 P0 P4
P4 R4 P4 R4 P4  | R4 P4 R4 P4
R4              | R4 P4 R4 P4 <-\
P*4 P0 P0 P0 P0 | P0 P0 P0 P0   |
^                               |
\--------------+----------------+
               |
          Not the same!

However, if we introduce an R*1, we can fix this quite easily:

Input            | Output
P0               | 
P0               |
P0               | 
P4 P0 P0 P0 P4   | P0 P0 P0 P4
R4               | P0 P0 P0 P4
P4 R4 P4 R4 P4   | R4 P4 R4 P4
R4               | R4 P4 R4 P4
P*4 P0 P0 P0 R*1 | P0 P0 P0 R*1
R*1              | R*1

It turns out we can encode R*1 = 03 13 00 00 00, so we are done. Use the following Python program to assemble and verify our DEFLATE quine:

import zlib

P0 = b'\x00\x00\x00\xff\xff'
P4 = b'\x00\x14\x00\xeb\xff'
R4 = b'B\x88!\xc4\x00'
R1_F = b'\x03\x13\x00\x00\x00'

comp = b''
comp += P0
comp += P0
comp += P0
comp += P4 + P0 + P0 + P0 + P4
comp += R4
comp += P4 + R4 + P4 + R4 + P4
comp += R4
comp += P4 + P0 + P0 + P0 + R1_F
comp += R1_F

print(zlib.decompress(comp, -zlib.MAX_WBITS) == comp)

Well done! You are now a certified deflate quine expert™.