Challenge: compress co-occuring columns into cartesian product

This challenge is inspired by a problem I encountered recently out in the real-world, though ultimately I cheated and found a workaround.

Challenge

The input is a space-separated table of characters (each line is a row, and every line will have the same number of columns). Your challenge is to group together co-occurring column values and output a 'compressed' table, and there may be duplicate rows. Let's look at an example:

input:

A X I
B X I
A Y I
B X I
B Y I


output:

AB XY I


The output is parsed as the cartesian product between each column:

AB XY = A X, B X, A Y, B Y


The union of cartesian products of columns in each output row must exactly match the set of all unique input rows. In other words, no skipping input rows and no specifying input rows that don't exist!

For further exampling, note that if the third column contained anything other than I, this would be harder to compress:

input:

A X I
B Y I
A Y $B X I B Y I  output: A X I A Y$
B XY I


Rules

• no non-standard libraries or calls to high-level compression functions. The spirit of the question is to write the algorithm yourself!
• output must be in the same format as inputs (zipping your output files is cheating)
• no hard-coding the answers of each test file. Your program needs to be able to handle arbitrary (correct) inputs.

Scoring (lowest wins)

• Size of your program in bytes (you may count the name of the input as a single byte)
• Plus the size (in bytes) of all of your output files on the given tests.
• -10% bonus if you have the fastest program (as it runs on my machine). Speed will be measured as the average time on 10 randomly generated test cases using generate.py 6 8 4 6 5 (the same 10 files for each program of course)

Test Files

These files are generated with this python script. The answer keys aren't guaranteed to be the minimum compression - you may be able to do better!

• So... if a person takes the output generated by one of these programs, they should be able to recreate each unique row of data, and nothing more, correct? May 7, 2014 at 16:54
• And... do I need to use a similar compression form, or will any form work provided a person can follow an algorithm to generate each unique row of data and nothing more? E.g., can my output for your example 2 look like 3:AXI|Y\$;BXI|YI May 7, 2014 at 16:56
• Yes, all unique rows of input can be generated from the output (which is how the generate script works). And no you may not use a different format for the output. Standardization. May 7, 2014 at 17:05
• And "fastest program" on which benchmark file? The sum of the three given ones? May 7, 2014 at 17:14
• This problem is known as Boolean tensor factorization. Finding the minimum number of output rows needed (Boolean tensor rank) is known to be NP-hard.
– xnor
May 8, 2014 at 0:40

print open('testfile').read()