Python
Encoding requires numpy, SciPy and scikit-image.
Decoding requires only PIL.
This is a method based on superpixel interpolation. To begin, each image is divided into 70 similar sized regions of similar color. For example, the landscape picture is divided in the following manner:
The centroid of each region is located (to the nearest raster point on a grid containing no more than 402 points), as well as it's average color (from a 216 color palette), and each of these regions is encoded as a number from 0 to 86832, capable of being stored in 2.5 printable ascii characters (actually 2.497, leaving just enough room to encode for a greyscale bit).
If you're attentive, you may have noticed that 140 / 2.5 = 56 regions, and not 70 as I stated earlier. Notice, however, that each of these regions is a unique, comparable object, which may be listed in any order. Because of this, we can use the permutation of the first 56 regions to encode for the other 14, as well as having a few bits left over to store the aspect ratio.
More specifically, each of the additional 14 regions is converted to a number, and then each of these numbers concatenated together (multiplying the current value by 86832, and adding the next). This (gigantic) number is then converted to a permutation on 56 objects.
For example:
from my_geom import *
# this can be any value from 0 to 56!, and it will map unambiguously to a permutation
num = 595132299344106583056657556772129922314933943196204990085065194829854239
perm = num2perm(num, 56)
print perm
print perm2num(perm)
will output:
[0, 3, 33, 13, 26, 22, 54, 12, 53, 47, 8, 39, 19, 51, 18, 27, 1, 41, 50, 20, 5, 29, 46, 9, 42, 23, 4, 37, 21, 49, 2, 6, 55, 52, 36, 7, 43, 11, 30, 10, 34, 44, 24, 45, 32, 28, 17, 35, 15, 25, 48, 40, 38, 31, 16, 14]
595132299344106583056657556772129922314933943196204990085065194829854239
The resulting permutation is then applied to the original 56 regions. The original number (and thus the additional 14 regions) can likewise be extracted by converting the permutation of the 56 encoded regions into its numerical representation.
When the --greyscale
option is used with the encoder, 94 regions are used instead (separated 70, 24), with 558 raster points, and 16 shades of grey.
When decoding, each of these regions is treated as a 3D cone extended into infinity, with its vertex at the centroid of the region, as viewed from above (a.k.a. a Voronoi Diagram). The borders are then blended together to create the final product.
Future Improvements
The dimensions of the Mona Lisa are a bit off, due to the way I'm storing the aspect ratio. I'll need to use a different system. Fixed, by assuming that the original aspect ratio is somewhere between 1:21 and 21:1, which I think is a reasonable assumption.
The Hindenburg could be improved a lot. The color palette I'm using only has 6 shades of grey. If I introduced a greyscale-only mode, I could use the extra information to increase the color depth, number of regions, number of raster points, or any combination of the three. I've added a --greyscale
option to the encoder, which does all three.
2d Shapes would probably look better with blending turned off. I'll likely add a flag for that. Added an encoder option to control the segmentation ratio, and a decoder option to turn off blending.
- More fun with combinatorics. 56! is actually large enough to store 15 additional regions, and 15! is large enough to store 2 more for a grand total of 73. But wait, there's more! The partitioning of these 73 object could also be used to store more information. For example, there are 73 choose 56 ways to select the initial 56 regions, and then 17 choose 15 ways to select the next 15. A grand total of 2403922132944423072 partitionings, big enough to store 3 more regions for a total of 76. I'd need to come up with a clever way to uniquely number all partitions of 73 into groups of 56, 15, 2 ... and back. Perhaps not practical, but an interesting problem to think about.
0VW*`Gnyq;c1JBY}tj#rOcKm)v_Ac\S.r[>,Xd_(qT6 >]!xOfU9~0jmIMG{hcg-'*a.s<X]6*%U5>/FOze?cPv@hI)PjpK9\iA7P ]a-7eC&ttS[]K>NwN-^$T1E.1OH^c0^"J 4V9X
0Jc?NsbD#1WDuqT]AJFELu<!iE3d!BB>jOA'L|<j!lCWXkr:gCXuD=D\BL{gA\ 8#*RKQ*tv\\3V0j;_4|o7>{Xage-N85):Q/Hl4.t&'0pp)d|Ry+?|xrA6u&2E!Ls]i]T<~)58%RiA
and
4PV 9G7X|}>pC[Czd!5&rA5 Eo1Q\+m5t:r#;H65NIggfkw'h4*gs.:~<bt'VuVL7V8Ed5{`ft7e>HMHrVVUXc.{#7A|#PBm,i>1B781.K8>s(yUV?a<*!mC@9p+Rgd<twZ.wuFnN dp
The second one encoded with the --greyscale
option.
3dVY3TY?9g+b7!5n`)l"Fg H$ 8n?[Q-4HE3.c:[pBBaH`5'MotAj%a4rIodYO.lp$h a94$n!M+Y?(eAR,@Y*LiKnz%s0rFpgnWy%!zV)?SuATmc~-ZQardp=?D5FWx;v=VA+]EJ(:%
Encoded with the --greyscale
option.
.9l% Ge<'_)3(`DTsH^eLn|l3.D_na,,sfcpnp{"|lSv<>}3b})%m2M)Ld{YUmf<Uill,*:QNGk,'f2; !2i88T:Yjqa8\Ktz4i@h2kHeC|9,P` v7Xzd Yp&z:'iLra&X&-b(g6vMq
Encoded with --ratio 60
, and decoded with --no-blending
options.
encoder.py
from __future__ import division
import argparse, numpy
from skimage.io import imread
from skimage.transform import resize
from skimage.segmentation import slic
from skimage.measure import regionprops
from my_geom import *
def encode(filename, seg_ratio, greyscale):
img = imread(filename)
height = len(img)
width = len(img[0])
ratio = width/height
if greyscale:
raster_size = 558
raster_ratio = 11
num_segs = 94
set1_len = 70
max_num = 8928 # 558 * 16
else:
raster_size = 402
raster_ratio = 13
num_segs = 70
set1_len = 56
max_num = 86832 # 402 * 216
raster_width = (raster_size*ratio)**0.5
raster_height = int(raster_width/ratio)
raster_width = int(raster_width)
resize_height = raster_height * raster_ratio
resize_width = raster_width * raster_ratio
img = resize(img, (resize_height, resize_width))
segs = slic(img, n_segments=num_segs-4, ratio=seg_ratio).astype('int16')
max_label = segs.max()
numpy.place(segs, segs==0, [max_label+1])
regions = [None]*(max_label+2)
for props in regionprops(segs):
label = props['Label']
props['Greyscale'] = greyscale
regions[label] = Region(props)
for i, a in enumerate(regions):
for j, b in enumerate(regions):
if a==None or b==None or a==b: continue
if a.centroid == b.centroid:
numpy.place(segs, segs==j, [i])
regions[j] = None
for y in range(resize_height):
for x in range(resize_width):
label = segs[y][x]
regions[label].add_point(img[y][x])
regions = [r for r in regions if r != None]
if len(regions)>num_segs:
regions = sorted(regions, key=lambda r: r.area)[-num_segs:]
regions = sorted(regions, key=lambda r: r.to_num(raster_width))
set1, set2 = regions[-set1_len:], regions[:-set1_len]
set2_num = 0
for s in set2:
set2_num *= max_num
set2_num += s.to_num(raster_width)
set2_num = ((set2_num*85 + raster_width)*85 + raster_height)*25 + len(set2)
perm = num2perm(set2_num, set1_len)
set1 = permute(set1, perm)
outnum = 0
for r in set1:
outnum *= max_num
outnum += r.to_num(raster_width)
outnum *= 2
outnum += greyscale
outstr = ''
for i in range(140):
outstr = chr(32 + outnum%95) + outstr
outnum //= 95
print outstr
parser = argparse.ArgumentParser(description='Encodes an image into a tweetable format.')
parser.add_argument('filename', type=str,
help='The filename of the image to encode.')
parser.add_argument('--ratio', dest='seg_ratio', type=float, default=30,
help='The segmentation ratio. Higher values (50+) will result in more regular shapes, lower values in more regular region color.')
parser.add_argument('--greyscale', dest='greyscale', action='store_true',
help='Encode the image as greyscale.')
args = parser.parse_args()
encode(args.filename, args.seg_ratio, args.greyscale)
decoder.py
from __future__ import division
import argparse
from PIL import Image, ImageDraw, ImageChops, ImageFilter
from my_geom import *
def decode(instr, no_blending=False):
innum = 0
for c in instr:
innum *= 95
innum += ord(c) - 32
greyscale = innum%2
innum //= 2
if greyscale:
max_num = 8928
set1_len = 70
image_mode = 'L'
default_color = 0
raster_ratio = 11
else:
max_num = 86832
set1_len = 56
image_mode = 'RGB'
default_color = (0, 0, 0)
raster_ratio = 13
nums = []
for i in range(set1_len):
nums = [innum%max_num] + nums
innum //= max_num
set2_num = perm2num(nums)
set2_len = set2_num%25
set2_num //= 25
raster_height = set2_num%85
set2_num //= 85
raster_width = set2_num%85
set2_num //= 85
resize_width = raster_width*raster_ratio
resize_height = raster_height*raster_ratio
for i in range(set2_len):
nums += set2_num%max_num,
set2_num //= max_num
regions = []
for num in nums:
r = Region()
r.from_num(num, raster_width, greyscale)
regions += r,
masks = []
outimage = Image.new(image_mode, (resize_width, resize_height), default_color)
for a in regions:
mask = Image.new('L', (resize_width, resize_height), 255)
for b in regions:
if a==b: continue
submask = Image.new('L', (resize_width, resize_height), 0)
poly = a.centroid.bisected_poly(b.centroid, resize_width, resize_height)
ImageDraw.Draw(submask).polygon(poly, fill=255, outline=255)
mask = ImageChops.multiply(mask, submask)
outimage.paste(a.avg_color, mask=mask)
if not no_blending:
outimage = outimage.resize((raster_width, raster_height), Image.ANTIALIAS)
outimage = outimage.resize((resize_width, resize_height), Image.BICUBIC)
smooth = ImageFilter.Kernel((3,3),(1,2,1,2,4,2,1,2,1))
for i in range(20):outimage = outimage.filter(smooth)
outimage.show()
parser = argparse.ArgumentParser(description='Decodes a tweet into and image.')
parser.add_argument('--no-blending', dest='no_blending', action='store_true',
help="Do not blend the borders in the final image.")
args = parser.parse_args()
instr = raw_input()
decode(instr, args.no_blending)
my_geom.py
from __future__ import division
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
self.xy = (x, y)
def __eq__(self, other):
return self.x == other.x and self.y == other.y
def __lt__(self, other):
return self.y < other.y or (self.y == other.y and self.x < other.x)
def inv_slope(self, other):
return (other.x - self.x)/(self.y - other.y)
def midpoint(self, other):
return Point((self.x + other.x)/2, (self.y + other.y)/2)
def dist2(self, other):
dx = self.x - other.x
dy = self.y - other.y
return dx*dx + dy*dy
def bisected_poly(self, other, resize_width, resize_height):
midpoint = self.midpoint(other)
points = []
if self.y == other.y:
points += (midpoint.x, 0), (midpoint.x, resize_height)
if self.x < midpoint.x:
points += (0, resize_height), (0, 0)
else:
points += (resize_width, resize_height), (resize_width, 0)
return points
elif self.x == other.x:
points += (0, midpoint.y), (resize_width, midpoint.y)
if self.y < midpoint.y:
points += (resize_width, 0), (0, 0)
else:
points += (resize_width, resize_height), (0, resize_height)
return points
slope = self.inv_slope(other)
y_intercept = midpoint.y - slope*midpoint.x
if self.y > midpoint.y:
points += ((resize_height - y_intercept)/slope, resize_height),
if slope < 0:
points += (resize_width, slope*resize_width + y_intercept), (resize_width, resize_height)
else:
points += (0, y_intercept), (0, resize_height)
else:
points += (-y_intercept/slope, 0),
if slope < 0:
points += (0, y_intercept), (0, 0)
else:
points += (resize_width, slope*resize_width + y_intercept), (resize_width, 0)
return points
class Region:
def __init__(self, props={}):
if props:
self.greyscale = props['Greyscale']
self.area = props['Area']
cy, cx = props['Centroid']
if self.greyscale:
self.centroid = Point(int(cx/11)*11+5, int(cy/11)*11+5)
else:
self.centroid = Point(int(cx/13)*13+6, int(cy/13)*13+6)
self.num_pixels = 0
self.r_total = 0
self.g_total = 0
self.b_total = 0
def __lt__(self, other):
return self.centroid < other.centroid
def add_point(self, rgb):
r, g, b = rgb
self.r_total += r
self.g_total += g
self.b_total += b
self.num_pixels += 1
if self.greyscale:
self.avg_color = int((3.2*self.r_total + 10.7*self.g_total + 1.1*self.b_total)/self.num_pixels + 0.5)*17
else:
self.avg_color = (
int(5*self.r_total/self.num_pixels + 0.5)*51,
int(5*self.g_total/self.num_pixels + 0.5)*51,
int(5*self.b_total/self.num_pixels + 0.5)*51)
def to_num(self, raster_width):
if self.greyscale:
raster_x = int((self.centroid.x - 5)/11)
raster_y = int((self.centroid.y - 5)/11)
return (raster_y*raster_width + raster_x)*16 + self.avg_color//17
else:
r, g, b = self.avg_color
r //= 51
g //= 51
b //= 51
raster_x = int((self.centroid.x - 6)/13)
raster_y = int((self.centroid.y - 6)/13)
return (raster_y*raster_width + raster_x)*216 + r*36 + g*6 + b
def from_num(self, num, raster_width, greyscale):
self.greyscale = greyscale
if greyscale:
self.avg_color = num%16*17
num //= 16
raster_x, raster_y = num%raster_width, num//raster_width
self.centroid = Point(raster_x*11 + 5, raster_y*11+5)
else:
rgb = num%216
r, g, b = rgb//36, rgb//6%6, rgb%6
self.avg_color = (r*51, g*51, b*51)
num //= 216
raster_x, raster_y = num%raster_width, num//raster_width
self.centroid = Point(raster_x*13 + 6, raster_y*13 + 6)
def perm2num(perm):
num = 0
size = len(perm)
for i in range(size):
num *= size-i
for j in range(i, size): num += perm[j]<perm[i]
return num
def num2perm(num, size):
perm = [0]*size
for i in range(size-1, -1, -1):
perm[i] = int(num%(size-i))
num //= size-i
for j in range(i+1, size): perm[j] += perm[j] >= perm[i]
return perm
def permute(arr, perm):
size = len(arr)
out = [0] * size
for i in range(size):
val = perm[i]
out[i] = arr[val]
return out