# UNIX Uniq Uknow?

## Challenge

You must write a program which takes a string and a flag as input and performs one of the following functions.

## Input

If your language does not support command line arguments, you may take the arguments as input via STDIN.

The flag will be two characters: a dash and an ASCII character such as -d or -c.

## Functions

The following is the list of functions which you must implement in your program. The title of each is the flag which corresponds to the function.

### Counting: -c

This counts the number of duplicate lines in the input file. For example, if the input is:

aa
aa
bb
bb
bb
xx


The output will be:

2 aa
3 bb
1 xx


Because there are two lines of aa, three lines of bb, and one line of xx. The output should be separated by newlines.

### Duplication: -d

This shows which lines are duplicates. For example, using the above example string (aa\naa\nbb\nbb\nbb\nxx), the output would be:

aa
bb


Because these are lines which have been duplicated. You should only output each line once.

### Duplication: -D

This is almost the same as -d except it outputs each line the number of times that it occurs in the input file. For example, using the example string the output will be:

aa
aa
bb
bb
bb


Because aa occurs twice and bb occurs three times.

### Unique: -u

Similar to above except it only prints unique lines. For example, using the example file, the output will be:

xx


Because xx is the only unique line. Also, due to xx being unique, you will only ever have to output this once.

## Example

Note that although the input string will not always be sorted, the output must be sorted alphabetically. For example, for the following input:

hello world
apple pie
zebra
dog
hello world
apple pie
apple pie
apple pie


The output for each of the flags would be:

-c

4 apple pie
1 dog
2 hello world
1 zebra


-d

apple pie
hello world


-D

apple pie
apple pie
apple pie
apple pie
hello world
hello world


-u

dog
zebra


## Rules

Your programs must not use the UNIX program uniq as, frankly, that is plain cheating.

All output should be to STDOUT.

The file will only ever contain lowercase, alphabetical characters, newlines and spaces:

abcdefghijklmnopqrstuvwxyz


The output should be sorted alphabetically.

In case there is any confusion, programs written on any operating system can enter. The challenge is not limited to UNIX.

As per PPCG standard, functions and anonymous functions are allowed, taking input via their arguments and outputting via a returned string.

## Winning

As this is , the shortest code in bytes wins.

• Would ["foo", "bar"], "-c" be considered a valid input? (array of strings + flag as separated string) – Arnauld Sep 5 '16 at 12:36
• @Arnauld Yes, it would – Beta Decay Sep 5 '16 at 12:43

## Perl, 92 + 4 (-sn flags) = 96 bytes

$h{$_}++}{$v=$h{$_},print$c?"$v$_":($D||$d)&&$v>1||$u&&$v<2?$_ x($d?1:$v):''for sort keys%h


This code needs to be in a file to run. And it needs -sn flags, as well as -M5.010 (free).
For instance :

$cat input.txt hello world apple pie zebra dog hello world apple pie apple pie apple pie$ cat prog.pl
$h{$_}++}{$v=$h{$_},print$c?"$v$_":($D||$d)&&$v>1||$u&&$v<2?$_ x($d?1:$v):''for sort keys%h
$perl -M5.010 -sn prog.pl -c < input.txt 4 apple pie 1 dog 2 hello world 1 zebra  ## GNU sed, 496 485 + 2(rn flags) = 487 bytes 1h;1b;$bb;2{:r;$!N;$!br;s/.*/echo "&"|sort/e};:b;x;/[dD]/bd;/c/bc;/u/!b;x;s/^/\n/
s/(\n.*)\1+//g;s/\n//p;b;:d;x;/^(.*)\n\1/!D;:l;x;/D/{x;P;x};x;s/^[^\n]*\n//
/^(.*)\n\1/bl;P;D;:c;s/$/#/;G;x;/^$/bp;s/[^\n]*\n?//;x;s/#(\n[^\n]*).*/\1/
/([^:\n]*)\n\1$/{s/\n[^\n]*$//;:i;s/9(@*:)/@\1/;ti;s/8(@*:)/9\1/;s/7(@*:)/8\1/
s/6(@*:)/7\1/;s/5(@*:)/6\1/;s/4(@*:)/5\1/;s/3(@*:)/4\1/;s/2(@*:)/3\1/;s/1(@*:)/2\1/
s/0(@*:)/1\1/;s/@+:/1&/;y/@/0/;bc};s/:/ /;s/[^\n]*$/1:&/;bc;:p;x;s/...//;s/..$//;s/:/ /gp


The code size is not going to compete, but it was fun emulating the uniq command. I wanted to add a simple selection sort to get rid of the sort call, thus solving it in pure sed only, but the code got too long already. The input is read entirely from STDIN, with the flag expected on the first line.

Run:

(echo "-c";cat input_file)|sed -rnf uniq_emulator.sed


Output:

4 apple pie
1 dog
2 hello world
1 zebra


Initially submitted code and explanation: (sections are separated by an extra newline)

# The main
# store the flag in hold space and start next cycle
1h;1b
# usually a cycle starts with each line read, but I also start it manually,
#so I need to skip some lines that should run only once
$bb 2{ # read the entire input into pattern space :r;$!N;$!br # executes a shell call to sort the pattern space s/.*/echo "&"|sort/e } # based on the flag, the appropriate jumps to "functions" are made :b;x;/[dD]/bd;/c/bc;/u/!b # Unique: -u # all duplicate lines are deleted and what remains is printed x;s/^/\n/;s/(\n.*)\1+//g s/\n//p;b # Duplication: -d and -D # if the first 2 lines are different, delete the first and start new cycle :d;x;/^(.*)\n\1/!D :l # if -D, print the first duplicate line x;/D/{x;P;x};x # otherwise (-d), delete the first duplicate line s/^[^\n]*\n// # loop condition, go back to :l if the first 2 lines are duplicate /^(.*)\n\1/bl # print only the first line and start a new cycle P;D # Counting: -c # in order to count line occurrences, I simulate reading the input again # In a loop I append all pattern space to hold space, delete first line of #pattern space and from hold space all lines except the first one appended. :c;s/$/#/;G
x;/^$/bp;s/[^\n]*\n?//;x;s/#(\n[^\n]*).*/\1/ # if the new line "read" is the same as the previous one, increment the #counter of that line (format is "counter:line"), delete and go back to :c /([^:\n]*)\n\1$/{s/\n[^\n]*$//;bi;:I;bc} # otherwise, stop the counter for the previous line, add a counter for the #current one and go back to :c s/:/ /;s/[^\n]*$/1:&/;bc
# when the loop ends, the hold space with all counts is printed
:p;x;s/...//;s/..$//;s/:/ /gp # Increment function that works by manually updating the changing digit(s) :i;s/9(@*:)/@\1/;ti s/8(@*:)/9\1/;s/7(@*:)/8\1/;s/6(@*:)/7\1/ s/5(@*:)/6\1/;s/4(@*:)/5\1/;s/3(@*:)/4\1/ s/2(@*:)/3\1/;s/1(@*:)/2\1/;s/0(@*:)/1\1/ s/@+:/1&/;y/@/0/;tI  • Can you save a few bytes by putting your increment function inline, instead of breaking it out? I only see it being used once, so you could delete the call (bi;:I) and return (tI) for 7 bytes. – Riley Sep 6 '16 at 18:07 • @Riley Yes, you're right. I pasted that increment f. from another program of mine and thought to leave it separate for easier explanation. I'll apply the change soon. Good catch and thanks for taking the time to read all that. – seshoumara Sep 6 '16 at 18:36 • I definitely didn't understand all of it, there is still a lot for me to learn. – Riley Sep 6 '16 at 18:37 • @Riley Yesterday I had no more edits available, today I changed it. Bytes count dropped by 8 (;tI), plus 1 I spotted during the edit. – seshoumara Sep 8 '16 at 5:29 # R, 128 bytes Using a different method than @Billywob and implemented as a unnamed function function(s,o){R=rle(sort(s));cat(switch(o,'-d'=R$v[R$l>1],'-D'=s[s%in%R$v[R$l>1]],'-c'=paste(R$l,R$v),'-u'=R$v[R$l<2]),sep=' ')}  Ungolfed function(s,o){ R=rle(sort(s)); # do the run length encoding for the sorted input cat( # output to STDOUT switch(o # switch on option to provide output lines ,'-d'=R$v[R$l>1] # RLE values with length > 1 ,'-D'=s[s%in%R$v[R$l>1]] # string values in RLE values with length > 1 ,'-c'=paste(R$l,R$v) # RLE length and values pasted together ,'-u'=R$v[R$l<2] # RLE values with length < 2, eg 1 ) ,sep=' ' # set separator to carriage return ) }  Test Run function(s,o){R=rle(sort(s));cat(switch(o,'-d'=R$v[R$l>1],'-D'=s[s%in%R$v[R$l>1]],'-c'=paste(R$l,R$v),'-u'=R$v[R$l<2]),sep=' ')} > s = c("hello world", "apple pie", "zebra", "dog", "hello world", "apple pie", "apple pie", "apple pie") > f = function(s,o){R=rle(sort(s));cat(switch(o,'-d'=R$v[R$l>1],'-D'=s[s%in%R$v[R$l>1]],'-c'=paste(R$l,R$v),'-u'=R$v[R$l<2]),sep=' ')} > f(s, "-c") 4 apple pie 1 dog 2 hello world 1 zebra > f(s, "-d") apple pie hello world > f(s, "-D") hello world apple pie hello world apple pie apple pie apple pie > f(s, "-u") dog zebra >  • Clever use of rle and switch, +1! – Billywob Sep 7 '16 at 9:39 ## R, 179 bytes f=function(s,o){s=sort(s);n="\n";z=cat;for(i in unique(s)){v=sum(s%in%i);if(v>1&o=="-d")z(i,n);if(o=="-c")z(v, i,n);if(o=="-D"&v>1)z(rep(c(i,n),v),n,sep="");if(o=="-u"&v<2)z(i,n)}}  ### Input Input is assumed to be an "R-vector" of strings (inspired by @Arnauld's JS array input, see below) and a string containing the type of function e.g. -d. ### Example An example of how to execute the function where I pre-define the vector of strings (not necessary). strings <- c("hello world", "apple pie", "zebra", "dog", "hello world", "apple pie", "apple pie", "apple pie") f(strings,"-c") f(strings, "-d") f(strings,"-D") f(strings, "-u")  ### Ungolfed f=function(s,o){ s=sort(s) # sort vector alphabetically n="\n" # generate string for newline (used 5x times) z=cat # generate alias for cat (used 4x times) for(i in unique(s)){ # for each unique string of vector v=sum(s%in%i) # count number of occurences if(o=="-d"&v>1)z(i,n) # if occurences >1 & "-d": print each unique string if(o=="-D"&v>1)z(rep(c(i,n),v),n,sep="") # repeat string #occurences and then print if(o=="-u"&v<2)z(i,n) # if occurences <2 & "-u": print each uniquely occuring string if(o=="-c")z(v,i,n) # if "-c": print number of occurence and each unique string } }  I'm sure this can be golfed further by shortening the control flow statements but couldn't figure out a smart way to do it yet. • It is generally acceptable to have the functions as unnamed, so you can save yourself a couple. – MickyT Sep 6 '16 at 23:25 # Javascript (ES6), 170 164 bytes (a,c)=>a.map(s=>l[s]=(l[s]|0)+1,l={})&&Object.keys(l).sort().map(k=>(n=l[k],o=[n+' '+k,n>1&&k,n>1&&${k}
.repeat(n),n<2&&k]['cdDu'.indexOf(c[1])])&&console.log(o))


Input:
array of strings + flag string, such as ["hello world", "apple pie"], "-c".

Output:
results are printed to the console

### Example

let f =
(a,c)=>a.map(s=>l[s]=(l[s]|0)+1,l={})&&Object.keys(l).sort().map(k=>(n=l[k],o=[n+' '+k,n>1&&k,n>1&&${k} .repeat(n),n<2&&k]['cdDu'.indexOf(c[1])])&&console.log(o)) console.log('-c');f([ "hello world", "apple pie", "zebra", "dog", "hello world", "apple pie", "apple pie", "apple pie" ], "-c"); console.log('-d');f([ "hello world", "apple pie", "zebra", "dog", "hello world", "apple pie", "apple pie", "apple pie" ], "-d"); console.log('-D');f([ "hello world", "apple pie", "zebra", "dog", "hello world", "apple pie", "apple pie", "apple pie" ], "-D"); console.log('-u');f([ "hello world", "apple pie", "zebra", "dog", "hello world", "apple pie", "apple pie", "apple pie" ], "-u"); ## Lua, 270 Bytes Well, that ended being way longer than I though it would, but I'm happy with it. Takes in a filename via command-line and forms two array while reading it's content (t={line, repetitions} and T={index, line}). Sorts T then outputs via STDOUT depending on the flag. f,m=...io.input(f)t={}T={}for l in io.lines()do l=l.."\n"t[l]=t[l]and t[l]+1or 1T[#T+1]=t[l]<2 and l or nil end table.sort(T)o=""for i=1,#T do k=T[i]v=t[k]o=o..(m=="-c"and v.." "..k or m=="-D"and k:rep(v)or m=="-d"and v>1 and k or m=="-u" and v<2 and k or'')end print(o)  ### Explanation f,m=... -- shorthands for the inputs io.input(f) -- set the file f as the current input stream t={} -- set the array linking a line to its number of occurrence T={} -- set the array containing each line once for sorting purpose for l in io.lines() -- iterate over each line in the file do l=l.."\n" -- add a newline at the end of it t[l]=t[l] -- if it's not the first time we see this line and t[l]+1 -- increment its number of occurence or 1 -- else set it to 1 occurence T[#T+1]=t[l]<2 and l or nil-- and set it in our listing end table.sort(T) -- sort the list of line o="" -- initialise the output to an empty string for i=1,#T -- iterate over each entry in T do k=T[i] -- shorthand for the line's string v=t[k] -- shorthand for its number of occurence -- the following nested ternaries are basically a chained if/elseif o=o.. -- at the end of our current output string (m=="-c" -- if the flag is -c(ount) and v.." "..k -- add the number of occurrence and the corresponding string or m=="-D" -- if the flag is -D(uplicates) and k:rep(v) -- add v times the line or m=="-d" and v>1 -- if the flag is -d(uplicates) and it has appeared >2 times and k -- add this line or m=="-u" and v<2 -- if the flag is -u(niq) and it has appeared only once and k -- add this line or '') -- if neither of these cases are true, had an empty string end print(o) -- output what we formatted previously :)  # Bash(coreutils, grep), 265 bytes IFS=$'\n'
for l in sort -u $1;do r=$r$(grep -c "$l" $1)"$l\n"
done
case $2 in -c)printf$r;;-d)printf $r|grep -v "1.*"|cut -d' ' -f2-;;-D)s=$(sort $1) for l in sh$0 $1 -u;do s=$(grep -v "$l"<<<"$s")
done
echo "$s";;-u)printf$r|grep "1.*"|cut -d' ' -f2-;;esac


Usage

$sh unique.bash input -D apple pie apple pie apple pie apple pie hello world hello world$ sh unique.bash input -c
4 apple pie
1 dog
2 hello world
1 zebra
$sh unique.bash input -u dog zebra$ sh unique.bash input -d
apple pie
hello world


Explanation

• Constructs \$r which is the output of -c. To construct it, it uses sort -u which sorts the input file and returns only unique results (I know, somewhat cheat but it wasn't specified in the rules. Feel free to ignore the answer if you don't like it :-). Also, it uses grep -c to count the occurrences of each word.
• For -d it gets the output of -c and with grep it prints everything that does not start with 1. With cut it ignores the first column (number of occurrences).
• For -u it does exactly the same as -d except from grep, where it selects everything that starts with 1.
• For -D it get the sorted input and remove all the words that are printed with -u in a for loop by using grep -v again.

# Java, 19(import java.util.*;) + 315 = 334 bytes

String f(String[]r,String q){TreeMap<String,Integer>m=new TreeMap();for(String s:r)m.put(s,m.get(s)==null?1:m.get(s)+1);int a=q.charAt(1),v;String o="",k;for(Map.Entry<String,Integer>e:m.entrySet()){v=e.getValue();k=e.getKey()+"\n";while(68==a&&v-->1)o+=k;o+='u'==a&&v<2||'d'==a&&v>1?k:99==a?v+" "+k:"";}return o;}}


Usage

String[] input = {"hello world","apple pie","zebra","dog","hello world","apple pie","apple pie","apple pie"};
System.out.println("-c");
System.out.println(f(input,"-c"));
System.out.println("-d");
System.out.println(f(input,"-d"));
System.out.println("-D");
System.out.println(f(input,"-D"));
System.out.println("-u");
System.out.println(f(input,"-u"));


Output

-c
4 apple pie
1 dog
2 hello world
1 zebra

-d
apple pie
hello world

-D
apple pie
apple pie
apple pie
hello world

-u
dog
zebra


Explanation A TreeMap<String, Integer> is used to store pairs of words and how many times they were repeated. The TreeMap is sorted so no extra call to sort it is required. Ternary operator is used to construct the output string regarding the input argument.

# Python 2, 115 bytes

f,a=input()
f=ord(f[1])%5
for w in sorted(set(a)):C=a.count(w);exec"print'%d '%C*(f>3)+w;"*[C>1,0,C<2,(C>1)*C,1][f]


Example:

golf % python2.7 uniq.py <<<'["-c",["hello world","apple pie","zebra","dog","hello world","apple pie","apple pie","apple pie"]]'
4 apple pie
1 dog
2 hello world
1 zebra


# Groovy (185 Bytes)

## Golf'd:

x={m,l->u=[];u.addAll(l);u.unique();c=u.collect{v->[v,l.count{it==v}]};D=c.collect{a,b->if (b>1){(1..b).collect{a}}};D.remove(null);m=='d'?D.collect{it.unique()}:['D':D,'c':c,'u':u][m]}


https://groovyconsole.appspot.com/edit/5173563388592128

## Ungolf'd (Kinda):

x = {
m,l->
u=[];
u.unique();
c = u.collect {
v->
[v, l.count {it==v}]
}
D = c.collect {
a,b->
if (b>1) {
(1..b).collect{a}
}
}
D.remove(null)
m == 'd' ? D.collect{it.unique()} : ['D':D,'c':c,'u':u][m]
}


## Output

• u Results aa bb xx

• c Results [aa, 3] [bb, 2] [xx, 1]

• D Results [aa, aa, aa] [bb, bb]

• d Results [aa] [bb]