UNIX Uniq Uknow?

Question

Challenge

You must write a program which takes a string and a flag as input and performs one of the following functions.

Input

If your language does not support command line arguments, you may take the arguments as input via STDIN.

The flag will be two characters: a dash and an ASCII character such as -d or -c.

Functions

The following is the list of functions which you must implement in your program. The title of each is the flag which corresponds to the function.

Counting: -c

This counts the number of duplicate lines in the input file. For example, if the input is:

aa
aa
bb
bb
bb
xx

The output will be:

2 aa
3 bb
1 xx

Because there are two lines of aa, three lines of bb, and one line of xx. The output should be separated by newlines.

Duplication: -d

This shows which lines are duplicates. For example, using the above example string (aa\naa\nbb\nbb\nbb\nxx), the output would be:

aa
bb

Because these are lines which have been duplicated. You should only output each line once.

Duplication: -D

This is almost the same as -d except it outputs each line the number of times that it occurs in the input file. For example, using the example string the output will be:

aa
aa
bb
bb
bb

Because aa occurs twice and bb occurs three times.

Unique: -u

Similar to above except it only prints unique lines. For example, using the example file, the output will be:

xx

Because xx is the only unique line. Also, due to xx being unique, you will only ever have to output this once.

Example

Note that although the input string will not always be sorted, the output must be sorted alphabetically. For example, for the following input:

hello world
apple pie
zebra
dog
hello world
apple pie
apple pie
apple pie

The output for each of the flags would be:

-c

4 apple pie
1 dog
2 hello world
1 zebra

-d

apple pie
hello world

-D

apple pie
apple pie
apple pie
apple pie
hello world
hello world

-u

dog
zebra

Rules

Your programs must not use the UNIX program uniq as, frankly, that is plain cheating.

All output should be to STDOUT.

The file will only ever contain lowercase, alphabetical characters, newlines and spaces:

abcdefghijklmnopqrstuvwxyz

The output should be sorted alphabetically.

In case there is any confusion, programs written on any operating system can enter. The challenge is not limited to UNIX.

As per PPCG standard, functions and anonymous functions are allowed, taking input via their arguments and outputting via a returned string.

Winning

As this is code-golf, the shortest code in bytes wins.

Answer 1

GNU sed, 496 485 + 2(rn flags) = 487 bytes

1h;1b;$bb;2{:r;$!N;$!br;s/.*/echo "&"|sort/e};:b;x;/[dD]/bd;/c/bc;/u/!b;x;s/^/\n/
s/(\n.*)\1+//g;s/\n//p;b;:d;x;/^(.*)\n\1/!D;:l;x;/D/{x;P;x};x;s/^[^\n]*\n//
/^(.*)\n\1/bl;P;D;:c;s/$/#/;G;x;/^$/bp;s/[^\n]*\n?//;x;s/#(\n[^\n]*).*/\1/
/([^:\n]*)\n\1$/{s/\n[^\n]*$//;:i;s/9(@*:)/@\1/;ti;s/8(@*:)/9\1/;s/7(@*:)/8\1/
s/6(@*:)/7\1/;s/5(@*:)/6\1/;s/4(@*:)/5\1/;s/3(@*:)/4\1/;s/2(@*:)/3\1/;s/1(@*:)/2\1/
s/0(@*:)/1\1/;s/@+:/1&/;y/@/0/;bc};s/:/ /;s/[^\n]*$/1:&/;bc;:p;x;s/...//;s/..$//;s/:/ /gp

The code size is not going to compete, but it was fun emulating the uniq command. I wanted to add a simple selection sort to get rid of the sort call, thus solving it in pure sed only, but the code got too long already. The input is read entirely from STDIN, with the flag expected on the first line.

Run:

(echo "-c";cat input_file)|sed -rnf uniq_emulator.sed

Output:

4 apple pie
1 dog
2 hello world
1 zebra

Initially submitted code and explanation: (sections are separated by an extra newline)

# The main
    # store the flag in hold space and start next cycle
1h;1b
    # usually a cycle starts with each line read, but I also start it manually,
    #so I need to skip some lines that should run only once
$bb
2{
        # read the entire input into pattern space
    :r;$!N;$!br
        # executes a shell call to sort the pattern space
    s/.*/echo "&"|sort/e
}
    # based on the flag, the appropriate jumps to "functions" are made
:b;x;/[dD]/bd;/c/bc;/u/!b

# Unique: -u
    # all duplicate lines are deleted and what remains is printed
x;s/^/\n/;s/(\n.*)\1+//g
s/\n//p;b

# Duplication: -d and -D
    # if the first 2 lines are different, delete the first and start new cycle
:d;x;/^(.*)\n\1/!D
:l
        # if -D, print the first duplicate line
    x;/D/{x;P;x};x
        # otherwise (-d), delete the first duplicate line
    s/^[^\n]*\n//
        # loop condition, go back to :l if the first 2 lines are duplicate
/^(.*)\n\1/bl
    # print only the first line and start a new cycle
P;D

# Counting: -c
    # in order to count line occurrences, I simulate reading the input again
    # In a loop I append all pattern space to hold space, delete first line of
    #pattern space and from hold space all lines except the first one appended.
:c;s/$/#/;G
x;/^$/bp;s/[^\n]*\n?//;x;s/#(\n[^\n]*).*/\1/
    # if the new line "read" is the same as the previous one, increment the
    #counter of that line (format is "counter:line"), delete and go back to :c
/([^:\n]*)\n\1$/{s/\n[^\n]*$//;bi;:I;bc}
    # otherwise, stop the counter for the previous line, add a counter for the
    #current one and go back to :c
s/:/ /;s/[^\n]*$/1:&/;bc
    # when the loop ends, the hold space with all counts is printed
:p;x;s/...//;s/..$//;s/:/ /gp

# Increment function that works by manually updating the changing digit(s)
:i;s/9(@*:)/@\1/;ti
s/8(@*:)/9\1/;s/7(@*:)/8\1/;s/6(@*:)/7\1/
s/5(@*:)/6\1/;s/4(@*:)/5\1/;s/3(@*:)/4\1/
s/2(@*:)/3\1/;s/1(@*:)/2\1/;s/0(@*:)/1\1/
s/@+:/1&/;y/@/0/;tI

Answer 2

Perl, 92 + 4 (-sn flags) = 96 bytes

$h{$_}++}{$v=$h{$_},print$c?"$v $_":($D||$d)&&$v>1||$u&&$v<2?$_ x($d?1:$v):''for sort keys%h

This code needs to be in a file to run. And it needs -sn flags, as well as -M5.010 (free).
For instance :

$ cat input.txt
hello world
apple pie
zebra
dog
hello world
apple pie
apple pie
apple pie
$ cat prog.pl
$h{$_}++}{$v=$h{$_},print$c?"$v $_":($D||$d)&&$v>1||$u&&$v<2?$_ x($d?1:$v):''for sort keys%h
$ perl -M5.010 -sn prog.pl -c < input.txt
4 apple pie
1 dog
2 hello world
1 zebra

Answer 3

R, 128 bytes

Using a different method than @Billywob and implemented as a unnamed function

function(s,o){R=rle(sort(s));cat(switch(o,'-d'=R$v[R$l>1],'-D'=s[s%in%R$v[R$l>1]],'-c'=paste(R$l,R$v),'-u'=R$v[R$l<2]),sep='
')}

Ungolfed

function(s,o){
   R=rle(sort(s));               # do the run length encoding for the sorted input
   cat(                          # output to STDOUT
      switch(o                   # switch on option to provide output lines
        ,'-d'=R$v[R$l>1]         # RLE values with length > 1
        ,'-D'=s[s%in%R$v[R$l>1]] # string values in RLE values with length > 1
        ,'-c'=paste(R$l,R$v)     # RLE length and values pasted together
        ,'-u'=R$v[R$l<2]         # RLE values with length < 2, eg 1
       )
      ,sep='
'                                # set separator to carriage return
   )
}

Test Run

function(s,o){R=rle(sort(s));cat(switch(o,'-d'=R$v[R$l>1],'-D'=s[s%in%R$v[R$l>1]],'-c'=paste(R$l,R$v),'-u'=R$v[R$l<2]),sep='
')}
> s = c("hello world", "apple pie", "zebra", "dog", "hello world", "apple pie", "apple pie", "apple pie")
> f =
function(s,o){R=rle(sort(s));cat(switch(o,'-d'=R$v[R$l>1],'-D'=s[s%in%R$v[R$l>1]],'-c'=paste(R$l,R$v),'-u'=R$v[R$l<2]),sep='
')}
> f(s, "-c")
4 apple pie
1 dog
2 hello world
1 zebra
> f(s, "-d")
apple pie
hello world
> f(s, "-D")
hello world
apple pie
hello world
apple pie
apple pie
apple pie
> f(s, "-u")
dog
zebra
>

Answer 4

R, 179 bytes

f=function(s,o){s=sort(s);n="\n";z=cat;for(i in unique(s)){v=sum(s%in%i);if(v>1&o=="-d")z(i,n);if(o=="-c")z(v, i,n);if(o=="-D"&v>1)z(rep(c(i,n),v),n,sep="");if(o=="-u"&v<2)z(i,n)}}

Input

Input is assumed to be an "R-vector" of strings (inspired by @Arnauld's JS array input, see below) and a string containing the type of function e.g. -d.

Example

An example of how to execute the function where I pre-define the vector of strings (not necessary).

strings <- c("hello world", "apple pie", "zebra", "dog", "hello world", "apple pie", "apple pie", "apple pie")

f(strings,"-c")
f(strings, "-d")
f(strings,"-D")
f(strings, "-u")

Ungolfed

f=function(s,o){
    s=sort(s)                   # sort vector alphabetically
    n="\n"                      # generate string for newline (used 5x times)
    z=cat                       # generate alias for cat (used 4x times)
    for(i in unique(s)){        # for each unique string of vector
        v=sum(s%in%i)           # count number of occurences
        if(o=="-d"&v>1)z(i,n)   # if occurences >1 & "-d": print each unique string  
        if(o=="-D"&v>1)z(rep(c(i,n),v),n,sep="") # repeat string #occurences and then print
        if(o=="-u"&v<2)z(i,n)   # if occurences <2 & "-u": print each uniquely occuring string  
        if(o=="-c")z(v,i,n)     # if "-c": print number of occurence and each unique string
   }
}

I'm sure this can be golfed further by shortening the control flow statements but couldn't figure out a smart way to do it yet.

Answer 5

JavaScript (V8),  170 164  140 bytes

Expects (array_of_strings)(flag_string). Prints the output.

a=>c=>[...new Set(a)].sort().map(k=>(a.map(v=>n+=v==k,n=0),o={c:n+' '+k,d:n>1&&k,D:n>1&&Array(n).fill(k).join`
`,u:n<2&&k}[c[1]])&&print(o))

Try it online!

Answer 6

Bash(coreutils, grep), 265 bytes

IFS=$'\n'
for l in `sort -u $1`;do r=$r$(grep -c "$l" $1)" $l\n"
done
case $2 in -c)printf $r;;-d)printf $r|grep -v "1.*"|cut -d' ' -f2-;;-D)s=$(sort $1)
for l in `sh $0 $1 -u`;do s=$(grep -v "$l"<<<"$s")
done
echo "$s";;-u)printf $r|grep "1.*"|cut -d' ' -f2-;;esac

Usage

$ sh unique.bash input -D
apple pie
apple pie
apple pie
apple pie
hello world
hello world
$ sh unique.bash input -c
4 apple pie
1 dog
2 hello world
1 zebra
$ sh unique.bash input -u
dog
zebra
$ sh unique.bash input -d
apple pie
hello world

Explanation

• Constructs $r which is the output of -c. To construct it, it uses sort -u which sorts the input file and returns only unique results (I know, somewhat cheat but it wasn't specified in the rules. Feel free to ignore the answer if you don't like it :-). Also, it uses grep -c to count the occurrences of each word.
• For -d it gets the output of -c and with grep it prints everything that does not start with 1. With cut it ignores the first column (number of occurrences).
• For -u it does exactly the same as -d except from grep, where it selects everything that starts with 1.
• For -D it get the sorted input and remove all the words that are printed with -u in a for loop by using grep -v again.

Answer 7

jq -r, 115 bytes

Takes an array of strings and the flag as separate string.

group_by(.)|map(select(has(1)))as$m|{c:map("\(length) \(.[0])"),d:[$m[][0]],D:[$m[][]],u:[.-$m|.[][]]}[input[1:]][]

Example and explanation:

echo '["aa", "aa", "bb", "bb", "bb", "xx"] "-c"' | jq -r '
  group_by(.) |                  # sort and group by item
  map(select(has(1))) as $m |    # store the slice of non-uniques
  {                              # create a lookup object
    c: map("\(length) \(.[0])"), # map to length and one item
    d: [    $m    [][0]],        # take one of each from the slice
    D: [    $m    [][ ]],        # take all of them from the slice
    u: [. - $m | .[][ ]]         # subtract the slice from the input
  }                              #     and take all of them
  [input[1:]]                    # lookup by the 2nd char of flag
  []                             # iterate and output
'

2 aa
3 bb
1 xx

Try it online!

Answer 8

J, 84 bytes

('cDdu'i.1{[)>@{[:(~.@]((],' ',[)&":"p.;D;~.@D;[-.D=.#~]*1<])1#.=@])@/:~];._1@(LF,])

Try it online!

A bit cumbersome, but I've progressively golfed it down from about 100b. I expect more golfing opportunities may be had, but I'm not quite sure where at this point.

Explanation

This answer can be divided into a bunch of components. Here's an easier to read version of the above (Try it online!) (technically functions slightly differently since we're not actually doing answer_list@sort, but rather, (answer_list@/:)~, which is happens to be the same thing.)

get_index =: 'cDdu' i. 1 { [
lines =: ];._1@(LF , ])
sort =: /:~
multi_dup =: #~ ] * 1 < ]
only_once =: [ -. multi_dup
single_dup =: ~.@multi_dup
count_list =: (] , ' ' , [)&":"1 0
counts =: 1 #. =@]
unique =: ~.@]
answer_list =: unique (count_list ; multi_dup ; single_dup ; only_once) counts
uniq_ans =: get_index pick answer_list@sort@lines

Answer 9

Python 2, 115 bytes

f,a=input()
f=ord(f[1])%5
for w in sorted(set(a)):C=a.count(w);exec"print'%d '%C*(f>3)+w;"*[C>1,0,C<2,(C>1)*C,1][f]

Example:

golf % python2.7 uniq.py <<<'["-c",["hello world","apple pie","zebra","dog","hello world","apple pie","apple pie","apple pie"]]'
4 apple pie
1 dog
2 hello world
1 zebra

Answer 10

Lua, 270 Bytes

Well, that ended being way longer than I though it would, but I'm happy with it.

Takes in a filename via command-line and forms two array while reading it's content (t={line, repetitions} and T={index, line}). Sorts T then outputs via STDOUT depending on the flag.

f,m=...io.input(f)t={}T={}for l in io.lines()do
l=l.."\n"t[l]=t[l]and t[l]+1or 1T[#T+1]=t[l]<2 and l or nil
end
table.sort(T)o=""for i=1,#T
do
k=T[i]v=t[k]o=o..(m=="-c"and v.." "..k
or m=="-D"and k:rep(v)or
m=="-d"and v>1 and k
or m=="-u" and v<2 and k or'')end
print(o)

Explanation

f,m=...                        -- shorthands for the inputs
io.input(f)                    -- set the file f as the current input stream
t={}                           -- set the array linking a line to its number of occurrence
T={}                           -- set the array containing each line once for sorting purpose
for l in io.lines()            -- iterate over each line in the file
do
    l=l.."\n"                  -- add a newline at the end of it
    t[l]=t[l]                  -- if it's not the first time we see this line
           and t[l]+1          -- increment its number of occurence
         or 1                  -- else set it to 1 occurence
    T[#T+1]=t[l]<2 and l or nil-- and set it in our listing
end
table.sort(T)                  -- sort the list of line
o=""                           -- initialise the output to an empty string
for i=1,#T                     -- iterate over each entry in T
do
    k=T[i]                     -- shorthand for the line's string
    v=t[k]                     -- shorthand for its number of occurence

    -- the following nested ternaries are basically a chained if/elseif
    o=o..                      -- at the end of our current output string
      (m=="-c"                 -- if the flag is -c(ount)
          and v.." "..k        -- add the number of occurrence and the corresponding string
        or m=="-D"             -- if the flag is -D(uplicates)
          and k:rep(v)         -- add v times the line
      or m=="-d" and v>1       -- if the flag is -d(uplicates) and it has appeared >2 times
          and k                -- add this line
      or m=="-u" and v<2       -- if the flag is -u(niq) and it has appeared only once
          and k                -- add this line
        or '')                 -- if neither of these cases are true, had an empty string
end
print(o)                       -- output what we formatted previously :)

Answer 11

Java, 19(import java.util.*;) + 315 = 334 bytes

String f(String[]r,String q){TreeMap<String,Integer>m=new TreeMap();for(String s:r)m.put(s,m.get(s)==null?1:m.get(s)+1);int a=q.charAt(1),v;String o="",k;for(Map.Entry<String,Integer>e:m.entrySet()){v=e.getValue();k=e.getKey()+"\n";while(68==a&&v-->1)o+=k;o+='u'==a&&v<2||'d'==a&&v>1?k:99==a?v+" "+k:"";}return o;}}

Usage

String[] input = {"hello world","apple pie","zebra","dog","hello world","apple pie","apple pie","apple pie"};
System.out.println("-c");
System.out.println(f(input,"-c"));
System.out.println("-d");
System.out.println(f(input,"-d"));
System.out.println("-D");
System.out.println(f(input,"-D"));
System.out.println("-u");
System.out.println(f(input,"-u"));

Output

-c
4 apple pie
1 dog
2 hello world
1 zebra

-d
apple pie
hello world

-D
apple pie
apple pie
apple pie
hello world

-u
dog
zebra

Explanation A TreeMap<String, Integer> is used to store pairs of words and how many times they were repeated. The TreeMap is sorted so no extra call to sort it is required. Ternary operator is used to construct the output string regarding the input argument.

Answer 12

Vyxal, 44 bytes

⁽R‡'tċ;vh‡'tċ¬;vh≬'tċ;ƛ÷$wẋ;fW‛Ǎ⟇?tḟi$sĊ$†W⁋

Try it Online!

A fun little answer that gets to use a list of lambdas. Takes input as flag, list of lines.

Explained

⁽R‡'tċ;vh‡'tċ¬;vh≬'tċ;ƛ÷$wẋ;fW‛Ǎ⟇?tḟi$sĊ$†W⁋
                             W               # Wrap the stack into a single list:
⁽R                                           # A function that reverses each item in a list
  ‡'tċ;vh                                    # A function that keeps only non-unique items
         ‡'tċ¬;vh                            # A function that keeps only unique items
                 ≬'tċ;ƛ÷$wẋ;f                # A function that repeats non-unique items by their count
                              ‛Ǎ⟇?tḟi        # Get the function at index "cdu".find(input[-1])
                                     $sĊ$†   # Call that function on a list of the counts of each item in the input, sorted alphabetically
                                          W⁋ # Wrap into a single list (needed because of a bug) and join on newlines.

Answer 13

05AB1E, 25 24 bytes

C|ÅγøíIÅγ2@ÏIDÅγÏDŠK)ćè»

05AB1E doesn't support custom system arguments, so it takes the inputs through STDIN.
The first input-line is the flag, and every other input-line is the input-file.

Try it online or verify all four flags.

Explanation:

C         # Convert the first (implicit) input-flag from a binary-string to an integer†
|         # Get all other input-lines as a list of strings
 Åγ       # Run-length encode it, and push the list of values and counts separated to
          # the stack
   ø      # Pair the values and lengths together
    í     # Reverse each from [value,count] to [count,value]
I         # Push the list of input-lines again
 Åγ       # Run-length encode it again
   2@     # Check for each count if it's >=2
     Ï    # Only leave the values at the truthy indices
I         # Push the list of input-lines yet again
 D        # Duplicate it
  Åγ      # Run-length encode it a third time
    Ï     # Only leave the values at the truthy (==1) indices
     D    # Duplicate this list of unique items
      Š   # Triple-swap the input,unique,unique lines to unique,input,unique lines
       K  # Remove the unique lines from the input
)         # Wrap the entire stack into a list
 ć        # Extract head; pop list and push remainder-list and first item separately to
          # the stack
  è       # Use that C(input-flag) to 0-based modular index it into the list
   »      # Join each inner list by spaces first (for the list of [count,value] pairs),
          # and then join each string by newlines
          # (after which the result is output implicitly)

† C on strings basically gets the index of each character in the string 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzǝʒαβγδεζηθвимнт ΓΔΘιΣΩ≠∊∍∞₁₂₃₄₅₆ !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~Ƶ€Λ‚ƒ„…†‡ˆ‰Š‹ŒĆŽƶĀ‘’“”–—˜™š›œćžŸā¡¢£¤¥¦§¨©ª«¬λ®¯°±²³\nµ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ•, and then converts this list of indices to a base-2 integer:

Flag	C	0-based modular index
"-c"	252	0
"-d"	253	1
"-D"	270	2
"-u"	227	3

Answer 14

Groovy (185 Bytes)

Golf'd:

x={m,l->u=[];u.addAll(l);u.unique();c=u.collect{v->[v,l.count{it==v}]};D=c.collect{a,b->if (b>1){(1..b).collect{a}}};D.remove(null);m=='d'?D.collect{it.unique()}:['D':D,'c':c,'u':u][m]}

https://groovyconsole.appspot.com/edit/5173563388592128

Ungolf'd (Kinda):

x = {
    m,l->
        u=[];
        u.addAll(l);
        u.unique();
        c = u.collect {
            v->
                [v, l.count {it==v}]
        }
        D = c.collect {
            a,b->
                if (b>1) {
                    (1..b).collect{a}
                }
        }
        D.remove(null)
        m == 'd' ? D.collect{it.unique()} : ['D':D,'c':c,'u':u][m]
}

Output

• u Results aa bb xx

• c Results [aa, 3] [bb, 2] [xx, 1]

• D Results [aa, aa, aa] [bb, bb]

• d Results [aa] [bb]