Remove repeated words from a string

Question

Remove all repeating words from an inputted sentence.

Input will be something like cat dog cat dog bird dog Snake snake Snake and the output should be cat dog bird Snake snake. There will always be a single space separating words.

Output order must be the same as input. (Refer to the example)

You don't need to handle punctuation but capital letter handling is required.

Answer 1

gs2, 3 bytes

,É-

Encoded in CP437.

STDIN is pushed at the start of the program. , splits it over spaces. É is uniq, which filters duplicates. - joins by spaces.

Answer 2

CJam, 7 chars

qS/_&S*

Can probably be much shorter... but whatever I've almost never used CJam. ^.^

q reads input, S/ splits on spaces, _& duplicates and applies a setwise AND (therefore getting rid of duplicates), and S* re-joins on space.

Online interpreter link

Answer 3

Haskell, 34 bytes

import Data.List
unwords.nub.words

Usage example: (unwords.nub.words) "cat dog cat dog bird dog Snake snake Snake" -> "cat dog bird Snake snake".

Answer 4

APL, 22 20 bytes

{1↓∊∪(∊∘' '⊂⊢)' ',⍵}

This creates an unnamed monadic function that accepts a string on the right and returns a string.

Explanation:

               ' ',⍵}    ⍝ Prepend a space to the input string
     (∊∘' '⊂⊢)          ⍝ Split the string on spaces using a fork
    ∪                    ⍝ Select the unique elements
{1↓∊                     ⍝ Join into a string and drop the leading space

Try it online

Saved 2 bytes thanks to Dennis!

Answer 5

Ruby, 21 chars

->s{s.split.uniq*' '}

Answer 6

JavaScript (ES6) 33

(see this answer)

Test running the snippet below in an EcmaScript 6 compliant browser (implementing Set, spread operator, template strings and arrow functions - I use Firefox).

Note: the conversion to Set drop all the duplicates and Set mantains the original ordering.

f=s=>[...Set(s.split` `)].join` `

function test() { O.innerHTML=f(I.value) }

test()

#I { width: 70% }

<input id=I value="cat dog cat dog bird dog Snake snake Snake"/><button onclick="test()">-></button>
<pre id=O></pre>

Answer 7

TeaScript, 12 bytes

TeaScript is JavaScript for golfing.

xs` `u()j` `

This is pretty short. It splits on each space, filters out duplicates, then rejoins.

Try it online

Answer 8

PowerShell, 15 Bytes

$args|select -u

Whoa, an actual entry where PowerShell is somewhat competitive? That's unpossible!

Takes the string as input arguments, pipes to Select-Object with the -Unique flag. Spits out an array of strings, preserving order and capitalization as requested.

Usage:

PS C:\Tools\Scripts\golfing> .\remove-repeated-words-from-string.ps1 cat dog cat dog bird dog Snake snake Snake
cat
dog
bird
Snake
snake

---

If this is too "cheaty" in assuming the input can be as command-line arguments, then go for the following, at 24 21 Bytes _{(saved some bytes thanks to blabb)}. Interestingly, using the unary operator in this direction happens to also work if the input string is demarcated with quotes or as individual arguments, since the default -split is by spaces. Bonus.

-split$args|select -u

Answer 9

Julia, 29 bytes

s->join(unique(split(s))," ")

This creates an unnamed function that splits the string into a vector on spaces, keeps only the unique elements (preserving order), and joins the array back into a string with spaces.

Answer 10

R, 22 bytes

cat(unique(scan(,"")))

This reads a string from STDIN and splits it into a vector on spaces using scan(,""), selects only unique elements, then concatenates them into a string and prints it to STDOUT using cat.

Answer 11

Retina, 22 bytes

 (\w+)\b(?<=\b\1\b.+)

Save the file with a trailing linefeed and run it with the -s flag.

This is fairly straight forward in that it matches a single word, and the lookbehind checks whether that same word has appeared in the string before. The trailing linefeed causes Retina to work in Replace mode with an empty replacement string, removing all matches.

Answer 12

Mathematica, 43 39 bytes

StringRiffle@*Keys@*Counts@*StringSplit

Answer 13

Pyth - 9 bytes

Well this is why we're all waiting for Pyth5, could have been 5 bytes.

jdoxzN{cz

Try it online here.

Answer 14

C++11, 291 bytes

#include<iostream>
#include<string>
#include<list>
#include<sstream>
#include<algorithm>
using namespace std;main(){string s;getline(cin,s);list<string>m;stringstream b(s);while(getline(b,s,' '))if(find(m.begin(),m.end(),s)==m.end())m.push_back(s);for(auto a:m)cout<<a<<' ';cout<<endl;}

I don't see a whole lot of C++ answers compared to golfing languages, so why not. Note that this uses C++11 features, and so if your compiler is stuck in the dark ages sufficiently old enough, you may need to pass a special compilation switch to make it use the C++11 standard. For g++, it's -std=c++11 (only needed for versions < 5.2). Try it online

Answer 15

K5, 9 bytes

" "/?" "\

FYI, this is a function.

Explanation

     " "\    Split the input on spaces
    ?        Find all the unique elements
" "/         Join them back together

Answer 16

Matlab: 18 Bytes

unique(d,'stable')

where d is d = {'cat','dog','cat','dog','bird','dog','Snake','snake','Snake'}.

The result is 'cat' 'dog' 'bird' 'Snake' 'snake'

Answer 17

Python 3, 55

l=[]
for x in input().split():l+=[x][x in l:]
print(*l)

Yeesh, this is long. Unfortunately, Python's set doesn't keep the order of the elements, so we have to do the work ourselves. We iterate through the input words, keeping a list l of elements that aren't yet in l. Then, we print the contents of l space-separated.

A string version of l would not work if some words are substrings of other words.

Answer 18

awk, 25

BEGIN{RS=ORS=" "}!c[$0]++

Output:

$ printf "cat dog cat dog bird dog Snake snake Snake" | awk 'BEGIN{RS=ORS=" "}!c[$0]++'
cat dog bird Snake snake $ 
$ 

Answer 19

C#, 38 bytes

String.Join(" ",s.Split().Distinct());

Answer 20

Perl 6, 14 bytes

As a whole program the only way you would write it is 21 bytes long

say $*IN.words.unique # 21 bytes

---

As a lambda expression the shortest is 14 bytes

*.words.unique # 14 bytes

say ( *.words.unique ).('cat dog cat dog bird dog Snake snake Snake')

my &foo = *.words.unique;
say foo $*IN;

While the output is a List, if you put it in a stringifying context it will put a space between the elements. If it was a requirement to return a string you could just add a ~ to the front ~*.words.unique.

---

If snippets were allowed, you could shorten it to 13 bytes by removing the *.

$_ = 'cat dog cat dog bird dog Snake snake Snake';

say .words.unique

Answer 21

Python 3, 87 80 bytes

turns out the full program version is shorter

s=input().split(' ')
print(' '.join(e for i,e in enumerate(s)if e not in s[:i]))

Did it without regex, I am happy

Try it online

Answer 22

Lua, 94 bytes

function c(a)l={}return a:gsub("%S+",function(b)if l[b]then return""else l[b]=true end end)end

Answer 23

JavaScript, 106 102 100 bytes

function(s){o={};s.split(' ').map(function(w){o[w]=1});a=[];for(w in o)a.push(w);return a.join(' ')}

// way too long for JS :(

Answer 24

Hassium, 91 bytes

func main(){d=[]foreach(w in input().split(' '))if(!(d.contains(w))){d.add(w)print(w+" ")}}

Run online and see expanded here

Answer 25

PHP 64 59 bytes

function r($i){echo join(" ",array_unique(split(" ",$i)));}

Answer 26

AppleScript, 162 bytes

Interestingly, this is almost identical to the non-repeating characters thing.

set x to(display dialog""default answer"")'s text returned's words
set o to""
repeat with i in x
considering case
if not i is in o then set o to o&i&" "
end
end
o

I didn't actually know the considering keyword before this. the more you know...

Answer 27

Burlesque, 6 bytes

blsq ) "cat dog cat dog bird dog Snake snake Snake"wdNBwD
cat dog bird Snake snake

Rather simple: split words, nub (nub = remove duplicates), convert back to words.

Answer 28

Gema, 21 characters

*\S=${$0;$0}@set{$0;}

(Very similar to the unique character solution, as there are no arrays in Gema, so allowing built-in unique functions not helps us much.)

Sample run:

bash-4.3$ gema '*\S=${$0;$0}@set{$0;}' <<< 'cat dog cat dog bird dog Snake snake Snake'
cat dog bird Snake snake 

Answer 29

Scala, 44 47 bytes

(s:String)=>s.split(" ").distinct.mkString(" ")

EDIT: using toSet might not preserve order, so I'm now using distinct // that just cost me 3 bytes :(

Answer 30

PHP, 37 Bytes

Assuming $s is the input string.

print_r(array_flip(explode(' ',$s)));