Turn a sentence into a dictionary

For this challenge, you will be given a long string as an input. With this string, you must create a dictionary.

Details:

This dictionary will contain all the words from the string - a word being any sequence of letters surrounded by non-letter characters (ASCII). Letter case of the entries of the dictionary should always be consistent (i.e. all lower case or all upper case, or only the first letter of each word is capitalised). The dictionary can be a list, array, or any other type so long as the words are distinct from each other. Finally, as this is a dictionary, the words must be in alphabetical order, and there may not be any duplicates.

You can assume no contractions will be used (the input will never contain it's, there's, etc.), and that there will always be at least one word

Some examples:

"this is an eXaMpLe of an Input" -> ["an","example","input","is","of","this"]
"Here is another example of an input, this time with Punctuation!" -> ["an","another","example","here","input","is","of","punctuation","this","time","with"]
"290(&79832aAa(*& *79 --=BBBb)bbBb ( x )*d -cxaAa_" -> ["aaa","bbbb","cxaaa","d","x"]


Scoring

Shortest code wins

• Are we guaranteed the input will contain at least one word? Jun 19 '20 at 16:59
• May we include the empty word at the start of our dictionary? Jun 19 '20 at 17:27
• yes, it's guaranteed at least one word, I'll update it Jun 19 '20 at 19:17
• Note that in standard computer-science terminology, a "dictionary" data structure is a map: each entry has associated storage of some type beyond the mere fact that it's present. You're using the common English meaning, which is fine, but real dictionaries have definitions. This is just a word-list. Probably fine to leave the title and description as-is, but in programming terms the problem is to tokenize a string into an ordered set of words. Jun 19 '20 at 23:00
• Could you answer @JonathanAllan's question as well, please? And, if we can include it, must we do so consistently? Jun 20 '20 at 12:41

JavaScript (ES6),  57  51 bytes

Returns a Set of words in lower case.

NB: In JS, the elements of a Set are guaranteed to appear in insertion order. In this case, this is the order defined in the sorted array the Set is built from.

s=>new Set(s.toLowerCase().match(/[a-z]+/g).sort())


Try it online!

Stax, 7 bytes

èñ≤!á~¬


Run and debug it

The output dictionary is produced as a space separated word list.

• It would be amazing if Stax's source compression also does Huffman coding.
– user92069
Jun 20 '20 at 0:53
• It doesn't now, but that's an interesting idea... Jun 20 '20 at 3:09

Pyth, 16 bytes

S{c:r0Q"[^a-z]"d


Try it online!

S{c:r0Q"[^a-z]"d   Implicit: Q=input(), d=" "
r0Q            Convert input to lowercase
:   "[^a-z]"d   Regex replace non-alphas with a space
c                Split on spaces
{                 Deduplicate
S                  Sort, implicit print


p$_.upcase.scan(/[A-Z]+/).sort|[]  Try it online! gawk -F[^a-zA-Z]+, 93 bytes {for(i=1;i<=NF;i++){if($i!=""){a[tolower($i)]=1}}n=asorti(a,b);for(j=1;j<=n;j++){print b[j]}}  Try it online! Works for GNU AWK, not regular AWK, due to the use of the asorti function. The input is split on anything which isn't a letter, leaving the words in $1, $2, etc. We iterate over the numbered variables, and, if they aren't equal to the empty string, we put them, lowercased, into an associative array a as an index. Once done, we sort the indices of the array a and put the result into an array b. Finally, we print the elements of b, in order. We need the test for an empty string, because if the input string ends with a non-letter, the last numbered variable will be an empty string. • This trims off some bytes... {for(;i<NF;)$++i?a[tolower($i)]=1:0;for(;j<asorti(a,b);)print b[++j]} May 17 at 10:17 Retina, 2423 19 bytes thanks to @FryAmTheEggman and @Neil for -5 bytes! TLlpll¶ D G. O  Try it online! TLlpll¶ convert the letters to lowercase and everything else to newlines. D deduplicates lines. G. removes empty lines. O sorts the lines. Japt v2.0a0, 12 11 bytes Two bytes can be saved if we can include the empty string in the "dictionary". Will update explanation upon confirmation. v q\L f â n  Try it - Includes all test cases v f"%a+" â n :Implicit input of string v :Lowercase f :Match "%a+" : RegEx /[a-z]/g â :Deduplicate n :Sort  Python 3, 89 $$\\cdots\$$ 61 60 bytes Saved 2 bytes thanks to 640KB!!! lambda s:sorted({*re.findall("[a-z]+",s.lower())}) import re  Try it online! • Why the downvote? Jun 19 '20 at 22:52 Vyxal, 13 bytes ⇩[a-z]+$ẎUs


Try it Online!

-3 thanks to Razetime.

+1 because I forgot to sort it.

• May 17 at 9:26
• May 17 at 9:26
• @Razetime Nice, I forgot I could lowercase it first! May 17 at 9:27
• Unless I'm missing something, this doesn't seem to work for testcase #3. Edit: It also doesn't seem to sort the list at any point. May 17 at 16:42
• @Underslash Oops, forgot to sort the list. Fixed now. May 17 at 19:28

perl -Mfeature=say -MList::Util=uniq -n, 31 bytes

say for sort@e=uniq lc=~/\pL+/g


Try it online!

Lowercases the input string (read from STDIN), extracts sequences of letters, removes duplicates, sorts and prints them.

• Nice! I'm sure there's a way to overuse -F which I might play with later! For this, you can save two bytes removing @e= by swapping sort and uniq: Try it online! Jun 20 '20 at 8:58

Raku, 29 bytes

{sort unique m:g/<:L>+/».lc}


Try it online!

m:g/<:L>+/ returns an array of match objects, one for each sequence of letters in the input. ».lc calls the lc (lowercase) method on each of those match objects, coercing them to strings in the process. sort and unique are self-explanatory, hopefully.

Red, 93 bytes

func[s][a: charset[#"a"-#"z"]sort unique parse lowercase s[collect[any[keep some a | skip]]]]


Try it online!

Perl 5MList::Util=uniq -F'[^a-zA-Z]+', 27 bytes

say for uniq sort map{lc}@F


Try it online!

05AB1E, 9 bytes

05AB1E has a built-in that I don't even know!

l.γa}ʒa}ê


Try it online!

Explanation

l         Lowercase
.γ       Group by:
a}     Is alphabetic?
ʒa}  Filter: is alphabetic?
ê sorted uniquify


PostgreSQL, 80 bytes

SELECT regexp_split_to_table(lower($1),'[^a-z]')UNION SELECT''ORDER BY 1OFFSET 1  Input is given as a query parameter and output is given as one word per row. Changing lower to upper outputs in uppercase instead. initcap also works but is 2 more bytes. Explanation regexp_split_to_table splits a string based on the regular expression provided. UNION SELECT'' is used to add the empty string and also remove duplicates. ORDER BY 1 is used to sort the results OFFSET 1 is used to not output the empty string. If the empty string is sometimes allowed, the following 67 byte solution works: SELECT DISTINCT regexp_split_to_table(lower($1),'[^a-z]')ORDER BY 1


Husk, 13 12 bytes

↓¬uO†_mf√ġK√


Try it online!

-1 byte from Jo King.

Explanation

↓¬uO†_mf√ġK√
ġK√ Group the input on non alphabet chars
mf√    filter out non-alphabet chars.
†_       convert all characters to lowercase
O         sort in ascending order
u          uniquify
↓¬           drop all empty strings at the beginning

• @Jo King ah, it works since they're ordered. Nice find. Oct 4 '20 at 13:37

Burlesque, 16 bytes

zz"[a-z]+"~?NB><


Try it online!

Explanation:

zz               # Lowercase input
"[a-z]+"~?     # Get list of all regex matches
NB   # Remove duplicates
>< # Sort


Charcoal, 39 bytes

Ｆ⪫  ↧Ｓ¿№βι⊞§υ±¹ι⊞υ⟦⟧≔⟦⟧ζＷ⁻υζ⊞ζ⌊ι✂Ｅζ⪫ιω¹


Try it online! Link is to verbose version of code. Explanation:

Ｆ⪫  ↧Ｓ


Wrap the lowercased input in spaces and loop over the characters.

¿№βι


If this is a letter, ...

⊞§υ±¹ι


... then append it to the last entry, ...

⊞υ⟦⟧


... otherwise start a new entry.

≔⟦⟧ζ


Start a list of unique entries.

Ｗ⁻υζ


While there are more entries, ...

⊞ζ⌊ι


... add the lexicographically first entry to the list of unique entries. This also ends up sorting the deduplicated list.

✂Ｅζ⪫ιω¹


Join the entries back into strings and print all but the first (which is always empty).

Jelly, 14 bytes

13 bytes if we may include the empty word as a leading entry - exclude the trailing Ḋ (maybe?).
11 bytes if we may output an empty word, but not always - exclude ⁸Ż also (probably not).

e€ØẠa⁸Żṣ0ŒlQṢḊ


A monadic Link accepting a list of characters which yields a list of lists of characters.

Try it online!

How?

e€ØẠa⁸Żṣ0ŒlQṢḊ - Link: list of charachters, S       e.g. "I((or))a"
ØẠ           - Latin alphabet characters               "AB...Zab...z"
e€             - for each (c in S): exists in (ØẠ)?      [1,0,0,1,1,0,0,1]
a⁸         - logical AND (vectorises) (with chain's left argument = S)       ['i',0,0,'o','r',0,0,'a']
Ż        - prepend a zero (for S like "abc")       [0,1,0,0,1,1,0,0,1]
ṣ0      - split at zeros                          [[],['I'],[],['o','r'],[],['a']]
Œl    - lower-case                              [[],['i'],[],['o','r'],[],['a']]
Q   - deduplicate                             [[],['i'],['o','r'],['a']]
Ṣ  - sort                                    [[],['a'],['i'],['o','r']]
Ḋ - dequeue (remove the empty word)         [['a'],['i'],['o','r']]


J, 44 bytes

Drops the potential empty word, otherwise -4 bytes.

}.@~.@/:~@([;._2~2|1+'@Z'&I.)@toupper@,&'  '


Try it online!

How it works

}.@~.@/:~@([;._2~2|1+'@Z'&I.)@toupper@,&'  '
,&'  ' append two spaces
toupper        one of the few functions, non
J user can guess by name :)
(          '@Z'&I.)                index into intervals …@](A…Z](_…
2|1+                        add 0 and mod 2, so A-Z is 0, else 1
[;._2~                            partition the string into groups,
each group ends with 1 (exclusive).
the end is based on the last item,
which is space -> 1
/:~                                    sort the strings
~.                                        remove duplicates
}.                                           remove head,
which will be the space group


MATL, 13 8 bytes

5 bytes removed thanks to @Sanchises!

k3Y4XXuS


Output is in lowercase.

How it works

k          % Implicit input: string. Convert to lowercase
3Y4        % Push string '[A-Za-z]+' (predefined literal)
XX         % Regexp match. Gives a cell array of substrings
u          % Unique
S          % Sort. Implicit display

• Why not k3Y4XXuS? Jun 20 '20 at 12:47
• @Sanchises Thank you! I had completely forgotten about the Y4 literals Jun 20 '20 at 12:57

R, 60 bytes

sort(setdiff(strsplit(tolower(readline()),"[^a-z]")[[1]],"")


Used setdiff to remove possible empty strings, but it deduplicates as well.

Haskell, 88 bytes

import Data.Char
import Data.List
s=sort.nub.words.map f
f x|isAlpha x=toLower x
f _=' '


Replace non-alpha characters with spaces so we can use words to split it.

Try it online!

• If you remove the Data.Char import, f x|x>'',x<'{'=x|x>'@',x<'['=[x..]!!32;f _=' ' saves a single byte :).
– Lynn
Oct 5 '20 at 13:41

Rust, 201 bytes

fn main(){let y=&mut"".into();std::io::stdin().read_line(y);y.retain(|c|c.is_alphabetic()||c==' ');*y=y.to_lowercase();let mut v=y.split(" ").collect::<Vec<_>>();v.sort();v.dedup();println!("{:?}", v)}


Try it online!

fn main(){
let y = &mut"".into();std::io::stdin().read_line(y); //Get input into string
y.retain(|c|c.is_alphabetic()||c==' '); //Retain spaces and letters in string
*y=y.to_lowercase(); //Convert to lowercase
let mut v=y.split(" ").collect::<Vec<_>>(); //Split string by space and collect into vector
v.sort(); //sort Vec
v.dedup(); //delete duplicates (only works on sorted Vec)
println!("{:?}", v) //debug-print vector (because no std::fmt::Display for Vec<&str>)
}


Haskell, 93 bytes

import Data.List
s=sort.nub.words.map(((do c<-"q69";(" "<*['1'..c])++['a'..'z'])!!).fromEnum)


Try it online!

It's not shorter than the other Haskell answer, but I thought this approach was pretty fun.

It creates a big string like "... abc...xyz abc...xyz ..." so that indexing into this string performs the same mapping as f in the other answer.

I'm including s= because the other answer also is.

QuadS, 15 bytes (SBCS)

∪⍵[⍋⍵]
\pL+
\l&


Try it online!

∪Unique elements of
⍵[] the data reordered to:
⍋⍵ the order that would sort it ascending

where the data is:

\pL+ all runs of Letters

\l& mapped to lowercase

$a=array_filter(array_unique(preg_split("/[^a-z]+/",strtolower($argn))));sort($a);  Try it online! I'm still not satisfied by it, but so far the best I could find.. Yeah array functions names are looooong in PHP :S The question doesn't ask to display the dictionary, so it's in the footer ;P Clojure, 59 46 bytes #(sort(set(re-seq #"[a-z]+"(.toLowerCase %))))  Try it online! 13 bytes saved by NikoNyrh. • Java-interop to .toLowerCase is bit shorter :) Jun 22 '20 at 18:01 • Indeed, I was so frustrated dealing with the fact that Clojure doesn't expose string functions in the core, that didn't even try thinking about interop :) Jun 22 '20 at 20:23 05AB1E (legacy), 6 bytes áмS¡lê  Explanation: á # Only leave letters of the (implicit) input-string м # Remove all those letters from the (implicit) input-string S # Convert the remaining characters to a list of characters ¡ # Split the (implicit) input-string on those characters l # Convert everything to lowercase ê # Sort and uniquify the words  NOTE: This doesn't work in the new version of 05AB1E for two reasons: 1. The ¡ would also keep empty string items. 2. A single word wouldn't be wrapped into a list, so something like input "test" would result in output "est" due to the ê working directly on this single string instead of a list. See the result of all test cases in the new version of 05AB1E. AWK (+"sort" callout), 87 bytes gsub("[^a-zA-Z]"," ")&&$0=tolower($0){for(;$++a;)!b[$a]++?c=c$a"\n":0;printf c|"sort"}


Try it online!

This AWK version is longer than the entry Abigail submitted (once it's golfed a bit) but it shows a trick I thought was worth highlighting.

gsub("[^a-zA-Z]"," ")&&$0=tolower($0){ ... }


The gsub call translates all the non-alphabetic characters to spaces, then the $0=tolower($0) bit does two things. It converts everything to lowercase, then the assignment effectively re-maps the string to positional parameters, dealing with all the added spaces. As a result if $1 was =BBBb)))bbBdb originally, after running through those calls, the data would set $1="BBBd" and $2="bbDdb". Once the input line has been cleaned up, for(;$++a;)!b[$a]++?c=c$a"\n":0


loops through all the words via $++a, and for each one increments a frequency counter array for that word b[$a]++. If the value of that frequency counter is 0, the ternary appends the word plus a LF to an accumulator string.

Then the code prints the accumulated string and pipes it to "sort" with this line.

printf c|"sort"
`