7
\$\begingroup\$

Background

Benford's Law is freaky!

Of all the numbers in the world, roughly 30% of them start with a 1! 17.6% start with a 2, 12.5% start with a 3, and so on, and so on.

For example, take a look at the following graph. Take the largest 120000 towns in the world, take their altitude, and voila - they obey Benford's law. Divide these numbers by ~3 to convert them into meters, and amazingly, they still obey this law!

Distribution of first digits of altitude of top 120,000 towns in the World

In fact, it can be generalized past the first digit:

enter image description here

Source


Challenge

So I wondered, how does this apply to letters and words?

Your challenge is to parse the big list of words below and formulate the table above (printing a matrix will suffice). Unlike this work, you will not only need to find the occurrence of the first letter, but also the second, third, fourth, and fifth. If a word does not have the nth digit, you can ignore that word (i.e. if you are compiling the 5th column (looking at the fifth digit), you should ignore all words less than 5 characters long)

Big List of Words: https://raw.githubusercontent.com/kevina/wordlist/master/alt12dicts/2of4brif.txt

Due to suggestion in the comments, this contest is now a code golf competiton, so shortest code in bytes wins. The entry must be a complete program and must output the table specified above. Built-ins are allowed.

Good luck!

\$\endgroup\$
  • 2
    \$\begingroup\$ I think it's confusing that you have a table of numbers and talk about Benford's Law, but the challenge isn't about that at all. (Benford's Law applies to numbers, not to words.) \$\endgroup\$ – mbomb007 Sep 11 '15 at 18:34
  • \$\begingroup\$ Can I just output a 2D matrix of the data (percentages)? \$\endgroup\$ – mbomb007 Sep 11 '15 at 19:17
  • \$\begingroup\$ @mbomb007 Surely, I'd assume the rows are a-z, and the columns are char# 1-5? Preferably, it would say what the rows/columns are next to/on top of the data \$\endgroup\$ – dberm22 Sep 11 '15 at 19:19
  • \$\begingroup\$ Incidentally, Benford's law comes from the fact that units are arbitrary, and so the distribution should stay the same if you change the units. Since words don't have units, and most certainly can't be converted to 'different units', there's really no way to have an equivalent pattern for words. On the other hand, there might be something possible within cryptography... (that is, cryptography as the equivalent of a change of units) \$\endgroup\$ – Glen O Sep 12 '15 at 14:28
  • \$\begingroup\$ Challenge seems to relate more to Zipf's Law \$\endgroup\$ – Poke Jun 27 '16 at 18:23
1
\$\begingroup\$

R: 88, 95, or 126 bytes

z=sapply(readr::read_lines("http://tinyurl.com/pshoqyo"),Vectorize(substr),1:5,1:5)
y=apply(z,1,table,exclude="")
y/colSums(y)

That's 126 bytes where that the data is downloaded from the provided URL (a tinyurl equivalent to be exact).

I can shorten the code to 88 bytes if you allow to read the input from stdin:

z=sapply(scan(,""),Vectorize(substr),1:5,1:5)
y=apply(z,1,table,exclude="")
y/colSums(y)

or 95 bytes if the data is provided via a t.txt file:

z=sapply(scan("t.txt",""),Vectorize(substr),1:5,1:5)
y=apply(z,1,table,exclude="")
y/colSums(y)

The output:

          aah         <NA>        <NA>        <NA>         <NA>
a 0.054250087 0.1366519284 0.091410860 0.062576399 0.0656915218
b 0.057777336 0.0070112047 0.023141714 0.020833697 0.0171560104
c 0.099582311 0.0178502654 0.053853798 0.046119198 0.0308013314
d 0.065925417 0.0080894018 0.032457317 0.040240449 0.0279619439
e 0.044177897 0.1499494924 0.068491563 0.115709739 0.1336258142
f 0.045208406 0.0035603690 0.019392694 0.018553559 0.0122387071
g 0.033566827 0.0028011669 0.026038614 0.031346432 0.0212462947
h 0.035818471 0.0396356269 0.009065709 0.021577492 0.0288307086
i 0.042716723 0.1027737583 0.061586103 0.079272029 0.1020188291
j 0.009065709 0.0002815175 0.002997334 0.001872970 0.0003181567
k 0.006557703 0.0032954113 0.004342637 0.025971634 0.0180791157
l 0.030602613 0.0469071140 0.057569618 0.058683031 0.0625465746
m 0.050139230 0.0185368141 0.045014732 0.032771954 0.0237633928
n 0.017230697 0.0724559354 0.080629937 0.054978721 0.0516144003
o 0.026307990 0.1417192442 0.067001176 0.058609030 0.0616554195
p 0.082236243 0.0202858231 0.041453955 0.040891509 0.0257849684
q 0.004835478 0.0032155407 0.002880155 0.002475636 0.0009935913
r 0.056736060 0.0873758770 0.099060304 0.069567953 0.0749333466
s 0.124834642 0.0096061647 0.069021478 0.057992614 0.0674600544
t 0.053731760 0.0237137132 0.066835577 0.085112378 0.0682529848
u 0.022802921 0.0768211701 0.043144600 0.036018688 0.0333164804
v 0.014224916 0.0093814228 0.018101442 0.014540002 0.0068392204
w 0.026536498 0.0070329376 0.010477867 0.009521917 0.0073194562
x 0.000184196 0.0124653498 0.004669879 0.001490387 0.0012265464
y 0.003312471 0.0105155083 0.009124480 0.006613406 0.0145012475
z 0.001506947 0.0003146373 0.003828814 0.004336978 0.0029114873
\$\endgroup\$
  • \$\begingroup\$ You seem to have the shortest answer. Please provide a printout and I will accept your solution! \$\endgroup\$ – dberm22 Sep 13 '15 at 14:30
5
\$\begingroup\$

Mathematica 246 bytes

The probabilities are based on all the words in the file.

p=Prepend;t=Transpose;d="Digit"; 
p[t[p[N[#/Total@#,5]&/@ t[Sort[Cases[Tally[Position[Characters
@Import["t.txt","List"],#][[All,2]]],{n_,_} /;n<6]]/.{_,k_}->k&/@
(c=CharacterRange["a","z"])],c]],{d,"1st "<> d,"2nd "<>d,"3rd "<>d,"4th "<>d,"5th "<>d}]
//Grid

table


In the given list of 60387 words, the occurrences are distributed as follows. Letters "s" and "c" are the most likely to appear at the beginning of a word (7455 and 6008, respectively). Only 11 words begin with the letter "x".

frequencies


Without headers in table 162 bytes

t=Transpose;t[N[#/Total@#]&/@t[Sort[Cases[Tally[Position[Characters@Import["t.txt", "List"], #][[All,2]]],{n_,_} /;n<6]]/.{_,k_}->k&/@CharacterRange["a","z"]]]//Grid
\$\endgroup\$
3
\$\begingroup\$

MATLAB, 113 117 bytes

I made this a script that loads our text file, conveniently called t.txt, and prints the table without headers to the screen.

Old version (113 bytes) contained an error:

D=max(1,strvcat(importdata('t.txt'))-95);F=[];for n=1:5,F=[F,accumarray(D(:,n),1)/length(D)];end;disp(F(2:end,:))

New corrected version is slightly longer unfortunately.

D=max(1,strvcat(importdata('t.txt'))-95);F=[];for n=1:5,E=D(:,n);F=[F,accumarray(E,1)/sum(E~=1)];end;disp(F(2:end,:))

Giving the output:

0.0543    0.1367    0.0914    0.0626    0.0657
0.0578    0.0070    0.0229    0.0200    0.0181
0.0995    0.0177    0.0512    0.0466    0.0324
0.0652    0.0077    0.0325    0.0407    0.0294
0.0420    0.1499    0.0686    0.1169    0.1391
0.0452    0.0036    0.0194    0.0186    0.0122
0.0336    0.0028    0.0258    0.0301    0.0224
0.0358    0.0392    0.0086    0.0218    0.0304
0.0422    0.0976    0.0616    0.0802    0.1073
0.0086    0.0003    0.0030    0.0019    0.0003
0.0066    0.0033    0.0043    0.0260    0.0181
0.0306    0.0469    0.0570    0.0564    0.0658
0.0501    0.0183    0.0428    0.0331    0.0250
0.0170    0.0688    0.0807    0.0556    0.0543
0.0250    0.1417    0.0671    0.0592    0.0642
0.0822    0.0203    0.0415    0.0409    0.0258
0.0048    0.0032    0.0029    0.0024    0.0010
0.0567    0.0864    0.0942    0.0703    0.0789
0.1235    0.0091    0.0691    0.0586    0.0710
0.0510    0.0237    0.0669    0.0860    0.0711
0.0228    0.0768    0.0431    0.0360    0.0333
0.0142    0.0094    0.0179    0.0140    0.0072
0.0265    0.0070    0.0100    0.0096    0.0077
0.0002    0.0118    0.0047    0.0015    0.0013
0.0031    0.0105    0.0091    0.0067    0.0151
0.0015    0.0003    0.0038    0.0043    0.0029
\$\endgroup\$
  • \$\begingroup\$ Nice use of accumarray! \$\endgroup\$ – Luis Mendo Sep 12 '15 at 11:05
  • \$\begingroup\$ Kind of cheating by downloading the file first, but that's alright. I bet you can remove the extension and save four chars. \$\endgroup\$ – dberm22 Sep 12 '15 at 11:06
  • \$\begingroup\$ @dberm22 Do you mean it should be obtained directly from the intenet? \$\endgroup\$ – Luis Mendo Sep 12 '15 at 11:07
  • \$\begingroup\$ It looks like you are not disregarding short in the count of each position, so the computed frequencies are too low \$\endgroup\$ – Luis Mendo Sep 12 '15 at 11:26
  • \$\begingroup\$ @LuisMendo thanks for pointing that out. Fixed it now at the cost of 4 bytes. \$\endgroup\$ – slvrbld Sep 12 '15 at 17:06
2
\$\begingroup\$

Python 3, 143 129 bytes

This is taking a long time to run on the provided data. I have no idea if it will finish any time soon.

I think the interpreter I'm using is refusing to run the program because of the input size. My program did the first 1000 words in 1 second.

Edit: Now stops at 5 letters/columns.

s=input().split()
l=96
while l<122:l+=1;print([sum(i<len(w)and w[i]==chr(l)for w in s)/sum(i<len(w)for w in s)for i in range(5)])

With the letters on the left (136):

s=input().split()
l=96
while l<122:l+=1;print(chr(l),[sum(i<len(w)and w[i]==chr(l)for w in s)/sum(i<len(w)for w in s)for i in range(5)])

Example Output

This is the output for input of hello\nworld:

[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.5]
[0.0, 0.5, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.5, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.5, 1.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.5, 0.0, 0.0, 0.5]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.5, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.5, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]

Output for the first 2000 words (not so nice to look at :/):

[1.0, 0.0015, 0.082164328657314628, 0.045523520485584217, 0.067875647668393782]
[0.0, 0.11899999999999999, 0.027555110220440882, 0.0096105209914011131, 0.017616580310880828]
[0.0, 0.1565, 0.096192384769539077, 0.015680323722812341, 0.046632124352331605]
[0.0, 0.14499999999999999, 0.03406813627254509, 0.019221041982802226, 0.042487046632124353]
[0.0, 0.019, 0.04458917835671343, 0.17653009610520992, 0.098963730569948186]
[0.0, 0.050000000000000003, 0.033066132264529056, 0.0030349013657056147, 0.0020725388601036268]
[0.0, 0.049000000000000002, 0.037575150300601205, 0.0096105209914011131, 0.03367875647668394]
[0.0, 0.0030000000000000001, 0.016533066132264528, 0.032372281234193223, 0.021243523316062177]
[0.0, 0.052499999999999998, 0.045591182364729456, 0.13606474456246839, 0.09689119170984456]
[0.0, 0.00050000000000000001, 0.025050100200400802, 0.0, 0.0]
[0.0, 0.00050000000000000001, 0.0075150300601202402, 0.00050581689428426911, 0.00051813471502590671]
[0.0, 0.123, 0.050601202404809621, 0.084977238239757211, 0.034715025906735753]
[0.0, 0.094, 0.036072144288577156, 0.027314112291350532, 0.030569948186528497]
[0.0, 0.1865, 0.03406813627254509, 0.030349013657056147, 0.080310880829015538]
[0.0, 0.0, 0.077655310621242479, 0.11987860394537178, 0.053886010362694303]
[0.0, 0.0, 0.027555110220440882, 0.018715225088517955, 0.013471502590673576]
[0.0, 0.0, 0.014529058116232466, 0.0015174506828528073, 0.0041450777202072537]
[0.0, 0.0, 0.095190380761523044, 0.076884167931208905, 0.11968911917098446]
[0.0, 0.0, 0.039579158316633264, 0.036418816388467376, 0.078238341968911912]
[0.0, 0.0, 0.10671342685370741, 0.039453717754172987, 0.077720207253886009]
[0.0, 0.0, 0.030561122244488977, 0.098128477491148211, 0.029015544041450778]
[0.0, 0.0, 0.033567134268537073, 0.0015174506828528073, 0.013989637305699482]
[0.0, 0.0, 0.00050100200400801599, 0.0075872534142640367, 0.0041450777202072537]
[0.0, 0.0, 0.0, 0.0, 0.0067357512953367879]
[0.0, 0.0, 0.002004008016032064, 0.0040465351542741529, 0.024352331606217616]
[0.0, 0.0, 0.0015030060120240481, 0.0050581689428426911, 0.0010362694300518134]

Ungolfed

s=input().split()
m=[]
l=96
while l<122:
    l+=1;r=[]
    for i in range(max(map(len,s))):                # Max length of words in list
        L=sum(i<len(w)for w in s)                   # How many words of this length
        f=sum(i<len(w)and w[i]==chr(l)for w in s)   # Number of char in that pos
        r+=[f/L]
    m+=[r]
for r in m:print(r)
\$\endgroup\$
  • \$\begingroup\$ What percent of the words began with "a"? (I'm having trouble understanding the table.) \$\endgroup\$ – DavidC Sep 11 '15 at 21:12
  • \$\begingroup\$ @DavidCarraher 100%. I only have output for the first 2000 words (all beginning with a). Neither interpreter I tried (repl.it and ideone) will work for much larger input. \$\endgroup\$ – mbomb007 Sep 11 '15 at 21:22
  • \$\begingroup\$ I see. Why not save the file on your hard disk, then import the file and work on it? \$\endgroup\$ – DavidC Sep 11 '15 at 21:52
  • \$\begingroup\$ @DavidCarraher Because many people cough are at work cough cough. >.> \$\endgroup\$ – mbomb007 Sep 11 '15 at 21:58
2
\$\begingroup\$

bash with gawk 4 and sort, 125

Pretty much what awk was made for

awk '{for(;NF;c[NF--]++)a[$NF][NF]++}END{for(i in a){printf i;for(j=0;j++<5;)printf" %.9f",a[i][j]/c[j];print""}}' FS= wordlist.txt|sort

"wordlist.txt" needs to be replaced by the name of your file containing the list of words, and I counted the whole filename as one character, because it could be ;)

Be sure to leave the space between FS= and the filename.

It takes into account, that not all words have five letters. So for the average of the second to fifth position it divides only by the number of words with at least that much letters.

awk part readable

{
    for(;NF;c[NF--]++)    # count number of words with at least that length
        a[$NF][NF]++      # count occurences of character at that position
}

END{
    for(i in a)
    {
        printf i;
        for(j=0;j++<5;)
            printf" %.5f",a[i][j]/c[j];    # print averages
        print""
    }
}

You can easily utilize the sort to gather some information. For instance the option -rk2 sorts by the most common letter in the first position, -rk3 sorts by the most common letter in the second position and so on.

Output without sort options

a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522
b 0.057777336 0.007004819 0.022906584 0.020010382 0.018061682
c 0.099491612 0.017652806 0.051200027 0.046635074 0.032427344
d 0.065196151 0.007683773 0.032486906 0.040690567 0.029411252
e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
f 0.045208406 0.003560369 0.019392694 0.018553559 0.012238707
g 0.033566827 0.002798616 0.025774050 0.030107671 0.022367893
h 0.035785848 0.039197178 0.008618975 0.021818852 0.030352691
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
j 0.008611125 0.000281518 0.003000066 0.001892195 0.000331247
k 0.006557703 0.003295411 0.004342637 0.025971634 0.018079116
l 0.030602613 0.046864391 0.056984685 0.056363971 0.065848428
m 0.050093563 0.018331760 0.042796526 0.033138532 0.025017870
n 0.017040091 0.068822760 0.080703441 0.055593697 0.054289649
o 0.024988822 0.141719244 0.067062256 0.059210636 0.064192193
p 0.082236243 0.020285823 0.041453955 0.040891509 0.025784968
q 0.004835478 0.003212612 0.002850892 0.002377803 0.001046043
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
s 0.123453723 0.009124480 0.069084400 0.058641303 0.070956607
t 0.051037475 0.023713713 0.066896506 0.085986035 0.071061211
u 0.022802921 0.076821170 0.043144600 0.036018688 0.033316480
v 0.014224916 0.009372878 0.017917523 0.013965405 0.007200265
w 0.026512329 0.006955139 0.009961546 0.009628426 0.007705853
x 0.000182158 0.011840297 0.004674136 0.001507058 0.001290120
y 0.003146373 0.010515508 0.009132799 0.006681291 0.015097892
z 0.001506947 0.000314637 0.003828814 0.004336978 0.002911487

Output sorted after occurrence as first letter

s 0.123453723 0.009124480 0.069084400 0.058641303 0.070956607
c 0.099491612 0.017652806 0.051200027 0.046635074 0.032427344
p 0.082236243 0.020285823 0.041453955 0.040891509 0.025784968
d 0.065196151 0.007683773 0.032486906 0.040690567 0.029411252
b 0.057777336 0.007004819 0.022906584 0.020010382 0.018061682
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522
t 0.051037475 0.023713713 0.066896506 0.085986035 0.071061211
m 0.050093563 0.018331760 0.042796526 0.033138532 0.025017870
f 0.045208406 0.003560369 0.019392694 0.018553559 0.012238707
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
h 0.035785848 0.039197178 0.008618975 0.021818852 0.030352691
g 0.033566827 0.002798616 0.025774050 0.030107671 0.022367893
l 0.030602613 0.046864391 0.056984685 0.056363971 0.065848428
w 0.026512329 0.006955139 0.009961546 0.009628426 0.007705853
o 0.024988822 0.141719244 0.067062256 0.059210636 0.064192193
u 0.022802921 0.076821170 0.043144600 0.036018688 0.033316480
n 0.017040091 0.068822760 0.080703441 0.055593697 0.054289649
v 0.014224916 0.009372878 0.017917523 0.013965405 0.007200265
j 0.008611125 0.000281518 0.003000066 0.001892195 0.000331247
k 0.006557703 0.003295411 0.004342637 0.025971634 0.018079116
q 0.004835478 0.003212612 0.002850892 0.002377803 0.001046043
y 0.003146373 0.010515508 0.009132799 0.006681291 0.015097892
z 0.001506947 0.000314637 0.003828814 0.004336978 0.002911487
x 0.000182158 0.011840297 0.004674136 0.001507058 0.001290120

Second letter

e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
o 0.024988822 0.141719244 0.067062256 0.059210636 0.064192193
a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102

Third letter

r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522
n 0.017040091 0.068822760 0.080703441 0.055593697 0.054289649
s 0.123453723 0.009124480 0.069084400 0.058641303 0.070956607
e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764

Fourth letter

e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
t 0.051037475 0.023713713 0.066896506 0.085986035 0.071061211
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522

Fifth letter

e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
t 0.051037475 0.023713713 0.066896506 0.085986035 0.071061211
s 0.123453723 0.009124480 0.069084400 0.058641303 0.070956607

So it looks like there is no Benfords Law of Words ;)

\$\endgroup\$
  • \$\begingroup\$ Great outputs/ sorting. It certainly looks like there is no Benford's Law of Words. Oh well =/ \$\endgroup\$ – dberm22 Sep 14 '15 at 20:10
  • \$\begingroup\$ Well, thanks. I was forced to use the formatting string there, since some averages were too short for just using tab, which would have messed up the whole table. About the law: I like it netter that way. With numbers it is at least somehow intuitively comprehensible. With letters it would be only spooky. ;) \$\endgroup\$ – Cabbie407 Sep 15 '15 at 4:33
1
\$\begingroup\$

Dyalog APL (76)

⎕ML←3⋄L{R÷+⌿R←0 1↓K[⍋⊃K←,∘≢⌸⍺⌷¨⍵;]}¨(↓(L←⍳5)∘.≤≢¨G)/¨⊂G←W⊂⍨10≠W←83 ¯1⎕MAP'w'

This assumes the input is in a file called w in the working directory.

Output:

      ⎕ML←3⋄L{R÷+⌿R←0 1↓K[⍋⊃K←,∘≢⌸⍺⌷¨⍵;]}¨(↓(L←⍳5)∘.≤≢¨G)/¨⊂G←W⊂⍨10≠W←83 ¯1⎕MAP'w'
 0.05425008694    0.1366519284     0.09141085991   0.06257639947   0.06569152182  
 0.05777733618    0.007004818918   0.02290658357   0.02001038196   0.01806168169  
 0.09949161243    0.01765280607    0.05120002652   0.04663507426   0.03242734357  
 0.06519615149    0.007683772997   0.03248690579   0.04069056749   0.02941125194  
 0.04196267409    0.1499494924     0.06855400119   0.1168974698    0.1391237644   
 0.04520840578    0.003560368954   0.01939269376   0.01855355917   0.01223870709  
 0.0335668273     0.002798615596   0.02577405026   0.03010767093   0.02236789344  
 0.03578584795    0.0391971782     0.008618975005  0.02181885162   0.03035269095  
 0.04224419163    0.09762034875    0.06164224624   0.08015874345   0.1073066127   
 0.008611124911   0.0002815175452  0.0030000663    0.001892195114  0.000331247058 
 0.006557702817   0.003295411264   0.004342637406  0.02597163382   0.01807911574  
 0.03060261315    0.04686439134    0.05698468474   0.05636397127   0.06584842832  
 0.05009356318    0.01833176015    0.04279652589   0.03313853213   0.02501786991  
 0.01704009141    0.06882275987    0.08070344096   0.05559369715   0.0542896494   
 0.0249888221     0.1417192442     0.06706225552   0.05921063648   0.06419219303  
 0.0822362429     0.02028582311    0.04145395478   0.04089150857   0.02578496836  
 0.004835477835   0.003212611986   0.002850891732  0.002377802709  0.001046043341 
 0.05668438571    0.08640932651    0.09417887688   0.070346121     0.07888910197  
 0.1234537235     0.009124480435   0.06908439966   0.05864130344   0.07095660664  
 0.05103747495    0.02371371322    0.066896506     0.0859860346    0.07106121097  
 0.02280292116    0.07682117012    0.04314459988   0.03601868752   0.03331648041  
 0.01422491596    0.009372878269   0.01791752304   0.01396540465   0.007200264998 
 0.02651232881    0.006955139351   0.009961546112  0.009628426464  0.007705852612 
 0.0001821584116  0.01184029675    0.004674136445  0.001507058055  0.001290120121 
 0.003146372564   0.0105155083     0.009132798515  0.006681290711  0.01509789222  
 0.001506946859   0.0003146372564  0.003828813896  0.004336978181  0.002911487299 

Explanation:

  • ⎕ML←3: set migration level to 3. (Among other things, this changes the behaviour of from 'split' into 'partition', so we don't have to get rid of the linefeeds.)
  • W←83 ¯1⎕MAP'w': read the file w as a string of ASCII values, store it in W.
  • G←W⊂⍨10≠W: partition W on linefeeds (character 10) and store the words in G.
  • (↓(L←⍳5)∘.≤≢¨G)/¨⊂G: store the numbers 1 to 5 in L, and for each number, select those words from G which have at least that many letters.
  • L{...: for each of the numbers 1 to 5, and the matching list of words,:
    • ⍺⌷¨⍵: for each word, select the ⍺-th letter
    • K←,∘≢⌸: for each letter, find out how many times it occurs, store this in K.
    • K[⍋⊃K...;]: sort K by the value of the letter (so a is at the top, and z at the bottom).
    • R←0 1↓: drop the first column (the letter), leaving only the amount of times it occurred. Store this in R.
    • R÷+⌿R: divide the value for each letter by the total amount.
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.