Benfords Law of Words

Question

Background

Of all the numbers in the world, roughly 30% of them start with a 1! 17.6% start with a 2, 12.5% start with a 3, and so on, and so on.

For example, take a look at the following graph. Take the largest 120000 towns in the world, take their altitude, and voila - they obey Benford's law. Divide these numbers by ~3 to convert them into meters, and amazingly, they still obey this law!

In fact, it can be generalized past the first digit:

Source

Challenge

So I wondered, how does this apply to letters and words?

Your challenge is to parse the big list of words below and formulate the table above (printing a matrix will suffice). Unlike this work, you will not only need to find the occurrence of the first letter, but also the second, third, fourth, and fifth. If a word does not have the nth digit, you can ignore that word (i.e. if you are compiling the 5th column (looking at the fifth digit), you should ignore all words less than 5 characters long)

Big List of Words: https://raw.githubusercontent.com/kevina/wordlist/master/alt12dicts/2of4brif.txt

Due to suggestion in the comments, this contest is now a code golf competiton, so shortest code in bytes wins. The entry must be a complete program and must output the table specified above. Built-ins are allowed.

Good luck!

I think it's confusing that you have a table of numbers and talk about Benford's Law, but the challenge isn't about that at all. (Benford's Law applies to numbers, not to words.) — mbomb007, Commented Sep 11, 2015 at 18:34
@mbomb007 Surely, I'd assume the rows are a-z, and the columns are char# 1-5? Preferably, it would say what the rows/columns are next to/on top of the data — dberm22, Commented Sep 11, 2015 at 19:19
Incidentally, Benford's law comes from the fact that units are arbitrary, and so the distribution should stay the same if you change the units. Since words don't have units, and most certainly can't be converted to 'different units', there's really no way to have an equivalent pattern for words. On the other hand, there might be something possible within cryptography... (that is, cryptography as the equivalent of a change of units) — Glen O, Commented Sep 12, 2015 at 14:28

Community · Accepted Answer · 2020-06-17 09:04:33Z

Mathematica 246 bytes

The probabilities are based on all the words in the file.

p=Prepend;t=Transpose;d="Digit"; 
p[t[p[N[#/Total@#,5]&/@ t[Sort[Cases[Tally[Position[Characters
@Import["t.txt","List"],#][[All,2]]],{n_,_} /;n<6]]/.{_,k_}->k&/@
(c=CharacterRange["a","z"])],c]],{d,"1st "<> d,"2nd "<>d,"3rd "<>d,"4th "<>d,"5th "<>d}]
//Grid

In the given list of 60387 words, the occurrences are distributed as follows. Letters "s" and "c" are the most likely to appear at the beginning of a word (7455 and 6008, respectively). Only 11 words begin with the letter "x".

Without headers in table 162 bytes

t=Transpose;t[N[#/Total@#]&/@t[Sort[Cases[Tally[Position[Characters@Import["t.txt", "List"], #][[All,2]]],{n_,_} /;n<6]]/.{_,k_}->k&/@CharacterRange["a","z"]]]//Grid

Nice work. Your numbers match mine, too. docs.google.com/spreadsheets/d/… . Good job! — dberm22, Commented Sep 11, 2015 at 22:45

slvrbld · Accepted Answer · 2015-09-12 17:05:45Z

MATLAB, 113 117 bytes

I made this a script that loads our text file, conveniently called t.txt, and prints the table without headers to the screen.

Old version (113 bytes) contained an error:

D=max(1,strvcat(importdata('t.txt'))-95);F=[];for n=1:5,F=[F,accumarray(D(:,n),1)/length(D)];end;disp(F(2:end,:))

New corrected version is slightly longer unfortunately.

D=max(1,strvcat(importdata('t.txt'))-95);F=[];for n=1:5,E=D(:,n);F=[F,accumarray(E,1)/sum(E~=1)];end;disp(F(2:end,:))

Giving the output:

0.0543    0.1367    0.0914    0.0626    0.0657
0.0578    0.0070    0.0229    0.0200    0.0181
0.0995    0.0177    0.0512    0.0466    0.0324
0.0652    0.0077    0.0325    0.0407    0.0294
0.0420    0.1499    0.0686    0.1169    0.1391
0.0452    0.0036    0.0194    0.0186    0.0122
0.0336    0.0028    0.0258    0.0301    0.0224
0.0358    0.0392    0.0086    0.0218    0.0304
0.0422    0.0976    0.0616    0.0802    0.1073
0.0086    0.0003    0.0030    0.0019    0.0003
0.0066    0.0033    0.0043    0.0260    0.0181
0.0306    0.0469    0.0570    0.0564    0.0658
0.0501    0.0183    0.0428    0.0331    0.0250
0.0170    0.0688    0.0807    0.0556    0.0543
0.0250    0.1417    0.0671    0.0592    0.0642
0.0822    0.0203    0.0415    0.0409    0.0258
0.0048    0.0032    0.0029    0.0024    0.0010
0.0567    0.0864    0.0942    0.0703    0.0789
0.1235    0.0091    0.0691    0.0586    0.0710
0.0510    0.0237    0.0669    0.0860    0.0711
0.0228    0.0768    0.0431    0.0360    0.0333
0.0142    0.0094    0.0179    0.0140    0.0072
0.0265    0.0070    0.0100    0.0096    0.0077
0.0002    0.0118    0.0047    0.0015    0.0013
0.0031    0.0105    0.0091    0.0067    0.0151
0.0015    0.0003    0.0038    0.0043    0.0029

Kind of cheating by downloading the file first, but that's alright. I bet you can remove the extension and save four chars. — dberm22, Commented Sep 12, 2015 at 11:06
@dberm22 Do you mean it should be obtained directly from the intenet? — Luis Mendo, Commented Sep 12, 2015 at 11:07
It looks like you are not disregarding short in the count of each position, so the computed frequencies are too low — Luis Mendo, Commented Sep 12, 2015 at 11:26
@LuisMendo thanks for pointing that out. Fixed it now at the cost of 4 bytes. — slvrbld, Commented Sep 12, 2015 at 17:06

mbomb007 · Accepted Answer · 2015-09-11 21:22:16Z

Python 3, 143 129 bytes

~~This is taking a long time to run on the provided data. I have no idea if it will finish any time soon.~~

I think the interpreter I'm using is refusing to run the program because of the input size. My program did the first 1000 words in 1 second.

Edit: Now stops at 5 letters/columns.

s=input().split()
l=96
while l<122:l+=1;print([sum(i<len(w)and w[i]==chr(l)for w in s)/sum(i<len(w)for w in s)for i in range(5)])

With the letters on the left (136):

s=input().split()
l=96
while l<122:l+=1;print(chr(l),[sum(i<len(w)and w[i]==chr(l)for w in s)/sum(i<len(w)for w in s)for i in range(5)])

Example Output

This is the output for input of hello\nworld:

[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.5]
[0.0, 0.5, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.5, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.5, 1.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.5, 0.0, 0.0, 0.5]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.5, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.5, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]
[0.0, 0.0, 0.0, 0.0, 0.0]

Output for the first 2000 words (not so nice to look at :/):

[1.0, 0.0015, 0.082164328657314628, 0.045523520485584217, 0.067875647668393782]
[0.0, 0.11899999999999999, 0.027555110220440882, 0.0096105209914011131, 0.017616580310880828]
[0.0, 0.1565, 0.096192384769539077, 0.015680323722812341, 0.046632124352331605]
[0.0, 0.14499999999999999, 0.03406813627254509, 0.019221041982802226, 0.042487046632124353]
[0.0, 0.019, 0.04458917835671343, 0.17653009610520992, 0.098963730569948186]
[0.0, 0.050000000000000003, 0.033066132264529056, 0.0030349013657056147, 0.0020725388601036268]
[0.0, 0.049000000000000002, 0.037575150300601205, 0.0096105209914011131, 0.03367875647668394]
[0.0, 0.0030000000000000001, 0.016533066132264528, 0.032372281234193223, 0.021243523316062177]
[0.0, 0.052499999999999998, 0.045591182364729456, 0.13606474456246839, 0.09689119170984456]
[0.0, 0.00050000000000000001, 0.025050100200400802, 0.0, 0.0]
[0.0, 0.00050000000000000001, 0.0075150300601202402, 0.00050581689428426911, 0.00051813471502590671]
[0.0, 0.123, 0.050601202404809621, 0.084977238239757211, 0.034715025906735753]
[0.0, 0.094, 0.036072144288577156, 0.027314112291350532, 0.030569948186528497]
[0.0, 0.1865, 0.03406813627254509, 0.030349013657056147, 0.080310880829015538]
[0.0, 0.0, 0.077655310621242479, 0.11987860394537178, 0.053886010362694303]
[0.0, 0.0, 0.027555110220440882, 0.018715225088517955, 0.013471502590673576]
[0.0, 0.0, 0.014529058116232466, 0.0015174506828528073, 0.0041450777202072537]
[0.0, 0.0, 0.095190380761523044, 0.076884167931208905, 0.11968911917098446]
[0.0, 0.0, 0.039579158316633264, 0.036418816388467376, 0.078238341968911912]
[0.0, 0.0, 0.10671342685370741, 0.039453717754172987, 0.077720207253886009]
[0.0, 0.0, 0.030561122244488977, 0.098128477491148211, 0.029015544041450778]
[0.0, 0.0, 0.033567134268537073, 0.0015174506828528073, 0.013989637305699482]
[0.0, 0.0, 0.00050100200400801599, 0.0075872534142640367, 0.0041450777202072537]
[0.0, 0.0, 0.0, 0.0, 0.0067357512953367879]
[0.0, 0.0, 0.002004008016032064, 0.0040465351542741529, 0.024352331606217616]
[0.0, 0.0, 0.0015030060120240481, 0.0050581689428426911, 0.0010362694300518134]

Ungolfed

s=input().split()
m=[]
l=96
while l<122:
    l+=1;r=[]
    for i in range(max(map(len,s))):                # Max length of words in list
        L=sum(i<len(w)for w in s)                   # How many words of this length
        f=sum(i<len(w)and w[i]==chr(l)for w in s)   # Number of char in that pos
        r+=[f/L]
    m+=[r]
for r in m:print(r)

What percent of the words began with "a"? (I'm having trouble understanding the table.) — DavidC, Commented Sep 11, 2015 at 21:12
@DavidCarraher 100%. I only have output for the first 2000 words (all beginning with a). Neither interpreter I tried (repl.it and ideone) will work for much larger input. — mbomb007, Commented Sep 11, 2015 at 21:22
I see. Why not save the file on your hard disk, then import the file and work on it? — DavidC, Commented Sep 11, 2015 at 21:52
@DavidCarraher Because many people cough are at work cough cough. >.> — mbomb007, Commented Sep 11, 2015 at 21:58

Community · Accepted Answer · 2020-06-17 09:04:33Z

bash with gawk 4 and sort, 125

Pretty much what awk was made for

awk '{for(;NF;c[NF--]++)a[$NF][NF]++}END{for(i in a){printf i;for(j=0;j++<5;)printf" %.9f",a[i][j]/c[j];print""}}' FS= wordlist.txt|sort

"wordlist.txt" needs to be replaced by the name of your file containing the list of words, and I counted the whole filename as one character, because it could be ;)

Be sure to leave the space between FS= and the filename.

It takes into account, that not all words have five letters. So for the average of the second to fifth position it divides only by the number of words with at least that much letters.

awk part readable

{
    for(;NF;c[NF--]++)    # count number of words with at least that length
        a[$NF][NF]++      # count occurences of character at that position
}

END{
    for(i in a)
    {
        printf i;
        for(j=0;j++<5;)
            printf" %.5f",a[i][j]/c[j];    # print averages
        print""
    }
}

You can easily utilize the sort to gather some information. For instance the option -rk2 sorts by the most common letter in the first position, -rk3 sorts by the most common letter in the second position and so on.

Output without sort options

a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522
b 0.057777336 0.007004819 0.022906584 0.020010382 0.018061682
c 0.099491612 0.017652806 0.051200027 0.046635074 0.032427344
d 0.065196151 0.007683773 0.032486906 0.040690567 0.029411252
e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
f 0.045208406 0.003560369 0.019392694 0.018553559 0.012238707
g 0.033566827 0.002798616 0.025774050 0.030107671 0.022367893
h 0.035785848 0.039197178 0.008618975 0.021818852 0.030352691
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
j 0.008611125 0.000281518 0.003000066 0.001892195 0.000331247
k 0.006557703 0.003295411 0.004342637 0.025971634 0.018079116
l 0.030602613 0.046864391 0.056984685 0.056363971 0.065848428
m 0.050093563 0.018331760 0.042796526 0.033138532 0.025017870
n 0.017040091 0.068822760 0.080703441 0.055593697 0.054289649
o 0.024988822 0.141719244 0.067062256 0.059210636 0.064192193
p 0.082236243 0.020285823 0.041453955 0.040891509 0.025784968
q 0.004835478 0.003212612 0.002850892 0.002377803 0.001046043
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
s 0.123453723 0.009124480 0.069084400 0.058641303 0.070956607
t 0.051037475 0.023713713 0.066896506 0.085986035 0.071061211
u 0.022802921 0.076821170 0.043144600 0.036018688 0.033316480
v 0.014224916 0.009372878 0.017917523 0.013965405 0.007200265
w 0.026512329 0.006955139 0.009961546 0.009628426 0.007705853
x 0.000182158 0.011840297 0.004674136 0.001507058 0.001290120
y 0.003146373 0.010515508 0.009132799 0.006681291 0.015097892
z 0.001506947 0.000314637 0.003828814 0.004336978 0.002911487

Output sorted after occurrence as first letter

s 0.123453723 0.009124480 0.069084400 0.058641303 0.070956607
c 0.099491612 0.017652806 0.051200027 0.046635074 0.032427344
p 0.082236243 0.020285823 0.041453955 0.040891509 0.025784968
d 0.065196151 0.007683773 0.032486906 0.040690567 0.029411252
b 0.057777336 0.007004819 0.022906584 0.020010382 0.018061682
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522
t 0.051037475 0.023713713 0.066896506 0.085986035 0.071061211
m 0.050093563 0.018331760 0.042796526 0.033138532 0.025017870
f 0.045208406 0.003560369 0.019392694 0.018553559 0.012238707
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
h 0.035785848 0.039197178 0.008618975 0.021818852 0.030352691
g 0.033566827 0.002798616 0.025774050 0.030107671 0.022367893
l 0.030602613 0.046864391 0.056984685 0.056363971 0.065848428
w 0.026512329 0.006955139 0.009961546 0.009628426 0.007705853
o 0.024988822 0.141719244 0.067062256 0.059210636 0.064192193
u 0.022802921 0.076821170 0.043144600 0.036018688 0.033316480
n 0.017040091 0.068822760 0.080703441 0.055593697 0.054289649
v 0.014224916 0.009372878 0.017917523 0.013965405 0.007200265
j 0.008611125 0.000281518 0.003000066 0.001892195 0.000331247
k 0.006557703 0.003295411 0.004342637 0.025971634 0.018079116
q 0.004835478 0.003212612 0.002850892 0.002377803 0.001046043
y 0.003146373 0.010515508 0.009132799 0.006681291 0.015097892
z 0.001506947 0.000314637 0.003828814 0.004336978 0.002911487
x 0.000182158 0.011840297 0.004674136 0.001507058 0.001290120

Second letter

e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
o 0.024988822 0.141719244 0.067062256 0.059210636 0.064192193
a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102

Third letter

r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522
n 0.017040091 0.068822760 0.080703441 0.055593697 0.054289649
s 0.123453723 0.009124480 0.069084400 0.058641303 0.070956607
e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764

Fourth letter

e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
t 0.051037475 0.023713713 0.066896506 0.085986035 0.071061211
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
a 0.054250087 0.136651928 0.091410860 0.062576399 0.065691522

Fifth letter

e 0.041962674 0.149949492 0.068554001 0.116897470 0.139123764
i 0.042244192 0.097620349 0.061642246 0.080158743 0.107306613
r 0.056684386 0.086409327 0.094178877 0.070346121 0.078889102
t 0.051037475 0.023713713 0.066896506 0.085986035 0.071061211
s 0.123453723 0.009124480 0.069084400 0.058641303 0.070956607

So it looks like there is no Benfords Law of Words ;)

Great outputs/ sorting. It certainly looks like there is no Benford's Law of Words. Oh well =/ — dberm22, Commented Sep 14, 2015 at 20:10
Well, thanks. I was forced to use the formatting string there, since some averages were too short for just using tab, which would have messed up the whole table. About the law: I like it netter that way. With numbers it is at least somehow intuitively comprehensible. With letters it would be only spooky. ;) — Cabbie407, Commented Sep 15, 2015 at 4:33

flodel · Accepted Answer · 2015-09-13 16:43:56Z

R: 88, 95, or 126 bytes

z=sapply(readr::read_lines("http://tinyurl.com/pshoqyo"),Vectorize(substr),1:5,1:5)
y=apply(z,1,table,exclude="")
y/colSums(y)

That's 126 bytes where that the data is downloaded from the provided URL (a tinyurl equivalent to be exact).

I can shorten the code to 88 bytes if you allow to read the input from stdin:

z=sapply(scan(,""),Vectorize(substr),1:5,1:5)
y=apply(z,1,table,exclude="")
y/colSums(y)

or 95 bytes if the data is provided via a t.txt file:

z=sapply(scan("t.txt",""),Vectorize(substr),1:5,1:5)
y=apply(z,1,table,exclude="")
y/colSums(y)

The output:

          aah         <NA>        <NA>        <NA>         <NA>
a 0.054250087 0.1366519284 0.091410860 0.062576399 0.0656915218
b 0.057777336 0.0070112047 0.023141714 0.020833697 0.0171560104
c 0.099582311 0.0178502654 0.053853798 0.046119198 0.0308013314
d 0.065925417 0.0080894018 0.032457317 0.040240449 0.0279619439
e 0.044177897 0.1499494924 0.068491563 0.115709739 0.1336258142
f 0.045208406 0.0035603690 0.019392694 0.018553559 0.0122387071
g 0.033566827 0.0028011669 0.026038614 0.031346432 0.0212462947
h 0.035818471 0.0396356269 0.009065709 0.021577492 0.0288307086
i 0.042716723 0.1027737583 0.061586103 0.079272029 0.1020188291
j 0.009065709 0.0002815175 0.002997334 0.001872970 0.0003181567
k 0.006557703 0.0032954113 0.004342637 0.025971634 0.0180791157
l 0.030602613 0.0469071140 0.057569618 0.058683031 0.0625465746
m 0.050139230 0.0185368141 0.045014732 0.032771954 0.0237633928
n 0.017230697 0.0724559354 0.080629937 0.054978721 0.0516144003
o 0.026307990 0.1417192442 0.067001176 0.058609030 0.0616554195
p 0.082236243 0.0202858231 0.041453955 0.040891509 0.0257849684
q 0.004835478 0.0032155407 0.002880155 0.002475636 0.0009935913
r 0.056736060 0.0873758770 0.099060304 0.069567953 0.0749333466
s 0.124834642 0.0096061647 0.069021478 0.057992614 0.0674600544
t 0.053731760 0.0237137132 0.066835577 0.085112378 0.0682529848
u 0.022802921 0.0768211701 0.043144600 0.036018688 0.0333164804
v 0.014224916 0.0093814228 0.018101442 0.014540002 0.0068392204
w 0.026536498 0.0070329376 0.010477867 0.009521917 0.0073194562
x 0.000184196 0.0124653498 0.004669879 0.001490387 0.0012265464
y 0.003312471 0.0105155083 0.009124480 0.006613406 0.0145012475
z 0.001506947 0.0003146373 0.003828814 0.004336978 0.0029114873

You seem to have the shortest answer. Please provide a printout and I will accept your solution! — dberm22, Commented Sep 13, 2015 at 14:30

Community · Accepted Answer · 2020-06-17 09:04:33Z

Dyalog APL (76)

⎕ML←3⋄L{R÷+⌿R←0 1↓K[⍋⊃K←,∘≢⌸⍺⌷¨⍵;]}¨(↓(L←⍳5)∘.≤≢¨G)/¨⊂G←W⊂⍨10≠W←83 ¯1⎕MAP'w'

This assumes the input is in a file called w in the working directory.

Output:

      ⎕ML←3⋄L{R÷+⌿R←0 1↓K[⍋⊃K←,∘≢⌸⍺⌷¨⍵;]}¨(↓(L←⍳5)∘.≤≢¨G)/¨⊂G←W⊂⍨10≠W←83 ¯1⎕MAP'w'
 0.05425008694    0.1366519284     0.09141085991   0.06257639947   0.06569152182  
 0.05777733618    0.007004818918   0.02290658357   0.02001038196   0.01806168169  
 0.09949161243    0.01765280607    0.05120002652   0.04663507426   0.03242734357  
 0.06519615149    0.007683772997   0.03248690579   0.04069056749   0.02941125194  
 0.04196267409    0.1499494924     0.06855400119   0.1168974698    0.1391237644   
 0.04520840578    0.003560368954   0.01939269376   0.01855355917   0.01223870709  
 0.0335668273     0.002798615596   0.02577405026   0.03010767093   0.02236789344  
 0.03578584795    0.0391971782     0.008618975005  0.02181885162   0.03035269095  
 0.04224419163    0.09762034875    0.06164224624   0.08015874345   0.1073066127   
 0.008611124911   0.0002815175452  0.0030000663    0.001892195114  0.000331247058 
 0.006557702817   0.003295411264   0.004342637406  0.02597163382   0.01807911574  
 0.03060261315    0.04686439134    0.05698468474   0.05636397127   0.06584842832  
 0.05009356318    0.01833176015    0.04279652589   0.03313853213   0.02501786991  
 0.01704009141    0.06882275987    0.08070344096   0.05559369715   0.0542896494   
 0.0249888221     0.1417192442     0.06706225552   0.05921063648   0.06419219303  
 0.0822362429     0.02028582311    0.04145395478   0.04089150857   0.02578496836  
 0.004835477835   0.003212611986   0.002850891732  0.002377802709  0.001046043341 
 0.05668438571    0.08640932651    0.09417887688   0.070346121     0.07888910197  
 0.1234537235     0.009124480435   0.06908439966   0.05864130344   0.07095660664  
 0.05103747495    0.02371371322    0.066896506     0.0859860346    0.07106121097  
 0.02280292116    0.07682117012    0.04314459988   0.03601868752   0.03331648041  
 0.01422491596    0.009372878269   0.01791752304   0.01396540465   0.007200264998 
 0.02651232881    0.006955139351   0.009961546112  0.009628426464  0.007705852612 
 0.0001821584116  0.01184029675    0.004674136445  0.001507058055  0.001290120121 
 0.003146372564   0.0105155083     0.009132798515  0.006681290711  0.01509789222  
 0.001506946859   0.0003146372564  0.003828813896  0.004336978181  0.002911487299

Explanation:

⎕ML←3: set migration level to 3. (Among other things, this changes the behaviour of ⊂ from 'split' into 'partition', so we don't have to get rid of the linefeeds.)
W←83 ¯1⎕MAP'w': read the file w as a string of ASCII values, store it in W.
G←W⊂⍨10≠W: partition W on linefeeds (character 10) and store the words in G.
(↓(L←⍳5)∘.≤≢¨G)/¨⊂G: store the numbers 1 to 5 in L, and for each number, select those words from G which have at least that many letters.
L{...}¨: for each of the numbers 1 to 5, and the matching list of words,:
- ⍺⌷¨⍵: for each word, select the ⍺-th letter
- K←,∘≢⌸: for each letter, find out how many times it occurs, store this in K.
- K[⍋⊃K...;]: sort K by the value of the letter (so a is at the top, and z at the bottom).
- R←0 1↓: drop the first column (the letter), leaving only the amount of times it occurred. Store this in R.
- R÷+⌿R: divide the value for each letter by the total amount.

Stack Exchange Network

Benfords Law of Words

6 Answers 6

Mathematica 246 bytes

MATLAB, 113 117 bytes

Python 3, 143 129 bytes

Example Output

Ungolfed

bash with gawk 4 and sort, 125

awk part readable

Output without sort options

Output sorted after occurrence as first letter

Second letter

Third letter

Fourth letter

Fifth letter

R: 88, 95, or 126 bytes

Dyalog APL (76)

Your Answer

Not the answer you're looking for? Browse other questions tagged
code-golf
or ask your own question.

Hot Network Questions

Benfords Law of Words

6 Answers 6

Mathematica 246 bytes

MATLAB, 113 117 bytes

Python 3, 143 129 bytes

Example Output

Ungolfed

bash with gawk 4 and sort, 125

awk part readable

Output without sort options

Output sorted after occurrence as first letter

Second letter

Third letter

Fourth letter

Fifth letter

R: 88, 95, or 126 bytes

Dyalog APL (76)

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged code-golf or ask your own question.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
code-golf
or ask your own question.