Help! I have more homework!

Question

My teacher was more than unhappy with my Martian Homework. I followed all the rules, but she says that what I output was gibberish... when she first looked at it, she was highly suspicious. "All languages should follow Zipf's law blah blah blah"... I didn't even know what Zipf's law was!

It turns out Zipf's law states that if you plot the logarithm of the frequency of each word on the y axis, and the logarithm of the "place" of each word on the x axis (most common = 1, second most common = 2, third most commmon = 3, and so on), then the plot will show a line with a slope of about -1, give or take about 10%.

For example, here's a plot for Moby Dick:

The x axis is the nth most common word, the y-axis is the number of occurences of the nth most common word. The slope of the line is about -1.07.

Now we're covering Venutian. Thankfully, Venutians use the latin alphabet. The rules are as follows:

• Each word must contain at least one vowel (a, e, i, o, u)
• In each word there can be up to three vowels in a row, but no more than two consonants in a row (a consonant is any letter that's not a vowel).
• No words longer than 15 letters
• Optional: group words into sentences 3-30 words long, delimited by periods

Because the teacher feels that I cheated on my Martian homework, I've been assigned to write an essay at least 30,000 words long (in Venutian). She's going to check my work using Zipf's law, so when a line is fitted (as described above) the slope has to be at most -0.9 but no less than -1.1, and she wants a vocabulary of at least 200 words. The same word should not be repeated more than 5 times in a row.

This is CodeGolf, so shortest code in bytes wins. Please paste the output to Pastebin or another tool where I can download it as a text file.

Answer 1

Mathematica, 102 bytes

""<>RandomChoice[1/Range@215->Rest@Flatten@Outer[StringJoin,a={"v","a","e","i","o","u"},a,a,{" "}],8!]

Unnamed function taking no input and returning a string consisting of 40,320 three-letter Venusian words with trailing spaces.

Outer[StringJoin,a={"v","a","e","i","o","u"},a,a,{" "}] produces the 216 three-letter words possible using only the letters "vaeiou", each with its own trailing space. The first of these words, "vvv", is not valid Venusian, but Rest throws it away.

Then RandomChoice[1/Range@215->...,8!] makes 8! = 40,320 random choices from the resulting 215-word list, with frequency weights determined by the reciprocals of the first 215 integers (1/Range@215). Finally, <>""... concatenates the strings in the resulting list.

The output is far from deterministic; one run yielded this Venusian essay.

Mathematica, 129 bytes

#2&@@@Sort[Join@@Table[{i,Rest@Flatten@Outer[StringJoin,a={"v","a","e","i","o","u"},a,a,{" "}]~Part~j},{j,215},{i,0,1,j/7!}]]<>""

This one is deterministic. The base set of 215 words is the same, but now each word is repeated an exact number of times (word #j is repeated roughly 7!/j times) to force zipf's law to hold. Then the words are interleaved equally to avoid repetitions. (Imagine each word is laid out on a ruler, with all the copies of that word equally spaced; when all the words are read in order, no particular word will repeat much, perhaps not at all.) The result is a 30,117-word Venusian essay.

Answer 2

05AB1E, 34 33 32 bytes

525DL/9*vNy<FD}})žMDâžNâJè3ô.rðý

525DL                            Yield [1, ..., 525]
     /                           Yield [525/1, ..., 525/525]
      9*                         Yield [4725/1, ..., 4725/525]. It's the number of occurences of each word. The sum of this array is greater than 30000
        v                        For each value (y = value, N = iteration counter starting from 0)
         N                       Push iteration counter
          y<FD}                  Push an array of "int(value)" times the iteration counter
               }                 End for
                )                Wrap everything in an array. At this point the array countains the sorted indices of all words that matches the frequency specs
                 žM              Push "aeiou"
                   Dâ            Cartesian product with itself (["aa", "ae", ...])
                     žN          Push the consonants
                       âJ        Cartesian product and join the values to make valid venutian words
                         è       Compute a big string with all words that correspond to the formerly computed indices
                          3ô     Since all words are concatenated, separate them into blocks of 3 letters
                            .r   Shuffle
                              ðý Join with whitespaces and implicitly display

Try it online!

I think it is still pretty golfable! For instance the numeric constants and vNy<FD} might be golfable.

Output example

How does it work?

It generates all combinations of words following the rule "vowel+vowel+consonant", which makes 525 unique valid words (more than 200). It then associates to each of them a frequency that satisfies the law f(x) = 4725/x where x is the rank of the current word, starting at 1 and ending at 525. Then the frequencies are normalized and multiplied so there are at least 30000 words. This code always yields 32074 words to make the involved constants golfable (please see the code explanation). So each word is repeated the amount of times corresponding to the frequency of the same word. Finally the words are shuffled. However it does not guarantee that a word is never repeated five times in a row. Hence the programs generates more than the needed 200 unique words in order to decrease the probability of having a word repeated five times in a row. Please note that this code always generates the same word sequence. The only thing that differs between two runs is the result of the shuffling operation.

How to evaluate frequency?

I made a simple Python3 code that takes the text in the file named "output" (from the algorithm point of view, it makes sense!) and outputs to "stats.csv".

from collections import Counter
from math import log10

with open("output", "r") as f:
    with open("stats.csv", "w") as stats:
        words = f.read().split()
        freqs = Counter(words)
        freqs = sorted([(i,freqs[i]) for i in freqs],key=lambda x:-x[1])

        print(len(words), "words")
        stats.write("logX;logF\n")
        for i, (key, f) in enumerate(freqs):
            stats.write(str(log10(i+1))+";"+str(log10(f))+"\n")

Which always yields the following distribution for my code:

So the slope is -1.0138. This value is now less close to -1 than the slope of the previous code, but it still satisfies the slope constraints.

Answer 3

Bash/Core Utils, 122 110 bytes

for w in {b,c,d}{a,e,i}{f,g,h}{o,u,a}{j,k,l};{ let ++x;yes $w|head -$[5575/x];}|shuf --random-source=<(yes ae)

Unrolled:

for w in {b,c,d}{a,e,i}{f,g,h}{o,u,a}{j,k,l};{
    let ++x
    yes $w|head -$[5575/x]
}|shuf --random-source=<(yes ae)

The for w loop generates 243 distinct words. let ++x; increments initially unset x (per arithmetic expression rules during that first execution, x is treated as 0 and thus its increment sets it to 1). The next line thus generates succeeding words at frequency 5575/x to approximate zipf frequency.

The next step is to permute this deterministically to fit the repetition requirement; despite --random-source being a terribly large flag name, using it with shuf beats the char count of hand rolling a mul-mod selector. yes ae is actually the shortest fixed "random" device I found to comply.

This generates this 33729 word essay [pastebin].

Bash/Core Utils, 96 84 bytes (non-competing)

For a non-deterministic approach, just chop off shuf flags:

for w in {b,c,d}{a,e,i}{f,g,h}{o,u,a}{j,k,l};{ let ++x;yes $w|head -$[5575/x];}|shuf

Analysis

The zipf slope is tuned to be straight. Using Excel to plot on logarithmic scales:

The teacher should notice a zipf slope of =-1.000764.