18
\$\begingroup\$

My teacher was more than unhappy with my Martian Homework. I followed all the rules, but she says that what I output was gibberish... when she first looked at it, she was highly suspicious. "All languages should follow Zipf's law blah blah blah"... I didn't even know what Zipf's law was!

It turns out Zipf's law states that if you plot the logarithm of the frequency of each word on the y axis, and the logarithm of the "place" of each word on the x axis (most common = 1, second most common = 2, third most commmon = 3, and so on), then the plot will show a line with a slope of about -1, give or take about 10%.

For example, here's a plot for Moby Dick:

enter image description here

The x axis is the nth most common word, the y-axis is the number of occurences of the nth most common word. The slope of the line is about -1.07.

Now we're covering Venutian. Thankfully, Venutians use the latin alphabet. The rules are as follows:

  • Each word must contain at least one vowel (a, e, i, o, u)
  • In each word there can be up to three vowels in a row, but no more than two consonants in a row (a consonant is any letter that's not a vowel).
  • No words longer than 15 letters
  • Optional: group words into sentences 3-30 words long, delimited by periods

Because the teacher feels that I cheated on my Martian homework, I've been assigned to write an essay at least 30,000 words long (in Venutian). She's going to check my work using Zipf's law, so when a line is fitted (as described above) the slope has to be at most -0.9 but no less than -1.1, and she wants a vocabulary of at least 200 words. The same word should not be repeated more than 5 times in a row.

This is CodeGolf, so shortest code in bytes wins. Please paste the output to Pastebin or another tool where I can download it as a text file.

\$\endgroup\$
  • \$\begingroup\$ Yes and if you want to you could just do a 32767 word sentence. The restriction is that the frequencies of words in the sentence must follow zipf's law \$\endgroup\$ – J. Antonio Perez Dec 3 '16 at 22:41
  • 1
    \$\begingroup\$ The traditional adjective for "from Venus" is Veneral, but for some reason it's declined in popularity. Venusian is commonly used in sci-fi. \$\endgroup\$ – Peter Taylor Dec 3 '16 at 22:46
  • \$\begingroup\$ I'm conjecturing that building a list of words following zipf distribution and shuffling it would have a high probability of producing a sequence with pairs of consecutive word also following zipf distribution. Moreover, with enough different words in the list the probability of having the same word repeated more than 5 times in a row would be really small. In case I tried this approach and managed to produce a valid essay, would it be accepted? \$\endgroup\$ – Leo Dec 4 '16 at 0:09
  • \$\begingroup\$ Your conjecture is reasonable, although the slope would be -0.35 \$\endgroup\$ – J. Antonio Perez Dec 4 '16 at 0:09
  • \$\begingroup\$ It would still look like a straight line; it's just the slope would be too great \$\endgroup\$ – J. Antonio Perez Dec 4 '16 at 0:10
3
\$\begingroup\$

Mathematica, 102 bytes

""<>RandomChoice[1/Range@215->Rest@Flatten@Outer[StringJoin,a={"v","a","e","i","o","u"},a,a,{" "}],8!]

Unnamed function taking no input and returning a string consisting of 40,320 three-letter Venusian words with trailing spaces.

Outer[StringJoin,a={"v","a","e","i","o","u"},a,a,{" "}] produces the 216 three-letter words possible using only the letters "vaeiou", each with its own trailing space. The first of these words, "vvv", is not valid Venusian, but Rest throws it away.

Then RandomChoice[1/Range@215->...,8!] makes 8! = 40,320 random choices from the resulting 215-word list, with frequency weights determined by the reciprocals of the first 215 integers (1/Range@215). Finally, <>""... concatenates the strings in the resulting list.

The output is far from deterministic; one run yielded this Venusian essay.

Mathematica, 129 bytes

#2&@@@Sort[Join@@Table[{i,Rest@Flatten@Outer[StringJoin,a={"v","a","e","i","o","u"},a,a,{" "}]~Part~j},{j,215},{i,0,1,j/7!}]]<>""

This one is deterministic. The base set of 215 words is the same, but now each word is repeated an exact number of times (word #j is repeated roughly 7!/j times) to force zipf's law to hold. Then the words are interleaved equally to avoid repetitions. (Imagine each word is laid out on a ruler, with all the copies of that word equally spaced; when all the words are read in order, no particular word will repeat much, perhaps not at all.) The result is a 30,117-word Venusian essay.

\$\endgroup\$
  • \$\begingroup\$ Doesn't making 8! pseudo-random choice mean you could wind up with 6 consecutive repetitions of the same word? \$\endgroup\$ – Dennis Dec 4 '16 at 4:18
  • \$\begingroup\$ Yes, in theory. \$\endgroup\$ – Greg Martin Dec 4 '16 at 5:23
  • \$\begingroup\$ @GregMartin Actually... the essay you linked to is non-compliant; vva appears six times consecutively. I think there's possibly a bigger issue though... shouldn't challenge answers work every time? (And if not, how do you draw the line of how likely they should be to work?) \$\endgroup\$ – H Walters Dec 4 '16 at 5:40
  • \$\begingroup\$ This is a fair critique, and I'm interested to see how it plays out. \$\endgroup\$ – Greg Martin Dec 4 '16 at 8:28
2
\$\begingroup\$

05AB1E, 34 33 32 bytes

525DL/9*vNy<FD}})žMDâžNâJè3ô.rðý

525DL                            Yield [1, ..., 525]
     /                           Yield [525/1, ..., 525/525]
      9*                         Yield [4725/1, ..., 4725/525]. It's the number of occurences of each word. The sum of this array is greater than 30000
        v                        For each value (y = value, N = iteration counter starting from 0)
         N                       Push iteration counter
          y<FD}                  Push an array of "int(value)" times the iteration counter
               }                 End for
                )                Wrap everything in an array. At this point the array countains the sorted indices of all words that matches the frequency specs
                 žM              Push "aeiou"
                   Dâ            Cartesian product with itself (["aa", "ae", ...])
                     žN          Push the consonants
                       âJ        Cartesian product and join the values to make valid venutian words
                         è       Compute a big string with all words that correspond to the formerly computed indices
                          3ô     Since all words are concatenated, separate them into blocks of 3 letters
                            .r   Shuffle
                              ðý Join with whitespaces and implicitly display

Try it online!

I think it is still pretty golfable! For instance the numeric constants and vNy<FD} might be golfable.

Output example

How does it work?

It generates all combinations of words following the rule "vowel+vowel+consonant", which makes 525 unique valid words (more than 200). It then associates to each of them a frequency that satisfies the law f(x) = 4725/x where x is the rank of the current word, starting at 1 and ending at 525. Then the frequencies are normalized and multiplied so there are at least 30000 words. This code always yields 32074 words to make the involved constants golfable (please see the code explanation). So each word is repeated the amount of times corresponding to the frequency of the same word. Finally the words are shuffled. However it does not guarantee that a word is never repeated five times in a row. Hence the programs generates more than the needed 200 unique words in order to decrease the probability of having a word repeated five times in a row. Please note that this code always generates the same word sequence. The only thing that differs between two runs is the result of the shuffling operation.

How to evaluate frequency?

I made a simple Python3 code that takes the text in the file named "output" (from the algorithm point of view, it makes sense!) and outputs to "stats.csv".

from collections import Counter
from math import log10

with open("output", "r") as f:
    with open("stats.csv", "w") as stats:
        words = f.read().split()
        freqs = Counter(words)
        freqs = sorted([(i,freqs[i]) for i in freqs],key=lambda x:-x[1])

        print(len(words), "words")
        stats.write("logX;logF\n")
        for i, (key, f) in enumerate(freqs):
            stats.write(str(log10(i+1))+";"+str(log10(f))+"\n")

Which always yields the following distribution for my code: Frequency law

So the slope is -1.0138. This value is now less close to -1 than the slope of the previous code, but it still satisfies the slope constraints.

\$\endgroup\$
  • \$\begingroup\$ Thank you for the script to evaluate frequency, note that you have an extra ` at the end. Also, why do you use semicolons as separators? csv usually stands for comma separated values ;) \$\endgroup\$ – Leo Dec 4 '16 at 18:44
  • 1
    \$\begingroup\$ Yes you're right haha! I got confused with markdown for a second. I used inline code first, then realized it was more appropriate to use a code block but I forgot to remove the extra `. I use semicolons as csv separators because I'm French and some softwares or companies are used to putting decimal values with comas instead of dots as we do with handwritten decimal values. Although I always use the dot to separate the integer part from the fractional part, I use the semicolon without further thinking. But hey, ssv is great as well ;) \$\endgroup\$ – Osable Dec 4 '16 at 18:51
0
\$\begingroup\$

Bash/Core Utils, 122 110 bytes

for w in {b,c,d}{a,e,i}{f,g,h}{o,u,a}{j,k,l};{ let ++x;yes $w|head -$[5575/x];}|shuf --random-source=<(yes ae)

Unrolled:

for w in {b,c,d}{a,e,i}{f,g,h}{o,u,a}{j,k,l};{
    let ++x
    yes $w|head -$[5575/x]
}|shuf --random-source=<(yes ae)

The for w loop generates 243 distinct words. let ++x; increments initially unset x (per arithmetic expression rules during that first execution, x is treated as 0 and thus its increment sets it to 1). The next line thus generates succeeding words at frequency 5575/x to approximate zipf frequency.

The next step is to permute this deterministically to fit the repetition requirement; despite --random-source being a terribly large flag name, using it with shuf beats the char count of hand rolling a mul-mod selector. yes ae is actually the shortest fixed "random" device I found to comply.

This generates this 33729 word essay [pastebin].

Bash/Core Utils, 96 84 bytes (non-competing)

For a non-deterministic approach, just chop off shuf flags:

for w in {b,c,d}{a,e,i}{f,g,h}{o,u,a}{j,k,l};{ let ++x;yes $w|head -$[5575/x];}|shuf

Analysis

The zipf slope is tuned to be straight. Using Excel to plot on logarithmic scales:

The teacher should notice a zipf slope of =-1.000764.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.