About Markov Writer
A typewriter that learns to write by reading. Pick a source (or paste your own text), choose how it writes and press WRITE. It types a fresh page that sounds a bit like the source, without understanding a word of it.
- The original is where this app started: a little Python script, random-sentences.py, that builds sentences by picking an article, noun, verb, preposition, article and noun, each at random from five fixed words ("Some dog walked on a town.").
- Words, order 1 to 4: each word is picked from the words that followed the last 1, 2, 3 or 4 words somewhere in the source, with the same odds.
- Letters, order 1 to 8: the same game one character at a time. Low orders make babble that looks like English; by order 5 it spells real words.
Click any word (or letter) on the page to see the choices the chain had at that point and how likely each was. Click a choice to put it there and carry on writing from it, or RE-ROLL to let chance pick again. Mix with blends a second source, typed in red like a two-colour ribbon; the balance sets how much each counts, whatever their lengths. THE LADDER writes a short line on each of four rungs (the original and orders 1 to 3, or in letters mode orders 1, 3, 5 and 8), so you can read the climb from random words to copying. Underline what it copied underlines every stretch of 8 words or more (40 letters in letters mode) that appears word for word in the source; the longest run gets a double line.
THE STORY
Andrey Markov, a Russian mathematician, had shown in 1906 that the law of large numbers can still hold when each event depends on the one before. In 1913 he tried it on real text: he went through the first 20,000 letters of Pushkin's verse novel Eugene Onegin by hand, counting how often a vowel follows a vowel and a consonant follows a consonant. That was the first time a Markov chain (a process where what happens next depends only on where you are now) was put to work on real data.
In 1948 Claude Shannon's paper A Mathematical Theory of Communication, which founded information theory, showed the same idea writing English. Picking letters with English frequencies, then by pairs and triples of letters, then whole words by pairs, gave lines that looked more and more like English ("THE HEAD AND IN FRONTAL ATTACK ON AN ENGLISH WRITER..."). He did it with books: open one at random, find the last letters you wrote, and copy what comes next.
Programs that write this way went on to make "Mark V. Shaney", a Usenet poster of the 1980s whose strange messages came from a word chain, and, in spirit, the next-word suggestions on phone keyboards. Modern AI language models also write one piece at a time from the odds of what comes next; the difference is that they look back over thousands of words and learn the odds from a vast amount of text, rather than counting exact matches.
HOW IT WORKS
Reading: for every run of n words in the source (the "context"), it counts which word came next. Writing: it starts at a real sentence opening, then repeatedly looks up the last n words and picks the next one at random, weighted by those counts. The source is treated as a loop, so it never gets stuck at the end. It stops at the first full stop after the length you chose; a quarter past that length it will also stop at a comma, semicolon or colon, and at half as long again it stops wherever it is (poems can run for lines without a full stop).
With a small source, high orders have only one choice almost everywhere, so the chain just copies the book. Order 1 nearly always has a choice, so it wanders off after every word. The sweet spot depends on how long the source is: these extracts are one or two thousand words, so order 2 is where it reads like a tipsy author. When two sources are mixed, it can only cross from one to the other where both used the same words, which is why the red and black come in runs.
The source texts are all long out of copyright: extracts from the start of each book, a selection of the poems, psalms and rhymes. Nothing you paste leaves this tab or is stored; closing the tab forgets it.
FURTHER WATCHING AND READING