netpoetic.com

Hi all! The other day Jim Andrews commented that poetry generator development involves making parameters configurable by variables and graphic user interface elements.

Now, over the last year or so I’ve been asking myself: ytf am I doing this? When I code a poetry generator, what exactly am I exploring? How can I formalize, track, measure what I’m working on? There are no pre-existing representations. But I figured I’d use Jim Andrew’s comment as a starting point for thinking about charNG, the most recent generator I’ve developed.

charNG is a character n-gram generator in the tradition of Dissociated Press and Travesty. I coded it in my spare time over the past couple of months, posting various output on Gnoetry Daily, including for a chapbook of poems remixing the text of HP Lovecraft. (advert: if you enjoy surreal gothic arcana, you should take a look: Lovecraft Remixed.)

Anyway, after the cut I’ll trace the development of charNG, focusing on which aspects of the character n-gram generation I implemented, which I parameterized, which I made accessible in the graphic user interface, and why. See below for more.

Intro

The reason I need to think this through is because I do poetry generation the same way a cat stalks the room. I basically do whatever I’m interested in at the moment without much long-term planning. In part this is because my funded research is by nature fairly structured. So I figure if I keep my poetry generation as unstructured as possible, I’ll open myself to research and technologies I might not otherwise have a chance to work with. And since I’m doing poetry generation for myself, I might as well enjoy it.

Poetry generation is process; generators and outputs trace explorations.

Before I started charNG, I knew what character n-grams are, and I had a fairly good idea about how Dissociated Press and Travesty are related, but I wanted to do some coding on it to get a really good feel for the algorithms. I want to emphasize that what’s important here are the algorithms involved, not the code itself. Before too long the platform will be obsolete and the code will be usable only through emulators or antiques. But the algorithms will be the same as they were for Andrey Markov in 1906, as they were for Ausonius in the 4th Century AD, as they were for everybody before and in-between.

Output is subjective and software becomes obsolete, but output sets are infinite and methods and algorithms are eternal.

So: one reason to develop poetry generators it to acquire an intimate understanding of computional poetry algorithms. Once you have this understanding, you will have a stronger mastery of your tools.

Early Versions

First, what is a character n-gram? If you wanna know the details in as unmathematical way as possible, see the explanation I wrote up for charNG. A summary is: it’s a sequence of characters in a text.

I started with a simple interface, built in JavaScript. I chose client-side JavaScript because I wanted something that could be easily saved by anyone who found it interesting. Part of the reason I developed charNG was because there used to be a server-side Travesty generator on a url including eskimo/rstarr/poormfa, but that generator disappeared a couple months ago. But a client-side generator can be saved by anyone, and either re-used personally or posted publically if it’s got a free-software license (as charNG does).

The top textarea contains the corpus, from which the n-gram model is built. I’ve used Shakespeare as a “hello world” ever since I read The American Council of Trustees and Alumni’s report on “The Vanishing Shakespeare“, which talks about how Tha Bard is no longer required in many courses.

The next (grayed-out) textarea is a log of the computations so I can see what’s happening. Even when I’m using a good IDE I like to include logs as well as use debuggers, so if something wacky happens in a run I can see why it happened.

The two buttons show charNG’s first two capabilities: first was “analyze” to show the n-gram model being built; next was “Write”. At first charNG only output character bigrams.

Finally, the lowermost textarea shows the output of the generation.

So the first thing that charNG did was take a corpus (the topmost textarea), count all the different sequences of characters in it, and print it out in the greyed-out area. For example, if the corpus begins with “From fairest creatures we desire increase,”, and we’re building a 4-gram language model, then our model will include “From”, “rom “, “om f”, “m fa”, ” fai”, “fair”, and so on. Some of these may seem kind of strange, but as it turns out the 4-gram “om f” occurs 7 times in Tha Sonnets, in places like “whom fortune”, and “random from”.

The next thing it did was to determine how to use these n-gram models to write poetry. Well, if you’ve come up with the string “cre” so far, and your 4-gram model tells you (among other things) that these are the 4-grams it’s seen beginning with “cre”:

crea 13,
cred 3,
cree 2,
crep 1,
cres 1,
cret 1,

then you may want to just randomly pick a number between 1 and 21; if it’s 13 or less you write “a” to make “crea”; if it’s 14-16 you write “d” to make “cred”; if it’s 17 or 18 you write “e” to make “cree”; if it’s 19 you write “p” to make “crep”; if it’s 20 you pick “s” to make “cres”; if it’s 21 you write “t” to make “cret”. This sort of thing was developed by the engineer Claude Shannon around the 1940s, based on ideas by mathematician Andrey Markov from the early 1900s.

In charNG’s as shown above, the only options are “Write” (to generate text) and “Analyze” (to show the n-gram models in the corpus.) The n-gram model being used (bigram) is hard-coded, along with everything else except the choice of corpus.

Now as a tool, charNG by this point is not very flexible: it only can output something like this:

kn ich flomalveeane thin,
Whe pay s ld tharde whenl,
Wheecheluselauset’dothallseinkn’sselothrkn eav

Which sounds kinda like getting your teeth pulled out. But, it has made me familiar with character n-gram generation. In particular, I was curious about the extent to which you had to represent an n-gram model in memory (i.e. have a data structure that contained a string like “crea” paired to a number like “13″) or whether you could just generate by looking at the data model. The way I did it by just looking at the data model was inspired by how Dissociated Press, one of the earliest character n-gram generators, does it: every time you want to generate a number, you pick the string you’re looking for (like “cre”), you look through the corpus to see how many times “cre” exists, you pick one of those, and you print out the next character you want. It’s equivalent to explicitly building a model for generation, because every instance of an n-gram built at model-building time also exists at Dissociated Press-style-generation time. You’re basically just doing a run-time tradeoff: either you can have an initial model that takes up memory and initializiation time to build, or you can generate a bit slower. Why would anyone want to explore this question? That’s the joy of unfunded efforts: as long as what you’re doing is more fun than TV, you WIN.

When you generate poetry as described above, you have no funders to report to, no program managers to satisfy, no auditors to review your code, and no audience to concern you. Your knowledge and abilities are constrained only by your will. This in itself is poetry.

Mid-development

Anyways, at first charNG only output character bigrams, and the code was not that clean: I had to re-write it to generalize beyond bigrams. After I’d convinced myself that generating from corpus was the same as generating from models (I could have just written out a mathematical proof but this was more fun!) I had to decide whether to continue working on this. I figured I would, and I generalized the n from which n-grams could be built as a parameter in code:

Note that I’ve de-emphasized the corpus window at the expense of the analysis and the generation output. From my time working with designers in a web shop I’ve come to appreciate that I know nothing about design, so in general I just focus completely on functionality: an unadorned interface focusing on the view I need of the algorithms I’m exploring.

Since I’m working on this in my spare time, I don’t want to waste too much time recalling context (i.e. remembering how everything works, what I was working on, and what I thought might be fun to work on next) so I usually end a programming session by noting somewhere what I thought would be interesting next. In this case, I did so by listing the parameters and I was frequently changing on the GUI but not yet making them GUI-editable. (in part because, for example, verbose detail printing was not completely implemented.)

At this point in the development, I was thinking of ways to add newlines and spaces to make the output look more like 20th century free verse. I got the idea from the generator JanusNode, which has an “eecummingsify” button that uses rules to add spaces given a configuration file of text to match for. I think that’s brilliant, but I wanted something simpler. At this point I was just adding newlines every once in a while and generating from higher-order n-grams. But I was already getting more interesting results:

But I worse cold differite,
And han steep in Grece crossed inher’d,
And to despacious cons my memoving,
And did moan walks;
And be.
And this,
And cleasure brange canot me, death of look,
And besmell’ to sic plague-tiend’ring paid dear;
They he can all a faults, of forbeauty, new-fould would ashinks bles up his shal, do you shalf wouldst you no me
When love’s not gones on sakes brain worth to bide,
That shall is thiefs his in our befor ther or time tendom leaven by mortunearseth showers gavength hell?

The above is a contiguous selection from unsupervised generation (i.e. generating a bunch of lines and selecting several adjacent lines without breaking them up or editing them) from Tha Sonnets. The repeated “And” in the selection gives it a bit of coherence, and it has some nice phrases. 4-grams are great for portmanteaux: you’re looking at just enough context to make interesting new words, without getting words that are unnaturally long. The created words sound like Old English or something, and for me the feeling is of a meaning that is possibly comprehensible but just out of reach.

Near-complete

By the time I announced the generator on Gnoetry Daily a couple weeks ago, I had explored a number of new features:

I’d implemented GUI versions of the parameters I’d been playing around with, such as type of n-grams model, number of characters to generate, and verbosity of generation details. I’d implemented free-style transformations by adding a percent chance of inserting a newline after any word, or initial spaces before a line. I’d messed around with these after using them at 0% chance each, which only uses newlines and initial spaces that are seen in the corpus.

I’d added a feature to show the portmanteaux in the output. Basically, this looks at the words in the output, and prints any that it doesn’t see in the words in the corpus. Sometimes it lists words that are in the larger set of English language words; in the image above, charNG would list the word “infect” as a portmanteau, because the corpus only contains “infection” (“infect” is created from the output element “infect,” which is created from the 5-gram “fect,” (note the comma) which is derived from such corpus elements as ” defect,”) One of the ways I like to use charNG is to just paste in a corpus, generate 4-grams, and look at the portmanteaux

Of course, just generating a bunch of text and picking out what you like is fun too. I posted some of this on Gnoetry Daily:

• Crowning the blood
• Pain of Manhood
• you see me (sucka for love)

I especially like how 4-grams worked when processed with codework parenthetical insertions:

all the reployal :f.u[n]tually she lawsui:t.er[ror] to had objections who aborited minationstar mand
his some, assed and, in Alliarace on the :w.ith[ered]eir relights
be:f.or[get]m, thearenticational to the
1.2 bi:s.ho[ut]page which on :l.us[ts]iona, while rule as Howed
t:h.or[ror]res:t.ri[ck]al familigate.]

 
– from be:g.al[lows] US govergned

There are many features that could still be added, but sometimes I think it’s best to keep the generator simple, so I don’t get caught up on one approach for too long. There are so many possibilities out there.

A clockwork toy rattles on the tabletop;
listen to it
and build another.

One of the most interesting things in developing charNG was varying the chaining approach during generation. I tried choosing between “Markov”, “one-character overlap”, or “cento/cut-up”, as described in the documentation I wrote for charNG. Basically, one-char overlap only considers the last character when deciding which n-gram to choose next. And cento/cut-up doesn’t consider any context when deciding what n-gram to pick next; it basically picks a random set of cut-ups.

This gave me the opportunity to read up on Centos a bit. Basically they’re supervised cut-ups, traditionally using classical poetry as source text; the practice is from Greek poets from before 400 BC and Roman poets thereafter. In particular, I tracked down Ausonius’ “Cento Nuptialis”, which was composed to amuse the Emperor Valentinian in the 4th century AD. On pg 373- of a Loeb Classical Library translation that’s been scanned online, Ausonius introduces his cento by saying:

This book, then hurriedly composed in a single day with some lamp-lit hours thrown in, I lately found among my rough drafts; and so great is my confidence in your sincerity and affection, that for all your gravity I could not withhold even a ludicrous production. So take a little work, continuous, though made of disjointed tags; one, though of various scraps; absurd, though of grave materials; mine, though the elements are another’s…

And if you will suffer me, who need instruction myself, to instruct you, I will expound what a cento is. It is a poem compactly built out of a variety of passages and different meanings, in such a way that either two half-lines are joined together to form one, or of one line and the following half with another half. For to place two (whole) lines side by side is weak, and three in succession is mere trifling.

Decimius Magnus Ausonius, consul, soldier, and mediocre poet, from the 21st century I your peer salute you.

You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.