Monthly Archives: February 2017

Can a recurrent neural network write good prose?

At this stage in my PhD research into literary style I am looking to machine learning and neural networks, and moving away from stylostatistical methodologies, partially out of fatigue. Statistical analyses are intensely process-based and always open, it seems to me, to fairly egregious ‘nudging’ in the name of reaching favourable outcomes. This brings a kind of bathos to some statistical analyses, as they account, for a greater extent than I’d like, for methodology and process, with the result that the novelty these approaches might have brought us are neglected. I have nothing against this emphasis on process necessarily, but I do also have a thing for outcomes, as well as the mysticism and relativity machine learning can bring, alienating us as it does from the process of the script’s decision making.

I first heard of the sci-fi writer from a colleague of mine in my department. It’s Robin Sloan’s plug-in for the script-writing interface Atom which allows you to ‘autocomplete’ texts based on your input. After sixteen hours of installing, uninstalling, moving directories around and looking up stackoverflow, I got it to work.I typed in some Joyce and got stuff about Chinese spaceships as output, which was great, but science fiction isn’t exactly my area, and I wanted to train the network on a corpus of modernist fiction. Fortunately, I had the complete works of Joyce, Virginia Woolf, Gertrude Stein, Sara Baume, Anne Enright, Will Self, F. Scott FitzGerald, Eimear McBride, Ernest Hemingway, Jorge Luis Borges, Joseph Conrad, Ford Madox Ford, Franz Kafka, Katherine Mansfield, Marcel Proust, Elizabeth Bowen, Samuel Beckett, Flann O’Brien, Djuna Barnes, William Faulkner & D.H. Lawrence to hand.

My understanding of this recurrent neural network, such as it is, runs as follows. The script reads the entire corpus of over 100 novels, and calculates the distance that separates every word from every other word. The network then hazards a guess as to what word follows the word or words that you present it with, then validates this against what its actuality. It then does so over and over and over, getting ‘better’ at predicting each time. The size of the corpus is significant in determining the length of time this will take, and mine required something around twelve days. I had to cut it off after twenty four hours because I was afraid my laptop wouldn’t be able to handle it. At this point it had carried out the process 135000 times, just below 10% of the full process. Once I get access to a computer with better hardware I can look into getting better results.

How this will feed into my thesis remains nebulous, I might move in a sociological direction and take survey data on how close they reckon the final result approximates literary prose. But at this point I’m interested in what impact it might conceivably have on my own writing. I am currently trying to sustain progress on my first novel alongside my research, so, in a self-interested enough way, I pose the question, can neural networks be used in the creation of good prose?

There have been many books written on the place of cliometric methodologies in literary history. I’m thinking here of William S. Burroughs’ cut-ups, Mallarmé’s infinite book of sonnets, and the brief flirtation the literary world had with hypertext in the 90’s, but beyond of the avant-garde, I don’t think I could think of an example of an author who has foregrounded their use of numerical methods of composition. A poet friend of mine has dabbled in this sort of thing but finds it expedient to not emphasise the aleatory aspect of what she’s doing, as publishers tend to give a frosty reception when their writers suggest that their work is automated to some extent.

And I can see where they’re coming from. No matter how good they get at it, I’m unlikely to get to a point where I’ll read automatically generated literary art. Speaking for myself, when I’m reading, it is not just about the words. I’m reading Enright or Woolf or Pynchon because I’m as interested in them as I am in what they produce. How synthetic would it be to set Faulkner and McCarthy in conversation with one another if their congruencies were wholly manufactured by outside interpretation or an anonymous algorithmic process as opposed to the discursive tissue of literary sphere, if a work didn’t arise from material and actual conditions? I know I’m making a lot of value-based assessments here that wouldn’t have a place in academic discourse, and on that basis what I’m saying is indefensible, but the probabilistic infinitude of it bothers me too. When I think about all the novelists I have yet to read I immediately get panicky about my own death, and the limitless possibilities of neural networks to churn out tomes and tomes of literary data in seconds just seems to me to exacerbate the problem.

However, speaking outside of my reader-identity, as a writer, I find it invigorating. My biggest problem as a writer isn’t writing nice sentences, given enough time I’m more than capable of that, the difficulty is finding things to wrap them around. Mood, tone, image, aren’t daunting, but a text’s momentum, the plot, I suppose, eludes me completely. It’s not something that bothers me, I consider plot to be a necessary evil, and resent novels that suspend information in a deliberate, keep-you-on-the-hook sort of way, but the ‘what next’ of composition is still a knotty issue.

The generation of text could be a useful way of getting an intelligent prompt that stylistically ‘borrows’ from a broad base of literary data, smashing words and images together in a generative manner to get the associative faculties going. I’m not suggesting that these scripts would be successful were they autonomous, I think we’re a few years off one of these algorithms writing a good novel, but I hope to demonstrate that my circa 350 generated words would be successful in facilitating the process of composition:

be as the whoo, put out and going to Ingleway effect themselves old shadows as she was like a farmers of his lake, for all or grips — that else bigs they perfectly clothes and the table and chest and under her destynets called a fingers of hanged staircase and cropping in her hand from him, “never married them my said?” know’s prode another hold of the utals of the bright silence and now he was much renderuched, his eyes. It was her natural dependent clothes, cattle that they came in loads of the remarks he was there inside him. There were she was solid drugs.

“I’m sons to see, then?’ she have no such description. The legs that somewhere to chair followed, the year disappeared curl at an entire of him frwented her in courage had approached. It was a long rose of visit. The moment, the audience on the people still the gulsion rowed because it was a travalious. But nothing in the rash.

“No, Jane. What does then they all get out him, but? Or perfect?”

“The advices?”

Of came the great as prayer. He said the aspect who, she lay on the white big remarking through the father — of the grandfather did he had seen her engoors, came garden, the irony opposition on his colling of the roof. Next parapes he had coming broken as though they fould

has a sort. Quite angry to captraita in the fact terror, and a sound and then raised the powerful knocking door crawling for a greatly keep, and is so many adventored and men. He went on. He had been her she had happened his hands on a little hand of a letter and a road that he had possibly became childish limp, her keep mind over her face went in himself voice. He came to the table, to a rashes right repairing that he fulfe, but it was soldier, to different and stuff was. The knees as it was a reason and that prone, the soul? And with grikening game. In such an inquisilled-road and commanded for a magbecross that has been deskled, tight gratulations in front standing again, very unrediction and automatiled spench and six in command, a

I don’t think I’d be alone in thinking that there’s some merit in parts of this writing. I wonder if there’s an extent to which Finnegans Wake has ‘tainted’ the corpus somewhat, because stylistically, I think that’s the closest analogue to what could be said to be going on here. Interestingly, it seems to be formulating its own puns, words like ‘unrediction,’ ‘automatiled spench’ (a tantalising meta-textual reference I think) and ‘destynets’, I think, would all be reminiscent of what you could expect to find in any given section of the Wake, but they don’t turn up in the corpus proper, at least according to a ctrl + f search. What this suggests to me is that the algorithm is plotting relationships on the level of the character, as well as phrasal units. However, I don’t recall the sci-fi model turning up paragraphs that were quite so disjointed and surreal — they didn’t make loads of sense, but they were recognisable, as grammatically coherent chunks of text. Although this could be the result of working with a partially trained model.

So, how might they feed our creative process? Here’s my attempt at making nice sentences out of the above.

— I have never been married, she said. — There’s no good to be gotten out of that sort of thing at all.

He’d use his hands to do chin-ups, pull himself up over the second staircase that hung over the landing, and he’d hang then, wriggling across the awning it created over the first set of stairs, grunting out eight to ten numbers each time he passed, his feet just missing the carpeted surface of the real stairs, the proper stairs.

Every time she walked between them she would wonder which of the two that she preferred. Not the one that she preferred, but the one that were more her, which one of these two am I, which one of these two is actually me? It was the feeling of moving between the two that she could remember, not his hands. They were just an afterthought, something cropped in in retrospect.

She can’t remember her sons either.

Her life had been a slow rise, to come to what it was. A house full of men, chairs and staircases, and she wished for it now to coil into itself, like the corners of stale newspapers.

The first thing you’ll notice about this is that it is a lot shorter. I started off by traducing the above, in as much as possible, into ‘plain words’ while remaining faithful to the n-grams I liked, like ‘bright silence’ ‘old shadows’ and ‘great as prayer’. In order to create images that play off one another, and to account for the dialogue, sentences that seemed to be doing similar things began to cluster together, so paragraphs organically started to shrink. Ultimately, once the ‘purpose’ of what I was doing started to come out, a critique of bourgeois values, memory loss, the nice phrasal units started to become spurious, and the eight or so paragraphs collapsed into the three and a half above. This is also ones of my biggest writing issues, I’ll type three full pages and after the editing process they’ll come to no more than 1.5 paragraphs, maybe?

The thematic sense of dislocation and fragmentation could be a product of the source material, but most things I write are about substance-abusing depressives with broken brains cos I’m a twenty-five year old petit-bourgeois male. There’s also a fairly pallid Enright vibe to what I’ve done with the above, I think the staircases line could come straight out of The Portable Virgin.

Maybe a more well-trained corpus could provide better prompts, but overall, if you want better results out of this for any kind of creative praxis, it’s probably better to be a good writer.

Modelling Humanities Data Blog Post #1 Deleuze, Descartes and Data to Knowledge

While dealing with the distinctions between data, knowledge and information in class, a pyramidal hierarchy was proposed, which can be seen on the left. This diagram discloses the process of making data (which have been defined as ‘facts’ which exist in the world), into information, and thereafter knowledge. These shifts from one state to another are not as neat as the diagram might suggest; it is just one interpretation giving shape to a highly dynamic and unsettled process; any movement from one of these levels to another is fraught. It is ‘a bargaining system,’ as every dataset has its limitations and aporias, not to speak of the process of interpretation or subsequent dissemination. This temporal dimension to data, its translation from a brute state is too often neglected within certain fields of study, fields in which data is more often understood as unambiguous, naturally hierarchicalised, and not open to contextualisation or debate.

This blog post aims to consider these issues within the context of a dataset obtained from The Central Statistics Office. The dataset contains information relating to the relative risk of falling into poverty based on one’s level of education between the years 2004 and 2015 inclusive. The data was analysed through use of the statistical analysis interface SPSS.

The purpose of the CSO is to compile and disseminate information relating to economic and social conditions within the state in order to give direction to the government in the formulation of policy. Therefore it was decided that the most pertinent information to be derived from the dataset would be the correlations between level of education and the likelihood of falling into poverty. The results appear below.

Correlation Between Risk of Poverty and Level of Education Achieved

Correlation Between Consistent Poverty (%) and Level of Education Received

Correlation Between Deprivation Rate (%) and Level of Education Received

Poverty Risk Based on Education Level

Deprivation Rate Based on Education Level

Consistent Poverty Rate based on Education Level

It can be seen that there is a very strong negative correlation between one’s level of education and one’s risk of exposure to poverty; the higher one ascends through the education system, the less likely it is one will fall into economic liminality. This is borne out both in the bar charts and the correlation tables, the latter of which yield p-values of .000, underlining the certainty of the finding. It should be noted that both graphing the data, and detecting correlations through use of the Spearman’s rho are elementary statistical procedures, but as the trend revealed here is consistent with more elaborate modelling of the relationship,[1] the parsimonious analysis carried out here is all that is required.

It should not be assumed that just because these graphs are informative that it is impossible to garner information from data in any other way. Even in its primary state, as it appears on the website, one could obtain information from a dataset through qualitative means. It is unlikely that this information will be as coherent as that which that can be gleaned from even the most basic graph, but it is important to emphasise the fact that the border that separates data from information is fluid.

It is unlikely to be a novel finding that those who have a third level education have higher incomes than those who do not; there is a robust body of research detailing the many benefits of attending university. [2] Therefore, can it be said that the visualisation of the dataset above has contributed to knowledge? One would answer this question relative to one’s initial research question, and how the information complicates or advances it. If the causal relationship between exposure to poverty and level of education has been confirmed, and a government agency makes the recommendation that further investment in educational support programmes are necessary, it is somewhere in this process that the boundary separating information from knowledge has been crossed.

The above diagram actualises the temporal nature of data to a greater extent than the pyramid, but in doing so it perpetuates a linearisation of the process, a line along which René Descartes’ notion of thought could be said to align. Descartes understood thought as a positive function which tends towards the good and toward truth. This ‘good sense’, allows us to ‘judge correctly and to distinguish the true from the false’.[3] Gilles Deleuze believes Descartes instantiates a model of thought which is oppressive, and which perceives thinking relative to external needs and values rather than in its actuality: ‘It cannot be regarded as fact that thinking is the natural exercise of a faculty, and that this faculty is possessed of a good nature and a good will.’[4]

In Deleuze’s conception, thought takes on a sensual disposition, reversing the Cartesian notion of mental inquiry beginning from a state of disinterestedness in order to arrive at a moment at which one recognises ‘rightness’. Deleuze argues that there is no such breakthrough moment or established methodology to thought, and argues for regarding it as more invasive, or unwelcome, a point of encounter when ‘something in the world forces us to think.’[5]

Rather than taking the neat, schematic movement from capturing data to modelling to interpreting for granted, Deleuze is engaged by these moments of crisis, points just before or just after the field of our understanding is qualitatively transformed into something different:

How else can one write but of those things which one doesn’t know, or know badly?…We write only at the frontiers of our knowledge, at the border which separates our knowledge from our ignorance and transforms one into the other.[6]

Deleuze’s comments have direct bearing upon our understanding of data, and how they should be understood within the context of the wider questions we ask of them. Deleuze argues that, ‘problems must be considered not as ‘givens’ (data) but as ideal ‘objecticities’ possessing their own sufficiency and implying acts of constitution and investment in their respective symbolic fields.’[7] While it is possible that Deleuze would risk overstating the case, were we to apply his theories to this dataset, it is nonetheless crucial to recall that data, and the methodologies we use to unpack and present them participate in wider economies of significance, ones with indeterminate horizons.


[1] Department for Business, Education and Skills, ‘BIS Research Paper №146: The Benefits of Higher Education and Participation for Individuals and Society: Key Findings and Reports’, (Department for Business, Education and Skills: 2013)

[2] OECD, Education Indicators in Focus, (OECD: 2012)

[3] Descartes, René, Discourse on the Method of Rightly Conducting the Reason, and Seeking Truth in the Sciences (Gutenberg: 2008),

[4] Deleuze, Gilles, Difference and Repetition (Bloomsbury Academic: 2016), p.175

[5] Ibid.

[6] Ibid, p. xviii

[7] Ibid, p.207


Deleuze, Gilles, Difference and Repetition (Bloomsbury Academic: 2016), p.175

Department for Business, Education and Skills, ‘BIS Research Paper №146: The Benefits of Higher Education and Participation for Individuals and Society: Key Findings and Reports’, (Department for Business, Education and Skills: 2013)

Descartes, René, Discourse on the Method of Rightly Conducting the Reason, and Seeking Truth in the Sciences (Gutenberg: 2008),

OECD, Education Indicators in Focus, (OECD: 2012)

A Derridean account of literary style

The boldness of the title here needs to be put in check immediately, I’ve only read the Grammatology recently, and though this was the first reading where I think I made the sense of it, I still haven’t read Lévi-Strauss or Rousseau, so in actuality, my reading can g.t.f.o.

Helpfully, Gayatri Chakravorty Spivak provides one of the better introductions of all time, which allows us to get underway in terms of considering style.

It could be said that Derrida’s philosophy launched a thousand styles, much to David Foster Wallace’s chagrin; his MFA students had an uncanny capacity to encourage many of them down the route of irony-poisoned multi-vocality, typographical playfulness, all in the name of his casting adrift an economy of relativised meaning, at the expense of bourgeois, post-Enlightenment certainty. This led him eventually to begin each semester by writing the names of the doyennes of deconstruction on the chalkboard, to announce ‘I’m read all these guys. You don’t need to remind me of them’.

However, we must not mistake the historicised post-structuralist movement for Derrida’s stated views on style, which, as far as I can see are unfortunately absent from the Grammatology. But, if we say that Derrida viewed the pursuit of stable meaning sceptically, it’s probable that seeking to discover a single, unified style in any textual artefact, would be likewise wrong-headed, as Spivak points out in her introduction

The desire for unity and order compels the author and the reader to balance the equation that is the text’s symptom.

A single authorial style is a romantic contrivance, and violates Derrida’s sense of textuality, which is autonomous from such concerns, and does not answer to ‘proper names.’

We know that the metaphor that would describe the genealogy of the text correctly is still forbidden. In its syntax and its lexicon, its spacing, by its punctuation, its lacunae, its margins, the historical appurtenance of a text, is never a straight line. It is neither causality by contagion nor the simple accumulation of layers. Not even the pure juxtaposition of borrowed pieces.

Of all the words in this rather dazzling paragraph, it is ‘layers’ that strikes me most forcefully, if only for the reason that ‘layers’ is the way in which I decided to envision and visualise literary data in the course of my thesis. There is a risk in giving into Derrida that one would merely come away with some nebulous kind of radical indeterminacy, rather than a constructivist paradigm, which would be more necessary in the carrying out of quantitative analytical procedures. In fact, the kind of theological everything/nothing that Derrida, ironically tends to engender, is exactly what I want to avoid.

What does stand out in this paragraph, is the text’s interconnectedness, and every part’s responsiveness to every other part. This is a key feature of Derrida’s mode of critique; in later chapters, he will locate minor, or tangential sections of Rousseau or Lévi-Strauss, minor details or afterthoughts, that reveal themselves to be the precise juncture at which their systems of thought lapse into incoherence and uncertainty. A ‘total’ view of style then, one which reveals it to be wide-ranging and prone to upset, composed on a granular level of fragmentary particles, is something that Derrida might offer us in comprehending style.

A Heideggerian account of literary style

Martin Heidegger is a philosopher who had a very specific idea of the kind of philosophy he wished to practice and as such, he doesn’t make it easy for those who wish to extract something of use from his system of thought for use elsewhere, as in, for example, literary studies. His primary interest was in the nature of Being, what we might simplistically define as ontology, less simplistically, the ontology of ontology.

His style is famous for its obtuseness and difficulty, and in my own estimation, Heidegger would be less an author who demands multiple readings, than one who requires a lifetime of serious study. Unlike Nietzsche, it can hardly be said that he endorses this praxis as a proper stylistic mode. Instead, he envisioned literature, which he refers to mostly as ‘poetry’, as an extension of his own philosophical work, in establishing the nature of Being.

The only material that we can harvest from his collection of hermeneutic writings, Poetry, Language, Thought which seem relevant to literary stylistics, comes in the second chapter, ‘The Origin of the Work of Art’. Enquiring into the nature of poetry involves, for Heidegger, an enquiry into its origin, in the artist and the artist’s activities. Getting to what the artist is is a difficult matter also; both seem to depend on one another as categories:

it is the work that first lets the artist emerge as a master of his art. The artist is the origin of the work. The work is the origin of the artist. Neither is one without the other. Nevertheless, neither is the sole support of the other. In themselves and in their interrelations artist and work are each of them by virtue of a third thing which is prior to both…art.

This is an unfashionable view; reflecting the increasingly social, collaborative nature of the humanities, we might increasingly wish to understand style as a likewise collaborative phenomenon, a social entity which allows for both the expression of a historical tradition and an individual idiom simultaneously.

Not for Heidegger. For him, style is somewhat beside the point, and elucidating it is a symptom of our decadent modernity, our tendency towards using things as means to ends, rather than ends in themselves. Elaborating on a text’s stylistic features, is to engage with rather facile aspects of its thingliness:

a thing is not merely an aggregate of traits, nor an accumulation of properties by which…an aggregate arises. A thing…is that around which the properties have assembled.

A style in which a thing, (and I should say, he’s talking about a jug or a stone here, hardly a novel or poem) appears does not define its thingliness completely, but it definitely partakes in it. Rather than having a secure sense of style, we have an aporia, direct information on the difficulty of confronting it methodologically, because of our fallen culture. Rather than grappling with style in its actuality, we only list traits, and thereby we come to an understanding of a thing-concept, rather than thing.

In resolving this, we might construct ‘a free field to display its tingly character directly,’ in such a way that that which interposes itself between the interpreter and an understanding of thingly nature, would be set aside. Of course, Heidegger is a pessimist regarding the success of this endeavour:

There is much in being that man cannot master. There is but little that comes to be known. What is known remains inexact, what is mastered insecure…When we contemplate this whole as one, then we apprehend, so it appears, al that is — though we grasp it crudely enough.

A Nietzschean account of literary style

When we think about Frederich Nietzsche and literary style, we might think of one of two things, the first being the conceptual distinction he drew in his first major work, The Birth of Tragedy, between the Apollonian and Dionysian modes. I won’t be pursuing this mode of analysis because I haven’t found a way to make it relevant to my own dissertation, yet, but I will draw our attention to what might be the second thing we think about which connects Nietzsche to style, and that is his own style.

Speaking purely about the only two works of Nietzsche I’ve read, Human, All too Human and Beyond Good and Evil, it can seem as though Nietzsche’s interest in style is less in dealing with the matter systematically, but more in embodying the notion, in his own epigrammatic, aphoristic and restless manner of writing, which seems, as time goes on, to become more and more pertinent to more contemporary forms of media. He’s been usefully described as a newspaper columnist or pundit (in that the philosophical antecedents he engages with he probably hasn’t read as systematically as he lets on, he was a philologist and a classicist rather than a philosopher), blogger (for the same reason) and I’m sure I wouldn’t be the first to say he would’ve fit in well on twitter, were I a vulgar enough character to make those sorts of analogies.

This restlessness when it comes to explicating the finer points of his work is what I mean when I say he embodies his own notion of style; by paying attention to his works’ formal strictures, or the artifice of their non-existence, we might be able to develop a more engaging model of Nietzsche’s stylistics than he can provide us with. For example, in the introduction to Human, all too Human, Ray Furness describes how Nietzsche perceives truth as feminine:

and those dogmatic souls afflicted with a terrible seriousness will be unable to woo her. A light, more elegant path is needed, a fresh and sparkling approach.

This ‘fresh and sparkling approach’ is Nietzsche’s ‘gay science’, which could be summed up as a curative to the Enlightenment project, which Nietzsche viewed as restrictive and overly systematic. As the translator says:

his preference for a darting perspectivism allows him to use dazzling contradictions to disorientate the reader and force them…to do the necessary work of weighting…and evaluating.

His commitment to a multiplicity of thought in his philosophy would encourage us to align Nietzsche’s commitment to multiperspectivalism, with a certain disposition towards multiplicity in literary style. This isn’t really an extravagant interpretation, Nietzsche is positively effusive about artists who display a capacity to be ‘on the move’ stylistically.

One author who he singles out for praise on this basis is Laurence Sterne. Sterne’s novel, The Life and Opinions of Tristram Shandy, A Gentleman, is so stylistically restless, it fails to assume any kind of stable narrative shape at all, which Nietzsche endorses unequivocally:

an artistic style in which the definite form is continually broken, thrust aside and transferred to the realm of the indefinite, so that is signifies one and the other at the same time

So his tastes are fairly obvious, multi-vocality, oscillation of the writing’s attention regarding its subject, ambiguity/irony. The reasons for this are fairly obvious, everything about Nietzsche’s thought repudiates convention and the formation of habit, which makes us overly familiar with and lax in our approaches to objects under inquiry.

Interestingly, for the purposes of my own research into modernism, Nietzsche sees plurality of styles as a modern tendency, and partially attributable to the growth in urban living. Modern iterations of literary forms will be formally restless, Nietzsche argues, as this is the new paradigm: “they [the moderns] are in all things rather too thorough to be able to settle like the men of other days.”

While I don’t think I could get away with calling Richard Wagner modernist, he was one of Nietzsche’s contemporaries, and Nietzsche regards him as pertinent in delineating the modern turn, as he saw it at the time:

This kind of music expresses best what I think of the Germans: they belong to the day before yesterday and the day after tomorrow — they have as yet no today.

Like Hans Georg-Gadamer after him, Nietzsche understood style as a reactionary force in many respects, enforcing normative standards on particular works. In Nietzsche’s estimation, pre-modern works were written for performance, for the benefit of the ear. In the modern age, a key distinguishing feature of style is its appeal to the eye. In both cases, Nietzsche regards this as having their own impediments, demonstrating that he is unlikely to commit to signing off any one kind of style as sufficient in itself, it will always be a matter of overturning, experimenting, pushing the boundaries of hegemonic categories, until they break apart.