Automatic Writing With Deep Learning
Making an AI system capable of producing anything sensible is hard. Take the task of sentiment analysis, where it is quite unclear what the agreement between experts is.
Join the DZone community and get the full member experience.Join For Free
A quite many machine learning and deep learning problems are directed at building a mapping function of roughly the following form:
Input X ---> Output Y
X is some sort of an object, e.g. an email text, an image, or a document.
Y is either a single class label from a finite set of labels, like spam/no spam, detected object or a cluster name for this document, or some number, like salary, in the next month or stock price.
While such tasks can be daunting to solve (like sentiment analysis or predicting stock prices in real-time), they require rather clear steps to achieve good levels of mapping accuracy. Again, I'm not discussing situations with lack of training data to cover the modeled phenomenon or poor feature selection.
In contrast, somewhat less straightforward areas of AI are the tasks that present you with a challenge of predicting as fuzzy structures as words, sentences, or complete texts. What are the examples? Machine translation for one, natural language generation for another. One may argue, that transcribing audio to text is also such type of mapping, but I'd argue it is not. Audio is a "wave" and the speech detection is an okay-solved task (with state of the art above 90% of accuracy). However, such an algorithm does not capture the meaning of the produced text, except for where it is necessary to do the disambiguation of what was said. Again, I have to make it clear that the audio -> text problem is not at all easy with and comes with its own intricacies, like handling speaker self-corrections, noise and so on.
Lately, the task of writing texts with a machine (e.g. here) caught my eye on Twitter. Previously, papers from Google on writing poetry or other text-producing software were giving me creepy feelings. I somehow undermined the role of such algorithms in the space of natural language processing and language understanding and saw only diminishing value of such systems to users. Again, any challenging tasks might be solved and even bring value to solving other challenging tasks. But who would use an automatic poetry writing system? Why would somebody, I thought, use these systems just for fun? My practical mind battled against such "fun" algorithms. Again, making an AI/NLProc system capable of producing anything sensible is hard. Take the task of sentiment analysis, where it is quite unclear what the agreement between experts is, not to mention non-experts.
I think this post has poured enough text onto the heads of my readers. I will use this post as a self-motivating mechanism to continue the research with systems producing text. My target is to complete the neural network training on the text from my Master thesis and show you some examples for your judgment of the usefulness of such systems.
Published at DZone with permission of Dmitry Kan, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.