Generating diverse successive sentences via context learning

Abstract

A number of neural models have been proposed in the literature to generate meaningful and grammatically correct sentences. However, those models have drawbacks such as lack of diversity or inability to generate successive sentences. To apply NLP neural models in real-world tasks like dialogues, simply predicting a next sentence at a time using autoregressive models or sampling sentences from trained generative models is not enough. In this thesis, as an extension of existing sentence generation research, we propose a new approach for generating diverse successive sentences. We base our model in variational auto-encoder (VAE) and combine it with an additional recurrent network for successive context learning. We evaluate formulated sentences with the type token ratio (TTR) measure and show that our model is superior to the base model on diversity. Furthermore, by presenting a method of conditioning our model on another latent codes, we show that it is also possible to change flows of generated sentences differently depending on the given latent code.