Text generation is one of the important problems in Natural Language Processing field. The former methods for text generation that are based on language modeling by the teacher forcing approach encounter the problem of discrepancy between the training and test phases an More
Text generation is one of the important problems in Natural Language Processing field. The former methods for text generation that are based on language modeling by the teacher forcing approach encounter the problem of discrepancy between the training and test phases and also employing an inappropriate objective (i.e., Maximum Likelihood estimation) for generation. In the past years, Generative Adversarial Networks (GANs) have achieved much popularity due to their capabilities in image generation. These networks have also attracted attention for sequence generation in the last few years. However, since text sequences are discrete, GANs cannot be easily employed for text generation, and new approaches like Reinforcement Learning and approximation have been utilized for this purpose. Furthermore, the instability problem of GANs training causes new challenges.
In this paper, a new GAN-based ensemble method is proposed for sequence generation problem. The idea of the proposed method is based on the ratio estimation which enables the model to overcome the problem of discreteness in data. Also, the proposed method is more stable than the other GAN-based methods. It also should be noted that the exposure bias problem of teacher forcing approach does not exist in the proposed method.
Experiments show the superiority of the proposed method to previous GAN-based methods for text generation.
Manuscript profile
