Image captioning with 2-layer LSTM network for combining visual attributes

Abstract

Image captioning is a rising ？eld of study in Arti？cial Intelligence, since both computer vision and natural language processing are involved in this task. We propose a novel 2-layer Long Short-Term Memory network architecture for generating a caption which describes an image. Our model consists of two LSTMs which play different roles in generating sentences: one is combining visual attributes extracted from a convolutional neural network and the other is decoding it. We train an image feature extractor for the purpose of extracting multiple objects as well. Our model is validated with Microsoft COCO dataset, and the results show that it outperforms other state-of-the-art models in evaluation metrics, BLEU, METEOR and CIDEr.