Capturing semantics scattered across entire text is one of the important issues for NLP tasks. It would be particularly critical with long text embodying a flow of themes. This paper proposes a new text modeling method that can handle thematic flows of text with Deep Neural Networks (DNN) in such a way that discourse information and distributed representations of text are incorporate. Unlike previous DNN-based document models, the proposed model enables discourse-aware analysis of text and composition of sentence-level distributed representations guided by the discourse structure. More specifically, my method identifies Elementary Discourse Units (EDUs) and their discourse relations in a given document by applying Rhetorical Structure Theory (RST)-based discourse analysis. The result is fed into a tree-structured neural network that reflects the discourse information including the structure of the document and the discourse roles and relation types. I evaluate the document model for two document-level text classification tasks, sentiment analysis and sarcasm detection, with comparisons against the reference systems that also utilize discourse information. In addition, I conduct additional experiments to evaluate the impact of neural network types and adopted discourse factors on modeling documents vis-à-vis the two classification tasks. Furthermore, I investigate the effects of various learning methods, input units on the quality of the proposed discourse-aware document model.