استفاده از شبکه های بیزی سلسله مراتبی در تجزیه و تحلیل موضوع احساسات ساده و پیچیده

Traditional emotion models, when tagging single emotions in documents, often ignore the fact that most documents convey complex human emotions. In this paper, we join emotion analysis with topic models to find complex emotions in documents, as well as the intensity of the emotions, and study how the document emotions vary with topics. Hierarchical Bayesian networks are employed to generate the latent topic variables and emotion variables. On average, our model on single emotion classification outperforms the traditional supervised machine learning models such as SVM and Naive Bayes. The other model on the complex emotion classification also achieves promising results. We thoroughly analyze the impact of vocabulary quality and topic quantity to emotion and intensity prediction in our experiments. The distribution of topics such as Friend and Job are found to be sensitive to the documents’ emotions, which we call emotion topic variation in this paper. This reveals the deeper relationship between topics and emotions.

Emotion analysis aims to reveal the delicate human emotions found in texts, which could help to promote the quality of the human–computer interface, create more accurate review analysis, and diagnose some mental diseases (Ren, 2009 and Ren, 2010). Traditional emotion models often simplify the problem by assigning a single emotion to each document, while the document could, in fact, have multiple emotions conveyed as a whole. This situation has been found in Ren-CECps (Quan and Ren, 2010) as well as other emotion corpora, and a single-emotion assumption would not only reduce the coverage of emotions but also affect the accuracy of emotion prediction. In this paper, we employ hierarchical Bayesian networks to model the complex emotions found in documents and the emotion intensities, interpreting the significance of each single emotion. The traditional studies of emotions and topics are distinctly performed. A topic is a particular distribution over the vocabulary, and in most cases represents a cluster of words when only the top N possible words in the vocabulary are considered. One of the most discussed problems of the automatically generated topics ( Blei et al., 2003) is that we can never know the semantic meanings of these topics before generation; we only receive enlightenment by watching the top N words generated from each topic. On the other hand, through a thorough observation of the document emotions in Ren-CECps, we find that people's choice of topics (words) in writing is closely related to their emotional states. For example, the topics of Child and Family are more often observed in the Joy and Love documents than the other documents, while the words under the Job topic are found with higher frequencies in the Hate documents. For the majority of words in a corpus, which although do not directly indicate emotions, could be divided into some emotion indicative latent topics. Following this intuition, in this paper we provide a generative process to jointly encode the choice of words and topics under different emotions in the documents, and try to interpret the latent topics on the emotion level. To model the emotions and emotion-related topics, we build hierarchical Bayesian networks by incorporating the emotion and emotion intensity variables into the LDA model (Blei et al., 2003). The idea behind this is straightforward: in the emotion topic models, the word generation procedure is determined not only by the document topics that are the core distribution in LDA, but also by the emotions (intensities) of that document. This extension of the LDA model enables us to naturally incorporate topics into the variation of document emotions. In addition to the word variables, the emotion intensity variables are generated from emotion variables. This ensures important parallelism between the existing emotions and the emotion intensities. We claim the emotion topic model is a half-supervised model since the generation of document emotions, emotion intensities, and words needs previous counts that could only be collected from a training set, suggesting a supervised procedure, while the topic generation component is similar to the LDA model, which is an unsupervised procedure. Two emotion models are studied in the framework of hierarchical Bayesian networks: the simple emotion topic (SET) model for predicting simple emotions and latent topics in documents, which covers the similar task of traditional emotion classification, and the complex emotion topic (CET) model for discovering the complex emotions and topics in documents. As we would discuss in detail later, the imbalance of emotion instances in real blog articles is very common, since some emotions such as Love and Hate are statistically more often than Surprise in people's attitude toward objects. It would be difficult for machine learning algorithms to correctly recognize the infrequent emotions without proper regularizations. In our hierarchical Bayesian networks, we employ Dirichlet and Beta priors on the proportional variables for emotion and word distributions, which in turn leads to the regularization of these distributions given a training set. In this sense, our emotion topic models are expected to be more robust to the real data. We develop different methods for evaluating simple and complex emotion predictions, not only for the accuracies of emotion(s) per document, but also for the quality of single emotions contained in the complex emotions. The emotion topic variation is also examined for both models, which reveals the special emotion distribution among different topics. The rest of this paper is arranged as follows. Section 2 makes a brief review of the emotion system in Ren-CECps. Section 3 introduces the simple and complex emotion topic models. Section 4 illustrates our Gibbs sampling method for inference. In Section 5 we perform experiments on emotion prediction and emotion topic variation, and discuss the results and models’ complexities. Related works are introduced in Section 6. Finally, Section 7 concludes.

We provided two generative models for extracting the latent topics in a corpus, estimating the emotions and the emotion intensities in documents, and analyzing the emotion topic variation. The idea behind was based on the strong correlation between the written emotions and the choice of emotional words and topics in the blog articles. By incorporating prior beliefs about the word and emotion frequencies into the corresponding probabilistic distributions, the models could generalize well in the new data. In our experiment, the impact of vocabulary quality and topic quantity to emotion classification and intensity prediction was thoroughly analyzed. On average, the simple emotions generated from our models were found with higher accuracy than the output of the traditional supervised machine learning approaches, such as Naive Bayes and Support Vector Machines. For the complex emotions, we provided three emotion evaluation criteria: the E1 accuracy for the predicted single emotions among complex emotions and the accuracies of E2 and E3 for complex emotions under different matching strategies. Our experiment showed that a small word frequency threshold (e.g., four) on the vocabulary promised better results for the simple emotion classification, and a relatively larger threshold (e.g., 16) would lead to higher overall accuracies for the complex emotion prediction. Both simple and complex emotion were not sensitive to large topic numbers (e.g., J > 20), but significant increment of accuracies have been observed by increasing J from 1 to 10. Both models showed robustness for the imbalance in the data. Because emotion intensities were paired with emotions in our study, we used V1 to evaluate the accuracy of the emotion and emotion intensity pairs and V2 to evaluate the accuracy of the emotion intensities with correct emotion tags. In our experiments, the topic number and word frequency threshold have shown similar impacts on the emotion-intensity accuracy (V1) and the emotion accuracy, which indicated a topic- and vocabulary-insensitive procedure of intensity prediction (V2) in our two models. We also studied the emotion topic variations by examining the topic meanings on the emotion level and comparing the emotion proportions within different topics. Finally, we examined the models’ complexities with topic number J and corpus size D, and gave suggestions on parameter settings for different applications. Currently, our models employ the half-supervised approach to predict emotions and intensities from the documents, and the inference procedures depend on previously annotated emotion corpus. The Dirichlet and Beta priors directly take observations in a training set as parameters. One of the possible solutions to release such strict dependencies might be turning to discover the connections between document emotions and more thoroughly divided word/topic features in the unsupervised manner, such as the degree and negative modifiers and the disjunctive conjunctions, which have been observed to suggest emotions and intensities in our emotion corpus, as well as the features which indicate context information (e.g., the dependency structures). More flexibilities can be put in the priors, such as the higher level informative distributions in Dirichlet process, so that the model could adjust parameters (e.g., division of the word features) better to the real text, and the connection of different word features and document emotions could be updated through the inference procedure given new examples. In addition, the performance of predicting document emotions from the latent topics could be affected the “quality” of these generated topics. Because the size of our current emotion corpus is relatively small, we do not have sufficient statistics to develop precise enough parameters. Nonetheless, with the unsupervised technologies, we hope to explore the emotions of large amount of documents from the Internet, and develop the topics of better quality for emotion prediction. Last but not least, the accurate emotional labels on sentences and words could provide extremely useful evidence for predicting the document emotions, as shown by the examples in Ren-CECps. However, because manually annotating these labels are very expensive, in practice we are expecting to develop models that automatically generate the word/sentence emotions so as to take advantage of these features for the document emotion classification in the future work.