A Brief Guide to Emotion Cause Pair Extraction in NLP

- Advertisement -


With the rapid development of social network platforms, more and more people share their experiences and feelings online. Therefore, the task of sentiment analysis of online texts is important in natural language processing. Sometimes, it is also important to know the cause of the observed emotion. You might be wondering why we need to identify the cause of a particular emotion. To understand this, for example, Samsung wants to find out why people love or hate the Note 7 rather than the distribution of different emotions. In such situations, one must understand the reason behind the feeling.
As the name suggests, this act of extracting the cause of emotion and emotion is known as ECPE (Emotion Cause Pair Extraction). ECPE is an emerging natural language processing task based on ECE (Emotion Cause Extraction), which involves interpreting a document to extract the causes of emotion. ECE and ECPE are two different approaches to extracting feelings and causes of feelings. Without any delay, let’s get into the details of ECE and ECPE.

- Advertisement -

Example of Emotion Cause Pair Extraction

- Advertisement -

Figure 1 shows the general function of the ECPE task. It is essential to begin by numbering the sections in a document before identifying the sentiment and its cause. Sections are what make up a document. Hence ECPE can be considered as work at clause level. Figure 1 shows an example statement: “Last week, I lost my phone while shopping; I’m feeling sad now.” The first section refers to the last week, the second the day I lost my phone while shopping, and the third how I’m feeling. After the numbering is done, it comes down to the emotion and its cause. The current emotion of the person is sadness and the reason why he is experiencing sadness is because he lost his phone while shopping. I hope this illustration explains the ECPE in general. Without that, let’s delve deeper into this topic.

- Advertisement -

What is the difference between ECE and Emotion Cause Pair Extraction?

Before getting into ECPE, let us first understand the previous approach of ECPE and its drawbacks. The purpose of the ECE is to remove the possible causes of a given emotion clause. This requires annotating emotional cues in a document. To get a clear idea about ECE and ECPE, let’s look at a comparison example shown in Figure 2.

Difference Between ECE and Emotion Cause Pair Extraction |  emotion causal pair extraction

The document in Figure 2 depicts the statement, “Yesterday morning, a policeman visited the old man with the lost money, and told him that the thief had been caught. The old man was overjoyed and put the money in the bank.” Deposited.”, 1st section “Yesterday morning”, “A policeman visited old man with lost money” 2nd section and so on. The next step is to draw the causal clause in case of ECE with the sentiments that have already been annotated. In the given example, the “happy” emotion is already annotated in the document, and the task now is to identify the causal clauses associated with the emotion. Whereas in the case of ECPE, the feeling “happy” is not interpreted. So the function of ECPE is to extract both the emotion clause and its reason clause.

Drawbacks of ECE

You might be wondering what are the limitations of ECE now that you know the differences between ECE and ECPE; There are two major drawbacks here:

Emotions must be annotated prior to causal extraction in the test set, limiting the applications of ECE to real-world scenarios. The second is that the way of first interpreting the feeling and then extracting the reasons is mutually indicative. These are the two main justifications for the new ECPE approach.

Emotion’s workflow causes pair extraction

Let’s first look at the definition of the emotion-causal pair extraction task. Given a document with multiple volumes, d = [c1, c2, …, c|d|]The goal of ECPE is to extract a set of emotion-causal pairs in d:

p = {…, (what, cc), …}

where CE is a sentiment clause and CC is the corresponding reason clause. After understanding the problem statement, let’s come to the approach to the ECPE task.

The ECPE task has been proposed as a two-step framework, which performs individual emotion extraction and cause extraction and then performs emotion-cause pairing and filtering. Now, the goal of the first step is to extract a set of sentiment clauses E = {c1e, …, cme} and reason clauses C = {c1c, …, cnc} for each document. The next step is to combine the emotional set E and the reason set C by applying a Cartesian product, which gives a set of emotion-causal pairs. A filter is then trained to eliminate pairs that do not have a causal relationship between emotion and reason. This is how ECPE as a whole works.

A Brief Overview of the Approach

After discussing the workflow of ECPE, let us now come to its approach. As already mentioned, a two-stage approach has been proposed for extracting emotion-causal pairs. The first step is to extract a set of emotions and a set of reasons, and the second is to perform emotion-causal pairing and filtering. Looking at the first task approach, two types of multi-task learning networks i.e. independent multi-task learning and interactive multi-task learning can be used to accomplish the first task. Let’s explore the approaches in more detail.

independent multi-task learning

A document section d = . is a collection of [c1, c2, …, c|d|] , A clause ci, in turn, is a collection of words ci = [wi, 1, wi, 2, …, wi, |ci|], A two-layer hierarchical bi-LSTM network is employed to capture the word-segment document structure. The lowest layer consists of a collection of word-level bi-LSTM modules that correspond to a single segment and collect the relevant data for each word. The hidden position of the jth term in ith section hi,j is obtained on the basis of bi-directional LSTM. Then the attention mechanism is adopted to obtain a segment representation. It’s all about the bottom layer.
Now, moving on to the upper layer, the upper layer consists of two components: emotion extraction and reason extraction. The output of the lowest layer is fed as the input of the upper layer, i.e. independent clause representations received at the lower layer [s1, s2, …, s|d| ],

Model for Independent Multitasking Learning (INDIP) |  emotion causal pair extraction

The context-aware representation of the CI phrase is one that represents the hidden states of a two-component bi-LSTM, Re and Rec. These hidden states are then fed to the softmax layer, which predicts emotions and causes.

The superscripts E and C denote emotion and reason, respectively. Then the model loss can be obtained as the weighted sum of the two components.

where le and lc are the cross-entropy error of sentiment prediction and causal prediction, respectively, and is a tradeoff parameter.

Interactive Multi-Task Learning:

Unlike independent multi-task learning, the two components in the upper layer are not independent of each other in interactive multi-task learning. Since emotion and reason extraction are not mutually independent, the interactive multi-task learning approach, being not independent, gives us better results than independent multi-task learning. On the one hand, providing feelings can help to better discover the reasons; On the other hand, knowing the reasons can help to extract feelings more accurately.

Two Models for Interactive Multi-Task Learning

There are two methods called Inter-EC and Inter-CE. The method that uses emotion extraction to improve causality is called Inter-EC, and the method that uses causal extraction to improve emotion extraction is called Inter-CE. Figure 4 shows two models proposed for interactive multi-tasking learning. The task of the lower layer is similar to that of independent multitasking learning. The upper layer consists of two components that are used to predict sentiment extraction and to interactively cause extraction.
Looking at the Inter-EC in detail, the outputs of the lower layer [s1, s2, …, s|d| ], independent segment representations are fed as inputs to the first component for sentiment extraction. The hidden state of the segment-level bi-LSTM rie is used as a feature to predict the distribution of the ith segment. , The predicted label of the ith clause is embedded as a vector Yie, which is used for the second component. For causal extraction, the second component takes as input, whereRepresents combination operations. The hidden state of the segment-level bi-LSTM ric is used to predict the distribution of the ith segment. , The loss function is the same as that used for independent multitasking learning.


This blog has been a tutorial to understand the basic overview of Emotion Cause Pair Extraction. Summing up the article, let’s note that we discussed earlier what Emotion Cause Pair Extraction is, its historical perspective, why we need it, and how it works. There are many techniques, such as Graph Convolutional Networks, to extract emotion and reason. Although there are many more methods than those mentioned in this article, this is the most popular.


Source link

- Advertisement -

Recent Articles

Related Stories