This model analyzes text to find the sentiment expressed in the text. It takes as input a sentence of text. It produces sentiment scores for the polarity, i.e. how positive or negative the sentence is, and the strength of the emotions in the sentence. This model can be applied in many situations where it is important to understand the emotions of someone writing text. It can be used to determine whether customers, constituents, suppliers, or others are pleased or unhappy, so that an organization can respond appropriately.
See the model in action with a Modzy MLOps platform demo or start a trial
96% F1 Score – F1 Score is the harmonic mean of the precision and recall, with best value of 1. It measures the balance between the two metrics.
88% Ground Truth Correlation – The correlation of computed raw sentiment intensity rating to the mean sentiment rating of human individuals.
99% Precision – A higher precision score indicates that the majority of labels predicted by the model for different classes are accurate.
94% Recall – A higher recall score indicates that the model finds and predicts correct labels for the majority of the classes it is supposed to find.
This model performs well on social media, where the correlation to ground truth (judgement of human reviewers) was 0.88, with precision of 0.99, recall of 0.94, and F1 score of 0.96. For the other types of data, these metrics were lower, but higher than for other lexicon-based tools.
The model does not use training data, but is constructed from a generalizable, valence-based, human-curated sentiment lexicon. The lexicon is sensitive to both the polarity and the intensity of sentiments expressed in social media but applicable to other text sources. The rule base captures conventional uses of grammatical and syntactical aspects of text for assessing sentiment intensity.
The lexicon used in the model was built by examining previous lexicons, and adding lexical features including emoticons, sentiment-related acronyms (e.g., LOL), and slang (e.g., ‘meh’). This process resulted in over 9,000 lexical feature candidates. A wisdom-of-crowds approach was used to acquire a valid point estimate for the sentiment valence (intensity) of each context-free candidate feature, resulting in about 7,500 final features. The lexical features were combined with heuristics based on syntax, capitalization and punctuation, each of which directly indicates sentiment intensity.
The VADER model was evaluated by measuring a) the correlation of computed raw sentiment intensity rating to gold standard ground truth, i.e., the mean sentiment rating from 20 pre-screened and appropriately trained human raters, as well as b) the multiclass (positive, negative, neutral) classification metrics of precision, recall, and F1 score. The validation set included social media (4,000 tweets), 10,605 movie reviews, 3,078 product reviews, and 5,190 opinion news articles.
The input(s) to this model must adhere to the following specifications:
This model will output the following:
See how quickly you can deploy and run models, connect to pipelines, autoscale resources, and integrate into workflows with Modzy MLOps platform
d o n o t fill t h i s . f i e l d d o n o t fill t h i s . f i e l d
