What Are Sentiment Scores and How Are They Calculated?

When conducting market research, it is critical to cast a wide net in order to capture an array of relevant content and data, be it company documents, expert calls, and broker research or alternative data sources—ensuring your research is full of multifaceted perspectives and voices. At the same time, attempting to manually glean insights from thousands of pages of text is a recipe for research blind spots and information fatigue.

In the age of digital transformation, arguably the largest development in research processes has been the adoption of artificial intelligence (AI). AI has the ability to parse through vast amounts of qualitative and quantitative data and surface the most relevant and crucial insights in seconds, removing guesswork and uncertainty almost entirely. It’s the secret sauce of every forward-thinking corporation or financial services firm that aims to maximize time, improve efficiency, and minimize risk. 

One of the most overlooked, yet deeply powerful, sources of insight is sentiment.

Sentiment refers to the tone and nuance beyond the semantic meaning of what an executive leader, news source, or customer is saying. This is particularly important when analyzing earnings transcripts, as inflection points in tone often serve as a proxy signal anticipating stock movement and shifting corporate strategy. 

The problem is that it can be tricky for humans to read between the lines and pick up on subtle shifts in tone and inflection, particularly when working with pages of data. By relying on an AI model that is trained to understand natural language with powerful natural language processing (NLP) capabilities, you can identify the most meaningful language indicators and track any changes in sentiment from QoQ. 

Sentiment analysis is a machine learning and NLP-based feature that sifts through documents, extrapolating tone and language nuance to give you the subtext within seconds. This feature is built into the AlphaSense platform, enabling you to quickly recognize the positive, negative, or neutral sentiment in the source you are analyzing, ensuring you have the right information to make better, more informed decisions.

What Are Sentiment Scores?

Sentiment scores are the measures of how positive or negative the tone of a particular transcript is. Their accuracy and reliability is strongly dependent on the caliber of the AI powering the sentiment analysis model. 

AlphaSense’s sentiment scoring is enabled by machine learning algorithms that assess the tone of a transcript on a spectrum of positive 100 to negative 100, with zero being neutral. It includes an overall score, as well as the delta (Δ) which shows the difference in total sentiment from the previous quarter’s earnings transcript of any one particular company.  

One of the biggest advantages of AlphaSense sentiment scores is the ability to rank documents by the biggest shift in sentiment from the previous quarter. Comparing scores can be useful for gathering quick industry insights or getting an overall macroeconomic picture. 

Within seconds, you can apply a sentiment lens to earnings call transcripts to learn:

• The companies with the largest sentiment score shift from previous quarters
• The most negative or positive documents, on a relative score from -100 to 100
• Highlights of phrase-level negative or positive sentiment underlying score shifts, honing in on the specific tonal text that our sentiment analysis model has identified with high confidence

How to Calculate Sentiment Scores

Calculating sentiment for a specific document first entails counting the raw inputs. That includes counting:

• Total negative statements in the document
• Total positive statements in the document
• Total statements in the document

The raw inputs drive the raw sentiment score for a document. Raw sentiment is the percent of the document that is positive minus the percent that is negative.

From there, the model calculates a normalized sentiment score, which is the raw score normalized across all companies such that the mean sentiment is 0 and the scores are stretched between -99 and 99. This score matches what you see in the AlphaSense platform.

Each sentence in a transcript gets a prediction of positive, negative or neutral along with a confidence value (i.e. 99% confidence this statement is positive). 

First, the model forms a contextual understanding of each sentence based on the surrounding context and what is important from within that sentence or paragraph. It then uses that information to make a sentence level prediction and output a confidence value for that prediction. The model also highlights important phrases and sentences that are driving its predictions, generating a summary for users.

For the document level score, we take the count of positive statements minus the count of negative statements divided by the total number of statements. We then normalize these scores by:

• The confidence of each prediction
• The mean and standard deviation of scores across all transcripts in the last two years such that the average score is 0. Some examples of what this means:
  • A score of 0 is average across all transcripts
  • A score of 40 (or -40) is in the top 20% (or bottom 20%) of all transcripts
  • A score of 99 (or -99) is in the top 2% (or bottom 2%) of all transcripts

AlphaSense sentiment scoring can provide an instant snapshot of how positive or negative the language in a transcript is, as well as how it has changed from the previous quarter.  

Sentiment Score Use Cases

Spot Potential Stock Movement Before It Happens

Shifts in executive commentary and sentiment QoQ may provide underlying context to specific topic areas and can directly influence a company’s stock price.

With AlphaSense’s sentiment trends, you can identify inflection points in sentiment that could indicate a potential red flag in stock performance. With highlighting of phrase-level sentiment, you can then quickly hone in on the most powerful statements to get the context you need. Sudden shifts in sentiment often directly parallel shifts in stock prices as well, so with this tool, you can predict stock movement before anyone else does.

Gain a Deeper Understanding of a Trend or Market Event

When monitoring a company or industry, it can sometimes be challenging to know if a certain development is beneficial or concerning, particularly when you are far from the source. With AlphaSense’s sentiment tool, you can search specific trends and see how the industry is speaking about them, as well as how the conversation has evolved over time. 

For example, several years ago, oilfield water management was identified as a key emerging risk to the sustainability of Permian Basin oil production. In addition to using AlphaSense’s advanced search technology to keep up with all the trends, sentiment analysis proved integral to understanding how the industry itself was discussing the water issue. 

Rather than expressing concern or hinting at future cutbacks, oil producers were actually speaking positively about the issue, their strategies, and cost control. This revealed an unexpected industry pattern—oilfield water is an industry that tends to find opportunity in technical challenges.

Yet the sentiment tool also revealed that the sentiment in the niche space of oilfield water was counter to the larger oil and gas industry’s sentiment on the issue. Looking at the earnings reports of the most recent US independent oil producers it was quickly apparent that the sentiment score was low and on a downward trajectory. This made it clear that US oil producers were dealing with negative sentiment from investors toward their sector, and this was reflected in their conference calls. 

The sentiment tool showed that while the overall industry was being negatively impacted by the oilfield water issue, the oilfield water niche itself was unexpectedly thriving.

Earnings Research

By accurately analyzing the tone and sentiment in financial documents, sentiment analysis empowers you to quickly understand the market landscape and outlook on specific companies. With AlphaSense’s sentiment analysis feature, you can apply a sentiment lens to earnings call transcripts within moments of them appearing on the AlphaSense platform and screen documents to see:

• Companies with the largest sentiment score shift from the previous quarter
• The most negative or positive documents, on a relative score from -100 to 100
• Highlighting of phrase-level positive or negative sentiment underlying score shifts, honing in on the specific tonal text that our model has identified with high confidence

With this tool, you can quickly uncover the subtle messaging and tonal cues that will ultimately allow you to draw conclusions and reveal relationships between key topics or themes that would otherwise be easily missed. This allows you to make better strategic decisions and gives you a competitive advantage.

Limitations of Sentiment Scores

Sentiment analysis accuracy can be variable, and there are certain potential issues that can arise when relying on AI-generated sentiment scores, particularly when it comes to customer data. 

One challenge is sarcasm detection. People frequently express negative sentiments in positive words, particularly in user-generated content such as tweets. For a machine, it can be extremely challenging to identify instances of sarcasm, particularly when it doesn’t have the context of the specific situation, topic, or environment. However, deep learning and machine learning models—like those found in the AlphaSense platform—have proven to be some of the most effective at sarcasm detection. 

Another challenge is word ambiguity, wherein a model cannot define polarity in advance because the polarity of certain words is dependent on the context (i.e. the same word can be positive in one context and negative in another). 

Lastly, a challenge for some models is understanding multipolarity, where a single phrase contains both positive and negative sentiments about a topic. For models that automatically score the entire sentence, there is a real risk of missing the mark. That’s why the most accurate sentiment analysis models will score each sentence by word or phrase, rather than by only considering the sentence as a whole.

The AlphaSense sentiment tool is designed to circumvent these issues and to be even more effective than a human at assessing and scoring financial data—without making false claims or over-generalizing.

How to Choose the Right Sentiment Analysis Tool

While there are many sentiment analysis tools available, it’s important to choose one that was trained on a reliable, validated dataset that will both enhance your research process and create trust in your analysis. 

When selecting a sentiment analysis model, ask the following questions:

What kind of data are you looking to analyze with sentiment analysis? 

If you are interested in measuring customer satisfaction, the data you need to analyze will come from social media platforms, customer feedback forms, and product reviews. In that case, you need a model that is trained on and will understand the typical consumer lexicon.

If you are interested in company or industry sentiment, your data will be mined from financial documents, news, and expert calls. Make sure the model is trained to understand the nuances of financial language. AlphaSense’s model is trained on 10+ years of human-curated financial documents, so it is particularly well-suited for analyzing earnings transcripts, conference calls, company presentations, expert calls, and other financial content.

How accurate is this model? 

As we’ve seen with recent innovations in generative AI, there is an inherent risk in relying on AI for any kind of text analysis or definitive truth-telling. However, confidence values can tell you how reliable a tool’s analysis is, so that you know what may require a second look. 

In AlphaSense’s sentiment analysis tool, each sentence gets a prediction of positive, negative, or neutral along with a confidence value (i.e. 99% confidence this statement is positive). Overall, our sentiment analysis model has been shown to be over 90% accurate in its predictions.

What metrics does the model use to assign a sentiment score to the content?

One of the key characteristics in a sentiment analysis model should be transparency. It should be clear why the tool is assigning a particular sentiment score to the piece of text. The AlphaSense sentiment tool highlights important phrases and sentences driving its predictions, color-coding positive and negative words accordingly. 

The final score for the document is calculated by subtracting the count of negative statements from the count of positive statements and dividing by the total number of statements. The model automatically generates a summary of all sentiment predictions for users, taking the guesswork out of the process. 

Take Your Research to the Next Level with AlphaSense

Sentiment analysis plays a vital role in understanding company outlook and nuances in language on earnings calls. Having a firm grasp on the implications behind earnings messaging will result in more confident decision-making and help you stay on top of market movements before your competitors.

Sentiment scoring is just one piece of AlphaSense’s advanced AI and automation toolset. When combined with Smart Synonyms™ search technology, real-time alerts, Smart Summaries, table tools, and a host of collaboration tools, AlphaSense allows you to get the most out of its vast content universe. 

Don’t miss our on-demand webcast on navigating market volatility, where we discuss how market leaders stay ahead of the curve and find growth opportunities in an uncertain macroeconomic environment. 

Start your free trial with AlphaSense today.