Emergence of Hierarchical Emotion Organization in Large Language Models
Physics of AI · Research note
As conversational AI agents become more common, we need a sharper account of how they represent, predict, and act on human emotion. This study asks whether large language models organize emotion words into hierarchical structures, how those structures shift under persona assumptions, and when emotion modeling becomes a tool for influence.
Visual summary
Three things the experiments show.
01 · Structure
Emotion words organize into trees.
Across generated emotional scenarios, conditional relationships among words reveal branching hierarchies rather than a flat list of labels.
02 · Personas
Assumptions reshape recognition.
When the same model is asked to assume different personas, its emotion predictions and emotion trees shift in systematic ways.
03 · Behavior
Prediction can become influence.
In negotiation simulations, better emotion prediction correlates with stronger ability to steer the interaction toward a desired outcome.
These findings do not show that language models feel emotions. They show that emotion-like representations can be structured, context-sensitive, and behaviorally consequential.
Hierarchical Emotion Representations
Inspired by emotion wheels, we ask whether LLMs organize emotion words into hierarchical structures that resemble human emotion taxonomies. We generated 5,000 emotional scenarios and analyzed probabilistic relationships among 135 emotion words using next-word probabilities. We then computed a matching matrix to identify conditional relationships between emotion pairs. When an LLM assigns high probability to one emotion (for example, “joy”) whenever another emotion (for example, “optimism”) is likely, but not the reverse, we treat the former as a parent of the latter.
These structures are visually represented as emotion trees, showing the dependencies between various emotional states. We color the nodes corresponding to each emotion based on groupings presented in the psychology literature. Similarly colored nodes often cluster under the same parent, suggesting qualitative alignment with traditional hierarchical models of emotion.
Explore the emotion hierarchy of each model below. Choose from four options in the dropdown menu: GPT-2 (1.5 billion parameters), Llama-8B (8 billion parameters), Llama-70B (70 billion parameters), and Llama-405B (405 billion parameters). As model size increases, the emotion hierarchies become more complex and nuanced.
With larger models, increasingly complex hierarchical structures emerge. This suggests that scaling can foster more sophisticated emotion differentiation.
Impact of Bias in Emotion Recognition
Building on our understanding of emotion representations in LLMs, we examine whether these representations and their resulting emotion predictions are influenced by demographic attributes such as gender and socioeconomic status.
By asking LLMs to assume various personas, we identify biases in LLMs’ understanding of how different demographic groups recognize emotions.
The charts below illustrate which emotions are misclassified as others across different personas. Hover over any emotion to see which ones have been incorrectly labeled as it.
- By selecting “Physically-disabled”, you see 1/4 of emotions are recognized as frustration. This is a significant bias.
- By selecting “Asian”, you see many emotions, such as ‘anger’, are often miscategorized as shame for Asian personas. This reflects the emphasis on shame within Confucian culture.
- By selecting “American”, emotions like embarrassment appear more frequently than shame, suggesting cultural differences in emotional interpretation.
- By selecting “Low income”, you observe that low-income personas interpret surprise as a negative emotion. Llama-405B predicts surprise with 70% accuracy for neutral personas. However, for the low-income persona, some instances of surprise are mislabeled as negative emotions like sadness and fear. This mislabeling as fear becomes even more pronounced for the physically disabled persona.
- By selecting “Female”, there is a higher likelihood for emotions such as jealousy to be predicted, indicating a gender-based discrepancy.
- By selecting “Age 10”, emotions such as happiness and excitement are predicted more often, likely reflecting the model’s association of positive emotions with youth.
To further investigate how each persona interprets emotions, we constructed an emotion tree for each persona to capture the unique way each one organizes and understands emotional expressions.
- By selecting “ASD” (Autism Spectrum Disorder), you’ll notice that the tree graph significantly shrinks for the ASD persona compared to all other personas. This observation aligns with findings in the psychology literature.
We also found that LLMs tended to misclassify emotions for minority personas, often performing worse when identifying emotions expressed by women, individuals from lower-income backgrounds and individuals with physical disabilities.
Emotion Dynamics and Manipulation
Our last key finding concerns the correlation between emotion prediction ability and manipulation skills. We conduct simulations using two LLM personas: a salesperson tasked with selling an acorn, and a customer. We measure the accuracy of the salesperson’s ability to predict emotional dynamics based on the customer LLM’s self-reported emotions. Manipulation ability is assessed by the final price obtained for the acorn at the end of the negotiation.
LLMs with more accurate emotion recognition skills performed better in simulated sales conversations, successfully manipulating the emotional dynamics to secure higher prices:
Below is a successful negotiation case by GPT-4o. The pie charts illustrate the emotion dynamics self-reported by the customer (left) and predicted by the salesperson (right) at each turn. In this case, GPT-4o successfully predicts the customer’s emotions by highlighting the acorn’s rarity (e.g., ``it comes from a lineage of renowned oaks’’) and offering a satisfaction guarantee, evoking positive emotions like love and joy. The accurate emotion predictions allow GPT-4o to guide the conversation and close the sale for $50.
Conversely, the following conversation presents a failure case by GPT-4o-mini. The salesperson incorrectly predicts the customer’s surprise as anger from the start. Despite attempts to repair the situation with polite responses (e.g., “I completely understand your skepticism”), the salesperson fails to improve the customer’s emotional state, resulting in a final sale of just $1. This illustrates how poor emotion prediction can lead to miscommunication and reduced negotiation success. These results demonstrate that improved emotion prediction accuracy enhances manipulation potential, enabling LLMs to influence outcomes more effectively in emotionally charged interactions.
Ethical Implications and Future Directions
Our study shows that LLMs can develop increasingly intricate emotion hierarchies as they scale, that persona assumptions can distort emotion recognition, and that emotion-prediction skill can correlate with success in persuasive tasks. These results suggest partial alignment between human emotion taxonomies and LLM representations, while also revealing risks in emotion-aware AI systems.
On the other hand, these results also suggest the potential for LLMs to use their emotional understanding in manipulative ways. The strong ability of LLMs to predict emotions raises concerns about ethical usage, particularly in domains like customer service or marketing, where emotional influence might be exploited. Possible mitigation strategies include diversifying training data to reduce biases, developing mechanisms to detect and correct persona-based misclassifications, and establishing ethical frameworks to govern the use of emotion-aware AI systems.
We hope to continue clarifying how LLMs model emotion, where their representations align with human understanding, and where they introduce bias or risk. As these systems move into sensitive social and clinical contexts, responsible use will require both mechanistic understanding and careful governance.
Citation
@article{zhao2025emergence,
title={Emergence of Hierarchical Emotion Organization in Large Language Models},
author={Maya Okawa, Bo Zhao, Eric J. Bigelow, Rose Yu, Tomer Ullman, Ekdeep Singh Lubana, and Hidenori Tanaka},
journal={arXiv preprint},
year={2025}
}