PersonaLLM
Scaffolding
Despite the many use cases for large language models (LLMs) in the design of chatbots in various industries and the research showing the importance of personalizing chatbots to cater to different personality traits, little work has been done to evaluate whether the behaviors of personalized LLMs can reflect certain personality traits accurately and consistently. We consider studying the behavior of LLM-based simulated agents which refer to as LLM personas and present a case study with GPT-3.5 (text-davinci-003) to investigate whether LLMs can generate content with consistent, personalized traits when assigned Big Five personality types and gender roles. We created 320 LLM personas (5 females and 5 males for each of the 32 Big Five personality types) and prompted them to complete the classic 44-item Big Five Inventory (BFI) and then write an 800-word story about their childhood. Results showed that LLM personas' self-reported BFI scores are consistent with their assigned personality types, with large effect sizes found on all five traits. Moreover, significant correlations were found between assigned personality types and some Linguistic Inquiry and Word Count (LIWC) psycholinguistic features of their writings. For instance, extroversion is associated with pro-social and active words, and neuroticism is associated with words related to negative emotions and mental health. Besides, we only found significant differences in using technological and cultural words in writing between LLM-generated female and male personas. This work provides a first step for further research on personalized LLMs and their applications in Human-AI conversation.
Home > Pilots & Programs > PersonaLLM
More Scaffolding
Anthology
An AI interface that turns raw conversation audio into interactive maps
Research
Dialogue to Decision
An LLM-Powered Framework for Analyzing Collective Idea Evolution and Voting Dynamics in Deliberative Assemblies
Research
Voice to Vision
Supporting decision makers in effective and efficient constituency-informed, AI-supported decision-making. Communicating how constituen...
Research
DisCourse
A curated social experience that transforms dinner between strangers into an opportunity to reimagine how we listen, speak, and share
Research
FRONTLINE Social Dialogue Network
Archived
Over 40 young people participated in a social listening experiment exploring how personal experiences build bridges to better understanding current events and creating mo
Pilots & Programs
realtalk@Boston
A new civic infrastructure in Boston grounded in dialogue as a way to building “civic muscle” of democracy
Pilots & Programs
