PersonaLLM
Scaffolding
Despite the many use cases for large language models (LLMs) in the design of chatbots in various industries and the research showing the importance of personalizing chatbots to cater to different personality traits, little work has been done to evaluate whether the behaviors of personalized LLMs can reflect certain personality traits accurately and consistently. We consider studying the behavior of LLM-based simulated agents which refer to as LLM personas and present a case study with GPT-3.5 (text-davinci-003) to investigate whether LLMs can generate content with consistent, personalized traits when assigned Big Five personality types and gender roles. We created 320 LLM personas (5 females and 5 males for each of the 32 Big Five personality types) and prompted them to complete the classic 44-item Big Five Inventory (BFI) and then write an 800-word story about their childhood. Results showed that LLM personas' self-reported BFI scores are consistent with their assigned personality types, with large effect sizes found on all five traits. Moreover, significant correlations were found between assigned personality types and some Linguistic Inquiry and Word Count (LIWC) psycholinguistic features of their writings. For instance, extroversion is associated with pro-social and active words, and neuroticism is associated with words related to negative emotions and mental health. Besides, we only found significant differences in using technological and cultural words in writing between LLM-generated female and male personas. This work provides a first step for further research on personalized LLMs and their applications in Human-AI conversation.
Home > Pilots & Programs > PersonaLLM
More Scaffolding
New Social Forms
Exploring new rituals, formats, and structures for coming together
Pilots & Programs
Storyline
An auditable AI framework for tracing competing narratives across podcasts, conversations, and news
Research
AI-Assisted Conversation Highlighting
Discovering semantically or emotionally salient moments in spoken discourse using LLMs
Research
realtalk@Boston
A new civic infrastructure in Boston grounded in dialogue as a way to building “civic muscle” of democracy
Pilots & Programs
Anthology
An AI interface that turns raw conversation audio into interactive maps
Research
Agora
A training ground to practice consensus-finding with real human perspectives
Research
