Ziwei (Sara) Gong

I am a Ph.D. candidate in Computer Science at Columbia University under the supervision of Professor Julia Hirschberg, with expected graduation in July 2027.

My research focuses on multimodal learning at the intersection of NLP, speech processing, and psychology. My thesis is on modeling human affect and psychological dynamics in interaction.

Email  /  CV  /  Bio  /  Scholar  /  Twitter  /  Github

profile photo

Research

* denotes equal contribution.

Empath: Tracing Multi-Level Emotion Dynamics in Crisis Counseling Dialogues
Ziwei Gong*, Yuchen Huang*, Nicholas Deas, Wen Liang, Shana Kleiner, Henry Willis, Melanie Subbiah, Kathleen McKeown, Julia Hirschberg
Under review, October 2026
pdf  /  arXiv
MEDIAT: Multi-Stakeholder, Emotion-Aware Dialogue Management
Ziwei Gong, Yifan Song, Jiateng Liu, Uttaran Bhattacharya, Saayan Mitra, Julia Hirschberg, Victor Bursztyn
Under review, October 2026
Right Destination, Wrong Journey: Synthetic Counseling Dialogues Miss Human Interactional Dynamics
Ziwei Gong*, Yuchen Huang*, Nicholas Deas, Melanie Subbiah, Julia Hirschberg
Under review, September 2026
E²Sim: Long-Horizon Simulation of Everyday Human Behavior with Evolving States
Ziwei Gong*, Xiangyang Yin*, Thai Quoc On, Lin Ai, Julia Hirschberg, Haotong Yang
Under review, September 2026
Huntington Disease Automatic Speech Recognition with Biomarker Supervision
Charles L Wang, Cady Chen, Ziwei Gong, Julia Hirschberg
Under review, March 2026
pdf  /  arXiv
A Survey on Mental Health Datasets and Resources
Ziwei Gong, Chengtao Dai, Bolei Ma, Melody Ma, Manasi Sharma, Thai Quoc On, Medha Morparia, Benjamin Enos, Yuwen Yu, Philip Resnik, Sarah Ita Levitan, Julia Hirschberg
Under review, January 2026
pdf

CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonese
Dazhong Chen, Yi-Cheng Lin, Yuchen Huang, Ziwei Gong, Di Jiang, Zeying Xie, Yi R. Fung
Under review, January 2026
pdf  /  arXiv
Factuality on Demand: Controlling the Factual-Informative Trade-off in Text Generation
Ziwei Gong*, Yanda Chen*, Julia Hirschberg, Chen Zhao, He He, Zhou Yu, Kathleen McKeown
arXiv preprint, 2025
pdf  /  arXiv

YUE-PUB-Speech: A Speech-based Pragmatic Understanding Benchmark for Cantonese
Yajie Wen*, Ziwei Gong*, Chengyan Wu*, Xiyun Gong, Yun Xue, Julia Hirschberg, Bolei Ma
Interspeech 2026, Sydney, Australia
paper  /  pdf
YUE-PUB: A Speech-based Pragmatic Understanding Benchmark for Cantonese
Chengyan Wu*, Ziwei Gong*, Jinzi Chen*, Yajie Wen, Mengmeng Su, Yun Xue, Bolei Ma
AACL-IJCNLP 2026, Hengqin, China
SURE: Synergistic Uncertainty-Aware Reasoning for Multimodal Emotion Recognition in Conversations
Yiqiang Cai*, Chengyan Wu*, Bolei Ma, Bo Chen, Yun Xue, Julia Hirschberg, Ziwei Gong
ICASSP 2026, Barcelona, Spain
pdf  /  arXiv

The Mind in the Machine: A Survey of Incorporating Psychological Theories in LLMs
Zizhou Liu*, Ziwei Gong*, Lin Ai*, Zheng Hui*, Run Chen*, Colin Wayne Leach, Michelle R. Greene, Julia Hirschberg
EACL 2026, Rabat, Morocco
paper  /  pdf  /  arXiv

Detecting Mental Manipulation in Speech via Synthetic Multi-Speaker Dialogue
Run Chen, Wen Liang, Ziwei Gong, Lin Ai, Julia Hirschberg
IWSDS 2026, Trenton, Italy
paper  /  pdf  /  arXiv

SMARTMiner: Extracting and Evaluating SMART Goals from Low-Resource Health Coaching Notes
Iva Bojic, Qi Chwen Ong, Stephanie Hilary Xinyi Ma, Lin Ai, Zheng Liu, Ziwei Gong, Julia Hirschberg, Andy Hau Yan Ho, Andy W. H. Khong
Findings of EMNLP 2025, Suzhou, China
paper  /  pdf

Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospects
Chengyan Wu, Yiqiang Cai, Yang Liu, Pengxu Zhu, Yun Xue, Ziwei Gong, Julia Hirschberg, Bolei Ma
Findings of EMNLP 2025, Suzhou, China
paper  /  pdf  /  arXiv

Learning More with Less: Self-Supervised Approaches for Low-Resource Speech Emotion Recognition
Ziwei Gong, Pengyuan Shi, Kaan Donbekci, Lin Ai, Run Chen, David Sasu, Zehui Wu, Julia Hirschberg
Interspeech 2025, Rotterdam, The Netherlands
paper  /  pdf  /  arXiv

CREAM: Comparison-Based Reference-Free ELO-Ranked Automatic Evaluation for Meeting Summarization
Ziwei Gong, Lin Ai, Harshsaiprasad Deshpande, Alexander Johnson, Emmy Phung, Zehui Wu, Ahmad Emami, Julia Hirschberg
Interspeech 2025, Rotterdam, The Netherlands
paper  /  pdf  /  arXiv

In this paper we propose an automatic, comparison-based evaluator for long-context summarization.

Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges
Bolei Ma, Yuting Li, Wei Zhou, Ziwei Gong, Yang Janet Liu, Katja Jasinskaja, Annemarie Friedrich, Julia Hirschberg, Frauke Kreuter, Barbara Plank
ACL 2025, Vienna, Austria
paper  /  pdf  /  arXiv

Akan Cinematic Emotions (ACE): A Multimodal Multi-party Dataset for Emotion Recognition in Movie Dialogues
David Sasu, Zehui Wu, Ziwei Gong, Run Chen, Pengyuan Shi, Lin Ai, Julia Hirschberg, Natalie Schluter
Findings of ACL 2025, Vienna, Austria
paper  /  pdf  /  arXiv

Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuances
Zehui Wu, Ziwei Gong, Lin Ai, Pengyuan Shi, Kaan Donbekci, Julia Hirschberg
Findings of NAACL 2025, Albuquerque, New Mexico
paper  /  pdf  /  arXiv

NovAScore: A New Automated Metric for Evaluating Document Level Novelty
Lin Ai, Ziwei Gong, Harshsaiprasad Deshpande, Alexander Johnson, Emmy Phung, Ahmad Emami, Julia Hirschberg
COLING 2025, Abu Dhabi, UAE
paper  /  pdf  /  arXiv

A Survey on Open Information Extraction from Rule-based Model to Large Language Model
Pai Liu*, Wenyang Gao*, Wenjie Dong*, Lin Ai*, Ziwei Gong*, Songfang Huang, Zongsheng Li, Ehsan Hoque, Julia Hirschberg, Yue Zhang
Findings of EMNLP 2024, Miami, FL
paper  /  pdf  /  arXiv

Multimodal Multi-Loss Fusion Network for Sentiment Analysis
Zehui Wu*, Ziwei Gong*, Jaywon Koo, Julia Hirschberg
NAACL 2024, Mexico City, Mexico
paper  /  pdf  /  arXiv

A Mapping on Current Classifying Categories of Emotions used in Multimodal Models for Emotion Recognition
Ziwei Gong, Xinyi Hu, Muyin Yao, Xiaoning Zhu, Julia Hirschberg
18th Linguistic Annotation Workshop, EACL 2024, Malta
paper  /  pdf

Eliciting Rich Positive Emotions in Dialogue Generation
Ziwei Gong, Qingkai Min, Yue Zhang
First Workshop on Social Influence in Conversations (SICon), ACL 2023, Toronto, Canada (Oral Presentation)
paper  /  pdf

Exploring New Methods for Identifying False Information and the Intent Behind It on Social Media: COVID-19 Tweets
Lin Ai, Run Chen, Ziwei Gong, Julia Guo, Shayan Hooshmand, Zixiaofan Yang, Julia Hirschberg
Sixth International Workshop on Social Sensing (SocialSens 2021), ICWSM, Atlanta
pdf

Teaching & Services

Instructor, COMS 4995 Semantic Representations for NLP, Columbia University, Summer 2025. Sole instructor for a 3-credit course (14 students); developed lectures, assignments, and final projects.

Mentor, Mental Health × AI Hackathon, Teachers College, Columbia University, Spring 2025. Mentored 3 teams on research design, NLP methods, and evaluation.

Teaching Assistant, COMS 6998 Advanced Topics in Spoken Language Processing, Columbia University, Spring 2023 and Spring 2024.

Mentor, Women In Science at Columbia (WISC) undergraduate mentoring program, Spring 2023.

Empirical Reasoning Fellow, ERC, Barnard College, Fall 2019 and Spring 2020. Supported quantitative-methods office hours in Excel, R, Python, and GIS.

Teaching Assistant, PSYC BC1101 Statistics, Barnard College, Fall 2018.

Professional Experience

Adobe Research, Research Scientist Intern, May–August 2026.
Developed a training-free dialogue control framework for dynamically routing stakeholder policies.

JPMorgan Chase & Co, AI & Data Science Summer Associate Program, June–August 2024.
Proposed a comparison-based evaluator for long-context meeting summarization without requiring references.

Amazon Web Services, Full-time SDE, Deep Learning at AWS, February–August 2022.
Designed and deployed Amazon SageMaker Debugger rules to optimize ML training and identify bottlenecks.

Amazon Web Services, SDE I Intern, Deep Learning at AWS, June–September 2021.
Built and optimized a distributed YOLOv3 object detection model for SageMaker, achieving 40% training speed optimization.

Grants & Awards

NSF ARNI Research Support (Doctoral Researcher), 2025–2026.
Towards Safe, Robust, Interpretable Dialogue Agents for Democratized Medical Care.

PhD Service Award, Columbia University, 2026.

Citadel PhD Travel Fund, 2025.

CRA-WP Grad Cohort for Women Travel Fund, 2024.

Beyond Barnard GHC Award, 2019 and 2020.

Invited Talks

Modeling Human Affect and Psychological Dynamics in Interaction, Advanced Speech Processing Course, Columbia University, November 2026 (scheduled).

Exploring Cultural Differences in LLMs for Mental Health, CSC Workshop, Barnard College, September 2026.

NLP+Mental Health, Advanced Speech Processing Course, Columbia University, April 2026.

Decoding Human Emotions, NLP Seminar, LMU Munich, April 2026; NLP Seminar, Columbia University, March 2025.

Semantic Representations, COMS W4705 Natural Language Processing, Columbia University, 2025 and 2026.

Emotion Detection, Advanced Speech Processing Course, Columbia University, 2023–2026.

Presentations

Semi-Occluded Vocal Tract Exercises on YouTube: Content Analysis and Sentiment Analysis
Pinhong Wu, Ziwei Gong, Yuchen Huang, Julia Hirschberg, Keith Chadwick.
147th Annual Meeting of the American Laryngological Association (ALA), Phoenix, Arizona, 2026.
poster

Research Mentoring

Zehui Wu (MS, Columbia) → 2 NAACL papers and 1 Interspeech paper; InstaLILY AI.

Jaywon Koo (MS, Columbia) → NAACL paper; Rice University CS Ph.D.

Pengyuan Shi (MS, Columbia) → Interspeech, NAACL, and ACL papers; Uber.

Kaan Donbekci (MS, Columbia) → Interspeech and NAACL papers.

Yuchen Huang (BA, Barnard) → 2 papers under review and ALA 2026 poster; UCLA MS.

Wen Liang (MS, Columbia) → IWSDS paper; Redhat.

Chengtao Dai (MS, Columbia) → paper under review.

Melody Ma (MS, Columbia) → paper under review; Meta Research.

Manasi Sharma (Palo Alto High) → paper under review; Columbia SEAS undergraduate.

David Smythe (MS, Columbia) → Datawise.

Richard Kim (MS, Columbia) → Headline.

Eleanor Lin (BS, Columbia) → University of Michigan CS Ph.D.

Thai Quoc On (MS, Columbia) → paper under review.

Medha Morparia (BA, Barnard) → paper under review.

Benjamin Enos (BA, Boston University) → paper under review.

Cady Chen (BA, Columbia) → paper under review; Stanford University RA.

Charles Wang (BS, Columbia) → paper under review.

Service

Organizing: The User, Plural: Heterogeneity and Interaction in User Simulation, workshop at COLING 2027, Macau, China, May 2027 (upcoming); BoF on User Simulation, ACL 2026, San Diego, CA, July 2026.

Reviewing: ACL Rolling Review (ARR), 2024–2026 (ACL, EMNLP, NAACL, EACL, AACL-IJCNLP); ICLR 2026; Linguistic Annotation Workshop (LAW) 2025; Social Influence in Conversations (SICon) 2025; Transactions on Affective Computing.

Department and community: Pre-application review for underrepresented candidates, Columbia University, 2023; CS Panel for Prospective Students, Barnard College, 2025 and 2026.

Member, Psi Chi, International Honor Society in Psychology, since August 2020.

News

May 2027: Co-organizing The User, Plural: Heterogeneity and Interaction in User Simulation at COLING 2027 (upcoming).

November 2026: Scheduled invited talk, Modeling Human Affect and Psychological Dynamics in Interaction, Columbia University.

Summer 2026: Research Scientist Intern at Adobe Research.


Website design from Jon Barron.