Building trustworthy generative AI 构建可信的生成式 AI
I am an Assistant Professor at the College of Artificial Intelligence, Shenzhen University, where I lead MUSE Lab (Multimodal Understanding and Synthesis for Explainability). I received my Ph.D. in Computer Science from Fudan University in 2025, advised by Prof. Junping Zhang, and my B.S. in Physics from Fudan University in 2019. I am a member of the Technical Committee on Hybrid Intelligence and the Science Popularization Committee of the Chinese Association of Automation (CAA).
我是深圳大学人工智能学院助理教授、硕士生导师，负责 MUSE Lab（Multimodal Understanding and Synthesis for Explainability Lab）。2025 年博士毕业于复旦大学计算机科学技术专业，师从张军平教授；2019 年本科毕业于复旦大学物理学专业。现为中国自动化学会混合智能专委会、普及工作专委会会员。
My research aims to make generative AI controllable, interpretable, and trustworthy. I study why text-to-image diffusion models succeed or fail on specific concepts and how to edit images and videos precisely and consistently; how large language models and agents stay reliable — free of hallucination, bias, and state collapse — over long-horizon interaction; and how generative agents can perceive and express human emotion. I am also interested in bringing these models to real-world applications such as emotional companionship, virtual try-on, and digital humans.
我的研究目标是让生成式 AI 更加可控、可解释、可信：研究文生图扩散模型为何在特定概念上成功或失败，以及如何实现精确、一致的图像与视频编辑；研究大语言模型与智能体如何在长程交互中保持可靠，避免幻觉、偏见与状态崩塌；研究生成式智能体如何感知并表达人类情感。同时探索这些技术在情感陪伴、虚拟试衣、数字人等真实场景中的落地应用。
Welcome research or business collaborations!
欢迎科研与商业合作！
Why do diffusion models fail on specific concepts, and how can we edit images and videos precisely while keeping semantics consistent and identities intact?
扩散模型为何会在特定概念上失效？如何在保持语义一致与身份不变的前提下，实现精确的图像与视频编辑？
Keeping LLMs and agents reliable: constraint and state integrity in long-horizon dialogue and agent handoffs, reflective search for code generation, and AI-for-AI systems that write academic surveys.
让大模型与智能体更可靠：长程对话与多智能体交接中的约束与状态保持、面向代码生成的反思式搜索，以及自动撰写学术综述的 AI for AI 系统。
From facial expression and micro-expression analysis to emotional agents that understand and respond to people, extending toward embodied AI and world models.
从人脸表情与微表情分析，到能够理解并回应人的情感智能体，并进一步拓展到具身智能与世界模型。
Applications: emotional companionship, virtual try-on, end-to-end apparel automation, and digital humans.应用场景：情感陪伴、虚拟试衣、服装全流程自动化、数字人建模。
(* equal contribution · † corresponding author)（* 共同第一作者 · † 通讯作者）
Text-to-image diffusion models usually follow prompts well, yet we found an overlooked failure: a concept such as “ink sketch” works for most objects but fails again and again on a few — Flux draws grapes as an ink sketch but keeps painting a cake in full color. These failures persist across random seeds, revealing systematic blind spots in how the model has learned the concept rather than sampling noise. We train sparse autoencoders at every denoising step, compare successful and failed generations in this sparse space where concepts become separable, locate the missing or weakened concept dimensions, and repair them at inference time by nudging features toward a concept prototype built from reliable samples, with no retraining. Across multiple diffusion backbones this markedly improves concept consistency, text fidelity and repair success, and the earlier the intervention, the stronger the correction.
文生图模型通常能很好地按提示词作画，但我们发现了一种被忽视的失败：同一个概念（如“水墨素描”）用在大多数物体上都没问题，却在少数物体上反复失败——Flux 能把“葡萄”画成水墨素描，“蛋糕”却总是彩色的。这类失败换随机种子也照样出现，说明模型对概念的掌握存在系统性盲区，而不是偶然的采样噪声。我们在去噪的每一步训练稀疏自编码器，在这个概念更易区分的稀疏空间里对比成功与失败的生成，找出缺失或减弱的概念维度，再在推理时把特征向由可靠样本构建的概念原型轻量插值，无需重新训练即可修复。在多种主流扩散模型上，概念一致性、文本忠实度和修复成功率都显著提升，而且越早干预、纠正效果越好。
Assistants must keep obeying user-set constraints — formats, personas, safety boundaries — over many turns, but evaluations mostly test instruction following in a single turn or a short exchange. We identify a subtle but dangerous failure, State Integrity Collapse: asked about it, the model can still restate the constraint, yet its next answers stop following it — unlike ordinary forgetting, this is not fixed by longer context or memory retrieval. We build StateTrack-Bench, which plants one persistent constraint (numerical, format, persona, factual-memory or logical) in a conversation, checks compliance at increasing dialogue depths, and uses a forked recall audit to tell forgetting apart from remembering without complying. Across seven models, 88.6% of failures are such behavioral drift rather than forgetting, and our Structured State Anchoring recovers up to 24.4 points.
对话助手需要在很多轮对话里一直遵守用户设定的约束（格式、角色、安全边界等），但现有评测大多只看单轮或短对话里能否遵守指令。我们发现一种隐蔽而危险的失败：问起来模型还能复述约束，接下来的回答却不再执行——我们称之为“状态完整性崩塌”，它不同于普通的遗忘，单靠更长的上下文或记忆检索解决不了。我们构建了 StateTrack-Bench：在对话中设定一条持续约束（数值、格式、角色、事实记忆、逻辑规则五类），在不同对话深度检验模型是否仍然遵守，并用“分叉回忆审计”区分“记不住”和“记得但不做”。在七个主流模型上，88.6% 的失败属于“记得但不做”的行为漂移而非遗忘；我们提出的结构化状态锚定最多可挽回 24.4 个百分点。
Code written by large language models often looks right yet fails on edge cases and hidden tests, and common self-repair keeps patching a single attempt, so an early wrong algorithmic choice is hard to escape. Existing tree-search methods rank candidates with numeric rewards but do not share the textual diagnoses of failures across branches, and without tests they may mistake missing evidence for success. ReMCTS organizes candidate programs as a search tree, runs them in a sandbox for feedback, keeps each branch's debugging context, stores failure diagnoses in a shared memory that other branches retrieve, and tells failed checks apart from missing evidence. On HumanEval and MBPP it beats direct generation in 8 of 10 model–dataset pairs, and ablations show where the gains come from and where they stop.
大模型写的代码常常“看起来对”，却在边界条件或隐藏测试上出错；常见的自我修复只沿着一次尝试反复打补丁，一旦最初的算法思路错了就很难跳出来。已有的树搜索方法又主要依赖数值奖励，失败时的文字诊断没法分享给其他分支，在没有测试用例时还容易把“缺少证据”误当成“通过”。ReMCTS 把多个候选程序组织成一棵搜索树，在沙箱里执行获取反馈，保留每条分支的调试上下文，把失败经验存入全局记忆供其他分支检索复用，并区分“检查未通过”和“证据不足”。在 HumanEval 和 MBPP 上，ReMCTS 在 10 组模型—数据集组合中有 8 组优于直接生成，消融实验也厘清了收益的来源与边界。
Publications are growing so fast that writing a good survey is ever more costly, and current LLM survey generators mostly retrieve papers as a flat pile of independent documents. A good survey explains how a field evolved — which work extends which, which methods compete, which benchmarks mark turning points — structure that similarity-based retrieval cannot see, so generated surveys read like a collage of summaries. SurveyGraph builds a Methods-Entity Knowledge Graph that records follow-on, competition and shared-resource relations between methods, and uses it as the backbone for core-paper discovery, graph-aware retrieval, outline generation and evolution-aware writing. Graph-aware retrieval nearly doubles recall over flat lexical baselines (p = 0.004), and citation audits find that 80.6% of generated sentences are fully or partly supported by the papers they cite.
论文越来越多，写一篇好的综述越来越费时费力，而现有的大模型自动综述系统大多把文献当作一堆彼此独立的文档来检索。好的综述要讲清楚一个领域如何演化——哪项工作延续了谁、哪些方法相互竞争、哪些基准标志着转折，只按相似度检索看不到这些结构，生成的综述容易变成零散摘要的拼贴。SurveyGraph 构建“方法—实体知识图谱”（MEKG），显式记录方法之间的延续、竞争和共享资源等关系，并以它为主干完成核心论文发现、图感知检索、大纲生成和按演化脉络组织的写作。图感知检索的召回率接近平面检索基线的两倍（p=0.004），引用核查显示 80.6% 的生成句子能在所引文献中找到完全或部分支撑。
When a text-to-image diffusion model is used for image-to-image translation — turning a photo into a painting, say — everything hinges on an initial noise that keeps the original image's information, and the common choice, SDEdit, simply adds random noise to the image. We find that this causes “semantic discrepancy”: the model misreads relations in the image and invents content that was never there, because the noise breaks the image's semantic integrity and makes U-Net upsampling link unrelated regions. We propose a training-free, plug-and-play fix for Stable Diffusion that exploits the determinism of DDIM inversion: a single inversion step of the reference image supplies accurate features and correlations, which replace the faulty ones in generation. Across translation tasks on COCO, ImageNet and ImageNet-R, it reduces semantic discrepancy and improves fidelity, outperforming PnP and similar methods with fewer priors while running 11.2× faster.
用文生图扩散模型做图像转换（比如把照片变成油画）时，关键是找到既保留原图信息、又便于转换的初始噪声，常用的 SDEdit 做法是直接给原图加随机噪声。我们发现这会引起“语义偏差”：模型误解原图中物体之间的关系，凭空生成原图里没有的内容；剖析 Stable Diffusion 的网络后，我们定位到原因是加噪破坏了图像的语义完整性，使 U-Net 上采样时在不相关的区域之间建立了错误关联。我们提出一种免训练、即插即用的方法，利用 DDIM 反演的确定性，只需对参考图做一步反演，就能得到正确的特征和关联，用来替换生成过程中的错误部分。在 COCO、ImageNet 和 ImageNet-R 的多种图像转换任务上，该方法有效缓解了语义偏差、提升了保真度，用更少的先验超过 PnP 等方法，速度快 11.2 倍。
