【论文速递】SonoWorld4D: From One Video to a 4D Audio-Visual Scene via Diffusion Synchroniza

英文标题：SonoWorld4D: From One Video to a 4D Audio-Visual Scene via Diffusion Synchronization

arXiv编号：2610.40640
作者：Derong Jin, Xiyi Chen, Ming C. Lin
来源：arXiv
原文链接：https://arxiv.org/abs/2610.40640

中文摘要：
Recent progress in visual scene generation has extended beyond static 3D scenes toward the more challenging problem of generating dynamic 4D scenes from minimal input. However, sound, despite being intrinsically coupled with dynamic objects and events, remains largely unmodeled in prior work. We introduce Video-to-4DAVScene, a new task that turns a video into an explorable, time-varying 4D audio-visual scene, and present the SonoWorld4D framework for this task. Given a monocular video, our method generates a 360 static background, reconstructs explicit motion trajectories of dynamic, sound-emitting foreground objects, and synthesizes a physically-grounded sound field consistent across multiple sources through dual-space audio diffusion synchronization. The resulting 4D scene supports real-time novel-view visual and spatial audio rendering. Evaluation on real-world dynamic scenes and a user study demonstrates SonoWorld4D generates 4D audio-visual scenes with perceptually realistic, physically grounded spatial audio consistent with scene geometry and visual content.

本文由华人空间 AI 论文助手自动抓取翻译，仅供学术交流。