【论文速递】Repurposing Video Diffusion Transformers for Cross-View Temporal Object Correspo

英文标题：Repurposing Video Diffusion Transformers for Cross-View Temporal Object Correspondence

arXiv编号：2610.37633
作者：Siyoon Jin, Dahyun Chung, Honggyu An
来源：arXiv
原文链接：https://arxiv.org/abs/2610.37633

中文摘要：
Cross-view object correspondence identifies the same object across views. However, existing methods often rely on frame-wise matching or sequential tracking, which can propagate correspondence errors under appearance changes or occlusions. Therefore, we introduce cross-view temporal object correspondence, which requires preserving object identity across both views and time from a query mask. To address this, we propose XTrack, a unified framework which repurposes a pretrained video DiT with semantic and spatiotemporal priors for RGB-to-mask latent transport, jointly modeling cross-view matching and temporal tracking. However, naive video DiTs do not guarantee accurate cross-view grounding and often attend to semantically similar yet incorrect objects across views. Thus, we introduce focal cross-view attention alignment, hard negative conditioning, and local context crop strategies to ensure reliable grounding. Finally, beyond these architectural enhancements, we also tackle the lack of dense annotation by introducing a pseudo-label curation pipeline that converts sparse Ego-Exo4D annotations into dense supervision to enable effective video-level training. Our results demonstrate the effectiveness of XTrack as a unified framework, ensuring robust temporal consistency and identity preservation.

本文由华人空间 AI 论文助手自动抓取翻译，仅供学术交流。