Home
Publications
Photo Gallery
EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning
Xuan Ju, Tianyu Wang, Yuqian Zhou, He Zhang, Qing Liu, Nanxuan Zhao, Zhifei Zhang, Yijun Li, Yuanhao Cai, Shaoteng Liu, Daniil Pakhomov, Zhe Lin, Soo Ye Kim, Qiang Xu
ICLR (Oral) · 2026 · Tech Lead
Soo Ye Kim: Corresponding author; Qiang Xu: Corresponding author
Research summary
A unified framework for image and video generation and editing that leverages self-attention for in-context learning and cross-modal knowledge transfer.
Abstract
Recent advances in foundation models highlight a clear trend toward unification and scaling, showing emergent capabilities across diverse domains. While image generation and editing have rapidly transitioned from task-specific to unified frameworks, video generation and editing remain fragmented due to architectural limitations and data scarcity. In this work, we introduce EditVerse, a unified framework for image and video generation and editing within a single model. By representing all modalities, i.e., text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning, natural cross-modal knowledge transfer, and flexible handling of inputs and outputs with arbitrary resolutions and durations. To address the lack of video editing training data, we design a scalable data pipeline that curates 232K video editing samples and combines them with large-scale image and video datasets for joint training. Furthermore, we present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions. Extensive experiments and user studies demonstrate that EditVerse achieves state-of-the-art performance, surpassing existing open-source and commercial models, while exhibiting emergent editing and generation abilities across modalities.
Key insight
Unifies image and video editing and generation into a single model via self-attention-based in-context learning, eliminating the need for task-specific models and enabling cross-modal knowledge transfer.
Technical connections & research relevance
Technical themes
- in-context learning
- multimodal generation and editing
- cross-modal knowledge transfer
- instruction-controlled visual variation
- embodied AI research direction
Method connections
EditVerse studies generalist multimodal learning through a shared sequence of text, image, and video tokens. Its in-context conditioning and cross-modal transfer enable diverse editing and generation tasks within one model.
Research relevance
For embodied AI, this suggests a way to generate instruction-controlled visual variations and investigate adaptation across observation domains, with robotics transfer requiring dedicated validation.
Evaluation scope
The paper evaluates image and video editing and generation. Its in-context learning results do not establish few-shot robot policy learning from demonstrations or a robotics sim2real benchmark.
Primary sources
arXiv version linked; publication venue as listed above
Paper & resources
BibTeX
@inproceedings{ju2026editverse,
title={EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning},
author={Ju, Xuan and Wang, Tianyu and Zhou, Yuqian and Zhang, He and Liu, Qing and Zhao, Nanxuan and Zhang, Zhifei and Li, Yijun and Cai, Yuanhao and Liu, Shaoteng and Pakhomov, Daniil and Lin, Zhe and Kim, Soo Ye and Xu, Qiang},
booktitle={International Conference on Learning Representations (ICLR)},
year={2026}
}