Exciting models have been developed in multimodal video understanding and generation, such as video LLM and video diffusion model. One emerging pathway to the ultimate intelligence is to create one single foundation model that can do both understanding and generation. In this work, we present Show-o, one single transformer that handles both multimodal understanding and generation. Unlike fully autoregressive models, Show-o is the first to unify autoregressive and discrete diffusion modeling, flexibly supporting a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation of any input/output format, all within one single transformer. Across various benchmarks, Show-o demonstrates comparable or superior performance, shedding light for building the next-generation foundation model. At the end, I will share our latest edition, Show-o2, which further integrates video understanding and generation into this single transformer.

Unified multimodal models aim to integrate visual understanding and generation within a single model, yet their visual representations often remain fragmented by task-specific encoders and inherited representation biases. In this talk, I present our exploration of unified visual representations, from TUNA to TUNA-2. We begin with TUNA, which unifies understanding and generation in a shared continuous visual space by cascading a reconstruction-oriented VAE with a semantic representation encoder. This design eliminates the representation mismatch of decoupled approaches. Building on this, we ask a more fundamental question: if a unified model is trained end-to-end, are pretrained vision encoders necessary at all? TUNA-2 progressively removes both the VAE and representation encoder, learning directly from raw pixels through simple patch embeddings. To address the increased difficulty and redundancy of pixel-space learning, we introduce masking-based feature learning to encourage more robust representations. At scale, this encoder-free design remains competitive in generation while yielding stronger fine-grained visual understanding. Together, these works suggest a path toward truly unified multimodal models: moving from unifying pretrained representations to learning unified representations natively from shared multimodal objectives.

I will begin this talk by introducing Granite Vision, our vision-language model that specializes in visual document understanding and enables efficient content extraction from tables, charts, infographics, plots, diagrams, sketches, and more. In particular, I will discuss the notion of code-guided data generation for multimodal comprehension. Next, I will expand to speech and audio, discussing the challenges of building a unified model for simultaneous comprehension of multimodal video and generation of prosodic speech for sports AI commentary. Our work was deployed at the US Open and Wimbledon tennis tournaments to provide commentary for matches that would otherwise lack coverage. I will conclude by discussing our current research directions and future opportunities in this space.

Unified multimodal models promise to understand and generate in a single architecture, but most of today’s designs treat the camera in oversimplified ways, assuming fixed viewpoints or predefined fields of view. This limits their ability to handle real-world scenarios where perspectives shift and contexts are dynamic. In this talk, I will introduce Puffin, our new multimodal framework that brings the camera dimension into the picture. Puffin combines autoregressive and diffusion modeling to interpret and generate scenes from arbitrary viewpoints with continuous fields of view. A key idea behind Puffin is to treat the camera as language, allowing the model to “think with camera.” This aligns visual cues with photographic terms and grounds reasoning in physical context, making the model more spatially aware. Puffin is trained on a large dataset of four million vision–language–camera triplets, with both global camera parameters and pixel-level camera maps. This enables precise and flexible control over spatial generation. I’ll show how Puffin outperforms specialized baselines in controllable generation and camera understanding, and how, with instruction tuning, it generalizes to diverse tasks like spatial imagination, world exploration, and photography guidance.

Visual creation is a long-horizon, communicative process: we plan, sketch, get feedback, and revise, over many rounds and often with many collaborators. Today’s generative models keep getting stronger, yet iterating with them still hurts: text cannot name a specific element, untouched content drifts across rounds, and every edit re-renders everything. What is missing is a shared, unified communication medium—a grounded symbolic canvas that humans, understanding models, and generation models can all read and write, the way code, diffs, and tests already serve coding agents. I will trace my work building pieces of this interface, from LLM-based layout planning to localized critique and revision, and close with a vision for models that learn the visual creation process itself.