: Beyond Talking Avatar via Streaming Multimodal References and Proactive Agent Planning

FlowAct-R2 introduces a Streaming Multimodal Reference DiT for streaming digital human video generation, built by adapting the pretrained Seedance 2.0 Mini reference-to-video backbone. By extending Diffusion Forcing to streaming multimodal references, the model continuously incorporates rolling action prompts, audio, and dynamically updated image, audio, and video references while preserving bidirectional spatiotemporal modeling. A video-driven RoPE aligns incoming reference chunks with the generation timeline, enabling references to be updated or switched during an ongoing stream. FlowAct-R2 further combines explicitly trained reference-plus-image conditioning with partially noised historical motion frames to preserve identity and appearance while reducing error accumulation.

Complementing the video generator, the Proactive Interaction Agent enables a digital human to proactively sustain a live session while remaining responsive to audience interactions. Offline, the agent derives a persona from multimodal character assets, organizes the session into a long-horizon agenda, and compiles complex behaviors into reusable multimodal skills. Online, it combines the prepared agenda and skills with audience events and summarized interaction history to determine what behavior to perform and how it should be scheduled. By initiating, overlaying, deferring, interrupting, or resuming behaviors according to the current context, the digital human can continue purposeful activities without continuous user prompts while responding naturally when new interactions arise.