Zhang Yi

I am currently a Senior Researcher working on generative AI. I obtained my Ph.D. in 2023 from the MMLab at The Chinese University of Hong Kong (CUHK), where I was fortunate to be advised by Prof. Hongsheng Li and Prof. Xiaogang Wang. Prior to that, I earned my bachelor's degree from Nanjing University in 2019.

Email  /  Google Scholar  /  Github /  CV

My research spans multimodal models, diffusion models, and computational photography, with a focus on advancing state-of-the-art techniques and addressing real-world challenges. My contributions have been recognized with the SIGGRAPH Asia 2024 Best Paper Award, and most of my publications have been successfully deployed in commercial products.

RIGen (Reciprocally Interactive Generation) goes beyond action following in generative gameplay: it combines persistent game state and configurable rules, contextual interaction reasoning, and action-conditioned audio-video generation, so that player actions have lasting consequences and the generated world acts on its own state. Trained on a large Xonotic gameplay dataset, it powers RIGen-Arena, an interactive generative FPS prototype.

Radian adds real-data adversarial supervision to on-policy DMD for few-step autoregressive video generation. Discriminator heads operate on frozen visual foundation model features of sparsely decoded rollouts, curbing error accumulation and improving perceptual quality on Wan2.1 for chunk-wise, frame-wise, and minute-long generation at zero extra inference cost.

Self-Aligned Forcing aligns each block's K/V history to the noise level of the block being denoised and keeps it differentiable, so a whole denoising stage runs in one causally masked pass. It trains up to 1.8x faster with less memory, reaches 49.1 FPS on 4 GPUs, and better balances visual quality and motion in long-horizon streaming generation.

Salt++ is a two-stage post-training framework for few-step streaming audio-video generation. Causal Self-Flow supervises contextual representations directly, and context-aligned AR DMD shares one causal context across generator, fake score, and real score. In the same 4-step causal setting it improves visual quality by 57% and motion quality by 45% over OmniForcing, and scales to 4-step 1664×960 generation.

Omni-LiveAvatar is the first framework for minute-level, real-time streaming joint audio-video avatar generation. It distills a bidirectional audio-video diffusion teacher (LTX-2) into a few-step autoregressive generator, with synchronized long-short-term memory and hierarchical rolling prompt planning, achieving a 33× speedup on a single H200.

Salt distills video generators down to 2–4 NFEs by combining Self-Consistent Distribution Matching Distillation (SC-DMD) with Cache-Distribution-Aware training, treating the KV cache as a quality-parameterized condition. It improves low-NFE quality on both non-autoregressive (Wan 2.1) and real-time autoregressive (Self-Forcing) backbones.

AR-CoPO adapts Neighbor GRPO to few-step streaming autoregressive video generators. By forking neighborhood candidates at a randomly chosen chunk and blending on-policy exploration with a replay buffer of reference rollouts, it delivers genuine RLHF alignment gains on Self-Forcing without falling into reward hacking.

Neighbor GRPO enables GRPO for flow matching models by bypassing SDEs entirely. By reinterpreting SDE-based GRPO from a distance optimization perspective, we generate diverse trajectories via perturbed initial noise and optimize using a distance-based surrogate policy, preserving ODE efficiency while outperforming SDE-based methods.

We propose Blend-Aware Latent Diffusion (BlendRecon + BlendGen ) to address boundary discontinuity and content inconsistency in diffusion-based inpainting, mitigating stitched seams and improving quality.

We address convergence difficulties of Distribution Matching Distillation (DMD) on large-scale flow-based models. By proposing implicit distribution alignment (IDA) and intra-segment guidance (ISG), SenseFlow enables DMD to converge for SD 3.5 and FLUX.1 dev, achieving superior distillation performance for both diffusion and flow-matching models.

The study introduces the Curriculum Consistency Model (CCM) to address inconsistent learning complexity in Consistency Distillation for diffusion and flow models, balancing complexity via a PSNR-based curriculum.

A novel AR mirror system that allows a seamless, perspective-aligned user experience, which is enabled by placing the camera behind a transparent display.

We introduce Motion-I2V, a novel framework for consistent and controllable image-to-video generation (I2V). In contrast to previous methods that directly learn the complicated image-to-video mapping, Motion-I2V factorizes I2V into two stages with explicit motion modeling.

A unified conditional framework based on diffusion models for image restoration.

A general-purpose backbone for image restoration tasks (e.g. denoising, deraining, and deblurring). A novel kernel basis attention (KBA) module has been proposed to effectively aggregate the spatial information via a series of learnable kernel bases.

We propose a grouped spatial-temporal shift module for effective video restoration. It surpasses previous SOTA methods with only 43% computational cost on video deblurring and denoising.

A practical low-light imaging system for burst raw image denoising. It has been deployed to some commercial smartphones.

Learning degradation representations for blurry images. The learned degradation representation shows pleasing improvements on both image deblurring and reblurring.

A self-supervised image denoising method using noisy images and the noise model. We achieve SOTA results on both real-world noise and synthetic noises (both point-wise and spatial correlated noise types).

A method to synthesize "almost the most realistic" raw image noise by sampling directly from the real noise distribution. We also found existing comparisons of noise synthesis methods are based on inaccurate noise parameters.

We achieve over 93% winning rate of Protoss against the most difficult non-cheating built-in AI (level-7) of Terran, training within two days using a single machine with only 48 CPU cores and 8 K40 GPUs.

The website template was borrowed from here.