Zhixin Shu

Zhixin Shu

Address: 345 Park Ave, San Jose, CA 95110

Email: zshu AT adobe DOT com

LinkedIn

Google Scholar



About

I'm a research scientist at Adobe Research since 2020. I obtained my Ph.D. in Computer Science from Stony Brook University in 2019.

I'm interested in research topics related to humans in the intersection of computer vision, computer graphics, and machine learning. My recent work has a focus on human-related controllable generation, 3D modeling, neural rendering, relighting, and editing.

Publications

 
Generative Portrait Shadow Removal
Jae Shin Yoon, Zhixin Shu, Mengwei Ren, Xuaner Zhang, Yannick Hold-Geoffroy, Krishna Kumar Singh, He Zhang
SIGGRAPH Asia 2024 (TOG)
[paper]

 
COMPOSE: Comprehensive Portrait Shadow Editing
Andrew Hou, Zhixin Shu, Xuaner Zhang, He Zhang, Yannick Hold-Geoffroy, Jae Shin Yoon, Xiaoming Liu
ECCV 2024
[paper]

 
LRM-Zero: Training Large Reconstruction Models with Synthesized Data
Desai Xie, Sai Bi, Zhixin Shu, Kai Zhang, Zexiang Xu, Yi Zhou, Sören Pirk, Arie Kaufman, Xin Sun, Hao Tan
NeurIPS 2024
[paper]

 
Perm: A Parametric Representation for Multi-Style 3D Hair Modeling
Chengan He, Xin Sun, Zhixin Shu, Fujun Luan, Sören Pirk, Jorge Alejandro Amador Herrera, Dominik L. Michels, Tuanfeng Y. Wang, Meng Zhang, Holly Rushmeier, Yi Zhou
arXiv 2024
[paper]

 
Rig3DGS: Creating Controllable Portraits from Casual Monocular Videos
Alfredo Rivero, ShahRukh Athar, Zhixin Shu, Dimitris Samaras
arXiv 2024
[paper]

 
Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning
Desai Xie, Jiahao Li, Hao Tan, Xin Sun, Zhixin Shu, Yi Zhou, Sai Bi, Sören Pirk, Arie E. Kaufman
CVPR 2024
[paper]

 
In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition
Yiran Xu, Zhixin Shu, Cameron Smith, Seoung Wug Oh, Jia-Bin Huang
CVPR 2024
[paper]

 
Holo-relighting: Controllable volumetric portrait relighting from a single image
Yiqun Mei, Yu Zeng, He Zhang, Zhixin Shu, Xuaner Zhang, Sai Bi, Jianming Zhang, HyunJoon Jung, Vishal M. Patel
CVPR 2024
[paper]

 
Relightful Harmonization: Lighting-aware Portrait Background Replacement
Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, He Zhang
CVPR 2024
[paper]

 
Controllable Dynamic Appearance for Neural 3D Portraits
ShahRukh Athar, Zhixin Shu, Zexiang Xu, Fujun Luan, Sai Bi, Kalyan Sunkavalli, Dimitris Samaras
3DV 2024
[paper]

 
Consistent Multimodal Generation via a Unified GAN Framework
Zhen Zhu, Yijun Li, Weijie Lyu, Krishna Kumar Singh, Zhixin Shu, Sören Pirk, Derek Hoiem
WACV 2024
[paper]

 
LightPainter: Interactive Portrait Relighting with Freehand Scribble
Yiqun Mei, He Zhang, Xuaner Zhang, Jianming Zhang, Zhixin Shu, Yilin Wang, Zijun Wei, Shi Yan, HyunJoon Jung, Vishal M. Patel
CVPR 2023
[paper]

 
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields
Zhengfei Kuang, Fujun Luan, Sai Bi, Zhixin Shu, Gordon Wetzstein, Kalyan Sunkavalli
CVPR 2023
[paper]

 
FLAME-in-NeRF: Neural control of Radiance Fields for Free View Face Animation
ShahRukh Athar, Zhixin Shu, Dimitris Samaras
FG 2023
[paper]

 
Learning an Isometric Surface Parameterization for Texture Unwrapping
Sagnik Das, Ke Ma, Zhixin Shu, Dimitris Samaras
ECCV 2022
[paper]

 
3D-FM GAN: Towards 3D-Controllable Face Manipulation
Yuchen Liu, Zhixin Shu, Yijun Li, Zhe Lin, Richard Zhang, S.Y. Kung
ECCV 2022
[paper]

 
Learning From Documents in the Wild to Improve Document Unwarping
Ke Ma, Sagnik Das, Zhixin Shu, Dimitris Samaras
SIGGRAPH 2022
[paper]

 
RigNeRF: Fully Controllable Neural 3D Portraits
ShahRukh Athar, Zexiang Xu, Kalyan Sunkavalli, Eli Shechtman, Zhixin Shu
CVPR 2022
[paper]

 
Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera
Jae Shin Yoon, Duygu Ceylan, Tuanfeng Y. Wang, Jingwan Lu, Jimei Yang, Zhixin Shu, Hyun Soo Park
CVPR 2022
[paper]

 
Point-NeRF: Point-based Neural Radiance Fields
Qiangeng Xu, Zexiang Xu, Julien Philip, Sai Bi, Zhixin Shu, Kalyan Sunkavalli, Ulrich Neumann
CVPR 2022 (oral)
[paper][bib]

 
Temporally Consistent Relighting for Portrait Videos
Sreenithy Chandran, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Zhixin Shu, Suren Jayasuriya
WACV 2021
[paper][bib]

 
Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
Badour AlBahar, Jingwan Lu, Jimei Yang, Zhixin Shu, Eli Shechtman, Jia-Bin Huang
SIGGRAPH Asia (TOG) 2021
[paper][bib]

 
Single-image Full-body Human Relighting
Manuel Lagunas, Xin Sun, Jimei Yang, Ruben Villegas, Jianming Zhang, Zhixin Shu, Belen Masia, Diego Gutierrez
EGSR 2021
[paper][bib]

 
Content-Aware GAN Compression
Yuchen Liu, Zhixin Shu, Yijun Li, Zhe Lin, Federico Perazzi, S.Y. Kung
CVPR 2021
[paper][bib]

 
Self-supervised Deformation Modeling for Facial Expression Editing
ShahRukh Athar, Zhixin Shu, Dimitris Samaras
FG 2020
[paper][bib]

 
Learning Monocular Face Reconstruction using Multi-View Supervision
Zhixin Shu, Duygu Ceylan, Kalyan Sunkavalli, Eli Shechtman, Sunil Hadap, Dimitris Samaras
FG 2020 (best paper runner up)
[paper][bib]

 
DewarpNet: Single-Image Document Unwarping with Stacked 3D and 2D Regression Networks
Sagnik Das*, Ke Ma*, Zhixin Shu, Roy Shilrot, Dimitris Samaras
ICCV 2019
[paper]

 
Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model using Deep Non-Rigid Structure from Motion
Mihir Sahasrabudhe*, Zhixin Shu*, Edward Bartrum, Rıza Alp Güler, Dimitris Samaras, Nikos Paragios, Iasonas Kokkinos 
ICCV Workshop 2019
[paper][bib]

 
An Adversarial Neuro-Tensorial Approach for Learning Disentangled Representations
Mengjiao Wang, Zhixin Shu, Shiyang Cheng, Yannis Panagakis, Dimitris Samaras, Stefanos Zafeiriou
IJCV 2019
[paper][bib]

 
Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
Zhixin Shu, Mihir Sahasrabudhe, Rıza Alp Güler, Dimitris Samaras, Nikos Paragios, Iasonas Kokkinos 
ECCV 2018
[paper][bib][code]

 
DocUNet: Document Image Unwarping via A Stacked U-Net
Ke Ma, Zhixin Shu, Xue Bai, Jue Wang, Dimitris Samaras
CVPR 2018
[paper][bib]

 
Neural Face Editing with Intrinsic Image Disentangling
Zhixin Shu, Ersin Yumer, Sunil Hadap, Kalyan Sunkavalli, Eli Shechtman, Dimitris Samaras
CVPR 2017 (oral)
[paper][bib][code]

 
Portrait Lighting Transfer using a Mass Transport Approach
Zhixin Shu, Sunil Hadap, Eli Shechtman, Kalyan Sunkavalli, Sylvain Paris, Dimitris Samaras
TOG (presented at SIGGRAPH) 2017
[paper][bib]

 
EyeOpener: Editing Eyes in the Wild
Zhixin Shu, Eli Shechtman, Dimitris Samaras, Sunil Hadap
TOG (presented at SIGGRAPH) 2016
[paper][bib]