Zhile Ren

Email: jrenzhile -at- gmail.com

I work at Apple on hardware-aware efficient-ML frameworks and algorithms, and optimizing on-device Apple Intelligence large language models and diffusion models.

Before that, I worked at Georgia Tech as a Postdoc with Dhruv Batra, Devi Parikh, and Irfan Essa. I got my PhD in Brown University working with Erik Sudderth in the computer science department. I did my undergrad in statistics at Zhejiang University.

Google Scholar | LinkedIn | Curriculum Vitae (in PDF)

Research Articles

Apple Intelligence Foundation Language Models
arXiv, 2024
link
Deploying Attention-Based Vision Transformers to Apple Neural Engine
Apple research article, 2024
link

Publications

Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference
Fred Hohman, Chaoqun Wang, Jinmook Lee, Jochen Görtler, Dominik Moritz, Jeffrey Bigham, Zhile Ren, Cecile Foret, Qi Shan, Xiaoyi Zhang
ACM Conference on Human Factors in Computing Systems (CHI 2024, Honorable Mention)
Paper
UPSCALE: Unconstrained Channel Pruning
Alvin Wan, Hanxiang Hao, Kaushik Patnaik, Sam Xu, Omer Hadad, David Güera, Zhile Ren, Qi Shan
International Conference on Machine Learning (ICML 2023)
Paper Code
AutoFocusFormer: Image Segmentation off the Grid
Chen Ziwen, Kaushik Patnaik, Shuangfei Zhai, Alvin Wan, Zhile Ren, Alexander G. Schwing, Alex Colburn, Li Fuxin
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2023)
Paper Code
Generative Multiplane Images: Making a 2D GAN 3D-Aware
Xiaoming Zhao, Fangchang Ma, David Güera, Zhile Ren, Alexander G. Schwing, Alex Colburn
European Conference on Computer Vision (ECCV 2022 oral presentation)
Paper Project Page
FvOR: Robust Joint Shape and Pose Optimization for Few-view Object Reconstruction
Zhenpei Yang, Zhile Ren, Miguel Angel Bautista, Zaiwei Zhang, Qi Shan, Qixing Huang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)
Paper Code
MVS2D: Efficient Multi-view Stereo via Attention-Driven 2D Convolutions
Zhenpei Yang, Zhile Ren, Qi Shan, Qixing Huang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)
Paper Project Page
Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Views
Vincent Cartillier, Zhile Ren, Neha Jain, Stefan Lee, Irfan Essa, Dhruv Batra
AAAI Conference on Artificial Intelligence (AAAI 2021)
Media coverage: Venture Beat, MIT Technology Review, Digital Trends, ZDNet
Paper Project Page
Clouds of Oriented Gradients for 3D Detection of Objects, Surfaces, and Indoor Scene Layouts
Zhile Ren, Erik Sudderth
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI 2020)
Paper
Cross-Channel Communication Networks
Jianwei Yang, Zhile Ren, Chuang Gan, Hongyuan Zhu, Devi Parikh
Neural Information Processing Systems (NeurIPS 2019)
Paper Poster Code
Embodied Amodal Recognition: Learning to Move to Perceive Objects
Jianwei Yang*, Zhile Ren*, Mingze Xu, Xinlei Chen, David Crandall, Devi Parikh, Dhruv Batra
(Equal Contribution*)
IEEE International Conference on Computer Vision (ICCV 2019)
Paper ML@GT Blog
3D Scene Reconstruction with Multi-layer Depth and Epipolar Transformers
Daeyun Shin, Zhile Ren, Erik Sudderth, Charless Fowlkes
IEEE International Conference on Computer Vision (ICCV 2019)
Paper Supplementary Video Project Page
A Fusion Approach for Multi-Frame Optical Flow Estimation
Zhile Ren, Orazio Gallo, Deqing Sun, Ming-Hsuan Yang, Jan Kautz, Erik Sudderth
IEEE Winter Conference on Applications of Computer Vision (WACV 2019)
Nov 2019: MFF consistently ranks top-2 among published flow methods in KITTI and MPI Sintel
Paper Project Page Supplementary Video
3D Object Detection with Latent Support Surfaces
Zhile Ren, Erik Sudderth
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2018)
Paper Code (Latent-SSVM)
Cascaded Scene Flow Prediction using Semantic Segmentation
Zhile Ren, Deqing Sun, Jan Kautz, Erik Sudderth
International Conference on 3D Vision (3DV 2017 oral presentation)
Paper Supplementary Talk Slides
Three-Dimensional Object Detection and Layout Prediction using Clouds of Oriented Gradients
Zhile Ren, Erik Sudderth
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016 oral presentation)
Paper Supplementary Detection Results Talk Slides Talk Recording

Robust Graph SLAM in Dynamic Environments with Moving Landmarks
Lingzhu Xiang, Zhile Ren, Mengrui Ni, Odest Chadwicke Jenkins
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015)
Paper
Transient Attributes for High-Level Understanding and Editing of Outdoor Scenes
Pierre-Yves Laffont, Zhile Ren, Xiaofeng Tao, Chao Qian and James Hays
ACM Transactions on Graphics (SIGGRAPH 2014)
Media coverage: Brown News, NBC News, IEEE Spectrum, PBS, Mic Gizmodo
Paper Project Page Code (Color Transformation) Talk Recording
Image Segmentation by Cascaded Region Agglomeration
Zhile Ren, Greg Shakhnarovich
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013)
Paper Supplementary Results

Thesis

Semantic Three-Dimensional Understanding of Dynamic Scenes
Zhile Ren
Doctoral Thesis, Brown University, May 2018
Thesis

Miscellaneous

  • (Almost) Everyone calls me "Ren"
  • You can find me in social networks: Facebook Instagram Twitter Goodreads Strava