Yuandong Tian's webpage

	GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection [link] [code] Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, Yuandong Tian arXiv 2024
	MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases [link] Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, Liangzhen Lai, Vikas Chandra arXiv 2024
	*Beyond A: Better Planning with Transformers via Search Dynamics Bootstrapping** [link] Lucas Lehnert, Sainbayar Sukhbaatar, Paul Mcvay, Michael Rabbat, Yuandong Tian arXiv 2024
	Efficient Streaming Language Models with Attention Sinks [link] [code] [MIT News] [Yannic Kilcher's video introduction] [VentureBeat] [Huggingface library] [Intel extension of Transformers] [MLC Chat] [Y-combinator] Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis ICLR 2024
	JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention [link] [5min ICLR talk] [5min ICLR slides] Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon Du ICLR 2024
	RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment [link] [code] Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, Yuandong Tian ICLR 2024
	H-GAP: Humanoid Control with a Generalist Planner [link] [website] Zhengyao Jiang, Yingchen Xu, Nolan Wagener, Yicheng Luo, Michael Janner, Edward Grefenstette, Tim Rocktaschel, Yuandong Tian (* = Equal 1st authors) ICLR 2024 *(Spotlight)*
	End-to-end Story Plot Generator [link] Hanlin Zhu, Andrew Cohen, Danqing Wang, Kevin Yang, Xiaomeng Yang, Jiantao Jiao, Yuandong Tian (* = Equal 1st authors) arXiv 2023
	Learning Personalized Story Evaluation [link] [code] Danqing Wang, Kevin Yang, Hanlin Zhu, Xiaomeng Yang, Andrew Cohen, Lei Li, Yuandong Tian arXiv 2023
	H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models [link] [code] Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Re, Clark Barrett, Zhangyang Wang, Beidi Chen NeurIPS 2023
	Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer [link] [talk] [slides] [5min NeurIPS talk] [5min NeurIPS slides] [poster] Yuandong Tian, Yiping Wang, Beidi Chen, Simon Du NeurIPS 2023
	Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information [link] [code] Arman Zharmagambetov, Brandon Amos, Aaron Ferber, Taoan Huang, Bistra Dilkina, Yuandong Tian NeurIPS 2023
	Extending Context Window of Large Language Models via Positional Interpolation [link] Shouyuan Chen, Sherman Wong, Liangjian Chen, Yuandong Tian arXiv 2023
	DOC: Improving Long Story Coherence With Detailed Outline Control [link] [code] Kevin Yang, Dan Klein, Nanyun Peng, Yuandong Tian ACL 2023
	Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time [link] Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, Beidi Chen ICML 2023 *(Oral)*
	A Cookbook of Self-Supervised Learning [link] Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yann LeCun, Micah Goldblum arXiv 2023
	Searching Large Neighborhoods for Integer Linear Programs with Contrastive Learning [link] Taoan Huang, Aaron Ferber, Yuandong Tian, Bistra Dilkina, Benoit Steiner ICML 2023
	SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems [link] [code] [Slides] Aaron Ferber, Taoan Huang, Daochen Zha, Martin Schubert, Benoit Steiner, Bistra Dilkina, Yuandong Tian ICML 2023 *(Outstanding paper in Sampling and Optimization in Discrete Space (SODS) Workshop)*
	Learning Compiler Pass Orders using Coreset and Normalized Value Prediction [link] Youwei Liang, Kevin Stone, Ali Shameli, Chris Cummins, Mostafa Elhoushi, Jiadong Guo, Benoit Steiner, Xiaomeng Yang, Pengtao Xie, Hugh Leather, Yuandong Tian (* = Equal 1st authors) ICML 2023
	Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models [link] [code] Daochen Zha, Louis Feng, Liang Luo, Bhargav Bhushanam, Zirui Liu, Yusuo Hu, Jade Nie, Yuzhen Huang, Yuandong Tian, Arun Kejariwal, Xia Hu MLSys 2023
	Local Branching Relaxation Heuristics for Integer Linear Programs [link] Taoan Huang, Aaron Ferber, Yuandong Tian, Bistra Dilkina, Benoit Steiner CPAIOR 2023
	Efficient Planning in a Compact Latent Action Space [link] [code] [website] Zhengyao Jiang, Tianjun Zhang, Michael Janner, Yueying Li, Tim Rocktaschel, Edward Grefenstette, Yuandong Tian ICLR 2023
	Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning [link] [code] [workshop version] [workshop poster] [5min talk] Yuandong Tian ICLR 2023
	MACTA: A Multi-agent Reinforcement Learning Approach for Cache Timing Attacks and Detection [link] Jiaxun Cui, Xiaomeng Yang, Geunbae Lee, Mulong Luo, Peter Stone, Hsien-Hsin S. Lee, Benjamin Lee, G. Edward Suh, Wenjie Xiong, Yuandong Tian* (* = Equal 2nd authors, ** = Equal advising) ICLR 2023
	Modeling Scattering Coefficients using Self-Attentive Complex Polynomials with Image-based Representation [link] Andrew Cohen, Weiping Dou, Jiang Zhu, Slawomir Koziel, Peter Renner, Jan-Ove Mattsson, Xiaomeng Yang, Beidi Chen, Kevin Stone, Yuandong Tian (* = Equal 1st authors) arXiv 2023
	AutoCAT: Reinforcement Learning for Automated Exploration of Cache Timing-Channel Attacks [link] [code] Mulong Luo, Wenjie Xiong, Geunbae Lee, Yueying Li, Xiaomeng Yang, Amy Zhang, Yuandong Tian, Hsien Hsin S Lee, G Edward Suh (* = Equal 1st authors) HPCA 2023
	Re3: Generating Longer Stories With Recursive Reprompting and Revision [link] [code] Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein EMNLP 2022
	Understanding Deep Contrastive Learning via Coordinate-wise Optimization [link] [code] [video] [5min talk slides] [poster] Yuandong Tian NeurIPS 2022 *(Oral)*
	DreamShard: Generalizable Embedding Table Placement for Recommender Systems [link] [code] Daochen Zha, Louis Feng, Qiaoyu Tan, Zirui Liu, Kwei-Herng Lai, Bhargav Bhushanam, Yuandong Tian, Arun Kejariwal, Xia Hu NeurIPS 2022
	AutoShard: Automated Embedding Table Sharding for Recommender Systems [link] [code] Daochen Zha, Louis Feng, Bhargav Bhushanam, Dhruv Choudhary, Jade Nie, Yuandong Tian, Jay Chae, Yinbin Ma, Arun Kejariwal, Xia Hu KDD 2022
	Denoised MDPs: Learning World Models Better Than the World Itself [link] [code] [website] Tongzhou Wang, Simon S Du, Antonio Torralba, Phillip Isola, Amy Zhang, Yuandong Tian ICML 2022
	On the Importance of Asymmetry for Siamese Representation Learning [link] [code] Xiao Wang, Haoqi Fan, Yuandong Tian, Daisuke Kihara, Xinlei Chen (* = Equal 1st authors) CVPR 2022
	Understanding Dimensional Collapse in Contrastive Self-supervised Learning [link] [code] Li Jing, Pascal Vincent, Yann LeCun, Yuandong Tian ICLR 2022
	NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training [link] [code] Chengyue Gong, Dilin Wang, Meng Li, Xinlei Chen, Zhicheng Yan, Yuandong Tian, Vikas Chandra ICLR 2022
	Multi-objective Optimization by Learning Space Partitions [link] [code] Yiyang Zhao, Linnan Wang, Kevin Yang, Tianjun Zhang, Tian Guo, Yuandong Tian ICLR 2022
	Towards demystifying representation learning with non-contrastive self-supervision [link] Xiang Wang, Xinlei Chen, Simon S Du, Yuandong Tian arXiv 2021
	Sample-Efficient Neural Architecture Search by Learning Action Space [link] [code] Linnan Wang, Saining Xie, Teng Li, Rodrigo Fonseca, Yuandong Tian T-PAMI 2021
	Learning Bounded Context-Free-Grammar via LSTM and the Transformer: Difference and Explanations [link] [code] Hui Shi, Sicun Gao, Yuandong Tian, Xinyun Chen, Jishen Zhao AAAI 2022
	CompilerGym: robust, performant compiler optimization environments for AI research [link] [code] Chris Cummins, Bram Wasti, Jiadong Guo, Brandon Cui, Jason Ansel, Sahir Gomez, Somya Jain, Jia Liu, Olivier Teytaud, Benoit Steiner, Yuandong Tian, Hugh Leather CGO 2022 *(Outstanding Paper)*
	NovelD: A Simple yet Effective Exploration Criterion [link] [code] [video] Tianjun Zhang, Huazhe Xu, Xiaolong Wang, Yi Wu, Kurt Keutzer, Joseph E. Gonzalez, Yuandong Tian NeurIPS 2021
	MADE: Exploration via Maximizing Deviation from Explored Regions [link] [code] Tianjun Zhang, Paria Rashidinejad, Jiantao Jiao, Yuandong Tian, Joseph Gonzalez, Stuart Russell (* = Equal 1st authors) NeurIPS 2021
	Learning Space Partitions for Path Planning [link] [code] Kevin Yang, Tianjun Zhang, Chris Cummins, Brandon Cui, Benoit Steiner, Linnan Wang, Joseph E. Gonzalez, Dan Klein, Yuandong Tian (* = Equal 1st authors) NeurIPS 2021
	Latent Execution for Neural Program Synthesis Beyond Domain-Specific Languages [link] [code] Xinyun Chen, Dawn Song, Yuandong Tian NeurIPS 2021
	Understanding Self-supervised Learning Dynamics without Contrastive Pairs [link] [code] [video] [Slides] [Blogpost] [Independent Reproduction] Yuandong Tian, Xinlei Chen, Surya Ganguli ICML 2021 *(Outstanding Paper Award Honorable Mention)*
	Few-shot Neural Architecture Search [link] [code] [Blogpost] Yiyang Zhao, Linnan Wang, Yuandong Tian, Rodrigo Fonseca, Tian Guo (* = Equal 1st authors) ICML 2021 *(Long Oral)*
	Learn-to-Share: A Hardware-friendly Transfer Learning Framework Exploiting Computation and Parameter Sharing [link] Cheng Fu, Hanxian Huang, Xinyun Chen, Yuandong Tian, Jishen Zhao (UCSD) ICML 2021 *(Long Oral)*
	Network Planning with Deep Reinforcement Learning [link] [code] Hang Zhu (JHU), Varun Gupta, Satyajeet Singh Ahuja, Yuandong Tian, Ying Zhang, Xin Jin SIGCOMM 2021
	FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining [link] Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Bichen Wu, Zijian He, Zhen Wei, Kan Chen, Yuandong Tian, Matthew Yu, Peter Vajda, Joseph Gonzalez CVPR 2021
	FPNAS: Fast Probabilistic Neural Architecture Search [link] Zhicheng Yan, Xiaoliang Dai, Peizhao Zhang, Yuandong Tian, Bichen Wu, Matt Feiszli CVPR 2021
	Understanding Robustness in Teacher-Student Setting: A New Perspective [link] [Slides] Zhuolin Yang, Zhaoxi Chen, Tiffany Cai, Xinyun Chen, Bo Li, Yuandong Tian (* = Equal 1st authors) AIStats 2021
	Multi-Agent Collaboration via Reward Attribution Decomposition [link] [code] [video] [website] Tianjun Zhang, Huazhe Xu, Xiaolong Wang, Yi Wu, Kurt Keutzer, Joseph E. Gonzalez, Yuandong Tian arxiv 2020
	Joint Policy Search for Multi-agent Collaboration with Imperfect Information [link] [code] [video] Yuandong Tian, Qucheng Gong, Tina Jiang NeurIPS 2020
	Understanding Self-supervised Learning with Dual Deep Networks [link] [code] [video] Yuandong Tian, Lantao Yu, Xinlei Chen, Surya Ganguli arXiv 2020
	Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension [link] [code] Yuandong Tian ICML 2020
	Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP [link] Haonan Yu, Sergey Edunov, Yuandong Tian, Ari S. Morcos ICLR 2020
	Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search [link] [code] Linnan Wang, Rodrigo Fonseca, Yuandong Tian NeurIPS 2020
	Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction [link] Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, Xia Hu KDD 2020
	FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions [link] [code] Alvin Wan, Xiaoliang Dai, Peizhao Zhang, Zijian He, Yuandong Tian, Saining Xie, Bichen Wu, Matthew Yu, Tao Xu, Kan Chen, Peter Vajda, Joseph Gonzalez CVPR 2020
	N-Bref : A High-fidelity Decompiler Exploiting Programming Structures [link] [code] [Blogpost] Cheng Fu, Kunlin Yang, Xinyun Chen, Yuandong Tian, Jishen Zhao arxiv 2020
	AlphaX: Exploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search [link] Linnan Wang, Yiyang Zhao, Yuu Jinnai, Yuandong Tian, Rodrigo Fonseca AAAI 2020
	Deep Symbolic Superoptimization Without Human Knowledge [link] [code] Hui Shi, Yang Zhang, Xinyun Chen, Yuandong Tian, Jishen Zhao ICLR 2020
	Hierarchical Decision Making by Generating and Following Natural Language Instructions [link] [code] Hengyuan Hu, Denis Yarats, Qucheng Gong, Yuandong Tian, Mike Lewis (* = Equal 1st authors) NeurIPS 2019
	ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero [link] [code] [website] [pretrained model and game records] [Blogpost] [Talk] [Forbes] [TechCrunch] [reddit] An open source reimplementation of DeepMind's zero-knowledge training and its application to the game of Go. Trained on 2000 GPUs for 9 days. With a single GPU and 50 seconds per move, the model won 20-0 versus 4 top 30 professional players, given human unlimited thinking time. It also won 980-18 versus LeelaZero (version Apr. 25). Yuandong Tian, Jerry Ma, Qucheng Gong, Shubho Sengupta, Zhuoyuan Chen, James Pinkerton, Larry Zitnick ( = Equal 2nd authors) ICML 2019 *(Long Oral)*
	Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees [link] [code] Yuping Luo, Huazhe Xu, Yuanzhi Li, Yuandong Tian, Trevor Darrell, Tengyu Ma (* = Equal 1st authors) ICLR 2019
	M^3RL: Mind-aware Multi-agent Management Reinforcement Learning [link] [code] Tianmin Shu, Yuandong Tian ICLR 2019
	Luck Matters: Understanding Training Dynamics of Deep ReLU Networks [link] [code] [Poster] Yuandong Tian, Tina Jiang, Qucheng Gong, Ari Morcos ICML-workshop 2019
	One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers [link] Ari S. Morcos, Haonan Yu, Michela Paganini, Yuandong Tian NeurIPS 2019
	Real-world video adaptation with reinforcement learning [link] Hongzi Mao, Shannon Chen, Drew Dimmery, Shaun Singh, Drew Blaisdell, Yuandong Tian, Mohammad Alizadeh, Eytan Bakshy ICML-Workshop 2019
	FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search [link] [code] Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, Kurt Keutzer CVPR 2019
	Learning to Perform Local Rewriting for Combinatorial Optimization [link] [code] Xinyun Chen, Yuandong Tian NeurIPS 2019
	Coda: An End-to-End Neural Program Decompiler [link] Cheng Fu, Huili Chen, Haolan Liu, Xinyun Chen, Yuandong Tian, Farinaz Koushanfar, Jishen Zhao NeurIPS 2019
	Building Generalizable Agents with a Realistic and Rich 3D Environment [link] [code] Yi Wu, Yuxin Wu, Georgia Gkioxari, Yuandong Tian ICLR-Workshop 2018
	A Theoretical Framework for Deep Locally Connected ReLU Network [link] [Poster] Yuandong Tian arxiv 2018
	Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima [link] Simon S. Du, Jason D. Lee, Yuandong Tian, Barnabas Poczos, Aarti Singh ICML 2018 *(Long Oral)*
	When is a Convolutional Filter Easy To Learn? [link] Simon S. Du, Jason D. Lee, Yuandong Tian ICLR 2018
	ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games [link] [code] [video] Yuandong Tian, Qucheng Gong, Wenling Shang, Yuxin Wu, Larry Zitnick NeurIPS 2017 *(Oral)*
	Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning [link] Yuxin Wu, Yuandong Tian ICLR 2017
	An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis [link] [code] Yuandong Tian ICML 2017
	Semantic Amodal Segmentation [link] Yan Zhu, Yuandong Tian, Dimitris Mexatas, Piotr Dollár CVPR 2017
	Better Computer Go Player with Neural Network and Long-term Prediction [link] [code] [pretrained model] [mit tech review] [wired] Yuandong Tian, Yan Zhu ICLR 2016
	Single Image 3D Interpreter Network [link] Jiajun Wu, Tianfan Xue, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, William T. Freeman (* = Equal 1st authors) ECCV 2016 *(Oral)*
	Simple Baseline for Visual Question Answering [link] [code] Bolei Zhou, Yuandong Tian, Sainbayar Sukhbaatar, Arthur Szlam, Rob Fergus arxiv 2016
	Theory and Practice of Hierarchical Data-driven Descent for Optimal Deformation Estimation [link] Yuandong Tian, Srinivasa G. Narasimhan IJCV 2015
	Theory and Practice of Globally Optimal Deformation Estimation [link] Yuandong Tian PhD thesis 2013
	Hierarchical Data-Driven Descent for Efficient Optimal Deformation Estimation [link] [Proofs] Yuandong Tian, Srinivasa G. Narasimhan ICCV 2013 *(Marr Prize Honorable Mentions)*
	Integrating Perceptual Learning with External World Knowledge in a Simulated Student [link] Nan Li, Yuandong Tian, William W. Cohen, Ken Koedinger AIED 2013
	Exploring the Spatial Hierarchy of Mixture Models for Human Pose Estimation [link] [code] [website] Yuandong Tian, Larry Zitnick, Srinivasa G. Narasimhan ECCV 2012
	Learning from Crowds in the Presence of Schools of Thought [link] [code] [Slides] [Dataset] Yuandong Tian, Jun Zhu KDD 2012
	Depth from Optical Turbulence [link] [website] Yuandong Tian, Srinivasa G. Narasimhan, Alan J. Vannevel CVPR 2012
	A Combined Theory of Defocused Illumination and Global Light Transport [link] Mohit Gupta, Yuandong Tian, Srinivasa G. Narasimhan, Li Zhang IJCV 2011
	Globally Optimal Estimation of Nonrigid Image Distortion [link] Yuandong Tian, Srinivasa G. Narasimhan IJCV 2011
	Rectification and 3D reconstruction of Curved Document Images [link] [website] Yuandong Tian, Srinivasa G. Narasimhan CVPR 2011 *(Oral)*
	Local Isomorphism to Solve the Pre-image Problem in Kernel Methods [link] Dong Huang, Yuandong Tian, Fernando De la Torre CVPR 2011
	A Globally Optimal Data-Driven Approach for Image Distortion Estimation [link] [website] Yuandong Tian, Srinivasa G. Narasimhan CVPR 2010 *(Oral)*
	Seeing through water: Image restoration using model-based tracking [link] [website] Yuandong Tian, Srinivasa G. Narasimhan ICCV 2009
	(De) Focusing on Global Light Transport for Active Scene Recovery [link] Mohit Gupta, Yuandong Tian, Srinivasa G. Narasimhan, Li Zhang CVPR 2009 *(Oral)*

Brief Bio

News