Xinyun Chen (陈 昕昀) Email: xinyunchen at google dot com   I am a senior research scientist at Google Brain. My research interests are neural program synthesis and adversarial machine learning. I obtained my Ph.D. degree in Computer Science from UC Berkeley in 2022, and my B.S. degree in Computer Science from ACM Honored Class, Shanghai Jiao Tong University. I also spent time at Facebook AI Research, DeepMind, and National Institute of Informatics.   Publications [Google Scholar] | Awards | Education | Talks | Services | Teaching | Misc.

Publications

• Teaching Large Language Models to Self-Debug

  Xinyun Chen, Maxwell Lin, Nathanael Schärli, Denny Zhou.




• Large Language Models as Optimizers

  Chengrun Yang*, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, Xinyun Chen*. (* Equal contribution)




• Large Language Models Can Be Easily Distracted by Irrelevant Context

  Freda Shi*, Xinyun Chen*, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, Denny Zhou. (* Equal contribution)

  International Conference on Machine Learning (ICML), 2023.




• Large Language Models as Tool Makers

  Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, Denny Zhou.




• Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models

  Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Albert Webson, Yunxuan Li, Vincent Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, Denny Zhou.




• Symbol tuning improves in-context learning in language models

  Jerry Wei, Le Hou, Andrew Lampinen, Xiangning Chen, Da Huang, Yi Tay, Xinyun Chen, Yifeng Lu, Denny Zhou, Tengyu Ma, Quoc V. Le.




• Larger Language Models Do In-Context Learning Differently

  Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, Tengyu Ma.




• Compositional Semantic Parsing with Large Language Models

  Andrew Drozdov*, Nathanael Schärli*, Ekin Akyürek, Nathan Scales, Xinying Song, Xinyun Chen, Olivier Bousquet, Denny Zhou. (* Equal contribution)

  International Conference on Learning Representations (ICLR), 2023.




• Towards Defending Multiple Lp-norm Bounded Adversarial Perturbations via Gated Batch Normalization

  Aishan Liu, Shiyu Tang, Xinyun Chen, Lei Huang, Haotong Qin, Xianglong Liu, Dacheng Tao.

  International Journal of Computer Vision (IJCV), 2023.




• Re-appraising the Schema Linking for Text-to-SQL

  Yujian Gan, Xinyun Chen, Matthew Purver.

  Findings of the Association for Computational Linguistics (ACL Findings), 2023.




• DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection

  Yizheng Chen, Zhoujie Ding, Lamya Alowain, Xinyun Chen, David Wagner.

  nternational Symposium on Research in Attacks, Intrusions and Defenses (RAID), 2023.




• BioCoder: A Benchmark for Bioinformatics Code Generation with Contextual Pragmatic Knowledge

  Xiangru Tang*, Bill Qian*, Rick Gao*, Jiakang Chen*, Xinyun Chen, Mark Gerstein. (* Equal contribution)




• Competition-Level Code Generation with AlphaCode

  Yujia Li*, David Choi*, Junyoung Chung*, Nate Kushman*, Julian Schrittwieser*, Rémi Leblond*, Tom Eccles*, James Keeling*, Felix Gimeno*, Agustin Dal Lago*, Thomas Hubert*, Peter Choy*, Cyprien de Masson d'Autume*, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, Oriol Vinyals. (* Joint first authors).

  Science Magazine, 2022.

  Featured as the front cover in Science Magazine. AlphaCode achieved an estimated rank within the top 54% of participants in programming competitions. Please check out the DeepMind blog post for more details.




• Scaling Instruction-Finetuned Language Models

  Hyung Won Chung*, Le Hou*, Shayne Longpre*, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, Jason Wei*. (* Equal contribution)




• Learning-Based Program Synthesis: Towards Synthesizing Complex Programs from Multi-Modal Specifications in the Wild

  Xinyun Chen.

  Ph.D. Dissertation, 2022.




• Benchmarking Language Models for Code Syntax Understanding

  Da Shen, Xinyun Chen, Chenguang Wang, Koushik Sen, Dawn Song.

  Findings of Conference on Empirical Methods in Natural Language Processing (EMNLP Findings), 2022.




• Perturbation Type Categorization for Multiple Adversarial Perturbation Robustness

  Pratyush Maini, Xinyun Chen, Bo Li, Dawn Song.

  Conference on Uncertainty in Artificial Intelligence (UAI), 2022.




• Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

  Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, Tom Goldstein.

  IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022.




• SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs

  Hongyu Ren, Hanjun Dai, Bo Dai, Xinyun Chen, Denny Zhou, Jure Leskovec, Dale Schuurmans.

  ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD), 2022.




• Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment

  Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver.

  Findings of Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL Findings), 2022.




• Learning Bounded Context-Free Grammar via LSTM and the Transformer: Difference and Explanations

  Hui Shi, Sicun Gao, Yuandong Tian, Xinyun Chen, Jishen Zhao.

  AAAI Conference on Artificial Intelligence (AAAI), 2022.




• Latent Execution for Neural Program Synthesis

  Xinyun Chen, Dawn Song, Yuandong Tian.

  Advances in Neural Information Processing Systems (NeurIPS), 2021.




• SpreadsheetCoder: Formula Prediction from Semi-structured Context

  Xinyun Chen, Petros Maniatis, Rishabh Singh, Charles Sutton, Hanjun Dai, Max Lin, Denny Zhou.

  International Conference on Machine Learning (ICML), 2021.

  Our work was released to support Google Sheets formula suggestions. Please check out our Google AI blog for more details.




• LEGO: Latent Execution-Guided Reasoning for Multi-Hop Question Answering on Knowledge Graphs

  Hongyu Ren, Hanjun Dai, Bo Dai, Xinyun Chen, Michihiro Yasunaga, Haitian Sun, Dale Schuurmans, Jure Leskovec, Denny Zhou.

  International Conference on Machine Learning (ICML), 2021.




• Learn-to-Share: A Hardware-friendly Transfer Learning Framework Exploiting Computation and Parameter Sharing

  Cheng Fu, Hanxian Huang, Xinyun Chen, Yuandong Tian, Jishen Zhao.

  International Conference on Machine Learning (ICML), 2021. (Long Talk)




• PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context

  Xinyun Chen, Linyuan Gong, Alvin Cheung, Dawn Song.

  Annual Meeting of the Association for Computational Linguistics (ACL), 2021.




• Towards Robustness of Text-to-SQL Models against Synonym Substitution

  Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R. Woodward, Jinxia Xie, Pengsheng Huang.

  Annual Meeting of the Association for Computational Linguistics (ACL), 2021.




• REFIT: a Unified Watermark Removal Framework for Deep Learning Systems with Limited Data

  Xinyun Chen*, Wenxiao Wang*, Chris Bender, Yiming Ding, Ruoxi Jia, Bo Li, Dawn Song. (* Equal contribution)

  ACM Asia Conference on Computer and Communications Security (AsiaCCS), 2021.




• Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization

  Yujian Gan, Xinyun Chen, Matthew Purver.

  Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021.




• Natural SQL: Making SQL Easier to Infer from Natural Language Specifications

  Yujian Gan, Xinyun Chen, Jinxia Xie, Matthew Purver, John R. Woodward, John Drake, Qiaofu Zhang.

  Findings of Conference on Empirical Methods in Natural Language Processing (EMNLP Findings), 2021.




• Understanding Robustness in Teacher-Student Setting: A New Perspective

  Zhuolin Yang*, Zhaoxi Chen, Tiffany (Tianhui) Cai, Xinyun Chen, Bo Li, Yuandong Tian*. (* Equal contribution)

  International Conference on Artificial Intelligence and Statistics (AISTATS), 2021.




• RobustART: Benchmarking Robustness on Architecture Design and Training Techniques

  Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xianglong Liu, Dawn Song, Alan Yuille, Philip H.S. Torr, Dacheng Tao.




• Compositional Generalization via Neural-Symbolic Stack Machines

  Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, Denny Zhou.

  Advances in Neural Information Processing Systems (NeurIPS), 2020.




• Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis

  Kavi Gupta, Peter Ebert Christensen*, Xinyun Chen*, Dawn Song. (* Equal contribution)

  Advances in Neural Information Processing Systems (NeurIPS), 2020.




• Spatiotemporal Attacks for Embodied Agents

  Aishan Liu, Tairan Huang, Xianglong Liu, Yitao Xu, Yuqing Ma, Xinyun Chen, Stephen Maybank, Dacheng Tao.

  European Conference on Computer Vision (ECCV), 2020.




• Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension

  Xinyun Chen, Chen Liang, Adams Wei Yu, Denny Zhou, Dawn Song, Quoc V. Le.

  International Conference on Learning Representations (ICLR), 2020. (Spotlight)




• Deep Symbolic Superoptimization Without Human Knowledge

  Hui Shi, Yang Zhang, Xinyun Chen, Yuandong Tian, Jishen Zhao.

  International Conference on Learning Representations (ICLR), 2020.




• Learning to Perform Local Rewriting for Combinatorial Optimization

  Xinyun Chen, Yuandong Tian.

  Advances in Neural Information Processing Systems (NeurIPS), 2019.




• Coda: An End-to-End Neural Program Decompiler

  Cheng Fu, Huili Chen, Haolan Liu, Xinyun Chen, Yuandong Tian, Farinaz Koushanfar, Jishen Zhao.

  Advances in Neural Information Processing Systems (NeurIPS), 2019.




• Execution-Guided Neural Program Synthesis

  Xinyun Chen, Chang Liu, Dawn Song.

  International Conference on Learning Representations (ICLR), 2019.




• Tree-to-tree Neural Networks for Program Translation

  Xinyun Chen, Chang Liu, Dawn Song.

  Advances in Neural Information Processing Systems (NeurIPS), 2018.

   




• Fooling Vision and Language Models Despite Localization and Attention Mechanism

  Xiaojun Xu, Xinyun Chen, Chang Liu, Anna Rohrbach, Trevor Darrell, Dawn Song.

  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018




• Towards Synthesizing Complex Programs from Input-Output Examples

  Xinyun Chen, Chang Liu, Dawn Song.

  International Conference on Learning Representations (ICLR), 2018.




• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

  Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, Dawn Song.

  Media coverage: Motherboard | The Register




• Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong

  Warren He, James Wei, Xinyun Chen, Nicholas Carlini, Dawn Song.

  USENIX Workshop on Offensive Technologies (WOOT), 2017.




• Delving into Transferable Adversarial Examples and Black-box Attacks

  Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song.

  International Conference on Learning Representations (ICLR), 2017.




• A General Retraining Framework for Scalable Adversarial Classification

  Bo Li, Yevgeniy Vorobeychik, Xinyun Chen.

  NeurIPS Workshop on Adversarial Training, 2016.




• Latent Attention For If-Then Program Synthesis

  Xinyun Chen, Chang Liu, Richard Shin, Dawn Song, Mingcheng Chen.

  Advances in Neural Information Processing Systems (NeurIPS), 2016.

Honors and Awards

• Rising Stars in Machine Learning, 2021.

• Rising Stars in EECS, 2021.

• Facebook Fellowship, 2020 (blog spotlight).

• Rising Stars in EECS, 2020.

• Departmental Fellowship of EECS, UC Berkeley, 2017.

• The Prize of Excellent Bachelor Thesis (top 1% in Shanghai Jiao Tong University), 2017.

• Outstanding Graduate of Shanghai Jiao Tong University, 2017.

• Gold Medal of Asia-Pacific Informatics Olympiad in China District, 2012.

• Silver Medal of Chinese Team Selection Contest, 2012.

Education

• Aug 2017 - May 2022: Ph.D., Computer Science, UC Berkeley. Advisor: Prof. Dawn Song.

• Sep 2013 - Jun 2017: B.S., ACM Honored Class, Shanghai Jiao Tong University. Rank: 1/30.

Talks

• August 2023: Leveraging Execution Feedback for Program Synthesis with Large Language Models, LLMs for Code Seminar.

• April 2023: Adversarial Learning Meets Large Language Models, SAO seminar, CSIRO.

• April 2023: Language Models for Program Synthesis, guest lecture in CPSC 663: Deep Learning Theory and Applications, Yale University.

• February 2023: Learning-Based Program Synthesis, REWORK Deep Learning Summit.

• December 2022: Program Synthesis from Semi-Structured Context, NeurIPS Workshop on Table Representation Learning.

• November 2022: Learning-Based Program Synthesis, guest lecture in CSE 527A: Natural Language Processing, Washington University in St. Louis.

• November 2022: Learning-Based Program Synthesis, Amazon AWS AI Research.

• October 2022: Learning-Based Program Synthesis, guest lecture in CMSC 828W: Foundations of Deep Learning, University of Maryland.

• September 2022: Learning to Model Structures and Execution for Program Synthesis, Language Technologies Institute Topical Seminar, Carnegie Mellon University.

• April 2022: Learning to Model Structures and Execution for Program Synthesis, ICLR Workshop on Deep Learning For Code.

• April 2022: Program Synthesis via Learning, guest lecture in CS188: Introduction to Artificial Intelligence.

• December 2021: Learning-Based Program Synthesis, Microsoft New England Machine Learning Seminar.

• December 2021: Learning-Based Program Synthesis, Microsoft Research Asia.

• November 2021: Deep Learning for Program Synthesis: Towards Human-like Reasoning, ML Rising Stars Series, University of Maryland.

• October 2021: Neural Program Synthesis for Language Understanding in the Wild, Neurosym Webinar Series.

• October 2021: Deep Learning for Program Synthesis: Towards Human-like Reasoning, Stanford Software Research Lunch.

• September 2021: Deep Learning for Program Synthesis: Towards Human-like Reasoning, Facebook Fellowship Summit.

• August 2021: Deep Learning for Program Synthesis: Towards Human-like Reasoning, University of Southern California.

• June 2021: Adversarial Attacks in Computer Vision: An Overview, CVPR Tutorial on Adversarial Machine Learning in Computer Vision.

• April 2021: Deep Learning for Program Synthesis, SJSU SCE Spark Tech Conference.

• March 2021: Neural-Symbolic Reasoning for Language Understanding, WSDM Workshop on Machine Reasoning.

• December 2020: Deep Learning for Program Synthesis from Input-Output Examples, NeurIPS Workshop on Computer-Assisted Programming.

• June 2020: Neural Program Synthesis for Navigation and Language Understanding, CVPR Tutorial on Neuro-Symbolic Visual Reasoning and Program Synthesis.

• October 2019: Neural Program Synthesis from Natural Language Specification, Open Virtual Assistant Lab, Stanford University.

• February 2019: Neural Program Synthesis from Input-Output Examples, UC San Diego.

• November 2018: Towards Synthesizing Complex Programs from Input-Output Examples, guest lecture in CS294-157: Deep Learning and Program Synthesis.

• October 2018: Neural Program Synthesis from Input-Output Examples, Facebook Big Code Summit.

• May 2018: Deep Learning for Program Synthesis, guest lecture in CS379C: Computational Models of the Neocortex, Stanford University.

Services

• Co-chair of AISec Workshop at CCS 2023.

• Co-chair of AISec Workshop at CCS 2022.

• Co-organizer of the Workshop on Adversarial Machine Learning on Computer Vision: Art of Robustness at CVPR 2023.

• Co-organizer of the Workshop on Structured and Unstructured Knowledge Integration at NAACL 2022.

• Co-organizer of the Workshop on The Art of Robustness: Devil and Angel in Adversarial Machine Learning at CVPR 2022.

• Co-organizer of the Workshop on Workshop on Adversarial Robustness in the Real World at ECCV 2022.

• Co-organizer of the Workshop on Practical Deep Learning in the Wild at AAAI 2022.

• Co-organizer of the Workshop on Security and Safety in Machine Learning Systems at ICLR 2021.

• Co-organizer of the Workshop on Adversarial Machine Learning in Real-World Computer Vision Systems and Online Challenges at CVPR 2021.

• Co-organizer of the Tutorial on Adversarial Machine Learning in Computer Vision at CVPR 2021.

• Co-organizer of the Workshop on Socially Responsible Machine Learning at ICML 2021.

• Co-organizer of the Workshop on Adversarial Robustness in the Real World at ICCV 2021.

• Co-organizer of the Workshop on Adversarial Learning for Multimedia at ACM Multimedia 2021.

• Co-organizer of the Workshop on Adversarial Machine Learning in Computer Vision at CVPR 2020.

• Program Committee / Reviewer of: NeurIPS (Outstanding Reviewer Award in 2020 and 2021), ICLR (Notable Reviewer Award in 2023), IJCAI (senior PC in 2021), ICML (expert reviewer in 2021), AAAI, ACL, EMNLP, NAACL, ICCV, ECCV, TPAMI, IJCV, TIP, TIFS, TDSC.

Teaching

• Graduate Student Instructor of CS 285: Deep Reinforcement Learning, Fall 2021, UC Berkeley.

• Graduate Student Instructor of CS 188: Introduction to Artificial Intelligence, Spring 2021, UC Berkeley.

Miscellaneous

In my spare time, I enjoy listening to all kinds of music. I have played the piano since kindergarten, and received the highest-level certificates of piano and music theory in China. I used to play some other instruments, including recorder and harmonica.