Xinyun Chen (陈 昕昀) Email: xinyun.chen at berkeley dot edu   Hi! I am a Ph.D. candidate at UC Berkeley, advised by Prof. Dawn Song. I received my B.S. degree in Computer Science from ACM Honored Class, Shanghai Jiao Tong University. I was a research intern at DeepMind (2021), working with Yujia Li; a research intern at Google Brain (2019, 2020), working with Denny Zhou, Rishabh Singh, Petros Maniatis, and Charles Sutton; a research intern at Facebook AI Research (2018), working with Yuandong Tian; and a visiting student at National Institute of Informatics (2015), working with Prof. Helmut Prendinger. My research lies at the intersection of deep learning, programming languages, and security. Currently, I mainly focus on neural program synthesis and adversarial machine learning. I received the Facebook Fellowship in 2020 (blog spotlight).   Publications [Google Scholar] | Awards | Education | Talks | Services | Teaching | Misc.

I am on the job market, and I am looking for both academic and industrial job opportunities starting from Summer/Fall 2022. Feel free to connect!

Publications

• Competition-Level Code Generation with AlphaCode

  Yujia Li*, David Choi*, Junyoung Chung*, Nate Kushman*, Julian Schrittwieser*, Rémi Leblond*, Tom Eccles*, James Keeling*, Felix Gimeno*, Agustin Dal Lago*, Thomas Hubert*, Peter Choy*, Cyprien de Masson d'Autume*, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, Oriol Vinyals. (* Joint first authors).

  AlphaCode achieved an estimated rank within the top 54% of participants in programming competitions. Please check out the DeepMind blog post for more details.




• Perturbation Type Categorization for Multiple Adversarial Perturbation Robustness

  Pratyush Maini, Xinyun Chen, Bo Li, Dawn Song.

  Conference on Uncertainty in Artificial Intelligence (UAI), 2022.




• Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

  Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, Tom Goldstein.

  IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022.




• SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs

  Hongyu Ren, Hanjun Dai, Bo Dai, Xinyun Chen, Denny Zhou, Jure Leskovec, Dale Schuurmans.

  ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD), 2022.




• Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment

  Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver.

  Findings of Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL Findings), 2022.




• Learning Bounded Context-Free Grammar via LSTM and the Transformer: Difference and Explanations

  Hui Shi, Sicun Gao, Yuandong Tian, Xinyun Chen, Jishen Zhao.

  AAAI Conference on Artificial Intelligence (AAAI), 2022.




• Latent Execution for Neural Program Synthesis

  Xinyun Chen, Dawn Song, Yuandong Tian.

  Advances in Neural Information Processing Systems (NeurIPS), 2021.




• SpreadsheetCoder: Formula Prediction from Semi-structured Context

  Xinyun Chen, Petros Maniatis, Rishabh Singh, Charles Sutton, Hanjun Dai, Max Lin, Denny Zhou.

  International Conference on Machine Learning (ICML), 2021.

  Our work was released to support Google Sheets formula suggestions. Please check out our Google AI blog for more details.




• LEGO: Latent Execution-Guided Reasoning for Multi-Hop Question Answering on Knowledge Graphs

  Hongyu Ren, Hanjun Dai, Bo Dai, Xinyun Chen, Michihiro Yasunaga, Haitian Sun, Dale Schuurmans, Jure Leskovec, Denny Zhou.

  International Conference on Machine Learning (ICML), 2021.




• Learn-to-Share: A Hardware-friendly Transfer Learning Framework Exploiting Computation and Parameter Sharing

  Cheng Fu, Hanxian Huang, Xinyun Chen, Yuandong Tian, Jishen Zhao.

  International Conference on Machine Learning (ICML), 2021. (Long Talk)




• PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context

  Xinyun Chen, Linyuan Gong, Alvin Cheung, Dawn Song.

  Annual Meeting of the Association for Computational Linguistics (ACL), 2021.




• Towards Robustness of Text-to-SQL Models against Synonym Substitution

  Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R. Woodward, Jinxia Xie, Pengsheng Huang.

  Annual Meeting of the Association for Computational Linguistics (ACL), 2021.




• REFIT: a Unified Watermark Removal Framework for Deep Learning Systems with Limited Data

  Xinyun Chen*, Wenxiao Wang*, Chris Bender, Yiming Ding, Ruoxi Jia, Bo Li, Dawn Song. (* Equal contribution)

  ACM Asia Conference on Computer and Communications Security (AsiaCCS), 2021.




• Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization

  Yujian Gan, Xinyun Chen, Matthew Purver.

  Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021.




• Natural SQL: Making SQL Easier to Infer from Natural Language Specifications

  Yujian Gan, Xinyun Chen, Jinxia Xie, Matthew Purver, John R. Woodward, John Drake, Qiaofu Zhang.

  Findings of Conference on Empirical Methods in Natural Language Processing (EMNLP Findings), 2021.




• Understanding Robustness in Teacher-Student Setting: A New Perspective

  Zhuolin Yang*, Zhaoxi Chen, Tiffany (Tianhui) Cai, Xinyun Chen, Bo Li, Yuandong Tian*. (* Equal contribution)

  International Conference on Artificial Intelligence and Statistics (AISTATS), 2021.




• RobustART: Benchmarking Robustness on Architecture Design and Training Techniques

  Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xianglong Liu, Dawn Song, Alan Yuille, Philip H.S. Torr, Dacheng Tao.




• Compositional Generalization via Neural-Symbolic Stack Machines

  Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, Denny Zhou.

  Advances in Neural Information Processing Systems (NeurIPS), 2020.




• Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis

  Kavi Gupta, Peter Ebert Christensen*, Xinyun Chen*, Dawn Song. (* Equal contribution)

  Advances in Neural Information Processing Systems (NeurIPS), 2020.




• Spatiotemporal Attacks for Embodied Agents

  Aishan Liu, Tairan Huang, Xianglong Liu, Yitao Xu, Yuqing Ma, Xinyun Chen, Stephen Maybank, Dacheng Tao.

  European Conference on Computer Vision (ECCV), 2020.




• Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension

  Xinyun Chen, Chen Liang, Adams Wei Yu, Denny Zhou, Dawn Song, Quoc V. Le.

  International Conference on Learning Representations (ICLR), 2020. (Spotlight)




• Deep Symbolic Superoptimization Without Human Knowledge

  Hui Shi, Yang Zhang, Xinyun Chen, Yuandong Tian, Jishen Zhao.

  International Conference on Learning Representations (ICLR), 2020.




• Learning to Perform Local Rewriting for Combinatorial Optimization

  Xinyun Chen, Yuandong Tian.

  Advances in Neural Information Processing Systems (NeurIPS), 2019.




• Coda: An End-to-End Neural Program Decompiler

  Cheng Fu, Huili Chen, Haolan Liu, Xinyun Chen, Yuandong Tian, Farinaz Koushanfar, Jishen Zhao.

  Advances in Neural Information Processing Systems (NeurIPS), 2019.




• Execution-Guided Neural Program Synthesis

  Xinyun Chen, Chang Liu, Dawn Song.

  International Conference on Learning Representations (ICLR), 2019.




• Tree-to-tree Neural Networks for Program Translation

  Xinyun Chen, Chang Liu, Dawn Song.

  Advances in Neural Information Processing Systems (NeurIPS), 2018.

   




• Fooling Vision and Language Models Despite Localization and Attention Mechanism

  Xiaojun Xu, Xinyun Chen, Chang Liu, Anna Rohrbach, Trevor Darrell, Dawn Song.

  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018




• Towards Synthesizing Complex Programs from Input-Output Examples

  Xinyun Chen, Chang Liu, Dawn Song.

  International Conference on Learning Representations (ICLR), 2018.




• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

  Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, Dawn Song.

  Media coverage: Motherboard | The Register




• Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong

  Warren He, James Wei, Xinyun Chen, Nicholas Carlini, Dawn Song.

  USENIX Workshop on Offensive Technologies (WOOT), 2017.




• Delving into Transferable Adversarial Examples and Black-box Attacks

  Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song.

  International Conference on Learning Representations (ICLR), 2017.




• A General Retraining Framework for Scalable Adversarial Classification

  Bo Li, Yevgeniy Vorobeychik, Xinyun Chen.

  NeurIPS Workshop on Adversarial Training, 2016.




• Latent Attention For If-Then Program Synthesis

  Xinyun Chen, Chang Liu, Richard Shin, Dawn Song, Mingcheng Chen.

  Advances in Neural Information Processing Systems (NeurIPS), 2016.

Honors and Awards

• Rising Stars in Machine Learning, 2021.

• Rising Stars in EECS, 2021.

• Facebook Fellowship, 2020.

• Rising Stars in EECS, 2020.

• Departmental Fellowship of EECS, UC Berkeley, 2017.

• The Prize of Excellent Bachelor Thesis (top 1% in Shanghai Jiao Tong University), 2017.

• Outstanding Graduate of Shanghai Jiao Tong University, 2017.

• Gold Medal of Asia-Pacific Informatics Olympiad in China District, 2012.

• Silver Medal of Chinese Team Selection Contest, 2012.

Education

• Aug 2017 - Present: Ph.D., Computer Science, UC Berkeley. Advisor: Prof. Dawn Song.

• Sep 2013 - Jun 2017: B.S., ACM Honored Class, Shanghai Jiao Tong University. Rank: 1/30.

Talks

• April 2022: Learning to Model Structures and Execution for Program Synthesis, ICLR Workshop on Deep Learning For Code.

• April 2022: Program Synthesis via Learning, guest lecture in CS188: Introduction to Artificial Intelligence.

• December 2021: Learning-Based Program Synthesis, Microsoft New England Machine Learning Seminar.

• December 2021: Learning-Based Program Synthesis, Microsoft Research Asia.

• November 2021: SpreadsheetCoder: Formula Prediction from Semi-structured Context, GRID, Iceland.

• November 2021: Deep Learning for Program Synthesis: Towards Human-like Reasoning, ML Rising Stars Series, University of Maryland.

• October 2021: Neural Program Synthesis for Language Understanding in the Wild, Neurosym Webinar Series.

• October 2021: Deep Learning for Program Synthesis: Towards Human-like Reasoning, Stanford Software Research Lunch.

• September 2021: Deep Learning for Program Synthesis: Towards Human-like Reasoning, Facebook Fellowship Summit.

• August 2021: Deep Learning for Program Synthesis: Towards Human-like Reasoning, University of Southern California.

• June 2021: Adversarial Attacks in Computer Vision: An Overview, CVPR Tutorial on Adversarial Machine Learning in Computer Vision.

• April 2021: Deep Learning for Program Synthesis, SJSU SCE Spark Tech Conference.

• March 2021: Neural-Symbolic Reasoning for Language Understanding, WSDM Workshop on Machine Reasoning.

• December 2020: Deep Learning for Program Synthesis from Input-Output Examples, NeurIPS Workshop on Computer-Assisted Programming.

• June 2020: Neural Program Synthesis for Navigation and Language Understanding, CVPR Tutorial on Neuro-Symbolic Visual Reasoning and Program Synthesis.

• April 2020: Learning to Perform Local Rewriting for Combinatorial Optimization, Google.

• February 2020: Neural-Symbolic Reader for Reading Comprehension, Google, Mountain View.

• January 2020: Learning to Perform Local Rewriting in Discrete Search Spaces, Alibaba Group, Sunnyvale.

• October 2019: Neural Program Synthesis from Natural Language Specification, Open Virtual Assistant Lab, Stanford University.

• February 2019: Neural Program Synthesis from Input-Output Examples, UC San Diego.

• November 2018: Towards Synthesizing Complex Programs from Input-Output Examples, guest lecture in CS294-157: Deep Learning and Program Synthesis.

• October 2018: Neural Program Synthesis from Input-Output Examples, Facebook Big Code Summit.

• May 2018: Deep Learning for Program Synthesis, guest lecture in CS379C: Computational Models of the Neocortex, Stanford University.

Services

• Co-organizer of the Workshop on Structured and Unstructured Knowledge Integration at NAACL 2022.

• Co-organizer of the Workshop on The Art of Robustness: Devil and Angel in Adversarial Machine Learning at CVPR 2022.

• Co-organizer of the Workshop on Practical Deep Learning in the Wild at AAAI 2022.

• Co-organizer of the Workshop on Security and Safety in Machine Learning Systems at ICLR 2021.

• Co-organizer of the Workshop on Adversarial Machine Learning in Real-World Computer Vision Systems and Online Challenges at CVPR 2021.

• Co-organizer of the Tutorial on Adversarial Machine Learning in Computer Vision at CVPR 2021.

• Co-organizer of the Workshop on Socially Responsible Machine Learning at ICML 2021.

• Co-organizer of the Workshop on Adversarial Robustness in the Real World at ICCV 2021.

• Co-organizer of the Workshop on Adversarial Learning for Multimedia at ACM Multimedia 2021.

• Co-organizer of the Workshop on Adversarial Machine Learning in Computer Vision at CVPR 2020.

• Program Committee / Reviewer of: IJCAI (senior PC in 2021), NeurIPS (Outstanding Reviewer Award in 2020 and 2021), ICML (expert reviewer in 2021), ICLR, AAAI, ACL, EMNLP, NAACL, ICCV, ECCV, TPAMI, IJCV, TIP, TIFS, TDSC.

Teaching

• Graduate Student Instructor of CS 285: Deep Reinforcement Learning, Fall 2021, UC Berkeley.

• Graduate Student Instructor of CS 188: Introduction to Artificial Intelligence, Spring 2021, UC Berkeley.

Miscellaneous

In my spare time, I enjoy listening to all kinds of music. I have played the piano since kindergarten, and received the highest-level certificates of piano and music theory in China. I used to play some other instruments, including recorder and harmonica.