Ensembles of Networks Produced from Neural Architecture Search

presented by Emily Herron form the University of Tennessee.

Abstract

Neural architecture search (NAS) is a popular topic at the intersection of deep learning and high performance computing. NAS focuses on optimizing the architecture of neural networks along with their hyperparameters in order to produce networks with superior performance. Much of the focus has been on how to produce a single best network to solve a machine learning problem, but as NAS methods produce many networks that work very well, this affords the opportunity to ensemble these networks to produce an improved result. Additionally, the diversity of network structures produced by NAS drives a natural bias towards diversity of predictions produced by the individual networks. This results in an improved ensemble over simply creating an ensemble that contains duplicates of the best network architecture retrained to have unique weights.

Slides

Paper pre-print

About the Speaker

Emily Herron is a third year PhD Student in the Bredesen Center Data Science and Engineering Program at the University of Tennessee. This is a joint program between the University of Tennessee and Oak Ridge National Laboratory. She holds a B.S. in Computational Science from Mercer University. Her research utilizes high performance computing, with a focus on evolutionary algorithms and neural architecture search.