Session: AI Meets Optimization (Part 2)
Chair: Wotao Yin
Cluster: Optimization for Emerging Technologies (LLMs, Quantum Computing, ...)
Talk 1: Scientific Machine Learning for Optimization and Control
Speaker: Jan Drgona
Abstract: This talk presents a scientific machine learning perspective (SciML) on modeling, optimization, and control. Specifically, we will discuss the opportunity to develop a unified SciML framework for modeling dynamical systems, learning to optimize, and learning to control methods. We demonstrate the application of these emerging SciML methods in a range of engineering case studies, including modeling of networked dynamical systems, building control, and dynamic economic dispatch problem in power systems. Furthermore, we will introduce the NeuroMANCER open-source library facilitating the implementation and prototyping of diverse SciML methods for a broad range of application problems.
Talk 2: What Can and Cannot Be Learned by Implicit Models?
Speaker: Jialin Liu
Abstract: Implicit models represent an efficient class of deep neural networks. Unlike explicit models, which are trained to map inputs to outputs in an end-to-end manner, implicit models are designed so that their fixed points align with the data. Their main advantage, as often noted in the literature, is memory efficiency—they store only a single layer but can function at any depth. However, this talk focuses on another important but overlooked benefit: greater expressive power. Specifically, we identify a class of target mappings that implicit models can represent, whereas explicit models cannot. Conversely, we also establish that implicit models are inherently limited to this class of mappings. To illustrate the practical relevance of these findings, we present case studies across imaging, scientific computing, and operations research, demonstrating the broad applicability of these target mappings.
Talk 3: Learning-to-Optimize via Implicit Networks
Speaker: Samy Wu Fung
Abstract: Learning-to-Optimize (or L2O) is an emerging approach where machine learning is used to learn an optimization algorithm. It automates the design of an optimization method based on its performance on a set of training problems. However, learning optimization algorithms in an end-to-end fashion can be challenging due to their asymptotic nature. This talk discusses a class of network architectures, called implicit networks, whose outputs are defined by a fixed point (or optimality) condition, which makes them naturally suited for L2O. We will cover how to train and design these networks efficiently.
