Session: Algebraic Methods in Optimization (Part 1)
Chair: Shixuan Zhang
Cluster: Conic and Semidefinite Optimization
Talk 1: Spurious minima in nonconvex sum-of-square optimization via syzygies
Speaker: Shixuan Zhang
Abstract: We study spurious local minima in a nonconvex low-rank formulation of sum-of-squares optimization on a real variety X. We reformulate the problem of finding a spurious local minimum or stationary points in terms of syzygies of the underlying linear series, and also bring in topological tools to study this problem. When the variety X is of minimal degree, there exist spurious second-order stationary points if and only if both the dimension and the codimension of the variety are greater than one, answering a question by Legat, Yuan, and Parrilo. Moreover, for surfaces of minimal degree, we provide sufficient conditions to exclude points from being spurious local minima. In particular, we characterize all spurious second-order stationary points on the Veronese surface, corresponding to ternary quartics, which either have finitely many Gram matrices or can be written as a binary quartic, complementing work by Scheiderer on decompositions of ternary quartics as a sum of three squares. For general varieties of higher degree, we give examples and characterizations of spurious local minima, and provide extensive numerical experiments demonstrating the effectiveness of the low-rank formulation.
Talk 2: Optimizing rational neural networks with trainable parameters
Speaker: Jiayi Li
Abstract: In the modern practice of deep learning, the performance of an artificial neural network is heavily dependent on the architecture and the choice of non-linear activations between each layer. We consider feedforward neural networks with activations being rational functions, whose coefficients are trainable. We characterize the critical points and study the geometry of the optimization landscape. This is joint work with Angelica Torres and Guido Montufar.
Talk 3: Pythagoras Numbers of Ternary Forms
Speaker: Alex Dunbar
Abstract: The representation of nonnegative polynomials as sums of squares is a fundamental idea in polynomial optimization. We study the \emph{Pythagoras number} $py(3,2d)$ of real ternary forms, defined as the minimal number $r$ such that every degree $2d$ form which is a sum of squares can be written as the sum of at most $r$ squares of degree $d$ forms. It is well-known that $d+1\leq py(3,2d)\leq d+2$. We show that $py(3,2d) = d+1$ for $2d = 8,10,12$. The main technical tool is Diesel's characterization of height 3 Gorenstein algebras. Based on joint work with Greg Blekherman and Rainer Sinn
