Stability Analysis of Gradient-Based Neural Networks for Optimization Problems

Qiaoming Han*, Lizhi LIAO, Houduo Qi, Liqun Qi

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

34 Citations (Scopus)

Abstract

The paper introduces a new approach to analyze the stability of neural network models without using any Lyapunov function. With the new approach, we investigate the stability properties of the general gradient-based neural network model for optimization problems. Our discussion includes both isolated equilibrium points and connected equilibrium sets which could be unbounded. For a general optimization problem, if the objective function is bounded below and its gradient is Lipschitz continuous, we prove that (a) any trajectory of the gradient-based neural network converges to an equilibrium point, and (b) the Lyapunov stability is equivalent to the asymptotical stability in the gradient-based neural networks. For a convex optimization problem, under the same assumptions, we show that any trajectory of gradient-based neural networks will converge to an asymptotically stable equilibrium point of the neural networks. For a general nonlinear objective function, we propose a refined gradient-based neural network, whose trajectory with any arbitrary initial point will converge to an equilibrium point, which satisfies the second order necessary optimality conditions for optimization problems. Promising simulation results of a refined gradient-based neural network on some problems are also reported.

Original languageEnglish
Pages (from-to)363-381
Number of pages19
JournalJournal of Global Optimization
Volume19
Issue number4
DOIs
Publication statusPublished - Apr 2001

Scopus Subject Areas

  • Computer Science Applications
  • Management Science and Operations Research
  • Control and Optimization
  • Applied Mathematics

User-Defined Keywords

  • Asymptotic stability
  • Equilibrium point
  • Equilibrium set
  • Exponential stability
  • Gradient-based neural network

Fingerprint

Dive into the research topics of 'Stability Analysis of Gradient-Based Neural Networks for Optimization Problems'. Together they form a unique fingerprint.

Cite this