Convergence analysis of the Levenberg-Marquardt method

Xin Long Luo; Lizhi LIAO; Hon Wah TAM

doi:10.1080/10556780601079233

Convergence analysis of the Levenberg-Marquardt method

Xin Long Luo^*, Lizhi LIAO, Hon Wah TAM

^*Corresponding author for this work

Research output: Contribution to journal › Journal article › peer-review

29 Citations (Scopus)

Abstract

The Levenberg-Marquardt method is a popular method for both optimization problems and equilibrium problems in dynamical systems. In this article, we study the convergence properties of the Levenberg-Marquardt method with the standard matrix update scheme. In our global convergence proof, we relax the condition that update matrices be bounded, and only require that their norms increase at most linearly. Furthermore, we analyze its local convergence for the uniformly convex function. In this case, the Levenberg-Marquardt method has superlinear convergence, and the initial matrix can be chosen arbitrarily for the Broyden-Fletcher-Goldfarb-Shanno (BFGS) formula.

Original language	English
Pages (from-to)	659-678
Number of pages	20
Journal	Optimization Methods and Software
Volume	22
Issue number	4
DOIs	https://doi.org/10.1080/10556780601079233
Publication status	Published - Aug 2007

Scopus Subject Areas

Software
Control and Optimization
Applied Mathematics

User-Defined Keywords

Levenberg-Marquardt method
Ordinary differential equations
Quasi-Newton method
Trust region method
Unconstrained optimization

Access to Document

10.1080/10556780601079233

Cite this

@article{e80ae8b2dc2e474eb665899ccd27fd58,

title = "Convergence analysis of the Levenberg-Marquardt method",

abstract = "The Levenberg-Marquardt method is a popular method for both optimization problems and equilibrium problems in dynamical systems. In this article, we study the convergence properties of the Levenberg-Marquardt method with the standard matrix update scheme. In our global convergence proof, we relax the condition that update matrices be bounded, and only require that their norms increase at most linearly. Furthermore, we analyze its local convergence for the uniformly convex function. In this case, the Levenberg-Marquardt method has superlinear convergence, and the initial matrix can be chosen arbitrarily for the Broyden-Fletcher-Goldfarb-Shanno (BFGS) formula.",

keywords = "Levenberg-Marquardt method, Ordinary differential equations, Quasi-Newton method, Trust region method, Unconstrained optimization",

author = "Luo, {Xin Long} and Lizhi LIAO and TAM, {Hon Wah}",

note = "Funding Information: The authors are very grateful to Prof. Ya-Xian Yuan, the Associate Editor, and two anonymous referees for their constructive and helpful suggestions on earlier versions of this paper. This research was supported in part by FRG grants from Hong Kong Baptist University and grants from the Research Grant Council of Hong Kong.",

year = "2007",

month = aug,

doi = "10.1080/10556780601079233",

language = "English",

volume = "22",

pages = "659--678",

journal = "Optimization Methods and Software",

issn = "1055-6788",

publisher = "Taylor and Francis",

number = "4",

}

TY - JOUR

T1 - Convergence analysis of the Levenberg-Marquardt method

AU - Luo, Xin Long

AU - LIAO, Lizhi

AU - TAM, Hon Wah

N1 - Funding Information: The authors are very grateful to Prof. Ya-Xian Yuan, the Associate Editor, and two anonymous referees for their constructive and helpful suggestions on earlier versions of this paper. This research was supported in part by FRG grants from Hong Kong Baptist University and grants from the Research Grant Council of Hong Kong.

PY - 2007/8

Y1 - 2007/8

N2 - The Levenberg-Marquardt method is a popular method for both optimization problems and equilibrium problems in dynamical systems. In this article, we study the convergence properties of the Levenberg-Marquardt method with the standard matrix update scheme. In our global convergence proof, we relax the condition that update matrices be bounded, and only require that their norms increase at most linearly. Furthermore, we analyze its local convergence for the uniformly convex function. In this case, the Levenberg-Marquardt method has superlinear convergence, and the initial matrix can be chosen arbitrarily for the Broyden-Fletcher-Goldfarb-Shanno (BFGS) formula.

AB - The Levenberg-Marquardt method is a popular method for both optimization problems and equilibrium problems in dynamical systems. In this article, we study the convergence properties of the Levenberg-Marquardt method with the standard matrix update scheme. In our global convergence proof, we relax the condition that update matrices be bounded, and only require that their norms increase at most linearly. Furthermore, we analyze its local convergence for the uniformly convex function. In this case, the Levenberg-Marquardt method has superlinear convergence, and the initial matrix can be chosen arbitrarily for the Broyden-Fletcher-Goldfarb-Shanno (BFGS) formula.

KW - Levenberg-Marquardt method

KW - Ordinary differential equations

KW - Quasi-Newton method

KW - Trust region method

KW - Unconstrained optimization

UR - http://www.scopus.com/inward/record.url?scp=34147222084&partnerID=8YFLogxK

U2 - 10.1080/10556780601079233

DO - 10.1080/10556780601079233

M3 - Journal article

AN - SCOPUS:34147222084

SN - 1055-6788

VL - 22

SP - 659

EP - 678

JO - Optimization Methods and Software

JF - Optimization Methods and Software

IS - 4

ER -

Convergence analysis of the Levenberg-Marquardt method

Abstract

Scopus Subject Areas

User-Defined Keywords

Access to Document

Other files and links

Fingerprint

Cite this