The ill-conditioning issue in the penalty function method

The penalty function method has been widely used in many constrained optimization problems. However, a companion of such method is the ill-conditioning problem in the corresponding Hessian matrix. In this paper, a thorough investigation is provided to address this ill-conditioning problem. Our study reveals that this ill-conditioning is mathematically tractable. Furthermore, the resulting ill-conditioned system can be converted into a well-conditioned one. Consequently, this ill-conditioning problem can be overcome. Some theoretical results will be also provided. Our preliminary numerical tests have confirmed our findings.