Enhanced Conjugate Gradient Method for Unconstrained Optimization and its Application in Neural Networks

Abstract

In this study, we present a novel conjugate gradient method specifically designed for addressing with unconstrained optimization problems. Traditional conjugate gradient methods have shown effectiveness in solving optimization problems, but they may encounter challenges when dealing with unconstrained problems. Our method addresses this issue by introducing modifications that enhance its performance in the unconstrained setting. We demonstrate that, under certain conditions, our method satisfies both the descent and the sufficient descent criteria and establishes global convergence, ensuring progress towards the optimal solution at each iteration. Moreover, we establish the global convergence of our method, providing confidence in its ability to find the global optimum. To showcase the practical applicability of our approach, we apply this novel method to a dataset, applying a feed-forward neural network value estimation for continuous trigonometric function value estimation. To evaluate the efficiency and effectiveness of our modified approach, we conducted numerical experiments on a set of well-known test functions.  These experiments reveal that our algorithm significantly reduces computational time due to its faster convergence rates and increased speed in directional minimization. These compelling results highlight the advantages of our approach over traditional conjugate gradient methods in the context of unconstrained optimization problems.