Implementing the Steepest Descent Algorithm in Python from Scratch

Optimization is the process of finding the set of variables x that minimize or maximize an objective function f(x). Let us now introduce the steepest descent algorithm and implement it from scratch. Our goal is to solve the optimization problem and find the minimum [4.5, 2.3]. To solve the optimization problem minₓ f(x), we start by positioning ourselves in some point in the coordinate space. This approach requires a proper selection of the step size α and the search direction p.