Solution of a linear system
Solution of a linear system [ edit ] The steepest descent algorithm applied to the Wiener filter [11] Gradient descent can be used to solve a system of linear equations � � − � = 0 reformulated as a quadratic minimization problem. If the system matrix � is real symmetric and positive-definite , an objective function is defined as the quadratic function, with minimization of � ( � ) = � � � � − 2 � � � , so that ∇ � ( � ) = 2 ( � � − � ) . For a general real matrix � , linear least squares define � ( � ) = ‖ � � − � ‖ 2 . In traditional linear least squares for real � and � the Euclidean norm is used, in which case ∇ � ( � ) = 2 � � ( � � − � ) . The line search minimization, finding the locally optimal step size � on every iteration, can be performed analytically for quadratic functions, and explicit formulas for the locally optimal � are known. [5] [12] For example, for real symmetric and positive-definite matrix � , a simple algorithm can be as follows, [5