Iteratively reweighted least squares

The method of iteratively reweighted least squares (IRLS) is used to solve certain optimization problems with objective functions of the form of a p-norm, a r g m i n β ⁡ ∑ i = 1 n | y i − f i ( β ) | p , {\displaystyle \operatorname {arg\,min} _{\boldsymbol {\beta }}\sum _{i=1}^{n}{\big |}y_{i}-f_{i}({\boldsymbol {\beta }}){\big |}^{p},} by an iterative method in which each step involves solving a weighted least squares problem of the form β ( t + 1 ) = a r g m i n β ⁡ ∑ i = 1 n w i ( β ( t ) ) | y i − f i ( β ) | 2 . {\displaystyle {\boldsymbol {\beta }}^{(t+1)}=\operatorname {arg\,min} _{\boldsymbol {\beta }}\sum _{i=1}^{n}w_{i}({\boldsymbol {\beta }}^{(t)}){\big |}y_{i}-f_{i}({\boldsymbol {\beta }}){\big |}^{2}.} IRLS is used to find the maximum likelihood estimates of a generalized linear model, and in robust regression to find an M-estimator, as a way of mitigating the influence of outliers in an otherwise normally distributed data set, for example, by minimizing the least absolute errors rather than the least square errors.

Source: Wikipedia — Iteratively reweighted least squares (CC BY-SA 4.0)