ほくそ笑む

R言語と統計解析について

統計的機械学習は単純な最適化問題ではない

It might seem that minimization of  K_n(w) is equivalent to minimization of  K(w). If these two minimization problems were equivalent, then maximization of  L_n(w) would be the best method in statistical estimation. However, minimization and expectation cannot be commutative.

 E[\min_w K_n(w) ] \neq \min_w E[K_n(w)] = \min_w K(w)

Hence maximization of  L_n(w) does not mean minimization of  K(w). This is the basic reason why statistical learning does not result in a simple optimization problem.

p.7

Algebraic Geometry and Statistical Learning Theory (Cambridge Monographs on Applied and Computational Mathematics)

Algebraic Geometry and Statistical Learning Theory (Cambridge Monographs on Applied and Computational Mathematics)