Least Squares

Because of the connection to the mean, and other nice properties, the third choice is very commonly used.

$$\min_f \sum_{i=1}^n (y_i-f(x_i))^2$$

In the constant case, we want $\min_c E(c)$, $E(c) = \sum_{i=1}^n (y_i-c)^2$.