As someone else mentioned, the BLUE is what we are usually looking for the Gauss-Markov theorem linked tells us why it is least squares.
EDIT: I am a bit suspicious though, since the definition of variance implicitly uses least squares. Maybe someone else can explain this better, but my expectation is that variance and the reason why it also uses squares (and not something else) should follow from some set of first principles.
Comments
As someone else mentioned, the BLUE is what we are usually looking for the Gauss-Markov theorem linked tells us why it is least squares.
EDIT: I am a bit suspicious though, since the definition of variance implicitly uses least squares. Maybe someone else can explain this better, but my expectation is that variance and the reason why it also uses squares (and not something else) should follow from some set of first principles.