Yes, but it's a little more complicated than that. In most cases you cannot trust the solution of straight Gaussian elimination. You need to take additional measures to avoid excessive error growth, such as using partial pivoting.

see http://en.wikipedia.org/wiki/Pivoting and http://en.wikipedia.org/wiki/Gaussian_elimination