Ah, the feature that turned into a bug. Lion automatically saves the windows of any application when it's quit, and restores those window when you next start the app. So to get it to go away, close the window, then quit; don't just quit right away.