I think that is an important observation. The total population might be stable or slightly rising; however, that should not be interpreted as no one leaving and only a few people moving in. It's has to do with the churn, or velocity of people coming in and then going out (leaving) as children begin reaching school age. The (public) schools, which very few exceptions, are terrible and just about everyone I know who has young children has plans on moving somewhere with better schools when it comes time.

Yes the city has many cultural and civic attractions, but they do not outweigh the educational deficit, for those with young children. For singles and for empty nesters the city is perfect, for people with young families the dearth of good public educational institutions is maddening. The wealthy can afford to send their children to private schools, everyone else cannot.