That's okay, as long that effect is roughly equal in every period of increasing unemployment. This and many other factors make it meaningless to treat unemployment additively, e.g. to say that 8% unemployment is "twice as bad" as 4%, but it doesn't diminish the number's comparative value, telling us whether we're better or worse off than before.