That’s an interesting phrasing. Even you (who seems to support the policy) describe the convicted criminal as a convicted criminal, yet they’re able to get a piece of paper that says they aren’t one.

They can get that paper after they payed their dues to society (served their time).

Does everyone have to produce such a paper routinely in life

Not routinely but I don't think anybody goes through their life without ever needing that paper. It is required for a vast variety of government programs and some jobs also require it. Working in IT I had to get it a few times when applying for jobs in private sector.