Freedom to roam does not need to be freedom to enter people's garden. At least in Norway, the law requires you to stay a minimum distance from dwellings, and has a variety of requirements to ensure you're not a nuisance.

Sweden has similar restrictions.