It depends a lot on the kind of material being recycled. Paper has been getting recycled very effectively for decades. Metals are pretty recyclable. Plastic is notoriously hard, especially the thin plastic from bags, apparently.

Whatever the case, it shouldn't be dumped in rivers.