I think peer-graded assignments might do the job. I am taking Gamification course on Coursera right now, and I liked peer-graded assignments a lot.

If there would be peer-graded assignments in machine learning course, I would definitely have tried them out.