Nice work! I'd be interested in how we could improve the UI to cater better to your third point. If I'm interested in how a language looks I tend to find one good solution, then look at the poster's profile and browse other of their solutions in that language. I'd welcome your thoughts :) Would like to open an issue on https://github.com/exercism/exercism and we can discuss this more widely with the Exercism community?
Great! Actually I had opened an issue a while ago but with a much larger scope in mind -- to also do some AST analysis and try to clump the solutions by similarity, so that the UI could present "There are three basic ways to solve this..."
I was really into exercism before they changed the interface. And I know that change is good, but the peer mentorship I got when the previous interface existed was decent, and at least pretty consistent.
With the new interface, I need to wait for official mentors, and while some have been way above and beyond, the dearth of reactions from anyone for long periods can be a real downer. I preferred the semi-OK amateur mentorship over the wildly inconsistent/non-existent mentor assistance.
There was one mentor that I would have happily paid directly to keep giving me feedback, but there was no mechanism in exercism for me to make that offer, even if exercism took a cut.
Also in the old interface, the community solutions were sorted by recency, so it was easier to interact with other fellow learners, and the new interface is not that, I'm not sure what the order is, but it's of no use for me to comment on a piece of code submitted two years ago.
I preferred the semi-OK amateur mentorship over the wildly inconsistent/non-existent mentor assistance.
Just to chime in with data here, there is 5x more feedback per day on the new system than there was with the old system. Depending on the track, you'll get feedback in 3hrs to 7days (very occasionally longer), so it is definitely inconsistent in that respect, but you will get feedback. In the old system, you'd get feedback at all in under 10% of submissions.
There was one mentor that I would have happily paid directly to keep giving me feedback, but there was no mechanism in exercism for me to make that offer, even if exercism took a cut.
In the future we'll be adding the ability to "pair" mentor/mentees who have both rated the other highly, so this will sort-of happen for you.
I got things like kudos etc... and comments which were helpful, I would have considered either one feedback. In general, there seemed to be more interaction, regardless of whether you call it feedback, or social validation.
The pair mentor/mentee sounds good, I look forward to it!
So there are two modes:
1) Mentored Mode: Submit, get structured feedback from mentors, then publish and get stars (ie upvotes/kudos) and/or public comments.
2) Independent Mode: Submit, publish, get stars (ie upvotes/kudos) and/or public comments.
So Independent Mode is very similar to "classic" Exercism.
I'm defining "feedback" really as (1) - so there are 5x more comments (from mentors + public) now than there were in old Exercism (public).
So for a tangible example:
- Feb 1st 2018 (classic): 162 comments, 110 stars
- Feb 1st 2019 (new): 831 comments, 46 stars
I randomly picked that date, but every date is basically the same sort of ratio.
Lots of people say the same as you do that it feels like there's less interaction, but really there's a lot more. I feel like we're failing to make Exercism feel as alive as it actually is for some reason, but I'm not sure why :)
Are these numbers normalized to the number of people making the interactions? or the number of learners? Did you have 5x growth over the past year? Sorry, I am kind of being critical, but mainly because my experienced degraded so much. I trust you to run the place in what ever way you see fit, and any gripe I have is because I really want exercism to work for me. It's the best way I've up-leveled my skills outside of coding on the job, and I really feel like I hit a huge speedbump when the new site went live.
They're not normalised, no. But the theme holds true. We have about 80% more solutions submitted per day and about 500% more comments per day. So engagement per solution is much higher.
It's also worth pointing out that this whole thing is dramatically evolving. It may be that you're a more "expert" developer and doing the more complex problems that means you're effectively lower in the queue (harder to mentor than basic solutions). Things like https://exercism.io/blog/automated-mentoring-support-project aim to free up our mentors to give them more time to deal with the harder problems. And things like https://github.com/exercism/exercism/issues/4658 and https://exercism.io/blog/track-anatomy-project aim to make mentoring easier and more enjoyable, which means we keep mentors around for longer and they mentor more because it's more fun for them. It may also be that you're on a track that has had 5x or 20x growth (I'm not sure if there are any but you get the idea) in which case the speedbump might have corresponded more to the growth of that track than v2.
If you don't mind telling me your exercism handle, I'd be interested to dig into your situation further to understand why you hit that speedbump. It's easy for me to talk in terms of data, but I'm limited to seeing the general patterns rather than hearing real people's stories such as yours, so it's really valuable for me to understand. Feel free to email me (jeremy@exercism.io) if you don't want to disclose it on here :)
I've been independently doing the Javascript track. I've been waiting for feedback on my Forth-like solution for a couple of weeks now. I can understand why, since it's a volunteer effort, and mentored mode users are a higher priority. I am trying to resist the temptation to rewrite it until I get feedback on the version I originally submitted.
I haven't received any peer feedback on my submissions, either, and I've seen almost no stars or comments on any of the solutions I've viewed. There seems to be very little of either one. I think giving feedback is hard and deserves care, which is why I haven't given any myself.
These aren't intended as complaints, just observations on my experience. I've decided the main value for me is that it gives me a set of practice problems, and the opportunity to learn from other people's solutions.
If I’m not confused about the new feedback mechanism, the site specifically discourages comments about the quality of the code, in favor of questions about it. I assume that’s to avoid discouraging people with overly negative feedback.
This is frustrating to me; in the Erlang space, we don’t have enough people to abuse the system, so I used to be able to provide and receive feedback from anyone without any negativity.
Comments
I think exercism.io is hugely valuable for solving three problems:
- Where can I find a set of practice problems which will force me to really engage with learning a new language?
- Given a specific problem, what does the solution look like in various langauges?
- Given a specific language, what do the solutions to a variety of problems look like?
Unfortunately, exercism's web interface is really only set up to cater to the first of those.
So I made an couple of indexes to solve for the other two:
https://gist.github.com/cellularmitosis/ae76bb09973f55418d97...
https://gist.github.com/cellularmitosis/2aec8d88a76771452f35...
Nice work! I'd be interested in how we could improve the UI to cater better to your third point. If I'm interested in how a language looks I tend to find one good solution, then look at the poster's profile and browse other of their solutions in that language. I'd welcome your thoughts :) Would like to open an issue on https://github.com/exercism/exercism and we can discuss this more widely with the Exercism community?
Great! Actually I had opened an issue a while ago but with a much larger scope in mind -- to also do some AST analysis and try to clump the solutions by similarity, so that the UI could present "There are three basic ways to solve this..."
https://github.com/exercism/exercism/issues/4333
(that's my work account)
Ah awesome. Sorry I dropped the ball on that. I'll reply now. Have you seen about https://exercism.io/blog/automated-mentoring-support-project ?
I was really into exercism before they changed the interface. And I know that change is good, but the peer mentorship I got when the previous interface existed was decent, and at least pretty consistent.
With the new interface, I need to wait for official mentors, and while some have been way above and beyond, the dearth of reactions from anyone for long periods can be a real downer. I preferred the semi-OK amateur mentorship over the wildly inconsistent/non-existent mentor assistance.
There was one mentor that I would have happily paid directly to keep giving me feedback, but there was no mechanism in exercism for me to make that offer, even if exercism took a cut.
Also in the old interface, the community solutions were sorted by recency, so it was easier to interact with other fellow learners, and the new interface is not that, I'm not sure what the order is, but it's of no use for me to comment on a piece of code submitted two years ago.
Just to chime in with data here, there is 5x more feedback per day on the new system than there was with the old system. Depending on the track, you'll get feedback in 3hrs to 7days (very occasionally longer), so it is definitely inconsistent in that respect, but you will get feedback. In the old system, you'd get feedback at all in under 10% of submissions.
In the future we'll be adding the ability to "pair" mentor/mentees who have both rated the other highly, so this will sort-of happen for you.
How do you define feedback?
I got things like kudos etc... and comments which were helpful, I would have considered either one feedback. In general, there seemed to be more interaction, regardless of whether you call it feedback, or social validation.
The pair mentor/mentee sounds good, I look forward to it!
So there are two modes: 1) Mentored Mode: Submit, get structured feedback from mentors, then publish and get stars (ie upvotes/kudos) and/or public comments. 2) Independent Mode: Submit, publish, get stars (ie upvotes/kudos) and/or public comments.
So Independent Mode is very similar to "classic" Exercism.
I'm defining "feedback" really as (1) - so there are 5x more comments (from mentors + public) now than there were in old Exercism (public).
So for a tangible example: - Feb 1st 2018 (classic): 162 comments, 110 stars - Feb 1st 2019 (new): 831 comments, 46 stars
I randomly picked that date, but every date is basically the same sort of ratio.
Lots of people say the same as you do that it feels like there's less interaction, but really there's a lot more. I feel like we're failing to make Exercism feel as alive as it actually is for some reason, but I'm not sure why :)
Are these numbers normalized to the number of people making the interactions? or the number of learners? Did you have 5x growth over the past year? Sorry, I am kind of being critical, but mainly because my experienced degraded so much. I trust you to run the place in what ever way you see fit, and any gripe I have is because I really want exercism to work for me. It's the best way I've up-leveled my skills outside of coding on the job, and I really feel like I hit a huge speedbump when the new site went live.
They're not normalised, no. But the theme holds true. We have about 80% more solutions submitted per day and about 500% more comments per day. So engagement per solution is much higher.
It's also worth pointing out that this whole thing is dramatically evolving. It may be that you're a more "expert" developer and doing the more complex problems that means you're effectively lower in the queue (harder to mentor than basic solutions). Things like https://exercism.io/blog/automated-mentoring-support-project aim to free up our mentors to give them more time to deal with the harder problems. And things like https://github.com/exercism/exercism/issues/4658 and https://exercism.io/blog/track-anatomy-project aim to make mentoring easier and more enjoyable, which means we keep mentors around for longer and they mentor more because it's more fun for them. It may also be that you're on a track that has had 5x or 20x growth (I'm not sure if there are any but you get the idea) in which case the speedbump might have corresponded more to the growth of that track than v2.
If you don't mind telling me your exercism handle, I'd be interested to dig into your situation further to understand why you hit that speedbump. It's easy for me to talk in terms of data, but I'm limited to seeing the general patterns rather than hearing real people's stories such as yours, so it's really valuable for me to understand. Feel free to email me (jeremy@exercism.io) if you don't want to disclose it on here :)
I've been independently doing the Javascript track. I've been waiting for feedback on my Forth-like solution for a couple of weeks now. I can understand why, since it's a volunteer effort, and mentored mode users are a higher priority. I am trying to resist the temptation to rewrite it until I get feedback on the version I originally submitted.
I haven't received any peer feedback on my submissions, either, and I've seen almost no stars or comments on any of the solutions I've viewed. There seems to be very little of either one. I think giving feedback is hard and deserves care, which is why I haven't given any myself.
These aren't intended as complaints, just observations on my experience. I've decided the main value for me is that it gives me a set of practice problems, and the opportunity to learn from other people's solutions.
If I’m not confused about the new feedback mechanism, the site specifically discourages comments about the quality of the code, in favor of questions about it. I assume that’s to avoid discouraging people with overly negative feedback.
This is frustrating to me; in the Erlang space, we don’t have enough people to abuse the system, so I used to be able to provide and receive feedback from anyone without any negativity.
For public comments, this is correct. For mentoring (in Mentored Mode) you will receive comments about the quality of code.
That's unfortunate.