An oft overlooked part of being involved in open source is the networking and mentorship, both technical and personal. Being a part of one or more well run projects lets you interact with some incredibly smart people discussing technical topics well outside your day job. Not all of that will pan out to something directly useful, but over time it adds up to both a wider skillset at actual development and dealing with a distributed team. The evolution of your written communication skills alone are probably worth it.