Unless I have the source I'm screwed.

But even having the source doesn't mean you aren't screwed. Even if you have the source, you still need a license that allows you to do something with the source. It's possible for the upstream team to give you the source, e.g., for viewing, but not grant you any rights to do anything with it. This is why the free software and open source licenses go on about things like guaranteeing redistributability and derived works, rather than just "Hey, you know. You can see the source code." Consider this: you have unfettered access to the "source" of a book in your library. That doesn't mean you have rights to do any of the stuff in the vein of what open source, free software, or most Creative Commons licensing allows you to do.

(It's possible that the "have" in your "have the source" was shorthand for having these rights.)