A lot of the confusion and criticism surrounding submodules comes from people not understanding that you do not need to learn a lot of new commands to use them: you just cd into the submodule and use it as a normal git repo. Then you cd out and commit the submodule hash change if that's what you want to do.
Some things are annoying, like removing them. But in general they're extremely useful and designed basically exactly how you'd want: it's extremely consistent since to a good approximation, a submodule behaves the same as a normal git repo once you're inside it.
It's the only way to do certain things but the devx is burdensome. It might be easy enough to do a thing but the major issue is that you have to constantly fight wonky behavior when working in a team.
If someone adds a new submodule, you need to know to pull it or at least init it. You need to know the code is in a submodule and treat it differently, complicating integration procedures.
It certainly does not behave the same in that you certainly can't ignore whether it exists once it's set up.
I disagree, even when you understand them there are still big problems with them:
1. Git doesn't support them very well. Bugs are disappointing frequent. Try changing a submodule to a directory or vice versa and feel the pain.
2. They are often used to split up projects (especially commercially) but this makes testing much harder and cross-repo changes way way harder. Even changes that only touch the submodule become way more tedious because you have to always do another PR to update the pointer.
3. The submodules are referenced by URL, which means you can't easily move or copy projects that have them because they still point to the old submodule location. Sometimes they're even misconfiguration to always use ssh:// or whatever, breaking unauthenticated clones.
4. Switching branches where submodules have been added/removed is a mess.
5. Having to constantly remember `git submodule update --init --recursive` is a right pain.
6. When used as a crap dependency manager you easily end up with duplicates. A project I worked on had 12 copied of a common submodule via transitive dependencies.
Git doesn't support them very well. Bugs are disappointing frequent
I can't remember any "bug" in the submodule machinery. If you find one, report to the Git mailing list, they very receptive.
Having to constantly remember `git submodule update --init --recursive`
Set this config: `git config submodule.recurse true`
IMO they're almost always the wrong solution.
Yeah... but sometimes they are the only feasible solution, especially when the dependency needs to be shared across projects that use incompatible dependency managers and Git is the only common denominator between them...
Set this config: `git config submodule.recurse true`
Well that does save a flag at least so thanks! But I still have to run the command.
There is actually a flag to do submodule update automatically when you checkout or switch branches but DO NOT USE IT! It is known to be buggy and can break your .git directory in a way that is pretty much "delete everything and clone from scratch".
Submodules aren't perfect and can be really annoying. I tried to avoid them at first.
But if you work with larger projects I think you'll see how they are useful.
For example, some projects might commit their node_modules or composer vendor folder as submodules so that they have an exact snapshot of their dependencies that can be quickly cloned in place and referenced without mixing it with the parent repos source.
Submodules are certainly not the only way to do things with git. Sometimes monorepos or something else makes more sense.
But if you care about maintainability then what are you supposed to do when the majority of your commits are unrelated to your project because you're using git subtree or something similar? Or the majority of your files are also not directly relevant to your project but need to be there?
What about stuff like optional tests or maintenance scripts or even binary dependencies that you want to have closely associated with your project but don't want to clone every time?
My thinking is that the porcelain is incomplete but very open. The way submodules are represented can be added on to.
And the plumbing is consistent enough that you easily script around it.
You should check out the Datalad project to see some really clever usage of submodules.
And the url key can be annoying; I usually try to use relative paths in my .gitmodules folder when possible. You're actually free to use remote helpers and relative paths to get around hard a github url into it.
You can also embed and read arbitrary keys from your .gitmodules file with git-config -f which is really useful. You don't even have to use a .gitmodules file, in theory.
Well indeed. I think the ultimate point is that Git is severely lacking here. There are clear use cases (you listed some) that drive people to submodules or LFS, but they are both shitty solutions.
We need Git (or something else) to provide a third solution that actually works. I'm really hoping that JJ realises that this is a problem that needs to be solved but I'm not holding my breath.
I think submodules are good enough for what they do and there's room in Git's model for forward development and it's also flexible enough that third party tools can piggyback on it.
Like Git in general it's a bit of a pain till you practice with it more.
Git can do some amazing stuff, I use it for data as well as code.
Comments
A lot of the confusion and criticism surrounding submodules comes from people not understanding that you do not need to learn a lot of new commands to use them: you just cd into the submodule and use it as a normal git repo. Then you cd out and commit the submodule hash change if that's what you want to do.
Some things are annoying, like removing them. But in general they're extremely useful and designed basically exactly how you'd want: it's extremely consistent since to a good approximation, a submodule behaves the same as a normal git repo once you're inside it.
It's the only way to do certain things but the devx is burdensome. It might be easy enough to do a thing but the major issue is that you have to constantly fight wonky behavior when working in a team.
If someone adds a new submodule, you need to know to pull it or at least init it. You need to know the code is in a submodule and treat it differently, complicating integration procedures.
It certainly does not behave the same in that you certainly can't ignore whether it exists once it's set up.
I disagree, even when you understand them there are still big problems with them:
1. Git doesn't support them very well. Bugs are disappointing frequent. Try changing a submodule to a directory or vice versa and feel the pain.
2. They are often used to split up projects (especially commercially) but this makes testing much harder and cross-repo changes way way harder. Even changes that only touch the submodule become way more tedious because you have to always do another PR to update the pointer.
3. The submodules are referenced by URL, which means you can't easily move or copy projects that have them because they still point to the old submodule location. Sometimes they're even misconfiguration to always use ssh:// or whatever, breaking unauthenticated clones.
4. Switching branches where submodules have been added/removed is a mess.
5. Having to constantly remember `git submodule update --init --recursive` is a right pain.
6. When used as a crap dependency manager you easily end up with duplicates. A project I worked on had 12 copied of a common submodule via transitive dependencies.
IMO they're almost always the wrong solution.
I can't remember any "bug" in the submodule machinery. If you find one, report to the Git mailing list, they very receptive.
Set this config: `git config submodule.recurse true`
Yeah... but sometimes they are the only feasible solution, especially when the dependency needs to be shared across projects that use incompatible dependency managers and Git is the only common denominator between them...
Well that does save a flag at least so thanks! But I still have to run the command.
There is actually a flag to do submodule update automatically when you checkout or switch branches but DO NOT USE IT! It is known to be buggy and can break your .git directory in a way that is pretty much "delete everything and clone from scratch".
Submodules aren't perfect and can be really annoying. I tried to avoid them at first.
But if you work with larger projects I think you'll see how they are useful.
For example, some projects might commit their node_modules or composer vendor folder as submodules so that they have an exact snapshot of their dependencies that can be quickly cloned in place and referenced without mixing it with the parent repos source.
Submodules are certainly not the only way to do things with git. Sometimes monorepos or something else makes more sense.
But if you care about maintainability then what are you supposed to do when the majority of your commits are unrelated to your project because you're using git subtree or something similar? Or the majority of your files are also not directly relevant to your project but need to be there?
What about stuff like optional tests or maintenance scripts or even binary dependencies that you want to have closely associated with your project but don't want to clone every time?
My thinking is that the porcelain is incomplete but very open. The way submodules are represented can be added on to.
And the plumbing is consistent enough that you easily script around it.
You should check out the Datalad project to see some really clever usage of submodules.
And the url key can be annoying; I usually try to use relative paths in my .gitmodules folder when possible. You're actually free to use remote helpers and relative paths to get around hard a github url into it.
You can also embed and read arbitrary keys from your .gitmodules file with git-config -f which is really useful. You don't even have to use a .gitmodules file, in theory.
Well indeed. I think the ultimate point is that Git is severely lacking here. There are clear use cases (you listed some) that drive people to submodules or LFS, but they are both shitty solutions.
We need Git (or something else) to provide a third solution that actually works. I'm really hoping that JJ realises that this is a problem that needs to be solved but I'm not holding my breath.
What problem are you trying to solve exactly?
I think submodules are good enough for what they do and there's room in Git's model for forward development and it's also flexible enough that third party tools can piggyback on it. Like Git in general it's a bit of a pain till you practice with it more.
Git can do some amazing stuff, I use it for data as well as code.