What all the critics miss is that Lein is "Easy" but tools.deps, tools.build are "Simple". Rich wants all of clojure to be built from simple orthogonal parts that compose together. Ideally a multi-purpose tool like leiningen should be built on top of those simple parts. Other tools can re-use those parts in different ways. With Leiningen - its a great tool but it's a complex thing that you take on an all-or-nothing basis.
Theory is all very well, but adding a library to a project is a critical requirement for all projects. A new starter comes over to Clojure from Python and the Deps guide [0] confidently explains too them that they add
to their deps.edn and they're good to go. Raising only the questions of:
1. What are the available keys of this map?
2. Why do I have to repeat clojure.java-time twice? Is this arbitrary? When are these different? (turns out it depends, if anyone is trying to figure that out)
3. What is a ":mvn"?
4. Is there a list of libraries I should be consulting to work out what versions mvn supports?
5. Is mvn my only choice here? Are there other repositories?
6. The map is one of Clojure's most flexible data structures because it can take arbitrary keys. Circling back to 1 - What other keys are supported?
Answering 1/6 is particularly interesting because the Deps reference [1] is long, undecipherable and frankly not-very-well written reference material. It starts off as more of a tutorial than a reference and requires the reader to engage with how Clojure programs run rather than how to download a library.
Someone coming over from Python would look at this, recall "pip install library" and then a lot of them would do the sensible thing and give up. Potentially never realising that the correct thing to do is go with leiningen. I'm not even sure if pip has command line flags, I've basically never been exposed to them.
Simple is all very good, I'm sure the people who struggle through to figure out how deps actually works are the better for it, but it would be better if adding a library were easy as well as simple.
While I agree that Leiningen is easier to get going with, many of the questions you asked also apply to Leiningen's project file.
I use Leiningen as it's convenient, but I check back on the tools.* ecosystem occasionally to see the progress being made. My suspicion is that tools.* will eventually supersede Leiningen and Boot, but that it will take two to three years at least.
Currently it's easier to use "lein new", but I can imagine a Babashka wrapper around tools.deps and tools.build eventually supplanting this. Being able to write, for example:
bbb new foobar; cd foobar; bbb add clojure.java.time
to automatically generate and update a deps.edn would be convenient, and potentially with far lower latency than Lein.
As a Clojure developer focused on creating correct/working software, I find the "Simple"-ness of my tooling's internal architecture to be a lower priority than it not leaking additional cognitive load onto my dev experience.
It may be the case that lein's internals are a blob of complexity, without boundaries defined in the "right" places. Maybe Clojure Tools' 3 or 4 components have it correct (though I'm skeptical of that, since there's overlap between them -- even moreso now with tools.build).
Conversely, it may be that build tooling is by its nature a messy affair (dealing with dependencies, resources on the filesystem/network, interaction with tools, tests, etc.), and that making those problems go away to the extent technically possible does your community of developers the greatest service. That would explain why talk of CLI and deps invariably has pushback and lein advocacy.
I wouldn't know if lein is internally complex because I've never had to go digging in it beyond the documentation project.clj file, even to do rather complex build definitions. By comparison, I've easily spent man-weeks on getting to the same place with deps.edn, and I would still find even defining the roles of its components difficult if pressed. I also worry that if we hit new Clojure devs with a wall of tooling issues like this, they won't stick around, even if we tell them the underlying theory of their tooling is more sound.
Clojure projects can start with literally no files (not even a deps.edn). By default, you get Clojure as a dep and src/ as a directory. Run `clj` to get a repl. If you make a function, you can call it with `clj -X my.ns/foo`. If you push it to github, others can consume it immediately by using your git url + sha.
There are lots of other things - they are all additive. Many Clojure libraries and tools don't need a build.clj or tools.build at all.
My experience is that deps.edn is much easier to get started with. With a template from something like clj-new you can get testing, a basic jar building script, etc to get started with and it's all there for you to muck with if needed.
I also worry that if we hit new Clojure devs with a wall of tooling issues like this, they won't stick around, even if we tell them the underlying theory of their tooling is more sound.
This sounds like a educational problem rather than "lets dumb down our tools to cater to new developers".
Clojure seems to balance "cater to professionals VS being easy to get started with" to the first mentioned segment, something I myself has no problem with.
Simple and easy are not mutually exclusive, and simple software is not automatically good software.
Other criteria, ease of use, documentation, also matter, especially for a build tool intended to be used by a whole community.
At the moment deps feels more of a 'make your own build tool' kit.
Simple, sure, but not easy to use: you need to tell it how to build jars or run tests, which is IMO essential functionality of a build tool.
I don't want to tell my build tool how to achieve these essential things, and risk ending up with a mess of projects that all use different aliases to run different tests runners and different jar building plugins.
Build tooling is one place where convention over configuration matters to me, and why I still prefer leiningen.
I'm hoping deps and tools.build will evolve from a 'make your own build tool kit' to a build tool that exposes the kit it was built on, so that most projects don't require anything custom, and be flexible enoug so that one special project can leverage the 'simple orthogonal parts' in the kit.
I see tools.build as a step forwards, but it's not there yet IMO, the user still needs to configure a bunch of stuff to get essential functionality.
I hope the core team will make a build tool on top of the 'make your own build tool kit' that is deps and tools.build, that comes with good defaults and essential features for the common projects, while exposing the 'simple orthogonal parts' in the kit so these can be used for complex projects.
A bit of context, because your post could easily be taken in bad faith:
Ideally a multi-purpose tool like leiningen should be built on top of those simple parts
tools.build is 2 weeks old according to the announcement.
Leiningen is nearly 12 years old.
I haven't looked into tools.build, so not arguing for or against it.
Disclaimer: Have been doing some work on leiningen from time to time.
Also I first ran into Clojure in 2011, and while I liked the language, I simply loved how leiningen just worked and I was instantly a fan, not even thinking about how it's built, just nice for the users[0]. And I've always hated projects/ecosystems that made you jump through hoops or some arcane syntax and exploded half the times you tried to build. So "easy" is really important.
Still not sure I agree here, as long as it's flexible enough to let me get my work done.
The whole thing was "it must be simple, even if the user needs to spend more time learning it" which is only true for people who learn the thing and keep on using it. For us who come back to a project after months (because we switch between tech stacks) it might be just twice the amount of work, if you need to relearn it every time.
It's not just that, or I don't think that is even the primary reason for tools.x.
Spec and tools are being developed to lay a foundation, so one can talk about and build upon the ideas of granular, function level dependencies, change (breakage vs. accretion) and so on. Ultimately this is about how code changes over time.
Comments
What all the critics miss is that Lein is "Easy" but tools.deps, tools.build are "Simple". Rich wants all of clojure to be built from simple orthogonal parts that compose together. Ideally a multi-purpose tool like leiningen should be built on top of those simple parts. Other tools can re-use those parts in different ways. With Leiningen - its a great tool but it's a complex thing that you take on an all-or-nothing basis.
Theory is all very well, but adding a library to a project is a critical requirement for all projects. A new starter comes over to Clojure from Python and the Deps guide [0] confidently explains too them that they add
{:deps {clojure.java-time/clojure.java-time {:mvn/version "0.3.2"}}}
to their deps.edn and they're good to go. Raising only the questions of:
1. What are the available keys of this map?
2. Why do I have to repeat clojure.java-time twice? Is this arbitrary? When are these different? (turns out it depends, if anyone is trying to figure that out)
3. What is a ":mvn"?
4. Is there a list of libraries I should be consulting to work out what versions mvn supports?
5. Is mvn my only choice here? Are there other repositories?
6. The map is one of Clojure's most flexible data structures because it can take arbitrary keys. Circling back to 1 - What other keys are supported?
Answering 1/6 is particularly interesting because the Deps reference [1] is long, undecipherable and frankly not-very-well written reference material. It starts off as more of a tutorial than a reference and requires the reader to engage with how Clojure programs run rather than how to download a library.
Someone coming over from Python would look at this, recall "pip install library" and then a lot of them would do the sensible thing and give up. Potentially never realising that the correct thing to do is go with leiningen. I'm not even sure if pip has command line flags, I've basically never been exposed to them.
Simple is all very good, I'm sure the people who struggle through to figure out how deps actually works are the better for it, but it would be better if adding a library were easy as well as simple.
[0] https://clojure.org/guides/deps_and_cli#_running_a_repl_and_...
[1] https://clojure.org/reference/deps_and_cli
While I agree that Leiningen is easier to get going with, many of the questions you asked also apply to Leiningen's project file.
I use Leiningen as it's convenient, but I check back on the tools.* ecosystem occasionally to see the progress being made. My suspicion is that tools.* will eventually supersede Leiningen and Boot, but that it will take two to three years at least.
Currently it's easier to use "lein new", but I can imagine a Babashka wrapper around tools.deps and tools.build eventually supplanting this. Being able to write, for example:
to automatically generate and update a deps.edn would be convenient, and potentially with far lower latency than Lein.These are good points and I'd welcome feedback at https://github.com/clojure/clojure-site/issues
As a Clojure developer focused on creating correct/working software, I find the "Simple"-ness of my tooling's internal architecture to be a lower priority than it not leaking additional cognitive load onto my dev experience.
It may be the case that lein's internals are a blob of complexity, without boundaries defined in the "right" places. Maybe Clojure Tools' 3 or 4 components have it correct (though I'm skeptical of that, since there's overlap between them -- even moreso now with tools.build).
Conversely, it may be that build tooling is by its nature a messy affair (dealing with dependencies, resources on the filesystem/network, interaction with tools, tests, etc.), and that making those problems go away to the extent technically possible does your community of developers the greatest service. That would explain why talk of CLI and deps invariably has pushback and lein advocacy.
I wouldn't know if lein is internally complex because I've never had to go digging in it beyond the documentation project.clj file, even to do rather complex build definitions. By comparison, I've easily spent man-weeks on getting to the same place with deps.edn, and I would still find even defining the roles of its components difficult if pressed. I also worry that if we hit new Clojure devs with a wall of tooling issues like this, they won't stick around, even if we tell them the underlying theory of their tooling is more sound.
Clojure projects can start with literally no files (not even a deps.edn). By default, you get Clojure as a dep and src/ as a directory. Run `clj` to get a repl. If you make a function, you can call it with `clj -X my.ns/foo`. If you push it to github, others can consume it immediately by using your git url + sha.
There are lots of other things - they are all additive. Many Clojure libraries and tools don't need a build.clj or tools.build at all.
My experience is that deps.edn is much easier to get started with. With a template from something like clj-new you can get testing, a basic jar building script, etc to get started with and it's all there for you to muck with if needed.
This sounds like a educational problem rather than "lets dumb down our tools to cater to new developers".
Clojure seems to balance "cater to professionals VS being easy to get started with" to the first mentioned segment, something I myself has no problem with.
Simple and easy are not mutually exclusive, and simple software is not automatically good software. Other criteria, ease of use, documentation, also matter, especially for a build tool intended to be used by a whole community.
At the moment deps feels more of a 'make your own build tool' kit. Simple, sure, but not easy to use: you need to tell it how to build jars or run tests, which is IMO essential functionality of a build tool.
I don't want to tell my build tool how to achieve these essential things, and risk ending up with a mess of projects that all use different aliases to run different tests runners and different jar building plugins. Build tooling is one place where convention over configuration matters to me, and why I still prefer leiningen.
I'm hoping deps and tools.build will evolve from a 'make your own build tool kit' to a build tool that exposes the kit it was built on, so that most projects don't require anything custom, and be flexible enoug so that one special project can leverage the 'simple orthogonal parts' in the kit.
I see tools.build as a step forwards, but it's not there yet IMO, the user still needs to configure a bunch of stuff to get essential functionality.
I hope the core team will make a build tool on top of the 'make your own build tool kit' that is deps and tools.build, that comes with good defaults and essential features for the common projects, while exposing the 'simple orthogonal parts' in the kit so these can be used for complex projects.
A bit of context, because your post could easily be taken in bad faith:
tools.build is 2 weeks old according to the announcement.
Leiningen is nearly 12 years old.
I haven't looked into tools.build, so not arguing for or against it.
Disclaimer: Have been doing some work on leiningen from time to time.
Also I first ran into Clojure in 2011, and while I liked the language, I simply loved how leiningen just worked and I was instantly a fan, not even thinking about how it's built, just nice for the users[0]. And I've always hated projects/ecosystems that made you jump through hoops or some arcane syntax and exploded half the times you tried to build. So "easy" is really important.
[0]: https://f5n.org/blog/2011/a-week-of-clojure/
"Easy" is important for someone to start playing around with something, to initially hook the user (the developer in this case).
But for professionals who spend a lot of time with the same tool, "simple" is more important in the long run.
Still not sure I agree here, as long as it's flexible enough to let me get my work done.
The whole thing was "it must be simple, even if the user needs to spend more time learning it" which is only true for people who learn the thing and keep on using it. For us who come back to a project after months (because we switch between tech stacks) it might be just twice the amount of work, if you need to relearn it every time.
It's not just that, or I don't think that is even the primary reason for tools.x.
Spec and tools are being developed to lay a foundation, so one can talk about and build upon the ideas of granular, function level dependencies, change (breakage vs. accretion) and so on. Ultimately this is about how code changes over time.