I can see where you're coming from in any other language. But in Julia it's important to realise that this "premature abstraction" isn't actually any extra work, it's just the default. If we write `f(x) = x+x` then `f` takes anything that can be added, which can be any custom number or matrix type, or really anything else. Adding type restrictions to make it work with only a limited set of types is completely doable, but actually more work than just leaving it generic.
We didn't at any point decide "it's worth the extra effort/complexity to make Flux work with custom number types"; it's just inadvertently been that way from day one, and I didn't even know anyone one was making use of it until today.
This is the same for many languages that treat operators with type class patterns. It’s still usually bad design choice. I find a lot of language designers & programmers like it, but they are super disconnected from the realities.
For an example consider breeze and spire in Scala. There’s so much effort to create these bloated numeric type hierarchies that abstract out things like monoids, rings, fields, iterability, sortability, etc.
It’s not good. Just having really boring repetitive implementations for each distinct data structure would be better! No joke! Being able to write type generic functions over sortable matrix subclasses turns out to not be valuable unless you’re also writing a highly abstracted library, which is never, certainly not when you’re using it for experiments.
Nobody needs to be able to make a DenseMatrix[Quaternion] and get it to automatically pick up implementations of fancy indexing. No. You can just write your own helper methods, and this is better, more convenient, applies less pressure for DenseMatrix to have some indecipherably complicated abstract implementation so it can be more free to just specialize on linear algebra functionality that works for DenseMatrix[Double] which is what is needed 99.999999999% of the time.
You really should try Julia, before making claims about its complexity.
The numeric type systems are simple, and designed for convenience, not to satisfy mathematical theory. In the case of FloatX, It's basically Any <: Number <: AbstractReal <: AbstractFloat <: FloatX
For complex datatypes, like vectors, matrices, dicts, etc, you have templatable datatypes, but that is no more complex than C++, and actually far cleaner in implementation.
For the most part, you do not NEED to make a Matrix{Quaternion}. And that's fine. However, if you do, the standard library will do the right thing, as if you had made a Matrix{Int32} or a Matrix{8BitGaloisField}. And if you choose to use Matrix{Float32}, the type system interacts with the compiler, and in the standard library it picks up the fortran BLAS library so you get faster-than-c performance.
On the other hand, you might be deploying a really large matrix on a supercomputing cluster, and it might be useful to re-index the matrix as a datatype that fits in the L1 cache of your Knights Landing chips. In which case, you have the option of redeploying as an AbstractMatrix{Float64}, implementing index catching functions, and dropping it in to you code (probably about 100 lines of code total, if even) without having to rewrite every single matrix operation everywhere.
It’s so funny to me how Julia proponents often make it an ad hominem attack as if the writer hasn’t used the language. I’ve been using and following Julia closely since late 2012, and even attended a few meetups / talks at MIT about it since I was a grad student at the time, and even took a random matrices class with Alan Edelman in which he talked quite a bit about early julia.
Julia is by no means the only language to have patterns like this either, and in fact it’s not even a language where these patterns are particularly easy to use (I would reserve that for Haskell, but admit there may be other languages I don’t know which also make the cut — not julia though).
Your two ending paragraphs read to me like a super naive restatement of the company line memo for why these types of parametric abstractions are supposed to be good. It’s like a political platform, and just like a political platform it doesn’t keep its promise.
I have worked on projects where we needed to customize bit packing, not for cache performance, but for control over a modified version of sparse matrix types.
And I’m telling you the idea that we’d ever rely on the language’s chosen abstraction and do something like AbstractSparseMatrix{Float64} to pick up a bunch of interface properties “for free” while making the underlying logic specialized for our sparse format is crazy. It’s a naive false promise that grad students believe and it gets quickly beaten out of them in the real world once you realize how the type constraints and inheritance / type class extension constraints this places on you are too limiting and end up requiring just too much boilerplate that can’t quite be autogenerated because the way the abstract interface was chosen just doesn’t quite match your use case.
Finally you realize going down this road was the wrong idea all along, and you just write a super short implementation of MyCustomSparseMatrix or MyCustomCachePropertyMatrix in your case, and you fill in the logic manually that you thought you’d be clever by getting “for free” via plugging into some abstraction hierarchy, and often realize for your use case you don’t need to re-implement hardly any of it, and can do the boring parts pretty easily with converters or helper functions that marshal between whatever “for free” functionality you hoped to get and your simple custom not-parametric-abstraction type.
I’ve been down this road too many times, in many languages. I just leave it for the grad students who like playing with abstraction toys, and instead I just get back to actual work, solving problems economically, which warrants a super strong heuristic of avoiding this type of parametric abstraction pattern as much as possible.
Comments
I can see where you're coming from in any other language. But in Julia it's important to realise that this "premature abstraction" isn't actually any extra work, it's just the default. If we write `f(x) = x+x` then `f` takes anything that can be added, which can be any custom number or matrix type, or really anything else. Adding type restrictions to make it work with only a limited set of types is completely doable, but actually more work than just leaving it generic.
We didn't at any point decide "it's worth the extra effort/complexity to make Flux work with custom number types"; it's just inadvertently been that way from day one, and I didn't even know anyone one was making use of it until today.
This is the same for many languages that treat operators with type class patterns. It’s still usually bad design choice. I find a lot of language designers & programmers like it, but they are super disconnected from the realities.
For an example consider breeze and spire in Scala. There’s so much effort to create these bloated numeric type hierarchies that abstract out things like monoids, rings, fields, iterability, sortability, etc.
It’s not good. Just having really boring repetitive implementations for each distinct data structure would be better! No joke! Being able to write type generic functions over sortable matrix subclasses turns out to not be valuable unless you’re also writing a highly abstracted library, which is never, certainly not when you’re using it for experiments.
Nobody needs to be able to make a DenseMatrix[Quaternion] and get it to automatically pick up implementations of fancy indexing. No. You can just write your own helper methods, and this is better, more convenient, applies less pressure for DenseMatrix to have some indecipherably complicated abstract implementation so it can be more free to just specialize on linear algebra functionality that works for DenseMatrix[Double] which is what is needed 99.999999999% of the time.
You really should try Julia, before making claims about its complexity.
The numeric type systems are simple, and designed for convenience, not to satisfy mathematical theory. In the case of FloatX, It's basically Any <: Number <: AbstractReal <: AbstractFloat <: FloatX
For complex datatypes, like vectors, matrices, dicts, etc, you have templatable datatypes, but that is no more complex than C++, and actually far cleaner in implementation.
For the most part, you do not NEED to make a Matrix{Quaternion}. And that's fine. However, if you do, the standard library will do the right thing, as if you had made a Matrix{Int32} or a Matrix{8BitGaloisField}. And if you choose to use Matrix{Float32}, the type system interacts with the compiler, and in the standard library it picks up the fortran BLAS library so you get faster-than-c performance.
On the other hand, you might be deploying a really large matrix on a supercomputing cluster, and it might be useful to re-index the matrix as a datatype that fits in the L1 cache of your Knights Landing chips. In which case, you have the option of redeploying as an AbstractMatrix{Float64}, implementing index catching functions, and dropping it in to you code (probably about 100 lines of code total, if even) without having to rewrite every single matrix operation everywhere.
It’s so funny to me how Julia proponents often make it an ad hominem attack as if the writer hasn’t used the language. I’ve been using and following Julia closely since late 2012, and even attended a few meetups / talks at MIT about it since I was a grad student at the time, and even took a random matrices class with Alan Edelman in which he talked quite a bit about early julia.
Julia is by no means the only language to have patterns like this either, and in fact it’s not even a language where these patterns are particularly easy to use (I would reserve that for Haskell, but admit there may be other languages I don’t know which also make the cut — not julia though).
Your two ending paragraphs read to me like a super naive restatement of the company line memo for why these types of parametric abstractions are supposed to be good. It’s like a political platform, and just like a political platform it doesn’t keep its promise.
I have worked on projects where we needed to customize bit packing, not for cache performance, but for control over a modified version of sparse matrix types.
And I’m telling you the idea that we’d ever rely on the language’s chosen abstraction and do something like AbstractSparseMatrix{Float64} to pick up a bunch of interface properties “for free” while making the underlying logic specialized for our sparse format is crazy. It’s a naive false promise that grad students believe and it gets quickly beaten out of them in the real world once you realize how the type constraints and inheritance / type class extension constraints this places on you are too limiting and end up requiring just too much boilerplate that can’t quite be autogenerated because the way the abstract interface was chosen just doesn’t quite match your use case.
Finally you realize going down this road was the wrong idea all along, and you just write a super short implementation of MyCustomSparseMatrix or MyCustomCachePropertyMatrix in your case, and you fill in the logic manually that you thought you’d be clever by getting “for free” via plugging into some abstraction hierarchy, and often realize for your use case you don’t need to re-implement hardly any of it, and can do the boring parts pretty easily with converters or helper functions that marshal between whatever “for free” functionality you hoped to get and your simple custom not-parametric-abstraction type.
I’ve been down this road too many times, in many languages. I just leave it for the grad students who like playing with abstraction toys, and instead I just get back to actual work, solving problems economically, which warrants a super strong heuristic of avoiding this type of parametric abstraction pattern as much as possible.