The type safety argument is total BS. First of all the training script will fail for the very first time if there is a type error. You'd be a moron to pass an argument of a different type 'a couple of hours' into the training. No sane programmer writes such code. What kind of nonsensical argument is this.
What I have found static typing to be really useful for is in remembering what I have coded. It's quite hard to remember a dynamic type while you are writing code, given the number of variables you are dealing with. Seeing that type definition next to your variable name is a handy reference. I find it helpful to speed up coding a bit and being able to remember a lot more clearly what I have done.
Static typing is also a nice way to communicate design intentions. But for this to work, the annotations have to be very expressive.
I don’t know the first thing about OCaml, but I have worked professionally with Haskell and static typing is a joy when it adds clarity and makes the contracts of functions instantly readable.
Contrast this with Scala, which I have also worked with professionally and the difference is stark. Scala type annotations are much harder to read, and the mechanism of implicits can make for extremely mysterious code that looks like it shouldn’t compile and only once you track down some distant implicit that’s somehow in scope, can you make sense of the way types are flowing through some function contract.
Yep agreed. Though you can communicate design intentions in comments, no ? There is another argument that programmers might not follow the comments, so strict enforcement by types helps - I don't believe in such a philosophy though. Most programmers will do the right thing and mistakes are not intentional.
It's not that they don't follow the comments, it's the fact that comments aren't executable, so they rot. People forget to update them. That can't happen with type definitions, because your code stops compiling.
Sure, comments serve their purpose, but that purpose only slightly overlaps with that of static types.
You'd be a moron to pass an argument of a different type 'a couple of hours' into the training
Huh?! At least in non-ML code this happens all the time, data fetched by whatever thinggie that uses zillion chained libraries of code nobody has time to audit, comes in hours or days late in a long running service blowing it up... eg. "oops, point.x is now no longer and integer but more like a map[ErrorObject->vector[int]]" bc something blew up in a very unexpected way in some other nodejs code light years away from the business logic you hold in your head... (yeah, the service gets restarted, but at some point some data that should have been saved in the DB hasn't been am may need to be recovered manually from some obscure log if even recoverable)
Yeah I should have been more explicit that this comment is in reference to ML code. Training is nothing but a loop so its unlikely you pass type A in iteration 1 and type B in iteration 200. If that happens most likely your training data is messed up and type safety cannot help you as you would have compiled and tested for type A.
You could also have a complex training job that trains a shallow model for a while, then uses it to train a deeper model, or extract an embedding from some layer and train a classical prediction model that uses the embedding as the feature vector.
But the point is that the right way to ensure safety is with realistic fixture-based integration testing. That’s not what static typing is for in that type of use case and is not a de facto benefit of static typing.
> What I have found static typing to be really useful for is in remembering what I have coded. It's quite hard to remember a dynamic type while you are writing code, given the number of variables you are dealing with. Seeing that type definition next to your variable name is a handy reference.
If you consider this a benefit, then (for example with Python), don't you get the same benefit just by using docstrings? Stated another way, if all you want is a visual cue about what you're passing to a function and getting returned, why bother with all the scaffolding of type safety? You can get that just by using documentation facilities outlined in various languages' style guides. Those language facilities (such as docstrings) tend to be very useful and a good engineering practice in general.
The point of type safety is actually to obviate what you're talking about. Smart developers can and do make the mistakes you're saying only a moron would make, regardless of available visual cues. Offloading that decision making process to a language that complains when you make that mistake instead of being forgiving about it is entirely the point.
So I guess what I'm saying is that I'm struggling to understand why you think type safety is BS. If I read you correctly, it sounds like you'd also say that developers committing memory corruption vulnerabilities are morons, and that the scaffolding of memory management and garbage collection is BS. Why not just have explicit references a developer can read while coding to make sure they're not overflowing a container, right?
Comments
The type safety argument is total BS. First of all the training script will fail for the very first time if there is a type error. You'd be a moron to pass an argument of a different type 'a couple of hours' into the training. No sane programmer writes such code. What kind of nonsensical argument is this.
What I have found static typing to be really useful for is in remembering what I have coded. It's quite hard to remember a dynamic type while you are writing code, given the number of variables you are dealing with. Seeing that type definition next to your variable name is a handy reference. I find it helpful to speed up coding a bit and being able to remember a lot more clearly what I have done.
Static typing is also a nice way to communicate design intentions. But for this to work, the annotations have to be very expressive.
I don’t know the first thing about OCaml, but I have worked professionally with Haskell and static typing is a joy when it adds clarity and makes the contracts of functions instantly readable.
Contrast this with Scala, which I have also worked with professionally and the difference is stark. Scala type annotations are much harder to read, and the mechanism of implicits can make for extremely mysterious code that looks like it shouldn’t compile and only once you track down some distant implicit that’s somehow in scope, can you make sense of the way types are flowing through some function contract.
Yep agreed. Though you can communicate design intentions in comments, no ? There is another argument that programmers might not follow the comments, so strict enforcement by types helps - I don't believe in such a philosophy though. Most programmers will do the right thing and mistakes are not intentional.
It's not that they don't follow the comments, it's the fact that comments aren't executable, so they rot. People forget to update them. That can't happen with type definitions, because your code stops compiling.
Sure, comments serve their purpose, but that purpose only slightly overlaps with that of static types.
Huh?! At least in non-ML code this happens all the time, data fetched by whatever thinggie that uses zillion chained libraries of code nobody has time to audit, comes in hours or days late in a long running service blowing it up... eg. "oops, point.x is now no longer and integer but more like a map[ErrorObject->vector[int]]" bc something blew up in a very unexpected way in some other nodejs code light years away from the business logic you hold in your head... (yeah, the service gets restarted, but at some point some data that should have been saved in the DB hasn't been am may need to be recovered manually from some obscure log if even recoverable)
in some other nodejs code
ML/DL is nothing at all like webdev :-) but these days you can compile OCaml to JavaScript if you want, I encourage you to check it out
Yeah I should have been more explicit that this comment is in reference to ML code. Training is nothing but a loop so its unlikely you pass type A in iteration 1 and type B in iteration 200. If that happens most likely your training data is messed up and type safety cannot help you as you would have compiled and tested for type A.
You could also have a complex training job that trains a shallow model for a while, then uses it to train a deeper model, or extract an embedding from some layer and train a classical prediction model that uses the embedding as the feature vector.
But the point is that the right way to ensure safety is with realistic fixture-based integration testing. That’s not what static typing is for in that type of use case and is not a de facto benefit of static typing.
> What I have found static typing to be really useful for is in remembering what I have coded. It's quite hard to remember a dynamic type while you are writing code, given the number of variables you are dealing with. Seeing that type definition next to your variable name is a handy reference.
If you consider this a benefit, then (for example with Python), don't you get the same benefit just by using docstrings? Stated another way, if all you want is a visual cue about what you're passing to a function and getting returned, why bother with all the scaffolding of type safety? You can get that just by using documentation facilities outlined in various languages' style guides. Those language facilities (such as docstrings) tend to be very useful and a good engineering practice in general.
The point of type safety is actually to obviate what you're talking about. Smart developers can and do make the mistakes you're saying only a moron would make, regardless of available visual cues. Offloading that decision making process to a language that complains when you make that mistake instead of being forgiving about it is entirely the point.
So I guess what I'm saying is that I'm struggling to understand why you think type safety is BS. If I read you correctly, it sounds like you'd also say that developers committing memory corruption vulnerabilities are morons, and that the scaffolding of memory management and garbage collection is BS. Why not just have explicit references a developer can read while coding to make sure they're not overflowing a container, right?