This is true, but it's also because Tensorflow is a typed language, but it uses the syntax of python. (At least the default way) in tensorflow, the graph is type checked on startup, and it'll fail if anything is wrong.
Contrast this with pytorch, chainer, or tensorflow's dynamic computation graphs and they're much more likely to have a bug that happens later, since their graphs aren't verified up front.
Unfortunately, typed languages won't help you much there. A big reason people use pytorch is because of its flexibility (i.e. they were bumping up against the constraints of a static graph system and wanted out)
I admit I don’t have much experience with those, I’m a Keras and CNTK guy but the principles will be the same: marshal your data into a huge matrix of floats/one-hot and hand it off to training where it will spend 99.9% of it’s time.
I am a fan of strong/static typing and was once very active in the OCaml community but that just struck me as a very odd thing for the OP to say... it’s just not something that people doing DL worry about. It could be valuable in the marshalling phase but that all happens before DL begins and (in my experience) in a separate program.
Comments
This is true, but it's also because Tensorflow is a typed language, but it uses the syntax of python. (At least the default way) in tensorflow, the graph is type checked on startup, and it'll fail if anything is wrong.
Contrast this with pytorch, chainer, or tensorflow's dynamic computation graphs and they're much more likely to have a bug that happens later, since their graphs aren't verified up front.
Unfortunately, typed languages won't help you much there. A big reason people use pytorch is because of its flexibility (i.e. they were bumping up against the constraints of a static graph system and wanted out)
I admit I don’t have much experience with those, I’m a Keras and CNTK guy but the principles will be the same: marshal your data into a huge matrix of floats/one-hot and hand it off to training where it will spend 99.9% of it’s time.
I am a fan of strong/static typing and was once very active in the OCaml community but that just struck me as a very odd thing for the OP to say... it’s just not something that people doing DL worry about. It could be valuable in the marshalling phase but that all happens before DL begins and (in my experience) in a separate program.