You kind of are claiming they're minimal, though. Because if they're not, your statement that "if you found any patterns in there, you could exploit the regularity..." implies nothing. Yeah, the patterns are there, and people are exploiting them.
Your socioeconomic argument just doesn't hold either. People don't delay releasing models until they've minimized it to the theoretical limit. They ship it when it's good enough for whatever job they're making it for.
No kidding. Clearly there's some redundancy, or however you want to frame it, in real-world released models. That still means your argument for why they can never be interpretable is based an incorrect premise.
Comments
You kind of are claiming they're minimal, though. Because if they're not, your statement that "if you found any patterns in there, you could exploit the regularity..." implies nothing. Yeah, the patterns are there, and people are exploiting them.
Your socioeconomic argument just doesn't hold either. People don't delay releasing models until they've minimized it to the theoretical limit. They ship it when it's good enough for whatever job they're making it for.
If somebody finds patterns in them, that means they can be improved by reducing the patterns. Patterns and symmetry are forms of redundancy.
The less memory the weights take to meet a level of competency, the more random, and therefore patternless and inscrutable they will be.
No kidding. Clearly there's some redundancy, or however you want to frame it, in real-world released models. That still means your argument for why they can never be interpretable is based an incorrect premise.