>>11081507This.
There are things, whose complexity is such, that their explanation, in a readable and comprehensive format for humans, simply isn't possible.
You can water down your explanation, but there's a lower threshold below which that explanation is rubbish (0 meaning), and an upper threshold above which it is gibberish (as in, no one can understand what the fuck is happening, because it's too complex).
Those two thresholds, for any sufficiently complex problem, can be in such an order, that there's no Goldilocks zone where the human can even hope to understand anything of the data of the trained model.
We're talking about millions if not billions of parameters, associated in EVEN MORE possible ways, how the hell do you want anyone to learn anything from that much information?
The way we should deal with that problem, is by restricting our NN in clever ways, as to reduce the overall complexity of the processes involve.
Bring down the problem to our level, without tampering with the efficiency of the program.
Convolution layers are such an example of how we should temper with our NN.
They're easy to understand and, even better, they improve our NN in an amazing way.
This is what machine learning should be all about. Not just relying on one mathematical concept such as the gradient descent to do all the work and shit out a "trained" model, but cleverly coming up and combining concepts in order to fill in the blanks of our roadmaps to our AIs.
We must connect the dots. You can't just draw a straight line between your starting point and your goal.
When people understand that, we will have avoided the third AI winter.