>>11417603This is true.
Theoretical work is advancing because of this though, while it was always outpaced before, it no longer is.
Maybe this will lead to cool stuff.
>>11416540The general idea is similar to how stat research works. Either you have a problem you want to model, so you collect data and setup a model which you believe might be able to express information contained in the data that you're interested in, or you're trying to figure out methods to make the process work better, such as new tests, new models, new optimization methods, etc. However there are many weird mysteries in deep learning, such as how it's working so much better than any theoretical bounds predict, or how it can work so well despite the loss surface being so non-convex while gradient descent is convex. This enables another axis, which is an attempt at describing these phenomena, as described (very poorly) in
>>11416581 
