>>12686575I'm gonna answer with some thoughts on building the simplest "AI as pop-sci portrays it"
It would be stacking architectures more than anything
GANs with style or similarly transfer-learning with RNNs are an approach that I think will go far when applied more widely- that is, training a network, freezing the weights, then adding another layer on top of it
Adversarial networks are a great design; I'd like to see some form of dynamic, multiple VAE/RNN generators with adversarial networks pruning shitty learners/keeping the salient information, freezing weights, and incorporating them into larger networks
I think we'll se some really interesting emergent properties, the problem and reason no one has done it yet is because its hell to learn. I have some dual-RNN architectures that take more time to get the spark session + GPU learning right than any other part of the process, and that's just so I can actually get it to learn in <1 week.
Stacking architectures will just require 1) small datasets, and 2) a lot of hyperparameter optimization (especially to avoid exploding gradient with any RNN type stuff; yes, even LSTMs and GRUs explode and I have to tweak the learning rate on some models), although while we are at it, building an RNN generator (I encompass LSTM/GRUs in the overall RNN name) that uses different hyperparameters, and get pruned by an adversarial network would be fun.
CNNs for image inputs to represent vision, encoder/decoder with attention for NLP to represent hearing, VAE latent space encoding and specific RNN weights for memory, and generate from those spaces when queried by another part of the network, etc.
I think it will be linking together generic program logic with stacked architectures to represent different circuits that will lead us to the first spooky emergent properties of the system, where it does something "human like" that wasn't expected from the individual components along
