So it's a known problem in AI that even if you know every input and output coming out of every node of a neural network, you still can't interpret the information of the hidden layers in any meaningful way. You have a perfect real time map of what is happening there, but you can derive nothing useful from it.

If we have a perfect real time mapping of the human brain, knowing exactly how and when each neuron is activated, every input and output coming out of them, will we still have the black box problem?