/sci/ - Science & Math » Thread #14259242

114KiB, 1917x1079, 364286860.png

View Same Google iqdb SauceNAO

Anonymous Wed 23 Feb 19:42:37 2022 No.14259242 View Reply Original Report

Quoted By: >>14259248 >>14259279

>hebbian learning
Why is it not applied in modern neural networks if it captures how neurons work biologically? All good results are from back propagation variants. Is hebbian learning a meme theory? What's going on here?

Anonymous

Anonymous Wed 23 Feb 2022 19:44:23 No.14259248 Report

Quoted By:

>>14259242
Because you need to actually run the model on a computer. If it doesn't have an effective realization it's useless in practice. Regular backprop is very computationally efficient.

Anonymous

Anonymous Wed 23 Feb 2022 19:46:14 No.14259254 Report

Quoted By:

Why don't we use nano processors that assemble and configure themselves in accordance with the needs of information flow? Cause we don't have them?

Anonymous

Anonymous Wed 23 Feb 2022 19:54:09 No.14259279 Report

Quoted By: >>14259467

>>14259242
Hebbian learning can basically be reduced to PCA, and we have faster ways to do PCA.
Perhaps more importantly, Hebbian learning doesn't have an error signal, but for most problems in machine learning we actually can create an error signal. In general its faster to learn with an error signal, if you have one available.
If you think about the brain too, its not just Hebbian learning happening. There are neurotransmitters and what not that are playing a role too, essentially acting like reinforcement signals. I'm really simplifying here, but you could kind of kind of something like Dopamine as adding a "direction" to the Hebbian learning that happens in the brain.

But all of that aside, the simplest answer is, backprop works really fucking well, and its really easy to use. So we just use that.

Anonymous

Anonymous Wed 23 Feb 2022 20:57:02 No.14259467 Report

Quoted By: >>14259545

>>14259279
What's even more confusing is that the weights can be somehow modified to human intelligence level. Here I assume babys are born with random weights in their brains so they are basically retarded at the start. Maybe they got good initializations from their genes so they were not completely dumb, but it's still amazing how local optimization turns out to reach a superb intelligence level. Perhaps they are born smart but lack knowledge, and all they need to do is use their smart neuron updating rules to fill the knowledge in their weights.

Anonymous

Anonymous Wed 23 Feb 2022 21:22:18 No.14259545 Report

Quoted By:

>>14259467
The extreme rapid learning in babies seems to be, at least partially, explained by the dramatic synaptic culling they undergo. Babies are born with a huge number of synapses and these quickly cull themselves down.
It seems that very rapid learning can be achieved in this way, throw spaghetti and the wall and see what sticks.

But yes it is incredible how local optimization can work so well. I would chalk most of this up to evolution and lots of little tricks that drive learning into certain directions. For example babies have an kind of innate attraction to staring at moving objects. It would seem that this was kind of hardwired in by evolution, because staring at moving objects is a great way to jump start your learning of intuitive physics, shapes of objects etc. The evolutionary hardwiring drives the attention, and then once you are paying attention the neurons can do their unsupervised learning.
This trick seems to be all over the place if you really closely look at human development and behavior. Even animals too.

Capcode	All Only User Posts Only Moderator Posts Only Admin Posts Only Developer Posts
Show Posts	All Only With Images Only Without Images
Deleted Posts	All Only Deleted Posts Only Non-Deleted Posts
Ghost Posts	All Only Ghost Posts Only Non-Ghost Posts
Post Type	All Only Sticky Threads Only Opening Posts Only Reply Posts
Results	All Grouped By Threads
Order	Latest Posts First Oldest Posts First

Your latest searches