/sci/ - Science & Math » Thread #10145233

88KiB, 1199x364, DsG4fYAVsAA32EZ.jpg

View Same Google iqdb SauceNAO

Training times in Machine Learning over time

Anonymous Fri 16 Nov 15:59:36 2018 No.10145233 View Reply Original Report

Quoted By: >>10146354

https://arxiv.org/abs/1811.05233

First training time example is from 2016
Latest lowest training time example is recent

30 hours -> 3.7 minutes
~500x faster

Machine learning is still delivering results

from https://twitter.com/karpathy a person I follow

Anonymous

Anonymous Fri 16 Nov 2018 16:46:28 No.10145334 Report

Quoted By: >>10145418 >>10145442

Neural nets are going to serve as the basis for a lot of future technology, but progress will be gradual and incremental simply because there is a lot of work to be done coming up with network architectures and training approaches. This makes redditors mad because it fits neither extreme (AI will make humans obselete in 2 weeks!/AI is useless and will never happen).

Prove me wrong.

Anonymous

Anonymous Fri 16 Nov 2018 17:28:37 No.10145418 Report

Quoted By:

>>10145334
>but progress will be gradual and incremental simply because there is a lot of work to be done coming up with network architectures and training approaches
Everything is achieved slowly and step by step.
Most people now just want the best a technology can offer on the first day after it is found.

Anonymous

Anonymous Fri 16 Nov 2018 17:41:30 No.10145442 Report

Quoted By: >>10145503

>>10145334
You are right. But, there will be new technology. Current DNN have serious problems, and all the study going on is avoiding them, which will not be fully avoided untill something changes in more fundamental level. One big problem is incremental aspect of learning.

Anonymous

Anonymous Fri 16 Nov 2018 18:10:50 No.10145503 Report

Quoted By: >>10146205

>>10145442
It's not a big problem if you understand how intelligence works.

You see, our low level structures are inherited and not trained "quickly". The quick learning happens only in certain parts of the brain that operate on high levels using memory and other tricks.

You wont ever get a faster backprop technique.

Anonymous

Anonymous Fri 16 Nov 2018 22:53:54 No.10146205 Report

Quoted By: >>10146221 >>10146234

>>10145503
This. Our brain is basically a sparse and extremely deep net shaped by evolution. Most of the shit like the basic structure required for image processing, sound processing, sensory integration, etc is "hard coded" in the structure of the network. You're not taking full matrices of initial values, you already start with a physical net that just needs adjustment and tuning.

Looking at the way actor models are developing I think AI is going down much the same path. People figuring out what architectures work and patching the model together from smaller parts built and trained to handle specific operations. The initial craze about just letting a network of dense matrices learn on its own is obviously incredibly difficult to shape into anything practical.

Anonymous

Anonymous Fri 16 Nov 2018 23:00:49 No.10146221 Report

Quoted By:

>>10146205
Yeah, it's pretty easy to see. It's not like our NN is changing entirely when we sort through images ourselves. We just make some mental notes, imagine the different categories, and then can easily do it.

The NN isn't really making huge changes. We are just doing different things at the highest level of the brain. Memory and imagination is heavily involved for "fast" training.

The next stage is getting imagination, memory, attention, and integrating RL all into it. You can call synthetic data creation imagination as well. Even if it's clunky.

Anonymous

Anonymous Fri 16 Nov 2018 23:04:59 No.10146234 Report

Quoted By: >>10146290

>>10146205
This is true. People blind from birth smile... I’d say a majority of stuff you do in your daily life are built in

Anonymous

Anonymous Fri 16 Nov 2018 23:26:02 No.10146290 Report

Quoted By:

>>10146234
anyone who has undergone an instinctual reaction to spotting some threat knows this. AKA spider/scorpion/snake etc

It's a very unique experience, same as say why we are attracted to certain physical traits.

Anonymous

Anonymous Fri 16 Nov 2018 23:55:56 No.10146354 Report

Quoted By: >>10146359

>>10145233
Most of these advances aren't really "machine learning" though

Anonymous

Anonymous Fri 16 Nov 2018 23:57:37 No.10146359 Report

Quoted By: >>10146371

>>10146354
?

computation power is such a huge requirement of all algos currently though

Anonymous

Anonymous Sat 17 Nov 2018 00:04:06 No.10146371 Report

Quoted By:

>>10146359
Yes, but advances in machine learning aren't driving this result, it's the millions of dollars in hardware

Capcode	All Only User Posts Only Moderator Posts Only Admin Posts Only Developer Posts
Show Posts	All Only With Images Only Without Images
Deleted Posts	All Only Deleted Posts Only Non-Deleted Posts
Ghost Posts	All Only Ghost Posts Only Non-Ghost Posts
Post Type	All Only Sticky Threads Only Opening Posts Only Reply Posts
Results	All Grouped By Threads
Order	Latest Posts First Oldest Posts First

Your latest searches