/sci/ - Science & Math » Thread #13797391

1020KiB, 825x1021, 1633976864856.png

View Same Google iqdb SauceNAO

Anonymous Thu 28 Oct 10:23:47 2021 No.13797391 View Reply Original Report

Quoted By: >>13797396 >>13798790 >>13799000 >>13799039 >>13799484 >>13800576 >>13801189 >>13802804 >>13805057 >>13806263 >>13806530 >>13806939 >>13809654 >>13809874

how do neural networks and machine self-learning work and how does it differ from traditional AI?

Anonymous

Anonymous Thu 28 Oct 2021 10:25:55 No.13797395 Report

Quoted By:

its all just classification regression lmao

Anonymous

Anonymous Thu 28 Oct 2021 10:27:24 No.13797396 Report

Quoted By: >>13797398 >>13803553 >>13809694

>>13797391
fully connected neural network works by passing through your sample (feedforward), then you get a bunch of values for your nodes and an error in your output node. After that you propogate the error from your last layer back through your layer (backpropogation) and acquire weights to update your network with. Conceptually, it is quite easy to understand and the math isnt hard either.

Anonymous

Anonymous Thu 28 Oct 2021 10:28:12 No.13797398 Report

Quoted By: >>13797405 >>13797409 >>13797414

>>13797396
but why do people talk about it as though it's categorically different from all the AI that came before it?
isn't it all still just algorithms?

Anonymous

Anonymous Thu 28 Oct 2021 10:33:59 No.13797405 Report

Quoted By: >>13797410 >>13800595 >>13801246

>>13797398
a FCNN can solve any classification task. This follows from the universal approximation theorem and is the reason nerual networks are fundamentally more interesting than any other classifcation method

Anonymous

Anonymous Thu 28 Oct 2021 10:35:37 No.13797409 Report

Quoted By:

>>13797398
http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Its different because it's a way to get vastly better performance using simple statistical methods than can be achieved by putting human knowledge into the code directly.

Anonymous

Anonymous Thu 28 Oct 2021 10:36:55 No.13797410 Report

Quoted By:

>>13797405
OP is asking about NN vs older AI not about NN vs older statistical methods lmao

Anonymous

Anonymous Thu 28 Oct 2021 10:42:50 No.13797414 Report

Quoted By: >>13797461

>>13797398
It doesn’t suffer from the bias-variance tradeoff like the others.

Anonymous

Anonymous Thu 28 Oct 2021 11:11:12 No.13797461 Report

Quoted By: >>13797778

>>13797414
Pretty sure it does, it can just deal with much higher dimensional features with high cross entropy.

Anonymous

View Same Google iqdb SauceNAO 3E76EE87-C7E4-4961-87D3-2ADCBCC5 (...).jpg, 580KiB, 2208x997

Anonymous Thu 28 Oct 2021 14:23:37 No.13797778 Report

Quoted By: >>13798986 >>13799470 >>13800613 >>13800620

>>13797461
The latest analyses say different.

https://arxiv.org/pdf/1812.11118.pdf

Anonymous

Anonymous Thu 28 Oct 2021 20:00:40 No.13798790 Report

Quoted By: >>13798856 >>13798865

>>13797391
You're looking at a creature.
It has an orange beak, and you have a neuron that has learned there's a 30% chance if it has an orange beak that it's a duck.
It has webbed feet, and you have a neuron that has learned if it has webbed feet theres a 30% chance it's a duck.
It quacks, and you have a neuron that has learned if it quacks theres a 95% chance it's a duck.

There's a neuron in your brain that measures "duckness" among other things and its connected to all of these neurons.
If it has all of features, it has a 99% chance it's a duck.
You interpret it as a duck with a reasonable confidence.

It's all a huge weighted graph of trillions of probabilities weighing upon each other.
And not just one-directionally, they loop back into each other and recurse and sometimes have errant connections.

Anonymous

Anonymous Thu 28 Oct 2021 20:24:06 No.13798856 Report

Quoted By: >>13798894

>>13798790
The brain doesnt work like that. A neuron doesnt know anything, much less a concept like duckness nor what a probability is.

Anonymous

Anonymous Thu 28 Oct 2021 20:29:01 No.13798865 Report

Quoted By: >>13798894

>>13798790
This is your brain on tech-company hype. The brain does not work that way at all, the entire process as you described does not occur in nature.
Not that genuine AI has to work like a brain at all, but come on, stop with the blatantly bad info.

Anonymous

Anonymous Thu 28 Oct 2021 20:39:03 No.13798894 Report

Quoted By: >>13798997

>>13798856
>>13798865
Somebody with assumedly no knowledge of neural networks asked how neural networks.
So I gave a simplified description.

Two more people with obviously no knowledge of neural networks objected.

Anonymous

Anonymous Thu 28 Oct 2021 21:10:24 No.13798986 Report

Quoted By:

>>13797778
Oh my. That's very interesting. It actually does make a bit of sense as well.

I wonder if there's a connection between this and splining. A lot of standard models (regression for example) are used because straight up polynomial interpolation is unstable and does not converge for many situations. But this is also why splines exist.
You can obviously spline things for classical datasets (people do splined regressions all the time), but I wonder if this just means that machine learning algorithms with many parameters are essentially finding the stable set of splines for a dataset (which would get you over the classical variance bias U)

Anonymous

Anonymous Thu 28 Oct 2021 21:15:45 No.13798997 Report

Quoted By: >>13799030

>>13798894
Grandmother neurons don't exist

Anonymous

Anonymous Thu 28 Oct 2021 21:16:25 No.13799000 Report

Quoted By: >>13799470 >>13800599

>>13797391
Matrix multiplication

Anonymous

Anonymous Thu 28 Oct 2021 21:30:57 No.13799030 Report

Quoted By: >>13800582

>>13798997
Right, but i'm trying to explain how to visualize the basic mechanic of a neural network.

And a "grandmother neuron" is an easier concept to extrapolate from than saying describing "duckness" as another arbitrary cloud of recursive "semi-duckness" neurons that flow off in another trillion directions

I can just describe that cloud of "duckness" as a singular entity.
You know.
For the sake of simplicity.
For people that don't really understand neural networks and want to understand better.
Like this thread.

Anonymous

View Same Google iqdb SauceNAO 1634788095816m.jpg, 135KiB, 1024x692

Anonymous Thu 28 Oct 2021 21:36:25 No.13799039 Report

Quoted By:

>>13797391
Lmao thats the unwritten contract when you're born. Hey this is life, you were somewhere before but now you're here. You'll go somewhere else once you die. But you're here now. So have fun. Or don't.

Anonymous

Anonymous Fri 29 Oct 2021 01:07:43 No.13799470 Report

Quoted By: >>13800569

>>13799000
"Math done by a computer". Nice numbers tho.
>>13797778
See also:
https://openai.com/blog/deep-double-descent/
And
https://www.google.com/url?q=https://mathai-iclr.github.io/papers/papers/MATHAI_29_paper.pdf

Anonymous

Anonymous Fri 29 Oct 2021 01:11:37 No.13799484 Report

Quoted By:

>>13797391
something something minimizing error

Anonymous

Anonymous Fri 29 Oct 2021 03:04:19 No.13799696 Report

Quoted By:

AI that have neural networks have neural networks. Machine learning is just you running a computer over and over again trying to optimize the problem based on the input. Traditional AI don't have these things by definition. They may do some raw brute forcing or guided brute forcing, state machines, do some fancy statistics, and predefined output by input. Obviously in a pure machine learned system you don't have any of these things and the machine learns itself.

Anonymous

Anonymous Fri 29 Oct 2021 12:51:38 No.13800569 Report

Quoted By:

>>13799470
Do you want to get into the details of forward propagation (leterally matrix multiplication with a sigmoid or other similar transformation to binary output) and backward propagation (tuning of parameters in the matrices) or are you looking for the mathematical ideas behind it? Because there's sufficient resources on forward/back propagation if you literally just Google training a neutral network

Anonymous

Anonymous Fri 29 Oct 2021 12:55:35 No.13800576 Report

Quoted By:

>>13797391
Neural nets are just constrained optimization problems with millions of variables. Feed it enough data and algorithms exist to find such optima. Classic AI on the other hand doesn't use any of that shit and is rather simple code that is limited by the hardware it's controlling. Tons of nested if xyz then ABC. This is in contrast to neural nets which say there is a defined minimum cost function, here's a million variables and the data to determine it.

Anonymous

Anonymous Fri 29 Oct 2021 13:00:33 No.13800582 Report

Quoted By: >>13800806 >>13809626

>>13799030
>For people that don't really understand neural networks and want to understand better.
You mean like you. It's just linear algebra, and that's it. That's literally all there is to it. Just linear algebra with matrices that have extremely large dimensions. It has nothing to do with brains. And using an analogy to compare it brains highlights how stupid you are and how you understand neither brains nor neural nets. Fun fact: the neuronal model using a voltage spike was a mathematical description of how the brain worked (and was clearly imperfect). Then ML arises and models their nets after the "neuron" claiming to be modeled after the brain... No it's modeled after a model of the brain. Lo and behold, a neural net capitulated the features of the mathematical neuronal model! Golly gee willikers look at that! Who could have possibly guessed that? It's not how brains work in the slightest.

Anonymous

Anonymous Fri 29 Oct 2021 13:12:45 No.13800595 Report

Quoted By:

>>13797405
>a FCNN can solve any classification task.
So can typing in numbers at random.

Anonymous

Anonymous Fri 29 Oct 2021 13:14:50 No.13800599 Report

Quoted By: >>13800824

>>13799000
no its like... layers instead, what you think this is for. but there are also those

Anonymous

Anonymous Fri 29 Oct 2021 13:25:30 No.13800613 Report

Quoted By: >>13800620

>>13797778
HAven't read anything yet, but that pic just really seems like "IT WILL WORK WITH MORE DATA AND PARAMETERS I ASSURE YOU PLEASE KEEP MY FUNDING DESPITE THE FACT THAT WE'RE STUCK ON EFFICACY"

Anonymous

Anonymous Fri 29 Oct 2021 13:30:41 No.13800620 Report

Quoted By: >>13800628 >>13800708 >>13801200 >>13803315

>>13797778
>>13800613
OK, read it diagonally, isn't this kind of an obvious thing that everyone talks about since forever?
I even remember François Chollet (name??) on Lex Fridman briefly mentioning this.

If your training space is sufficiently dense and you sufficient nodes you might get to a point where any given data point to pass on your hypothesis function will be calculated by interpolation, in the (impossible) limit, every possible point would already be considered in your hypothesis and it would be omnisicient. THis is obviously impossible because there are uncountably infinite possible data points and you cannot gather uncountably infinite data point to train your hypothesis.

Anonymous

Anonymous Fri 29 Oct 2021 13:33:13 No.13800628 Report

Quoted By: >>13800633

>>13800620
>in the (impossible) limit,
It's as though they don't care about having an actual theoretical basis. "A FCNN can solve any classification task" means nothing since so can monkeys on typewriters. Generating solutions at random would accomplish the same thing.

Anonymous

Anonymous Fri 29 Oct 2021 13:37:34 No.13800633 Report

Quoted By: >>13800639

>>13800628
They also assume they can converge to a solution to the inference problem proposed by the network architecture, and obviously there are infinite many cases where they cannot. Specially with GrAdIeNt DeScEnT.

I am not against using simple and standard methods to try and see if it works on practical problems, if it does: GREAT!

What I fucking hate are these fags always trying to convince themselves that they can "prove" that they will solve everything.

Anonymous

Anonymous Fri 29 Oct 2021 13:40:54 No.13800639 Report

Quoted By:

>>13800633
>converge to a solution to the inference problem proposed by the network architecture, and obviously there are infinite many cases where they cannot. Specially with GrAdIeNt DeScEnT.
I don't know what that is, especially the second half. All I know is it doesn't seem to tend or scale efficiently without diminishing returns towards being able to do certain things humans can do intelligently.

Anonymous

Anonymous Fri 29 Oct 2021 14:21:34 No.13800708 Report

Quoted By: >>13800738

>>13800620
This is actually not guaranteed. Maybe with ML techniques it's guaranteed, but for the general interpolation problem that is simply not true.

For many problems, interpolation will not converge unless the underlying structure of the interpolating function is similar to the true function or nodes are chosen carefully. That's, personally, why I find this double descent factor interesting.

In simple standard regression, the reason you stop at the optimal bias variance trade off is because with more parameters you typically reach issues with polynomials and interpolation. Like interpolation on a data set has no guarantee of convergence, regardless of how much information is provided. You could provide an infinite amount of information with Infinite parameters and still not reach convergence (and in fact more information and more parameters typically leads to Wilder fluctuations of the interpolation). There are methods around this though, such as using carefully selected nodes or splines.

I can only assume, based on my limited knowledge of ML, that essentially similar workarounds to the interpolation problem are being found and solved in the process.

Anonymous

Anonymous Fri 29 Oct 2021 14:33:28 No.13800738 Report

Quoted By:

>>13800708
No, you're overestimating the capacities of the niggers who wrote that preprint.
They do not consider interpolation convergence at all, that's not their problem. They consider that their inference algorithm converges to an interpolating behaviour given enough data, and that's why it's supposed to become infinitely better with more data.

For me, it's just cope, since we clearly see in practice tons of problems that never get better results just because you throw more data at them (Tesla autodriving is an infamous example).

Anonymous

Anonymous Fri 29 Oct 2021 15:03:10 No.13800806 Report

Quoted By: >>13800809

>>13800582
Yeah dude.
Neural Networks are "just linear algebra".
That's really useful to people with no clue.

I can't even tell what you're mad about.
I never even said it was "exactly" like a human brain.
In fact I said I was simplifying it to help illustrate.

>It's not how brains work in the slightest.
It literally is how brains work "in the slightest".
Take your elitist head out of your ass.

Anonymous

Anonymous Fri 29 Oct 2021 15:05:58 No.13800809 Report

Quoted By: >>13800827 >>13803320

>>13800806
>I never even said it was "exactly" like a human brain.
Putting quotes around exactly doesn't make you any more correct. You're wrong, simply put. Plus, when I say NNs are just linear algebra, you admit you don't know how that helps you (implying you don't know what they are). Hence how can you claim that they're how brains work? This is very simple logic.
>It literally is how brains work "in the slightest".
Source from a neuroscientist?

Anonymous

Anonymous Fri 29 Oct 2021 15:12:14 No.13800824 Report

Quoted By:

>>13800599
In case you're the retard who doesn't know anything about linear algebra here, the number of layers in a neutral network is just the number of matrices you need to multiply to get the output.

Anonymous

Anonymous Fri 29 Oct 2021 15:14:23 No.13800827 Report

Quoted By: >>13800831 >>13800833 >>13803331

>>13800809
You've got brain damage my guy.
Go out and socialize.

The purpose of thread is to describe neural networks to somebody who doesn't understand them.
Just telling them "linear algebra" doesn't explain or visualize anything.

You have to have literal autism if you can't understand that.

Literally any neuroscientist will tell you yes neural networks are modeled after how neurons work.
It's a model.
It's not exactly how a brain works.
But obviously because it's modeled after how neurons work, it does in fact resemble a brain or neuron "in the slightest".

We're looking at a pixel?
Somebody asks what color it is, I tell them purple, and you're getting mad because ACKSHUALLY it's a little red light next to a blue light.

Anonymous

Anonymous Fri 29 Oct 2021 15:17:19 No.13800831 Report

Quoted By: >>13800834

>>13800827
Literal soience mentality. You're the kind of pseud like NDT who says we're stardust then everyone goes WHOA SO DEEP while not actually learning anything at all.
>modeled after how neurons work
And as I explained, the neuronal model of brains has little to nothing to do with how the brain actually works. Maybe if you knew what you were talking about, and weren't just parroting bullshit from pop scientists you see on YouTube, you'd realize this.
>a pixel has multiple colors
Are you fucking stupid?

Anonymous

Anonymous Fri 29 Oct 2021 15:18:11 No.13800833 Report

Quoted By:

>>13800827
It gives you a starting point to go read something or take a (notoriously easy) math class to learn. If you aren't looking to learn and just want to be spoon fed information, go somewhere else

Anonymous

Anonymous Fri 29 Oct 2021 15:18:12 No.13800834 Report

Quoted By:

>>13800831
Best of luck my guy

Anonymous

Anonymous Fri 29 Oct 2021 17:40:12 No.13801189 Report

Quoted By: >>13801697

>>13797391

Artificial intelligence is a misnomer. To recreate something you have to know what it is. At the end of the day its just another human extension to be marketed, hyped, and sold middle quartile intelligence humans by the upper quartile intelligence humans.

Anonymous

Anonymous Fri 29 Oct 2021 17:47:02 No.13801200 Report

Quoted By: >>13801220

>>13800620
François Chollet is pozzed
https://twitter.com/fchollet/status/816040451799207937

Anonymous

Anonymous Fri 29 Oct 2021 17:57:00 No.13801220 Report

Quoted By:

>>13801200
>pokemon picture
Why does he have to be woke when he does this?

Anonymous

Anonymous Fri 29 Oct 2021 18:02:40 No.13801234 Report

Quoted By: >>13801277 >>13812698

Neural networks honestly seem magical to me. Is there a proof of why they work better than other ML approaches. Even CMU's ML course states that most traditional ML methods have been rendered obsolete by neural networks.

Anonymous

Anonymous Fri 29 Oct 2021 18:07:41 No.13801246 Report

Quoted By:

>>13797405
>a FCNN can solve any classification task. This follows from the universal approximation theorem and is the reason nerual networks are fundamentally more interesting than any other classification method
Gaussian processes can model any smooth function also. Really the only reason for the preference for neural networks is because we just have noticed that they work better.

Anonymous

Anonymous Fri 29 Oct 2021 18:17:59 No.13801277 Report

Quoted By:

>>13801234
Uh, maybe? I mean it essentially makes sense, all neural networks do is define basis functions for your final regression.

The standard regression process has YOU defining the basis functions and regressors or iterating over groupings of potential basis functions and regressors and finding the best fit for your data. Neural Networks instead just 'build the best'.

Also it's baloney that it has rendered other techniques obsolete. Sure for complex data where the interactions aren't fully understood it's best, but if the relationship is simple, or well understood by a field, regression techniques are preferred. Most sciences are doing linear or nonlinear regressions for their models because they form their model (i.e. their basis functions and regressors) independent of ML/Statistical methods/neural nets. Often time, in these situations, the coefficients/parameters are much more important than predictions and makes neural networks useless (as it's more difficult to reason out what the final coefficients/parameters mean in a NN).

Anonymous

Anonymous Fri 29 Oct 2021 20:47:56 No.13801697 Report

Quoted By:

>>13801189
>To recreate something you have to know what it is.
"Artificial" is in the name nigmire. Therefore it's not a misnomer.

Anonymous

Anonymous Sat 30 Oct 2021 06:16:33 No.13802804 Report

Quoted By: >>13803170 >>13803345

>>13797391
How long before neural networks plateau. The last 6 years have been explosive for them but I believe it is about time we stopped.

Anonymous

Anonymous Sat 30 Oct 2021 11:12:17 No.13803170 Report

Quoted By: >>13803435 >>13803479 >>13803487

>>13802804
>How long before neural networks plateau.
There’s no end in sight. They keep improving. Nvidia and Microsoft recently released a 530 billion parameter model and they said they’re planning for models with at least 100 trillion parameters this decade. For reference, that’s about the same as the number of synapses in a human brain.

Anonymous

Anonymous Sat 30 Oct 2021 12:42:20 No.13803315 Report

Quoted By:

>>13800620
Small pedantry: If the math is done on a digital computer then the space is countable and finite.

Anonymous

Anonymous Sat 30 Oct 2021 12:46:50 No.13803320 Report

Quoted By:

>>13800809
U r def modeling NN correct as far as I can tell, but u do not seem to be modeling other humans correct like.

Anonymous

Anonymous Sat 30 Oct 2021 12:53:19 No.13803331 Report

Quoted By: >>13803410

>>13800827
It was kinda annoying how u said
>There's a neuron in your brain that measures "duckness"
Without qualification... you are explaining neuroscience wrong while explaining NN correct like afaik based on colah's research.
I do like your explanation tho. It's a lot better then linear algebra fag, who could be more pro social by saying
>if you really want to understand NN you must have a grasp of linear algebra

Anonymous

Anonymous Sat 30 Oct 2021 13:01:41 No.13803345 Report

Quoted By: >>13803487

>>13802804
I suspect this is like asking
>how long until computers plateau. The last 80 years have been explosive for them but I believe it is about time we stopped.
(Which is true desu)
With the amount of changing and building on top of the concept of a nn that goes on it seems likely that this might be a new paradigm upon which most future development will just build on top of.

Anonymous

Anonymous Sat 30 Oct 2021 13:41:13 No.13803410 Report

Quoted By: >>13803783

>>13803331
Those are different statements. Your modification of my statement permits the mysterious incorrect pseud shit about NNs. They arent black boxes. They are well understood as constrained optimization problems in linear algebra with billions of variables. That's all there is to it. These stupid terms like "training" and "supervised learning" and other nonsense do nothing but occlude the underlying mathematics of the algorithms. I suspect because of people like the pseud I'm responding to who barely understands what it does, runs a few lines of packaged code, and goes woah!

Anonymous

Anonymous Sat 30 Oct 2021 13:48:32 No.13803435 Report

Quoted By: >>13803661

>>13803170
There is an end. Look up how many millions of dollars Google spent just to train Alpha Zero. It's tens of millions for just a fractional improvement in performance. And despite throwing the kitchen sink at this problem, they STILL can't design algorithms to read language upside down, or to do very simple tasks such as recognizing what a sentence is if you remove all the vowels.
>ny hmn cn rd ths bt stt f th rt nrl nts tht cst tns f mllns cnnt
Google can increase their funding by an order of 100 to spend billions on these programs, and I guarantee you they will get only a marginal improvement. It doesn't matter how many nerve cells are in the brain... These neural nets work nothing like how the brain does.

Anonymous

Anonymous Sat 30 Oct 2021 14:14:47 No.13803479 Report

Quoted By:

>>13803170
Goddamn you weren't joking. But t.bh I'd like to see some improvements on the RL front too.

Anonymous

Anonymous Sat 30 Oct 2021 14:18:15 No.13803487 Report

Quoted By: >>13803499 >>13803546

>>13803170
https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/

Goddamn you weren't joking.

>>13803345
I wouldn't be surprised if advances in ML automate all low skill jobs in the next 50 years. DALL-E probably already threatens illustrators.

Anonymous

Anonymous Sat 30 Oct 2021 14:28:03 No.13803499 Report

Quoted By: >>13803509 >>13803546 >>13803594 >>13803874

>>13803487
>cost of operation both monetarily and environmentally conspicuously absent

Anonymous

Anonymous Sat 30 Oct 2021 14:32:50 No.13803509 Report

Quoted By: >>13803532 >>13803594 >>13803874

>>13803499
True. I believe we should reduce our dependence on neural networks because they are very computationally expensive which is unsustainable. A greater focus should be on development of alternate methods for these tasks that may probably not be as a versatile as neural networks but do better on individual tasks.

Anonymous

Anonymous Sat 30 Oct 2021 14:44:05 No.13803532 Report

Quoted By: >>13803546

>>13803509
Exactly. Even in the article the responses aren't that impressive. The classifier is fed a simple sentence and asked to verify the truth value. It spits out true both times with the justification being just regurgitating the sentence. There's nothing of substance there. Really, if people started calling these things classifiers and identifiers, rather than "artificial intelligence", and if we mentioned that the constraints leading to such classifications and identifications get optimized rather than "trained" or "learned", people would more easily see these limitations. There's nothing intelligent about these algorithms.

Anonymous

Anonymous Sat 30 Oct 2021 14:48:14 No.13803546 Report

Quoted By:

>>13803487
Me again
>>13803499
>>13803532
What bothers me most about this article is this line
>We live in a time where AI advancements are far outpacing Moore’s law.
This has absolutely nothing to do with Moore's law. Moore's law is merely a statement about the number of transistors in a CPU/GPU over time. Saying that these neural nets outpace Moore's law is akin to saying that Deep Blue's rocket architecture far outpace Newton's Law. It makes no sense and demonstrates a fundamental misunderstanding of the law you're citing. That's my biggest issue with all these neural nets and AI hype. They use all these terms without demonstrating an actual understanding of the topic, and then fluff up the very fundamental nature of such algorithms (which are simply classifiers) as "artificial general intelligence". It's absurd word games, and I don't understand why more people don't catch on to it.

Anonymous

Anonymous Sat 30 Oct 2021 14:50:19 No.13803553 Report

Quoted By:

>>13797396
feetforward* like a slide

Anonymous

Anonymous Sat 30 Oct 2021 15:02:00 No.13803594 Report

Quoted By: >>13803622

>>13803499
>>13803509
https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/

This might interest you.

Anonymous

Anonymous Sat 30 Oct 2021 15:11:32 No.13803622 Report

Quoted By:

>>13803594
Read it. Unamused. It's just more pandering. This guy also conspicuously leaves out cost of operation. Why do they hide it?

Anonymous

Anonymous Sat 30 Oct 2021 15:21:35 No.13803649 Report

Quoted By: >>13803886 >>13806521

In university we have such a great focus on neural networks, NLP, reinforcement learning but reproducing these experiments or making projects becomes difficult because of the limited computational resources we have as students. So I really would appreciate cheaper methods.

I have also heard that there is a growing sentiment at DeepMind that reinforcement learning can be the solution to all possible tasks that ML does now. Is this true?

Anonymous

Anonymous Sat 30 Oct 2021 15:25:31 No.13803661 Report

Quoted By: >>13803675

>>13803435
Depends on what you mean by plateauing. Right now there’s no reason to believe that if you did keep scaling exponentially they wouldn’t keep improving. Sure, even companies with a lot of money can’t throw exponentially increasing amounts of money at it, but it’s currently still the easiest way to do it and there are other ways to improve neural networks, such as experimenting with different architectures (the current ones such as transformers are used because they’re easily parallelizable and not specifically designed for these tasks) or providing them with different data (I don’t know anyone who was raised purely on reading text and there’s no reason to believe that given the same data as a human growing up it wouldn’t be able to become as functional as a human).

>These neural nets work nothing like how the brain does.
How do you know this?

Anonymous

Anonymous Sat 30 Oct 2021 15:31:36 No.13803675 Report

Quoted By: >>13805027

>>13803661
>How do you know this?
Let me know when a neural net gets angry, and starts visibly shaking due to emotions. Or, more generously, when such a neural net can do fundamental human tasks like parse ambiguous statements off context. Or perhaps write a short fantasy story even 3 pages long. If the neural net is anything like a brain, why can't it do such simple tasks any human can do?

Anonymous

Anonymous Sat 30 Oct 2021 16:16:17 No.13803783 Report

Quoted By: >>13803826

>>13803410
>These stupid terms like "chair" and "table" and other nonsense do nothing but occluded the underlying carpentry of the furniture.
I also get really wet thinking about the mathematics of NNs, but that doesn't mean having words to describe different aspects and associated concepts "obscure" the math. Supervised as opposed to unsupervised is a useful distinction yo.

Anonymous

Anonymous Sat 30 Oct 2021 16:30:27 No.13803826 Report

Quoted By:

>>13803783
"Supervised learning" = identifier with input parameters
"Unsupervised learning" = identifier without input parameters.
See? Much better terms that actually highlight what it does. Rather than pretending it's some kind of "intelligence" that's being "trained"

Anonymous

Anonymous Sat 30 Oct 2021 16:52:46 No.13803874 Report

Quoted By: >>13803893

>>13803499
>>13803509
The bulk of the cost is in training

Anonymous

Anonymous Sat 30 Oct 2021 16:58:11 No.13803886 Report

Quoted By: >>13805988

>>13803649
You really wouldn't complain this way if you were studying like how to put things into orbit
>rockets are way outside of the student budget for studying orbital mechanics. I really would appreciate a cheaper method

Anonymous

Anonymous Sat 30 Oct 2021 16:59:45 No.13803893 Report

Quoted By:

>>13803874
OK, and? The point is they throw orders of magnitude of more money at these problems for marginal improvement in efficacy. It's unsustainable. They're already throwing hundreds of millions of dollars at these algorithms for little gain. There's only a couple more orders of magnitude more they can throw, and if they get just another marginal improvement, they're stuck and can't do any better.

Anonymous

Anonymous Sat 30 Oct 2021 23:12:31 No.13805027 Report

Quoted By: >>13806341

>>13803675
>fundamental human tasks like parse ambiguous statements off context
Even good old GPT-2 could do that.

Anonymous

Anonymous Sat 30 Oct 2021 23:23:13 No.13805057 Report

Quoted By:

>>13797391
How to double the neural connections in your brain (3min video)
https://youtu.be/4dpEQ68cG0g

Anonymous

Anonymous Sun 31 Oct 2021 06:38:28 No.13805735 Report

Quoted By:

I love reinforcement learning!!

Anonymous

Anonymous Sun 31 Oct 2021 09:53:44 No.13805988 Report

Quoted By: >>13806020

>>13803886
Rocket science isn't as hyped as ML.

Anonymous

Anonymous Sun 31 Oct 2021 10:14:47 No.13806020 Report

Quoted By: >>13806932

>>13805988
My point was, these are not things you should expect to have access to as a student. You don't build orbital rockets as a student. You don't build power grids as a student. You don't train billion parameter models as a student. Those are projects that respective students learn about so they could go on to work on them, but not something they should expect to make.

Anonymous

Anonymous Sun 31 Oct 2021 12:35:27 No.13806263 Report

Quoted By: >>13810808

>>13797391
There is no AI, it's just optimization.

Anonymous

Anonymous Sun 31 Oct 2021 13:05:58 No.13806341 Report

Quoted By:

>>13805027
No.

Anonymous

Anonymous Sun 31 Oct 2021 14:21:57 No.13806521 Report

Quoted By: >>13806932

>>13803649
>In university we have such a great focus on neural networks, NLP, reinforcement learning but reproducing these experiments or making projects becomes difficult because of the limited computational resources we have as students. So I really would appreciate cheaper methods.
You can use any of these methods on a normal computer, I do it all the time. I work at a small ML company and 95% of the models I build are run on a 3 year old mac. I only jump to the GPU clusters for big-parameter models, but even a simple transformer as run in the OG paper can be trained overnight easily on a regular ol' desktop.
>I have also heard that there is a growing sentiment at DeepMind that reinforcement learning can be the solution to all possible tasks that ML does now. Is this true?
I dunno, but everything I've seen is more about integrating reinforcement learning with other architectures; RL has taken off in a big way and can do a lot that classical ML couldn't, but it's extremely resource heavy to achieve those differentiating results. Also, the problems addressed are usually of a different type where supervised ML wouldn't necessarily apply.

Anonymous

Anonymous Sun 31 Oct 2021 14:24:05 No.13806530 Report

Quoted By:

>>13797391
It is randomly generated. Changing it bit by bit creates video.

Anonymous

Anonymous Sun 31 Oct 2021 16:46:21 No.13806932 Report

Quoted By:

>>13806020
>>13806521
I don't disagree but we are sometimes expected to build models in PyTorch or Tensorflow that should do real time object detection, abstractive text summarisation etc. Even on Google Colab's free tier, it takes a lot of time to train or fine tune these models.

Anonymous

Anonymous Sun 31 Oct 2021 16:48:08 No.13806939 Report

Quoted By:

>>13797391
those things work only in the sense that the buzzwords trick the sois into willingly parting with their wealth

Anonymous

Anonymous Mon 01 Nov 2021 06:48:33 No.13808775 Report

Quoted By: >>13809336

How did a bunch of interconnected nodes with state memory turn out to be so great? I mean the underlying theory of neural networks is not even based on any advanced mathematics.

Anonymous

Anonymous Mon 01 Nov 2021 12:34:20 No.13809336 Report

Quoted By:

>>13808775
>500 billion free variables
That's how it turned out so great. There are patterns everywhere all these algorithms do is throw everything at a pattern, including the kitchen sink, and see what pops out. It's horribly optimized and doesn't scale well at all.

Anonymous

Anonymous Mon 01 Nov 2021 14:44:28 No.13809626 Report

Quoted By:

>>13800582
Wow what an asshole kek

Anonymous

Anonymous Mon 01 Nov 2021 15:02:07 No.13809654 Report

Quoted By: >>13809660

>>13797391
traditional AI was more about search and graphs. They had a lower emphasis on "learning" as being important.
machine learning and "connectionism", puts learning as the central problem . Neural Networks are perhaps the most extreme version of putting learning central to everything

Many Deep Learning butthurts in this thread. Trivializing Deep Learning doesn't make you smart, it only makes you feel smart. You will regret not paying more attention to deep learning later in your career, I promise you.

Anonymous

Anonymous Mon 01 Nov 2021 15:07:20 No.13809660 Report

Quoted By: >>13809676

>>13809654
Neural networks have done very well so far. But it is also true that it is infeasible. Small time actors cannot take full advantage of it because of the high costs.
So why are you so sure of DL's continued success?

Anonymous

Anonymous Mon 01 Nov 2021 15:18:23 No.13809676 Report

Quoted By: >>13809692

>>13809660
>Small time actors cannot take full advantage of it because of the high costs.
Unfortunately this is true. Doesn't change what I said however. Huge computational burden gate keeps a bunch of stuff in science in case you haven't realized yet.
>So why are you so sure of DL's continued success?
Lots of reasons that are hard to condense. But hardware is moving towards the neural orientation, and everything follows the hardware. That should tell you something BIG is happening here.

The key thing to understand is that yes artificial neural networks are not really models of brain neural networks. But that is really overly pedantic. The key insight is that nature itself settled on parallel distributed architecture for intelligence. And our technology and hardware is moving in that same direction. There are fundamental reasons why building an intelligent system with parallel distributed computing makes sense, and neural networks are the family of machine learning models that map DIRECTLY onto that paradigm.

Anonymous

Anonymous Mon 01 Nov 2021 15:24:59 No.13809692 Report

Quoted By: >>13809734

>>13809676
>But hardware is moving towards the neural orientation, and everything follows the hardware.
Not that guy but what do you mean by this?

Anonymous

Anonymous Mon 01 Nov 2021 15:25:20 No.13809694 Report

Quoted By: >>13809760

>>13797396
>the math isnt hard either
The advanced calculus and linear algebra required isn't hard? To me, the math involved in machine learning was always the most difficult part, but if you have a math background it could well look very different.

Anonymous

Anonymous Mon 01 Nov 2021 15:38:25 No.13809734 Report

Quoted By: >>13809758 >>13809801

>>13809692
No matter how you slice things, AI or machine learning or "Just a bunch of linear algebra" are computationally intense problems. This is just the nature of the domain.
When the solution on the software side, and the solution on the hardware side are the same, you should really, really, really, really pay attention. That solution being "make things look more neural networky". Or we can call it "parallel distributed processing" if neural network makes you butthurt, I know that is a trigger word for many people. What we call it doesn't matter though, this is the way it is going.

Shitty explanation I guess but sorry I don't feel like writing a whole essay right now.

If I can leave you guys with one thing it is this. Don't look at deep learning and think "huh, its just bunch of linear algebra. thats for low IQ comp sci majors".
Instead look at Deep Learning and see that "just a bunch of linear algebra" managed to
>learn to play starcraft 2
>learn to play go
>solve protein folding problems
>completely revolutionize computer vision
>learn to drive cars
etc etc etc

When something simple and dumb like "just linear algebra" can do all of that, pay attention.

Anonymous

Anonymous Mon 01 Nov 2021 15:43:49 No.13809758 Report

Quoted By: >>13809769

>>13809734
Wow so we just need "better" linear algebra and need to gear hardware even more towards deep learning.

Anonymous

Anonymous Mon 01 Nov 2021 15:44:18 No.13809760 Report

Quoted By:

>>13809694
If you have a mathematical background (even a half assed one like maths phys t. me) it's not hard. Optimisation and efficient implementation can be harder, as well as quantising and bounding predictive certainty, but at principles layer it's all straight forward.

Anonymous

Anonymous Mon 01 Nov 2021 15:45:39 No.13809769 Report

Quoted By: >>13809848

>>13809758
All the hardware dudes are working on deep learning right now. Jim fucking Keller is building deep learning chips. The elephant is in the room, all you need to do is look at it.

Anonymous

Anonymous Mon 01 Nov 2021 15:56:37 No.13809801 Report

Quoted By: >>13809815

>>13809734
>500 billion free parameters in a matrix can replicate the rules of a game that is constrained to a couple hundred billion finite moves
Whoa! So cool! Why, dare I say, soon such advanced linear algorithms will be able to tell me how it fell in love. *Swoon*

You're a pseud, simple as.

Anonymous

Anonymous Mon 01 Nov 2021 16:03:31 No.13809815 Report

Quoted By: >>13809886

>>13809801
I found the butthurt engineering major

Anonymous

Anonymous Mon 01 Nov 2021 16:09:32 No.13809831 Report

Quoted By:

The day AI starts consistently solving Leetcode questions is the day I give up on programming

Anonymous

Anonymous Mon 01 Nov 2021 16:16:22 No.13809848 Report

Quoted By: >>13809856

>>13809769
>deep learning is good because it has custom built hardware
This is just stating the obvious

Anonymous

Anonymous Mon 01 Nov 2021 16:19:18 No.13809856 Report

Quoted By:

>>13809848
That's the point. It is obvious. Hence the whole elephant in the room thing...

Anonymous

Anonymous Mon 01 Nov 2021 16:29:09 No.13809874 Report

Quoted By:

>>13797391
So i had to present an AI we made on an university project and we just hired some indians to pretend to be an AI over a php backend. The nice thing about indians is, that they have human intelligence but they still (usually) dont pass the turing test. Just use human intelligence its faster and superior

Anonymous

Anonymous Mon 01 Nov 2021 16:35:35 No.13809886 Report

Quoted By: >>13809895

>>13809815
More like a professor of physics who has seen multiple waves of AI fads over the past few decades. This isn't the first time people were excited about AI... Similar time in history in the '80s, then they plateaued due to computational inefficiency. Only recently did large scale computer clusters let them throw more money and resources at the algorithms during a resurgence of interest in AI. But they're starting to hit another plateau, funneling hundreds of millions more in funding for marginal improvement in classifications. By 2030 this fad will end.

Anonymous

Anonymous Mon 01 Nov 2021 16:41:33 No.13809895 Report

Quoted By: >>13809899

>>13809886
>By 2030 this fad will end
And by that time deep learning will be deeply embedded into nearly every aspect of our technology. That's a pretty fucking good trend if you ask me.

Also your history is a little bit wrong. Yes that's the textbook version of "muh AI winters", but I'd encourage you to think about what entity has been the central figure in AI from the earliest days.
protip: its DARPA

consider that some of us have access to information you don't. thats all i can say

Anonymous

Anonymous Mon 01 Nov 2021 16:45:04 No.13809899 Report

Quoted By:

>>13809895
>consider that some of us have access to information you don't. thats all i can say
Ah yes, like project STARGATE which was classified for decades... People knew things us meager civilians didn't. Turns out their knowledge and experiments were busts, lol.

Anonymous

Anonymous Mon 01 Nov 2021 17:08:37 No.13809950 Report

Quoted By: >>13809959

>/sci/ dimwits attempting to understand ML
kek, if you're filtered by python then you are most also filtered by tensorflow and keras

Anonymous

Anonymous Mon 01 Nov 2021 17:15:24 No.13809959 Report

Quoted By:

>>13809950
Way to out yourself as a fucking retard. Maybe /g/ is more your speed.

Anonymous

Anonymous Mon 01 Nov 2021 23:13:28 No.13810808 Report

Quoted By:

>>13806263
Any sufficiently well understood magic is indistinguishable from science.

Anonymous

Anonymous Tue 02 Nov 2021 08:08:31 No.13811804 Report

Quoted By:

bump

Anonymous

Anonymous Tue 02 Nov 2021 16:30:24 No.13812698 Report

Quoted By:

>>13801234
It's because they have low interpretability. This issue is recognised even by actual ML researchers so there is a lot of research ongoing on semi interpretable models

Capcode	All Only User Posts Only Moderator Posts Only Admin Posts Only Developer Posts
Show Posts	All Only With Images Only Without Images
Deleted Posts	All Only Deleted Posts Only Non-Deleted Posts
Ghost Posts	All Only Ghost Posts Only Non-Ghost Posts
Post Type	All Only Sticky Threads Only Opening Posts Only Reply Posts
Results	All Grouped By Threads
Order	Latest Posts First Oldest Posts First

Your latest searches