/sci/ - Science & Math » Thread #11716071

119KiB, 1083x722, 3IMG_1331.jpg

View Same Google iqdb SauceNAO

What kind of math background would help you in machine/statistical learning theory research?

Anonymous Wed 27 May 04:19:19 2020 No.11716071 View Reply Original Report

Quoted By: >>11716078 >>11716113 >>11716201 >>11716283 >>11716640 >>11716657

Like what are the most complementary math areas? Ofc you need undergrad probability and linear algebra as foundational background, I'm more so asking what grad/research level math might be helpful to contribute to learning theory in the future, or is it just it's own thing now

Anonymous

Anonymous Wed 27 May 2020 04:23:45 No.11716078 Report

Quoted By: >>11716095

>>11716071

to get a meme job or for actual research?

Anonymous

Anonymous Wed 27 May 2020 04:28:47 No.11716095 Report

Quoted By: >>11716158

>>11716078
Research
although interested in your meme job approach too

Anonymous

Anonymous Wed 27 May 2020 04:32:14 No.11716113 Report

Quoted By: >>11716201 >>11716283

>>11716071
>undergrad probability and linear algebra as foundational background
That's basically it, although you know, not linear algebra for noobs right, proper linear algebra.

Anonymous

Anonymous Wed 27 May 2020 04:46:52 No.11716158 Report

Quoted By: >>11716221 >>11716264

>>11716095
>undergrad probability and linear algebra

stay away from research unless it is a meme research

Anonymous

Anonymous Wed 27 May 2020 05:00:29 No.11716201 Report

Quoted By: >>11716465 >>11716519

>>11716071
>>11716113
Yeah that's it, and that's pushing it, since machine learning is already very well researched and you just need to code the algorithms.

Anonymous

Anonymous Wed 27 May 2020 05:05:39 No.11716221 Report

Quoted By:

>>11716158
Why?

Anonymous

Anonymous Wed 27 May 2020 05:16:10 No.11716264 Report

Quoted By: >>11716673

>>11716158
Meme = applied deep learning/CV/NLP?

Anonymous

Anonymous Wed 27 May 2020 05:22:24 No.11716283 Report

Quoted By: >>11716295

>>11716071
>>11716113
It depends. If you're doing mainstream ML research, undergrad probability theory, linear algebra, and calculus 1-3 are all you need as a practioner. If you're doing research into theory proper, especially if it's SLT and CoLT, you're going to need heavier analytic background.

Anonymous

Anonymous Wed 27 May 2020 05:25:35 No.11716295 Report

Quoted By: >>11716304

>>11716283
>you're going to need heavier analytic background.
Such as?

Anonymous

Anonymous Wed 27 May 2020 05:30:11 No.11716304 Report

Quoted By: >>11716309

>>11716295
At the least? A semester of baby Rudin as seen in undergrad. Preferably? You will have wanted to see measure spaces, their relationship with probability theory, and some functional analysis. See:
https://web.stanford.edu/~hastie/StatLearnSparsity_files/SLS_corrected_1.4.16.pdf
https://people.eecs.berkeley.edu/~wainwrig/stat241b/

Anonymous

Anonymous Wed 27 May 2020 05:34:38 No.11716309 Report

Quoted By: >>11716355

>>11716304
So keep studying analysis and probability to the highest levels you can take... is that all?

Anonymous

Anonymous Wed 27 May 2020 05:58:45 No.11716355 Report

Quoted By:

>>11716309
I mean for learning theory, that's more or less how it goes. You'll also want to learn computational theory, so go through Arora-Barak for complexity, and for CoLT, you want to have the following under your belt:
http://www.cs.columbia.edu/~rocco/Teaching/S12

https://doc.lagout.org/science/0_Computer%20Science/2_Algorithms/Boosting_%20Foundations%20and%20Algorithms%20%5BSchapire%20%26%20Freund%202012-05-18%5D.pdf

https://pdfs.semanticscholar.org/e923/9469aba4bccf3e36d1c27894721e8dbefc44.pdf

and probably some more material on the PAC model. Either way, what I list isn't exactly comprehensive, but it's a start

Anonymous

Anonymous Wed 27 May 2020 06:40:48 No.11716465 Report

Quoted By: >>11716479 >>11716504

>>11716201
What if i like to understand what I am doing. Am I in for 5 years of grind for little practical value? If so, what math?

Anonymous

Anonymous Wed 27 May 2020 06:47:34 No.11716479 Report

Quoted By: >>11716504

>>11716465
For the what math part scratch that, posts above suffice. Still intrested in your opinion on the first part of my question.

Anonymous

Anonymous Wed 27 May 2020 06:59:27 No.11716504 Report

Quoted By: >>11716519

>>11716465
>>11716479
ML theory is only a subfield of ML, everyone else is mostly publishing experimental work
Theory might be a dead end anyway, applications get all the attention, press, and money thrown behind it

Anonymous

Anonymous Wed 27 May 2020 07:08:37 No.11716519 Report

Quoted By: >>11716527

>>11716201
>>11716504
OP asked about statistical learning *theory*, not the loads of ML papers in experiment and application. Either way, theory is definitely not a dead end just by looking at publication activity and what they're writing about. Application gets all the press and attention due to it being easily marketable - everyone sees ML and thinks "AI." They see learning theory and see math and statistics.

Anonymous

Anonymous Wed 27 May 2020 07:12:29 No.11716527 Report

Quoted By: >>11716560

>>11716519
If the goal is to explain how the state of the art in application works with theory no one is even close to that

Anonymous

Anonymous Wed 27 May 2020 07:28:20 No.11716560 Report

Quoted By: >>11716613

>>11716527
I mean, that's one goal, but if you read these well circulated papers from JMLR, it's pretty clear the scope of theory is not relegated to explaining experiment.
http://www.jmlr.org/papers/volume20/15-192/15-192.pdf

http://www.jmlr.org/papers/volume20/16-243/16-243.pdf

http://www.jmlr.org/papers/volume20/16-585/16-585.pdf

http://www.jmlr.org/papers/volume20/17-501/17-501.pdf

Anonymous

Anonymous Wed 27 May 2020 07:52:33 No.11716613 Report

Quoted By:

>>11716560
Are you in this field? Do you think it's better than just doing CV/NLP?

Anonymous

Anonymous Wed 27 May 2020 08:04:39 No.11716640 Report

Quoted By: >>11716665

>>11716071
Statistical Learning Theory.
I also found differential geometry to be helpful.
If you want to get far ahead, learn quantum computing. Has the same foundation and can boost learning a lot.

Anonymous

Anonymous Wed 27 May 2020 08:09:27 No.11716657 Report

Quoted By:

>>11716071
For understanding what a FNN is, which is the single most important thing to understand, you need:
- Linear algebra
- Multi variable calculus

Other areas that will come up at some point:
- Real analysis
- Fourier Analysis
- Abstract algebra
- ODEs and numerics of ODEs, important for current research into neural ODEs

>Ofc you need undergrad probability
Not necessarily. NNs function largely without probability theory.

Anonymous

Anonymous Wed 27 May 2020 08:12:07 No.11716665 Report

Quoted By: >>11717349

>>11716640
>I also found differential geometry to be helpful.
Certainly not for neural networks. Where is diff geo useful?

Anonymous

Anonymous Wed 27 May 2020 08:17:32 No.11716673 Report

Quoted By: >>11716687

>>11716264

ok I not an expert or even anything but I would start with stochastic optimization, numerical methods, maybe some statistical physics or language theory (machine learning of what?)

Also complexity theory tells you some problems are intractable no matter what you throw at against them.

Ask some experts, why even bother with this basket weaving forum? You probably learn by doing projects there must be difference between running keras on your GTX 1060 and some large scale, like really large scale computations (matrix won't fit into memory) etc. Numerical analysis is not a completely trivial thing.

Last but not least ironically you need to learn to code. Writing clear, efficient code is not that popular again.

There are possibly some extremely smart, highly competent and experienced people doing research already and you with your "undergrad lin alg" is no match for them.

Anonymous

Anonymous Wed 27 May 2020 08:28:04 No.11716687 Report

Quoted By:

>>11716673
Just asked what he meant by meme, I guess he thinks applied ML is a meme

Anonymous

Anonymous Wed 27 May 2020 12:59:43 No.11717349 Report

Quoted By:

>>11716665
"Geometrical deep learning", basically for deformation- or whatever kind of transformation-invariant learning.
Also for topological data analysis. Berry numbers reveal some useful connections (or disconnections) in huge datasets.

Capcode	All Only User Posts Only Moderator Posts Only Admin Posts Only Developer Posts
Show Posts	All Only With Images Only Without Images
Deleted Posts	All Only Deleted Posts Only Non-Deleted Posts
Ghost Posts	All Only Ghost Posts Only Non-Ghost Posts
Post Type	All Only Sticky Threads Only Opening Posts Only Reply Posts
Results	All Grouped By Threads
Order	Latest Posts First Oldest Posts First

Your latest searches