>>11306865>Why is Data Science a “hot job”?Because everyone is generating huge amounts of data (mostly through spying on people), but sifting through that shit fast enough is impossible for human. The previous approaches was to get someone to try modeling assumptions until something was good enough. Data science is more intelligent (that is to say, it actually works) AND turns a 10 people team's 5 years of work into a single employee's afternoon's worth of messing around to achieve these superior results. Yet, there is non-trivial amount of work to do to properly make that work, so you want these employees.
But if data science works so well, surely you'd only need a few data scientists and be done with it right?
Correct! But it turns out that every single task a data scientist can do brings ridiculous value and it scales marvelously with how many of them you hire. A big reason for that is that most of their jobs can easily be done solo and then be brought together by a team of software engineers. This fixes the scale problem of regular collaborative teams.
>Won’t most of the work done by Data Scientists by streamlined through tools Amazon, Microsoft, and Google are working on like AutoML for example?No. Amazon, m$ and google tools don't work for anything but the most trivial problems, for which your data scientist will just run sklearn anyway, which is much cheaper for you than using these services.
AutoML doesn't work anywhere near as well as a real data scientist either.
Big companies' tools are not intended for real workloads, they're intended for people with no expertise who want baseline tools but don't know how to get them, for already "solved" tasks. AutoML is a very complicated problem and a whole area of expanding research. Doesn't seem it's going somewhere but who knows. So far it's plagued by the same problems as pre-ML methods: it's no more efficient than monte-carlo search methods, and those are too slow in practice.
