The last decade has seen a steep rise in the importance and application of data science.
There are several drivers behind this growth of data science importance.
Surge in amount of data
With mobile devices and internet, the amount of new data produced each years is surging. Here are some interesting statistics:
Each day, 500 million tweets are published,294 billion emails are sent,4 terabytes of data are created per connected car, etc.
Organizations have a wealth of data available and they are using it to produce valuable insights and drive their business decisions.
Rise of computing power
All of this data would be more or less unusable if we did not have powerful computers to automate their analysis. A CPU in today’s mobile device is by magnitudes more powerful than a 386 DX-40 PC from 1991. The rise of computation power has been a steady presence for decades, leading to empirical rule known as Moore’s law. Although in recent years there has been some slowing down in terms of CPU frequencies, this was alleviated by processors getting an increasing number of cores. See e.g. recent AMD power processors for example.
The third important driver behind rise of data science is that the combination of above – more data, better processors led to some methods become stunningly successful in predictions. I am talking about deep learning algorithms. The interesting thing is that many of these algorithms have been around for a relatively long time. E.g. one of the most famous ones – LSTM was invented in the 1990s. However, it was only the huge amounts of data from mobile devices and improvement in computing power which made these methods extremely successful in image detection and other tasks.
Once the first successes became important in businesses, driven especially by companies such as Google, Amazon and others, a lot of other companies also adopted the new trend of applying data science to its business problems.
As the skills demanded by the data science are rather unique what happened soon was shortage in data scientists and rise in our salaries. As a consequence, data scientist is near the top in recent years in salary charts. But not only that, a lot of us, data scientist consultants are highly satisfied with the jobs we are doing. They are intellectually demanding and interesting.