Recently, I have been asked by quite a number of executives from various industries on what are my thoughts on the role of data scientists in future, and how this might have an impact within any organization that aspire to be "more data driven". Honestly, I don’t exactly know what will happen in future but we can make some reasonable deductions based on several trends:
More sophisticated and robust data science algorithms will still likely continue to be open source and advancement in technologies will make deployment of algorithms much easier - This would "lower" the barrier for people with sufficient coding and basic understanding of statistics to build machine learning models, and it is a good thing. In the 1990s, you probably need to know HTML and CSS to build a decent looking website. Today, there are a variety of free or affordable "tools" and "packages" that can enable many people to build a beautiful website albeit they might still need to have HTML or CSS knowledge in order to build a stunning website. Observe how this had change the role of website developers from 1990s to date and you might find subtle similarities for data science in future.
You might have heard that discussion of ROI would stifle good and meaningful innovations. However, many companies right now are having those difficult conversations to "justify" ROI in innovations. And this is also a healthy discussion so that innovations can be made more sustainable. All of us would be very familiar with Uber innovations and how it has massively disrupted the transport industry many years ago. But the prolong inability to make profits in spite of its dominant platform isn't worth a lot from investors point of view, unless it can generate a consistent stream of profit to satisfy investors and most importantly, start to self-fund these innovations to build better platforms or solutions to benefit end consumers. Data Scientists, now more than ever, would need to generate real, mature and tangible actionable insights and present a business case to challenge the traditional commercial operating model or important business decisions.
As a direct outcome of the points above, there are 2 paths on how data scientists role would evolve. The first path would become research data scientists who are involved in highly research oriented work with the aim to evolve or build more complex algorithms or products (E.g AutoML) or approaches that would give a comparative advantage in the market. I would encourage data scientists who enjoy working on the "deep" to explore this path. And these jobs are likely to be found in tech companies like Microsoft, Amazon etc, IN THEIR headquarters or innovation powerhouse research labs.
The second path would be those who are close to the business, potentially occupying business management roles that was previously filled by MBAs. This group of data scientists would need to consciously learn how to be "less technical" (while still know the underlying technical to unpack the “fluffs”), learn the business and understand that less is more. On top of their ability to mine for insights from data, a data scientist who is also trained in a variety of soft skills such as communications, business acumen/sense, team management, commercial leadership, and ABILITY TO EXECUTE would be the best person to fill certain key position in a business function. Many companies will likely be thinking of hiring such profiles to take advantage of what machine learning has to offer.
The above is just my thoughts on how data science role will evolve in future. Nobody exactly know what's going to happen but I do hope that this article would trigger us to think deeper and have more meaningful discussion on the evolution of data scientist role.