The role of data scientists will be transformed as machine learning techniques become more widely used by businesses, according to Stephen Brobst, CTO of analytics firm Teradata.
While many of the principles behind ‘AI’ approaches are not new, interest within an enterprise setting has exploded in recent years. And as usage becomes more widespread and sophisticated, the role of data scientists will begin to evolve too, according to Brobst.
He explains that data scientists have typically spent much of their time ‘wrangling’ data to feed into predictive models. In future, more of this work will be automated and data scientists will instead be more focused on selecting which machine learning or deep learning tools to utilise for specific tasks.
“Instead of the data scientist spending most of their time working with the data itself, they are going to spend most of their time working with the algorithms – so you have to be much more sophisticated in algorithm selection and topology selection in a neural network and so on,” he says.
“You still have to understand how the nature of the data influences your algorithm selection, but you are going to be spending less time preparing data, because data scientists today spend more than 60 percent of their time preparing data, beating data over the head, torturing it, shoving it into a hole, and that is not going to be required anymore.”
He says that this is already becoming the case with advanced algorithms used for machine learning purposes currently, but as deep learning – a branch of machine learning – become more widely used, data scientists’ priorities will shift further.
“In deep learning there will be fewer requirements for domain knowledge and more requirements for algorithm selection based on the type of data that you have and so on,” he says, “so this shift of skill set will be very, very interesting.”