Data science and analytics have moved from being an investment in the future to a core component of corporate strategy and a strategic enabler for all areas of the business. Regardless of size or sector, the practice of analytics has become a necessity. Best-in-class organizations have been leveraging data science for years to develop new products, understand their customers, control their costs, and make better informed decisions, which has provided them a large competitive advantage.
As such, legacy companies continue to invest in the practice. In a survey by Deloitte, half of the executives who were interviewed said that they believe data science will fundamentally change their organizations in the next three years. Unfortunately, most data science projects end in failure. In studies by Gartner, VentureBeat, and Capgemini, failure rates were 85%, 87%, and 73% respectively. Executives know that it is a necessity, they know that it works, but organizations for the most part are unable to unlock value from their data science functions.
There are many reasons behind these failures, but the fundamental mistake is an over-vectoring on technology, and the misapplication of management models which were inelegantly adopted from technical disciplines like software engineering. Teams of highly skilled practitioners are onboarded with the mandate to “do data science”, and through prescriptive sprints they are directed to execute on plans which are conceived and coordinated by their would-be clients.
The control this gives the business is illusory, and the output of the function suffers as a result. The motivation behind this self-sabotage is one of the fundamental misunderstandings of data science, and one of the major contributing factors to the failures that are being seen – the perception of analytics professionals as technical resources, rather than as creatives.
People would never hire a master painter, hand them a reference photograph, and ask that they recreate it on the canvas. The painter is hired for their creativity, and ability to translate the desires of their client into a beautiful product. In a similar way, rather than acting as a pool of technical resources, data scientists should be depended on to understand the needs of their client and deliver value.
As a creative practice, practitioners have several unique requirements to be at their best:
Operating within a guild of peers
Data professionals, like other craftspeople, need to work within an environment that provides opportunities for hands-on learning, and apprenticeship opportunities under more experienced practitioners. Data science degree programs provide only the theoretical foundations – until they have worked with messy real-world data, dropped a table in production, and faced a code review board, they are often unprepared to work independently. The fraternal connection and support provided by a guild mentality gives the practice a sense of self-determination that cannot be had any other way.
Support by a credible leader
Leaders need to view themselves as a supporter and facilitator, willing to remove roadblocks and operating within a lightweight framework. Hierarchical top-down approaches to management for a team of craftspeople fosters dependency from less mature practitioners, and resentment from those more experienced.
At the same time, leaders need to be more than an apparatchik. Without respect from the team, and the ability to challenge their approach, deliverables will be inconsistent and uncontrolled. The role of an analytics leader is to provide a bridge between the function and the business, and to shape an egalitarian team that challenges and supports each member. These “translators”, as McKinsey calls them, do not need deep data engineering or architecture experience, but instead to have a broad understanding of both the analytics function and the business so that they can ask the right questions and to help the team navigate towards an effective solution.
Personal agency
Whether it is a firmly held position on variable naming conventions or a fondness for mechanical keyboards, practitioners should have the ability to express themselves fully inasmuch as it does not disrupt the team or impact their deliverables. Project delivery needs to be consistent, but all other aspects of how that is achieved should be under the control of the individual. Creative solutions will never emerge from a team built on compliance and uniformity.
Gabriel Marcel, a French philosopher, promoted the idea of creative fidelity. Marcel believed that for an individual to reach their fullest personal expression, they needed to be a thinking, emotive, creative being. There is nothing human in simply doing or reacting. He said that “a really alive person is not merely someone who has a taste for life, but somebody who spreads that taste, showering it, as it were, around him; and a person who is really alive in this way has, quite apart from any tangible achievements of his, something essentially creative about him.” The primary goal of team conventions needs to be in support of this idea of creative fidelity. Every individual on the team needs to consider themselves as participating in the strategizing, planning, execution, ownership, and delivery of the team. Leadership must be exercised, but practitioners need to have a creative outlet and agency over their professional lives in order to be at their best.
Summary
In the 2011 documentary, Jiro Dreams of Sushi, the titular Jiro describes himself as a Shokunin, a person who passionately dedicates himself completely to bettering his craft. He feels obligated on an almost spiritual level to improve. This level of passion is not uncommon among data scientists, and it is essential that they have a leader who is able to challenge them, understand them, and offer an environment where they can grow.
Data science and analytics practices can work without strategy. It will be misguided, but work will be completed. The practice can even function without tools or processes. Nothing can happen though without creative and passionate people working collaboratively on meaningful projects. Analytics as a function lives or dies on the strength of its people.
