Don’t get Caught Fumbling in Data Science and Miss the Moment for Action

data scientists routine tasks are exploratory data analysis, data cleaning, statistical modeling, and building machine learning models

The Artificial Intelligent (AI) world mirrors the Photography world very closely.  Have you every missed a great photograph because you were fumbling with your camera settings?  In the same way you can be stumbling collecting, cleaning and staging data when you should be analyzing, deciding and taking action to get a better business outcome.

The modern camera which takes digital pictures is a perfect example of how humans combine ART and SCIENCE to create outputs of work.  The automated technology allows the photographer to spend 10% of his energy on the science part and put 90% of his focus on the art of making a great picture.

Great Wedding Moment
Photo by Stephen Choo Quan –

The DSLR camera in a process called metering has an AI algorithm embedded in the camera that computes the perfect blend of ISO, aperture and shutter speed to create a great photographic exposure.

In the same manner as metering is to photography, a data scientist will use a very similar manual approach for supervised and unsupervised learning to arrive at statistical models.  The emerging automated AI tools are the equivalent of an intelligent digital camera  with automated tools for statistical analysis and machine learning like SAP’s Predictive Factory, Automatic Statistician, Ayasdi Core, and Skytree Infinity.

As in any industry this will reduced the barrier to entry.   There are many more photographers working professionally than there has ever been because of these intelligent cameras.  Data Science will have the same thing happen to this elite modern title as predictive tools are put in the hands of power users.  This must take place for this market place to evolve and transform business.

It Is Not the Strongest of the Species that Survives But the Most Adaptable

Charles Darwin

Now I am not saying you do not have to know the what a Bayesian Inference is. Everyone who want to work in the space needs to know the process of random forest and logistic regression but which package to use to solve a type of problem can easily be automated by a tool to be used by a sophisticated power user and eventually to the everyday business analyst.  User who know SPSS, SAS, R and Python will be able to navigate situations where the data sets are just too wide and vales needs to be tamed to create a proper working model.


It means we will have to enable all users with smart tools that can create models as good as the best of data scientists in one tenth (1/10th) the time it takes to do manually.

“The SAP predictive factory can produce as good a model as well known data scientist.  The difference is that the predictive factory takes 1 week and the data scientist takes 3 months”, says Pier Bobys, SAP predictive Analytics Specialist.

So we know that these “smart” tools have arrived.  They are as good as any reputable data scientists for routine tasks, such as exploratory data analysis, statistical modeling, and building machine learning models.

We can expect to see this become more main stream and pervasive in the coming year. Companies who are slow to adapt to the speed of change will become extinct.  Companies who empower business users with AI fundamentals and these tools will thrive.  It just follows the law of nature.

Stephen Choo Quan