As a technologist involved in analytics for decades, it is rewarding and exciting to see machine learning and applied analytics become mainstream. Suddenly what I do has become current and interesting, even outside tech circles. The downside, of course, is the amount of noise being generated around artificial intelligence, which makes it hard for organizations to focus on how they can deliver real value with their data.
Based on experience building quite a few analytic application platforms, I’ve learned that the last mile of analytics is usually the hardest part. Turning insights from machine learning into real action that impacts the bottom line is the only way our efforts produce tangible results, yet it’s often the most ignored part of a company’s analytic strategy.
Turning insights from machine learning into real action is the only way our AI efforts produce tangible results
For example, a common use case for analytics is understanding the profitability of products and customers. A firm may have invested in various ways to analyze and visualize past performance and to create algorithms to predict what is likely to happen in the future. But what happens when the business wants to apply this insight to adjust product pricing, create customer offers or adjust marketing strategies? This is the last mile of analytics and it is essential to any organization’s analytic strategy.
Understandably, many of the data and analytic executives I speak with are focused on the front-end of this process: data, quality, ETL and machine learning. The end user environment that ultimately leverages this information is often relegated to BI and visualization tools or embedded in static business applications. Unfortunately, BI tools aren’t powerful enough to optimize your pricing strategy, and business applications don’t deliver the self-serve agility required by businesses whose needs are constantly evolving and impossible to anticipate.
As a result, users continue to use spreadsheets as their tool of choice for consuming analytics. Many technologists I’ve met discount the spreadsheet as old technology and lament the fact that end-users are extracting data out of their data and analytic platforms, resulting in loss of traceability, duplication of data and a perception of all-around end-user chaos.
End-users aren’t continuing to use spreadsheets because they’re not embracing new technology, it’s because they don’t have an alternative
What many don’t recognize is that end-users aren’t continuing to use spreadsheets because they’re not embracing new technology, it’s because they don’t have an alternative. Spreadsheets provide full control for creating new calculations in seconds and financial projections in minutes. They can simulate how the business will perform based on variations of multiple variables and are able to bring in data from virtually any source and access it on the fly. Most importantly, spreadsheets can do all of this without having to write code and without waiting months for a solution from IT.
If spreadsheets are so powerful then what’s the problem? Why shouldn’t they continue to be the last mile of analytics? The answer is obvious to both technologists and the business. Invented in the 70s, spreadsheets have started to run their course for running today’s on-demand business. They are on the desktop, difficult to share, lack data security, can’t handle production data complexity or volume and can be tremendously fragile and opaque. Trying to understand a spreadsheet authored by a colleague is analogous to reading their mind. Most importantly, spreadsheets can’t be automated and are not real-time. They simply don’t fit with today’s online API driven architectures, which is why they’re often ignored as a necessary evil and non-strategic by IT.
Next generation tools for the business will connect to enterprise data and enable self-serve creation of applications that predict behavior, automate logic and make real-time decisions
I believe it is time to break the status-quo with spreadsheets and focus on delivering better self-serve tools for the last mile of analytics. Next generation tools for business analyst will be directly connected to enterprise data and allow them to not only explore and visualize data but create new analytic applications that automate business logic and make real-time decisions. The language they use will be declarative like a spreadsheet, but much more powerful and easy to understand and their environment will be online, shareable and multi-user.
I look forward to the next generation of analytic applications and we are excited to do our part to help create them. As I watch my two sons progress through their education, it strikes me that while not everyone wants a career in technology, all careers will need to leverage technology to be effective. The emerging workforce will be better prepared to work directly with data and scripting languages and will demand self-serve analytic solutions regardless of their discipline. My belief is that our job as technologists is beginning to shift dramatically from delivering point solutions to delivering self-serve platforms that empower the business.
