Matrix Goes Mainstream
Six Data Mining Companies Watching You
- Retailers employing same data mining techniques as NSA to drive growth
- Data Management’s mandate has expanded fromBiz Intelligence to Predictive Analytics
- Venture Capital investments in Artificial Intelligence setting new records
- Target’s vast data project provides a window into the future
“You look surprised to see us again Mr. Anderson. That’s the difference between us. I’ve been expecting you.” -Agent Smith, The Matrix
From Matrix to Minority Report, Avatar to Ex Machina, Hollywood keeps churning out conspiracy stories about big data, even as the rest of us voluntarily upload everything from location to likes… and retailers can’t get enough of us.
Take Target for example, where an app-based coupon program called Cartwheel has lit a fire under 25 million shoppers. The more you buy, the more it learns about what you like. The customized deals and discounts just keep coming. Scary? Hardly, more like manna from heaven.
We recently got the inside view on Target’s (TGT) data-driven shopping experience from man who designed it, Chief Marketing Officer Jeff Jones. He refers to customers as guests, and his centralized data collection hub the Center for Guest Excellence. We had the pleasure of interviewing him at J.P. Morgan Masters‘ Series in New York.
“The mobile phone is the most powerful shopping device ever created,” Jones tells a group of over 100 marketing professionals. To illustrate the point, he describes Target’s smartphone app, which directs people around the store to locate products, even maps their most efficient route for an entire shopping list. Out of stock? Not when the app sources it from another store and ships it to your home overnight.
Indoor mapping is made possible through a partnership with a company called Pointe Inside, while Target’s Cartwheel app was developed in-house. Both offer a glimpse into how the mega-retailer uses mega amounts of data to envelope shoppers into what’s become the Holy Grail of retailing: an integrated omni-channel experience (industry jargon for fully integrated, where customers don’t distinguish between digital and physical shopping, and stores double as distribution centers for online orders).
Think of Target’s Center for Guest Excellence as a giant sieve which funnels every conceivable piece of data into one central location for realtime analysis. Purchases, complaints, special requests –even intangibles like mannequins placement– are logged and analyzed to improve a guest’s experience. Ultimately, Target is indifferent between a sale in-store versus a sale online. They key says Jones, is creating a “seamless, excellent experience.”
“The dream is people would come to Target.com, but that’s not where it happens. Social platforms like Facebook are so pervasive… digital is now 51% of our ad spend. People go to Pinterest to figure out how they want their home to look, then they come to Target. It’s why we’re putting Buy buttons in posts.”
Jones previously served as CMO at The Gap (GPS), and soon after joining Target he embedded the retailer’s engineers at Facebook (FB). His goal was to foster collaboration on how to better identify individual preferences, and then direct the right merchandise at the right time to the right person. Data has become so critical to Target’s mission that every managers’ meeting begins with a review of real-time customer activity. In fact, each employee receives a morning email highlighting three critical trends gleaned from guest data the day before. The read-thru rate is over 90 percent.
Jones approach has put Target on the cutting edge of retail, but a new report from Forrester Research estimates between 60 and 73 percent of data available to large enterprises goes unused. It’s why VC funds like Data Collective, Bloomberg Beta, Khosla Ventures and Lux Capital have invested nearly $1B in Artificial Intelligence (AI) startups since 2010, according to data collected by CB Insights.
Former Apple (AAPL) and Pepsi (PEP) CEO John Scully saw the opportunity early on and founded Zeta Interactive in 2007. So did eBay (EBAY) co-founder, Peter Thiel, whose privately owned Palantir Technologies was most recently valued at $15B. Here’s how the company describes itself on it’s webpage:
We’re focused on creating the world’s best user experience for working with data, one that empowers people to ask and answer complex questions without requiring them to master querying languages, statistical modeling, or the command line. To achieve this, we build platforms for integrating, managing, and securing data on top of which we layer applications for fully interactive human-driven, machine-assisted analysis.
Data analysis on the scale employed by pioneers like Palantir, and customers like Target is a multifaceted task which requires massive storage and supercomputer-like power. In broad terms, the industry can be divided into three primary categories:
• Data Management (DM) gathering, scrubbing and organizing metadata
• Business Intelligence (BI) processing data and gleaning insight
• Predictive Analytics (PA) using data to anticipate future activity
Large amounts of data require significant plumbing… vast holding tanks for storage and interconnected pipes for transport. Adding more data is not as easy calling Cisco and purchasing another server, or even Amazon Web Services and renting more cloud space.
For anyone who remembers plugging data into dBase back in the 90s (my first job out of college at Merrill Lynch), we’ve come along way. Data strings are now so dense and cumbersome they require specific solutions. The primary architecture used to manage metadata is called Hadoop, an open source Javabased framework created by two engineers in 2006, and named for the toy elephant belonging to one of their sons. In case you think calling it Hadoop sounds silly or belies its importance, IDC values the market for Big Data at $122B.
Forrester describes Hadoop as “mandatory” for any enterprise looking to identify actionable strategies based on advanced analytics on large amounts of data. Its three primary components include a distributed file system (HDFS) which distributes large data blocks across commodity machines to maximize bandwidth, a resource management platform (YARN) which minimizes stress on any one CPU or server array and a compression program (MapReduce) to speed up processing.
Forrester ranks privately held Cloudera as the #1 Hadoop pure play in its 1Q 2016 report, followed closely by publicly traded Hortonworks (HDP). Other top rated companies include MapR and Pivotal Software, though like Cloudera they too are private. Many of these companies are less than ten years old, and while they are growing sales 40-50 percent annually, most are not yet profitable.
Early VC investors are effectively banking on data mining as the next core driver of enterprise marketing strategy. As John Sculley told me on BloombergTV. “This is the next big thing. It’s where I’m placing my bet and why I’ve hired 60 software engineers.”
Once enterprises have installed Hadoop across their networks to gather customer data, they can begin to analyze it with a host of products broadly described as Business Intelligence (BI). Author of the AtScale Blog Bruno Aziza, describes BI as the point at which company employees pivot from being “knowledge workers to knowledge seekers.”
Gartner published its annual 2016 Magic Quadrant for BI the first week of February and only three companies of 24 made the coveted leaders’ box: Microsoft (MSFT), Qlik (QLIK), Tableau (DATA). This year marks Gartner’s 10th annual report, and it has become the document of reference for buyers of BI technology. To make the so-called magic quadrant, companies have to demonstrate both completeness of vision and ability to execute.
Tableau pioneered the concept of converting multiple data sets into straight forward charts with a its dragand-drop desktop dashboard (demo at: http://www.tableau.com/products). Tableau’s software enables any information tracked by an enterprise (product sales, web traffic, store visits, etc) to be charted and compared across multiple factors with a few mouse clicks. Programmers call this visualization, and the goal is to empower marketing execs with tools to spot trends in realtime.
As an example, Target’s CMO Jeff Jones mentioned how sales associates reported 2-3 fold sales increases when certain items were featured on mannequins. Some further noted how mannequins placed in specific areas attracted more attention then others, driving additional sales. Data visualization would take all that information and create a map of the sales floor, illustrating optimum mannequin usage and placement. Charting over time might reveal further insights, like different traffic patterns on different days. Mannequins could be shifted after hours to maximize expected patterns for the following day.
Not surprisingly, Tableau’s consistent 90 percent gross margins are among the highest in the entire software sector, but growth has halved to 30 percent as competitors have entered the market.
Specifically, Bloomberg Intelligence BI analyst Mandeep Singh identifies fellow Gartner Magic Quadrant leaders Microsoft Power BI and Qlik Sense. Amazon (AMZN) and Salesforce.com (CRM) have also introduced their own BI solutions, QuickSite and Wave.
Critically, these newer entrants are taking business intelligence a step further, analyzing changes over time in order to generate forecasts about the future. Its called Predictive Analytics (PA) and its the latest evolution of data analysis. Other companies making the jump to more predictive software capabilities include Apigee (APIC), Splunk (SPLK), Alphabet (GOOGL) through a series of acquisitions and Microsoft via its Cortana Analytics acquisition.
According to Bloomberg Senior Software Analyst Anurag Rana:
“The ability of cloud application vendors to provide advanced analytics on their core products may become the single biggest distinguishing factor in the year ahead. Machine learning and other advanced data analysis methodologies are becoming increasingly important for clients to better understand their user-base. Rising use of Internet of Things (IoT) products will also fuel demand for analytics. Pure-play visualization vendors will likely expand their product portfolio as competition increases.”
Industrial manufacturers like GE have already joined retailers like Target in making a big push into predictive technologies. It’s part of the $517B IoT market, where machines loaded with sensors push data to a central hub and adapt to changes without human intervention. GE intends to triple revenue at its PA software subsidiary Predix to $15B by 2020. Per Mr. Rana’s calculation, that would make GE a global top 10 software company, competing directly with IBM and SAP.
Bottom Line: Access to data means every major enterprise looking to grow sales must become a software company. No wonder summer camps teach code.
