Skip to content

Posts tagged ‘Big Data’

10 Ways Analytics Are Accelerating Digital Manufacturing

  • 42% of manufacturers say big data and analytics as their highest priority in 2015.
  • 56% of power distribution providers rank big data and analytics within their top three priorities for 2015.
  • 61% of aviation companies consider big data and analytics their highest priority this year.

Bottom line: Digital manufacturing strategies are gaining ground as manufacturers adopt big data and analytics to improve operational effectiveness, time-to-market, new product development and increase product quality and reliability.

Analytics Are Fueling Digital Manufacturing Growth

Big data and analytics adoption by manufacturers is the first step many are taking to create a galvanized, intelligent digital thread that unifies every aspect of their value chains. For aerospace manufacturers whose supply chains are exceptionally complex, big data and analytics are revolutionizing value chains starting with suppliers and progressing through all operations.

The majority of manufacturers are relying on analytics to improve order accuracy, shipment & cycle time performance, and product quality. Those excelling at digital manufacturing strategies are gaining additional analytical insights into how they can make decisions more accurately, quicker and with lower potential costs and risks.

The manufacturing industry generates more data than any other sector of the global economy on a consistent basis.   The more complex a given manufacturers’ operations are, the more valuable the insights gained from big data and analytics. The following comparison of big data analytics priorities by industry from a recent speech given by Jeff Immelt, CEO and President of General Electric illustrates this point:

analytics customer survey

Source: GE Minds and Machines Presentation, Jeff Immelt, CEO & President, General Electric.

10 Ways Analytics Are Accelerating Digital Manufacturing 

The ten ways analytics is accelerating digital manufacturing adoption globally include the following:

  • Providing real-time operator intelligence (70%), remote monitoring and diagnostics (66%), and condition-based maintenance (59%) are the three most valuable areas for analytics GE customers mentioned in a recent survey. GE’s industrial customers are looking to tailor pre-built applications that can deliver the eight different functional areas shown in the graphic below.  Manufacturers are looking to asset performance management as an integral part of their digital thread’s analytics and insight.

industrial customer perspective

Source: GE Minds and Machines Presentation, Jeff Immelt, CEO & President, General Electric.

  • Using data modeling to improve production workflows is improving Earnings Before Interest & Taxes (EBIT) by 55% for a chemical manufacturer.  Using analytics and data modeling to make more accurate,  efficient decisions encompassing making or buying ingredients, choosing to substitute an ingredient or not, optimizing equipment usage and/or reliability and gaining incremental sales through increased production capacity is leading to a significant improvement in EBIT for a leading chemical manufacturer on a consistent basis.  The following graphic provides insights into the contributions of each factor in improving EBIT performance.

EBIT Growth

Source: Taming manufacturing complexity with advanced analytics. McKinsey & Company by Patrick Briest, Valerio Dilda, and Ken Somers February 2015. 

  • Planning-execution integration in production centers and real-time production integration are two areas where analytics are having the greatest impact on manufacturers’ operating expenses (OPEX). When analytics are integrated as part of a digital manufacturing strategy, supply chains benefit when Web-EDI (Electronic Data Interchange) and real-time order conformation are implemented and analyzed for continual improvement.

Digital initaitves impact

Source: Operational Excellence through Digital in Manufacturing Industries. Capgemini Consulting.

  • Optimization tools (56%), demand forecasting (53%), integrated business planning (48%) and supplier collaboration & risk analytics (46%) are being rapidly adopted by manufacturers today, setting the foundation for digital manufacturing growth.  Deloitte recently interviewed supply chain executives regarding the thirteen fastest-moving technical capacities they are using today and expect to use in the future. The following graphic provides an overview of supply chain capabilities current in use and what percent of each they expect to use in the future.

use of supply chain capabilities

Source: Supply Chain Talent of the Future Findings from the third annual supply chain survey. Deloitte.  2015.

  • Analytics is integral to making the vision of Industrie 4.0 a reality. Industrie 4.0 is a German government initiative that promotes automation of the manufacturing industry with the goal of developing Smart Factories. Analytics is extensively used in manufacturing centers who are in the process of reengineering their entire operations to attain Industrie 4.0 compliance. Manufacturing value chains in highly regulated industries that rely on German suppliers and manufacturers are also relying on analytics extensively to guide their Industrie 4.0 journey. A recent Deloitte study of Industrie 4.0 adoption found that research and development (43%) will see the greatest transformational contribution from Industry 4.0.

Industry 4.0 areas

Source: Industry 4.0: Challenges and solutions for the digital transformation and use of exponential technologies. Deloitte Consulting, 2015

  • Analytics is enabling manufacturers to also scale real time cloud-based operational intelligence, condition-based monitoring, monitoring & diagnostics and asset lifecycle management across global manufacturing centers.  Capturing, aggregating, analyzing and taking action on analytics across all production centers using the GE Predix Cloud will also accelerate digital manufacturing growth over time.  Integrating analytics, industrial and sensor data into a scalable series of data models and apps delivered as a Platform-as-a-Service (PaaS), GE will make this service commercially available in 2016.  The following graphic illustrates how complex manufacturers could use Predix Cloud to improve operational efficiency and quality.

horizontal capability controls

Source: Jeff Immelt Presentation on Pivot Strategy, December 16, 2014

  • Analytics is providing greater insights into product, process, program and service quality, forcing manufacturers to revamp existing production centers and make them more efficient.  Gaining greater insight into which production centers and factories are delivering the highest quality products and why is now possible.  The vision of unifying quality across an enterprise quality management and compliance (ECQM) framework is now a reality, driving greater digital manufacturing growth as a result. The following graphic from Tableau is an example of a manufacturing quality dashboard.

Mfg quality dashboard

Source: Manufacturing Analytics Quality Dashboard

  • Increasing production yields through the use of more effective supplier quality management and bill of material (BOM) planning integrated within production processes.  Analytics is extensively being used today for supplier audits, supplier quality management and traceability. Capitalizing on the full value of these analytics is a strong catalyst for manufacturers to move closer to digitizing their operations.
  • Using analytics to predict machine failures before they occur reduces downtime, production costs and increase customer satisfaction.  In highly regulated industries production equipment is periodically audited and reviewed for conformance to specific standards.  Integrating even the simplest sensor into production equipment can deliver valuable insights into what factors cause it to fail.  Analytics are providing Failure Mode and Effects Analysis (FMEA) in real-time today, providing manufacturers with a glimpse into which equipment and machinery will most likely fail when. Knowing this can save literally millions of dollars in lost production time.
  • Adopting Pareto Analysis to continually improve schedule, quality and cost performance to the cell or production center level is driving digital manufacturing adoption.  Determining which factors are enhancing or reducing product, process and program quality is now possible using advanced manufacturing analytics. Differentiating between the many symptoms of a quality problem and its root cause is now becoming possible, especially for companies pursuing digital manufacturing strategies.

Additional sources of information on the impact of analytics on digital manufacturing:


10 Ways Big Data Is Revolutionizing Supply Chain Management

supply chain managementBottom line: Big data is providing supplier networks with greater data accuracy, clarity, and insights, leading to more contextual intelligence shared across supply chains.

Forward-thinking manufacturers are orchestrating 80% or more of their supplier network activity outside their four walls, using big data and cloud-based technologies to get beyond the constraints of legacy Enterprise Resource Planning (ERP) and Supply Chain Management (SCM) systems. For manufacturers whose business models are based on rapid product lifecycles and speed, legacy ERP systems are a bottleneck.  Designed for delivering order, shipment and transactional data, these systems aren’t capable of scaling to meet the challenges supply chains face today.

Choosing to compete on accuracy, speed and quality forces supplier networks to get to a level of contextual intelligence not possible with legacy ERP and SCM systems. While many companies today haven’t yet adopted big data into their supply chain operations, these ten factors taken together will be the catalyst that get many moving on their journey.

The ten ways big data is revolutionizing supply chain management include:

Figure 1 SCM Data Volume Velocity Variety

  • Enabling more complex supplier networks that focus on knowledge sharing and collaboration as the value-add over just completing transactions.  Big data is revolutionizing how supplier networks form, grow, proliferate into new markets and mature over time. Transactions aren’t the only goal, creating knowledge-sharing networks is, based on the insights gained from big data analytics. The following graphic from Business Ecosystems Come Of Age (Deloitte University Press) (free, no opt-in) illustrates the progression of supply chains from networks or webs, where knowledge sharing becomes a priority.

figure 1 big data scm

  • Big data and advanced analytics are being integrated into optimization tools, demand forecasting, integrated business planning and supplier collaboration & risk analytics at a quickening pace. These are the top four supply chain capabilities that Delotte found are currently in use form their recent study, Supply Chain Talent of the Future Findings from the 3rd Annual Supply Chain Survey (free, no opt-in). Control tower analytics and visualization are also on the roadmaps of supply chain teams currently running big data pilots.

Figure 2 use of supply chain capabilities

  • 64% of supply chain executives consider big data analytics a disruptive and important technology, setting the foundation for long-term change management in their organizations.  SCM World’s latest Chief Supply Chain Officer Report provides a prioritization of the most disruptive technologies for supply chains as defined by the organizations’ members.  The following graphic from the report provides insights into how senior supply chain executives are prioritizing big data analytics over other technologies.

disruptive tech

  • Using geoanalytics based on big data to merge and optimize delivery networks.  The Boston Consulting Group provides insights into how big data is being put to use in supply chain management in the article Making Big Data Work: Supply Chain Management (free, opt-in). One of the examples provided is how the merger of two delivery networks was orchestrated and optimized using geoanalytics. The following graphic is from the article. Combining geoanalytics and big data sets could drastically reduce cable TV tech wait times and driving up service accuracy, fixing one of the most well-known service challenges of companies in that business.

Figure 4 geoanalytics

figure 6 big data


figure 7 big data

  • Greater contextual intelligence of how supply chain tactics, strategies and operations are influencing financial objectives.  Supply chain visibility often refers to being able to see multiple supplier layers deep into a supply network.  It’s been my experience that being able to track financial outcomes of supply chain decisions back to financial objectives is attainable, and with big data app integration to financial systems, very effective in industries with rapid inventory turns. Source: Turn Big Data Into Big Visibility.

figure 8 traceability

  • Traceability and recalls are by nature data-intensive, making big data’s contribution potentially significant. Big data has the potential to provide improved traceability performance and reduce the thousands of hours lost just trying to access, integrate and manage product databases that provide data on where products are in the field needing to be recalled or retrofitted.
  • Increasing supplier quality from supplier audit to inbound inspection and final assembly with big data. IBM has developed a quality early-warning system that detects and then defines a prioritization framework that isolates quality problem faster than more traditional methods, including Statistical Process Control (SPC). The early-warning system is deployed upstream of suppliers and extends out to products in the field.

2015 Roundup Of Analytics, Big Data & Business Intelligence Forecasts And Market Estimates

  • NYC SkylineSalesforce (NYSE:CRM) estimates adding analytics and Business Intelligence (BI) applications will increase their Total Addressable Market (TAM) by $13B in FY2014.
  • 89% of business leaders believe Big Data will revolutionize business operations in the same way the Internet did.
  • 83% have pursued Big Data projects in order to seize a competitive edge.

Despite the varying methodologies used in the studies mentioned in this roundup, many share a common set of conclusions. The high priority in gaining greater insights into customers and their unmet needs, more precise information on how to best manage and simplify sales cycles, and how to streamline service are common themes.

The most successful Big Data uses cases revolve around enterprises’ need to get beyond the constraints that hold them back from being more attentive and responsive to customers.

Presented below is a roundup of recent forecasts and estimates:

  • Wikibon projects the Big Data market will top $84B in 2026, attaining a 17% Compound Annual Growth Rate (CAGR) for the forecast period 2011 to 2026. The Big Data market reached $27.36B in 2014, up from $19.6B in 2013. These and other insights are from Wikibon’s excellent research of Big Data market adoption and growth. The graphic below provides an overview of their Big Data Market Forecast.  Source: Executive Summary: Big Data Vendor Revenue and Market Forecast, 2011-2026.

Wikibon big data forecast

  • IBM and SAS are the leaders of the Big Data predictive analytics market according to the latest Forrester Wave™: Big Data Predictive Analytics Solutions, Q2 2015. The latest Forrester Wave is based on an analysis of 13 different big data predictive analytics providers including Alpine Data Labs, Alteryx, Angoss Software, Dell, FICO, IBM,, Microsoft, Oracle, Predixion Software, RapidMiner, SAP, and SAS. Forrester specifically called out Microsoft Azure Learning is an impressive new entrant that shows the potential for Microsoft to be a significant player in this market. Gregory Piatetsky (@KDNuggets) has done an excellent analysis of the Forrester Wave Big Data Predictive Analytics Solutions Q2 2015 report here. Source: Courtesy of Predixion Software: The Forrester Wave™: Big Data Predictive Analytics Solutions, Q2 2015 (free, no opt-in).

Forrester Wave Big Data Predictive Analytics

  • IBM, KNIME, RapidMiner and SAS are leading the advanced analytics platform market according to Gartner’s latest Magic Quadrant. Gartner’s latest Magic Quadrant for advanced analytics evaluated 16 leading providers of advanced analytics platforms that are used to building solutions from scratch. The following vendors were included in Gartner’s analysis: Alpine Data Labs, Alteryx, Angoss, Dell, FICO, IBM, KNIME, Microsoft, Predixion, Prognoz, RapidMiner, Revolution Analytics, Salford Systems, SAP, SAS and Tibco Software, Gregory Piatetsky (@KDNuggets) provides excellent insights into shifts in Magic Quadrant for Advanced Platform rankings here.  Source: Courtesy of RapidMinerMagic Quadrant for Advanced Analytics Platforms Published: 19 February 2015 Analyst(s): Gareth Herschel, Alexander Linden, Lisa Kart (reprint; free, no opt-in).

Magic Quadrant for Advanced Analytics Platforms

  • Salesforce estimates adding analytics and Business Intelligence (BI) applications will increase their Total Addressable Market (TAM) by $13B in FY2014. Adding new apps in analytics is projected to increase their TAM to $82B for calendar year (CY) 2018, fueling an 11% CAGR in their total addressable market from CY 2013 to 2018. Source: Building on Fifteen Years of Customer Success Salesforce Analyst Day 2014 Presentation (free, no opt in).

Salesforce Graphic

  • 89% of business leaders believe big data will revolutionize business operations in the same way the Internet did. 85% believe that big data will dramatically change the way they do business. 79% agree that ‘companies that do not embrace Big Data will lose their competitive position and may even face extinction.’ 83% have pursued big data projects in order to seize a competitive edge. The top three areas where big data will make an impact in their operations include: impacting customer relationships (37%); redefining product development (26%); and changing the way operations is organized (15%).The following graphic compares the top six areas where big data is projected to have the greatest impact in organizations over the next five years. Source: Accenture, Big Success with Big Data: Executive Summary (free, no opt in).

Big Data Big Success Graphic

Frost & Sullivan Graphic


global text market graphic


  • Customer analytics (48%), operational analytics (21%), and fraud & compliance (21%) are the top three use cases for Big Data. Datameer’s analysis of the market also found that the global Hadoop market will grow from $1.5B in 2012 to $50.2B in 2020, and financial services, technology and telecommunications are the leading industries using big data solutions today. Source: Big Data: A Competitive Weapon for the Enterprise.

Big Data Use Cases in Business

  • 37% of Asia Pacific manufacturers are using Big Data and analytics technologies to improve production quality management. IDC found manufacturers in this region are relying on these technologies to reduce costs, increase productivity, and attract new customers. Source: Big Data and Analytics Core to Nex-Gen Manufacturing.

big data in manufacturing

  • Supply chain visibility (56%), geo-location and mapping data (47%) and product traceability data (42%) are the top three potential areas of Big Data opportunity for supply chain management. Transport management, supply chain planning, & network modeling and optimization are the three most popular applications of Big Data in supply chain initiatives. Source: Supply Chain Report, February 2015.

Big data use in supply chains

  • Finding correlations across multiple disparate data sources (48%), predicting customer behavior (46%) and predicting product or services sales (40%) are the three factors driving interest in Big Data analytics. These and other fascinating findings from InformationWeek’s 2015 Analytics & BI Survey provide a glimpse into how enterprises are selecting analytics applications and platforms. Source: Information Week 2015 Analytics & BI Survey.

factors driving interest in big data analysis

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Where Big Data Jobs Will Be In 2015

Big Data Drives Rapid ChangesDemand for Computer Systems Analysts with big data expertise increased 89.9% in the last twelve months and 85.40% for Computer and Information Research Scientists.

Demand for Python programming expertise increased 96.9% in big-data related positions in the last twelve months.

These and other key insights are from a recent analysis completed of big data hiring trends using WANTED Analytics, the leading provider of data analytics on the workplace.  For purposes of this analysis, the term “big data” is comprised of the four skill sets of data analysis, data acquisition, data mining and data structures. The WANTED Analytics taxonomy references these skill sets when queries are made on the term “big data”.

The company currently maintains a database of more than one billion unique job listings and is collecting hiring trend data from more than 150 countries. WANTED Analytics has never been a client, they provided complimentary access based on my requesting a trial account. Many Forbes readers are interested in staying current on big data hiring trends, which led me to complete this analysis.

Key Take-aways include the following:

  • Demand for big data expertise across a range of occupations saw significant growth over the last twelve months. There was a 123.60% jump in demand for Information Technology Project Managers with big data expertise, and an 89.8% increase for Computer Systems Analysts. The following table provides an overview of the distribution of open positions by occupation and the percentage growth in job demand over time.

job growth matrix

  • The five leading industries with the most job openings requiring big data expertise include Professional, Scientific and Technical Services (27.14%), Information Technologies (18.89%), Manufacturing (12.35%), Retail Trade (9.62%) and Sustainability, Waste Management & Remediation Services (8.20%). The following graphic shows the distribution of open positions between September 1, 2014 to today, December 29, 2014:

top 20 industries hiring

  • The Hiring Scale is 76 for jobs that require big data skills with 12 candidates per job opening as of December 29, 2014.  The higher the Hiring Scale score, the more difficult it is for employers to find the right applicants for open positions. Nationally an average job posting for an IT professional with cloud computing expertise is open just 47 days.

big data hiring scale

  • The median salary for professionals with big data expertise is $103,000 a year. Sample jobs in this category include Big Data Solution Architect, Linux Systems and Big Data Engineer, Big Data Platform Engineer, Lead Software Engineer, Big Data (Java, Hadoop, SQL) and others.  The distribution of median salaries across all industries shown below:

big data market salary

  • San Jose – Sunnyvale – Santa Clara, CA, San Francisco – Oakland – Fremont, CA, and Washington – Arlington – Alexandria, DC are the top three U.S. employment markets for big data related jobs as of today.  Mapping the distribution of job volume, salary range, candidate supply, posting period and hiring scale by Metropolitan Statistical Area (MSA) or states and counties is supported by WANTED Analytics and shown in the following graphic. A summary of the top twenty employment markets is also shown following the map:

US Hiring Map

US Top Markets

  • Cisco (NASDAQ:CSCO), IBM (NYSE: IBM) and Oracle (NYSE:ORCL) have the most open big data-related positions today. Cisco, its supplier, partner and support ecosystem companies have 3,613 related big data positions available.  The following table shows the top ten big data employers today, the distribution of jobs, and the number of new jobs added over the last year.

top ten employers

  • Python programming (96.90%), Linux expertise (76.60%) and Structured Query Language (SQL) (76%) are the three most in-demand skills in positions that mention big data as a requirement.  The following table provides an overview of the top 10 most in-demand skills:


Ten Ways Big Data Is Revolutionizing Manufacturing

quality1McKinsey & Company recently published How Big Data Can Improve Manufacturing which provides insightful analysis of how big data and advanced analytics can streamline biopharmaceutical, chemical and discrete manufacturing.

The article highlights how manufacturers in process-based industries are using advanced analytics to increase yields and reduce costs. Manufacturers have an abundance of operational and shop floor data that is being used for tracking today.  The McKinsey article shows through several examples how big data and advanced analytics applications and platforms can deliver operational insights as well.

The following graphic from the article illustrates how big data and advanced analytics are streamlining manufacturing value chains by finding the core determinants of process performance, and then taking action to continually improve them:

Advanced Analytics Big Data in Manufacturing

Big Data’s Impact on Manufacturing Is Growing

In addition to the examples provided in the McKinsey article, there are ten ways big data is revolutionizing manufacturing:

  • Increasing the accuracy, quality and yield of biopharmaceutical production.  It is common in biopharmaceutical production flows to monitor more than 200 variables to ensure the purity of the ingredients as well as the substances being made stay in compliance. One of the many factors that makes biopharmaceutical production so challenging is that yields can vary from 50 to 100% for no immediately discernible reason. Using advanced analytics, a manufacturer was able to track the nine parameters that most explained yield variation. Based on this insight they were able to increase the vaccine’s yield by 50%, worth between $5M to $10M in yearly savings for the single vaccine alone.
  • Accelerating the integration of IT, manufacturing and operational systems making the vision of Industrie 4.0 a reality. Industrie 4.0 is a German government initiative that promotes automation of the manufacturing industry with the goal of developing Smart Factories. Big data is already being used for optimizing production schedules based on supplier, customer, machine availability and cost constraints. Manufacturing value chains in highly regulated industries that rely on German suppliers and manufacturers are making rapid strides with Industrie 4.0 today.  As this initiative serves as a catalyst to galvanize diverse multifunctional departments together, big data and advanced analytics will become critical to its success.
  • Better forecasts of product demand and production (46%), understanding plant performance across multiple metrics (45%) and providing service and support to customers faster (39%) are the top three areas big data can improve manufacturing performance.   These findings are from a recent survey LNS Research and MESA International completed to see where big data is delivering the greatest manufacturing performance improvements today. You can find the original blog post here.

LNS Graphic

  • Integrating advanced analytics across the Six Sigma DMAIC (Define, Measure, Analyze, Improve and Control) framework to fuel continuous improvement.  Getting greater insights into how each phase of a DMAIC-driven improvement program is working, and how the efforts made impact all other areas of manufacturing performance is nascent today. This area shows great potential to make production workflows more customer-driven than ever before.
  • Greater visibility into supplier quality levels, and greater accuracy in predicting supplier performance over time.  Using big data and advanced analytics, manufacturers are able to view product quality and delivery accuracy in real-time, making trade-offs on which suppliers receive the most time-sensitive orders.  Managing to quality metrics becomes the priority over measuring delivery schedule performance alone.
  • Measuring compliance and traceability to the machine level becomes possible. Using sensors on all machinery in a production center provides operations managers with immediate visibility into how each is operating. Having advanced analytics can also show quality, performance and training variances by each machine and its operators.  This is invaluable in streamlining workflows in a production center, and is becoming increasingly commonplace.
  • Selling only the most profitable customized or build-to-order configurations of products that impact production the least.  For many complex manufacturers, customized or build-to-order products deliver higher-than-average gross margins yet also costs exponentially more if production processes aren’t well planned.  Using advanced analytics, manufacturers are discovering which of the myriad of build-to-order configurations they can sell with the most minimal impact to existing production schedules to the machine scheduling, staffing and shop floor level.
  • Breaking quality management and compliance systems out of their silos and making them a corporate priority.  It’s time for more manufacturers to take a more strategic view of quality and quit being satisfied with standalone, siloed quality management and compliance systems.  The McKinsey article and articles listed at the end of this post provide many examples of how big data and analytics are providing insights into which parameters matter most to quality management and compliance. The majority of these parameters are corporate-wide, not just limited to quality management or compliance departments alone.
  • Quantify how daily production impacts financial performance with visibility to the machine level. Big data and advanced analytics are delivering the missing link that can unify daily production activity to the financial performance of a manufacturer.  Being able to know to the machine level if the factory floor is running efficiently, production planners and senior management know how best to scale operations.  By unifying daily production to financial metrics, manufacturers have a greater chance of profitably scaling their operations.
  • Service becomes strategic and a contributor to customers’ goals by monitoring products and proactively providing preventative maintenance recommendations.  Manufacturers are starting to look at the more complex products they produce as needing an operating system to manage the sensors onboard. These sensors report back activity and can send alerts for preventative maintenance. Big data and analytics will make the level of recommendations contextual for the first time so customers can get greater value.  General Electric is doing this today with its jet engines and drilling platforms for example.

Additional sources of information on Big Data in Manufacturing:


84% Of Enterprises See Big Data Analytics Changing Their Industries’ Competitive Landscapes In The Next Year

NYC Skyline87% of enterprises believe Big Data analytics will redefine the competitive landscape of their industries within the next three years. 89% believe that companies that do not adopt a Big Data analytics strategy in the next year risk losing market share and momentum.

These and other key findings are from a Accenture and General Electric study published this month on how the combination of Big Data analytics and the Internet of Things (IoT) are redefining the competitive landscape of entire industries. Accenture and GE define the Industrial Internet as the use of sensor, software, machine-to-machine learning and other technologies to gather and analyze data from physical objects or other large data streams, and then use those analyses to manage operations and in some cases to offer new, valued-added services.

Big Data Analytics Now Seen As Essential For Competitive Growth

The Industrial Internet is projected to be worth $500B in worldwide spending by 2020, taking into account hardware, software and services sales according to Wikibon and previously published research from General Electric. This finding and others can be found on the home page of the Accenture and GE study here: How the Industrial Internet is Changing the Competitive Landscape of Industries.

The study also shows that many enterprises are investing the majority of their time in analysis (36%) and just 13% are using Big Data analytics to predict outcomes, and only 16% using their analytics applications to optimize processes and strategies. Moving beyond analysis to predictive analytics and optimization is the upside potential the majority of the C-level respondents see as essential to staying competitive in their industries in the future.

A summary of results and the methodology used are downloadable in PDF form (free, no opt in) from this link: Industrial Internet Insights Report For 2015.

Key take-aways from the study include the following:

  • 73% of companies are already investing more than 20% of their overall technology budget on Big Data analytics, and just over two in ten are investing more than 30%. 76% of executives expect spending levels to increase. The following graphic illustrates these results:

Figure 1 big data investments

  • Big Data analytics has quickly become the highest priority for aviation (61%), wind (45%) and manufacturing (42%) companies.  The following graphic provides insights into the relative level of importance of Big Data analytics relative to other priorities in the enterprises interviewed in the study:

Figure 2 industry overview

  • 74% of enterprises say that their main competitors are already using Big Data analytics to successfully differentiate their competitive strengths with clients, the media, and investors. 93% of enterprises are seeing new competitors in their market using Big Data analytics as a key differentiation strategy.  The single greatest risk enterprises see from not implementing a Big Data strategy is that competitors will gain market share at their expense.  Please see the following graphic for a comparison of the risks of not implementing Big Data strategy.

Figure 3 Unable to Implement

  • 65% of enterprises are focused on monitoring assets to identify operating issues for more proactive maintenance. 58% report having capabilities such as connecting equipment to collect operating data and analyzing the data to produce insights. The following graphic provides an overview of Big Data monitoring survey results:

Figure 4 big data monitoring

  • Increasing profitability (60%), gaining a competitive advantage (57%) and improving environmental safety and emissions compliance (55%) are the three highest industry priorities according to the survey. The following table provides an analysis of the top business priorities by industry for the next three years with the shaded areas indicating the highest-ranked priorities by industry:

Figure 5 industry priorities

  • The top three challenges enterprises face in implementing Big Data initiatives include the following: system barriers between departments prevent collection and correlation of data for maximum impact (36%); security concerns are impacting enterprises’ ability to implement a wide-scale Big Data initiative (35%); and  consolidation of disparate data and being able to use the resulting data store (29%), third. The following graphic provides an overview of the top three challenges organizations face in implementing Big Data initiatives:

Figure 6 challenges for big data analytics


Roundup Of Analytics, Big Data & Business Intelligence Forecasts And Market Estimates, 2014

NYC SkylineFrom manufacturers looking to gain greater insights into streamlining production, reducing time-to-market and increasing product quality to financial services firms seeking to upsell clients, analytics is now essential for any business looking to stay competitive.  Marketing is going through its own transformation, away from traditional tactics to analytics- and data-driven strategies that deliver measurable results.

Analytics and the insights they deliver are changing competitive dynamics daily by delivering greater acuity and focus.  The high level of interest and hype surrounding analytics, Big Data and business intelligence (BI) is leading to a proliferation of market projections and forecasts, each providing a different perspective of these markets.

Presented below is a roundup of recent forecasts and market estimates:

  • The Advanced and Predictive Analytics (APA) software market is projected from grow from $2.2B in 2013 to $3.4B in 2018, attaining a 9.9% CAGR in the forecast period.  The top 3 vendors in 2013 based on worldwide revenue were SAS ($768.3M, 35.4% market share), IBM ($370.3M, 17.1% market share) and Microsoft ($64.9M, 3% market share).  IDC commented that simplified APA tools that provide less flexibility than standalone statistical models tools yet have more intuitive graphical user interfaces and easier-to-use features are fueling business analysts’ adoption.  Source:
  • A.T. Kearney forecasts global spending on Big Data hardware, software and services will grow at a CAGR of 30% through 2018, reaching a total market size of $114B.  The average business expects to spend $8M on big data-related initiatives this year. Source: Beyond Big: The Analytically Powered Organization.
  • Cloud-based Business Intelligence (BI) is projected to grow from $.75B in 2013 to $2.94B in 2018, attaining a CAGR of 31%.  Redwood Capital’s recent Sector Report on Business Intelligence  (free, no opt in) provides a thorough analysis of the current and future direction of BI.  Redwood Capital segments the BI market into traditional, mobile, cloud and social business intelligence.   The following two charts from the Sector Report on Business Intelligence  illustrate how Redwood Capital sees the progression of the BI market through 2018.

redwood capital global intelligence market size

  • Enterprises getting the most value out of analytics and BI have leaders that concentrate more on collaboration, instilling confidence in their teams, and creating an active analytics community, while laggards focus on technology alone.  A.T. Kearney and Carnegie Mellon University recently surveyed 430 companies around the world, representing a wide range of geographies and industries, for the inaugural Leadership Excellence in Analytic Practices (LEAP) study.  You can find the study here.  The following is a graphic from the study comparing the characteristics of leaders and laggards’ strategies for building a culture of analytics excellence.

leaders and laggards2

  • The worldwide market for Big Data related hardware, software and professional services is projected to reach $30B in 2014.  Signals and System Telecom forecasts the market will attain a Compound Annual Growth Rate (CAGR) of 17% over the next 6 years.  Signals and Systems Telecom’s report forecasts Big Data will be a $76B market by 2020.  Source:
  • Big Data is projected to be a $28.5B market in 2014, growing to $50.1B in 2015 according to Wikkbon.  Their report, Big Data Vendor Revenue and Market Forecast 2013-2017 is outstanding in its accuracy and depth of analysis.  The following is a graphic from the study, illustrating Wikibon’s Big Data market forecast broken down by market component through 2017.

Big Data Wikibon

  • SAPIBMSASMicrosoftOracle, Information Builders, MicroStrategy, and Actuate are market leaders in BI according to Forrester’s latest Wave analysis of BI platforms.  Their report, The Forrester Wave™: Enterprise Business Intelligence Platforms, Q4 2013 (free PDF, no opt in, courtesy of SAS) provides a thorough analysis of 11 different BI software providers using the research firm’s 72-criteria evaluation methodology.
  • Amazon Web Services, Cloudera, Hortonworks, IBM, MapR Technologies, Pivotal Software, and Teradata are Big Data Hadoop market leaders according to Forrester’s latest Wave analysis of Hadoop Solutions.  Their report, The Forrester Wave™: Big Data Hadoop Solutions, Q1 2014 (free PDF, no opt in, courtesy of MapR Technologies) provides a thorough analysis of nine different Big Data Hadoop software providers using the research firm’s 32-criteria evaluation methodology.
  • IDC forecasts the server market for high performance data analysis (HPDA) will grow at a 23.5% compound annual growth rate (CAGR) reaching $2.7B by 2018.  In the same series of studies IDC forecasts the related storage market will expand to $1.6B also in 2018. HPDA is the term IDC created to describe the formative market for big data workloads using HPC. Source:
  • Global Big Data technology and services revenue will grow from $14.26B in 2014 to $23.76B in 2016, attaining a compound annual growth rate of 18.55%.  These figures and a complete market analysis are available in IDC’s Worldwide Big Data Technology and Services 2012 – 2016 Forecast.  You can download the full report here (free, no opt-in): Worldwide Big Data Technology and Services 2012 – 2016 Forecast.

big data analytics by market size

  • Financial Services firms are projected to spend $6.4B in Big Data-related hardware, software and services in 2015, growing at a CAGR of 22% through 2020.  Software and internet-related companies are projected to spend $2.8B in 2015, growing at a CAGR of 26% through 2020.  These and other market forecasts and projections can be found in Bain & Company’s Insights Analysis, Big Data: The Organizational Challenge.  An infographic of their research results are shown below.

Big-Data-infographic-Bain & Company

potential payback of big data initiatives

BCG’s Value Creators Report Shows How Software Is Driving New Business Models

boston-300x211Boston Consulting Group (BCG) recently released their fifth annual technology, media and telecommunications (TMT) value report. The 2013 TMT Value Creators Report: The Great Software Transformation, How to Win as Technology Changes the World (free, opt-in required, 41 pgs).

The five trends that serve as the foundation of this report include the increasing pervasiveness of software, affordable small devices, ubiquitous broadband connectivity, big-data analytics and cloud computing.  BCG’s analysis illustrates how the majority of TMT companies that deliver the most value to shareholders are concentrating on the explosive growth of new markets, the rise of software-enabled digital metasystems, and for many, both.

The study is based on an analysis of 191 companies, 76 in the technology industry, 62 from media and 53 from telecom.  To review the methodology of this study please see page 28 of the report.

Here are the key takeaways from this years’ BCG TMT Value Creators Report:

  • BCG is predicting 1B smartphones will be sold in 2013, the first year their sales will have exceeded those of features phones.  By 2018, there will be more than 5B “post-PC” products (tablets & smartphones) in circulation. There are nearly as many mobile connections in the world as people (6.8B) according to the United Nation’s International Telecommunication Union (ITU).

bcg figure 1

  • 27 terabytes of data is generated every second through the creation of video, images social networks, transactional and enterprise-based systems and networks.  90% of the data that is stored today didn’t exist two years ago, and the annual data growth rate in future years is projected to be 40% to 60% over current levels according to BCG’s analysis.

bcg figure 2

  • The ascent of communications speeds is surpassing Moore’s Law as a structural driver of growth.  BCG completed the following analysis graphing the progression of microprocessor transition count (Moore’s Law) relative to Internet speed (bps) citing Butter’s Law of Photonics which states that the amount of data coming out of an optical fiber is doubling every nine months. BCG states that these dynamics are democratizing information technology and will lead to the cloud computing industry (software and services) reaching nearly $250B in 2017.
    bcg figure 3
  • BCG predicts that India will see a fivefold increase in digitally-influenced spending, ascending from $30B in 2012 to $150B in 2016, among the fastest of all nations globally according to their study. India will also see the value of online purchases increase from $8B in 2012 t5o $50B in 2016.

bcg figure 4

  • 3D printing is forecast to become a $3.1B market by 2016, and will have an economic impact of $550B in 2025, fueling rapid price reductions in 3D printers through 2017.  BCG sees 3D printing, connected travel, genomics and smart grid technologies are central to their digital metasystem.   The following graphic illustrates the key trends in each of these areas along with research findings from BCG and other sources.

bcg figure 5

  • Only 7% of customers are comfortable with their information being used outside of the purpose for which it was originally gathered.

bcg figure 6

  • BCG reports that mobile infrastructure investments in Europe have fallen 67% from 2004 to 2014.  Less than 1% of mobile connections in Europe were 4B as of the end of 2012, compared to 11% in the U.S. and 28% in South Korea.   European operators have also been challenged to monetize mobile data as well, as the following figures illustrate.

bcg figure 7

bcg figure 8

  • Big Data is attracting $19B in funding across five key areas according to BCG’s analysis.  These include consumer data and marketing, enterprise data, analytical tools, vertical markets and data platforms.  A graphical analysis of these investments is shown below.

bcg figure 9

Cloud Predictive Analytics Most Used To Gain Customer Insight

AnalyticsUsing analytics to better understand customer satisfaction, profitability, retention and churn while increasing cross-sell and up-sell are the most dominant uses of cloud-based analytics today.

Jim Ericson and James Taylor presented the results of Decision Management Solutions’ cloud predictive analytics survey this week in the webinar Predictive Analytics in the Cloud 2013 – Opportunities, Trends and the Impact of Big Data.  The research methodology included 350 survey responses, with a Web-based survey used for data collection.  The survey centered on the areas of pre-packaged cloud-based solutions, cloud-based predictive modeling, and cloud deployment of predictive analytics.  You can see a replay of the webinar at this link.

Key takeaways of the study results released during the webinar include the following:

  • Customer Analytics (72%), followed by supply chain, business optimization, marketing optimization (57%), risk and fraud (52%), and marketing (58%) are the areas in which respondents reported the strongest interest.
  • When the customer analytics responses were analyzed in greater depth they showed most interest in customer satisfaction (50%) followed by customer profitability (34%), customer retention/churn (32%), customer management (30%), and cross-sell/up-sell (26%).
  • Adoption was increasingly widespread and growing, with over 90% of respondents reporting that they expected to deploy one or more type of predictive analytics in the cloud solution.
  • Industries with the most impact from predictive analytics include retail (13% more than average), Financial Services (12%) and hardware/software (4%). Lagging industries include health care delivery (-9%), insurance -11%) and (surprisingly) telecommunications (-33%).  The following graphic illustrates the relative impact of cloud-based predictive analytics applications by industry.

Adoption of Cloud-based Predictive Analytics by Industry

  • The most widespread analytics scenarios include prepackaged solutions (52%), cloud-based analytics modeling (47%) and cloud-based analytic embedding of applications (46%).  Comparing the 2011 and 2013 surveys showed significant gains in all three categories, with the greatest being in the area of cloud-based analytic modeling.  This category increased from 51% in 2011 to 75% in 2013, making it the most likely analytics application respondents are going to implement this year.

Comparison of Analytics Applications Most Likely To Deploy, 2011 versus 2013

  • 63% of respondents report that when predictive analytics are tightly integrated into operations using Decision Management, enterprises have the intelligence they need to transform their businesses.

Impact of Predictive Analytics Integration Across The Enterprise

  • Data security and privacy (61%) followed by regulatory compliance (50%) are the two most significant concerns respondent companies have regarding predictive analytics adoption in their companies.  Compliance has increased as a concern significantly since 2011, probably as more financial services firms are adopting cloud computing for mainstream business strategies.

Concerns of Enterprises Who Are Using Cloud-based Predictive Analytics Today

  • Internal cloud deployments (41%) are the most common approach to implementing central cloud platforms, followed by managed vendor clouds (23% and hybrid clouds (23%). Private and managed clouds continue to grow as preferred platforms for cloud-based analytics, as respondents seek greater security and stability of their applications.  The continued adoption of private and managed clouds are a direct result of respondents’ concerns regarding data security, stability, reliability and redundancy.

Approach To Cloud Deployment

  • The study concludes that structured data is the most prevalent type of data, followed by third party data and unstructured data.
  • While there was no widespread impact on results from Big Data, predictive analytics cloud deployments that have a Big Data component are more likely to contribute to a transformative impact on their organizations’ performance.  Similarly those with more experience deploying predictive analytics in the cloud were more likely to use Big Data.
  • In those predictive analytics cloud deployments already operating or having an impact, social media data from the cloud, voice or other audio data, and image or video data were all much more broadly used as the following graphic illustrates.

Which Data Types Deliver The Most Positive Impact In A Big Data Context

Making Analytics Pay In The Enterprise

global-analytics-300x2001With analytics and big data being so heavily hyped today, it is ironic the majority of business analysts often lack access to data and tools they need.

But things are changing with the next generation of analytics software coming to market.  A recent study by The Economist, “Big Data and the Democratisation of Decisions,” shows the severity of the big data analytics problem and which departments need the most support: customer service, human resources, marketing, strategy and business development.  The following is an infographic based on the study’s key findings. To be clear, all companies mentioned in this post are not and never have been clients of mine or companies I have worked for.

Unleashing Greater Insight in the Enterprise

The real analytics payoff in the enterprise begins when business analysts can achieve customer and market insights faster than their competitors.  In the consumer packaged goods industry, every week counts in a new product launch and product lifecycle.  In healthcare, lag times in customer service lead to patients seeking more responsive treatment alternatives.  The net result in each is lost revenue.

Analytics applications and platforms are increasingly being designed for self-service and the needs of business analysts first.  Instead of having to rely on IT for analytics, big data and advanced statistical analysis support, business analysts need to be able to complete projects on their own. Analytics applications are advancing quickly on this self-service dimension, making it possible for business analysts to get complex projects done in a fraction of the time it would have taken IT to staff and complete them.

Alliances and partnerships between analytics software providers are focused on getting business analysts the tools they need so they don’t have to rely on IT so much to get their work done.  The recent partnership announced between Alteryx and Revolution Analytics puts R-based predictive analytics directly in the hands business analysts is a case in point.

What’s noteworthy about this partnership above all others is the option it gives enterprises to integrate big data and other 3rd party sources into a common system of engagement. Business analysts can then use tools to design analytics and reporting workflows that align and stay in step with line-of-business needs over time.

alteryx-gallery1-300x1691Once an application or workflow is complete, business analysts can publish and distribute their analytics applications enterprise-wide. The Alteryx Analytics Gallery (shown to the right) gives customers the opportunity to share their analytics applications with each other.  The gallery is helping business analysts learn from each other, serving as a catalyst for broader analytic consumption.

This is the same model ServiceNow (NYSE:NOW) has been so successful with in the area of IT Service Management.  I attended Knowledge13 earlier this year and found their customer base to be one of the most enthusiastic I’ve ever met.  What ServiceNow has done IT Service Management, Alteryx is on its way to accomplishing in analytics.

Why All This Matters For Customers

Getting analytics applications and tools in the hands of business analysts significantly improves the customer experience and reduce errors at the same time. At Kaiser Permanente, business analysts focus on cost saving projects that improve customer service.

Kaiser has a continual stream of customer interactions across multiple channels going on daily.  Supported by legacy IT systems, Microsoft Excel spreadsheets and manual processes to keep the entire system working, the healthcare provider was seeing patient satisfaction levels drop as they didn’t have a clear view of their customers.  The legacy and manual systems also made coordinating customer service teams very difficult and replicating analytics tools very difficult.


Kaiser Permanente was able to aggregate and cleanse the myriad of data sources they rely on and gain greater insights into their customer’s needs. Creating analytics and reporting workflows that business analysts and lean leaders in their Service Organization use to stay on top of customer needs has led to a five-fold increase in customer service performance according to Greg Hall, Senior Service Optimization Leader.


Get every new post delivered to your Inbox.

Join 409 other followers

%d bloggers like this: