Posts from the ‘Big Data Forecast’ Category

Jun 2

10 Charts That Will Change Your Perspective Of Big Data’s Growth

Worldwide Big Data market revenues for software and services are projected to increase from $42B in 2018 to $103B in 2027, attaining a Compound Annual Growth Rate (CAGR) of 10.48% according to Wikibon.
Forrester predicts the global Big Data software market will be worth $31B this year, growing 14% from the previous year. The entire global software market is forecast to be worth $628B in revenue, with $302B from applications.
According to an Accenture study, 79% of enterprise executives agree that companies that do not embrace Big Data will lose their competitive position and could face extinction. Even more, 83%, have pursued Big Data projects to seize a competitive edge.
59% of executives say Big Data at their company would be improved through the use of AI according to PwC.

Sales and Marketing, Research & Development (R&D), Supply Chain Management (SCM) including distribution, Workplace Management and Operations are where advanced analytics including Big Data are making the greatest contributions to revenue growth today. McKinsey Analytics’ study Analytics Comes of Age, published in January 2018 (PDF, 100 pp., no opt-in) is a comprehensive overview of how analytics technologies and Big Data are enabling entirely new ecosystems, serving as a foundational technology for Artificial Intelligence (AI). McKinsey finds that analytics and Big Data are making the most valuable contributions in the Basic Materials and High Tech industries. The first chart in the following series of ten is from the McKinsey Analytics study, highlighting how analytics and Big Data are revolutionizing many of the foundational business processes of Sales and Marketing.

The following ten charts provide insights into Big Data’s growth:

Nearly 50% of respondents to a recent McKinsey Analytics survey say analytics and Big Data have fundamentally changed business practices in their sales and marketing functions. Also, more than 30% say the same about R&D across industries, with respondents in High Tech and Basic Materials & Energy report the greatest number of functions being transformed by analytics and Big Data. Source: Analytics Comes of Age, published in January 2018 (PDF, 100 pp., no opt-in).

Worldwide Big Data market revenues for software and services are projected to increase from $42B in 2018 to $103B in 2027, attaining a Compound Annual Growth Rate (CAGR) of 10.48%. As part of this forecast, Wikibon estimates the worldwide Big Data market is growing at an 11.4% CAGR between 2017 and 2027, growing from $35B to $103B. Source: Wikibon and reported by Statista.

According to NewVantage Venture Partners, Big Data is delivering the most value to enterprises by decreasing expenses (49.2%) and creating new avenues for innovation and disruption (44.3%). Discovering new opportunities to reduce costs by combining advanced analytics and Big Data delivers the most measurable results, further leading to this category being the most prevalent in the study. 69.4% have started using Big Data to create a data-driven culture, with 27.9% reporting results. Source: NewVantage Venture Partners, Big Data Executive Survey 2017 (PDF, 16 pp.)

The Hadoop and Big Data Market are projected to grow from $17.1B in 2017 to $99.31B in 2022 attaining a 28.5% CAGR. The greatest period of projected growth is in 2021 and 2022 when the market is projected to jump $30B in value in one year. Source: StrategyMRC and reported by Statista.

Big Data applications and analytics is projected to grow from $5.3B in 2018 to $19.4B in 2026, attaining a CAGR of 15.49%. Big Data market worldwide includes Professional Services is projected to grow from $16.5B in 2018 to $21.3B in 2026. Source: Wikibon and reported by Statista.

Comparing the worldwide demand for advanced analytics and Big Data-related hardware, services and software, the latter category’s dominance becomes clear. The software segment is projected to increase the fastest of all categories, increasing from $14B in 2018 to $46B in 2027 attaining a CAGR of 12.6%. Sources: Wikibon; SiliconANGLE; Statista estimates and reported by Statista.

Advanced analytics and Big Data revenue in China are projected to be worth ¥57.8B ($9B) by 2020. The Chinese market is predicted to be one of the fastest growing globally, growing at a CAGR of 31.72% in the forecast period. Sources: Social Sciences Academic Press (China) and Statista.

Non-relational analytic data stores are projected to be the fastest growing technology category in Big Data, growing at a CAGR of 38.6% between 2015 and 2020. Cognitive software platforms (23.3% CAGR) and Content Analytics (17.3%) round out the top three fastest growing technologies between 2015 and 2020. Source: Statista.

A decentralized general-merchandise retailer that used Big Data to create performance group clusters saw sales grow 3% to 4%. Big Data is the catalyst of a retailing industry makeover, bringing greater precision to localization than has been possible before. Big Data is being used today to increase the ROI of endcap promotions, optimize planograms, help to improve upsell and cross-sell sales performance and optimize prices on items that drive the greatest amount of foot traffic. Source: Use Big Data to Give Local Shoppers What They Want, Boston Consulting Group, February 8, 2018.

84% of enterprises have launched advanced analytics and Big Data initiatives to bring greater accuracy and accelerate their decision-making Big Data initiatives focused on this area also have the greatest success rate (69%) according to the most recent NewVantage Venture Partners Survey. Over a third of enterprises, 36%, say this area is their top priority for advanced analytics and Big Data investment. Sources: NewVantage Venture Partners Survey and Statista.

Additional Big Data Information Sources:

4 Pain Points of Big Data and how to solve them, Digital McKinsey via Medium, November 10, 2017

53% Of Companies Are Adopting Big Data Analytics, Forbes, December 24, 2017

6 Predictions For The $203 Billion Big Data Analytics Market, Forbes, Gil Press, January 20, 2017

Analytics Comes of Age, McKinsey Analytics, January 2018 (PDF, 100 pp.)

Big Data & Analytics Is The Most Wanted Expertise By 75% Of IoT Providers, Forbes, August 21, 2017

Big Data 2017 – Market Statistics, Use Cases, and Trends, Calsoft (36 pp., PDF)

Big Data and Business Analytics Revenues Forecast to Reach $150.8 Billion This Year, Led by Banking and Manufacturing Investments, According to IDC, March 14, 2017

Big Data Executive Survey 2018, Data and Innovation – How Big Data and AI are Driving Business Innovation, NewVantage Venture Partners, January 2018 (PDF, 18 pp.)

Big Data Tech Hadoop and Spark Get Slow Start in Enterprise, Information Week, March 20, 2018

Big Success With Big Data, Accenture (PDF, 12 pp.)

Gartner Survey Shows Organizations Are Slow to Advance in Data and Analytics, Gartner, February 5, 2018

How Big Data and AI Are Driving Business Innovation in 2018, MIT Sloan Management Review, February 5, 2018

IDC forecasts big growth for Big Data, Analytics Magazine. April 2018

IDC Worldwide Big Data Technology and Services 2012 – 2015 Forecast, Courtesy of EC Europa (PDF, 34 pp.)

Midyear Global Tech Market Outlook For 2017 To 2018, Forrester, September 25, 2017 (client access reqd.)

Oracle Industry Analyst Reports – Data-rich website of industry analyst reports

Ten Ways Big Data Is Revolutionizing Marketing And Sales, Forbes, May 9, 2016

The Big Data Payoff: Turning Big Data into Business Value, CAP Gemini & Informatica Study, (PDF, 12 pp.)

The Forrester Wave™: Enterprise BI Platforms With Majority Cloud Deployments, Q3 2017 courtesy of Oracle

May 26

The Best Big Data Companies And CEOs To Work For In 2018

Employees would most recommend Dataiku, StreamSets, 1010 Data, Reltio, Looker, SAP, Google, Salesforce, Tamr, DataRobot, Domo, Microsoft, Qubole, Attivio and Informatica to friends interested in working for business analytics, data science and machine learning, big data systems and platforms, or data management and integration company in 2018.
These 15 companies are the highest rated in their fields of expertise as defined by Computer Reseller News as of May 11, 2018.
Nine out of 10 employees of these top 15 companies would recommend the company they work for to a friend.

Forbes readers’ most common requests center on who the best companies are to work for in analytics, big data, data management, data science and machine learning. The latest Computer Reseller News‘ 2018 Big Data 100 list of companies is used to complete the analysis as it is an impartial, independent list aggregated based on CRN’s analysis and perspectives of the market. Using the CRN list as a foundation, the following analysis captures the best companies in their respective areas today.

Using the 2018 Big Data 100 CRN list as a baseline to compare the Glassdoor scores of the (%) of employees who would recommend this company to a friend and (%) of employees who approve of the CEO, the following analysis was completed today. 25 companies on the list have very few (less than 15) or no Glassdoor reviews, so they are excluded from the rankings. Based on analysis of Glassdoor score patterns over the last four years, the lower the number of rankings, the more 100% scores for referrals and CEOs. These companies, however, are included in the full data set available here. If the image below is not visible in your browser, you can view the rankings here.

The highest rated CEOs on Glassdoor as of May 11, 2018 include the following:

Dataiku	Florian Douetteau	100%
StreamSets	Girish Pancha	100%
MemSQL	Nikita Shamgunov	100%
1010 Data	Greg Munves	99%
Salesforce.com	Marc Benioff	98%
Attivio	Stephen Baker	98%
SAP	Bill McDermott	97%
Qubole	Ashish Thusoo	97%
Trifacta	Adam Wilson	97%
Zaloni	Ben Sharma	97%
Reltio	Manish Sood	96%
Microsoft	Satya Nadella	96%
Cloudera	Thomas J. Reilly	96%
Sumo Logic	Ramin Sayar	96%
Google	Sundar Pichai	95%
Looker	Frank Bien	93%
MongoDB	Dev Ittycheria	92%
Snowflake Computing	Bob Muglia	92%
Talend	Mike Tuchen	92%
Databricks	Ali Ghodsi	90%
Informatica	Anil Chakravarthy	90%

Apr 22

The State Of Cloud Business Intelligence, 2018

Cloud BI adoption is soaring in 2018, nearly doubling 2016 adoption levels.
Over 90% of Sales & Marketing teams say that Cloud BI is essential for getting their work done in 2018, leading all categories in the survey.
66% of organizations that consider themselves completely successful with Business Intelligence (BI) initiatives currently use the cloud.
Financial Services (62%), Technology (54%), and Education (54%) have the highest Cloud BI adoption rates in 2018.
86% of Cloud BI adopters name Amazon AWS as their first choice, 82% name Microsoft Azure, 66% name Google Cloud, and 36% identify IBM Bluemix as their preferred provider of cloud BI services.

These and other many other fascinating insights are from Dresner Advisory Services 2018 Cloud Computing and Business Intelligence Market Study (client access reqd.) of the Wisdom of Crowds® series of research. The goal of the 7^th annual edition of the study seeks to quantify end-user deployment trends and attitudes toward cloud computing and business intelligence (BI), defined as the technologies, tools, and solutions that employ one or more cloud deployment models. Dresner Advisory Services defines the scope of Business Intelligence (BI) tools and technologies to include query and reporting, OLAP (online analytical processing), data mining and advanced analytics, end-user tools for ad hoc query and analysis, and dashboards for performance monitoring. Please see page 10 of the study for the methodology. The study found the primary barriers to greater cloud BI adoption are enterprises’ concerns regarding data privacy and security.

Key takeaways from the study include the following:

Cloud BI’s importance continues to accelerate in 2018, with the majority of respondents considering it an important element of their broader analytics strategies. The study found that mean level of sentiment rose from 2.68 to 3.22 (above the level of “important”) between 2017 and 2018, indicating the increased importance of Cloud BI over the last year. By region, Asia-Pacific respondents continue to be the strongest proponents of cloud computing regarding both adjusted mean (4.2 or “very important”) and levels of criticality. The following graphic illustrates Cloud BI’s growing importance between 2012 and 2018.

Over 90% of Sales & Marketing teams say Cloud BI apps are important to getting their work done in 2018, leading all respondent categories in the survey. The study found that Cloud BI importance in 2018 is highest among Sales/Marketing and Executive Management respondents. One of the key factors driving this is the fact that both Sales & Marketing and Executive Management are increasingly relying on cloud-based front office applications and services that are integrated with and generate cloud-based data to track progress towards goals.

Cloud BI is most critical to Financial Services & Insurance, Technology, and Retail & Wholesale Trade industries. The study recorded its highest-ever levels of Cloud Bi importance in 2018. Financial Services has the highest weighted mean interest in cloud BI (3.8, which approaches “very important” status shown in the figure below). Technology organizations, where half of the respondents say cloud BI is “critical” or “very important,” are the next most interested. Close to 90% of Retail/Wholesale respondents say SaaS/cloud BI is at least “important” to them. As it has been over time, Healthcare remains the industry least open to managed services for data and business intelligence.

Cloud BI adoption is soaring in 2018, nearly doubling 2016 adoption levels. The study finds that the percentage of respondents using Cloud BI in 2018 nearly doubled from 25% of enterprise users in 2016. Year over year, current use rose from 31% to 49%. In the same time frame, the percentage of respondents with no plans to use cloud BI dropped by half, from 38% to 19%. This study has been completed for the last seven years, showing a steady progression of Cloud BI awareness and adoption, with 2018 being the first one showing the most significant rise in adoption levels ever.

Sales & Marketing leads all departments in current use and planning for Cloud BI applications. Business Intelligence Competency Centers (BICC) are a close second, each with over 60% adoption rates for Cloud BI today. Operations including manufacturing and supply chains and services are the next most likely to use Cloud BI currently. Marketing and BICC lead current adoption and are contributing catalysts of Cloud BI’s soaring growth between 2016 and 2018. Both of these departments often have time-constrained and revenue-driven goals where quantifying contributions to company growth and achievement ad critical.

Financial Services (62%), Technology (54%), and Education (54%) industries have the highest Cloud BI adoption rates in 2018. The retail/wholesale industry has the fourth-highest level of Cloud BI adoption and the greatest number of companies who are currently evaluating Cloud BI today. The least likely current or future users are found in manufacturing and security-sensitive healthcare organizations, where 45% respondents report no plans for cloud-based BI/analytics.

Dashboards, advanced visualization, ad-hoc query, data integration, and self-service are the most-required Cloud BI features in 2018. Sales & Marketing need real-time feedback on key initiatives, programs, strategies, and progress towards goals. Dashboards and advanced visualization features’ dominance of feature requirements reflect this department’s ongoing need for real-time feedback on the progress of their teams towards goals. Reporting, data discovery, and end-user data blending (data preparation) make up the next tier of importance.

Manufacturers have the greatest interest in dashboards, ad-hoc query, production reporting, search interface, location intelligence, and ability to write to transactional applications. Education respondents report the greatest interest in advanced visualization along with data integration, data mining, end-user data blending, data catalog, and collaborative support for group-based analysis. Financial Services respondents are highly interested in advanced visualization and lead all industries in self-service. Healthcare industry respondents lead interest only in in-memory support. Retail/Wholesale and Healthcare industry respondents are the least feature interested overall.

Interest in cloud application connections to Salesforce, NetSuite, and other cloud-based platforms has increased 12% this year. Getting end-to-end visibility across supply chains, manufacturing centers, and distribution channels requires Cloud BI apps be integrated with cloud-based platforms and on-premises applications and data. Expect to see this accelerate in 2019 as Cloud BI apps become more pervasive across Marketing & Sales and Executive Management, in addition to Operations including supply chain management and manufacturing where real-time shop floor monitoring is growing rapidly.

Retail/Wholesale, Business Services, Education and Financial Services & Insurance industries are most interested in Google Analytics connectors to obtain data for their Cloud BI apps. Respondents from Technology industries prioritize Salesforce integration and connectors above all others. Education respondents are most interested in MySQL and Google Drive integration and connectors. Manufacturers are most interested in connectors to Google AdWords, SurveyMonkey, and The Healthcare industry respondents prioritize SAP Cloud BI services and also interested in ServiceNow connectors.

Feb 5

Data Scientist Is The Best Job In America According Glassdoor

Data Scientist has been named the best job in America for three years running, with a median base salary of $110,000 and 4,524 job openings.
DevOps Engineer is the second-best job in 2018, paying a median base salary of $105,000 and 3,369 job openings.
There are 29,187 Software Engineering jobs available today, making this job the most popular regarding Glassdoor postings according to the study.

These and many other fascinating insights are from Glassdoor’s 50 Best Jobs In America For 2018. The Glassdoor Report is viewable online here. Glassdoor’s annual report highlights the 50 best jobs based on each job’s overall Glassdoor Job Score.The Glassdoor Job Score is determined by weighing three key factors equally: earning potential based on median annual base salary, job satisfaction rating, and the number of job openings. Glassdoor’s 2018 report lists jobs that excel across all three dimensions of their Job Score metric. For an excellent overview of the study by Karsten Strauss of Forbes, please see his post, The Best Jobs To Apply For In 2018.

LinkedIn’s 2017 U.S. Emerging Jobs Report found that there are 9.8 times more Machine Learning Engineers working today than five years ago with 1,829 open positions listed on their site as of last month. Data science and machine learning are generating more jobs than candidates right now, making these two areas the fastest growing tech employment areas today.

Key takeaways from the study include the following:

Six analytics and data science jobs are included in Glassdoor’s 50 best jobs In America for 2018. These include Data Scientist, Analytics Manager, Database Administrator, Data Engineer, Data Analyst and Business Intelligence Developer. The complete list of the top 50 jobs is provided below with the analytics and data science jobs highlighted along with software engineering, which has a record 29,817 open jobs today:

Median base salary of the 50 best jobs in America is $91,000 with the average salary of the six analytics and data science jobs being $94,167.
Across all six analytics and data science jobs there are 16,702 openings as of today according to Glassdoor.
Tech jobs make up 20 of Glassdoor’s 50 Best Jobs in America for 2018, up from 14 jobs in 2017.

Source: Glassdoor Reveals the 50 Best Jobs in America for 2018

Nov 11

6M Developers Are Creating Big Data And Advanced Analytics Apps Today

2M developers are working on IoT applications, increasing 34% since the last year.
Over 50% of the developers working on IoT applications are writing software that utilizes sensors in some capacity.
4M enterprise developers play decision-making roles when it comes to selecting organizational IT development resources. Another 5.2 million hold decision-making authority for selecting IT deployment resources.
4M developers (26% of all developers globally) are using the cloud as a development environment today
The APAC region leads the world with approximately 7.4M developers today, followed by EMEA with 7.2M, North America with 4.4M and Latin American with 1.9M.

These and many other fascinating insights are from the Evans Data Corporation Global Developer Population and Demographic Study 2016 (PDF, client access) published earlier this week. The methodology Evans Data has created to produce this report is the most comprehensive developed for aggregating, analyzing and predicting developer populations globally. The study combines Evans Data’s proprietary global developer population modeling with the current results of their semi-annual global developer survey.

Key takeaways from the study include the following:

6M developers (29% of all developers globally) are involved in a Big Data and Advanced Analytics project today. An additional 25% of developers, or 5.3M, are going to begin Big Data and Advanced Analytics projects within the next six 13% or 2.6M of all developers globally are going to start Big Data and Advanced Analytics projects within the next 7 to 12 months. The following graphic provides an overview of the involvement of 21M developers in Big Data and Advanced Analytics projects today. Please click on the image to expand for easier viewing.

4M developers (26% of all developers globally) are using the cloud as a development environment today. Developers creating new apps in the cloud had increased 375% since Evans began measuring developer participation in mobile development in 2009 when just slightly more than 1.2M developers were using the cloud as their development platform. 4.5M developers (21% of all global developers) plan on beginning app development on cloud platforms in the next six months, and 3.9M (18% of all global developers) plan on starting development on the cloud in 7 – 12 months. Please click on the image to expand for easier viewing.

8M developers in APAC (24% of all developers in the region) are currently developing on cloud platforms. 29% of APAC developers are planning to start cloud-based development in six months, and 20% in 7 – 12 months. The following graphic compares the number of developers currently using the cloud as a development environment today and the number who plan to in the future. Please click on the image to expand for easier viewing.

34% of all Commercial Independent Software Vendors (ISVs) globally today (1.8M developers) are using the cloud as a development environment. An additional 1.4M are planning to begin cloud development in the next six months. 28% of developers globally creating apps in the cloud are from custom system integrators (SI) and value-added resellers (VARs). 23% or approximately 1.2M are from enterprises. The following graphic compares the percent of developers by developer segment who are currently creating new apps in cloud environments. Please click on the image to expand for easier viewing.

30% of developers (6.2M developers globally) are currently developing software for connected devices or the Internet of Things today, with an additional 26% planning to begin projects in 6 months. Evans Data found that this increased 34% over the last year. Also, 2.1M developers plan to begin development in this area within the next 7 to 12 months. The following graphic compares the number of developers globally by stage of development for creating software for connected devices or the Internet of Things. Please click on the image to expand for easier viewing.

41% of global developers creating connected device and IoT software today are from 27% are from North America, 24% are from EMEA and 7% from Latin America. There are 6,072,048 developers currently working on connected device and IoT software today globally. The following graphic provides an overview of the distribution of developers creating connected device and IoT software by region today. Please click on the image to expand for easier viewing.

34% of developers actively creating software for connected devices or the Internet of Things work for custom System Integrators (SI) and VARs today. ISVs are the next largest segment of developers working on IoT projects (30%) followed by enterprises (21%). The following graphic provides an overview of the global base of developers creating software for connected devices and IoT. Evans Data found there are 6.1M developers currently creating apps and solutions in this area alone. Please click on the image to expand for easier viewing.

Jul 2

10 Ways Machine Learning Is Revolutionizing Manufacturing

Bottom line: Every manufacturer has the potential to integrate machine learning into their operations and become more competitive by gaining predictive insights into production.

Machine learning’s core technologies align well with the complex problems manufacturers face daily. From striving to keep supply chains operating efficiently to producing customized, built- to-order products on time, machine learning algorithms have the potential to bring greater predictive accuracy to every phase of production. Many of the algorithms being developed are iterative, designed to learn continually and seek optimized outcomes. These algorithms iterate in milliseconds, enabling manufacturers to seek optimized outcomes in minutes versus months.

The ten ways machine learning is revolutionizing manufacturing include the following:

Increasing production capacity up to 20% while lowering material consumption rates by 4%. Smart manufacturing systems designed to capitalize on predictive data analytics and machine learning have the potential to improve yield rates at the machine, production cell, and plant levels. The following graphic from General Electric and cited in a National Institute of Standards (NIST) provides a summary of benefits that are being gained using predictive analytics and machine learning in manufacturing today.

Source: Focus Group: Big Data Analytics for Smart Manufacturing Systems

Providing more relevant data so finance, operations, and supply chain teams can better manage factory and demand-side constraints. In many manufacturing companies, IT systems aren’t integrated, which makes it difficult for cross-functional teams to accomplish shared goals. Machine learning has the potential to bring an entirely new level of insight and intelligence into these teams, making their goals of optimizing production workflows, inventory, Work In Process (WIP), and value chain decisions possible.

Source: GE Global Research Stifel 2015 Industrials Conference

Improving preventative maintenance and Maintenance, Repair and Overhaul (MRO) performance with greater predictive accuracy to the component and part-level. Integrating machine learning databases, apps, and algorithms into cloud platforms are becoming pervasive, as evidenced by announcements from Amazon, Google, and Microsoft. The following graphic illustrates how machine learning is integrated into the Azure platform. Microsoft is enabling Krones to attain their Industrie 4.0 objectives by automating aspects of their manufacturing operations on Microsoft Azure.

Source: Enabling Manufacturing Transformation in a Connected World John Shewchuk Technical Fellow DX, Microsoft

Enabling condition monitoring processes that provide manufacturers with the scale to manage Overall Equipment Effectiveness (OEE) at the plant level increasing OEE performance from 65% to 85%. An automotive OEM partnered with Tata Consultancy Services to improve their production processes that had seen Overall Equipment Effectiveness (OEE) of the press line reach a low of 65 percent, with the breakdown time ranging from 17-20 percent. By integrating sensor data on 15 operating parameters (such as oil pressure, oil temperature, oil viscosity, oil leakage, and air pressure) collected from the equipment every 15 seconds for 12 months. The components of the solution are shown

Source: Using Big Data for Machine Learning Analytics in Manufacturing

Machine learning is revolutionizing relationship intelligence and Salesforce is quickly emerging as the leader. The series of acquisitions Salesforce is making positions them to be the global leader in machine learning and artificial intelligence (AI). The following table from the Cowen and Company research note, Salesforce: Initiating At Outperform; Growth Engine Is Well Greased published June 23, 2016, summarizes Salesforce’s series of machine learning and AI acquisitions, followed by an analysis of new product releases and estimated revenue contributions. Salesforce’s recent acquisition of e-commerce provider Demandware for $2.8B is analyzed by Alex Konrad is his recent post, Salesforce Will Acquire Demandware For $2.8 Billion In Move Into Digital Commerce. Cowen & Company predicts Commerce Cloud will contribute $325M in revenue by FY18, with Demandware sales being a significant contributor.

Revolutionizing product and service quality with machine learning algorithms that determine which factors most and least impact quality company-wide. Manufacturers often are challenged with making product and service quality to the workflow level a core part of their companies. Often quality is isolated. Machine learning is revolutionizing product and service quality by determining which internal processes, workflows, and factors contribute most and least to quality objectives being met. Using machine learning manufacturers will be able to attain much greater manufacturing intelligence by predicting how their quality and sourcing decisions contribute to greater Six Sigma performance within the Define, Measure, Analyze, Improve, and Control (DMAIC) framework.

Increasing production yields by the optimizing of team, machine, supplier and customer requirements are already happening with machine learning. Machine learning is making a difference on the shop floor daily in aerospace & defense, discrete, industrial and high-tech manufacturers today. Manufacturers are turning to more complex, customized products to use more of their production capacity, and machine learning help to optimize the best possible selection of machines, trained staffs, and suppliers.

The vision of Manufacturing-as-a-Service will become a reality thanks to machine learning enabling subscription models for production services. Manufacturers whose production processes are designed to support rapid, highly customized production runs are well positioning to launch new businesses that provide a subscription rate for services and scale globally. Consumer Packaged Goods (CPG), electronics providers and retailers whose manufacturing costs have skyrocketed will have the potential to subscribe to a manufacturing service and invest more in branding, marketing, and selling.

Machine learning is ideally suited for optimizing supply chains and creating greater economies of scale. For many complex manufacturers, over 70% of their products are sourced from suppliers that are making trade-offs of which buyer they will fulfill orders for first. Using machine learning, buyers and suppliers could collaborate more effectively and reduce stock-outs, improve forecast accuracy and met or beat more customer delivery dates.

Knowing the right price to charge a given customer at the right time to get the most margin and closed sale will be commonplace with machine learning. Machine learning is extending what enterprise-level price optimization apps provide today. One of the most significant differences is going to be just how optimizing pricing along with suggested strategies to close deals accelerate sales cycles.

Additional reading:

Cisco Blog: Deus Ex Machina: Machine Learning Acts to Create New Business Outcomes

Enabling Manufacturing Transformation in a Connected World John Shewchuk Technical Fellow DX, Microsoft

Focus Group: Big Data Analytics for Smart Manufacturing Systems

GE Predix: The Industrial Internet Platform

IDC Manufacturing Insights reprint courtesy of Cisco: Designing and Implementing the Factory of the Future at Mahindra Vehicle Manufacturers

Machine Learning: What It Is And Why It Matters

McKinsey & Company, An Executive’s Guide to Machine Learning

MIT Sloan Management Review, Sales Gets a Machine-Learning Makeover

Stanford University CS 229 Machine Learning Course Materials
The Economist Feature On Machine Learning

UC Berkeley CS 194-10, Fall 2011: Introduction to Machine Learning
Lecture slides, notes

University of Washington CSE 446 – Machine Learning – Winter 2014

Sources:

Lee, J. H., & Ha, S. H. (2009). Recognizing yield patterns through hybrid applications of machine learning techniques. Information Sciences, 179(6), 844-850.

Mackenzie, A. (2015). The production of prediction: What does machine learning want?. European Journal of Cultural Studies, 18(4-5), 429-445.

Pham, D. T., & Afify, A. A. (2005, July). Applications of machine learning in manufacturing. In Intelligent Production Machines and Systems, 1st I* PROMS Virtual International Conference (pp. 225-230).

Priore, P., de la Fuente, D., Puente, J., & Parreño, J. (2006). A comparison of machine-learning algorithms for dynamic scheduling of flexible manufacturing systems. Engineering Applications of Artificial Intelligence, 19(3), 247-255.

Oct 23

Businesses Adopting Big Data, Cloud & Mobility Grow 53% Faster Than Peers

Orchestrating big data, cloud and mobility strategies leads to 53% greater growth than peers not adopting these technologies.
73% of midmarket companies say the complexity of their stored data requires big data analytics apps and tools to better gain insights from.
41% of midmarket companies are using big data to better target marketing efforts.
54% of midmarket companies’ security budgets are invested in security plans versus reacting to threats.

These and many other insights are from Dell’s second annual Global Technology Adoption Index (GTAI 2015) released last week in collaboration with TNS Research. The Global Technology Adoption Index surveyed IT and business decision makers of mid-market organizations across 11 countries, interviewing 2,900 IT and business decision makers representing businesses with 100 to 4,999 employees.

The purpose of the index is to understand how business users perceive, plan for and utilize four key technologies: cloud, mobility, security and big data. Dell released the first wave of its results this week and will be publishing several additional chapters throughout 2016. You can download Chapter 1 of the study here (PDF, no opt-in, 18 pp.).

Key take-aways from the study include the following:

Orchestrating big data, cloud and mobility strategies leads to 53% greater growth than peers not adopting these technologies. Midmarket organizations adopting big data alone have the potential to grow 50% more than comparable organizations. Effective use of Bring Your Own Device (BYOD) mobility strategies has the potential to increase growth by 53% over laggards or late adopters..

73% of North American organizations believe the volume and complexity of their data requires big data analytics apps and tools. This is up from 54% in 2014, indicating midmarket organizations are concentrating on how to get more value from the massive data stores many have accumulated. This same group of organizations believe they are getting more value out of big data this year (69%) compared to last year (64%). Top outcomes of using big data include better targeting of marketing efforts (41%), optimization of ad spending (37%), and optimization of social media marketing (37%).

54% of an organization’s security budget is invested in security plans versus reacting to threats. Dell & TNS Research discovered that midmarket organizations both in North America and Western Europe are relying on security to enable new devices or drive competitive advantage. In North America, taking a more strategic approach to security has increased from 25% in 2014 to 35% today. In Western Europe, the percentage of companies taking a more strategic view of security has increased from 26% in 2014 to 30% this year.

IT infrastructure costs to support big data initiatives (29%) and costs related to securing the data (28%) are the two greatest barriers to big data adoption. For cloud adoption, costs and security are the two biggest barriers in midmarket organizations as is shown in the graphic below.

Cloud use by midmarket companies in France increased 12% in the last twelve months, leading all nations in the survey. Of the 11 countries surveyed, France had the greatest increase in cloud adoption within midmarket companies. French businesses increased their adoption of cloud applications and platforms from 70% in 2014 to 82% in 2015.

Sources: Dell Study Reveals Companies Investing in Cloud, Mobility, Security and Big Data Are Growing More Than 50 Percent Faster Than Laggards. October 13, 2015

Oct 18

2015 Big Data Market Update

42.6% of all big data apps developed for manufacturing are being created by enterprises today.
38.2% of all big data and advanced analytics apps in use today are in customer-facing departments including marketing, sales, and customer service.
33.2% of all big data and advanced analytics developers are concentrating on the software & computing industry.
19.2% of big data app developers say quality of data is the biggest problem they consistently face when building new apps.

These and other insights are from the recently published report Big Data and Advanced Analytics Survey 2015, Volume I by Evans Data Corporation. The survey is based on 444 in-depth interviews with developers who are currently working with analytics and databases and are both currently working on and planning big data and advanced analytics projects. The survey’s results provide a strategic view of the attitudes, adoption patterns and intentions of developers in relation to big data and analytics. You can more on the methodology of the report here.

Key take-aways from the report include the following:

Software & computing (18%), financial (11.6%), manufacturing (10.9%) and retail (9.8%) industries have the highest percentage of programmers creating big data and analytics applications today. Additional industries where big data app development is active and growing include entertainment (7.7%), telecommunications (7.5%), utilities & energy (6.6%) and healthcare (4.6%). The following graphic provides an overview of the industries addressed.

Capturing more information than traditional database practices (22.60%), capturing and analyzing unstructured data (21.10%) and the potential for visualizing or analyzing data differently (20.70%) are the three top use cases driving app development today. Evans Data found that capturing more information than traditional database practices allow increased 6% since last year, making it the top use case in 2015. The following graphic provides the distribution of responses by use cases from the developers surveyed.

Total size of the data being processed (40.8%), complex, unstructured nature of the data (38.1%) and the need for real-time data analysis (17.7%) are the top three factors driving big data adoption over traditional database solutions. Evans Data found that the size and complexity of structured and unstructured data is the catalyst that gets enterprises moving on the journey to big data adoption. The ability to gain greater insights into their data with descriptive, predictive and contextually-driven analytics is the fuel that keeps big data adoption moving forward in all companies.

33.2% of all big data and advanced analytics developers are concentrating on the software & computing industry. Of these developers, 36.7% are working in organizations of 101 to 1,000 employees, 32.9% are in enterprises of 1,000+ employees, and 30.1% are in organizations of 100 employees or less. 42.6% of all big data software development in manufacturing begins in enterprises (1K+ employees).

Enterprises competing in the software & computing industry (17.5%), manufacturing (15.8%) and financial industry (14%) are investing the heaviest in big data and analytics app development. Overall, 32% of big data and analytics projects are custom-designed and produced by system integrators and value-added resellers (SI, VAR). 70% of big data and advanced analytics apps for manufacturing are created by enterprise and system integrator/value-added reseller (SI/VAR) development teams. The following graphic provides an overview of industries targeted by big data, segmented by developer segment.

Sales and customer data (9.6%), IT-based data analysis (9.4%), informatics (8.7%) and financial transactions (8.4%) are the most common big data sets app developers are working with today. In addition marketing, system management, production and shop floor data, and web & social media-generated data are also included. Evans Data found that informatics data sets grew the fastest in the last six months, and scientific computing is now competing with transaction processing systems as a dominant data set developers rely on to create new apps.

Marketing departments have quickly become the most common users of big data and advanced analytics apps (14.4%) followed by IT (13.3%) and Research & Development (13%). Evans Data asked developers which departments in their organizations are putting big data and advanced analytics apps to use, regardless of where they were created. 38.2% of all big data use in organizations today are in customer-facing departments including marketing, sales, and customer service.

Availability of relevant tools (10.9%), storage costs (10.2%) and siloed business, IT, and analytics/data science teams (10.0%) are the top three barriers developers face in building new apps. It’s interesting to note that compliance and having to transition from legacy systems did not score higher in the survey, as these two areas are inordinately more complex in more regulated, older industries. For big data and advanced analytics to accelerate across manufacturing and financial industries, compliance and legacy systems integration barriers will need to first be addressed.

Quality of data (19.2%), relevance of data being acquired (13.5%), volume of data being processed (12.6%) and ability to adequately visualize big data (11.7%) are the four biggest problem areas faced by big data developers today. Additional problem areas include the volume of data in storage (10.5%), ability to gain insight from big data (10.1%) and the high rate of data acquisition (7.6%). The remainder of problem areas are shown in the graphic below.

Providing real-time correlation and anomaly detection of diverse security data (29.9%) and high-speed querying of security intelligence data (28.1%) are the two most critical areas vendors can assist developers with today. Big data and analytics app developers are looking to vendors to also provide more effective security algorithms for various use case scenarios (17.6%), flexible big data analytics across structured and unstructured data (14.2%) and more useful graphical front-end tools for visualizing and exploring big data (5.1%).

Sep 5

3 Comments

10 Ways Big Data Is Revolutionizing Supply Chain Management

Bottom line: Big data is providing supplier networks with greater data accuracy, clarity, and insights, leading to more contextual intelligence shared across supply chains.

Forward-thinking manufacturers are orchestrating 80% or more of their supplier network activity outside their four walls, using big data and cloud-based technologies to get beyond the constraints of legacy Enterprise Resource Planning (ERP) and Supply Chain Management (SCM) systems. For manufacturers whose business models are based on rapid product lifecycles and speed, legacy ERP systems are a bottleneck. Designed for delivering order, shipment and transactional data, these systems aren’t capable of scaling to meet the challenges supply chains face today.

Choosing to compete on accuracy, speed and quality forces supplier networks to get to a level of contextual intelligence not possible with legacy ERP and SCM systems. While many companies today haven’t yet adopted big data into their supply chain operations, these ten factors taken together will be the catalyst that get many moving on their journey.

The ten ways big data is revolutionizing supply chain management include:

The scale, scope and depth of data supply chains are generating today is accelerating, providing ample data sets to drive contextual intelligence. The following graphic provides an overview of 52 different sources of big data that are generated in supply chains Plotting the data sources by variety, volume and velocity by the relative level of structured/unstructured data, it’s clear that the majority of supply chain data is generated outside an enterprise. Forward-thinking manufacturers are looking at big data as a catalyst for greater collaboration. Source: Big Data Analytics in Supply Chain Management: Trends and Related Research. Presented at 6th International Conference on Operations and Supply Chain Management, Bali, 2014

Enabling more complex supplier networks that focus on knowledge sharing and collaboration as the value-add over just completing transactions. Big data is revolutionizing how supplier networks form, grow, proliferate into new markets and mature over time. Transactions aren’t the only goal, creating knowledge-sharing networks is, based on the insights gained from big data analytics. The following graphic from Business Ecosystems Come Of Age (Deloitte University Press) (free, no opt-in) illustrates the progression of supply chains from networks or webs, where knowledge sharing becomes a priority.

Big data and advanced analytics are being integrated into optimization tools, demand forecasting, integrated business planning and supplier collaboration & risk analytics at a quickening pace. These are the top four supply chain capabilities that Delotte found are currently in use form their recent study, Supply Chain Talent of the Future Findings from the 3^rd Annual Supply Chain Survey (free, no opt-in). Control tower analytics and visualization are also on the roadmaps of supply chain teams currently running big data pilots.

64% of supply chain executives consider big data analytics a disruptive and important technology, setting the foundation for long-term change management in their organizations. SCM World’s latest Chief Supply Chain Officer Report provides a prioritization of the most disruptive technologies for supply chains as defined by the organizations’ members. The following graphic from the report provides insights into how senior supply chain executives are prioritizing big data analytics over other technologies.

Using geoanalytics based on big data to merge and optimize delivery networks. The Boston Consulting Group provides insights into how big data is being put to use in supply chain management in the article Making Big Data Work: Supply Chain Management (free, opt-in). One of the examples provided is how the merger of two delivery networks was orchestrated and optimized using geoanalytics. The following graphic is from the article. Combining geoanalytics and big data sets could drastically reduce cable TV tech wait times and driving up service accuracy, fixing one of the most well-known service challenges of companies in that business.

Big data is having an impact on organizations’ reaction time to supply chain issues (41%), increased supply chain efficiency of 10% or greater (36%), and greater integration across the supply chain (36%). The Big Data Analytics in Supply Chain: Hype or Here to Stay? Accenture Global Operations Megatrends Study found that companies are achieving significant results using big data analytics to improve supply chain performance and gain greater contextual intelligence.

Embedding big data analytics in operations leads to a 4.25x improvement in order-to-cycle delivery times, and a 2.6x improvement in supply chain efficiency of 10% or greater. Accenture found that embedding big data into supply chain operations accelerates supply chain processes a minimum of 1.3x over using big data on an ad hoc basis. Source: Big Data Analytics in Supply Chain: Hype or Here to Stay? Accenture Global Operations Megatrends Study

Greater contextual intelligence of how supply chain tactics, strategies and operations are influencing financial objectives. Supply chain visibility often refers to being able to see multiple supplier layers deep into a supply network. It’s been my experience that being able to track financial outcomes of supply chain decisions back to financial objectives is attainable, and with big data app integration to financial systems, very effective in industries with rapid inventory turns. Source: Turn Big Data Into Big Visibility.

Traceability and recalls are by nature data-intensive, making big data’s contribution potentially significant. Big data has the potential to provide improved traceability performance and reduce the thousands of hours lost just trying to access, integrate and manage product databases that provide data on where products are in the field needing to be recalled or retrofitted.
Increasing supplier quality from supplier audit to inbound inspection and final assembly with big data. IBM has developed a quality early-warning system that detects and then defines a prioritization framework that isolates quality problem faster than more traditional methods, including Statistical Process Control (SPC). The early-warning system is deployed upstream of suppliers and extends out to products in the field.

Jun 5

1 Comment

2015 Roundup Of Analytics, Big Data & Business Intelligence Forecasts And Market Estimates

Salesforce (NYSE:CRM) estimates adding analytics and Business Intelligence (BI) applications will increase their Total Addressable Market (TAM) by $13B in FY2014.

89% of business leaders believe Big Data will revolutionize business operations in the same way the Internet did.

83% have pursued Big Data projects in order to seize a competitive edge.

Despite the varying methodologies used in the studies mentioned in this roundup, many share a common set of conclusions. The high priority in gaining greater insights into customers and their unmet needs, more precise information on how to best manage and simplify sales cycles, and how to streamline service are common themes.

The most successful Big Data uses cases revolve around enterprises’ need to get beyond the constraints that hold them back from being more attentive and responsive to customers.

Presented below is a roundup of recent forecasts and estimates:

Wikibon projects the Big Data market will top $84B in 2026, attaining a 17% Compound Annual Growth Rate (CAGR) for the forecast period 2011 to 2026. The Big Data market reached $27.36B in 2014, up from $19.6B in 2013. These and other insights are from Wikibon’s excellent research of Big Data market adoption and growth. The graphic below provides an overview of their Big Data Market Forecast. Source: Executive Summary: Big Data Vendor Revenue and Market Forecast, 2011-2026.

IBM and SAS are the leaders of the Big Data predictive analytics market according to the latest Forrester Wave™: Big Data Predictive Analytics Solutions, Q2 2015. The latest Forrester Wave is based on an analysis of 13 different big data predictive analytics providers including Alpine Data Labs, Alteryx, Angoss Software, Dell, FICO, IBM, KNIME.com, Microsoft, Oracle, Predixion Software, RapidMiner, SAP, and SAS. Forrester specifically called out Microsoft Azure Learning is an impressive new entrant that shows the potential for Microsoft to be a significant player in this market. Gregory Piatetsky (@KDNuggets) has done an excellent analysis of the Forrester Wave Big Data Predictive Analytics Solutions Q2 2015 report here. Source: Courtesy of Predixion Software: The Forrester Wave™: Big Data Predictive Analytics Solutions, Q2 2015 (free, no opt-in).

IBM, KNIME, RapidMiner and SAS are leading the advanced analytics platform market according to Gartner’s latest Magic Quadrant. Gartner’s latest Magic Quadrant for advanced analytics evaluated 16 leading providers of advanced analytics platforms that are used to building solutions from scratch. The following vendors were included in Gartner’s analysis: Alpine Data Labs, Alteryx, Angoss, Dell, FICO, IBM, KNIME, Microsoft, Predixion, Prognoz, RapidMiner, Revolution Analytics, Salford Systems, SAP, SAS and Tibco Software, Gregory Piatetsky (@KDNuggets) provides excellent insights into shifts in Magic Quadrant for Advanced Platform rankings here. Source: Courtesy of RapidMiner: Magic Quadrant for Advanced Analytics Platforms Published: 19 February 2015 Analyst(s): Gareth Herschel, Alexander Linden, Lisa Kart (reprint; free, no opt-in).

Salesforce estimates adding analytics and Business Intelligence (BI) applications will increase their Total Addressable Market (TAM) by $13B in FY2014. Adding new apps in analytics is projected to increase their TAM to $82B for calendar year (CY) 2018, fueling an 11% CAGR in their total addressable market from CY 2013 to 2018. Source: Building on Fifteen Years of Customer Success Salesforce Analyst Day 2014 Presentation (free, no opt in).

89% of business leaders believe big data will revolutionize business operations in the same way the Internet did. 85% believe that big data will dramatically change the way they do business. 79% agree that ‘companies that do not embrace Big Data will lose their competitive position and may even face extinction.’ 83% have pursued big data projects in order to seize a competitive edge. The top three areas where big data will make an impact in their operations include: impacting customer relationships (37%); redefining product development (26%); and changing the way operations is organized (15%).The following graphic compares the top six areas where big data is projected to have the greatest impact in organizations over the next five years. Source: Accenture, Big Success with Big Data: Executive Summary (free, no opt in).

The global Big Data market is projected reach $122B in revenue by 2025. Frost & Sullivan also forecasts the global data traffic will cross 100 Zettabytes annually by 2025. Source: World’s Top Global Mega Trends To 2025 and Implications to Business, Society and Cultures.

The global text analytics market has a potential to reach $6.5B by 2020, attaining a CAGR of 25.2% from 2014 to 2020. Customer Relationship Management (CRM), predictive analytics and brand reputation are the top three projected applications. The following graphic provides an overview of key findings from the report. Source: Global Text Analytics Market: Text Analytics, Visualizing and Analyzing Open-Ended Text Data.

Customer analytics (48%), operational analytics (21%), and fraud & compliance (21%) are the top three use cases for Big Data. Datameer’s analysis of the market also found that the global Hadoop market will grow from $1.5B in 2012 to $50.2B in 2020, and financial services, technology and telecommunications are the leading industries using big data solutions today. Source: Big Data: A Competitive Weapon for the Enterprise.

37% of Asia Pacific manufacturers are using Big Data and analytics technologies to improve production quality management. IDC found manufacturers in this region are relying on these technologies to reduce costs, increase productivity, and attract new customers. Source: Big Data and Analytics Core to Nex-Gen Manufacturing.

Supply chain visibility (56%), geo-location and mapping data (47%) and product traceability data (42%) are the top three potential areas of Big Data opportunity for supply chain management. Transport management, supply chain planning, & network modeling and optimization are the three most popular applications of Big Data in supply chain initiatives. Source: Supply Chain Report, February 2015.

Finding correlations across multiple disparate data sources (48%), predicting customer behavior (46%) and predicting product or services sales (40%) are the three factors driving interest in Big Data analytics. These and other fascinating findings from InformationWeek’s 2015 Analytics & BI Survey provide a glimpse into how enterprises are selecting analytics applications and platforms. Source: Information Week 2015 Analytics & BI Survey.

Banking, communications, media, utilities and wholesale trade increased their use of Big Data analytics the most in the last twelve months. Gartner’s latest survey of big data adoption completed in the 4^th quarter of 2014 found that columnar/in-memory databases, high-capacity data warehouses, search-based indexes and log data analytics all experienced greater than 5% growth in references reporting active use. Source: Survey Analysis: BI and Analytics Spending Intentions, 2015, 13 May 2015, Report G00274741 Analyst(s): Josh Parenteau & Rita L. Sallam (client access).

IDC predicts the global Big Data and Analytics market will reach $125B in hardware, software and services revenue this year. The professional services-to-technology ratio will also increase 25% this year, due to increasingly complex system configurations and enterprise-wide solutions. IDC also predicts the Internet of Things (IoT) market will also attain a 30% CAGR from 2014 to 2019. Source: IDC Predictions 2015: Accelerating Innovation — and Growth — on the 3rd Platform by Frank Gens & IDC Predictions 2015 Team (free, no opt-in courtesy of SAP).

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.