Posts from the ‘Big Data’ Category

Jun 16

What Matters Most In Business Intelligence, 2019

Improving revenues using BI is now the most popular objective enterprises are pursuing in 2019.
Reporting, dashboards, data integration, advanced visualization, and end-user self-service are the most strategic BI initiatives underway in enterprises today.
Operations, Executive Management, Finance, and Sales are primarily driving Business Intelligence (BI) adoption throughout enterprises today.
Tech companies’ Operations & Sales teams are the most effective at driving BI adoption across industries surveyed, with Advertising driving BI adoption across Marketing.

These and many other fascinating insights are from Dresner Advisory Associates’ 10^th edition of its popular Wisdom of Crowds® Business Intelligence Market Study. The study is noteworthy in that it provides insights into how enterprises are expanding their adoption of Business Intelligence (BI) from centralized strategies to tactical ones that seek to improve daily operations. The Dresner research teams’ broad assessment of the BI market makes this report unique, including their use visualizations that provide a strategic view of market trends. The study is based on interviews with respondents from the firms’ research community of over 5,000 organizations as well as vendors’ customers and qualified crowdsourced respondents recruited over social media. Please see pages 13 – 16 for the methodology.

Key insights from the study include the following:

Operations, Executive Management, Finance, and Sales are primarily driving Business Intelligence (BI) adoption throughout their enterprises today. More than half of the enterprises surveyed see these four departments as the primary initiators or drivers of BI initiatives. Over the last seven years, Operations departments have most increased their influence over BI adoption, more than any other department included in the current and previous survey. Marketing and Strategic Planning are also the most likely to be sponsoring BI pilots and looking for new ways to introduce BI applications and platforms into use daily.

Tech companies’ Operations & Sales teams are the most effective at driving BI adoption across industries surveyed, with Advertising driving BI adoption across Marketing. Retail/Wholesale and Tech companies’ sales leadership is primarily driving BI adoption in their respective industries. It’s not surprising to see the leading influencer among Healthcare respondents is resource-intensive HR. The study found that Executive Management is most likely to drive business intelligence in consulting practices most often.

Reporting, dashboards, data integration, advanced visualization, and end-user self-service are the most strategic BI initiatives underway in enterprises today. Second-tier initiatives include data discovery, data warehousing, data discovery, data mining/advanced algorithms, and data storytelling. Comparing the last four years of survey data, Dresner’s research team found reporting retains all-time high scores as the top priority, and data storytelling, governance, and data catalog hold momentum. Please click on the graphic to expand for easier reading.

BI software providers most commonly rely on executive-level personas to design their applications and add new features. Dresner’s research team found all vertical industries except Business Services target business executives first in their product design and messaging. Given the customer-centric nature of advertising and consulting services business models, it is understandable why the primary focus BI vendors rely on in selling to them are customer personas. The following graphic compares targeted users for BI by industry.

Improving revenues using BI is now the most popular objective in 2019, despite BI initially being positioned as a solution for compliance and risk management. Executive Management, Marketing/Sales, and Operations are driving the focus on improving revenues this year. Nearly 50% of enterprises now expect BI to deliver better decision making, making the areas of reporting, and dashboards must-have features. Interestingly, enterprises aren’t looking to BI as much for improving operational efficiencies and cost reductions or competitive advantages. Over the last 12 to 18 months, more tech manufacturing companies have initiated new business models that require their operations teams to support a shift from products to services revenues. An example of this shift is the introduction of smart, connected products that provide real-time data that serves as the foundation for future services strategies. Please click on the graphic to expand for easier reading.

In aggregate, BI is achieving its highest levels of adoption in R&D, Executive Management, and Operations departments today. The growing complexity of products and business models in tech companies, increasing reliance on analytics and BI in retail/wholesale to streamline supply chains and improve buying experiences are contributing factors to the increasing levels of BI adoption in these three departments. The following graphic compares BI’s level of adoption by function today.

Enterprises with the largest BI budgets this year are investing more heavily into dashboards, reporting, and data integration. Conversely, those with smaller budgets are placing a higher priority on open source-based big data projects, end-user data preparation, collaborative support for group-based decision-making, and enterprise planning. The following graphic provides insights into technologies and initiatives strategic to BI at an enterprise level by budget plans.

Marketing/Sales and Operations are using the greatest variety of BI tools today. The survey shows how conversant Operations professionals are with the BI tools in use throughout their departments. Every one of them knows how many and most likely which types of BI tools are deployed in their departments. Across all industries, Research & Development (R&D), Business Intelligence Competency Center (BICC), and IT respondents are most likely to report they have multiple tools in use.

Jun 2

1 Comment

10 Charts That Will Change Your Perspective Of Big Data’s Growth

Worldwide Big Data market revenues for software and services are projected to increase from $42B in 2018 to $103B in 2027, attaining a Compound Annual Growth Rate (CAGR) of 10.48% according to Wikibon.
Forrester predicts the global Big Data software market will be worth $31B this year, growing 14% from the previous year. The entire global software market is forecast to be worth $628B in revenue, with $302B from applications.
According to an Accenture study, 79% of enterprise executives agree that companies that do not embrace Big Data will lose their competitive position and could face extinction. Even more, 83%, have pursued Big Data projects to seize a competitive edge.
59% of executives say Big Data at their company would be improved through the use of AI according to PwC.

Sales and Marketing, Research & Development (R&D), Supply Chain Management (SCM) including distribution, Workplace Management and Operations are where advanced analytics including Big Data are making the greatest contributions to revenue growth today. McKinsey Analytics’ study Analytics Comes of Age, published in January 2018 (PDF, 100 pp., no opt-in) is a comprehensive overview of how analytics technologies and Big Data are enabling entirely new ecosystems, serving as a foundational technology for Artificial Intelligence (AI). McKinsey finds that analytics and Big Data are making the most valuable contributions in the Basic Materials and High Tech industries. The first chart in the following series of ten is from the McKinsey Analytics study, highlighting how analytics and Big Data are revolutionizing many of the foundational business processes of Sales and Marketing.

The following ten charts provide insights into Big Data’s growth:

Nearly 50% of respondents to a recent McKinsey Analytics survey say analytics and Big Data have fundamentally changed business practices in their sales and marketing functions. Also, more than 30% say the same about R&D across industries, with respondents in High Tech and Basic Materials & Energy report the greatest number of functions being transformed by analytics and Big Data. Source: Analytics Comes of Age, published in January 2018 (PDF, 100 pp., no opt-in).

Worldwide Big Data market revenues for software and services are projected to increase from $42B in 2018 to $103B in 2027, attaining a Compound Annual Growth Rate (CAGR) of 10.48%. As part of this forecast, Wikibon estimates the worldwide Big Data market is growing at an 11.4% CAGR between 2017 and 2027, growing from $35B to $103B. Source: Wikibon and reported by Statista.

According to NewVantage Venture Partners, Big Data is delivering the most value to enterprises by decreasing expenses (49.2%) and creating new avenues for innovation and disruption (44.3%). Discovering new opportunities to reduce costs by combining advanced analytics and Big Data delivers the most measurable results, further leading to this category being the most prevalent in the study. 69.4% have started using Big Data to create a data-driven culture, with 27.9% reporting results. Source: NewVantage Venture Partners, Big Data Executive Survey 2017 (PDF, 16 pp.)

The Hadoop and Big Data Market are projected to grow from $17.1B in 2017 to $99.31B in 2022 attaining a 28.5% CAGR. The greatest period of projected growth is in 2021 and 2022 when the market is projected to jump $30B in value in one year. Source: StrategyMRC and reported by Statista.

Big Data applications and analytics is projected to grow from $5.3B in 2018 to $19.4B in 2026, attaining a CAGR of 15.49%. Big Data market worldwide includes Professional Services is projected to grow from $16.5B in 2018 to $21.3B in 2026. Source: Wikibon and reported by Statista.

Comparing the worldwide demand for advanced analytics and Big Data-related hardware, services and software, the latter category’s dominance becomes clear. The software segment is projected to increase the fastest of all categories, increasing from $14B in 2018 to $46B in 2027 attaining a CAGR of 12.6%. Sources: Wikibon; SiliconANGLE; Statista estimates and reported by Statista.

Advanced analytics and Big Data revenue in China are projected to be worth ¥57.8B ($9B) by 2020. The Chinese market is predicted to be one of the fastest growing globally, growing at a CAGR of 31.72% in the forecast period. Sources: Social Sciences Academic Press (China) and Statista.

Non-relational analytic data stores are projected to be the fastest growing technology category in Big Data, growing at a CAGR of 38.6% between 2015 and 2020. Cognitive software platforms (23.3% CAGR) and Content Analytics (17.3%) round out the top three fastest growing technologies between 2015 and 2020. Source: Statista.

A decentralized general-merchandise retailer that used Big Data to create performance group clusters saw sales grow 3% to 4%. Big Data is the catalyst of a retailing industry makeover, bringing greater precision to localization than has been possible before. Big Data is being used today to increase the ROI of endcap promotions, optimize planograms, help to improve upsell and cross-sell sales performance and optimize prices on items that drive the greatest amount of foot traffic. Source: Use Big Data to Give Local Shoppers What They Want, Boston Consulting Group, February 8, 2018.

84% of enterprises have launched advanced analytics and Big Data initiatives to bring greater accuracy and accelerate their decision-making Big Data initiatives focused on this area also have the greatest success rate (69%) according to the most recent NewVantage Venture Partners Survey. Over a third of enterprises, 36%, say this area is their top priority for advanced analytics and Big Data investment. Sources: NewVantage Venture Partners Survey and Statista.

Additional Big Data Information Sources:

4 Pain Points of Big Data and how to solve them, Digital McKinsey via Medium, November 10, 2017

53% Of Companies Are Adopting Big Data Analytics, Forbes, December 24, 2017

6 Predictions For The $203 Billion Big Data Analytics Market, Forbes, Gil Press, January 20, 2017

Analytics Comes of Age, McKinsey Analytics, January 2018 (PDF, 100 pp.)

Big Data & Analytics Is The Most Wanted Expertise By 75% Of IoT Providers, Forbes, August 21, 2017

Big Data 2017 – Market Statistics, Use Cases, and Trends, Calsoft (36 pp., PDF)

Big Data and Business Analytics Revenues Forecast to Reach $150.8 Billion This Year, Led by Banking and Manufacturing Investments, According to IDC, March 14, 2017

Big Data Executive Survey 2018, Data and Innovation – How Big Data and AI are Driving Business Innovation, NewVantage Venture Partners, January 2018 (PDF, 18 pp.)

Big Data Tech Hadoop and Spark Get Slow Start in Enterprise, Information Week, March 20, 2018

Big Success With Big Data, Accenture (PDF, 12 pp.)

Gartner Survey Shows Organizations Are Slow to Advance in Data and Analytics, Gartner, February 5, 2018

How Big Data and AI Are Driving Business Innovation in 2018, MIT Sloan Management Review, February 5, 2018

IDC forecasts big growth for Big Data, Analytics Magazine. April 2018

IDC Worldwide Big Data Technology and Services 2012 – 2015 Forecast, Courtesy of EC Europa (PDF, 34 pp.)

Midyear Global Tech Market Outlook For 2017 To 2018, Forrester, September 25, 2017 (client access reqd.)

Oracle Industry Analyst Reports – Data-rich website of industry analyst reports

Ten Ways Big Data Is Revolutionizing Marketing And Sales, Forbes, May 9, 2016

The Big Data Payoff: Turning Big Data into Business Value, CAP Gemini & Informatica Study, (PDF, 12 pp.)

The Forrester Wave™: Enterprise BI Platforms With Majority Cloud Deployments, Q3 2017 courtesy of Oracle

May 26

The Best Big Data Companies And CEOs To Work For In 2018

Employees would most recommend Dataiku, StreamSets, 1010 Data, Reltio, Looker, SAP, Google, Salesforce, Tamr, DataRobot, Domo, Microsoft, Qubole, Attivio and Informatica to friends interested in working for business analytics, data science and machine learning, big data systems and platforms, or data management and integration company in 2018.
These 15 companies are the highest rated in their fields of expertise as defined by Computer Reseller News as of May 11, 2018.
Nine out of 10 employees of these top 15 companies would recommend the company they work for to a friend.

Forbes readers’ most common requests center on who the best companies are to work for in analytics, big data, data management, data science and machine learning. The latest Computer Reseller News‘ 2018 Big Data 100 list of companies is used to complete the analysis as it is an impartial, independent list aggregated based on CRN’s analysis and perspectives of the market. Using the CRN list as a foundation, the following analysis captures the best companies in their respective areas today.

Using the 2018 Big Data 100 CRN list as a baseline to compare the Glassdoor scores of the (%) of employees who would recommend this company to a friend and (%) of employees who approve of the CEO, the following analysis was completed today. 25 companies on the list have very few (less than 15) or no Glassdoor reviews, so they are excluded from the rankings. Based on analysis of Glassdoor score patterns over the last four years, the lower the number of rankings, the more 100% scores for referrals and CEOs. These companies, however, are included in the full data set available here. If the image below is not visible in your browser, you can view the rankings here.

The highest rated CEOs on Glassdoor as of May 11, 2018 include the following:

Dataiku	Florian Douetteau	100%
StreamSets	Girish Pancha	100%
MemSQL	Nikita Shamgunov	100%
1010 Data	Greg Munves	99%
Salesforce.com	Marc Benioff	98%
Attivio	Stephen Baker	98%
SAP	Bill McDermott	97%
Qubole	Ashish Thusoo	97%
Trifacta	Adam Wilson	97%
Zaloni	Ben Sharma	97%
Reltio	Manish Sood	96%
Microsoft	Satya Nadella	96%
Cloudera	Thomas J. Reilly	96%
Sumo Logic	Ramin Sayar	96%
Google	Sundar Pichai	95%
Looker	Frank Bien	93%
MongoDB	Dev Ittycheria	92%
Snowflake Computing	Bob Muglia	92%
Talend	Mike Tuchen	92%
Databricks	Ali Ghodsi	90%
Informatica	Anil Chakravarthy	90%

Feb 5

Data Scientist Is The Best Job In America According Glassdoor

Data Scientist has been named the best job in America for three years running, with a median base salary of $110,000 and 4,524 job openings.
DevOps Engineer is the second-best job in 2018, paying a median base salary of $105,000 and 3,369 job openings.
There are 29,187 Software Engineering jobs available today, making this job the most popular regarding Glassdoor postings according to the study.

These and many other fascinating insights are from Glassdoor’s 50 Best Jobs In America For 2018. The Glassdoor Report is viewable online here. Glassdoor’s annual report highlights the 50 best jobs based on each job’s overall Glassdoor Job Score.The Glassdoor Job Score is determined by weighing three key factors equally: earning potential based on median annual base salary, job satisfaction rating, and the number of job openings. Glassdoor’s 2018 report lists jobs that excel across all three dimensions of their Job Score metric. For an excellent overview of the study by Karsten Strauss of Forbes, please see his post, The Best Jobs To Apply For In 2018.

LinkedIn’s 2017 U.S. Emerging Jobs Report found that there are 9.8 times more Machine Learning Engineers working today than five years ago with 1,829 open positions listed on their site as of last month. Data science and machine learning are generating more jobs than candidates right now, making these two areas the fastest growing tech employment areas today.

Key takeaways from the study include the following:

Six analytics and data science jobs are included in Glassdoor’s 50 best jobs In America for 2018. These include Data Scientist, Analytics Manager, Database Administrator, Data Engineer, Data Analyst and Business Intelligence Developer. The complete list of the top 50 jobs is provided below with the analytics and data science jobs highlighted along with software engineering, which has a record 29,817 open jobs today:

Median base salary of the 50 best jobs in America is $91,000 with the average salary of the six analytics and data science jobs being $94,167.
Across all six analytics and data science jobs there are 16,702 openings as of today according to Glassdoor.
Tech jobs make up 20 of Glassdoor’s 50 Best Jobs in America for 2018, up from 14 jobs in 2017.

Source: Glassdoor Reveals the 50 Best Jobs in America for 2018

Jul 31

McKinsey’s State Of Machine Learning And AI, 2017

Tech giants including Baidu and Google spent between $20B to $30B on AI in 2016, with 90% of this spent on R&D and deployment, and 10% on AI acquisitions.
Artificial Intelligence (AI) investment has turned into a race for patents and intellectual property (IP) among the world’s leading tech companies.
U.S.-based companies absorbed 66% of all AI investments in 2016. China was second with 17% and growing fast.
By providing better search results, Netflix estimates that it is avoiding canceled subscriptions that would reduce its revenue by $1B annually.

These and other findings are from the McKinsey Global Institute Study, and discussion paper, Artificial Intelligence, The Next Digital Frontier (80 pp., PDF, free, no opt-in) published last month. McKinsey Global Institute published an article summarizing the findings titled How Artificial Intelligence Can Deliver Real Value To Companies. McKinsey interviewed more than 3,000 senior executives on the use of AI technologies, their companies’ prospects for further deployment, and AI’s impact on markets, governments, and individuals. McKinsey Analytics was also utilized in the development of this study and discussion paper.

Key takeaways from the study include the following:

Tech giants including Baidu and Google spent between $20B to $30B on AI in 2016, with 90% of this spent on R&D and deployment, and 10% on AI acquisitions. The current rate of AI investment is 3X the external investment growth since 2013. McKinsey found that 20% of AI-aware firms are early adopters, concentrated in the high-tech/telecom, automotive/assembly and financial services industries. The graphic below illustrates the trends the study team found during their analysis.

AI is turning into a race for patents and intellectual property (IP) among the world’s leading tech companies. McKinsey found that only a small percentage (up to 9%) of Venture Capital (VC), Private Equity (PE), and other external funding. Of all categories that have publically available data, M&A grew the fastest between 2013 And 2016 (85%).The report cites many examples of internal development including Amazon’s investments in robotics and speech recognition, and Salesforce on virtual agents and machine learning. BMW, Tesla, and Toyota lead auto manufacturers in their investments in robotics and machine learning for use in driverless cars. Toyota is planning to invest $1B in establishing a new research institute devoted to AI for robotics and driverless vehicles.

McKinsey estimates that total annual external investment in AI was between $8B to $12B in 2016, with machine learning attracting nearly 60% of that investment. Robotics and speech recognition are two of the most popular investment areas. Investors are most favoring machine learning startups due to quickness code-based start-ups have at scaling up to include new features fast. Software-based machine learning startups are preferred over their more cost-intensive machine-based robotics counterparts that often don’t have their software counterparts do. As a result of these factors and more, Corporate M&A is soaring in this area with the Compound Annual Growth Rate (CAGR) reaching approximately 80% from 20-13 to 2016. The following graphic illustrates the distribution of external investments by category from the study.

High tech, telecom, and financial services are the leading early adopters of machine learning and AI. These industries are known for their willingness to invest in new technologies to gain competitive and internal process efficiencies. Many start-ups have also had their start by concentrating on the digital challenges of this industries as well. The\ MGI Digitization Index is a GDP-weighted average of Europe and the United States. See Appendix B of the study for a full list of metrics and explanation of methodology. McKinsey also created an overall AI index shown in the first column below that compares key performance indicators (KPIs) across assets, usage, and labor where AI could contribute. The following is a heat map showing the relative level of AI adoption by industry and key area of asset, usage, and labor category.

McKinsey predicts High Tech, Communications, and Financial Services will be the leading industries to adopt AI in the next three years. The competition for patents and intellectual property (IP) in these three industries is accelerating. Devices, products and services available now and on the roadmaps of leading tech companies will over time reveal the level of innovative activity going on in their R&D labs today. In financial services, for example, there are clear benefits from improved accuracy and speed in AI-optimized fraud-detection systems, forecast to be a $3B market in 2020. The following graphic provides an overview of sectors or industries leading in AI addition today and who intend to grow their investments the most in the next three years.

Healthcare, financial services, and professional services are seeing the greatest increase in their profit margins as a result of AI adoption. McKinsey found that companies who benefit from senior management support for AI initiatives have invested in infrastructure to support its scale and have clear business goals achieve 3 to 15% percentage point higher profit margin. Of the over 3,000 business leaders who were interviewed as part of the survey, the majority expect margins to increase by up to 5% points in the next year.

Amazon has achieved impressive results from its $775 million acquisition of Kiva, a robotics company that automates picking and packing according to the McKinsey study. “Click to ship” cycle time, which ranged from 60 to 75 minutes with humans, fell to 15 minutes with Kiva, while inventory capacity increased by 50%. Operating costs fell an estimated 20%, giving a return of close to 40% on the original investment

Netflix has also achieved impressive results from the algorithm it uses to personalize recommendations to its 100 million subscribers worldwide. Netflix found that customers, on average, give up 90 seconds after searching for a movie. By improving search results, Netflix projects that they have avoided canceled subscriptions that would reduce its revenue by $1B annually.

Nov 11

6M Developers Are Creating Big Data And Advanced Analytics Apps Today

2M developers are working on IoT applications, increasing 34% since the last year.
Over 50% of the developers working on IoT applications are writing software that utilizes sensors in some capacity.
4M enterprise developers play decision-making roles when it comes to selecting organizational IT development resources. Another 5.2 million hold decision-making authority for selecting IT deployment resources.
4M developers (26% of all developers globally) are using the cloud as a development environment today
The APAC region leads the world with approximately 7.4M developers today, followed by EMEA with 7.2M, North America with 4.4M and Latin American with 1.9M.

These and many other fascinating insights are from the Evans Data Corporation Global Developer Population and Demographic Study 2016 (PDF, client access) published earlier this week. The methodology Evans Data has created to produce this report is the most comprehensive developed for aggregating, analyzing and predicting developer populations globally. The study combines Evans Data’s proprietary global developer population modeling with the current results of their semi-annual global developer survey.

Key takeaways from the study include the following:

6M developers (29% of all developers globally) are involved in a Big Data and Advanced Analytics project today. An additional 25% of developers, or 5.3M, are going to begin Big Data and Advanced Analytics projects within the next six 13% or 2.6M of all developers globally are going to start Big Data and Advanced Analytics projects within the next 7 to 12 months. The following graphic provides an overview of the involvement of 21M developers in Big Data and Advanced Analytics projects today. Please click on the image to expand for easier viewing.

4M developers (26% of all developers globally) are using the cloud as a development environment today. Developers creating new apps in the cloud had increased 375% since Evans began measuring developer participation in mobile development in 2009 when just slightly more than 1.2M developers were using the cloud as their development platform. 4.5M developers (21% of all global developers) plan on beginning app development on cloud platforms in the next six months, and 3.9M (18% of all global developers) plan on starting development on the cloud in 7 – 12 months. Please click on the image to expand for easier viewing.

8M developers in APAC (24% of all developers in the region) are currently developing on cloud platforms. 29% of APAC developers are planning to start cloud-based development in six months, and 20% in 7 – 12 months. The following graphic compares the number of developers currently using the cloud as a development environment today and the number who plan to in the future. Please click on the image to expand for easier viewing.

34% of all Commercial Independent Software Vendors (ISVs) globally today (1.8M developers) are using the cloud as a development environment. An additional 1.4M are planning to begin cloud development in the next six months. 28% of developers globally creating apps in the cloud are from custom system integrators (SI) and value-added resellers (VARs). 23% or approximately 1.2M are from enterprises. The following graphic compares the percent of developers by developer segment who are currently creating new apps in cloud environments. Please click on the image to expand for easier viewing.

30% of developers (6.2M developers globally) are currently developing software for connected devices or the Internet of Things today, with an additional 26% planning to begin projects in 6 months. Evans Data found that this increased 34% over the last year. Also, 2.1M developers plan to begin development in this area within the next 7 to 12 months. The following graphic compares the number of developers globally by stage of development for creating software for connected devices or the Internet of Things. Please click on the image to expand for easier viewing.

41% of global developers creating connected device and IoT software today are from 27% are from North America, 24% are from EMEA and 7% from Latin America. There are 6,072,048 developers currently working on connected device and IoT software today globally. The following graphic provides an overview of the distribution of developers creating connected device and IoT software by region today. Please click on the image to expand for easier viewing.

34% of developers actively creating software for connected devices or the Internet of Things work for custom System Integrators (SI) and VARs today. ISVs are the next largest segment of developers working on IoT projects (30%) followed by enterprises (21%). The following graphic provides an overview of the global base of developers creating software for connected devices and IoT. Evans Data found there are 6.1M developers currently creating apps and solutions in this area alone. Please click on the image to expand for easier viewing.

Sep 11

Roundup Of Analytics, Big Data & BI Forecasts And Market Estimates, 2016

Big Data & business analytics software worldwide revenues will grow from nearly $122B in 2015 to more than $187B in 2019, an increase of more than 50% over the five-year forecast period.
The market for prescriptive analytics software is estimated to grow from approximately $415M in 2014 to $1.1B in 2019, attaining a 22% CAGR.
By 2020, predictive and prescriptive analytics will attract 40% of enterprises’ net new investment in business intelligence and analytics.

Making enterprises more customer-centric, sharpening focus on key initiatives that lead to entering new markets and creating new business models, and improving operational performance are three dominant factors driving analytics, Big Data, and business intelligence (BI) investments today. Unleashing the insights hidden in unstructured data is providing enterprises with the potential to compete and improve in areas they had limited visibility into before. Examples of these areas include the complexity of B2B selling and service relationships, healthcare services, and maintenance, repair, and overhaul (MRO) of complex machinery. All organizations face the daunting task of integrating systems together to enable greater process visibility. enosiX is taking a leadership role in this area, offering real-time integration between SAP and Salesforce systems, giving enterprises the opportunity to be more responsive to suppliers, resellers, partners and most importantly, customers.

Presented below are a roundup of recent analytics and big data forecasts and market estimates:

Global revenue in the business intelligence (BI) and analytics market is forecast to reach $16.9B in 2016, an increase of 5.2% from 2015. Gartner predicts that the BI and analytics market is in the final stages of a multiyear shift from IT-led, system-of-record reporting to business-led, self-service analytics. Source: Gartner Says Worldwide Business Intelligence and Analytics Market to Reach $16.9 Billion in 2016.

Big data and business analytics worldwide revenues will grow from nearly $122B in 2015 to more than $187B in 2019, an increase of more than 50% over the five-year forecast period. The industries that present the largest revenue opportunities are Discrete Manufacturing ($22.8B in 2019), Banking ($22.1B), and Process Manufacturing ($16.4B). Source: Worldwide Big Data and Business Analytics Revenues Forecast to Reach $187 Billion in 2019, According to IDC.

The global big data market will grow from $18.3B in 2014 to $92.2B by 2026, representing a compound annual growth rate of 14.4 percent. Wikibon predicts significant growth in all four sub-segments of big data software through 2026. Data management (14% CAGR), core technologies such as Hadoop, Spark and streaming analytics (24% CAGR), databases (18% CAGR) and big data applications, analytics and tools (23% CAGR) are the four fastest growing sub-segments according to Wikibon. Source: Wikibon forecasts Big Data market to hit $92.2B by 2026.

In 2015, the Global Analytics and Business Intelligence applications market grew 4% to approach nearly $11.6B in license, maintenance and subscription revenues with SAP maintaining market leadership. SAP led the marketing with 10% market share and $1.2B in Analytics and Business Intelligence (BI) product revenues, riding on a 23% jump in license, maintenance, and subscription revenues. SAS Institute was No. 2 achieving 9% share; IBM was the third at 8%, and Oracle and Microsoft were fourth and fifth place with 7% and 5%, respectively. Source: Apps Run The World: Top 10 Analytics and BI Software Vendors and Market Forecast 2015-2020. For a useful comparison of Tableau vs QlikView vs Microsoft Power BI, please see the recent analysis from SelectHub here.

Through 2020, Spending on Cloud-Based Big Data and Analytics (BDA) Technology Will Grow 4.5x Faster Than Spending for On-Premises Solutions. IDC’s FutureScape: Worldwide Big Data and Analytics 2016 Predictions (PDF, 13 , no opt-in, courtesy of Cloudera) define ten predictions that define the future of big data and analytics (BDA). IDC also predicts that by 2020, 50% of all business analytics software will incorporate prescriptive analytics built on cognitive computing functionality. The Source: IDC FutureScape: Worldwide Big Data and Analytics 2016 Predictions courtesy of Cloudera

The Total Data market is expected to nearly double in size, growing from $69.6B in revenue in 2015 to $132.3B in 2020. The specific market segments included in 451 Research’s analysis are operational databases, analytic databases, reporting and analytics, data management, performance management, event/stream processing, distributed data grid/cache, Hadoop, and search-based data platforms and analytics. Source: Total Data market expected to reach $132bn by 2020; 451 Research, June 14, 2016.

U.S. and International Operations (29%) and Enterprises (27%) lead the adoption of Big Data globally. Midmarket firms have the fastest growth rate of Big Data initiatives and programs in their organizations between 2014 and 2015. Source: International Institute For Analytics. Advanced Analytics and Big Data Adoption Report, 2016.(free, opt-in reqd.)

Improving customer relationships (55%) and making the business more data-focused (53%) are the top two business goals or objectives driving investments in data-driven initiatives today. 78% of enterprises agree that collection and analysis of Big Data have the potential to change fundamentally the way they do business over the next 1 to 3 years. Source: IDG Enterprise 2016 Data & Analytics Research, July 5, 2016.

Venture capital (VC) investment in Big Data accelerated quickly at the beginning of the year with DataDog ($94M), BloomReach ($56M), Qubole ($30M), PlaceIQ ($25M) and others receiving funding. Big Data startups received $6.64B in venture capital investment in 2015, 11% of total tech VC. M&A activity has remained moderate (FirstMark noted 35 acquisitions since their latest landscape was published last year). Source: Matt Turck’s blog post, Is Big Data Still a Thing? (The 2016 Big Data Landscape).

Oracle (16.4%), SAP (13.1%), IBM (10.3%), Microsoft (9.1%) and SAS (6.1%) are the market share leaders in business analytics through 2015. During the last year, IDC also found the on-premises portion of the overall market contracted by 1.4%, while the public cloud services revenue grew 26.5%. Public cloud portion of the market now represents 17% of the market. Source: IDC, Worldwide Business Analytics Software Market Shares, 2015: Healthy Demand Despite Currency Exchange Rate Headwinds (PDF, free, courtesy of SAS).

IDC forecasts global spending on cognitive systems will reach nearly $31.3 billion in 2019 with a five-year compound annual growth rate (CAGR) of 55%. More than 40% of all cognitive systems spending throughout the forecast will go to software, which includes both cognitive applications (i.e., text and rich media analytics, tagging, searching, machine learning, categorization, clustering, hypothesis generation, question answering, visualization, filtering, alerting, and navigation). Also included in the forecasts are cognitive software platforms, which enable the development of intelligent, advisory, and cognitively enabled solutions. Source: Worldwide Spending on Cognitive Systems Forecast to Soar to More Than $31 Billion in 2019, According to a New IDC Spending Guide.

The market for prescriptive analytics software is estimated to grow from approximately $415M in 2014 to $1.1B in 2019, attaining a 22% CAGR. 10% of organizations currently have some form of prescriptive analytics. The portion of organizations adopting prescriptive analytics is forecasted to grow to 35% by 2020, and the bulk of new adoption will be from large organizations in mature economies. Source: Gartner Forecast Snapshot: Prescriptive Analytics, Worldwide, 2016; 5 February 2016 (client access reqd).

The global business intelligence and analytics software market is expected to increase from $17.9B in 2014 to $26.78B in 2019, attaining a CAGR of 8.4%. Banking, financial services, insurance, retail, IT, and telecom will account for the largest percentage of the analytics and BI market. Source: Blog post on Marketresearch.com, The Business Intelligence and Analytics Software Market.

By 2020, predictive and prescriptive analytics will attract 40% of enterprises’ net new investment in business intelligence and analytics. By 2020, only 50% of chief analytics officers will have successfully created a narrative that links financial objectives to business intelligence and analytics initiatives and investments. Source: Gartner, 100 Data and Analytics Predictions Through 2020 Published: 24 March 2016 ID: G00301430 Analyst(s): Douglas Laney | Ankush Jain (client access reqd).

Big Data Analytics & Hadoop Market accounted for $8.48B in 2015 and is expected to reach $99.31B by 2022 growing at a CAGR of 42.1% from 2015 to 2022. The rise of big data analytics and rapid growth in consumer data capture and taxonomy techniques are a few of the many factors fueling market growth. Source: Stratistics Market Research Consulting (PDF, opt-in, payment reqd).

Additional sources of market information:

Analytics Trends 2016 The Next Evolution, Deloitte.

Big data analytics, Ericsson White Paper Uen 288 23-3211 Rev B | October 2015

Big Data and the Intelligence Economy in Canada Big Data: Big Opportunities to Create Business Value, EMC.

The Forrester Wave™: Big Data Hadoop Distributions, Q1 2016

The Forrester Wave™: Big Data Hadoop Cloud Solutions, Q2 2016

The Forrester Wave™: Big Data Text Analytics Platforms, Q2 2016

The Forrester Wave™: Big Data Streaming Analytics, Q1 2016

The Forrester Wave™: Customer Analytics Solutions, Q1 2016

From Big Data to Better Decisions: The ultimate guide to business intelligence today (Domo)

Gartner Hype Cycle for Business Intelligence and Analytics, 2015

IBM: Extracting business value from the 4 V’s of big data

IDC Worldwide Big Data Technology and Services 2012 – 2015 Forecast

Opportunities in Telecom Sector: Arising from Big Data. Deloitte, November 2015

Who will win as Finance doubles down on analytics?

Jul 2

10 Ways Machine Learning Is Revolutionizing Manufacturing

Bottom line: Every manufacturer has the potential to integrate machine learning into their operations and become more competitive by gaining predictive insights into production.

Machine learning’s core technologies align well with the complex problems manufacturers face daily. From striving to keep supply chains operating efficiently to producing customized, built- to-order products on time, machine learning algorithms have the potential to bring greater predictive accuracy to every phase of production. Many of the algorithms being developed are iterative, designed to learn continually and seek optimized outcomes. These algorithms iterate in milliseconds, enabling manufacturers to seek optimized outcomes in minutes versus months.

The ten ways machine learning is revolutionizing manufacturing include the following:

Increasing production capacity up to 20% while lowering material consumption rates by 4%. Smart manufacturing systems designed to capitalize on predictive data analytics and machine learning have the potential to improve yield rates at the machine, production cell, and plant levels. The following graphic from General Electric and cited in a National Institute of Standards (NIST) provides a summary of benefits that are being gained using predictive analytics and machine learning in manufacturing today.

Source: Focus Group: Big Data Analytics for Smart Manufacturing Systems

Providing more relevant data so finance, operations, and supply chain teams can better manage factory and demand-side constraints. In many manufacturing companies, IT systems aren’t integrated, which makes it difficult for cross-functional teams to accomplish shared goals. Machine learning has the potential to bring an entirely new level of insight and intelligence into these teams, making their goals of optimizing production workflows, inventory, Work In Process (WIP), and value chain decisions possible.

Source: GE Global Research Stifel 2015 Industrials Conference

Improving preventative maintenance and Maintenance, Repair and Overhaul (MRO) performance with greater predictive accuracy to the component and part-level. Integrating machine learning databases, apps, and algorithms into cloud platforms are becoming pervasive, as evidenced by announcements from Amazon, Google, and Microsoft. The following graphic illustrates how machine learning is integrated into the Azure platform. Microsoft is enabling Krones to attain their Industrie 4.0 objectives by automating aspects of their manufacturing operations on Microsoft Azure.

Source: Enabling Manufacturing Transformation in a Connected World John Shewchuk Technical Fellow DX, Microsoft

Enabling condition monitoring processes that provide manufacturers with the scale to manage Overall Equipment Effectiveness (OEE) at the plant level increasing OEE performance from 65% to 85%. An automotive OEM partnered with Tata Consultancy Services to improve their production processes that had seen Overall Equipment Effectiveness (OEE) of the press line reach a low of 65 percent, with the breakdown time ranging from 17-20 percent. By integrating sensor data on 15 operating parameters (such as oil pressure, oil temperature, oil viscosity, oil leakage, and air pressure) collected from the equipment every 15 seconds for 12 months. The components of the solution are shown

Source: Using Big Data for Machine Learning Analytics in Manufacturing

Machine learning is revolutionizing relationship intelligence and Salesforce is quickly emerging as the leader. The series of acquisitions Salesforce is making positions them to be the global leader in machine learning and artificial intelligence (AI). The following table from the Cowen and Company research note, Salesforce: Initiating At Outperform; Growth Engine Is Well Greased published June 23, 2016, summarizes Salesforce’s series of machine learning and AI acquisitions, followed by an analysis of new product releases and estimated revenue contributions. Salesforce’s recent acquisition of e-commerce provider Demandware for $2.8B is analyzed by Alex Konrad is his recent post, Salesforce Will Acquire Demandware For $2.8 Billion In Move Into Digital Commerce. Cowen & Company predicts Commerce Cloud will contribute $325M in revenue by FY18, with Demandware sales being a significant contributor.

Revolutionizing product and service quality with machine learning algorithms that determine which factors most and least impact quality company-wide. Manufacturers often are challenged with making product and service quality to the workflow level a core part of their companies. Often quality is isolated. Machine learning is revolutionizing product and service quality by determining which internal processes, workflows, and factors contribute most and least to quality objectives being met. Using machine learning manufacturers will be able to attain much greater manufacturing intelligence by predicting how their quality and sourcing decisions contribute to greater Six Sigma performance within the Define, Measure, Analyze, Improve, and Control (DMAIC) framework.

Increasing production yields by the optimizing of team, machine, supplier and customer requirements are already happening with machine learning. Machine learning is making a difference on the shop floor daily in aerospace & defense, discrete, industrial and high-tech manufacturers today. Manufacturers are turning to more complex, customized products to use more of their production capacity, and machine learning help to optimize the best possible selection of machines, trained staffs, and suppliers.

The vision of Manufacturing-as-a-Service will become a reality thanks to machine learning enabling subscription models for production services. Manufacturers whose production processes are designed to support rapid, highly customized production runs are well positioning to launch new businesses that provide a subscription rate for services and scale globally. Consumer Packaged Goods (CPG), electronics providers and retailers whose manufacturing costs have skyrocketed will have the potential to subscribe to a manufacturing service and invest more in branding, marketing, and selling.

Machine learning is ideally suited for optimizing supply chains and creating greater economies of scale. For many complex manufacturers, over 70% of their products are sourced from suppliers that are making trade-offs of which buyer they will fulfill orders for first. Using machine learning, buyers and suppliers could collaborate more effectively and reduce stock-outs, improve forecast accuracy and met or beat more customer delivery dates.

Knowing the right price to charge a given customer at the right time to get the most margin and closed sale will be commonplace with machine learning. Machine learning is extending what enterprise-level price optimization apps provide today. One of the most significant differences is going to be just how optimizing pricing along with suggested strategies to close deals accelerate sales cycles.

Additional reading:

Cisco Blog: Deus Ex Machina: Machine Learning Acts to Create New Business Outcomes

Enabling Manufacturing Transformation in a Connected World John Shewchuk Technical Fellow DX, Microsoft

Focus Group: Big Data Analytics for Smart Manufacturing Systems

GE Predix: The Industrial Internet Platform

IDC Manufacturing Insights reprint courtesy of Cisco: Designing and Implementing the Factory of the Future at Mahindra Vehicle Manufacturers

Machine Learning: What It Is And Why It Matters

McKinsey & Company, An Executive’s Guide to Machine Learning

MIT Sloan Management Review, Sales Gets a Machine-Learning Makeover

Stanford University CS 229 Machine Learning Course Materials
The Economist Feature On Machine Learning

UC Berkeley CS 194-10, Fall 2011: Introduction to Machine Learning
Lecture slides, notes

University of Washington CSE 446 – Machine Learning – Winter 2014

Sources:

Lee, J. H., & Ha, S. H. (2009). Recognizing yield patterns through hybrid applications of machine learning techniques. Information Sciences, 179(6), 844-850.

Mackenzie, A. (2015). The production of prediction: What does machine learning want?. European Journal of Cultural Studies, 18(4-5), 429-445.

Pham, D. T., & Afify, A. A. (2005, July). Applications of machine learning in manufacturing. In Intelligent Production Machines and Systems, 1st I* PROMS Virtual International Conference (pp. 225-230).

Priore, P., de la Fuente, D., Puente, J., & Parreño, J. (2006). A comparison of machine-learning algorithms for dynamic scheduling of flexible manufacturing systems. Engineering Applications of Artificial Intelligence, 19(3), 247-255.

Oct 23

Businesses Adopting Big Data, Cloud & Mobility Grow 53% Faster Than Peers

Orchestrating big data, cloud and mobility strategies leads to 53% greater growth than peers not adopting these technologies.
73% of midmarket companies say the complexity of their stored data requires big data analytics apps and tools to better gain insights from.
41% of midmarket companies are using big data to better target marketing efforts.
54% of midmarket companies’ security budgets are invested in security plans versus reacting to threats.

These and many other insights are from Dell’s second annual Global Technology Adoption Index (GTAI 2015) released last week in collaboration with TNS Research. The Global Technology Adoption Index surveyed IT and business decision makers of mid-market organizations across 11 countries, interviewing 2,900 IT and business decision makers representing businesses with 100 to 4,999 employees.

The purpose of the index is to understand how business users perceive, plan for and utilize four key technologies: cloud, mobility, security and big data. Dell released the first wave of its results this week and will be publishing several additional chapters throughout 2016. You can download Chapter 1 of the study here (PDF, no opt-in, 18 pp.).

Key take-aways from the study include the following:

Orchestrating big data, cloud and mobility strategies leads to 53% greater growth than peers not adopting these technologies. Midmarket organizations adopting big data alone have the potential to grow 50% more than comparable organizations. Effective use of Bring Your Own Device (BYOD) mobility strategies has the potential to increase growth by 53% over laggards or late adopters..

73% of North American organizations believe the volume and complexity of their data requires big data analytics apps and tools. This is up from 54% in 2014, indicating midmarket organizations are concentrating on how to get more value from the massive data stores many have accumulated. This same group of organizations believe they are getting more value out of big data this year (69%) compared to last year (64%). Top outcomes of using big data include better targeting of marketing efforts (41%), optimization of ad spending (37%), and optimization of social media marketing (37%).

54% of an organization’s security budget is invested in security plans versus reacting to threats. Dell & TNS Research discovered that midmarket organizations both in North America and Western Europe are relying on security to enable new devices or drive competitive advantage. In North America, taking a more strategic approach to security has increased from 25% in 2014 to 35% today. In Western Europe, the percentage of companies taking a more strategic view of security has increased from 26% in 2014 to 30% this year.

IT infrastructure costs to support big data initiatives (29%) and costs related to securing the data (28%) are the two greatest barriers to big data adoption. For cloud adoption, costs and security are the two biggest barriers in midmarket organizations as is shown in the graphic below.

Cloud use by midmarket companies in France increased 12% in the last twelve months, leading all nations in the survey. Of the 11 countries surveyed, France had the greatest increase in cloud adoption within midmarket companies. French businesses increased their adoption of cloud applications and platforms from 70% in 2014 to 82% in 2015.

Sources: Dell Study Reveals Companies Investing in Cloud, Mobility, Security and Big Data Are Growing More Than 50 Percent Faster Than Laggards. October 13, 2015

Oct 18

2015 Big Data Market Update

42.6% of all big data apps developed for manufacturing are being created by enterprises today.
38.2% of all big data and advanced analytics apps in use today are in customer-facing departments including marketing, sales, and customer service.
33.2% of all big data and advanced analytics developers are concentrating on the software & computing industry.
19.2% of big data app developers say quality of data is the biggest problem they consistently face when building new apps.

These and other insights are from the recently published report Big Data and Advanced Analytics Survey 2015, Volume I by Evans Data Corporation. The survey is based on 444 in-depth interviews with developers who are currently working with analytics and databases and are both currently working on and planning big data and advanced analytics projects. The survey’s results provide a strategic view of the attitudes, adoption patterns and intentions of developers in relation to big data and analytics. You can more on the methodology of the report here.

Key take-aways from the report include the following:

Software & computing (18%), financial (11.6%), manufacturing (10.9%) and retail (9.8%) industries have the highest percentage of programmers creating big data and analytics applications today. Additional industries where big data app development is active and growing include entertainment (7.7%), telecommunications (7.5%), utilities & energy (6.6%) and healthcare (4.6%). The following graphic provides an overview of the industries addressed.

Capturing more information than traditional database practices (22.60%), capturing and analyzing unstructured data (21.10%) and the potential for visualizing or analyzing data differently (20.70%) are the three top use cases driving app development today. Evans Data found that capturing more information than traditional database practices allow increased 6% since last year, making it the top use case in 2015. The following graphic provides the distribution of responses by use cases from the developers surveyed.

Total size of the data being processed (40.8%), complex, unstructured nature of the data (38.1%) and the need for real-time data analysis (17.7%) are the top three factors driving big data adoption over traditional database solutions. Evans Data found that the size and complexity of structured and unstructured data is the catalyst that gets enterprises moving on the journey to big data adoption. The ability to gain greater insights into their data with descriptive, predictive and contextually-driven analytics is the fuel that keeps big data adoption moving forward in all companies.

33.2% of all big data and advanced analytics developers are concentrating on the software & computing industry. Of these developers, 36.7% are working in organizations of 101 to 1,000 employees, 32.9% are in enterprises of 1,000+ employees, and 30.1% are in organizations of 100 employees or less. 42.6% of all big data software development in manufacturing begins in enterprises (1K+ employees).

Enterprises competing in the software & computing industry (17.5%), manufacturing (15.8%) and financial industry (14%) are investing the heaviest in big data and analytics app development. Overall, 32% of big data and analytics projects are custom-designed and produced by system integrators and value-added resellers (SI, VAR). 70% of big data and advanced analytics apps for manufacturing are created by enterprise and system integrator/value-added reseller (SI/VAR) development teams. The following graphic provides an overview of industries targeted by big data, segmented by developer segment.

Sales and customer data (9.6%), IT-based data analysis (9.4%), informatics (8.7%) and financial transactions (8.4%) are the most common big data sets app developers are working with today. In addition marketing, system management, production and shop floor data, and web & social media-generated data are also included. Evans Data found that informatics data sets grew the fastest in the last six months, and scientific computing is now competing with transaction processing systems as a dominant data set developers rely on to create new apps.

Marketing departments have quickly become the most common users of big data and advanced analytics apps (14.4%) followed by IT (13.3%) and Research & Development (13%). Evans Data asked developers which departments in their organizations are putting big data and advanced analytics apps to use, regardless of where they were created. 38.2% of all big data use in organizations today are in customer-facing departments including marketing, sales, and customer service.

Availability of relevant tools (10.9%), storage costs (10.2%) and siloed business, IT, and analytics/data science teams (10.0%) are the top three barriers developers face in building new apps. It’s interesting to note that compliance and having to transition from legacy systems did not score higher in the survey, as these two areas are inordinately more complex in more regulated, older industries. For big data and advanced analytics to accelerate across manufacturing and financial industries, compliance and legacy systems integration barriers will need to first be addressed.

Quality of data (19.2%), relevance of data being acquired (13.5%), volume of data being processed (12.6%) and ability to adequately visualize big data (11.7%) are the four biggest problem areas faced by big data developers today. Additional problem areas include the volume of data in storage (10.5%), ability to gain insight from big data (10.1%) and the high rate of data acquisition (7.6%). The remainder of problem areas are shown in the graphic below.

Providing real-time correlation and anomaly detection of diverse security data (29.9%) and high-speed querying of security intelligence data (28.1%) are the two most critical areas vendors can assist developers with today. Big data and analytics app developers are looking to vendors to also provide more effective security algorithms for various use case scenarios (17.6%), flexible big data analytics across structured and unstructured data (14.2%) and more useful graphical front-end tools for visualizing and exploring big data (5.1%).