How to use AI as a competitive intelligence tool

The digital information universe is growing exponentially. It was only ten years ago, in 2009, that the total amount of digital information exceeded 1 zettabyte – a hundred million times more than what fits in the U.S. Library of Congress, the world’s largest library. Since then it has doubled every second year and is expected to exceed 40 zettabyte next year. The vast majority of this data is unstructured, particularly in the form of text, images, video, and audio.

This constitutes a huge opportunity and an equally huge challenge to the competitive intelligence professional. On the one hand, the vast data universe contains information about almost any aspect of the contextual environment, including such that could be of critical importance to the business. On the other hand, this valuable information is unstructured and dispersed, so collecting, structuring, and making sense of it requires time and resources.

Thus, having tools and processes in place for continuously and effortlessly extracting strategic decision support from the digital information universe is a real competitive advantage. It should be a strategic imperative for any business. This article outlines what the AI-powered paradigm of competitive intelligence looks like.

What is competitive intelligence?

Competitive intelligence (CI) is the practice of gathering insights about the external business environment – products, customers, competitors, substitutes, supply chains, or any other aspect that can support strategic decisions in an organization. It uses a large number of approaches and taps into a wide range of sources, including industry experts, analyst and financial reports, trade and news media, internet searches, and various public data sources.

The typical CI cycle consists of planning and formulating the research question; collecting information; processing and analyzing the information; and disseminating insights. The next section will look at how AI tools have the potential to radically change how CI professionals distribute their time across these stages and how they carry out their work.

How does AI change the role of the CI professional?

There are three main ways in which AI is having an impact on CI, corresponding to the second, third, and fourth stages of the archetypal CI process, described below.

1. By expanding the domains where insights can be gathered

Natural language processing has been around for a long time, but it is only recently that it has become justified to call it natural language understanding. New, deep learning-powered techniques are now able to recognize sentiment, emotions, entities, categories, and topics in text with human precision. This enables structuring and finding insights in news, social media, analyst reports, patents, academic articles, financial reports, and any other text-based information source.

Over a time span of just a few years, AI has also closed the gap between humans and machines when it comes to speech recognition. Translating speech into text enables natural language processing of, and thus extraction of competitive intelligence from, sources such as customer support calls, podcasts, and any form of broadcasting.

Partly the same deep learning techniques that have improved speech recognition, have also transformed object recognition in images. It is now possible for the competitive intelligence professional to organize and make sense of large amounts of image and video data. Use cases include visual trend spotting, brand scan, and market insight based on images published and shared in traditional and social media.

2. By changing how information is searched and processed

AI leads to two major shifts in how information is searched and processed:

In the information search and collection phase, it makes a transition from manual and criteria-based search to cognitive and adaptive search. This is made possible by deep learning techniques that understand contextual similarity without being bound by the specific words used, and which revises its own understanding based on user feedback. No more need to spend time on defining complex, keyword-based search criteria.

In the data processing and analysis phase, it replaces statistical and hypothesis-based approaches with open-ended and deep learning-powered approaches. The enabler is an AI technique that clusters texts and images into a landscape of meaning, in combination with a visual, interactive way of navigating and exploring this landscape. Effortlessly switching between overview and depth helps the CI professional to interpret and make sense of large amounts of information. It has finally become possible to see the forest and the trees at the same time.

We’ve previously written a blog post on this more adaptive and open-ended paradigm for information search.

3. By enabling new ways of disseminating insights

Machines increasingly help CI professionals with the otherwise time-consuming work of codifying and disseminating CI insights. Natural language generation can be used to formulate brief texts based on the extracted insights. Automated email alerts and dashboards can be adapted to different roles to keep the organization up to date with important developments

More and more organizations are adopting knowledge graphs as a means to structuring and storing information. Relevant information is continuously extracted from various structured and unstructured data sources inside and outside the organization. It is then embedded into the knowledge graph, where it is easily browsed and queried. Structuring information in this way also enables integration with conversational interfaces such as chatbots.

To sum up: redefining what it means to be a CI professional The AI-powered CI approach is changing the way CI professionals conduct their work and what they spend their time on. If they traditionally spent 80% of the time on collecting and processing information and the remaining 20% making sense of it and disseminating the insights, these proportions are being flipped with the new, AI-powered paradigm. Rather than defining and revising static information search criteria, CI professionals are instead teaching deep learning models to capture relevant information. Instead of analyzing data using statistical approaches, they use deep learning to surface interesting patterns and their human knowledge to make sense of them. As an alternative to tedious report writing, they leverage technologies for automatically codifying insights and disseminating them in ways customized for different roles in the organization.



Covestro (formerly Bayer MaterialScience), a leading global chemicals company, wanted to accelerate its innovation by leveraging external startups in China. Their first step was to map the entire Chinese startup ecosystem, using data on hundreds of thousands of investment deals. Using Dcipher Analytics to structure this data and transforming it into a network of ownership relations helped them understand the structure and stakeholders of the startup landscape. Dcipher Analytics’ AI-driven CI toolbox also helped them single out the most interesting startups for collaboration.

To try out AI-powered competitive intelligence yourself, sign up for a free trial of Dcipher Analytics and follow our in-app tutorials.