Natural Language Generation in BI

Business Intelligence uses data analytics and visualizations to help organizations make data-driven decisions. Traditionally, BI tools used a variety of visualizations to represent data collected in domains like finance, e-commerce, supply chain, etc. Today, the use of BI has evolved to include insights that leverage Artificial Intelligence. At krtrimaIQ Cognitive Solutions, we refer to this evolved BI as NextGen BI. To understand more about our concept of NextGen BI, check out NextGen Business Intelligence and Visualization .

Some of the improved insights utilized in NextGen BI are Smart Narratives, Auto Insights, Demand Forecasting, Root Cause, Word Clouds and Natural Language Q&A. Each insight provides a unique perspective into the data beyond what can be represented by simple visualizations. To know more about our NextGen BI features and solutions, take a look at our dashboards .

In this article, we will explore Smart Narratives in BI and dive deeper into the engine that drives these narratives – Natural Language Generation.

What are Smart Narratives?

Let us explore Smart Narratives through a case study of DVD Store sales. The dataset used represents an online e-commerce store that sells DVDs. The dataset contains a list of customers, demographics, products, and orders collected from 2018 till 2021. A common insight present in most sales dashboards, presents sales across multiple regions. While understanding these charts is simpler with sales panning across fewer regions, it becomes challenging as more regions are added to the dataset.

Smart narratives simplify the analysis by helping derive insights from complex structured data in human-readable form. Conceptually, these narratives extract the key insights presented by charts into freeform text retaining important information and presenting additional information that can help take better decisions.

The use of these narratives can be taken a step ahead by incorporating Text-to-Speech that converts the narratives into audio form that can be played out loud while analysing the charts.

Smart Narratives

Integration of Smart Narratives

Smart Narratives can be integrated into any industry standard BI tools such as Power BI, Tableau, QlikView/QlikSense etc. or custom developed analytics application. Some common analytics are discussed below.

  1. Sales Analytics

    The dashboards where Smart Narratives commonly find their place are Sales Analytics Dashboards. There are numerous representations of sales, both independently and with respect to another variable like region, age, gender, time, etc.

    Sales Analytics
  2. Enterprise Analytics

    Smart Narratives also finds its place in analysis for large enterprises. Organizations derive insights for revenue, product, and customer growth which can be represented using these narratives.

    Enterprise Analytics
  3. Timeline Analytics

    Representing data over various periods of time is a common form of analytics. Smart narratives for these representations are critical since it increases the speed of data-driven decision-making.

    Timeline Analytics

Generating Smart Narratives

There are different methods to create smart narratives based on the data provided. A few of them are as follows.

  1. PowerBI Smart Narratives

    The smart narrative visualization helps you quickly summarize visuals and reports. It provides relevant innovative insights that you can customize. Available to all PowerBI dashboards for free, it can be easily integrated into your report. The only shortcoming – it cannot be used outside PowerBI.

    PowerBI Smart Narratives
  2. Arria NLG

    Unlike PowerBI which has developed its own Smart Narratives engine, not all BI tools have this feature. There are advanced proprietary and open-source tools available which can be integrated into BI applications. Arria NLG is one such proprietary tool that also provides a free trial. They provide integrations with PowerBI, Tableau, Qlik, and Excel.

    Arria NLG
  3. Create your own engine (RosaeNLG)

    If you are into NLP and programming, you can create your own Smart Narrative engine using the open-source RosaeNLG. It allows you to create and customize your own templates for different use cases rather than a more generic one and can be integrated with any kind of dashboard, be it PowerBI, GoodData, or your own dashboard on React.js.

    RosaeNLG

Let us now dive deeper into what drives Smart Narratives along with a brief introduction to RosaeNLG.

What is Natural Language Generation?

Natural Language Generation is focused on automatically generating text from structured data, in human-readable form.

A few use cases of NLG are listed below.

  • Describe a product based on its features (SEO)
  • Produce Structured Reports - Risk Reports, Fund Performance in the Financial Industry
  • Describe a situation - Client summary before a meeting, Report of the visit of an office for Real Estate
  • Generate well-formed chatbot answers

How does Natural Language Generation work?

The process of Natural Language Generation can be grouped into six stages.

  1. Content Analysis

    The structured data is processed, and the key information and their relationships are identified.

  2. Data Understanding

    Patterns are detected in the processed data and segregated to form the final output.

  3. Document Structuring

    A text structure is defined for the narrative based on the data processed and patterns detected.

  4. Sentence Aggregation

    Based on the patterns relevant phrases and their variations are combined to form complete meaningful sentences.

  5. Grammatical Structuring

    General rules of grammar are applied to the sentences to fine tune the syntactical structure of the sentences and rephrase the sentences.

  6. Language Presentation

    The final narrative is generated based on the templates selected for the data.

NLG typically relies on Machine Learning algorithms like Markov Chains, Recurrent Neural Networks and LSTM models. The more commonly used models are based on the Transformer architecture like Generative Pre-trained Transformer (GPT), Bidirectional Encoder Representations from Transformers (BERT) and XLNet.

With a deeper understanding of NLG, let us quickly look at RosaeNLG and its features.

RosaeNLG

RosaeNLG is a Natural Language Generation library built on the Pug template engine and intended for Node.js or client side (browser) execution. English, French, German, Italian, and Spanish are fully supported with grammar, gender, etc. Apart from these languages, text can be generated in other languages as well, with fewer features. RosaeNLG is powerful enough to create real-world NLG applications.

Some of the features of RosaeNLG are:

  • Support for JavaScript and Typescript generation
  • Complete support for languages like English, French, German, Italian and Spanish
  • Easy customized template creation for growing generic data with the use of conditions and loops
  • Multiple synonyms for phrases and sentences picked at random
  • Allows the use of HTML tags and CSS styling
  • Representation of dates and numbers in different locale-specific formats

Conclusion

Natural Language Generation is a trending concept and its use in BI Dashboards in the form of Smart Narratives is gaining traction in recent times.