In my previous articles (here the first and the second one), I have shown the use of ElasticSearch as a full-text search engine in e-commerce, the set and use of some advanced configurations and the creation of a products index which contains all saved products.

For demonstration purposes, we used the Bogus library for the dynamic generation of products and the NEST library for the CRUD on the ElasticSearch index.

Our model Product class is defined as:

public class Product
{
    public int Id { get; set; }
    public string Ean { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
    public Brand Brand { get; set; }
    public Category Category { get; set; }
    public Store Store { get; set; }
    public decimal Price { get; set; }
    public string Currency { get; set; }
    public int Quantity { get; set; }
    public float Rating { get; set; }
    public DateTime ReleaseDate { get; set; }
    public string Image { get; set; }
    public List<review> Reviews { get; set; }
}

where Brand, Category, Store, Review, and User classes are respectively:

public class Brand
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
}
 
public class Category
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
}
 
public class Store
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
}
 
public class Review
{
    public int Id { get; set; }
    public short Rating { get; set; }
    public string Description { get; set; }
    public User User { get; set; }
}
 
public class User
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string IPAddress { get; set; }
    public GeoIp GeoIp { get; set; }
}

GeoIP is a class from the NEST library used for geographic data.

The next step is to create a dashboard that can display the products, and the performed research carried out on them to elaborate more or less advanced statistics.

Rather than developing a custom solution with certain time-consuming efforts, we decided to use Kibana. It deals with a front end application that is part of the ElasticSearch stack that allows us to view data and searches for all indexed data and track the load of queries.

Kibana can also be used for monitoring, managing, and securing the same stack.

To install and configure it just go om the page https://www.elastic.co/downloads/kibana. Here we can find the installers for all platforms, and download the one suitable for our needs.

Once downloaded and unzipped in a given folder, in our case C:\ElasticSearch\Kibana, we open the config/kibana.yml file and set the elasticsearch.hosts parameter to point to our instance of ElasticSearch (for local versions http://localhost:9200):

elasticsearch.hosts: ["http://localhost:9200"]

Let’s run bin/kibana.bat and open the http://localhost:5601 from a browser page. The page we get is the following:

Kibana interface is divided into several sections.

In the open source version there are: Discover (data interactive exploration), Visualize (data analysis in graphs, tables, tags), Dashboards (complex data views), Canvas (documents creation), Maps (georeferenced data analysis), Dev Tools (tools to process and analyze queries) and Management (index and cluster management).

We can also install the X-Pack plugin to use the Graph and Monitoring sections.

Let’s go to the Management -> Elasticsearch -> Index Management section to verify that the Elasticsearch indexes have been correctly detected:

We can find our products index and verify its mapping and matching with our data model:

To create a Kibana index, just go to the Management section -> Kibana -> Index Patterns and type a text that allows you to link the new index to one or more ElasticSearch indexes.

In our case, we type products, so as to create our Kibana index.

Once created the index, in the Discover section, it is possible to filter the data by date or by one or more fields:

Using the search bar, we can query between products using the KQL language (Kibana Query language), which allows you to easily query using the autocomplete. For example, we could type:

category.name : Games AND rating > 0.5

to know all the products in the Games category with a rating higher than 0.5.

You can select some field and add them to Selected fields, in order to have a custom result view.

Once added the index and verified the correctness of the queries, we can create new data views.

The visualizations consist of various types of graphs (bar, cakes), tables, indicators, metrics, and tag clouds. Of course, they support data aggregations.

In the Visualize section, we can create a new data visualization by using graphs.

For statistical purposes, we create some graphs of products grouped by category, brand, using simple vertical bar graphs. We get results similar to:

You can also add filters to this view. For example, in our case, we want to see only products available in the store, so we type quantity > 0 in the filter bar. We click then on the Save button to save our view.

Another useful visualization is that of products by price range. In this case, we define buckets in the price field and use them for a pie chart. Let’s divide the products by the following price ranges:

0     50
50    100
100   200
200   400
400   800
800

We can get a result similar to the following:

We can also add some sub-buckets to have aggregated data and a nested visualization.

In the Maps section, we can use Elastic Maps (multilayer maps) to show georeferenced data.

We can click on the Add layer to add data to our index by selecting which field contains georeferences info (in our case geoIp.location).

Once we created all the views needed, we move on to our first Dashboard. This last is a set of views, searches, and maps, usually updated in real-time, which provides high-level information on the index data.

In the Dashboard section, let’s click on Create a new dashboard, then to Add and select the created views:

Let’s add them all and arrange them on the dashboard layout. We can obtain such a result:

Dashboards can be filtered through KQL queries, and the view is always dynamic. We can also share and integrate them within our Web applications through iframes. If we click on Share -> Copy Iframe Code we will get such a link:

<iframe src="http://localhost:5601/app/kibana#/dashboard?embed=true" height="600" width="800"></iframe>

Another interesting feature of Kibana is Canvas. It deals with a tool for viewing and presenting data that show real-time data and combines them with colors, images, and text to create dynamic views.

In the Canvas section, let’s click on Create workpad and start adding the metrics. In our case, we set parameters for the products and available items, brands, and categories, a pie chart for the brand-category pair and an average of the prices of the items by brand.

Once created, the workpad can be shared as a JSON file or downloaded as a PDF report.

Other interesting features are:

• Machine Learning: allows you to check anomalies in the data and create new indexes with normalized data;
• Graph: enables you to view the connections between the indexed objects;
• Logs: is used to view and manage the log data of our applications, and possibly check anomalies in real-time:
• REST API: allows you to communicate via HTTP with the Kibana engine and manage our dashboards;
• APM: allows you to monitor services, applications and related performances in real-time;
• Dev Tools: a set of tools for interacting with data, including the console and a search profiler.

Conclusions

In this article, we showed you how to use Kibana to process, manage, and get the best from the ElasticSearch engine.

We wish we raised your interest in the topic.

The sample project with the code used in this article is available here: https://github.com/enricobencivenga/ProductElasticSearchAdvanced.

See you at the next article!