Configure Kibana: Create Data Views and Master KQL

Post author:Noel Kamphoa
Post published:August 21, 2024
Post category:DevOps / Observability

1. Introduction

In the previous article, we saw how to install and configure Kibana using Docker. In this article, you will see some basic steps you must go through after the installation. You will see how to create and manage Dataviews. You will also learn how to query the data in a Dataview and Elasticsearch indices using the Kibana Query Language (KQL) and the Elasticsearch Query Language (ES|QL).

2. Prerequisites

You are expected to have Kibana installed before you can follow this tutorial. In case you have not yet installed Kibana, follow any of these two tutorials:

Install Kibana using Docker
Install Kibana using Docker Compose: This will also install Elasticsearch and Logstash.

3. Configure Kibana: Home Screen

Kibana is usually available at http://{servername}:5601, assuming you used the default port number during the installation. The first time you navigate to the Kibana URL you will see the following screen if there are already some data in Elasticsearch indices:

If no data exists in Elasticsearch indices, you might see a different screen inviting you to add Elastic integrations. We assume here that you have already populated some data to your Elasticsearch indices using a tool like Logstash or any other tool.

4. Create a DataView

A Dataview is a quick way to visualize the data in specific Elasticsearch indices. Without a Dataview, Kibana will display the data in all Elasticsearch indices, which might be very difficult to explore.
There are two ways to create a Dataview. The first way is by clicking the Create dataview button on the welcome page as shown here:

You may also create a Dataview by navigating to the Discover page using the Burger menu:

From the Discover page, click Create a new Dataview as shown below:

Either way, you must provide a name, an index pattern to your Dataview. You must also select the timestamp field to use for filtering. Once you type in an index pattern, Kibana will automatically detect the number of Elasticsearch indices matching the pattern.

Once you have filled up everything, click Save. The Dataview is then automatically selected and its content is displayed:

5. Set a Time Range

The first thing you need to do after creating a dataview is to adjust the time filter setting. This will ensure you are looking for data in the correct time frame.
Access the time filter field from the top right as shown here:

Adjust the time setting by selecting a value from the following zones:

Quick select: Offers more customization options
Commonly used
Recently used
Refreshing delay: By default, it’s 60 seconds.

6. Kibana Query Language (KQL)

By default, Kibana will load all the content of a dataview and use the pagination if there are multiple rows. Kibana Query Language(KQL) is a text-based query language that will help you filter the content of a dataview. It can be very helpful to look for specific text in an Elasticsearch index.

Here are a few ways of using KQL:

Filter a specific field

field_name: field_value

field_name is the name of the field you want to filter on. The field must exist within the given dataview. Here is an example using the message field of our Logstash dataview.

Filter without specifying a field name
In case you want to search for a value in any field of the documents, just omit the field name:

Pro-tip: If you are not seeing any data in the search result, ensure you have selected the correct time frame in the time filter field (top right).

Saving a Query
Whenever you create a query and would like to be able to reuse it later on, there is an option to save the query.

7. Elasticsearch Query Language (ES|QL)

Unlike KQL which is simply a text-filtering language, Elasticsearch Query Language (ES|QL) is a powerful language that allows you to filter, transform, and analyze data stored in Elasticsearch indices.

There are two ways of running an ES|QL query:

Using DevTools: This approach utilizes the ES|QL query API.
Using Discover: This is the simplest way from an end-user point of view.

We will be using the last option in this tutorial. To access it, go to Discover, and from the Dataview menu, select ES|QL as shown below:

You can then start typing your queries as shown in the image below.

ES|QL uses pipes (|) to manipulate and transform the data. To learn more about the language, read the official documentation.

Pro-tip: If you are not seeing any data in the search result, ensure you have selected the correct time frame in the time filter field (top right).

8. Conclusion

In this tutorial, you learned how to configure a Kibana dashboard. You saw how to manage Dataviews and also how to query Elasticsearch indices using Kibana Query Language(KQL) and Elasticsearch Query Language(ES|QL).