Index Templates

1. Overview

Index templates are the configurations of indices to be created. However, the configuration of an index can be altered after it is created individually.

Whether an index is created manually or automatically while indexing a document, the new index will be created in the light of the template settings.

A template can be either a composable index template (introduced in Elasticsearch 7.8, index template) or a component template. Component templates are, as their name indicates, building blocks that can be used again and again to create index templates, and they define mappings, settings, and aliases. Component templates make it easier and faster to create index templates.

The main purpose behind the component templates, making the templates more modular and more viable to apply with less effort.

Index templates may contain many component templates other than directly specified settings, mappings, and aliases. Just because an index template is composed of component templates, it doesn’t mean they are directly applied to a set of indices. 

In settings, specifications of an index are defined (e.g., number of shards, number of replicas, shard size, …)

In mappings, how the documents will be stored in the index is defined.

To remember an index’s name easily, aliases are used.

Basic template behavior:

• At the index creation, time index templates will not be merged; the one with the highest priority will be used
• Unless there is a composable index template that matches a particular index, a legacy template is available to be matched and applied
• Templates will be applied to a new index if the index’s name matches the template’s wildcard pattern
• Changes that have been applied to an index template won’t cause any change to the indices that have already been created
• Specified explicit settings weigh heavier when the index that’s being created matches with an index template. Explicit settings will be taken into account

For example, if “number_of_replicas” is set to 2 for the component index template and it is defined to be 0 at the index creation, then it will be 0 replicas.

logs-*-* and metrics-*-*are the 2 of the index patterns and their built-in index templates got a priority of 100.

If overriding these templates needs to be prevented, then the new index templates need to have a priority of less than 100.

2. Creating a Template

There are many ways to create index templates. Index template or component template APIs can be used. If the more interfacial way is preferred, Stack Management in Kibana UI can also be used for this purpose.

2.1 Create an Index Template

We defined the wildcard pattern as *cars. In the future, at the creation time, any index matching this pattern (e.g., used_cars) will inherit the configurations of this template.
The “manufacture_date” field is mapped in a type of date, and the format is dd-MM-yyyy, and it has a priority of 400, so it has precedence over the built-in index templates.

PUT _index_template/cars_template
{
  "index_patterns": ["*cars"],
  "priority": 400,
  "template": {
    "mappings": {
      "properties": {
        "make": {
          "type": "text"
        },  
        "manufacture_date": {
          "type": "date",
          "format":"dd-MM-yyyy"
        }
      }
    },
    "settings":{
      "number_of_shards":1,
      "number_of_replicas":1
    },
    "aliases":{
      "all_cars":{}
    }
  }
}

It is easy to create a new index matching the index template’s pattern now, and the index will acquire the mappings, settings, and aliases without effort. PUT request will create the index if it doesn’t exist or update it if it exists.

PUT used_cars

The “used_cars” index will match the pattern of the template (*cars) and inherit all of the configurations.

The indices that will also inherit the configurations of the template:

PUT european_cars
PUT japan-cars
PUT brandnewcars

The indices will not match with the pattern of the template so they won’t be able to inherit the configurations:

PUT classic_car
PUT europeancars2
PUT new-cars-2022

Let’s create some documents:

PUT used_cars/_doc/1
{
  "manufacture_date": "04-01-2022",
  "make": "ford",
  "model": "mustang"
}
​
PUT used_cars/_doc/2
{
  "manufacture_date": "26-03-2022",
  "make": "kia",
  "model": "niro"
}
​
PUT japan-cars/_doc/1
{
  "manufacture_date": "14-02-2022",
  "make": "subaru",
  "model": "forester"
}
​

Now it’s time to cast some magic.

Imagine we have lots of indices and we want to search all of them at once. In this manner we want to search the cars manufactured in the first quarter of 2022:

GET european_cars,japan-cars,brandnewcars/_search
{
  "query": {
    "range": {
      "manufacture_date": {
        "gte": "01-01-2022",
        "lte": "31-03-2022"
      }
    }
  }
}

This is not so problematic for three indices, but when we think big, there are lots of indices, and when the index number gets bigger, typos may be made easily.

Aliases become handy here:

GET all_cars/_search
{
  "query": {
    "range": {
      "manufacture_date": {
        "gte": "01-01-2022",
        "lte": "31-03-2022"
      }
    }
  }
}

GET all_cars/_search
{
  "query": {
    "match": {
      "make": "subaru"
    }
  }
}

Always consider priority value when working with high numbers of indices and index templates!

When there are more than one index templates affecting the same index patterns, the one with the highest priority calls the shot.

PUT _index_template/food_template
{
  "index_patterns": ["*fruits"],
  "priority": 400,
  "template": { . . . }
}

PUT _index_template/food_template_1
{
  "index_patterns": ["*fruits"],
  "priority": 500,
  "template": { . . . }
}

2.2 Create a Component Template

As aforementioned, component templates favor users in every way possible. Instead of creating many index templates for various desires or indices each time we need, it is more convenient to create various component templates and combine them to configure them again and again for new indices on different needs.

Re-usability is the keyword here.

component template API is used to create new component templates.

Let’s create component templates for settings, mappings, and aliases.

PUT _component_template/my_settings_template
{
  "template":{
    "settings":{
      "number_of_replicas":2,
      "number_of_shards":1
    }
  }
}

PUT _component_template/my_mappings_template
{
  "template": {
    "mappings": {
      "properties": {
        "manufacture_date": {
          "type": "date",
          "format":"dd-MM-yyyy"
        }
      }
    }
  }
}

PUT _component_template/my_aliases_template
{
  "template": {
    "aliases": {
      "all_cars": {},
      "manufactured_cars":{}
    }
  }
}

index pattern is not needed in component templates.

Now we have the building blocks to create a composable index template.

composed_of parameter will do the magic here:

PUT _index_template/composed_cars_template
{
  "index_patterns": [
    "*cars"
  ],
  "priority": 500,
  "composed_of": [
    "my_settings_template",
    "my_mappings_template",
    "my_aliases_template"
  ]
}

Component templates can be versioned optionally by “version” parameter.

PUT _component_template/my_settings_template
{
  "template":{
    "settings":{
      "number_of_replicas":2,
      "number_of_shards":2
    }
  },
  "version": 3
}