1. Overview<\/a><\/p>\n 2. Static Index Settings<\/a><\/p>\n 3. Dynamic Index Settings<\/a><\/p>\n 4. Index Shard Allocation<\/a><\/p>\n 4.1. Shard allocation filtering<\/a><\/p>\n 4.2. Index allocation filter settings<\/a><\/p>\n 4.3. Allocation delay on a node leave<\/a><\/p>\n 4.4. Index recovery order<\/a><\/p>\n 4.5. Shard allocation<\/a><\/p>\n 5. Logging Slow Searches<\/a><\/p>\n 6. Slow Log Index<\/a><\/p>\n 7. Store Module<\/a><\/p>\n <\/a><\/p>\n Index modules<\/strong> adjust the configurations of the indices and control how the indices will act. Index modules are created for all indices individually.<\/p>\n Adjusting the configurations means, for example, the primary shard<\/strong> number of an index or replica shard<\/strong> number, the compression method<\/strong> to store data, etc.<\/p>\n There are two types of index settings: Static<\/strong> and Dynamic<\/strong>. Static index<\/strong> settings have to be set ahead of time<\/strong> while creating an index or after an index is closed for a closed index.<\/span><\/p>\n Some basic static index settings:<\/span><\/p>\n <\/a><\/p>\n Dynamic index<\/strong> settings can be changed anytime<\/strong> desired at runtime<\/em>.\u00a0<\/span><\/p>\n <\/a><\/p>\n Shards can be allocated to nodes by using this module and settings can be configured index by index.<\/span> The shards are going to be allocated somewhere, but where? Shard allocation filters<\/strong> can be used to decide the location of the shards of an index.<\/span><\/p>\n In this manner, alongside the built-in attributes<\/em>, custom attributes can also be used as a shard allocation filter. The built-in attributes are _name<\/strong>, _host_ip<\/strong>, _publish_ip<\/strong>, _ip<\/strong>, _host<\/strong>, _id<\/strong>, _tier<\/strong>, and _tier_preference<\/strong>.<\/span><\/p>\n A custom attribute<\/em> can be added to a node by adding the attribute to the node’s elasticsearch.yml<\/strong> file: e.g.<\/strong>, if we have 3 nodes and we want to filter them on their order, the order attribute can be added by\u00a0<\/span>node.attr.order: first<\/span><\/strong>.<\/span> In order to allocate the shards of the index e.g used_cars to first or second nodes<\/span><\/p>\n PUT used_cars\/_settings When a node says goodbye to its fellows in the cluster, the master node takes action to prevent data loss. From then, there are 3 ways available to track.<\/span><\/p>\n Whenever a node goes missing, these steps shouldn’t be taken immediately. A timeout must be set in case of the node went missing because of some communication problem. If the node comes back alive while the master node trying to do some action, this causes extra work on resources.<\/span><\/p>\n Master node will wait 10 minutes before allocating the unassigned shards that the leaver node left-off:<\/span><\/p>\n <\/a><\/p>\n By default, the newest index will be recovered first and the oldest will be recovered last but the priority of an index can be set to change the order.<\/span><\/p>\n <\/a><\/p>\n In the perspective of the cluster, shards will be spread along the nodes, but is there a limit? So, how many nodes? By default, as many as possible.\u00a0<\/span><\/p>\n index.routing.allocation.total_shards_per_node<\/span><\/strong>\u00a0is used for defining how many shards will be allocated to a single node.<\/span><\/p>\n cluster.routing.allocation.total_shards_per_node<\/span><\/strong>\u00a0is used for defining how many shards will be allocated to each node.<\/span><\/p>\n These limits must be set carefully, because they are hard limits, and if the limit is not sufficient, some shards will be left unallocated.<\/span> Dynamic settings contain some configurations for logging slow queries. The operation is done at the shard level. Slow logs may enlighten us to see why the searches are slow.<\/span><\/p>\n To set timers to log setting should be put into the index:<\/span><\/p>\n There are two phases: query<\/strong> and fetch<\/strong>.<\/p>\n The time taken in the query phase<\/strong> means the time taken while getting a list of the documents from all the searched shards<\/em>.<\/p>\n The time taken in the fetch phase<\/strong> means how long it takes to get the correct shard containing the queried documents and get the result<\/em>.<\/p>\n The queries which take longer once in a while are not the problem most of the time, but if some queries always take a long time to result, that must be investigated.1. Overview<\/span><\/h2>\n
\n<\/a><\/p>\n2. Static Index Settings<\/h2>\n
\n
3. Dynamic Index Settings<\/span><\/h2>\n
\n
\n
\n
4. Index Shard Allocation<\/span><\/h2>\n
\n<\/a><\/p>\n4.1. Shard allocation filtering<\/h3>\n
\n<\/a><\/p>\n4.2. Index allocation filter settings<\/h3>\n
\n
\n{
\n“index.routing.allocation.include.order”: “first,second”
\n}
\n<\/a><\/p>\n4.3. Allocation delay on a node leave<\/h3>\n
\n
\n
PUT _all\/_settings\r\n{\r\n\"settings\": {\r\n \"index.unassigned.node_left.delayed_timeout\": \"10m\"\r\n }\r\n}<\/pre>\n<\/blockquote>\nIf the timeout is set to 0, the setting is disabled so the missing shards will be allocated as soon as possible.<\/span><\/h6>\n
4.4. Index recovery order<\/h3>\n
\n
PUT cars-000001\r\n\r\nPUT cars-000002\r\n{\r\n\"settings\": {\r\n \"index.priority\": 100\r\n }\r\n}\r\n\r\nPUT cars-000003\r\n\r\nPUT cars-000004\r\n{\r\n\"settings\": {\r\n \"index.priority\": 10\r\n }\r\n}<\/pre>\n<\/blockquote>\n4.5. Shard allocation<\/h3>\n
\n<\/a><\/p>\n5. Logging Slow Searches<\/h2>\n
\n
PUT *cars\/_settings\r\n{\r\n \"index.search.slowlog.threshold.query.warn\": \"15s\",\r\n \"index.search.slowlog.threshold.query.info\": \"8s\",\r\n \"index.search.slowlog.threshold.query.debug\": \"4s\",\r\n \"index.search.slowlog.threshold.query.trace\": \"1s\",\r\n \"index.search.slowlog.threshold.fetch.warn\": \"1500ms\",\r\n \"index.search.slowlog.threshold.fetch.info\": \"800ms\",\r\n \"index.search.slowlog.threshold.fetch.debug\": \"1s\",\r\n \"index.search.slowlog.threshold.fetch.trace\": \"500ms\"\r\n}<\/pre>\n<\/blockquote>\nSetting the values to -1 means the timer is off and 0ms will show all the processing queries.<\/span><\/h6>\n
\n<\/a><\/p>\n6. Slow Log Index<\/h2>\n