elasticsearch - Difference between curator-cli and curator action files

Question

I'm stuck on a similar problem seen on this post, but can't find a solution : https://github.com/elastic/curator/issues/1513

To snapshot my Elasticsearch cluster (7.7.1), I use curator (5.8) to daily snapshot all indices. I realised today that only my indices starting with "." are being snapshoted by Curator.

If I use the curator-cli, all indices are indeed seen by curator and snapshoted.

I tried to remove all filters in my action file, replaced them by :

filters:
  - filtertype: none

Nothing seems to work, my dry-runs always end up listing all indices beggining with a dot.

This is my action file :

---
actions:
  1:
    action: snapshot
    description: >-
      Snapshot all indices
    options:
      repository: backup
      name: testbackup6
      ignore_unavailable: False
      include_global_state: True
      partial: False
      wait_for_completion: True
      skip_repo_fs_check: False
      disable_action: False
    filters:
    - filtertype: none

Curator logs (I have anonymized some results)

2021-01-08 18:34:44,021 INFO      DRY-RUN: snapshot: testbackup6 in repository backup with arguments: {'ignore_unavailable': False, 'include_global_state': True, 'partial': False, 'indices': '.apm-XXX,.apm-customXXX,.async-sXXX,.kibana_1,.kibana_task_manager_1,.monitoring-alerts-7,.monitoring-es-7-2021.01.02,.monitoring-es-7-2021.01.03,.monitoring-es-7-2021.01.04
...
,.triggered_watches,.watches'}

I went to see the DEBUG logs, and the indices lifecycle seems to be a problem. Here are some accepted/rejected indices :

2021-01-08 19:54:07,925 DEBUG          curator.indexlist       __not_actionable:39   Index XXXX_supervision-server_logs-2020.12.31-000014 is not actionable, removing from list.

2021-01-08 19:54:07,925 DEBUG          curator.indexlist            __excludify:58   **Removed** from actionable list: XXX_supervision-server_logs-2020.12.31-000014 has index.lifecycle.name XXX_supervision-server_logs-policy

2021-01-08 19:54:07,925 DEBUG          curator.indexlist           __actionable:35   Index .monitoring-es-7-2021.01.05 is actionable and remains in the list.

2021-01-08 19:54:07,925 DEBUG          curator.indexlist            __excludify:58   **Remains** in actionable list: index.lifecycle.name is not set for index .monitoring-es-7-2021.01.05

2021-01-08 19:54:07,925 DEBUG          curator.indexlist       __not_actionable:39   Index XXX_logs-2021.01.05-000019 is not actionable, removing from list.

Has anyone experienced this ? I can't see the link between indices having ILM policies and curator not matching them.

I can't find a workaround with regex to help me match all my indices. With the same "filtertype: none" on curator-cli, everything is OK.

Thanks a lot

Answer 1

I just found it ><

"allow_ilm_indices: True" must be added in the action file in order to show all indices...

The curator_cli has this option on True by default, which is not the case of curator itself.