ES-Cluster 구성 - 3(DataNode)

2024.03.02 - [Elasticsearch] - ES-Cluster 구성 - 1(공통)

ES-Cluster 구성 - 1(공통)

2024.03.02 - [Elasticsearch] - ES-Cluster 구성 - 2(MasterNode, Kibana)

ES-Cluster 구성 - 2(MasterNode, Kibana)

2024.03.02 - [Elasticsearch] - ES-Cluster 구성 - 4(Fleet Server 설치)

---

ES Cluter 구성 - DataNode 설치

# Data Node에서 클러스터 가입 시 사용할 토큰 생성 (Master Node에서 실행)
sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node
eyJ2ZXIiOiI4LjEyLjEiLCJhZHIiOlsiMTAuMTAuMTIuMTgxOjkyMDAiXSwiZmdyIjoiNjJiNWZiN2UyZThjZDgzYTUwZGVhMDk2MjEyOGU5OTQ3MWI1MmJiM2UwN2VmOGExZmZmZDQxNmEyMWQzYzZhNCIsImtleSI6InE5N3h6NDBCN25KS3RfUF9tcDF1OmRXOUJFZUJsU1V5Z0xKX3RHNm5IaWcifQ==

# 위에서 생성한 토큰 활용하여 Data Node 클러스터 가입
sudo /usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token eyJ2ZXIiOiI4LjEyLjEiLCJhZHIiOlsiMTAuMTAuMTIuMTgxOjkyMDAiXSwiZmdyIjoiNjJiNWZiN2UyZThjZDgzYTUwZGVhMDk2MjEyOGU5OTQ3MWI1MmJiM2UwN2VmOGExZmZmZDQxNmEyMWQzYzZhNCIsImtleSI6InE5N3h6NDBCN25KS3RfUF9tcDF1OmRXOUJFZUJsU1V5Z0xKX3RHNm5IaWcifQ==

This node will be reconfigured to join an existing cluster, using the enrollment token that you provided.
This operation will overwrite the existing configuration. Specifically:
  - Security auto configuration will be removed from elasticsearch.yml
  - The [certs] config directory will be removed
  - Security auto configuration related secure settings will be removed from the elasticsearch.keystore
Do you want to continue with the reconfiguration process [y/N]y

#### Data node 클러스터 가입 실패 시 ES 삭제 후 재설치 ####
sudo apt autoremove --purge elasticsearch
sudo apt install -y elasticsearch

# elasticsearch.yml 설정
sudo vi /etc/elasticsearch/elasticsearch.yml         
cluster.name: xg-es                                      # 클러스터 이름
node.name: es-data1                                          # 노드 이름
node.roles: [ data, remote_cluster_client, ingest ]      # 노드 Role 설정 (master, data, data_content, transform, remote_cluster_client 등)
path.data: /data/elasticsearch/lib                               # 인덱스 데이터 디렉토리 설정
path.logs: /data/elasticsearch/log                               # 로그 디렉토리 설정
bootstrap.memory_lock: true                                      # ES 시작 시 메모리 락업하여 전용 사용
network.host: 0.0.0.0                                            # 허용할 네트워크 대역 설정

discovery.seed_hosts: ["es-master1", "es-data1", "es-data2", "es-data3"]        # 클러스터 내에서 통신할 Node List

cluster.initial_master_nodes: ["es-master1"]        # 초기 클러스터 생성 시 Master 노드 지정

xpack.security.enabled: true
xpack.security.enrollment.enabled: true

xpack.security.http.ssl:                             # http api 통신 시 ssl 사용gg
  enabled: true
  keystore.path: certs/http.p12

xpack.security.transport.ssl:                        # 노드끼리 통신할 때 ssl 사용
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12

http.host: 0.0.0.0
transport.host: 0.0.0.0
#----------------------- END SECURITY AUTO CONFIGURATION -------------------------
xpack.security.authc.api_key.enabled: true

path:                                                # ES 스냅샷 생성 시 활용할 디렉토리
  repo:
    - /data/elasticsearch/backup

indices.requests.cache.size: 2%
indices.queries.cache.size: 15%

thread_pool:                                         # Search Thread Pool 설정
    search:
        size: 30
        queue_size: 2000

# heap 메모리 설정
sudo vi /etc/elasticsearch/jvm.options
-Xms8g
-Xmx8g

# elasticsearch 실행
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch && sudo systemctl start elasticsearch
sudo systemctl status elasticsearch

# 클러스터 가입 확인
sudo curl -u elastic:'xgtestes!@' --cacert /etc/elasticsearch/certs/http_ca.crt -XGET 'https://localhost:9200/_cat/nodes'
10.10.x.184  4 74 56 0.70 0.17 0.05 dir   - es-data3
10.10.x.183 11 75 11 0.32 0.11 0.03 dir   - es-data2
10.10.x.182  8 75  0 0.01 0.07 0.03 dir   - es-data1
10.10.x.181  6 66  0 0.38 0.24 0.14 dimrs * es-master1

---

Master 노드 data role 삭제

# Master 노드에서 초기 구성시 생성된 샤드를 repurpose하여 삭제하고 Only Master로만 수행
# ElasticSearch 중지
sudo systemctl stop elasticsearch

# elasticsearch.yml에서 role 변경 (data_content Remove)
sudo vi /etc/elasticsearch/elasticsearch.yml

node.roles: [ master, remote_cluster_client, ingest ]

# repurpose 실행
sudo /usr/share/elasticsearch/bin/elasticsearch-node repurpose
Confirm [y/N] y

# elasticsearch 실행
sudo systemctl start elasticsearch
sudo systemctl status elasticsearch.service

# 클러스터에 확인
sudo curl -u elastic:'xgtestes!@' --cacert /etc/elasticsearch/certs/http_ca.crt -XGET 'https://localhost:9200/_cat/nodes'
10.10.x.181  1 64 29 0.49 0.26 0.16 imr * es-master1
10.10.x.183  2 75  1 0.07 0.08 0.02 dir - es-data2
10.10.x.184  4 75  1 0.14 0.14 0.06 dir - es-data3
10.10.x.182 14 75  1 0.38 0.19 0.08 dir - es-data1

• Kibana Tab → Management의 Stack Monitoring에서 클러스터(Node, Index) 상태 확인