Elasticsearch 3 Node Cluster (master_not_discovered_exception)

Sigh.... Been at this for a bit and am now looking to the community....

Have a brand new 3 node cluster for Elasticsearch. Ubunutu 18.04 TLS

I am getting this when I do curl -XGET '

{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503} 

Here is my elasticsearch.yml (cleaned)

# ---------------------------------- Cluster ----------------------------------- cluster.name: clustername # ------------------------------------ Node ------------------------------------ node.name: thisnode1 network.publish_host: thisnode1 # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /var/lib/elasticsearch/data # # Path to log files: # path.logs: /var/log/elasticsearch # # ---------------------------------- Network ----------------------------------- # network.host: 0.0.0.0 # http.port: 9200 # # --------------------------------- Discovery ---------------------------------- # discovery.zen.ping.unicast.hosts: ["10.0.0.2", "10.0.0.3", "10.0.0.4"] discovery.zen.minimum_master_nodes: 2 node.master: true node.data: true 

And the error in the log:

[2019-07-26T14:47:09,740][WARN ][o.e.c.c.ClusterFormationFailureHelper] [thisnode1] master not discovered or elected yet, an election requires a node with id [VEAPSAP3SMqYW-OR-J_24A], have discovered [{thisnode2}{scPfMQZcS4uigjLCUtafFQ}{IBMAZFVNRQ6zOJ2w3_RP8A}{thisnode2}{10.0.0.3:9300}{ml.machine_memory=16794214400, ml.max_open_jobs=20, xpack.installed=true}, {thisnode3}{t3lePkSJRlyRVCVHvN_WQA}{oUBItui6TaC1cIkc14YiJw}{thisnode3}{10.0.0.4:9300}{ml.machine_memory=16794214400, ml.max_open_jobs=20, xpack.installed=true}] which is not a quorum; discovery will continue using [10.0.0.3:9300, 10.0.0.4:9300] from hosts providers and [{thisnode1}{TNTCGgbJSPe6iDogwAajQg}{F3yaKygbT4O6PSG8kFPbIQ}{thisnode1}{10.0.0.2:9300}{ml.machine_memory=16794214400, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 12, last-accepted version 49 in term 12 

I am obviously missing a config. I am just not sure what.

Thanks in advance for any guidance!

1 Answer

For the benefit of others... I installed Elasticsearch on all three machines and started the service to be sure it was working. Apparently this was my mistake. In going through the logs I found an entry that said it was joining the wrong cluster. I deleted the contents of the data directory and restarted services. That was it. All three nodes happily saw each other