Elasticsearch Clusters on AWS: OpenSearch & EC2

To run an Elasticsearch or OpenSearch cluster on AWS in 2026 you have two honest options: use the fully managed Amazon OpenSearch Service (formerly Amazon Elasticsearch Service), or self-manage Elasticsearch or OpenSearch on EC2 using the discovery-ec2 plugin for node discovery and repository-s3 for snapshots. Managed OpenSearch removes almost all operational work; self-managed EC2 gives you full control over the version and configuration.

This guide is a 2026 rewrite of an older tutorial. The original used the cloud-aws plugin, Java 8, Elasticsearch 2.x, and node.master/node.data flags — all of which are obsolete. In 2021 Elastic relicensed Elasticsearch under SSPL/Elastic License v2, AWS forked the project into OpenSearch, and the discovery plugin was split into discovery-ec2. Modern Elasticsearch (8.x/9.x) also ships with security on by default. Everything below reflects how a cluster is actually formed today.

Key takeaways

• On AWS in 2026 you pick one of two paths: managed Amazon OpenSearch Service or self-managed Elasticsearch/OpenSearch on EC2 — there is no single "right" answer, it's an ops-vs-control trade-off.
• The old cloud-aws discovery plugin was removed; self-managed clusters now use discovery-ec2 for peer discovery and repository-s3 for snapshots.
• Amazon's managed service is OpenSearch Service, not "Elasticsearch" — for genuine Elastic-built Elasticsearch as a managed service, use Elastic Cloud.
• Cluster formation moved from the old discovery.zen.* settings to discovery.seed_hosts/discovery.seed_providers plus cluster.initial_master_nodes (Elasticsearch 7.0+).
• Use an EC2 IAM role, not static access_key/secret_key, so discovery and S3 snapshots authenticate without secrets in elasticsearch.yml.
• Modern Elasticsearch has TLS and authentication enabled by default — security is no longer an optional add-on.

What changed since the old cloud-aws tutorial?

If you are following a guide from a few years ago, several pieces no longer exist:

• cloud-aws plugin → discovery-ec2 (+ repository-s3). The monolithic cloud-aws plugin was split and the old name removed. EC2-based node discovery now lives in discovery-ec2; S3 snapshot/restore lives in repository-s3.
• Elasticsearch is no longer Apache-2.0. In 2021 Elastic relicensed under the Server Side Public License (SSPL) and Elastic License v2. AWS forked the last open version into OpenSearch (Apache-2.0). Amazon's managed offering was renamed from "Amazon Elasticsearch Service" to Amazon OpenSearch Service.
• discovery.zen.* is gone. Since 7.0 you bootstrap a cluster with cluster.initial_master_nodes and find peers with discovery.seed_hosts or a seed provider such as discovery.seed_providers: ec2.
• node.master/node.data booleans → node.roles. Roles are now a list, e.g. node.roles: [ master ] or node.roles: [ data, ingest ].
• Security is on by default. Elasticsearch 8.x/9.x enables TLS and authentication out of the box; you no longer bolt on a separate plugin to get it.

Managed Amazon OpenSearch Service vs self-managed on EC2

Before writing any config, decide which path fits your team. The managed service trades control for far less operational load; self-managing on EC2 trades effort for full version and configuration freedom.

Dimension	Amazon OpenSearch Service (managed)	Self-managed ES/OpenSearch on EC2
Ops burden	AWS handles patching, node replacement, and backups	You own the OS, JVM, upgrades, monitoring, and snapshots
Cost control	Pay for managed instances + storage; limited low-level tuning	Raw EC2/EBS; spot, reserved, and right-sizing all on the table — but more effort
Version freedom	OpenSearch versions AWS supports (plus legacy Elasticsearch 7.10); not Elastic's 8.x/9.x	Any Elasticsearch or OpenSearch version you choose
Scaling	Console/API blue-green deployments; some changes trigger a reconfig	Manual — add nodes and rebalance shards yourself
Security	IAM, fine-grained access control, encryption, and VPC built in	You configure TLS, authentication, network rules, and IAM yourself
Best for	Teams wanting low ops and tight AWS-native integration	Teams needing a specific ES/OpenSearch version or full control

A practical rule of thumb: choose managed OpenSearch Service unless you have a concrete reason to self-manage — for example you need a current Elastic-built Elasticsearch version, a custom plugin AWS does not allow, or very specific instance/storage economics.

What node roles do you need in a self-managed cluster?

A single Elasticsearch instance plays every role at once — master, data, and coordinating. For anything beyond a demo you separate these so one workload cannot starve another:

• Master-eligible nodes keep the cluster state: creating and deleting indices, tracking nodes, and deciding which shards live where. Run an odd number (three is typical) so master election always has a quorum.
• Data nodes hold the shards that contain your indexed documents and do the heavy I/O-, memory-, and CPU-bound work: indexing, search, and aggregations.
• Coordinating-only nodes (the modern name for the old "client" node) route requests, run the search reduce phase, and spread out bulk indexing. You get one by giving a node an empty node.roles: [ ].

Dedicated roles give a cluster more predictable performance. The setup below uses dedicated master and data nodes.

How do you set up a self-managed cluster on EC2?

First, install Elasticsearch and the discovery-ec2 plugin on each EC2 instance. Note the plugin name: it is not the removed cloud-aws.

# Install Elasticsearch 8.x on an EC2 instance (Debian/Ubuntu)
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch \
  | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] \
https://artifacts.elastic.co/packages/8.x/apt stable main" \
  | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
sudo apt-get update && sudo apt-get install -y elasticsearch

# Install the EC2 discovery plugin (the old "cloud-aws" plugin no longer exists)
sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install discovery-ec2

sudo systemctl enable --now elasticsearch

Give each EC2 instance an IAM role (instance profile) that allows ec2:DescribeInstances, so discovery-ec2 can list its peers without any static credentials. This is far safer than the old habit of pasting access_key/secret_key into elasticsearch.yml. If you are new to scoping these permissions, see our guide to AWS IAM roles and policies.

To keep clusters from accidentally merging, tag every instance (for example es-cluster=prod-search) and filter discovery on that tag. Choose a unique cluster.name per cluster — nodes that share a cluster.name and can see each other will form one cluster.

# /etc/elasticsearch/elasticsearch.yml  --  master-eligible node
cluster.name: prod-search
node.name: master-1
node.roles: [ master ]

network.host: _ec2_                 # bind to the instance's private IP

# EC2-based discovery (replaces the removed cloud-aws plugin)
discovery.seed_providers: ec2
discovery.ec2.tag.es-cluster: prod-search   # only join instances with this tag
discovery.ec2.host_type: private_ip

# Bootstrap the cluster ONCE on first start, then delete this line
cluster.initial_master_nodes: [ "master-1", "master-2", "master-3" ]

# Security is ON by default in 8.x/9.x (TLS + authentication)
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

# /etc/elasticsearch/elasticsearch.yml  --  data node
cluster.name: prod-search
node.name: data-1
node.roles: [ data, ingest ]

network.host: _ec2_

discovery.seed_providers: ec2
discovery.ec2.tag.es-cluster: prod-search
discovery.ec2.host_type: private_ip
# NOTE: cluster.initial_master_nodes is set ONLY on the bootstrapping
# master node(s) the first time the cluster forms -- never on data nodes.

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

How do you back the cluster up to S3?

For a self-managed cluster you own backups. Install repository-s3 and register an S3 snapshot repository. Because the EC2 instance already has an IAM role, the repository authenticates without storing keys — just make sure the role can read and write the snapshot bucket.

# Snapshots to S3 (install on every node, then restart)
sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install repository-s3
sudo systemctl restart elasticsearch

# Register an S3 snapshot repository (auth comes from the instance IAM role)
curl -k -u elastic:$ES_PASSWORD -X PUT \
  "https://localhost:9200/_snapshot/s3_backups" \
  -H 'Content-Type: application/json' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "my-es-snapshots",
    "region": "us-east-1",
    "base_path": "prod-search"
  }
}'

Verifying nodes, security, and logs

Once the nodes are up, list the cluster members to confirm discovery worked:

curl -k -u elastic:$ES_PASSWORD https://localhost:9200/_cat/nodes?v

On security: in modern Elasticsearch you no longer add a separate plugin to lock things down. The legacy approach in our older post on securing an Elasticsearch instance used Shield (later X-Pack Security) — that functionality is now built in as Elastic Security and is enabled by default, so you mainly configure TLS certificates and roles rather than installing anything. For managed clusters, Amazon OpenSearch Service provides fine-grained access control and encryption out of the box.

If this cluster is the search-and-storage tier behind a logging pipeline, pair it with the rest of the stack as described in our ELK stack guide for parsing your logs. And because self-managed clusters live on raw EC2/EBS, revisit instance sizing and storage regularly — our notes on AWS cost and performance optimization apply directly to right-sizing data nodes.

What about the managed path?

If you choose Amazon OpenSearch Service, there is no plugin install or elasticsearch.yml to maintain. You create a domain from the AWS Console, the API, or infrastructure-as-code (CloudFormation/CDK/Terraform), pick instance types and node counts, attach it to a VPC, and AWS handles discovery, master election, patching, and snapshots for you. You give up the ability to run arbitrary versions or plugins, but for most teams that trade is worth it.

Need help choosing or running it?

Whether you want a managed OpenSearch domain stood up cleanly or a self-managed cluster on EC2 that you fully control, our team can design, secure, and operate it for you. Explore our AWS consulting services — and if search is part of a larger move onto AWS, our cloud migration services cover the end-to-end path. We have delivered 50+ projects and use AI to ship and support infrastructure in days to weeks.

Frequently Asked Questions

Is the cloud-aws Elasticsearch plugin still supported?

No. The old cloud-aws plugin was removed years ago. For a self-managed cluster on AWS you now install discovery-ec2 for EC2-based node discovery and repository-s3 for snapshot and restore to Amazon S3. They are installed separately with bin/elasticsearch-plugin install.

Can I still get "Elasticsearch" as a managed service on AWS?

AWS's own managed service is Amazon OpenSearch Service, which runs OpenSearch (and a legacy Elasticsearch 7.10 engine), not Elastic's current Elasticsearch. If you specifically need genuine, Elastic-built Elasticsearch as a managed service, use Elastic Cloud, which runs on AWS and is available through the AWS Marketplace. So the answer is yes — via Elastic Cloud — but AWS's native service is OpenSearch.

What is the difference between Amazon OpenSearch Service and Elasticsearch?

In 2021 Elastic relicensed Elasticsearch under SSPL and the Elastic License v2, moving it away from the Apache-2.0 license. AWS forked the last open version and created OpenSearch, an Apache-2.0 project, and renamed its managed offering from Amazon Elasticsearch Service to Amazon OpenSearch Service. The two share roots but have diverged in features and licensing since.

Do I still need to set access_key and secret_key in elasticsearch.yml?

No, and you should not. Attach an IAM role (instance profile) to each EC2 instance granting ec2:DescribeInstances for discovery and the relevant S3 permissions for snapshots. The discovery-ec2 and repository-s3 plugins automatically use the instance role, so no static credentials live in your configuration files.

What replaced the discovery.zen settings for cluster formation?

Since Elasticsearch 7.0, Zen discovery is gone. You bootstrap a brand-new cluster once with cluster.initial_master_nodes and let nodes find each other through discovery.seed_hosts or a seed provider such as discovery.seed_providers: ec2. The discovery.zen.minimum_master_nodes setting no longer exists.

Do I have to configure security manually on a self-managed cluster?

Elasticsearch 8.x and 9.x ship with security on by default: TLS for transport and HTTP, plus authentication, are enabled automatically, and the first start generates certificates and an elastic password. You mainly manage certificates, users, and roles rather than installing a separate plugin. Amazon OpenSearch Service provides equivalent security features built in.