In this article I will outline the steps needed to setup an Amazon Elasticsearch Service cluster, a managed service from Amazon that makes it easy to deploy within minutes, operate, and scale Elasticsearch in the AWS Cloud, all within the AWS Management Console. Elasticsearch is a popular open-source search and analytics engine for use cases such as log analytics, real-time application monitoring, and click stream analytics. Amazon Elasticsearch Service provisions all the resources for your cluster and launches it. The service automatically detects and replaces failed Elasticsearch nodes, reducing the overhead associated with self-managed infrastructure and Elasticsearch software. Amazon Elasticsearch Service allows you to easily scale your cluster via a single API call or a few clicks in the AWS Management Console. With Amazon Elasticsearch Service, you get direct access to the Elasticsearch open-source API so that code and applications you’re already using with your existing Elasticsearch environments will work seamlessly. For a more detailed summary click here.
Here are the steps to have a working instance of Amazon ElasticSearch. These steps will all be performed within the web console, and are located under Analytics >> Elasticsearch Service.
- Create a domain — A domain encapsulates the Amazon ES engine instances that process Amazon ES requests, the indexed data that you want to search, snapshots of the domain, access policies, and metadata. You can create an Amazon ES domain by using the Amazon ES console, the AWS CLI, or the AWS SDK. The only information required to create a domain for ES is the name, which must:
- Must uniquely identify the domain
- First letter must be lower case
- Must be between 3 and 28 characters in length
- Contains only lowercase letters, numbers, and the hyphen (-)
- Configure the Cluster — The cluster is the collection of one or more data nodes, optional dedicated master nodes, and storage. Traffic, data, and availability requirements of your application will determine how you configure your cluster
- Instance count is the total number of EC2 instances you want created for the cluster
- Instance type is the hardware configuration of each EC2 instance
- Dedicated Master is optional, however it is recommended that for each cluster 3 are created. This provides a redundant and more stable environment for managing node clusters, indexes, and shards for each cluster created
- Zone Awareness is the option to have ElasticSearch deployed across 2 separate zones within the same region to provide higher availability of your cluster.
- Storage is where you configure the storage size and other options for your cluster. Consider indices, shards, and replicas as you create your storage. Click here for more details on EC2 storage options.
- Snapshot gives you the option to take a daily snapshot (copy) of your cluster at the time you specify.
- Access Policy is where you determine how you want to secure your ECS Domain. For this demonstration I have chosen to allow access from my IP Address only.
See screen shot below regarding configuring your instances.
Here is a screen shot that maps out the different types of storage options for EC2 instances.
As far as accessing your cluster, the screen shot you see here should only be used for development purposes.
That is it for configuration. Stay tuned for part 2 where I will explain how to create indexes and mappings, and part 3 where I will show you how to load data into and query the indexes within your newly configured AWS Elasticsearch cluster.