Author: Deepak Mishra
Introduction
Elastic Stack is a family of products that reliably and securely retrieves data in all formats from any source for real-time search, analysis, and visualization.
Elastic Stack components:
Elasticsearch is a distributed RESTful search engine built on Apache Lucene and released under the Apache license. Based on Java, you can search and index document files in various formats.
Logstash is a data collection engine that integrates, normalizes, and distributes data from different sources. This product was originally optimized for log data but has been expanded to allow data to be retrieved from any source.
Beats are "data shippers" that are small and lightweight and are installed on the servers where your application is hosted. These beats are used to provide various kinds of operational data, either directly or via Logstash, to Elasticsearch.
Kibana is an open-source tool for exploring and visualizing the data that is stored in elasticsearch. It is specifically designed to handle massive amounts of streaming and real-time data.
Elastic Stack provides three solutions:
ElasticSearch: It is a distributed, open-source search and analytics engine built on Apache Lucene and developed in Java. It started as a scalable version of the Lucene open-source search framework and then added the ability to horizontally scale Lucene indices. It allows you to store, search, and analyze huge volumes of data quickly and in near real-time and give back answers in milliseconds.
Elastic Observability: It provides you with a single stack to unify your logs, metrics, uptime data, application traces, user experience data, and synthetics.
Elastic Security: Integrates SIEM threat detection and endpoint prevention and response capabilities into one solution. Leveraging Elasticsearch's speed and scalability, these analytics and protection capabilities allow analysts to protect their organizations from threats before damage or loss occurs.
Implementation
This demonstration shows how to use Elastic Heartbeat to monitor services, EC2 instances, and more.
Elastic Observability offers a variety of services including, logging, metrics, uptime, application performance monitoring, and more.
This demo uses the Uptime app.
Step 1:
Before you can start the actual implementation, you must first create an Elastic Cloud account at http://elastic.co.
Step 2:
After creating an account, you need to create a new deployment.. To do this, click the Create Deployment button and set up the configuration as needed.
Step 3:
Once your deployment is up and running, you can click the menu and select Uptime. The Uptime app gives you the option to add data in the right corner.
Step 4:
ElasticSearch provides a guide to installing and setting up ElasticHeartbeat to send data to Elasticsearch. Therefore, follow Elastic's instructions to set the heartbeat.
After installing Heartbeat, you need to configure the monitors and output in the heartbeat.yml file. This file helps heartbeat to know which services to monitor and where to send the data collected from those services.
Example of how to set up a monitor:
heartbeat.monitors:
- type: http
schedule: '@every 5s'
urls: [“www.google.com”]
Id: google-service
name: Google
Step 5:
To set up and run the heartbeat, use the following command after configuration.
.\heartbeat.exe setup
Start-Service heartbeat
Once the heartbeat is up and running, go to the elastic cloud, and you will see the data pouring into the elastic cloud. You can see that the service is being monitored in the Uptime app. You can also set alerts to check the availability, SLAs, and certificates of monitored services.
Conclusion:
We use heartbeat to keep track of all the services that are active across many servers or instances. Also, we have the ability to track the availability of our services, which consistently offer high availability to users.
References:
https://www.elastic.co/guide/en/beats/heartbeat/current/index.html
https://www.elastic.co/guide/en/observability/current/monitor-uptime-synthetics.html