Awesome Elasticsearch

← Awesome TypeScript -= Awesome Elasticsearch =-

General

Elastic Stack

• Elasticsearch official website
• Logstash is a data pipeline that helps you process logs and other event data from a variety of systems
• Kibana is a data analysis tool that helps to visualize your data; Kibana Manual docs
• beats is the platform for building lightweight, open source data shippers for many types of data you want to enrich with Logstash, search and analyze in Elasticsearch, and visualize in Kibana.

Books

• Deep Learning for Search - teaches you how to leverage neural networks, NLP, and deep learning techniques to improve search performance. (2019)
• Relevant Search: with applications for Solr and Elasticsearch - demystifies relevance work. Using Elasticsearch, it teaches you how to return engaging search results to your users, helping you understand and leverage the internals of Lucene-based search engines. (2016)
• Elasticsearch in Action - teaches you how to build scalable search applications using Elasticsearch (2015)
• Elasticsearch in Action, Second edition - hands-on guide to developing fully functional search engines with Elasticsearch and Kibana. (2021)

Certifications

Elastic Certified Engineer

• Elastic Certified Engineer notes - notes and exercises to prepare the certification exam

Related (awesome) lists

• frutik/awesome-search I am building e-commerce search now. Below are listed some of my build blocks

Open-source and free products, based on Elasticsearch

• Fess is an open source full featured Enterprise Search, with a web-crawler
• Yelp/elastalert is a modular flexible rules based alerting system written in Python
• etsy/411 - an Alert Management Web Application https://demo.fouroneone.io (credentials: user/user)
• appbaseio/mirage is a 🔎 GUI for composing Elasticsearch queries
• exceptionless/Exceptionless is an error (exceptions) collecting and reporting server with client bindings for a various programming languages
• searchkit/searchkit is a UI framework based on React to build awesome search experiences with Elasticsearch
• appbaseio/reactivemaps is a React based UI components library for building Airbnb / Foursquare like Maps
• appbaseio/reactivesearch is a library of beautiful React UI components for Elasticsearch
• appbaseio/dejavu The missing UI for Elasticsearch; landing page
• Simple File Server is an Openstack Swift compatible distributed object store that can serve and securely store billions of large and small files using minimal resources.
• logagent a log shipper to parse and ship logs to Elasticsearch including bulk indexing, disk buffers and log format detection.
• ItemsAPI simplified search API for web and mobile (based on Elasticsearch and Express.js)
• Kuzzle - An open-source backend with advanced real-time features for Web, Mobile and IoT that uses ElasticSearch as a database. (Website)
• SIAC - SIAC is an enterprise SIEM built on the ELK stack and other open-source components.
• Sentinl - Sentinl is a Kibana alerting and reporting app.
• Praeco - Elasticsearch alerting made simple
• DataStation - Easily query, script, and visualize data from every database, file, and API.
• DocKit - GUI client for elasticsearch to query, manage and visualize your data.

Elasticsearch developer tools and utilities

Clients

• Python Elasticsearch Client - Official low-level elasticsearch client for python
• Elasticsearch DSL - High-level python client for Elasticsearch
• ESORM - Python Asyncio ORM based on the famous Pydantic
• Ruby Elasticsearch Client - Official low-level elasticsearch client for Ruby

Development and debugging

• Sense (from Elastic) A JSON aware developer console to Elasticsearch; official and very powerful
• ES-mode An Emacs major mode for interacting with Elasticsearch (similar to Sense)
• Elasticsearch Cheatsheet Examples for the most used queries, API and settings for all major version of Elasticsearch
• Elasticstat CLI tool displaying monitoring informations like htop
• Elastic for Visual Studio Code An extension for developing Elasticsearch queries like Kibana and Sense extention in Visual Studio Code
• Elastic Builder A Node.js implementation of the Elasticsearch DSL
• Bodybuilder A Node.js elasticsearch query body builder
• enju A Node.js elasticsearch ORM
• Peek An interactive CLI in Python that works like Kibana Console with additional features
• Logstash pipeline parser Python Parsing expression grammar (PEG) and Abstract syntax tree (AST) for Logstash pipeline syntax.

Import and Export

• Knapsack plugin is an "swiss knife" export/import plugin for Elasticsearch
• Elasticsearch-Exporter is a command line script to import/export data from Elasticsearch to various other storage systems
• esbulk Parallel elasticsearch bulk indexing utility for the command line.
• elasticdump - tools for moving and saving indices
• elasticsearch-loader - Tool for loading common file types to elasticsearch including csv, json, and parquet

Management

• Esctl - High-level command line interface to manage Elasticsearch clusters.
• Vulcanizer - Github's open sourced cluster management library based on Elasticsearch's REST API. Comes with a high level CLI tool

Elasticsearch plugins

Cluster

• sscarduzio/elasticsearch-readonlyrest-plugin Safely expose Elasticsearch REST API directly to the public
• mobz/elasticsearch-head is a powerful and essential plugin for managing your cluster, indices and mapping
• Bigdesk - Live charts and statistics for elasticsearch cluster
• Elastic HQ - Elasticsearch cluster management console with live monitoring and beautiful UI
• Cerebro is an open source(MIT License) elasticsearch web admin tool. Supports ES 5.x
• Kopf - Another management plugin that have REST console and manual shard allocation
• Search Guard - Elasticsearch and elastic stack security and alerting for free
• ee-outliers - ee-outliers is a framework to detect outliers in events stored in an Elasticsearch cluster.
• Elasticsearch Comrade - Elasticsearch admin panel built for ops and monitoring
• elasticsearch-admin - Web administration for Elasticsearch

Other

• SIREn Join Plugin for Elasticsearch This plugin extends Elasticsearch with new search actions and a filter query parser that enables to perform a "Filter Join" between two set of documents (in the same index or in different indexes).

Integrations and SQL support

• NLPchina/elasticsearch-sql - Query elasticsearch using familiar SQL syntax. You can also use ES functions in SQL.
• elastic/elasticsearch-hadoop - Elasticsearch real-time search and analytics natively integrated with Hadoop (and Hive)
• jprante/elasticsearch-jdbc - JDBC importer for Elasticsearch
• pandasticsearch - An Elasticsearch client exposing DataFrame API
• monstache - Go daemon that syncs MongoDB to Elasticsearch in near realtime

You know, for search

• jprante/elasticsearch-plugin-bundle A plugin that consists of a compilation of useful Elasticsearch plugins related to indexing and searching documents

Kibana plugins and applications

• elastic/timelion time-series analyses application. Overview and installation guide: Timelion: The time series composer for Kibana
• Kibana Alert App for Elasticsearch - Kibana plugin with monitoring, alerting and reporting capabilities
• VulnWhisperer - VulnWhisperer is a vulnerability data and report aggregator.
• Wazuh Kibana App - A Kibana app for working with data generated by Wazuh.
• Datasweet Formula - A real time calculated metric plugin Datasweet Formula.

Kibana Visualization plugins

• nbs-system/mapster - a visualization which allows to create live event 3d maps in Kibana
• Kibana Tag Cloud Plugin - tag cloud visualization plugin based on d3-cloud plugin
• LogTrail - a plugin for Kibana to view, analyze, search and tail log events from multiple hosts in realtime with devops friendly interface inspired by Papertrail
• Analyze API - Kibana 6 application to manipulate the _analyze API graphically
• kbn_network - This is a plugin developed for Kibana that displays a network node that link two fields that have been previously selected.

Discussions and social media

• /r/elasticsearch
• Elasticsearch forum
• Stackoverflow
• Books on Amazon does not fit well into this category, but worth checking out!
• TODO: Put some good twitter accounts

Tutorials

• Centralized Logging with Logstash and Kibana On Ubuntu 14.04 everything you need to now when you are creating your first Elasticsearch+Logstash+Kibana instance
• dwyl/learn-elasticsearch a getting started tutorial with a pack of valuable references
• Make Sense of your Logs: From Zero to Hero in less than an Hour! by Britta Weber demonstrates how you can build Elasticsearch + Logstash + Kibana stack to collect and discover your data
• $$ Elasticsearch 7 and Elastic Stack - liveVideo course that teaches you to search, analyze, and visualize big data on a cluster with Elasticsearch, Logstash, Beats, Kibana, and more.

Articles

System configuration

• A Useful Elasticsearch Cheat Sheet in Times of Trouble
• The definitive guide for Elasticsearch on Windows Azure
• Elasticsearch pre-flight checklist
• 9 Tips on Elasticsearch Configuration for High Performance
• Best Practices in AWS
• How to Secure Elasticsearch and Kibana with NGINX, LDAP and SSL :lock:
• Elasticsearch server on Webfaction using NGINX with basic authorization and HTTPS protocol
• Elasticsearch Guides Useful Elasticsearch guides with best practices, troubleshooting instructions for errors, tips, examples of code snippets and more.

Docker and Elasticsearch

• Running an Elasticsearch cluster with Docker

Java tuning

• Elasticsearch Java Virtual Machine settings explained
• Tuning Garbage Collection for Mission-Critical Java Applications
• G1: One Garbage Collector To Rule Them All
• Use Lucene’s MMapDirectory on 64bit platforms, please!
• Black Magic cookbook
• G1GC Fundamentals: Lessons from Taming Garbage Collection
• JVM Garbage Collector settings investigation PDF Comparison of JVM GC
• Garbage Collection Settings for Elasticsearch Master Nodes Fine tunine your garbage collector
• Understanding G1 GC Log Format To tune and troubleshoot G1 GC enabled JVMs, one must have a proper understanding of G1 GC log format. This article walks through key things that one should know about the G1 GC log format.

How to start using G1

#ES_JAVA_OPTS=""
ES_JAVA_OPTS="-XX:-UseParNewGC -XX:-UseConcMarkSweepGC -XX:+UseG1GC"

Scalable Infrastructure and performance

• The Authoritative Guide to Elasticsearch Performance Tuning (Part 1) Part 2 [Part