Unless you’ve been living underground for the last 10 years, you have at least heard of Elasticsearch. Perhaps you haven’t used it, but this open source tool from Elastic.co was created in 2004 and has become increasingly powerful since then.
For the uninitiated, Elasticsearch is an add-on tool in your application environment built specifically to search any documents in the stack. Originally called Compass, Elasticsearch was designed to be a scalable solution for indexing and searching documents, mainly for NoSQL solutions available in the market at the time. The idea was to use a commonly-used interface, specifically JSON over HTTP, so Elasticsearch could be used by languages like Ruby, Python, and PHP, not just the Java it was written in. This HTTP-first approach, plus how easy it was to get running, distinguished it from its main competitor, Solr.
Open source tools often grow beyond their creators’ original intentions. Maintaining its search functionality, Elasticsearch was enhanced adding real-time analytics, and multitenancy, or having the ability to reside in a server with multiple users with common access to the service. While distributed capabilities were something available with NoSQL solutions, with concepts such as sharding, this was something not commonly seen in search tools.
As time went on, Elasticsearch found newer applications as the needs of software developers grew. One such function is as an indexing tool for the ingest and visualization of log data: Elasticsearch was at the heart of centralized log management systems. This was such a success, Elasticsearch became part of what was known as the ELK (Elasticsearch, Logstash, Kibana) Stack, or later simply as the Elastic Stack - a set of complementary tools specifically dedicated to data ingest, retrieval, analysis and visualization.
Since Elasticsearch is, at its heart, a scalable distributed document store, it can hold a ton of data. Beyond being a secondary search index as it was originally designed, Elasticsearch is now a solid stand-alone data store.
Another place Elasticsearch has stretched its capabilities is in the distributed databases market. Using Elasticsearch instead of other options allows databases like CrateDB to collect information, distribute from nodes or devices, back to a centralized application for analysis by a user. This is a far cry from simple document search.
Under the hood, Elasticsearch is based on Apache Lucene, and makes as many features as possible available via JSON or the Java API clients commonly used. With new major versions coming out every year and patches being released even weekly, it can be difficult to keep up with Elasticsearch at its best and latest.
As you can see, Elasticsearch has come a long way since its humble beginnings as a search tool. There are many potential enhancements to functionality this tool can add to your application environment. There are many tools, but few offer the flexibility and functionality of Elasticsearch.
We may have mentioned we really like Elasticsearch. As such, we continue to keep up on Elasticsearch’s new versions, new functionality, and new implementations. We are always happy to share this knowledge and get your application streamlined by adding Elasticsearch to your application stack. If you aren’t sure where to start with Elasticsearch, get in touch - we are happy to help!