On Tuesday the 22nd of November, a group of Devoteam consultants visited Elastic’s “ElasticON” event in The Beurs van Berlage in Amsterdam. Elastic is from origin a Dutch IT “Search” company that builds self-managed and SaaS offerings for Search, Logging, Security, and Analytics use cases.
The company develops the open source Elastic Stack (Elasticsearch, Kibana, Beats, and Logstash) previously known as ELK Stack, free and paid proprietary features (previously X-Pack), Elastic Cloud and Elastic Cloud Enterprise. They offer solutions with which you can put large sets of data in one database through which you can search swiftly.
There were multiple sessions, announcements and product updates during the day. Devoteam was one of the core sponsors for this mega event that had well over 500 people in attendance.
Top 3 Takeaways
Takeaway #1: Upping the ante on search
Elastic is taking the world of search head-on! In multiple tech sessions leading to the big event, Elastic emphasized how they are improving on Search – a core component of their product suite. They also covered in length the transition away from traditional semantic search (simple text matching), to a more empowering vector search, the Elastic way. Where traditional approach relies on mentions of keywords, lexical similarity, and the frequency of word occurrences, vector search engines use distances in the embedding space to represent similarity.
Hierarchical Navigable Small Worlds (HNSW): Algorithms like k-Nearest Neighbor Algorithm (kNN), have been associated with excessive execution times and zaps between computational resources. Elastic has decided to utilize a more native ANN algorithm in Lucene 9. This sacrifices perfect accuracy in exchange for executing efficiently in high dimensional embedding spaces, at scale.
The result is a search which works seamlessly across textual data (documents), images, and audio. Additionally, the hybrid BM25 scoring algorithm that elastic employs, scores high on reciprocal ranking. Additionally, the built-in Eland library lets anyone load NLP models off-the-shelf without worrying about needing the expertise to train ML models.
Takeaway #2: Navigating into the unknown, fearlessly!
One of the early Keynotes of the day, laid the path forward for the future of Elastic Security. Elastic has been engaged for several years with the security team of MITRE. One of the outcomes of this collaboration is that Threat Detection, Investigation and Response (TIDR) is now at the heart of Elastic Security. Elastic’s SIEM boasts of industry leading threat response capabilities. With advanced Cloud Posture Management (K/CSPM) and Cloud Workload Monitoring and Protection (CWP) capabilities built-in, this solution is fully Cloud ready.
The shared responsibility model: How do partners and clients trust a third-party cloud service that can potentially save costs? Can the docker image, which helped speed up development, be a trojan? Who draws the boundaries of responsibility when critical layers like SSO are compromised? The teams behind Elastic’s Security orchestration, automation, and response (SOAR), Threat Intelligence (TI) and Endpoint Security illustrated how they are working in tandem to build a product that is continually addressing these critical questions which are of paramount importance in the world of developers.
Takeaway #3: Google Data Cloud Alliance
Google Cloud was also on stage, not just to talk about cloud but to give a highly impactful presentation that showed the outcome of years of serious collaboration with Elastic.
They showcased real world examples where customers leveraged Elastic’s Observability solution to continually monitor the migration of their workloads to Google Cloud. Google Cloud team also highlighted how major brands have benefitted by transitioning from a self-managed to a fully managed enterprise on GCP while partnering with Elastic for monitoring, search and security.
Bonus takeaway: The Future of Elasticsearch
Elastic-ON not just serves as a podium to talk about the difference Elastic makes to businesses, but it also serves as the forum where Elastic, the technology company, plants the seeds of its vision for the coming year. This year was no exception.
Bang on cue, Shay Banon (CTO of Elastic) spoke about how more emphasis will be laid on development across features in hot demand like frozen tier search, improving the already awesome Time Series DB, by popular demand ESQL and in general how Elastic is gearing to a Stateless future.
Curious about the atmosphere at ElasticON? Watch this short after-movie
Full-Stack Monitoring with Devoteam
Is your organization using multiple monitoring tools? Which means no single source of truth. Or are you experiencing challenges in monitoring license costs (multiple tools means multiple licenses)? We at Devoteam believe in helping you with creating a single source of the truth. Get in touch with us.