{"id":1968,"date":"2024-10-17T04:00:00","date_gmt":"2024-10-17T04:00:00","guid":{"rendered":"https:\/\/www.xome.com\/blog\/?p=1968"},"modified":"2024-11-26T23:16:33","modified_gmt":"2024-11-26T23:16:33","slug":"improving-property-search-performance-with-elasticsearch","status":"publish","type":"post","link":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/","title":{"rendered":"Improving Property Search Performance with Elasticsearch"},"content":{"rendered":"\n<p>The most important part of an application\u2019s user experience is its search response time. The longer users wait for a response to their request, the quicker they lose interest and exit the application.&nbsp;&nbsp;<\/p>\n\n\n\n<p>That\u2019s why Elasticsearch has become a go-to search and analytics engine used for building applications. At the core of <a href=\"https:\/\/www.elastic.co\/elastic-stack\" target=\"_blank\" rel=\"noreferrer noopener\">Elastic Stack<\/a>, Elasticsearch is used for indexing, searching, and analyzing large sets of data in near real-time. It is the perfect tool for real estate applications that require prompt searches across a variety of different use cases.&nbsp;<\/p>\n\n\n\n<p>At Xome\u00ae, we utilize Elasticsearch in areas of our digital properties where fast and relevant search responses are required. Even though Elasticsearch is already known for its efficiency, we follow a set of best practices to help optimize search performance and provide the best experience possible for our users.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-what-is-elasticsearch\">What is Elasticsearch?<\/h2>\n\n\n\n<p>Elasticsearch is a distributed open-source search and analytics engine built on top of <a href=\"https:\/\/lucene.apache.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">Apache Lucene<\/a>, a full-text search engine library written in Java. It was developed by Elastic as a part of their Elastic Stack, a collection of open-source tools that Elasticsearch can integrate with seamlessly.&nbsp;&nbsp;<\/p>\n\n\n\n<p>Other tools in the stack include Beats for data shippers, Logstash for data processing, and Kibana for data visualization and exploration.&nbsp;<\/p>\n\n\n\n<p>Elasticsearch is commonly used for various purposes including full-text search, monitoring and alerting, log and event data analysis, and business analytics, among many others.&nbsp;&nbsp;<\/p>\n\n\n\n<p>When used to build real estate applications, a few specific use cases include:&nbsp;&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Property search based on various criteria, such as price range and property type&nbsp;<\/li>\n\n\n\n<li>Geospatial search for properties based on their geographic location and proximity to places like schools and public transportation&nbsp;<\/li>\n\n\n\n<li>Geospatial mapping for integrating interactive maps&nbsp;<\/li>\n\n\n\n<li>Dynamic filtering for users to refine search results based on specific attributes&nbsp;<\/li>\n\n\n\n<li>Relevant property recommendations to users based on their search history, preferences, and behavior&nbsp;<\/li>\n<\/ul>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-seven-best-practices-for-improving-elasticsearch-search-performance-nbsp\">Seven best practices for improving Elasticsearch search performance&nbsp;<\/h2>\n\n\n\n<p>There are various ways to optimize the implementation of Elasticsearch when developing applications.&nbsp;&nbsp;<\/p>\n\n\n\n<p>Search speed depends on factors such as query complexity, data volume, cluster configuration, and hardware resources allocated to the search operation. Since Elasticsearch depends significantly on the filesystem cache to speed up search performance, increasing the size of that cache and using upgraded hardware are basic best practices.&nbsp;&nbsp;<\/p>\n\n\n\n<p>There are other best practices that are less obvious but just as important.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-1-search-fewer-fields-nbsp-nbsp\">1. Search fewer fields&nbsp;&nbsp;<\/h3>\n\n\n\n<p>Searching fewer fields reduces the amount of data that Elasticsearch needs to access and analyze during search operations. By limiting the search scope to only the most relevant fields, Elasticsearch can execute queries more efficiently, leading to faster search response times while reducing resource utilization.&nbsp;&nbsp;<\/p>\n\n\n\n<p>Using a <strong>copy_to <\/strong>directive is one way to improve search time.&nbsp;For example, with an index that contains properties that searches over both the property type and details of the property, the values are indexed into a single field.&nbsp;&nbsp;<\/p>\n\n\n\n<p>This field can be called something like a <strong>type_and_details<\/strong> field:&nbsp;<\/p>\n\n\n\n<p>\u201ccopy_to\u201d = \u201ctype_and_details\u201d&nbsp;<\/p>\n\n\n\n<p>When a user searches for home or auction properties, they may experience a more intuitive search with less latency.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-2-use-preference-to-optimize-cache-utilization-nbsp-nbsp\">2. Use preference to optimize cache utilization&nbsp;&nbsp;<\/h3>\n\n\n\n<p>Elasticsearch uses three different caches to help improve search performance. These include the filesystem cache, request cache, and query cache. However, these caches may not help much when multiple search requests are run in a row.&nbsp;<\/p>\n\n\n\n<p>By leveraging the preference parameter in search requests, subsequent searches can be directed to the same shard or replica where caching is more effective. This ensures that frequently accessed data and query results are cached at the shard level, increasing cache hit rates and reducing the need for redundant computations.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-3-use-term-query-for-exact-matching-and-on-keyword-fields-nbsp-nbsp\">3. Use term query for exact matching and on keyword fields&nbsp;&nbsp;<\/h3>\n\n\n\n<p>A term query is a type of query used to search for exact terms in the indexed data.&nbsp;When using the term query, Elasticsearch looks for exact matches of the search term in the inverted index without any text analysis or tokenization.&nbsp;<\/p>\n\n\n\n<p>The term query can be used to find documents based on values such as price<s>,<\/s> but should be avoided for text fields where the match query should be used instead.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-4-implement-features-like-elasticsearch-percolator-nbsp\">4. Implement features like Elasticsearch Percolator&nbsp;<\/h3>\n\n\n\n<p>Implementing other Elasticsearch features like <a href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/query-dsl-percolate-query.html\" target=\"_blank\" rel=\"noreferrer noopener\">Percolator<\/a> can help to further optimize a user\u2019s search experience. Elasticsearch Percolator is a reverse search where, when given a document, it finds all the search queries that match the document.&nbsp;&nbsp;<\/p>\n\n\n\n<p>When a user\u2019s search criteria does not return listings that interest them, they may set up alerts for those specific criteria. They will then be notified when new listings come on the market that match those criteria.&nbsp;&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-5-optimize-necessary-index-settings-nbsp\">5. Optimize necessary index settings&nbsp;<\/h3>\n\n\n\n<p>Configuring certain index settings can significantly impact indexing efficiency, such as the number of shards and replication settings.&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>As a best practice, the number of shards should be a multiple of the number of data nodes in the cluster. For example, if there are 3 data nodes, it is better to have the number of shards be a multiple of 3 (6, 9, etc.).<\/li>\n<\/ul>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>For replication settings, it is better to have at least one replica for each index, so that data is always available even if one data node goes down. The number of replicas should be less than the number of data nodes. If there are three data nodes, you should not have more than two replicas.&nbsp;&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-6-pre-index-data-nbsp\">6. Pre-index data&nbsp;<\/h3>\n\n\n\n<p>Pre-indexing data eliminates the need for real-time indexing during search requests because it involves indexing data before it is queried. In this approach, data is indexed and prepared in advance. This helps ensure that it is readily available for search operations without the need for additional indexing tasks at the time of the search request.&nbsp;&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-7-tune-for-indexing-speed-nbsp-nbsp\">7. Tune for indexing speed&nbsp;&nbsp;<\/h3>\n\n\n\n<p>Increasing indexing speed means that newly ingested data becomes available for search queries more quickly, improving overall search performance.&nbsp;<\/p>\n\n\n\n<p>There are a few practices recommended when tuning for indexing speed. Although indexing speed does not directly affect how fast searches are executed, there are indirect ways it improves search speed.&nbsp;<\/p>\n\n\n\n<p>A few ways to tune for indexing speed include:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Using bulk requests&nbsp;<\/li>\n\n\n\n<li>Using multiple workers\/threads to send data to Elasticsearch&nbsp;<\/li>\n\n\n\n<li>Unsetting or increasing the refresh interval&nbsp;<\/li>\n<\/ul>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>Improvements in indexing speed can alleviate resource bottlenecks and improve the overall stability of the Elasticsearch cluster, indirectly benefiting property search performance for application users.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-indexing-performance-vs-search-performance-nbsp\">Indexing performance vs. search performance&nbsp;<\/h2>\n\n\n\n<p>There is an inherent tradeoff between reducing indexing latency and solving for query latency. Optimizing for faster indexing may involve sacrificing query performance, as resources are prioritized for indexing tasks. On the other hand, optimizing search queries may require limiting indexing activity to avoid resource contention and ensure query responsiveness.&nbsp;<\/p>\n\n\n\n<p>Compromising between the two depends on the use case and its priorities. One use case may prioritize fresh data while another requires faster queries. For example, real-time analytics applications like Domo may prioritize data freshness, while interactive real estate search applications like Xome may prioritize query responsiveness.&nbsp;<\/p>\n\n\n\n<p>Improving Elasticsearch performance requires careful consideration of these tradeoffs.&nbsp;&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-optimizing-elasticsearch-for-low-latency-search-with-less-downtime-at-xome-nbsp\">Optimizing Elasticsearch for low latency search with less downtime at Xome&nbsp;<\/h2>\n\n\n\n<p>High performance property searches are essential when building powerful real estate applications. This continues to be a high priority of the Xome technical teams as we try to solve users\u2019 problems and promptly present them with the best-fitting properties that fit their search.&nbsp;&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Discover how Elasticsearch has improved Xome&#8217;s property search capabilities for fast and relevant search responses.<\/p>\n","protected":false},"author":2,"featured_media":1971,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[107],"tags":[157,158,160,118],"class_list":["post-1968","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-real-estate-tech","tag-elasticsearch","tag-property-search","tag-tech-innovation","tag-technology"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.3 (Yoast SEO v27.6) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Improving Property Search Performance with Elasticsearch - Xome | Blog<\/title>\n<meta name=\"description\" content=\"Discover how Elasticsearch has improved Xome&#039;s property search capabilities for fast and relevant search responses.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Improving Property Search Performance with Elasticsearch\" \/>\n<meta property=\"og:description\" content=\"Discover how Elasticsearch has improved Xome&#039;s property search capabilities for fast and relevant search responses.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/\" \/>\n<meta property=\"og:site_name\" content=\"Xome | Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/XomeRealEstate\" \/>\n<meta property=\"article:published_time\" content=\"2024-10-17T04:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-11-26T23:16:33+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/cloud-prod.xome.com\/blog\/wp-content\/uploads\/2024\/10\/Blog-Featured-Images.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Xome\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@XomeRealEstate\" \/>\n<meta name=\"twitter:site\" content=\"@XomeRealEstate\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Xome\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Improving Property Search Performance with Elasticsearch - Xome | Blog","description":"Discover how Elasticsearch has improved Xome's property search capabilities for fast and relevant search responses.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/","og_locale":"en_US","og_type":"article","og_title":"Improving Property Search Performance with Elasticsearch","og_description":"Discover how Elasticsearch has improved Xome's property search capabilities for fast and relevant search responses.","og_url":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/","og_site_name":"Xome | Blog","article_publisher":"https:\/\/www.facebook.com\/XomeRealEstate","article_published_time":"2024-10-17T04:00:00+00:00","article_modified_time":"2024-11-26T23:16:33+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/cloud-prod.xome.com\/blog\/wp-content\/uploads\/2024\/10\/Blog-Featured-Images.png","type":"image\/png"}],"author":"Xome","twitter_card":"summary_large_image","twitter_creator":"@XomeRealEstate","twitter_site":"@XomeRealEstate","twitter_misc":{"Written by":"Xome","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/#article","isPartOf":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/"},"author":{"name":"Xome","@id":"https:\/\/cloud-prod.xome.com\/blog\/#\/schema\/person\/9ba9f5a5815a35c244a30044cd1fb392"},"headline":"Improving Property Search Performance with Elasticsearch","datePublished":"2024-10-17T04:00:00+00:00","dateModified":"2024-11-26T23:16:33+00:00","mainEntityOfPage":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/"},"wordCount":1292,"publisher":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/#organization"},"image":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/#primaryimage"},"thumbnailUrl":"https:\/\/cloud-prod.xome.com\/blog\/wp-content\/uploads\/2024\/10\/Blog-Featured-Images.png","keywords":["Elasticsearch","property search","tech innovation","technology"],"articleSection":["Real Estate Tech"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/","url":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/","name":"Improving Property Search Performance with Elasticsearch - Xome | Blog","isPartOf":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/#primaryimage"},"image":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/#primaryimage"},"thumbnailUrl":"https:\/\/cloud-prod.xome.com\/blog\/wp-content\/uploads\/2024\/10\/Blog-Featured-Images.png","datePublished":"2024-10-17T04:00:00+00:00","dateModified":"2024-11-26T23:16:33+00:00","description":"Discover how Elasticsearch has improved Xome's property search capabilities for fast and relevant search responses.","breadcrumb":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/#primaryimage","url":"https:\/\/cloud-prod.xome.com\/blog\/wp-content\/uploads\/2024\/10\/Blog-Featured-Images.png","contentUrl":"https:\/\/cloud-prod.xome.com\/blog\/wp-content\/uploads\/2024\/10\/Blog-Featured-Images.png","width":1200,"height":628},{"@type":"BreadcrumbList","@id":"https:\/\/cloud-prod.xome.com\/blog\/improving-property-search-performance-with-elasticsearch\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog","item":"https:\/\/cloud-prod.xome.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Real Estate Tech","item":"https:\/\/cloud-prod.xome.com\/blog\/category\/real-estate-tech\/"},{"@type":"ListItem","position":3,"name":"Improving Property Search Performance with Elasticsearch"}]},{"@type":"WebSite","@id":"https:\/\/cloud-prod.xome.com\/blog\/#website","url":"https:\/\/cloud-prod.xome.com\/blog\/","name":"Xome | Blog","description":"","publisher":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/cloud-prod.xome.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/cloud-prod.xome.com\/blog\/#organization","name":"Xome | Blog","url":"https:\/\/cloud-prod.xome.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/cloud-prod.xome.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/cloud-prod.xome.com\/blog\/wp-content\/uploads\/2022\/10\/xome-logo-blaze.svg","contentUrl":"https:\/\/cloud-prod.xome.com\/blog\/wp-content\/uploads\/2022\/10\/xome-logo-blaze.svg","width":64,"height":30,"caption":"Xome | Blog"},"image":{"@id":"https:\/\/cloud-prod.xome.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/XomeRealEstate","https:\/\/x.com\/XomeRealEstate","https:\/\/www.instagram.com\/xome_com","https:\/\/www.linkedin.com\/company\/xome"]},{"@type":"Person","@id":"https:\/\/cloud-prod.xome.com\/blog\/#\/schema\/person\/9ba9f5a5815a35c244a30044cd1fb392","name":"Xome","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/6a23391c131d2bcbe1b80b2697e5f17546685a7b6e8f4f12dc0324ab4d535802?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/6a23391c131d2bcbe1b80b2697e5f17546685a7b6e8f4f12dc0324ab4d535802?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/6a23391c131d2bcbe1b80b2697e5f17546685a7b6e8f4f12dc0324ab4d535802?s=96&d=mm&r=g","caption":"Xome"},"url":"https:\/\/cloud-prod.xome.com\/blog\/author\/jfritsche\/"}]}},"_links":{"self":[{"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/posts\/1968","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/comments?post=1968"}],"version-history":[{"count":5,"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/posts\/1968\/revisions"}],"predecessor-version":[{"id":1974,"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/posts\/1968\/revisions\/1974"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/media\/1971"}],"wp:attachment":[{"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/media?parent=1968"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/categories?post=1968"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloud-prod.xome.com\/blog\/wp-json\/wp\/v2\/tags?post=1968"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}