This blog post concerns the intermittent issue with Sitecore Solr search results in Solr Cloud. Depending on the application and architecture implementation, root causes can be different. But I would like to share the problem we faced and its fix so it may help someone facing the same problem and can apply the same fix.
Challenge:
We created a sophisticated search results page that leverages SXA search components and filters. Additionally, we integrated custom code in various website sections to retrieve data from Solr indexes using Sitecore Content Search APIs. Everything functioned seamlessly in our local environment and even in higher-level environments. However, after some days, we encountered intermittent issues retrieving data from the indexes in a higher environment.
Intermittent Issues with Sitecore Solr Search Results Cases
Intermittent issues arose, meaning that when loading a page that relied on indexed data, it would occasionally fetch incomplete or no data. Refreshing the page would sometimes resolve the issue, providing access to all the data. In essence, the results were inconsistent.
Furthermore, we implemented a custom item resolver for Bucket item URLs, in which we indexed user-friendly URLs for bucket-able items as part of a computed field. While URL resolution worked at times, it also intermittently resulted in 404 errors.
In a nutshell, any feature dependent on the Solr search does not work consistently with all the requests.
Solution:
We did not discover any Sitecore error logs specific to features or indexes. Even after attempting to rebuild the indexes, these issues were not resolved. We contacted Sitecore Support to report the problem and learned that the replicas within the Solr cloud were not synchronized. This implies that the document count for a given index, such as ‘sitecore_web_index,’ differs across each replica node.
Confirm the Document Counts
To confirm the issue of ‘document counts not being in sync among replica nodes,’ we logged in to the Solr dashboard of the affected environment. We checked the Nodes tab under the Cloud tab on the left, as shown below.
Solr Cloud Nodes
As shown below, the document count is not in sync for the sitecore_web_index. sitecore_web_index_s1r1 and sitecore_web_index_s1r2 has 15.3k docs whereas sitecore_web_index_s1r4 has 12.9k docs.
The document count is not in sync for the sitecore_web_index
Click on more links to access other indexes and verify the document counts for the relevant indexes. These counts are not specific and are displayed in thousands. If the counts appear to be the same, check the specific count. To determine the specific count, check the steps outlined in the ‘Self-Diagnosis’ section of the Identical Queries Produce Inconsistent Results – SearchStax Docs.
Request Rolling Restart
Once we confirmed the issue, we proceeded with the solution to sync the document count across all replicas. This required a rolling restart of the Solr cluster. SearchStax Support has suggested the rolling restart to the Sitecore support team. We requested the Sitecore support team to perform the necessary rolling restart for the concerned environment, as they have the required access for this action.
After Rolling Restart
Following the rolling restart, we verified that the document counts were in sync, and the intermittent search issue was successfully resolved!
Please consider checking for the exact resolution in your case, if you encounter a similar situation. If the website is new, it is advisable to perform this document count check for all the indexes. If not in sync, it is good to have a rolling restart at least once before the UAT phase or going live.
Also, Good to check:
Computed field is getting updated in solr index only when we fully rebuild reindex. Only one Sitecore instance within the environment cluster must define an indexing role, meaning a single Sitecore instance actively handles indexing. When multiple servers assume indexing roles, it leads to issues within the indexes, such as potential clearing of computed field values or updates occurring only after a complete index rebuild.
Performance optimizations when using Solr
Special thanks to our dedicated colleagues for their invaluable support during this investigation, and thanks to the Sitecore Support Team and SearchStax Support for their fantastic assistance.
Hope this helps. Happy Sitecore searching!
