Search debugging tips

Once Enterprise Search is up and running, these tips may be useful in debugging and investigating day to day issues, or when developing new features.

Cleanup after versioning

When you version and activate a new index, be sure to delete the old index once you’re certain the new index is healthy and handling queries.

If the old version is not deleted, it will continue to be updated and indexed as content is created and modified, and this may impact indexing performance. You can use the health check CLI command to determine if multiple indexes exist.

You can delete the old index by referencing the version, or by relative reference (i.e., previous).

$ wp vip-search index-versions delete post previous

After imports, mass CLI changes, etc

Consider versioning the site’s index, this is the best way to ensure everything is correctly indexed.

Elasticsearch’s continual indexing is dependent on built-in WordPress event action hooks. Normally, the standard WordPress editing process uses these, and every change made by an editor to a post or page should be quickly reflected in the Elasticsearch index.

The primary exception to this is in custom code, or plugins, that use direct database queries or nonstandard ways of modifying the database. This is most likely to be evident in CLI commands that do bulk updates, or which disable these hooks by defining WP_IMPORTING to be true.

Often these issues are not immediately obvious.

If pages or posts (or their attributes such as meta or comments) are created, modified, or deleted with custom SQL queries (even if they use WPDB) that bypass these hooks, then the index state can become out of sync with the database. The only way to recover from this is to version and re-index, and the only long-term solution (to avoid having to version frequently) is to ensure only standard WordPress functions (for example, wp_update_post) are used to update posts, terms, post meta, and users.

If a plugin uses custom post types (or custom tables) to manage its data, and that plugin also uses direct database queries to add or modify the data, then it may be easiest to simply avoid using Elasticsearch to query the data, by disabling the custom post type.

Items in search results

If you feel something is not appearing in the correct place, or at all, in a search, you can use the Search Dev Tools to review the response data, and the CLI to do some basic checks.

Also, opening the particular post or page in the editor and clicking “Update” sometimes resolves indexing issues – an update will cause the post to be re-indexed.

If the item appears in the response JSON, but not in the page, verify that the item has the correct status and has not been filtered out of the results because it’s a draft or some other status.

If the item appears in the wrong position, review the “score” details in the response, these indicate relative rankings. It’s possible other items have a higher score because of greater weighting given to a particular attribute such as the title or date, or a higher frequency of keywords.

Items missing from search results

Check the post’s status and type, and ensure it’s in the index. For example, using CLI to check if post ID 123 is in the index:

$ wp vip-search documents get post 123

If it is in the index, try to search for the item by the exact title. If it can’t be found, check the response being returned in Search Dev Tools — if there are no results, review the query parameters and confirm that all of the attributes are being indexed (e.g. included in the allow lists). If a query includes all post types, and other posts are ranked higher, you may need to modify the query to only use that custom post type, or exclude generic posts and pages.

Search query is not being offloaded to Enterprise Search

There are a few reasons we recommend exploring to see why your search query may not be properly offloaded:

• Refer to the Search Dev Tools dashboard and verify if search queries are being handled by Enterprise Search.
• Review the steps in the setting up guide and verify that all constants are configured correctly.
• Ensure the search feature is activated. If not, use the activate-feature command for it:

$ wp vip-search activate-feature search

• There is a technical limitation of returning only the first 10000 documents during a search query. If the query is trying to retrieve more documents, it will not offload to ElasticSearch and use a regular DB query — pagination does not alleviate this limitation.
  E.g. Even if we would be searching only for 16 documents, if it is after the first 10000, the query will not run against ElasticSearch.

Error: index_not_found_exception

If you look in Search Dev Tools and see an index_not_found_exception error:

We can confirm that there is no index by CLI:

$ wp vip-search health validate-counts

Skipping validation of 'post' index as it doesn't exist.