Soapbox has developed an AI site-search tool, that allows users to ask questions – using natural language – that are answered using only the material in your content library.
Auto-generated overviews are popping up everywhere, from Google’s search results to summaries of Amazon reviews. One of the big talking points among search engine optimisation (SEO) specialists has been how these overviews are affecting click through rates (CTRs).
You might assume that if an AI overview answers a user’s query, they won’t go any further. However, studies have shown that the organic CTRs actually increase slightly for sources cited in the summary results.
Our theory is that users are leaning on the summaries to learn more about the content behind the link – in effect, helping them to make more informed click-throughs, compared to the traditional list of competing results. We would love to see if this also results in a lower bounce rate from those clicks. We expect it does.
Generating accurate AI overviews
A key concern often cited with AI generated texts is about ‘hallucinations’ – when an AI output includes fabricated or untrue answers, often when it does not ‘know’ a true answer.
The key to avoiding hallucinations (eg, when using AI in your website’s search functionality) is to avoid letting the AI lean on its training data to answer a query. Instead, we need to provide some context for the answer in the prompt. We do this using a vector database to retrieve highly relevant results by finding content closely dealing with the semantic meaning of the visitor’s query.
Read more about how vector database results are found and how they vastly improve relevance
If the results are not close matches, we can assume that your website does not have the answer to the user’s question and we can simply tell the user that straight away.
Only if we receive relevant results do we go to the next step of generating a response. To do this, we would take the top three to six ‘highly relevant’ results from the vector database to put the visitor’s question to a large language model (LLM) like OpenAI. At the same time, the content of the results from your site is also passed into the prompt (along with some other guidance). The AI is instructed to only respond if it can find the answer in that text.
The answer is marked as an AI-generated response, both visually and with words. The user is prompted to click through to the content items that were used to generate that answer, encouraging the visitor to read the original material.
See it in action
With the AI system we developed for the Nuffield Trust’s website, the search page suddenly does much more than mere keyword matching. The AI tool lets you ask anything at all that relates to their work, and generates condensed answers via the LLM. Please try it for yourself, and you’ll find find it produces excellent answers while encouraging further reading by citing which sources it has used.
The search page focuses on the AI-generated answer by default, clearly separating it from the traditional search section. There is an option, however, to combine the two sets of results, as well as an option to make the generated text more of an aside, focusing more on the traditional results.
Send us a message and we can talk through options with you.