When a chatbot generates an answer based on our prompt, the text may contain facts or claims that sound convincing but are not correct. This is called hallucinations and is due to the models generating text based on statistical probabilities in their training data, not verified knowledge. Hallucinations are therefore not a design flaw but an inherent property of the technology.
An AI can hallucinate to varying degrees based on how it is built (see for example hyperparameters such as top-p and temperature) but remember that there is always a risk of errors as long as a Large Languade Model (LLM) is involved. If you use chatbots for trying to find information or sources, you often encounter hallucinations in the form of made-up references. Always double check that the sources provided by a chatbot actually exist and read the original text before referring to them in your own texts!
There are AI tools that are specifically designed for information retrieval, such as Semantic Scholar, Scopus AI and Perplexity. The functionality of AI search services vary depending on whether you have created an account and paid subscription fees. We recommend using the AI tools available in the library's database list. AI search services can be useful when you are orienting yourself in a new topic or expanding your search beyond traditional scholarly databases. Information retrieval tools that include AI are not suitable for more formal systematic literature reviews because the generated hit list is not replicable.
Try it yourself! Try asking a chatbot questions about something you know a lot about - then you will notice if the chatbot generates incorrect, ambiguous or inadequate answers.
If a chatbot has connection to the internet (which was not always the case), it can be asked to generate information that includes URLs. The sources are generated in the same way as the rest of the answer – based on statistics and probabilities. Since this process is not deterministic, the sources will vary from time to time and the tool can still hallucinate incorrect references.
There are AI tools that use Retrieval-Augmented Generation (RAG). These information retrieval tools have a language model at their core but are also connected to a knowledge base that is used to answer the question. The idea is that the generated answers must be somewhere within the knowledge base. If an AI tool includes a RAG, there is therefore less risk of hallucinations.
Figure 4. The difference between chatbots based on generative AI or AI-based search functions. (Svanberg & Seo, 2024) CC BY 4.0 Translated.
In some chatbots and AI search services, you can choose to search using a function called “deep search” or “deep research.” The name may create high expectations, but the “search” usually works in the same way as a regular prompt to a language model. There are advanced techniques that can be used, for example Inference-Time Compute, which means that the AI tool takes longer to generate your answer. It can also result in more exhaustive and higher-quality responses. Hallucinations may still occur.
The question or instruction you submit to an AI tool is called a prompt. In order for the tools to generate helpful and effective responses, you often need to craft a good prompt. This can be done by experimenting with different versions of the prompt, for example specifying how many words you want the response to contain or that the language whould have a certain tone of voice. The process of formulating and designing a good prompt is called prompt engineering. Using chatbots may feel intuitive because they are built to handle natural language and have interfaces that are familiar to many. However, to get the most out of an AI tool, some form of prompt engineering is usually required.
Prompt engineering is a very new field where many frameworks and tips are published and evaluated all the time. One of the first to appear is a framework called CLEAR, where the acronym stands for concise, logical, explicit, adaptive and reflective.
Figure 5. The CLEAR Framework. (Lo, 2023).
There are suggestions and tips on how to structure your prompt for best results, including few-shot prompting, chain-of-thought prompting and prompt chaining. Learn more techniques: