What prompt instructions can be given to reduce the chance of the LLM hallucinating, by explicitly telling it to stick to the provided information?

To reduce the likelihood of an LLM hallucinating, prompts should explicitly instruct the model to rely strictly on the information provided and avoid inventing details. Start by clearly defining the scope of the task. For example, use directives like, “Answer using only the data provided in the following context,” followed by the specific information. This sets a boundary, signaling that the model should not fill gaps with external knowledge. Additionally, include warnings against speculation, such as, “If the answer isn’t found in the context, state that the information is unavailable.” This forces the model to acknowledge limitations rather than guess. For instance, if the task is to summarize a technical document, the prompt might say, “Summarize the key points below without adding new examples or assumptions. Omit any details not explicitly mentioned.”

Next, structure the prompt to reinforce context. Provide the necessary data upfront and reference it explicitly. For example, when querying about API documentation, include the actual text of the documentation and instruct the model to base its answers solely on that text. Use phrases like, “Refer to Section 2.3 of the provided API spec to explain authentication methods,” which ties the response directly to a source. You can also ask the model to cite where in the provided data it found the answer, like, “Include the paragraph number from the context that supports your response.” This creates accountability and makes it easier to verify accuracy. For example, a prompt might say, “Using the troubleshooting guide below, list three common errors and their solutions. Indicate which page each solution is described on.”

Finally, test and refine prompts iteratively. Use known data to check if the model stays within bounds. For example, if you provide a dataset of user feedback and ask, “What are the top three complaints mentioned?” verify that the answer matches the dataset exactly. If the model invents a complaint not present, adjust the prompt with stricter instructions, like, “Do not list any issues not explicitly written in the user feedback excerpts.” Additionally, use parameters like temperature (lower values reduce randomness) to make outputs more deterministic. For ambiguous queries, such as interpreting unclear requirements, add a step where the model asks for clarification: “If any part of the context is unclear, request additional details before answering.” This reduces the risk of incorrect assumptions.