Prompt Engineering with Llama 2

Prompting Techniques with Llama 2

Get the Most Out of Llama 2 with Prompts

Explicit Instructions

Detailed, explicit instructions produce better results than open-ended prompts:

Example

“Describe quantum physics in one short sentence of no more than 12 words.”

You can think about giving explicit instructions as using rules and restrictions to how Llama 2 responds to your prompt.

Stylization

• Explain this to me like a topic on a children's educational network show teaching elementary students.

• I'm a software engineer using large language models for summarization. Summarize the following text in under 250 words:

• Give your answer like an old timey private investigator hunting down a case step by step.

Formatting

• Use bullet points.

• Return as a JSON object.

• Use less technical terms and help me apply it in my work in communications

Restrictions

• Only use academic papers.

• Never give sources older than 2020.

• If you don't know the answer, say that you don't know.

Here's an example of giving explicit instructions to give more specific results by limiting the responses to recently created sources:

• Explain the latest advances in large language models to me.

• Explain the latest advances in large language models to me. Always cite your sources. Never cite sources older than 2020.

Prompting using Zero- and Few-Shot Learning

A shot is an example or demonstration of what type of prompt and response you expect from a large language model. This term originates from training computer vision models on photographs, where one shot was one example or instance that the model used to classify an image.

Zero-Shot Prompting

Large language models like Llama 2 are unique because they are capable of following instructions and producing responses without having previously seen an example of a task.

Prompting without examples is called "zero-shot prompting".

Few-Shot Prompting

Adding specific examples of your desired output generally results in more accurate, consistent output. This technique is called "few-shot prompting".

Role Prompting

Llama 2 will often give more consistent responses when given a role (Kong et al. (2023)). Roles give context to the Generative AI on what type of answers are desired.

Let's use Llama 2 to create a more focused, technical response for a question around the pros and cons of using PyTorch:

"Explain the pros and cons of using PyTorch."

More likely to explain the pros and cons of PyTorch covers general areas like documentation, the PyTorch community, and mentions a steep learning curve.

“Your role is a machine learning expert who gives highly technical advice to senior engineers who work with complicated datasets. Explain the pros and cons of using PyTorch."

Often results in more technical benefits and drawbacks that provide more technical details on how model layers.

Chain-of-Thought

Simply adding a phrase encouraging step-by-step thinking "significantly improves the ability of large language models to perform complex reasoning" (Wei et al. (2022)). This technique is called "CoT" or "Chain-of-Thought" prompting:

“Who lived longer Elvis Presley or Mozart?"

Sometime gives incorrect answer of "Mozart".

"Who lived longer Elvis Presley or Mozart? Let's think through this carefully, step by step."

Gives the correct answer "Elvis".

Self-Consistency

Generative AIs are probablistic, so even with Chain-of-Thought, a single generation might produce incorrect results. Self-Consistency (Wang et al. (2022)) introduces enhanced accuracy by selecting the most frequent answer from multiple generations (at the cost of higher compute).

Retrieval-Augmented Generation

You'll probably want to use factual knowledge in your application. You can extract common facts from today's large models out-of-the-box (i.e. using just the model weights).

However, more specific facts, or private information, cannot be reliably retrieved. The model will either declare it does not know or hallucinate an incorrect answer.

Retrieval-Augmented Generation, or RAG, describes the practice of including information in the prompt you've retrived from an external database (Lewis et al. (2020)).

It's an effective way to incorporate facts into your Generative AI application and is more affordable than fine-tuning which may be costly and negatively impact the foundational model's capabilities. This could be as simple as a lookup table or as sophisticated as a vector database) containing all of your company's knowledge.

Program-Aided Language Models

Generative AIs, by nature, aren't great at performing calculations.

Gao et al. (2022) introduced the concept of "Program-aided Language Models" (PAL). While Generative AIs are bad at arithmetic, they're great for code generation. PAL leverages this fact by instructing the Generative AI to write code to solve calculation tasks.

Limiting Extraneous Tokens

A common struggle is getting output without extraneous tokens (ex. "Sure! Here's more information on...").

Improvement can be made by combining a role, rules and restrictions, explicit instructions, and an example.

Authors

Edited by Dalton Flanagan with contributions from Mohsen Agsen, Bryce Bortree, Ricardo Juan Palma Duran, Kaolin Fire, Thomas Scialom.