Introduction

As a software developer, I see LLM model usage in two perspectives:

  • Using it as a tool powered by LLM models (for example, AI coding editors, various chatbots)
  • Using it as a component in software applications to power business-centric features (for example, building a chatbot for your customer-facing application, reviewing sentiment of customer reviews in an ecommerce application)

This post focuses on the second perspective: building applications that leverage LLM capabilities.

LLM models understand and interact in natural language like English. Simply put, they are brains trained on vast amounts of information that process input text and generate responses based on their training data, training methodology, and user input.

User input text, typically a query to an LLM, is referred to as a Prompt.

Major AI companies perform compute intensive LLM model training. As developers, we have control over the prompting aspect to get efficient responses from these models. Effective prompting can significantly differentiate results from the same models.

With the rise of AI and LLMs, prompt engineering has become a crucial skill for developers, businesses, and anyone looking to leverage the power of these models.

Over the past couple of years, better prompting techniques have been discovered and refined.

Write Clear and Specific Instructions

Clarity in instructions with emphasis on important points can make models respond in a particular way.

For example:

Vague prompt:

Reply to customer email with a resolution to their issue.
Customer email: {customer_email_content}

Better prompt:

Reply to following customer email in a professional and helpful tone.

Identify their issue and respond with possible solutions and timelines.
Include follow-up actions in the email, if any.
Do not include any information about our competitors in the email.

Customer email: {customer_email_content}

Set Persona for a Prompting Session

Like humans, LLMs can be assigned an Identity.

For example, a teacher has the qualities of responsible teaching, listening to students, asking questions, and an intent to make sure students understand the concept.

An identity of Sachin Tendulkar will have qualities of the best sportsman, team spirit, and one who knows cricket inside out.

An identity avoids describing a role in detail, but it gives a general idea to LLM about the qualities expected in that role.

For example, to respond to a customer email: You are a customer representative sets the identity as “Customer Representative”. LLM already knows the qualities expected in a customer representative.

You are a customer representative.

Identify the issue and respond with possible solutions and timelines.
...

Identity is also referred to as a Persona or Role.

To differentiate between identity information from regular interaction with LLM, it’s logically divided into System prompt and User prompt.

System Prompt

Prompt session with an LLM might not end in a single prompt. You might want to have multiple back-and-forth communications with an LLM. During this session, you want the LLM to remember its identity during the session.

For example,

You are Sachin Tendulkar, a legendary cricket player. Currently based in Mumbai.
I am interviewing you for a magazine whose audience is primarily high school students.

Here, it remembers that you are interviewing Sachin Tendulkar for a kids’ magazine.

In System prompt, focus is on “Who” part. It sets the identity of the LLM for the session.

Setting a system prompt for a session is a one-time task; further communication is based on user prompts.

User Prompt

Prompts used for regular interaction with LLMs are referred to as User Prompts. Here, we ask a query but don’t need to remember it for the whole session.

For example,

Query:

Any message for our readers?

Response:

(I pause, looking directly at the camera – as if speaking directly to the young readers)

Okay, listen up! This is something I really want to get across.

(I take a deep breath)

You guys are the future of cricket, the future of sport! You've grown up with incredible players, amazing technology, and a global game like never before. That's awesome. But don't just watch – participate.

As you can notice, the LLM remembers that it is Sachin Tendulkar and is addressing a young audience.

In user prompts, focus is on “What” part. It is used to ask queries to the LLM.

Frameworks like LangChain and Spring AI support setting System and User prompts while interacting with LLMs.

LLMs like Examples

Humans learn better with examples. Examples clarify many doubts which might not be clear with plain theory. LLMs too understand better with examples.

Prompts with no examples are referred to as Zero-Shot Prompting.

For example:

Identify the sentiment of the following:
"Item received in good shape but packaging was bad!"

Response can be following depending on the LLM being used, but it can vary significantly: Sentiment: Neutral/Negative

Zero-shot prompting is preferred for simple chat interactions, but for more complex tasks, it can lead to inaccurate responses.

Prompts with one example are termed One-Shot Prompting. These have more accuracy as compared to Zero-Shot.

For example:

Example:
"Item was received as expected but with small delay"
Sentiment: Positive

Identify the sentiment of the following:
"Item received in good shape but packaging was bad!"

In this case, the response can be: Sentiment: Neutral/Positive

The more examples covering different scenarios, the better the LLM understands how to perform on user prompts. Prompts with more examples are termed Few-Shot Prompting.

Break Complex Task into Subtasks

LLMs perform better when a complex task is split into smaller subtasks. Each subtask with clear instructions allows the LLM to perform better.

For example, instead of saying:

Get me today and tomorrow's weather report for my current location.

we can break it down it into subtasks with more details:

- Ask for the current user location, if not clear
- Get the current time
- Use the weather API to fetch the weather report for the current time and current time + one day
- Consolidate the response into a single report

We have split the vague prompt into multiple subtasks with additional details. Some subtasks may require the LLM to interact with external APIs, search the web, etc. Although ideally an LLM is isolated from external knowledge, techniques such as MCP and code execution allow it to gather information from the outside world.

MCP (Model Context Protocol) is a powerful technique which connects isolated LLMs to external world information and tools. This will be covered in a future post.

Please share your thoughts and feedback in the comments section below.