> ## Documentation Index
> Fetch the complete documentation index at: https://prompt.university/llms.txt
> Use this file to discover all available pages before exploring further.

# Lesson 3.5: Tool Integration & RAG

> Connect LLMs to external knowledge sources and tools through RAG and structured tool use

<Info>
  **Duration:** 75 minutes
</Info>

## Introduction

LLMs are powerful, but they have limitations: outdated knowledge, inability to access real-time data, and no way to perform actions. The solution? Connect them to external tools and knowledge sources. In this lesson, you'll learn Retrieval-Augmented Generation (RAG) and tool integration patterns that transform LLMs from isolated systems into connected, capable assistants.

## Why External Integration Matters

<CardGroup cols={2}>
  <Card title="Current Information" icon="clock">
    Access real-time data beyond the model's training cutoff
  </Card>

  <Card title="Factual Grounding" icon="anchor">
    Reduce hallucinations by grounding in verified sources
  </Card>

  <Card title="Specialized Knowledge" icon="book">
    Tap into domain-specific databases and documents
  </Card>

  <Card title="Action Capability" icon="bolt">
    Perform calculations, API calls, and real-world actions
  </Card>
</CardGroup>

## Retrieval-Augmented Generation (RAG)

RAG combines retrieval of relevant information with generation, dramatically improving accuracy.

### What is RAG?

<Steps>
  <Step title="User Query">
    User asks a question
  </Step>

  <Step title="Retrieve Context">
    System searches knowledge base for relevant information
  </Step>

  <Step title="Augment Prompt">
    Retrieved information is added to the prompt as context
  </Step>

  <Step title="Generate Answer">
    LLM generates answer based on provided context
  </Step>
</Steps>

### Basic RAG Pattern

```
Context: [RETRIEVED DOCUMENTS/INFORMATION]

Question: [USER QUESTION]

Instructions: Answer the question based ONLY on the provided context. 
If the answer is not in the context, explicitly state that you don't have 
enough information to answer.

Answer:
```

### Example: Company Knowledge Base

```
Context (Retrieved from company docs):
"Our return policy allows returns within 30 days of purchase. Items must be unused 
and in original packaging. Refunds are processed within 5-7 business days after we 
receive the return. Shipping costs are non-refundable unless the item was defective. 
To initiate a return, contact customer service at returns@company.com with your order number."

Question: How long do I have to return an item?

Instructions: Answer based ONLY on the provided context.

Answer: According to our return policy, you have 30 days from the date of purchase 
to return an item. The item must be unused and in its original packaging.
```

<Note>
  **Key Benefit:** The answer is grounded in actual company policy, not the model's general knowledge, eliminating hallucination risk.
</Note>

### When to Use RAG

<Accordion title="Knowledge-Intensive Tasks">
  **Use when:** Answers require specific, factual information

  **Example:** Customer support, technical documentation, research assistance

  **Why:** Ensures accuracy and reduces hallucinations
</Accordion>

<Accordion title="Dynamic Information">
  **Use when:** Information changes frequently

  **Example:** News, stock prices, weather, product catalogs

  **Why:** Provides current information beyond training data
</Accordion>

<Accordion title="Private/Proprietary Data">
  **Use when:** Need to access company-specific information

  **Example:** Internal policies, customer data, project documentation

  **Why:** Keeps sensitive data secure while enabling AI assistance
</Accordion>

<Accordion title="Citation Requirements">
  **Use when:** Answers must be traceable to sources

  **Example:** Legal research, academic work, compliance

  **Why:** Provides verifiable source attribution
</Accordion>

## RAG Prompt Patterns

### Pattern 1: Simple RAG

```
Retrieved Context:
[DOCUMENT 1]
[DOCUMENT 2]
[DOCUMENT 3]

Question: [USER QUESTION]

Answer based on the context above:
```

### Pattern 2: RAG with Source Citation

```
Context Documents:
[Doc 1]: [CONTENT]
[Doc 2]: [CONTENT]
[Doc 3]: [CONTENT]

Question: [USER QUESTION]

Instructions:
1. Answer based on the provided documents
2. Cite which document(s) support your answer
3. If information is not in the documents, say so

Answer:
[RESPONSE]

Sources: [Doc 1], [Doc 2]
```

### Pattern 3: RAG with Confidence

```
Retrieved Information:
[CONTEXT]

Question: [USER QUESTION]

Provide:
1. Your answer based on the context
2. Confidence level (High/Medium/Low)
3. What information would increase confidence

Answer:
Confidence:
Additional information needed:
```

### Example: Technical Documentation RAG

```
Retrieved Context:
[Doc 1 - API Authentication]: "All API requests require an API key in the header. 
Format: Authorization: Bearer YOUR_API_KEY. Keys can be generated in the dashboard 
under Settings > API Keys."

[Doc 2 - Rate Limits]: "Free tier: 100 requests/hour. Pro tier: 1000 requests/hour. 
Enterprise: Custom limits. Rate limit headers are included in all responses."

[Doc 3 - Error Codes]: "401: Invalid or missing API key. 429: Rate limit exceeded. 
500: Server error. Contact support for 500 errors."

Question: Why am I getting a 401 error when calling your API?

Answer: Based on the documentation, a 401 error indicates an invalid or missing API key. 
To resolve this:

1. Verify you're including the API key in the request header
2. Check the format: Authorization: Bearer YOUR_API_KEY
3. Ensure your API key is valid (generate a new one in Settings > API Keys if needed)

Source: [Doc 1 - API Authentication], [Doc 3 - Error Codes]
```

## External Tool Use

Enable LLMs to use calculators, APIs, databases, and other tools.

### Tool Use Pattern

```
Available Tools:
- calculator: Perform mathematical calculations
- search: Search the web for current information
- database: Query internal database
- api_call: Make API requests to external services

Task: [USER REQUEST]

Step 1: Determine which tool(s) are needed
Required tools: [LIST]

Step 2: Use tools
Tool: [TOOL NAME]
Input: [PARAMETERS]
Output: [RESULT]

Step 3: Generate response using tool outputs
Answer: [RESPONSE INCORPORATING TOOL RESULTS]
```

### Example: Calculator Integration

```
Available Tools:
- calculator(expression): Evaluates mathematical expressions

Question: If I invest $10,000 at 7% annual interest compounded monthly for 5 years, 
how much will I have?

Step 1: Identify calculation needed
Formula: A = P(1 + r/n)^(nt)
Where: P=$10,000, r=0.07, n=12, t=5

Step 2: Use calculator
Tool: calculator
Input: 10000 * (1 + 0.07/12)^(12*5)
Output: 14,176.25

Step 3: Generate response
Answer: After 5 years, your investment will grow to $14,176.25. This represents 
a total gain of $4,176.25 (41.76% return) from your initial $10,000 investment.
```

### Example: API Integration

```
Available Tools:
- weather_api(location): Get current weather data
- forecast_api(location, days): Get weather forecast

Question: Should I bring an umbrella to Seattle tomorrow?

Step 1: Determine needed information
Need: Tomorrow's weather forecast for Seattle

Step 2: Use weather API
Tool: forecast_api
Input: location="Seattle, WA", days=1
Output: {
  "date": "2024-01-16",
  "conditions": "Rain",
  "precipitation_chance": 85%,
  "temperature": "48°F"
}

Step 3: Generate recommendation
Answer: Yes, you should definitely bring an umbrella to Seattle tomorrow. 
The forecast shows rain with an 85% chance of precipitation and temperatures 
around 48°F. Consider bringing a light jacket as well.
```

## API Integration Patterns

### Structured Tool Calls

```
System: You have access to these tools:

1. get_stock_price(symbol: str) -> float
   Returns current stock price

2. get_company_info(symbol: str) -> dict
   Returns company information

3. calculate_percentage_change(old: float, new: float) -> float
   Calculates percentage change

User Query: How much has Apple stock changed since yesterday?

Assistant Response:
I'll need to use multiple tools to answer this:

Tool Call 1:
Function: get_stock_price
Parameters: {"symbol": "AAPL"}
Result: 185.50

Tool Call 2:
Function: get_stock_price
Parameters: {"symbol": "AAPL", "date": "yesterday"}
Result: 182.30

Tool Call 3:
Function: calculate_percentage_change
Parameters: {"old": 182.30, "new": 185.50}
Result: 1.76

Answer: Apple (AAPL) stock has increased by $3.20 (1.76%) since yesterday, 
currently trading at $185.50.
```

### Error Handling in Tool Use

```
Available Tools:
- database_query(sql: str) -> results

Question: How many customers do we have in California?

Attempt 1:
Tool: database_query
Input: "SELECT COUNT(*) FROM customers WHERE state = 'CA'"
Output: ERROR - Table 'customers' not found

Analysis: Table name might be different

Attempt 2:
Tool: database_query
Input: "SHOW TABLES"
Output: ['users', 'orders', 'user_profiles']

Analysis: Customer data likely in 'users' or 'user_profiles'

Attempt 3:
Tool: database_query
Input: "SELECT COUNT(*) FROM users WHERE state = 'CA'"
Output: 1,247

Answer: We have 1,247 customers in California.
```

## Knowledge Grounding

Prevent hallucinations by grounding responses in verified sources.

### Grounding Pattern

```
Knowledge Base: [VERIFIED INFORMATION]

Question: [USER QUESTION]

Rules:
1. Answer ONLY using information from the knowledge base
2. If information is not available, say "I don't have information about that"
3. Never make up or infer information not explicitly stated
4. Cite specific sources when possible

Answer:
```

### Example: Medical Information

```
Knowledge Base (from verified medical sources):
"Aspirin is a nonsteroidal anti-inflammatory drug (NSAID) used to reduce pain, 
fever, and inflammation. Common dosage for adults is 325-650mg every 4-6 hours. 
Maximum daily dose: 4000mg. Side effects may include stomach upset, heartburn, 
and increased bleeding risk. Consult a doctor before use if you have bleeding 
disorders, stomach ulcers, or are taking blood thinners."

Question: Can I take aspirin for my headache?

Grounded Answer: Based on the medical information available, aspirin can be used 
to reduce pain, including headaches. The typical adult dosage is 325-650mg every 
4-6 hours, with a maximum of 4000mg per day.

However, you should consult a doctor before taking aspirin if you have:
- Bleeding disorders
- Stomach ulcers
- Are taking blood thinners

Possible side effects include stomach upset, heartburn, and increased bleeding risk.

Note: This is general information. For personalized medical advice, please consult 
your healthcare provider.
```

### Preventing Hallucination

```
Context: [ACTUAL INFORMATION]

Question: [QUESTION THAT MIGHT TEMPT HALLUCINATION]

Instructions:
- Answer ONLY if the information is in the context
- If unsure, say "I don't have enough information"
- Never fill in gaps with assumptions
- Be explicit about limitations

Example:

Context: "Our product was launched in 2020. It has 50,000 active users."

Question: How many users did you have in 2021?

GOOD Answer: "I don't have information about our user count in 2021. The available 
data shows we have 50,000 active users currently, and the product was launched in 2020."

BAD Answer: "We had approximately 25,000 users in 2021." ❌ (Hallucinated)
```

## Citing Sources

### Citation Pattern

```
Sources:
[Source 1]: [CONTENT]
[Source 2]: [CONTENT]
[Source 3]: [CONTENT]

Question: [QUESTION]

Answer with inline citations:
[ANSWER with [Source X] citations]

References:
[Source 1]: [FULL CITATION]
[Source 2]: [FULL CITATION]
```

### Example: Research Assistant

```
Sources:
[1]: "A 2023 study in Nature found that regular exercise reduces cognitive decline 
by 30% in adults over 65." (Nature, Vol 615, 2023)

[2]: "The WHO recommends 150 minutes of moderate aerobic activity per week for adults." 
(WHO Physical Activity Guidelines, 2020)

[3]: "Research shows that combining aerobic and strength training provides the greatest 
cognitive benefits." (Journal of Aging Research, 2022)

Question: How does exercise affect cognitive health in older adults?

Answer: Regular exercise has significant benefits for cognitive health in older adults. 
Research shows that it can reduce cognitive decline by 30% in adults over 65 [1]. 
The World Health Organization recommends at least 150 minutes of moderate aerobic 
activity per week [2], and studies indicate that combining aerobic exercise with 
strength training provides the greatest cognitive benefits [3].

References:
[1] Nature, Vol 615, 2023 - Study on exercise and cognitive decline
[2] WHO Physical Activity Guidelines, 2020
[3] Journal of Aging Research, 2022 - Combined exercise benefits study
```

## Best Practices

<CardGroup cols={2}>
  <Card title="Verify Sources" icon="shield-check">
    Ensure retrieved information is from reliable sources
  </Card>

  <Card title="Explicit Limitations" icon="triangle-exclamation">
    Clearly state when information is not available
  </Card>

  <Card title="Source Attribution" icon="quote-left">
    Always cite where information came from
  </Card>

  <Card title="Error Handling" icon="bug">
    Handle tool failures gracefully with fallback strategies
  </Card>
</CardGroup>

## Practice Exercises

### Exercise 1: RAG Implementation

Design a RAG system for a customer support chatbot.

<Accordion title="Sample Solution">
  ```
  System Design:

  1. Knowledge Base:
     - Product documentation
     - FAQs
     - Return policies
     - Troubleshooting guides

  2. Retrieval Strategy:
     - Semantic search on user question
     - Retrieve top 3 most relevant documents
     - Include document metadata (source, date)

  3. Prompt Template:
  ```

  Retrieved Information:
  \[Doc 1 - Source: Product Manual, Section 3.2]
  \[CONTENT]

  \[Doc 2 - Source: FAQ #15]
  \[CONTENT]

  \[Doc 3 - Source: Troubleshooting Guide]
  \[CONTENT]

  Customer Question: \[QUESTION]

  Instructions:

  * Answer based on retrieved documents
  * Be helpful and friendly
  * Cite specific sources
  * If answer not found, offer to escalate to human agent

  Response:

  ```

  4. Fallback Strategy:
  - If confidence < 70%: Offer human escalation
  - If no relevant docs found: Acknowledge and escalate
  - Track unanswered questions for knowledge base improvement
  ```
</Accordion>

### Exercise 2: Tool Integration

Design a tool-using assistant for financial calculations.

<Accordion title="Sample Solution">
  ```
  Available Tools:

  1. stock_price(symbol, date=today)
     - Returns stock price for given symbol and date

  2. calculate(expression)
     - Evaluates mathematical expressions

  3. currency_convert(amount, from_currency, to_currency)
     - Converts between currencies

  4. portfolio_value(holdings)
     - Calculates total portfolio value

  Example Interaction:

  User: "I have 100 shares of AAPL bought at $150. What's my gain if I sell today?"

  Assistant Process:

  Step 1: Identify needed tools
  - Need current AAPL price: stock_price
  - Need to calculate gain: calculate

  Step 2: Execute tools
  Tool: stock_price("AAPL")
  Result: $185.50

  Tool: calculate("(185.50 - 150) * 100")
  Result: $3,550

  Step 3: Generate response
  "Your 100 shares of Apple (AAPL) have gained $3,550. You bought at $150/share 
  and the current price is $185.50/share, representing a 23.67% return on your 
  investment."
  ```
</Accordion>

### Exercise 3: Source Grounding

Create a grounded QA system that prevents hallucinations.

<Accordion title="Sample Solution">
  ```
  Grounding System:

  1. Strict Rules:
     - ONLY use provided context
     - Explicitly state when information is missing
     - Never infer or assume
     - Cite specific sources

  2. Template:
  ```

  Verified Sources:
  \[Source 1]: \[CONTENT]
  \[Source 2]: \[CONTENT]

  Question: \[QUESTION]

  Grounding Rules:
  ✓ Answer only from sources
  ✓ State "Not available" if missing
  ✓ Cite sources
  ✗ Never guess or infer

  Answer:

  ```

  3. Example:

  Sources:
  [1]: "Product X costs $99"
  [2]: "Product X has 5-star rating"

  Q: "What's the warranty on Product X?"

  GOOD: "I don't have information about the warranty for Product X in the 
  available sources. I can tell you it costs $99 [1] and has a 5-star rating [2], 
  but warranty details are not provided."

  BAD: "Product X likely has a 1-year warranty." ❌
  ```
</Accordion>

## Real-World Application: Knowledge-Grounded Assistant

```
You are a knowledge-grounded assistant with access to a verified knowledge base.

Knowledge Base: [RETRIEVED DOCUMENTS]

User Query: [QUESTION]

Response Protocol:

1. Search Knowledge Base
   Relevant documents: [LIST]
   Confidence: [HIGH/MEDIUM/LOW]

2. Generate Answer
   - Use ONLY information from knowledge base
   - Cite sources inline [Doc X]
   - If information incomplete, state explicitly

3. Quality Checks
   ✓ All facts from knowledge base?
   ✓ Sources cited?
   ✓ Limitations stated?
   ✓ No hallucinations?

4. Response Format:
   Answer: [GROUNDED RESPONSE]
   
   Sources:
   - [Doc 1]: [Citation]
   - [Doc 2]: [Citation]
   
   Confidence: [LEVEL]
   
   Limitations: [IF ANY]

5. Fallback (if needed):
   "I don't have enough information in my knowledge base to fully answer this question. 
   Would you like me to [alternative action]?"
```

## Key Takeaways

<Check>Use RAG to ground answers in verified sources</Check>
<Check>Integrate external tools for calculations and real-time data</Check>
<Check>Always cite sources and attribute information</Check>
<Check>Explicitly state limitations and missing information</Check>
<Check>Implement error handling for tool failures</Check>
<Check>Prevent hallucinations through strict grounding rules</Check>

## Module 3 Complete!

Congratulations! You've mastered advanced prompting techniques. You're now ready to tackle the Module 3 assessment and build a comprehensive AI assistant.

<Card title="Next: Module 3 Assessment" icon="clipboard-check" href="/module-3/assessment">
  Build a Multi-Capability AI Assistant combining all advanced techniques
</Card>