Lesson 3.1: Chain of Thought (CoT) Prompting

Duration: 75 minutes

Introduction

In 2022, researchers discovered something remarkable: simply asking an LLM to “think step-by-step” dramatically improved its reasoning abilities. This breakthrough, called Chain of Thought (CoT) prompting, transformed how we approach complex reasoning tasks. In this lesson, you’ll learn why CoT works and how to apply it effectively.

The CoT Breakthrough

Research Impact: Wei et al.’s 2022 paper “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” showed that CoT improved accuracy on math word problems from 17% to 58% on some benchmarks.

Why CoT Works

Chain of Thought prompting works because it:

Decomposition

Breaks complex problems into manageable steps

Error Correction

Allows the model to catch and fix mistakes mid-reasoning

Transparency

Makes the reasoning process visible and verifiable

Working Memory

Provides space to track intermediate results

Before vs. After CoT

Without CoT
With CoT

Prompt:

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. 
Each can has 3 tennis balls. How many tennis balls does he have now?

A:

Output:

11 tennis balls

❌ Incorrect - No reasoning shown, wrong answer

Prompt:

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. 
Each can has 3 tennis balls. How many tennis balls does he have now?

A: Let's think step-by-step.

Output:

Let's think step-by-step.
- Roger starts with 5 tennis balls
- He buys 2 cans of tennis balls
- Each can has 3 tennis balls
- So he gets 2 × 3 = 6 new tennis balls
- Total: 5 + 6 = 11 tennis balls

The answer is 11 tennis balls.

✅ Correct - Clear reasoning, right answer

Few-Shot CoT Pattern

Provide examples that demonstrate step-by-step reasoning.

Basic Few-Shot CoT

Q: A store sells notebooks for $3 each. If you buy 5 or more, you get 20% off the total. 
How much do 6 notebooks cost?

A: Let's work through this step-by-step:
- Regular price: 6 × $3 = $18
- Discount: 20% of $18 = $3.60
- Final price: $18 - $3.60 = $14.40
The answer is $14.40.

Q: A train travels 120 miles in 2 hours. If it increases speed by 20 mph for the next 3 hours, 
how far does it travel in total?

A: Let's work through this step-by-step:

Output:

- Initial speed: 120 miles ÷ 2 hours = 60 mph
- Distance in first segment: 120 miles
- New speed: 60 mph + 20 mph = 80 mph
- Distance in second segment: 80 mph × 3 hours = 240 miles
- Total distance: 120 miles + 240 miles = 360 miles
The answer is 360 miles.

Pro Tip: Include 1-3 examples with complete reasoning chains. More examples improve performance but use more tokens.

Multi-Domain Few-Shot CoT

Show CoT works across different problem types.

Example 1 - Math:
Q: If 3 apples cost $2, how much do 12 apples cost?
A: Let's solve this step-by-step:
- Cost per apple: $2 ÷ 3 = $0.67
- Cost of 12 apples: $0.67 × 12 = $8.04
The answer is $8.04.

Example 2 - Logic:
Q: All roses are flowers. Some flowers fade quickly. Do all roses fade quickly?
A: Let's reason through this:
- Premise 1: All roses are flowers (roses ⊂ flowers)
- Premise 2: Some flowers fade quickly (not all)
- Conclusion: We cannot determine if all roses fade quickly
The answer is: No, we cannot conclude that all roses fade quickly.

Example 3 - Commonsense:
Q: If I drop a glass on a concrete floor, what will likely happen?
A: Let's think about this:
- Glass is fragile and brittle
- Concrete is hard and unforgiving
- Impact force will exceed glass strength
- Result: The glass will likely shatter
The answer is: The glass will likely break or shatter.

Now solve this:
Q: [YOUR QUESTION]
A: Let's think step-by-step:

Zero-Shot CoT: The Magic Phrase

The simplest and most powerful CoT technique.

The Basic Pattern

Q: [QUESTION]

A: Let's think step-by-step.

That’s it! This simple phrase activates reasoning capabilities.

Variations That Work

Different phrasings that trigger CoT reasoning:

"Let's think step-by-step."
"Let's work through this systematically."
"Let's break this down."
"Let's solve this carefully."
"Let's approach this methodically."
"First, let's understand what we know..."

Research Finding: “Let’s think step-by-step” is the most consistently effective phrase across different models and tasks.

Zero-Shot CoT in Action

Q: A farmer has 17 sheep. All but 9 die. How many sheep are left?

A: Let's think step-by-step.

Output:

Let's think step-by-step.
- The farmer starts with 17 sheep
- "All but 9 die" means all except 9 die
- If all except 9 die, then 9 survive
- Therefore, 9 sheep are left

The answer is 9 sheep.

CoT Across Different Tasks

Arithmetic Problems

Q: A bakery makes 120 cupcakes. They sell 3/4 of them in the morning and 1/3 of the remainder 
in the afternoon. How many cupcakes are left?

A: Let's calculate step-by-step:

Output:

- Total cupcakes: 120
- Sold in morning: 3/4 × 120 = 90 cupcakes
- Remaining after morning: 120 - 90 = 30 cupcakes
- Sold in afternoon: 1/3 × 30 = 10 cupcakes
- Final remaining: 30 - 10 = 20 cupcakes

The answer is 20 cupcakes.

Commonsense Reasoning

Q: If you're running a race and you pass the person in second place, what place are you in?

A: Let's reason through this carefully:

Output:

- You were behind the person in second place
- You pass them, meaning you move ahead of them
- The person who was in second is now behind you
- You take their position
- Therefore, you are now in second place

The answer is: You are in second place.

Symbolic Reasoning

Q: If A > B and B > C, what is the relationship between A and C?

A: Let's work through this logically:

Output:

- Given: A > B (A is greater than B)
- Given: B > C (B is greater than C)
- By transitive property: If A > B and B > C, then A > C
- This is because A is greater than B, and B is greater than C
- Therefore, A must be greater than C

The answer is: A > C (A is greater than C)

Word Problems with Multiple Steps

Q: Sarah has $50. She spends 1/5 of it on lunch, then spends $12 on a book. 
She then earns $20 from babysitting. How much money does she have now?

A: Let's track her money step-by-step:

Output:

Step 1: Starting amount
- Sarah starts with $50

Step 2: Lunch expense
- Spends 1/5 of $50 = $10 on lunch
- Remaining: $50 - $10 = $40

Step 3: Book purchase
- Spends $12 on a book
- Remaining: $40 - $12 = $28

Step 4: Babysitting income
- Earns $20 from babysitting
- Final amount: $28 + $20 = $48

The answer is $48.

When CoT Helps Most

Multi-Step Problems

Best for: Problems requiring 2+ intermediate calculations or logical stepsExample: “If a car travels 60 mph for 2 hours, then 80 mph for 3 hours, what’s the average speed?”

Counter-Intuitive Problems

Best for: Problems where the obvious answer is wrongExample: “A bat and ball cost

1.10 total. The bat costs

1 more than the ball. How much does the ball cost?”

Problems Requiring Tracking

Best for: Scenarios where you need to track changing statesExample: “John has 5 apples. He gives 2 to Mary, who gives 1 to Tom. Tom gives 1 back to John. How many does each person have?”

Logical Reasoning

Best for: Deductive or inductive reasoning tasksExample: “All birds can fly. Penguins are birds. Can penguins fly?” (Requires recognizing the faulty premise)

Advanced CoT Patterns

CoT with Verification

Add a verification step to catch errors.

Q: [PROBLEM]

A: Let's solve this step-by-step, then verify our answer.

Solution:
[STEP-BY-STEP REASONING]

Verification:
[CHECK THE ANSWER]

Final answer:

Example:

Q: A rectangle has a perimeter of 24 cm and a length of 8 cm. What is its area?

A: Let's solve this step-by-step, then verify our answer.

Solution:
- Perimeter = 2(length + width) = 24 cm
- We know length = 8 cm
- So: 2(8 + width) = 24
- 16 + 2(width) = 24
- 2(width) = 8
- width = 4 cm
- Area = length × width = 8 × 4 = 32 cm²

Verification:
- Check perimeter: 2(8 + 4) = 2(12) = 24 cm ✓
- Check area calculation: 8 × 4 = 32 cm² ✓

Final answer: 32 cm²

CoT with Explicit Sub-Goals

Break the problem into clear sub-goals.

Q: [COMPLEX PROBLEM]

A: Let's identify our sub-goals and solve each:

Sub-goal 1: [FIRST OBJECTIVE]
[REASONING]

Sub-goal 2: [SECOND OBJECTIVE]
[REASONING]

Sub-goal 3: [FINAL OBJECTIVE]
[REASONING]

Final answer:

CoT with Alternative Approaches

Consider multiple solution paths.

Q: [PROBLEM]

A: Let's solve this using two different approaches to verify our answer.

Approach 1: [METHOD 1]
[REASONING]
Result: [ANSWER]

Approach 2: [METHOD 2]
[REASONING]
Result: [ANSWER]

Both approaches agree, so the answer is: [ANSWER]

Best Practices

Be Explicit

Use clear trigger phrases like “Let’s think step-by-step”

Show All Steps

Don’t skip intermediate calculations or logical steps

Label Steps

Number or label steps for clarity

Verify When Critical

Add verification for high-stakes problems

Common Pitfalls

Pitfall 1: Skipping CoT for Simple ProblemsDon’t use CoT for trivial questions like “What is 2+2?” It adds unnecessary overhead.Use CoT when: The problem requires 2+ steps or non-obvious reasoning

Pitfall 2: Vague Reasoning StepsBad: “Then we calculate the result”
Good: “Then we multiply 5 × 3 = 15”Be specific in each step.

Pitfall 3: Not Checking the Final AnswerAlways include a clear final answer statement: “The answer is: [ANSWER]“

Practice Exercises

Exercise 1: Math Word Problem

Apply CoT to this problem: “A store has a sale: Buy 2 items, get 25% off the total. Buy 4+ items, get 40% off. If shirts cost $20 each, how much do 5 shirts cost?”

Sample Solution

Let's calculate step-by-step:

Step 1: Determine which discount applies
- Buying 5 shirts (more than 4)
- Qualifies for 40% off

Step 2: Calculate original price
- 5 shirts × $20 = $100

Step 3: Calculate discount
- 40% of $100 = $40

Step 4: Calculate final price
- $100 - $40 = $60

The answer is $60.

Exercise 2: Logic Problem

Use CoT for this reasoning task: “In a family of 5, there are 2 parents and 3 children. The oldest child is twice the age of the youngest. The middle child is 3 years older than the youngest. If the youngest is 5, what is the total age of all children?”

Sample Solution

Let's work through this step-by-step:

Step 1: Identify the youngest child's age
- Youngest child: 5 years old

Step 2: Calculate the oldest child's age
- Oldest is twice the youngest
- Oldest: 5 × 2 = 10 years old

Step 3: Calculate the middle child's age
- Middle is 3 years older than youngest
- Middle: 5 + 3 = 8 years old

Step 4: Sum all children's ages
- Total: 5 + 8 + 10 = 23 years

The answer is 23 years.

Exercise 3: Commonsense Reasoning

Apply CoT to this scenario: “You have a 3-gallon jug and a 5-gallon jug. How can you measure exactly 4 gallons of water?”

Sample Solution

Let's think through this step-by-step:

Step 1: Fill the 5-gallon jug completely
- 5-gallon jug: 5 gallons
- 3-gallon jug: 0 gallons

Step 2: Pour from 5-gallon into 3-gallon until full
- 5-gallon jug: 5 - 3 = 2 gallons remaining
- 3-gallon jug: 3 gallons (full)

Step 3: Empty the 3-gallon jug
- 5-gallon jug: 2 gallons
- 3-gallon jug: 0 gallons

Step 4: Pour the 2 gallons from 5-gallon into 3-gallon
- 5-gallon jug: 0 gallons
- 3-gallon jug: 2 gallons

Step 5: Fill the 5-gallon jug again
- 5-gallon jug: 5 gallons
- 3-gallon jug: 2 gallons

Step 6: Pour from 5-gallon into 3-gallon until full
- 3-gallon jug can only take 1 more gallon (already has 2)
- 5-gallon jug: 5 - 1 = 4 gallons remaining
- 3-gallon jug: 3 gallons (full)

The answer is: The 5-gallon jug now contains exactly 4 gallons.

Real-World Application: Math Tutor System

Build a complete math tutoring system using CoT:

You are a math tutor helping students solve problems. Always use step-by-step reasoning.

Problem: [STUDENT'S PROBLEM]

Solution:
Let's work through this together, step-by-step:

Step 1: Understand what we're looking for
[Identify the goal]

Step 2: Identify what we know
[List given information]

Step 3: Determine our approach
[Explain the method]

Step 4: Execute the solution
[Show detailed calculations]

Step 5: Verify our answer
[Check if the answer makes sense]

Final Answer: [CLEAR ANSWER]

Explanation for student:
[Brief explanation of the key concept]

Key Takeaways

Use “Let’s think step-by-step” to activate CoT reasoning

Show all intermediate steps explicitly

CoT improves accuracy by 30-50% on reasoning tasks

Works across math, logic, and commonsense reasoning

Add verification steps for critical problems

Label and number steps for clarity

Next Steps

You’ve mastered Chain of Thought prompting. Now learn to decompose even more complex problems into manageable sub-problems.

Next: Lesson 3.2 - Problem Decomposition

Break complex problems into solvable pieces

​Introduction

​The CoT Breakthrough

​Why CoT Works

Decomposition

Error Correction

Transparency

Working Memory

​Before vs. After CoT

​Few-Shot CoT Pattern

​Basic Few-Shot CoT

​Multi-Domain Few-Shot CoT

​Zero-Shot CoT: The Magic Phrase

​The Basic Pattern

​Variations That Work

​Zero-Shot CoT in Action

​CoT Across Different Tasks

​Arithmetic Problems

​Commonsense Reasoning

​Symbolic Reasoning

​Word Problems with Multiple Steps

​When CoT Helps Most

​Advanced CoT Patterns

​CoT with Verification

​CoT with Explicit Sub-Goals

​CoT with Alternative Approaches

​Best Practices

Be Explicit

Show All Steps

Label Steps

Verify When Critical

​Common Pitfalls

​Practice Exercises

​Exercise 1: Math Word Problem

​Exercise 2: Logic Problem

​Exercise 3: Commonsense Reasoning

​Real-World Application: Math Tutor System

​Key Takeaways

​Next Steps

Next: Lesson 3.2 - Problem Decomposition

Introduction

The CoT Breakthrough

Why CoT Works

Before vs. After CoT

Few-Shot CoT Pattern

Basic Few-Shot CoT

Multi-Domain Few-Shot CoT

Zero-Shot CoT: The Magic Phrase

The Basic Pattern

Variations That Work

Zero-Shot CoT in Action

CoT Across Different Tasks

Arithmetic Problems

Commonsense Reasoning

Symbolic Reasoning

Word Problems with Multiple Steps

When CoT Helps Most

Advanced CoT Patterns

CoT with Verification

CoT with Explicit Sub-Goals

CoT with Alternative Approaches

Best Practices

Common Pitfalls

Practice Exercises

Exercise 1: Math Word Problem

Exercise 2: Logic Problem

Exercise 3: Commonsense Reasoning

Real-World Application: Math Tutor System

Key Takeaways

Next Steps