Advanced Robotics / Reinforcement Learning · Lesson 01
Why Robots Learn
Est. 60–75 min
Why
Why this lesson matters
For thirteen stages you built robots that follow rules and models: PID loops, kinematics, planners, grasp checks. They are fast, predictable and easy to debug — and they are only as good as the model inside them. When the world changes in a way the model did not expect (a new surface, a worn gripper pad, an object nobody measured), a pure rule-follower keeps making the same mistake.
Reinforcement learning (RL) is a different style. The robot is not told the right answer. It tries something, gets a number back that says how good the result was — the reward — and slowly prefers actions that earned more reward. In this lesson you will compare a rule-based robot and a learning robot on the same task, see where each one wins, and build a small UNO “head” that learns by trial and error where the light is.
Concept
Rules, models and learning from reward
1. Three ways to make a robot do the right thing
| Approach | What you give the robot | Example from earlier stages | Weak point |
|---|---|---|---|
| Hand-written rules | IF this THEN that | Stage 13 task sequencer, E-stop logic | Breaks in situations you did not imagine |
| Model + control / planning | Equations of the robot and world | PID (Stage 7), IK (Stage 8), RRT (Stage 13) | Only as good as the model; needs re-measuring when the world changes |
| Learning from reward (RL) | A goal written as a reward number | This stage | Needs many tries; reward can be gamed; harder to prove safe |
2. The agent–environment loop
RL describes every problem with the same loop. The agent (the robot’s decision-maker) looks at the state of the world, chooses an action, and the environment (the desk, the puck, physics) responds with a new state and a reward. Then the loop repeats. The agent’s job is to learn a policy — a rule for choosing actions — that collects as much reward as possible over time. Sutton and Barto’s textbook, the standard reference for this field, is built entirely around this loop.

3. The simplest learner: a table of averages
Our puck task has just one decision: which of 9 flick speeds to use. That is a multi-armed bandit — named after a row of slot machines, each with an unknown payout. The learner keeps one number per speed: the average reward it got from that speed. Most of the time it picks the speed with the best average (exploit); with a small probability ε (epsilon, say 0.1) it picks a random speed (explore). This ε-greedy rule is the simplest exploration method, and it appears in almost every RL course.
new estimate = old estimate + step × (reward − old estimate)
step = 1/n → plain average of all n tries (remembers everything equally)
step = 0.2 → recent tries count more (forgets an old desk surface)That one line is the seed of everything in this stage. Q-learning (Lesson 3 onwards) uses the same “move the estimate a little towards what you just saw” idea, just with states added.

Visual
Visualise it: walk a toy pick task
A toy pick task written as states, actions and rewards. Step through it by hand before any learning happens. (Educational simulation.)
Interactive · RL MDP explorer · Educational
State s
Approach
Last reward r
—
Episode return
0.00 · 0 steps
Actions a (from current state)
Pick a state, then take an action. Watch reward + next state.
Toy transition table (partial)
| s | a | r | s′ |
|---|---|---|---|
| Approach | Move closer | -0.10 | At grasp |
| Approach | Close gripper | -1.00 | Fail |
| Approach | Raise | -0.20 | Approach |
| Approach | Retreat | -0.05 | Approach |
| At grasp | Move closer | -0.05 | At grasp |
| At grasp | Close gripper | 0.50 | Lift |
| At grasp | Raise | -1.00 | Fail |
| At grasp | Retreat | -0.10 | Approach |
| Lift | Move closer | -0.05 | Lift |
| Lift | Close gripper | -0.02 | Lift |
| Lift | Raise | 1.00 | Done |
| Lift | Retreat | -0.50 | Fail |
| Done | Move closer | 0.00 | Done |
| Done | Close gripper | 0.00 | Done |
| Done | Raise | 0.00 | Done |
| Done | Retreat | 0.00 | Done |
Code lab
Code lab: rule vs learner on the puck desk
why_learn_lab.py simulates the puck desk with friction and a little random scatter. The RULE robot uses the formula with the μ it was told. The LEARNER knows nothing about friction; it only sees the reward −|miss in cm| after every flick. Standard library only — run it with python3 why_learn_lab.py. All numbers are classroom examples.
"""why_learn_lab.py - Stage 14 L01: rule vs learning on a puck-flick task.
A small arm flicks a puck along the desk. It should stop on a line 50 cm away.
The puck slides with friction: stopping distance d = v^2 / (2 * mu * g).
Two robots:
RULE - uses the physics formula with the mu it was told (a model).
LEARNER - knows nothing about friction. It tries 9 flick speeds and keeps
an average reward for each (an epsilon-greedy "bandit").
Reward = -|miss distance in cm| (closer is better, 0 is perfect).
All numbers are example classroom values. Standard library only.
"""
import math
import random
G = 9.81
TARGET = 0.50 # metres
SPEEDS = [0.90 + 0.05 * i for i in range(9)] # 0.90 ... 1.30 m/s
def flick(v, mu, rng, noise=0.03):
"""Real desk: stopping distance with friction plus a little random scatter."""
d = v * v / (2 * mu * G)
return d * (1 + rng.gauss(0, noise))
def rule_speed(mu_model):
"""Classical answer: invert the formula for the target distance."""
return math.sqrt(2 * mu_model * G * TARGET)
def nearest_speed(v):
return min(SPEEDS, key=lambda s: abs(s - v))
def run_rule(mu_model, mu_real, tries, rng):
v = nearest_speed(rule_speed(mu_model))
misses = [abs(flick(v, mu_real, rng) - TARGET) * 100 for _ in range(tries)]
return v, misses
def run_learner(mu_real, tries, eps, rng, q=None, n=None, alpha=None):
q = q if q is not None else [0.0] * len(SPEEDS)
n = n if n is not None else [0] * len(SPEEDS)
misses = []
for t in range(tries):
if 0 in n: # try every speed once first
a = n.index(0)
elif rng.random() < eps: # explore
a = rng.randrange(len(SPEEDS))
else: # exploit the best average so far
a = max(range(len(SPEEDS)), key=lambda i: q[i])
miss = abs(flick(SPEEDS[a], mu_real, rng) - TARGET) * 100
reward = -miss
n[a] += 1
step = alpha if alpha else 1 / n[a] # 1/n = plain average, alpha = forget old data
q[a] += step * (reward - q[a])
misses.append(miss)
best = max(range(len(SPEEDS)), key=lambda i: q[i])
return SPEEDS[best], misses, q, n
def avg(xs):
return sum(xs) / len(xs)
def main():
print("Target line:", TARGET * 100, "cm candidate speeds:",
", ".join(f"{s:.2f}" for s in SPEEDS), "m/s")
for mu in (0.10, 0.12, 0.15):
print(f" mu {mu:.2f}: perfect speed {rule_speed(mu):.3f} m/s")
print("\n1) Clean desk, model is right (mu told 0.12, real 0.12)")
rng = random.Random(1)
v, m = run_rule(0.12, 0.12, 40, rng)
print(f" RULE speed {v:.2f} first miss {m[0]:.1f} cm avg of 40 {avg(m):.1f} cm")
rng = random.Random(1)
vb, m2, q, n = run_learner(0.12, 40, 0.1, rng)
print(f" LEARNER best {vb:.2f} first 9 avg {avg(m2[:9]):.1f} cm last 10 avg {avg(m2[-10:]):.1f} cm total wasted cm {sum(m2) - sum(m):.0f}")
print("\n2) Someone puts a cloth mat on the desk (model still says 0.12, real 0.15)")
rng = random.Random(2)
v, m = run_rule(0.12, 0.15, 40, rng)
print(f" RULE speed {v:.2f} avg miss {avg(m):.1f} cm (keeps missing until a human re-measures mu)")
rng = random.Random(2)
vb, m2, q, n = run_learner(0.15, 40, 0.1, rng)
print(f" LEARNER best {vb:.2f} last 10 avg {avg(m2[-10:]):.1f} cm")
print(" learner's table (speed: average reward, tries):")
for s, qi, ni in zip(SPEEDS, q, n):
print(f" {s:.2f} m/s: {qi:7.1f} x{ni}")
print("\n3) Mat removed mid-way (real mu 0.15 -> 0.10 after 40 tries)")
for eps in (0.0, 0.1, 0.3):
rng = random.Random(3)
_, _, q, n = run_learner(0.15, 40, eps, rng)
vb, m3, _, _ = run_learner(0.10, 60, eps, rng, q, n)
print(f" eps {eps:.1f} plain average : best after change {vb:.2f} last 20 avg miss {avg(m3[-20:]):.1f} cm")
for eps in (0.1, 0.3):
rng = random.Random(3)
_, _, q, n = run_learner(0.15, 40, eps, rng, alpha=0.2)
vb, m3, _, _ = run_learner(0.10, 60, eps, rng, q, n, alpha=0.2)
print(f" eps {eps:.1f} step 0.2 (forgets): best after change {vb:.2f} last 20 avg miss {avg(m3[-20:]):.1f} cm")
print("\n4) Cost of learning on a real robot")
for tries in (9, 40, 200, 5000):
print(f" {tries:5d} flicks x 6 s each = {tries * 6 / 60:6.1f} min of arm time")
if __name__ == "__main__":
main()Output (run with python3 why_learn_lab.py)
Target line: 50.0 cm candidate speeds: 0.90, 0.95, 1.00, 1.05, 1.10, 1.15, 1.20, 1.25, 1.30 m/s
mu 0.10: perfect speed 0.990 m/s
mu 0.12: perfect speed 1.085 m/s
mu 0.15: perfect speed 1.213 m/s
1) Clean desk, model is right (mu told 0.12, real 0.12)
RULE speed 1.10 first miss 3.4 cm avg of 40 1.7 cm
LEARNER best 1.10 first 9 avg 9.7 cm last 10 avg 1.9 cm total wasted cm 93
2) Someone puts a cloth mat on the desk (model still says 0.12, real 0.15)
RULE speed 1.10 avg miss 9.3 cm (keeps missing until a human re-measures mu)
LEARNER best 1.20 last 10 avg 1.6 cm
learner's table (speed: average reward, tries):
0.90 m/s: -20.5 x1
0.95 m/s: -19.9 x1
1.00 m/s: -15.6 x1
1.05 m/s: -12.4 x1
1.10 m/s: -7.9 x1
1.15 m/s: -7.0 x1
1.20 m/s: -1.7 x30
1.25 m/s: -2.4 x3
1.30 m/s: -5.6 x1
3) Mat removed mid-way (real mu 0.15 -> 0.10 after 40 tries)
eps 0.0 plain average : best after change 1.10 last 20 avg miss 13.5 cm
eps 0.1 plain average : best after change 1.05 last 20 avg miss 7.7 cm
eps 0.3 plain average : best after change 1.05 last 20 avg miss 10.4 cm
eps 0.1 step 0.2 (forgets): best after change 1.00 last 20 avg miss 3.5 cm
eps 0.3 step 0.2 (forgets): best after change 1.00 last 20 avg miss 7.7 cm
4) Cost of learning on a real robot
9 flicks x 6 s each = 0.9 min of arm time
40 flicks x 6 s each = 4.0 min of arm time
200 flicks x 6 s each = 20.0 min of arm time
5000 flicks x 6 s each = 500.0 min of arm timeTry it: walk a toy pick task step by step. Close the gripper too early and see the negative reward; then approach, close and lift to collect the success reward. (Educational simulation — not MuJoCo, Isaac or a real RL library.)
Interactive · RL MDP explorer · Educational
State s
Approach
Last reward r
—
Episode return
0.00 · 0 steps
Actions a (from current state)
Pick a state, then take an action. Watch reward + next state.
Toy transition table (partial)
| s | a | r | s′ |
|---|---|---|---|
| Approach | Move closer | -0.10 | At grasp |
| Approach | Close gripper | -1.00 | Fail |
| Approach | Raise | -0.20 | Approach |
| Approach | Retreat | -0.05 | Approach |
| At grasp | Move closer | -0.05 | At grasp |
| At grasp | Close gripper | 0.50 | Lift |
| At grasp | Raise | -1.00 | Fail |
| At grasp | Retreat | -0.10 | Approach |
| Lift | Move closer | -0.05 | Lift |
| Lift | Close gripper | -0.02 | Lift |
| Lift | Raise | 1.00 | Done |
| Lift | Retreat | -0.50 | Fail |
| Done | Move closer | 0.00 | Done |
| Done | Close gripper | 0.00 | Done |
| Done | Raise | 0.00 | Done |
| Done | Retreat | 0.00 | Done |
Bench build: a light-seeking head that learns
light_seeker_bandit.ino turns a servo with a light sensor (LDR) into a 9-armed bandit. Each “arm” is an angle from 30° to 150°. The reward is the light reading. Nobody tells the UNO where the window or the torch is — it tries angles, keeps a value for each, and with ε = 0.15 sometimes explores. Because it uses a constant step of 0.2, it re-learns when you move the torch.
| Part | Connection | Note |
|---|---|---|
| SG90/MG90-class servo | signal → D9 | LDR glued or taped to the servo horn, pointing outwards |
| Servo supply | red → separate 5–6 V supply, brown → supply GND | join supply GND to UNO GND |
| LDR + 10 kΩ resistor | LDR between 5 V and A0; 10 kΩ between A0 and GND | brighter light → higher reading |
| Push button | D2 → button → GND | INPUT_PULLUP; press to clear the memory |
| Torch / phone light | — | the “goal” the robot has to discover |
// light_seeker_bandit.ino - Stage 14 L01
// A servo "head" with a light sensor (LDR) learns, by trial and error,
// which of 9 angles gets the most light. No light model, no map - only reward.
//
// Wiring (see lesson table):
// Servo signal -> D9, servo power from a SEPARATE 5-6 V supply, grounds joined
// LDR + 10k divider -> A0 (LDR from 5V to A0, 10k from A0 to GND)
// Button D2 -> GND (INPUT_PULLUP): press = forget everything, start again
// Serial 115200: prints each try and the learned table.
#include <Servo.h>
const int SERVO_PIN = 9;
const int LDR_PIN = A0;
const int RESET_PIN = 2;
const int N_ARMS = 9;
const int ANGLES[N_ARMS] = {30, 45, 60, 75, 90, 105, 120, 135, 150};
const float EPSILON = 0.15; // 15 % of tries explore at random
const float STEP = 0.2; // constant step: old readings fade (room light changes)
const unsigned long SETTLE_MS = 400;
Servo head;
float q[N_ARMS];
int n[N_ARMS];
int tryCount = 0;
int currentAngle = 90;
void forget() {
for (int i = 0; i < N_ARMS; i++) { q[i] = 0; n[i] = 0; }
tryCount = 0;
Serial.println(F("--- memory cleared, learning from scratch ---"));
}
void slowMoveTo(int target) {
// 2 degrees every 15 ms (~130 deg/s) so the servo never jerks
while (currentAngle != target) {
currentAngle += (target > currentAngle) ? min(2, target - currentAngle) : -min(2, currentAngle - target);
head.write(currentAngle);
delay(15);
}
}
int readLight() {
long sum = 0;
for (int i = 0; i < 8; i++) { sum += analogRead(LDR_PIN); delay(5); }
return sum / 8; // 0..1023, bigger = brighter with this divider
}
int bestArm() {
int b = 0;
for (int i = 1; i < N_ARMS; i++) if (q[i] > q[b]) b = i;
return b;
}
int chooseArm(bool &explored) {
explored = false;
for (int i = 0; i < N_ARMS; i++) if (n[i] == 0) return i; // try each once
if (random(1000) < EPSILON * 1000) { explored = true; return random(N_ARMS); }
return bestArm();
}
void printTable() {
Serial.println(F("angle value tries"));
int b = bestArm();
for (int i = 0; i < N_ARMS; i++) {
Serial.print(ANGLES[i]); Serial.print(F(" "));
Serial.print(q[i], 0); Serial.print(F(" "));
Serial.print(n[i]);
if (i == b) Serial.print(F(" <- best so far"));
Serial.println();
}
}
void setup() {
Serial.begin(115200);
pinMode(RESET_PIN, INPUT_PULLUP);
randomSeed(analogRead(A5)); // floating pin gives a different seed each start
head.attach(SERVO_PIN);
head.write(currentAngle);
delay(500);
forget();
}
void loop() {
if (digitalRead(RESET_PIN) == LOW) { forget(); delay(300); }
bool explored;
int a = chooseArm(explored);
slowMoveTo(ANGLES[a]);
delay(SETTLE_MS); // let the servo stop shaking before reading
int reward = readLight();
n[a]++;
if (n[a] == 1) q[a] = reward; // first reading: just store it
else q[a] += STEP * (reward - q[a]);
tryCount++;
Serial.print(F("try ")); Serial.print(tryCount);
Serial.print(explored ? F(" EXPLORE ") : F(" exploit "));
Serial.print(F("angle ")); Serial.print(ANGLES[a]);
Serial.print(F(" reward ")); Serial.println(reward);
if (tryCount % 20 == 0) printTable();
delay(200);
}--- memory cleared, learning from scratch ---
try 1 exploit angle 30 reward <reading>
try 2 exploit angle 45 reward <reading>
... (first 9 tries visit every angle once)
try 14 EXPLORE angle 135 reward <reading>
try 20 exploit angle <best> reward <reading>
angle value tries
30 <value> <n>
...
<best> <value> <n> <- best so farDebugging
| Symptom | Likely cause | Fix |
|---|---|---|
| All readings almost the same | room too bright or LDR covered | Dim the room, use a torch, shade the LDR with a paper tube so it “looks” in one direction |
| Readings get lower when brighter | LDR and resistor swapped | LDR from 5 V to A0, resistor from A0 to GND |
| Best angle jumps around | reading taken while servo still moving | Increase SETTLE_MS to 600–800 |
| UNO resets when the servo moves | servo powered from the UNO | Separate supply with grounds joined |
| Always the same angle even after moving the torch | EPSILON is 0 or STEP too small | Use EPSILON 0.1–0.2 and STEP 0.2 |
| Python learner never improves | reward sign flipped | Reward must be −miss (bigger is better); check max picks the best |
Real world
Robot learning in the real world

- Games first: RL became famous through games — backgammon, then Atari video games learned from pixels, then Go (AlphaGo). Games have perfect simulators and unlimited tries.
- Legged robots: walking over rough ground is full of contact and slipping that is hard to model by hand; learned controllers trained in simulation are now widely studied for legged robots.
- Manipulation: pushing, inserting and handling soft objects are active RL research areas, usually trained in simulation (Gymnasium, Isaac Lab, MuJoCo) and then carefully transferred.
- Honest note: most industrial robots you will meet in Kota or anywhere else still run classical control. Learning is growing, but a good engineer knows when *not* to use it.


Think
Think about it
Think
Your stage-7 PID line follower works perfectly on the lab floor. A friend says “replace it with RL, it is more advanced.” What do you say?
Experiment
Try it in the lab
About 60 minutes. Python 3 for the puck simulation; an Arduino UNO with a servo on its own supply, an LDR and 10 kΩ resistor, a push button and a torch.
Experiment
Rule vs learner, on screen and on the bench
You will need
- `why_learn_lab.py` (Python 3, no libraries)
- Toy MDP explorer (in your browser)
- Arduino UNO, SG90/MG90-class servo, separate 5–6 V supply, LDR, 10 kΩ resistor, push button, jumper wires
- Torch or phone light, paper tube to shade the LDR, notebook
Steps
- Run `why_learn_lab.py`. Write down the average miss of RULE and LEARNER in case 1 and case 2.
- Change the real μ in case 2 from 0.15 to 0.18. Who suffers more — the rule robot or the learner?
- In case 3, try step 0.1 and 0.5 instead of 0.2. Which one adapts fastest, and which one is noisiest?
- Build the light-seeking head. Put the torch at about 60°. Let it run 40 tries and copy the table.
- Move the torch to about 130°. Count how many tries until “best so far” moves. Repeat with EPSILON 0.05 and 0.3.
- Open the MDP explorer and play the toy pick task by hand until you get the full success reward.
Interactive · RL MDP explorer · Educational
State s
Approach
Last reward r
—
Episode return
0.00 · 0 steps
Actions a (from current state)
Pick a state, then take an action. Watch reward + next state.
Toy transition table (partial)
| s | a | r | s′ |
|---|---|---|---|
| Approach | Move closer | -0.10 | At grasp |
| Approach | Close gripper | -1.00 | Fail |
| Approach | Raise | -0.20 | Approach |
| Approach | Retreat | -0.05 | Approach |
| At grasp | Move closer | -0.05 | At grasp |
| At grasp | Close gripper | 0.50 | Lift |
| At grasp | Raise | -1.00 | Fail |
| At grasp | Retreat | -0.10 | Approach |
| Lift | Move closer | -0.05 | Lift |
| Lift | Close gripper | -0.02 | Lift |
| Lift | Raise | 1.00 | Done |
| Lift | Retreat | -0.50 | Fail |
| Done | Move closer | 0.00 | Done |
| Done | Close gripper | 0.00 | Done |
| Done | Raise | 0.00 | Done |
| Done | Retreat | 0.00 | Done |
A model gives instant good answers when it is right and repeated mistakes when it is wrong. A learner starts badly, improves from reward, and can follow a changing world — if it keeps exploring and does not trust old data forever.
Knowledge check
Check your understanding
Knowledge check
1. In reinforcement learning, the robot learns mainly from…
2. In the clean-desk case, why did the rule robot beat the learner at first?
3. ε-greedy with ε = 0.1 means…
4. With ε = 0 the learner did not adapt when the mat was removed because…
5. Why did a constant step of 0.2 adapt better than a plain average after the desk changed?
6. Why is robot RL usually trained in simulation first?
7. A one-decision problem like choosing the best flick speed is called a…
8. Where should the servo in the light-seeker get its power?
Mini assignment
Hands-on assignment
Mini assignment
Classical vs learning card
Decide, with evidence, where learning helps a robot and where it does not.
Deliverables
- A table of 3 robot tasks better done with rules/models/PID and 3 where learning from reward might help — one line of reason each.
- Your Python results for case 2 with μ 0.15 and 0.18 and your explanation of who suffered more.
- The light-seeker log: tries needed to switch after moving the torch, for ε 0.05, 0.15 and 0.3.
- One “hybrid” idea: a classical controller plus a small learned part, and how you would keep it safe.
Do this, step by step
- Make the 3 + 3 task table (rules/PID vs learning), one reason per line.
- Run
why_learn_lab.pywith real μ 0.15, then 0.18 in case 2; note the average miss of RULE and LEARNER. - Bench: servo on its own 5–6 V supply, grounds joined; upload
light_seeker_bandit.inoand let it run 40 tries. - Move the torch and count tries until “best so far” changes, for
EPSILON0.05, 0.15 and 0.3. - Write your hybrid idea: the classical controller stays in charge, the learned part only tunes one number inside safe limits.
Stuck? Hints, one at a time
- If the learner never changes its mind after you move the torch, check
EPSILON— at 0 it never explores. - The rule robot uses μ from its model; only the real μ changes in case 2, so the model is now wrong.
- If the head jitters or the UNO resets, the servo is drawing too much current — power it from the separate supply, never the 5 V pin.
Challenges (pick by level)
- 🟢 Beginner: Write three everyday examples of learning from reward (a dog trick, a bowler adjusting length, a phone keyboard learning your words). For each, name the action and the reward.
- 🟡 Intermediate: In the Python file, make ε decrease over time (start 0.3, multiply by 0.95 each try, minimum 0.02). Does the learner waste fewer flicks?
- 🟠 Advanced: Give the learner 17 speeds instead of 9 (step 0.025). Is the final miss smaller? How many extra tries did it cost?
- 🔴 Expert: Add a second LDR at a fixed place so the head’s reward is “light on LDR 1 minus light on LDR 2”. Now it must find a direction that is bright for one and dark for the other.
- ⚫ Research: Read Chapter 2 (“Multi-armed Bandits”) of Sutton & Barto. Find the name of the method that uses an “upper confidence bound” instead of random exploration, and explain it in two lines.
Next step
Our puck learner had only one decision and no “situation”. Real robots choose many actions in a row, and the right action depends on where they are. In Lesson 2 — States, Actions & Rewards you will write a robot task as a proper Markov decision process and build your own Gymnasium-style environment.
Look back: Stage 13 Lesson 16: Capstone. Next: Lesson 2: States, Actions & Rewards. Related: Stage 13 — Perception, Planning & Manipulation.
Sources
- [Sutton & Barto — Reinforcement Learning: An Introduction, 2nd edition (MIT Press, 2018), free online](http://incompleteideas.net/book/the-book-2nd.html) — the standard textbook; agent–environment loop, bandits (Chapter 2), incremental averages and constant step sizes for changing problems.
- Wikipedia — Reinforcement learning — agent/environment/reward framing, exploration vs exploitation, ε-greedy, applications including robot control and Go (AlphaGo), sample inefficiency.
- Wikipedia — Multi-armed bandit — slot-machine origin of the name, bandit as a one-state MDP, epsilon-greedy strategy.
- Gymnasium — Env API (Farama Foundation) — the standard Python interface (reset/step) used for RL environments; we build one in Lesson 2.
- Arduino Servo library (GitHub) — controlling RC hobby servos.
- Photos: Wikimedia Commons (credits under each image).
Self-attested for now — keep artifacts in your lab notebook.
Complete
Today's goal
0 of 1 done today
Mark this lesson complete when you have worked the experiment, knowledge check, and assignment (even roughly).