Stage 03 · Lesson 05/14

Linux & Dev Environment · Lesson 05

Processes & System Monitoring

Est. 40–50 min

Why

Why you must know what is running on your robot

Starting programs is easy. A good robot engineer also knows how to see what is running, measure what each process is using, and stop things in the right order — politely first, forcefully only if needed.

Concept

The big idea: programs become processes with IDs, states and signals

  • Program = a file on disk (/usr/bin/python3). Process = that program running, with its own PID (process ID).
  • Each process has a parent (the shell that started it), a state (running, sleeping, stopped, zombie) and uses CPU % and memory.
  • Foreground jobs own your terminal; background jobs (command &) run while you keep typing.
  • Signals are short messages to a process: please stop (SIGTERM), interrupt (SIGINT, from Ctrl+C), kill now (SIGKILL — cannot be caught, no cleanup).
Monitoring and control commands
CommandWhat it shows / does
ps auxSnapshot of every process (user, PID, %CPU, %MEM, command)
pgrep -a pythonFind processes by name, with their PIDs
top / htopLive view sorted by CPU; press q to quit (sudo apt install htop)
kill PIDSend SIGTERM (15) — polite stop request
kill -INT PIDSame as pressing Ctrl+C in that program’s terminal
kill -9 PIDSIGKILL — last resort, the program cannot clean up
jobs, fg, bg, Ctrl+ZManage jobs started from this terminal
free -h, df -h, uptimeMemory, disk space, load average
tegrastatsOn NVIDIA Jetson: CPU, GPU, RAM and temperature

How it works

How it actually works: the life of a robot process

  1. Start

    You type python3 camera_driver.py &. The shell asks the kernel to create a new process; the kernel gives it a PID (say 4312). $! in bash holds that PID.

  2. Run and sleep

    Most of the time the process is sleeping — waiting for the next camera frame. When a frame arrives the kernel wakes it and gives it CPU time.

  3. Measure

    top shows its %CPU. 100% on one core means it never rests — often a loop without sleep or a too-high frame rate.

  4. Stop politely

    kill 4312 sends SIGTERM. A well-written driver catches it, releases the camera and exits.

  5. Exit status

    Every process returns a number. 0 = success. A process stopped by signal N reports 128 + N (SIGTERM 15 → 143).

Visual

Visualise it: robot process monitor

A simulated ps-style view of four robot processes. Start and stop them and watch CPU and state change — nothing real is started.

Interactive · Robot process monitor

Simulated ps-style view — 3/4 running. No real processes are started.

PROCESSROLECPU%MEMSTATE
robot_controllerSense → decide → act loop1248MRUN
camera_driverUSB camera /dev/video0 reader2296MRUN
telemetry_loggerJSONL joint / sensor logger0—STOP
estop_watcherSafety monitor — halt on fault18MRUN
# process monitor ready — start/stop simulated only

Worked example

Worked example: the laggy arm

Arjun’s arm reacts one second late to a hand in front of the camera. He opens a second terminal:

StepCommandWhat he learns
1top (press P to sort by CPU)camera_driver.py at 99% CPU, controller at 3%
2pgrep -a cameraPID 4312, started with --fps 60
3kill 4312Driver prints “releasing /dev/video0” and exits cleanly
4Restart with --fps 15CPU drops to ~25%, the arm reacts on time
NoteThese numbers are an illustration of the method. On your machine, measure — never guess.

Code lab

Code lab: start, check and stop — the right way

First in bash: start a stand-in “driver” in the background, confirm it runs, stop it with SIGTERM and read the exit status.

process_drill.sh · Shell
#!/usr/bin/env bash
# process_drill.sh — start, check and stop a background process (Lesson 5)
sleep 30 &                     # pretend this is camera_driver
PID=$!
kill -0 "$PID" && echo "camera_driver (sleep) is running"
kill -TERM "$PID"              # polite stop request (SIGTERM = 15)
wait "$PID"
echo "exit status: $?  (128 + 15 means it was stopped by SIGTERM)"
kill -0 "$PID" 2>/dev/null || echo "process is gone"

Expected output

camera_driver (sleep) is running
exit status: 143  (128 + 15 means it was stopped by SIGTERM)
process is gone

Now the most important habit for robot code: a loop that catches Ctrl+C and SIGTERM so it can brake the motors and close the serial port before exiting.

graceful_stop.py · Python
# graceful_stop.py — a robot loop that stops politely on Ctrl+C or SIGTERM (Lesson 5)
import os
import signal
import time

running = True

def on_stop(signum, frame):
    global running
    print(f"got {signal.Signals(signum).name} -> braking motors, closing serial")
    running = False

signal.signal(signal.SIGINT, on_stop)    # Ctrl+C in the terminal
signal.signal(signal.SIGTERM, on_stop)   # kill <pid> / systemd stop

tick = 0
while running:
    tick += 1
    print(f"tick {tick}: reading sensors")
    if tick == 3:                         # demo only: send ourselves a SIGTERM
        os.kill(os.getpid(), signal.SIGTERM)
    time.sleep(0.1)

print("controller stopped cleanly")

Expected output

tick 1: reading sensors
tick 2: reading sensors
tick 3: reading sensors
got SIGTERM -> braking motors, closing serial
controller stopped cleanly

Reading logs

Watch the register: reading a running robot’s log

While processes run, they write logs. tail -f logs/robot_session.log follows the file live (Ctrl+C to stop following). To find problems fast you filter with grep. Try it on a real-looking robot log here — you will master pipes fully in Lesson 8.

Pick ERROR, then add the serial component filter. Turn on the per-component count to see which process complains the most.

Interactive · Pipe & grep filter lab

A 12-line robot log, like a kirana shop register. Each | passes the lines that survive to the next tool. Change one control and watch the line count at every stage.

  1. cat robot_session.log12 lines
  2. | grep ERROR3 lines
  3. | head -n 123 lines
$ cat robot_session.log | grep ERROR | head -n 12
10:00:05 ERROR serial  timeout waiting for reply
10:00:09 ERROR camera  no frames for 2 s
10:00:12 ERROR serial  checksum mismatch

Real world

Where process skills save a demo

  • Camera driver pegs the CPU → frames drop → controller acts on old detections.
  • A crashed logger leaves the arm working but with no record of what happened.
  • On a Jetson, tegrastats shows the GPU busy and the board temperature — thermal throttling slows everything.
  • free -h shows RAM almost full → the system starts swapping and the control loop timing jitters.

Mistakes

Common mistakes and debugging

Hardware & safety

Hardware lab and safety notes

On a real robot, stopping a process the wrong way can leave a motor running. Treat “stop” as a safety feature.

  • Test graceful_stop.py with motors disconnected first; then with the robot on blocks (wheels off the ground).
  • Use kill PID (SIGTERM) and wait a few seconds before ever trying kill -9.
  • Keep a physical power switch on the motor supply — software stop is not an emergency stop.
Hobby servo motors
Hobby servos — a crashed script can leave them holding position or twitching.Photo: Kborer · Public domain (released by the author) · Source: Wikimedia Commons · resized, converted to WebP

Debugging

Debugging: when it doesn’t work

Symptom → likely cause → fix
SymptomLikely causeFix
Ctrl + C does nothingScript ignores SIGINT or is stuck in a C callTry kill PID; fix the loop to check a stop flag
Next start says “device busy”Old process still holds the serial portps aux | grep python, then kill PID
One process at 100% CPULoop without sleepAdd time.sleep(0.02) or a rate limit
Exit status 137Something used SIGKILL (128 + 9) — maybe out of memoryCheck dmesg for “Out of memory”; use SIGTERM
NoteRead the last line of an error first — it usually names the problem. Change one thing at a time and re-run.

Think

Think about it

Think

Which process should you never casually kill on a moving robot, and in what order should you stop the rest?

Experiment

Try it in the lab: process drill

Use the process monitor as your robot. Narrate what a real arm would do at each step.

Experiment

Process drill

You will need

  • Robot process monitor
  • Pipe & grep filter lab
  • Optional: Ubuntu terminal

Steps

  1. Stop `camera_driver`, observe its state and CPU, then start it again.
  2. Stop `robot_controller` and write what the real arm must do (halt and hold).
  3. Keep `estop_watcher` running throughout. Why?
  4. In the pipe lab, find how many ERROR lines come from `camera`.
  5. On Ubuntu, run `process_drill.sh` and `graceful_stop.py`. Then open `top` and find your terminal’s bash process.

Interactive · Robot process monitor

Simulated ps-style view — 3/4 running. No real processes are started.

PROCESSROLECPU%MEMSTATE
robot_controllerSense → decide → act loop1248MRUN
camera_driverUSB camera /dev/video0 reader2296MRUN
telemetry_loggerJSONL joint / sensor logger0—STOP
estop_watcherSafety monitor — halt on fault18MRUN
# process monitor ready — start/stop simulated only

In the real drill the exit status should be 143 (128 + 15). If you see 137, something used SIGKILL (128 + 9).

Knowledge check

Check your understanding

Knowledge check

1. A process is…

2. Before killing processes on a live arm you should…

3. High CPU on camera_driver often causes…

4. `kill 4312` with no option sends…

5. A script stopped by SIGTERM reports exit status…

Mini assignment

Hands-on assignment

Mini assignment

Process map

Write the process map for your robot so anyone on the team can start and stop it safely.

Deliverables

  • Table of at least 4 processes: name, start command, what it uses (CPU/camera/serial), safe stop method
  • Stop order (with reason) for an emergency and for a normal shutdown
  • Your version of graceful_stop.py with one extra cleanup step

Do this, step by step

  1. In the process monitor, find the process eating CPU and stop it politely.
  2. Run process_drill.sh and confirm the exit status is 143.
  3. Run graceful_stop.py, press Ctrl + C, and see “got SIGINT”.
  4. Fill in your process-map table and the two stop orders.

Stuck? Hints, one at a time

NoteTry the task first. Read one hint, try again, and only then read the next one.
  1. Hint 1: ps aux | grep camera finds the PID; top shows who is using CPU.
  2. Hint 2: 143 = 128 + 15 (SIGTERM). 130 = 128 + 2 (SIGINT, Ctrl + C).
  3. Hint 3: Emergency stop order: motors first, then everything else. Normal shutdown: stop new commands, brake, then close ports.

Challenges (pick by level)

  • 🟢 Easy: Run sleep 100 &, find it with jobs and ps, then stop it.
  • 🟡 Medium: Add “close serial port” as an extra cleanup step in graceful_stop.py.
  • 🟠 Hard: Run a script inside tmux, detach, log out and reattach — it should still be running.

Next step

Look back: Lesson 4: Linux Permissions & Users. Next: Lesson 6: Package Management. Related: Stage 2 — Python + Robot Control.

Sources

Self-attested for now — keep artifacts in your lab notebook.

Complete

Today's goal

0 of 1 done today

0-day streak
Daily goal:lessons, quizzes or assignments a day

Mark this lesson complete when you have worked the experiment, knowledge check, and assignment (even roughly).

Stuck somewhere?

Ask a doubt about this lesson

Saved with this lesson's id and your email so we can reply. Privacy · or email contact@aihubstation.in