Lesson 2: Teaching BonicBot to Detect Faces

If you already installed these packages in Lesson 1, you don’t need to reinstall them — the same environment works for this lesson.

If pip install fails, try pip3 install ... instead, or use a virtual environment:

python3 -m venv bonicbot-env
source bonicbot-env/bin/activate   # On Windows: bonicbot-env\Scripts\activate
pip install bonicbot-bridge opencv-python numpy

1. Find your BonicBot’s IP address. Check the robot’s on-device display, its companion app, or your router’s connected-devices list.

2. Save the code below into a file, e.g. lesson2_faces.py.

3. Replace [IP_ADDRESS] in the code with your robot’s actual IP address (see the callout box under the code block).

4. Make sure your BonicBot is powered on and connected to the same network as your computer, and that at least one face can appear in front of its camera.

5. Run the script:

   python lesson2_faces.py

6. A window titled “BonicBot Vision” should open, showing colored boxes and landmarks around each detected face, plus a small stats panel in the top-left corner.

7. Watch the terminal — a new line is only printed when the number of visible faces changes.

8. Press q with the video window focused to stop the stream and exit cleanly.

1. Launch the BonicBot simulation:

   ros2 launch my_bot robot_system.launch.py use_sim_time:=true world:=obsworld.sdf use_real_camera:=True

   • use_real_camera:=True — the camera feed comes from your laptop’s webcam, while the robot’s body and motion are still simulated. This is the easiest way to test detection, since you can just hold whatever you want detected up to your laptop camera.
   • use_real_camera:=False — the camera feed comes from Gazebo instead (i.e. whatever the simulated camera sees inside the simulated world).

2. Once the simulation is running, use localhost as the host instead of a physical IP address:

   with BonicBot(host="localhost") as bot:

3. Everything else in the code — enabling face detection, drawing boxes, reading bot.get_faces() — works the same way, since the simulation exposes the same interface as a real robot. This path is mainly useful for exploring the lesson without hardware on hand; if you have a real BonicBot, connecting to its actual IP address is the recommended way to go through this lesson.

• Imports — time handles timestamps and FPS calculation; cv2 (OpenCV) draws boxes, labels, landmarks, and the HUD; numpy backs the image arrays; BonicBot is the SDK class used to connect to the robot.
• FACE_COLORS — A list of six distinct BGR colors. Each detected face is assigned one via i % len(FACE_COLORS), so colors repeat if more than six faces are visible at once.
• FLASH_DURATION / MIN_CONFIDENCE — Two tunable constants: FLASH_DURATION controls how many frames the border-flash effect lasts after the face count changes, and MIN_CONFIDENCE filters out any detection the model is less than 80% sure about.
• draw_faces(frame, faces) — For each face, reads its bounding box (handling both a {xmin, ymin, width, height} dict format and a plain [cx, cy, w, h] list format), converts normalized coordinates to pixels, and draws a colored rectangle, a filled label background, and the face number with confidence. It then loops over the face’s keypoints (eyes, nose, mouth, ears) and draws a small dot for each.
• draw_hud(frame, faces, fps, flash_frames_left) — Draws a semi-transparent black panel in the top-left corner using cv2.addWeighted, then writes the current face count and FPS on top of it. If flash_frames_left is greater than zero, it also draws a colored border around the whole frame — green if faces are present, red if the count just dropped to zero.
• with BonicBot(host='[IP_ADDRESS]') as bot: — Opens a connection to your robot, automatically closing it when the block ends.
• bot.system.start_camera() / bot.start_camera() / bot.camera.wait_for_image(...) — Powers on the camera, starts streaming, and waits (up to 5 seconds) for the first frame to arrive.
• bot.enable_detection('face') — Switches the robot’s vision pipeline from general object detection to the dedicated face-detection model.
• FPS tracking (prev_time, dt, fps = fps * 0.9 + ... * 0.1) — Measures the time between frames and smooths the frames-per-second estimate using an exponential moving average, so the displayed FPS doesn’t jitter wildly frame to frame.
• Change-triggered logging (if len(faces) != last_count:) — Compares the current face count to the last printed count. A line is only printed to the terminal when that count actually changes, keeping the output readable instead of flooding it every frame.
• flash_frames_left — Set to FLASH_DURATION whenever the face count changes, then counted down each loop iteration; this is what drives the temporary border-flash effect in draw_hud.
• Main loop — Grabs a frame, gets filtered faces, updates FPS and logging, draws faces and the HUD onto a copy of the frame, and shows it in the “BonicBot Vision” window.
• cv2.waitKey(1) & 0xFF == ord("q") — Checks each iteration whether q was pressed, breaking the loop if so.
• finally: cleanup block — Runs whether the loop exits normally or due to an error, ensuring face detection is disabled, the display window is closed, and the camera is stopped every time.

Visual Output:

Terminal Output:

📷 Starting vision system...
🧠 Loading face detection model...
✅ Vision system ready.

[14:08:23] 🙂 2 face(s) in view (99%, 96%)

The BonicBot vision window will display:

• A colored box around every detected face
• Face numbers (Face #1, Face #2, …)
• Detection confidence
• Six facial landmarks for each face
• A live face counter
• Current processing speed (FPS)
• A brief border flash whenever the number of visible faces changes

Every image received by BonicBot’s vision system is analyzed using MediaPipe Face Detection, a lightweight deep learning model designed for real-time robotics applications.

For every face the model finds, it predicts:

• a bounding box
• a confidence score
• six facial landmark points

The program converts these predictions into positions on the robot’s display, where each face receives:

• its own colored bounding box
• a unique face number
• confidence information
• landmark markers

The robot also maintains a small dashboard showing:

• the current number of detected faces
• the processing speed (Frames Per Second)

To avoid flooding the terminal with repeated messages, BonicBot only prints an event when the number of visible faces changes. This event-based logging is commonly used in robotics because it highlights important changes instead of repeating the same information hundreds of times every second.

The bounding box already contains the normalized center coordinates (cx_n, cy_n). Convert them into pixel coordinates using the display width and height before drawing them with cv2.putText().