Capstone 3 — BonicBot Confidence-Driven Personality

Learning Objective

Build a BonicBot application whose personality changes automatically, using YOLO object detection, confidence-based reasoning, a local LLM (Ollama) for persona-driven descriptions, and multi-voice text-to-speech (Piper) — all running together in a live, non-blocking loop.

Introduction

BonicBot can be configured to:

think with a fully local AI brain — no cloud service, listening through the mic and answering out loud with a local LLM served by Ollama
sound natural instead of robotic, by using a neural Piper voice, maintaining a continuous listen-think-speak pipeline
change personality dynamically, by using system prompts for different personas — like prompting BonicBot to answer as a pirate
combine vision with its AI brain, watching through the camera, recognizing objects with YOLO, and asking the local LLM to describe each one in a friendly sentence instead of just naming it

In this capstone project, you’ll combine all of these abilities into a single robot personality engine.

Project Scenario

Imagine BonicBot sitting on a desk, watching the room through its camera. Instead of naming everything it sees in a flat voice, it should react like it has moods — driven entirely by one signal that comes straight out of the detector, not by what the object actually is and not by a voice command:

Confidence — how sure YOLO is about the current detection.

From that signal alone, BonicBot picks one of three personas, split across two confidence cut points:

Band	Condition	Persona
1	`confidence >= CONFIDENT_THRESHOLD` (85%)	Confident
2	`CURIOUS_THRESHOLD <= confidence < CONFIDENT_THRESHOLD` (65%–85%)	Curious
3	`confidence < CURIOUS_THRESHOLD` (below 65%)	Unsure

Each persona now controls delivery (a spoken prefix like “I’m fairly sure that’s…”) and Piper voice settings, rather than using a different system prompt for each mood. The LLM itself generates a single factual use-phrase per object category. This keeps the robot’s factual knowledge completely stable, while its tone of voice and delivery shift automatically based on how confident the detection is.

In plain terms, here’s what happens every single frame:

Grab a frame and check what YOLO detected.
Pick the highest-confidence detection, skipping any class in IGNORED_CLASSES (like person) unless nothing else clears the confidence floor.
Wait for that same object to show up steadily for a few frames in a row, tracking the average confidence.
Once it’s confirmed, decide the persona purely from that average confidence.
Ask the local LLM for a factual use-phrase for the object (if we haven’t already cached it) — on a background thread, so the video window never freezes.
Combine the persona’s prefix with the factual phrase, and speak it out loud with that persona’s Piper voice.
Show the detection boxes and the latest line on screen.

Setup: Installing Packages

Before running the code, make sure your computer has the required Python packages and system libraries installed. Open a terminal and run:


pip install bonicbot-bridge opencv-python numpy ollama piper-tts pyaudio

What each package / tool does

Package / Tool	Purpose
`bonicbot-bridge`	The official BonicBot SDK. Connects to `BonicBot` to stream live camera frames (`bot.start_camera()`, `bot.get_image()`) and run YOLO object detection (`bot.enable_detection("yolo")`, `bot.vision.get_detections()`).
`piper-tts`	Fast, local neural Text-to-Speech (TTS) engine. Synthesizes voice audio streams for the active persona using ONNX voice models (`en_US-ryan-high`, `en_US-amy-medium`, `en_US-kathleen-low`).
`ollama`	Python client for local LLMs. Sends category names to `qwen3:0.6b` to generate short factual use-phrases.
`pyaudio`	Audio I/O library. Streams synthesized voice audio directly to your computer speakers.
`opencv-python`	Imported as `cv2`. Draws bounding boxes, confidence labels, and color-coded persona text banners.
`numpy`	Imported as `np`. Handles raw image arrays and matrix calculations.

If pip install fails, try pip3 install ... instead, or use a virtual environment:


python3 -m venv bonicbot-env
source bonicbot-env/bin/activate   # On Windows: bonicbot-env\Scripts\activate
pip install bonicbot-bridge opencv-python numpy ollama piper-tts pyaudio

How to run the program

Install Ollama (one-time setup):
- Windows / Mac: Download and run the installer from ollama.com/download .
- Linux: Run curl -fsSL https://ollama.com/install.sh | sh in your terminal.
Install PortAudio (system dependency for pyaudio):
- Linux: sudo apt install portaudio19-dev
- macOS: brew install portaudio
Pull the LLM model:
```
ollama pull qwen3:0.6b
```
Update IP address: Find your BonicBot’s IP address and set HOST = '[IP_ADDRESS]' (or 'localhost' for simulation).
Save the code into a file, e.g., capstone3_confidence_persona.py.
Run the script:
```
python capstone3_confidence_persona.py
```
Watch terminal logs as Ollama and all 3 Piper voice models (ryan, amy, kathleen) preload automatically.
Point the camera at objects in your room. When an object holds steady for 8 frames, BonicBot will map its average confidence score to one of 3 personas (Confident, Curious, Unsure), fetch a use-phrase from Ollama, and speak it aloud with matching voice settings and on-screen color coding!
Press ‘q’ in the video window to exit.

Don’t have a physical BonicBot? Try it in simulation (optional)

If you don’t have physical access to a BonicBot, you can still work through this capstone using the ROS 2 simulation environment:

Launch the BonicBot simulation:


ros2 launch my_bot robot_system.launch.py use_sim_time:=true world:=obsworld.sdf use_real_camera:=True

When HOST = 'localhost', BonicBot connects to the simulated camera feed and runs YOLO object detection, confidence-driven persona selection, Ollama LLM generation, and Piper TTS audio output seamlessly.

Code

Part 1 — Imports, Configuration, and Personas


import re
import shutil
import subprocess
import sys
import threading
import queue
import time
from pathlib import Path
 
import cv2
import ollama
import pyaudio
from piper import PiperVoice, SynthesisConfig
from piper.download_voices import download_voice
from bonicbot_bridge import BonicBot
 
# ============================== CONFIG =======================================
HOST = "[IP_ADDRESS]"
OLLAMA_MODEL = "qwen3:0.6b"
PIPER_VOICES_DIR = Path(__file__).resolve().parent / "piper_voices"
SERVER_START_TIMEOUT, SERVER_POLL_INTERVAL = 15.0, 0.5
 
DETECTION_CONFIDENCE_THRESHOLD = 0.35   # low bar so small handheld objects can compete with "person"
DESCRIBE_HOLD_FRAMES = 8                # frames an object must persist before it's "confirmed"
MISS_TOLERANCE_FRAMES = 5               # brief flicker tolerance
IGNORED_CLASSES = {"person"}            # never top pick unless it's literally the only thing seen
CONFIDENT_THRESHOLD, CURIOUS_THRESHOLD = 0.85, 0.65
MAX_TAIL_WORDS = 7                      # hard cap on the LLM's use-phrase -> guarantees one short line
EXIT_KEY = ord("q")
 
# The model now only fills in a short "typical use" phrase (object identity is
# injected from the YOLO class name in code, never generated — see compose_line).
# Low temperature + a small num_predict cost little for that narrow a task and
# buy a lot of factual consistency. num_predict still needs headroom for Qwen3's
# empty "<think>\n\n</think>\n\n" wrapper (emitted even with /no_think) before the
# real text starts — too low truncates inside the wrapper, not the answer.
LLM_OPTIONS = {
    "temperature": 0.3,
    "min_p": 0.05,
    "top_k": 40,
    "repeat_penalty": 1.3,
    "repeat_last_n": 64,
    "num_predict": 40,
}
THINK_TAG_RE = re.compile(r"<think>.*?</think>", re.S)  # belt-and-suspenders strip if any leaks through
 
# The model's only job now: name ONE well-known use for a bare category name — no
# shape, color, material, brand, size, or condition. Those were the real hallucination
# vectors (asked to "describe" an unseen object, a 0.6B model invents them freely;
# asked only for its category's textbook typical-use, it's drawing on real training
# data, not guessing). BANNED_WORDS is a cheap net for anything that slips through anyway.
SYSTEM_PROMPT = (
    "You label object categories factually. You are told ONLY a category name — never its "
    "actual color, material, brand, size, or condition, so never mention any of those. Reply "
    "with nothing but a short lowercase phrase (3-6 words) giving ONE well-known, general use "
    "for that category, e.g. 'used for holding hot drinks'. No sentence, no subject, no punctuation."
)
BANNED_WORDS = re.compile(
    r"\b(red|orange|yellow|green|blue|purple|pink|black|white|gray|grey|silver|gold|brown|"
    r"wood(en)?|plastic|metal|steel|glass|ceramic|leather|fabric|small|large|big|tiny|huge|"
    r"giant|new|old|shiny|glossy|matte|rusty)\b", re.IGNORECASE)
FALLBACK_TAIL = "used for its everyday purpose"  # always true; used whenever validation fails
 
# ============================== PERSONAS =====================================
# Personas now control DELIVERY only (how sure BonicBot sounds via the spoken
# prefix) — not content. The factual use-phrase is generated once per class and
# reused across all three, so tone can never change what's actually being claimed.
PERSONAS = {
    "confident": dict(display_name="Confident", prefix="That is, without a doubt, ",
                       voice_name="en_US-ryan-high", length_scale=0.95, volume=1.10),
    "curious": dict(display_name="Curious", prefix="Ooh, I'm fairly sure that's ",
                     voice_name="en_US-amy-medium", length_scale=1.0, volume=1.0),
    "unsure": dict(display_name="Unsure", prefix="Um, I think that might be ",
                    voice_name="en_US-kathleen-low", length_scale=1.1, volume=0.85),
}
PERSONA_COLORS = {"confident": (0, 255, 0), "curious": (0, 255, 255), "unsure": (0, 165, 255)}
 
def choose_persona(confidence):
    if confidence >= CONFIDENT_THRESHOLD:
        return "confident"
    if confidence >= CURIOUS_THRESHOLD:
        return "curious"
    return "unsure"

Replace [IP_ADDRESS] with the IP address of your own BonicBot. Every BonicBot may have a different IP address depending on your network configuration.

📌 Example:


HOST = "172.20.10.2"

Replace 172.20.10.2 with the IP address assigned to your BonicBot.

📦 Before running, install the extra libraries this capstone needs:


pip install ollama opencv-python numpy piper-tts pyaudio --break-system-packages

You’ll also need the Ollama app installed once from ollama.com/download — the script starts the server, downloads the LLM, and downloads every Piper voice it needs automatically after that, so no manual ollama or piper commands are required.

Part 2 — Ollama & Piper Setup Helpers


# ============================== OLLAMA / PIPER SETUP =========================
def ensure_ollama_ready(model_name):
    if shutil.which("ollama") is None:
        sys.exit("⚠️ Ollama isn't installed — get it from https://ollama.com/download and re-run.")
    try:
        ollama.list()
    except Exception:
        subprocess.Popen(["ollama", "serve"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
        start = time.time()
        while time.time() - start < SERVER_START_TIMEOUT:
            try:
                ollama.list()
                break
            except Exception:
                time.sleep(SERVER_POLL_INTERVAL)
    try:
        ollama.show(model_name)
    except Exception:
        print(f"Pulling '{model_name}' (one-time)...")
        for p in ollama.pull(model_name, stream=True):
            if p.get("status"):
                print(f"  {p['status']}")
    print("✅ Ollama ready.")
 
def preload_persona_voices():
    cache = {}
    for persona in PERSONAS.values():
        name = persona["voice_name"]
        if name in cache:
            continue
        model_path = PIPER_VOICES_DIR / f"{name}.onnx"
        cfg_path = PIPER_VOICES_DIR / f"{name}.onnx.json"
 
        def is_valid(p):
            return p.exists() and p.stat().st_size > 0
 
        for attempt in range(2):
            if not (is_valid(model_path) and is_valid(cfg_path)):
                print(f"Downloading voice '{name}' (~60MB, one-time)...")
                PIPER_VOICES_DIR.mkdir(parents=True, exist_ok=True)
                if model_path.exists(): model_path.unlink()
                if cfg_path.exists(): cfg_path.unlink()
                try:
                    download_voice(name, PIPER_VOICES_DIR)
                except Exception as e:
                    print(f"⚠️ Error downloading voice '{name}': {e}")
 
            try:
                cache[name] = PiperVoice.load(model_path, config_path=cfg_path)
                break
            except Exception as e:
                print(f"⚠️ Failed to load voice '{name}': {e}")
                if attempt == 0:
                    print("Retrying download...")
                    if model_path.exists(): model_path.unlink()
                    if cfg_path.exists(): cfg_path.unlink()
                else:
                    sys.exit(f"❌ Could not load voice '{name}' after retry.")
 
    print("✅ Voices ready.")
    return cache

Part 3 — Persona Selection & Object Debouncing


# ============================== YOLO DEBOUNCE =================================
class ObjectTracker:
    """Confirms a class only once it persists for `hold_frames`, and reports the
    AVERAGE confidence across that whole streak — not one noisy single-frame
    reading — so the persona choice and the LLM prompt both get a stable signal."""
 
    def __init__(self, hold_frames, miss_tolerance=MISS_TOLERANCE_FRAMES):
        self._hold, self._tol = hold_frames, miss_tolerance
        self._class = None
        self._streak = self._missed = 0
        self._conf_sum = 0.0
 
    def update(self, class_name, confidence):
        if class_name is not None and class_name == self._class:
            self._streak += 1
            self._missed = 0
            self._conf_sum += confidence
        elif class_name is not None:
            self._class, self._streak, self._missed, self._conf_sum = class_name, 1, 0, confidence
        else:
            self._missed += 1
            if self._missed > self._tol:
                self._class, self._streak, self._conf_sum = None, 0, 0.0
 
        if self._class is not None and self._streak == self._hold:
            return self._class, self._conf_sum / self._streak
        return None, None
 
 
def get_top_detection(detections, min_conf=DETECTION_CONFIDENCE_THRESHOLD):
    """Highest-confidence non-ignored detection wins; falls back to an ignored
    class (e.g. person) only if nothing else clears the confidence floor."""
    candidates = [d for d in detections if d.get("confidence", 0.0) >= min_conf]
    preferred = [d for d in candidates if d.get("class") not in IGNORED_CLASSES]
    pool = preferred or candidates
    return max(pool, key=lambda d: d.get("confidence", 0.0), default=None)

Part 4 — Background Worker (LLM + Speech)


# ============================== BACKGROUND WORKER =============================
def article_for(word):
    """Pick 'a' or 'an' for a bare category name (COCO class names are simple enough for this)."""
    return "an" if word[:1].lower() in "aeiou" else "a"
 
 
def clean_tail(raw, class_name):
    """Reduce raw LLM output to one short, validated use-phrase, or the safe fallback."""
    text = THINK_TAG_RE.sub("", raw).strip()
    text = re.split(r"[\n.!?]", text, maxsplit=1)[0]          # first clause only -> one line, guaranteed
    text = " ".join(text.split()).strip(' "\'.,').lower()
    text = " ".join(text.split()[:MAX_TAIL_WORDS])             # hard length cap regardless of model behavior
    if not text or BANNED_WORDS.search(text):
        return FALLBACK_TAIL
    return text
 
 
def compose_line(class_name, persona, tail):
    """Build the final line — class_name is code-injected fact; tail is the only generated part."""
    if class_name == "person":
        return persona["prefix"] + "someone standing in view, with nothing else standing out right now."
    return f"{persona['prefix']}{article_for(class_name)} {class_name}, {tail}."
 
 
class PersonaWorker:
    def __init__(self, model_name, voice_cache):
        self._model, self._voices = model_name, voice_cache
        self._pa = pyaudio.PyAudio()
        self._queue = queue.Queue(maxsize=1)
        self._lock = threading.Lock()
        self._latest = (None, "")
        self._tail_cache = {}  # class_name -> validated "typical use" phrase, shared by all personas
        threading.Thread(target=self._run, daemon=True).start()
 
    def request(self, class_name, persona_key):
        try:
            self._queue.put_nowait((class_name, persona_key))
        except queue.Full:
            pass
 
    def get_latest(self):
        with self._lock:
            return self._latest
 
    def _run(self):
        while True:
            class_name, persona_key = self._queue.get()
            persona = PERSONAS[persona_key]
            try:
                tail = "" if class_name == "person" else self._get_tail(class_name)
            except Exception as e:
                print(f"⚠️ LLM unreachable ({e}); using fallback description.")
                tail = FALLBACK_TAIL
            text = compose_line(class_name, persona, tail)
            with self._lock:
                self._latest = (persona_key, text)
            self._speak(text, persona)
 
    def _get_tail(self, class_name):
        if class_name in self._tail_cache:
            return self._tail_cache[class_name]
        messages = [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Category: '{class_name}'. /no_think"},
        ]
        try:
            resp = ollama.chat(model=self._model, messages=messages, options=LLM_OPTIONS, think=False)
        except TypeError:
            resp = ollama.chat(model=self._model, messages=messages, options=LLM_OPTIONS)
        tail = clean_tail(resp["message"]["content"], class_name)
        self._tail_cache[class_name] = tail
        return tail
 
    def _speak(self, text, persona):
        voice = self._voices[persona["voice_name"]]
        cfg = SynthesisConfig(length_scale=persona["length_scale"], volume=persona["volume"])
        print(f"BonicBot ({persona['display_name']}): {text}")
        stream = None
        try:
            for chunk in voice.synthesize(text, syn_config=cfg):
                if stream is None:
                    stream = self._pa.open(format=self._pa.get_format_from_width(chunk.sample_width),
                                            channels=chunk.sample_channels, rate=chunk.sample_rate, output=True)
                stream.write(chunk.audio_int16_bytes)
        finally:
            if stream:
                stream.stop_stream()
                stream.close()
 
    def shutdown(self):
        self._pa.terminate()

Part 5 — Drawing Helpers and Main Loop


# ============================== DRAWING HELPERS ================================
def draw_detections(frame, detections):
    h, w = frame.shape[:2]
    for d in detections:
        bbox = d.get("bbox")
        if not bbox or len(bbox) < 4:
            continue
        cx, cy, bw, bh = bbox
        x1, y1 = max(0, int((cx - bw / 2) * w)), max(0, int((cy - bh / 2) * h))
        x2, y2 = min(w - 1, int((cx + bw / 2) * w)), min(h - 1, int((cy + bh / 2) * h))
        cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
        cv2.putText(frame, f"{d.get('class', '?')} {d.get('confidence', 0.0):.0%}",
                    (x1 + 2, y1 - 4), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
    return frame
 
 
def draw_persona_bar(frame, persona_key, text):
    if not text:
        return frame
    h, w = frame.shape[:2]
    overlay = frame.copy()
    cv2.rectangle(overlay, (0, h - 46), (w, h), (0, 0, 0), -1)
    frame = cv2.addWeighted(overlay, 0.6, frame, 0.4, 0)
    label = f"[{PERSONAS.get(persona_key, {}).get('display_name', '?')}] {text}"
    cv2.putText(frame, label[:95] + ("..." if len(label) > 95 else ""), (10, h - 16),
                cv2.FONT_HERSHEY_SIMPLEX, 0.55, PERSONA_COLORS.get(persona_key, (255, 255, 255)), 1)
    return frame
 
 
# ============================== MAIN ============================================
def main():
    ensure_ollama_ready(OLLAMA_MODEL)
    voice_cache = preload_persona_voices()
    worker = PersonaWorker(OLLAMA_MODEL, voice_cache)
    tracker = ObjectTracker(DESCRIBE_HOLD_FRAMES)
    last_confirmed = None
 
    with BonicBot(host=HOST) as bot:
        bot.system.start_camera()
        bot.start_camera()
        bot.camera.wait_for_image(timeout=5.0)
        bot.enable_detection("yolo")
        while not bot.vision.yolo_enabled:
            time.sleep(0.1)
        print("✅ Live. Press 'q' to quit.")
        try:
            while True:
                frame = bot.get_image()
                if frame is not None:
                    detections = bot.vision.get_detections()
                    top = get_top_detection(detections)
                    class_name = top.get("class") if top else None
                    confidence = top.get("confidence", 0.0) if top else 0.0
                    confirmed_class, confirmed_conf = tracker.update(class_name, confidence)
 
                    if confirmed_class is not None and confirmed_class != last_confirmed:
                        persona_key = choose_persona(confirmed_conf)
                        print(f"🔎 {confirmed_class} @ {confirmed_conf:.0%} avg -> {persona_key}")
                        worker.request(confirmed_class, persona_key)
                        last_confirmed = confirmed_class
 
                    persona_key, latest_text = worker.get_latest()
                    display = draw_persona_bar(draw_detections(frame.copy(), detections), persona_key, latest_text)
                    cv2.imshow("BonicBot Vision — Confidence Personas", display)
 
                if cv2.waitKey(1) & 0xFF == EXIT_KEY:
                    break
        finally:
            cv2.destroyAllWindows()
            bot.disable_detection()
            bot.stop_camera()
            bot.system.stop_camera()
            worker.shutdown()
 
 
if __name__ == "__main__":
    main()

🔊BonicBot speaks through your computer’s default audio output via PyAudio in this capstone, not through the robot’s own speaker. Make sure your computer’s speakers or headphones are on before running the script.

Code Walkthrough

Line-by-line explanation

Imports & Configuration (lines 58–141) — Configures system parameters, thresholds (CONFIDENT_THRESHOLD = 0.85, CURIOUS_THRESHOLD = 0.65), SYSTEM_PROMPT, and PERSONAS dictionary mapping 3 confidence levels to spoken prefixes, Piper voice names, length scales, and volume levels.
Ollama & Piper Preloading (lines 176–236) — ensure_ollama_ready() starts the background Ollama daemon and pulls qwen3:0.6b. preload_persona_voices() downloads and loads all 3 required Piper ONNX voice models into memory on startup.
YOLO Debouncing & Class Filtering (lines 245–280) — ObjectTracker calculates average confidence over DESCRIBE_HOLD_FRAMES consecutive hits. get_top_detection() prioritizes non-ignored object classes over ambient person detections.
Async LLM Worker & Text Synthesis (lines 288–379) — PersonaWorker uses a background thread and a 1-element queue (queue.Queue(maxsize=1)). _get_tail() queries Ollama for short factual use-phrases (cached in _tail_cache). compose_line() injects the category name and prefix, while _speak() streams audio using PyAudio.
Visual Rendering Helpers (lines 388–413) — draw_detections() draws green bounding boxes; draw_persona_bar() overlays a color-coded persona banner at the bottom of the video window (Green = Confident, Yellow = Curious, Orange = Unsure).
Main Non-Blocking Loop (lines 417–464) — Initializes BonicBot camera streaming and YOLO detection, debounces detected objects, selects personas dynamically, triggers background worker requests, and renders visual output until q is pressed.

Expected Output

Click to see expected output

Visual Output:

📌 Note: This visual demonstration is using the ROS 2 simulation with use_real_camera:=True.

Terminal Output:


✅ Ollama ready.
✅ Voices ready.
✅ Live. Press 'q' to quit.
🔎 bottle @ 91% avg -> confident
BonicBot (Confident): That is, without a doubt, a bottle, used for holding liquids.
🔎 phone @ 74% avg -> curious
BonicBot (Curious): Ooh, I'm fairly sure that's a phone, used for making calls.
🔎 cup @ 52% avg -> unsure
BonicBot (Unsure): Um, I think that might be a cup, used for drinking beverages.

🔧 Under the Hood

How does BonicBot decide its mood, and why doesn’t the video freeze while it talks?

This project is built around two independent loops running at the same time: a fast main loop that owns the camera and the video window, and a slow background thread that owns the LLM call and the speech synthesis.

The main loop grabs a frame, reads YOLO’s detections, debounces them through ObjectTracker, and — the moment a class is newly confirmed — hands it off to PersonaWorker.request() and immediately moves on to drawing the next frame. It never waits for a response.
The background thread inside PersonaWorker pulls one request at a time off a queue, calls the local LLM, then synthesizes and plays the speech with Piper. Because the queue has maxsize=1, a burst of new detections can’t pile up requests faster than the worker can speak them — the newest confirmed object simply overwrites whatever was still waiting.

That split is why the video feed in Part 5’s main() keeps rendering smoothly even though a full LLM call plus Piper synthesis can easily take longer than a single video frame.

The persona itself is decided entirely by choose_persona(), which checks the confirmed detection’s average confidence against two thresholds:

Confident — confidence >= CONFIDENT_THRESHOLD (85%). BonicBot is sure enough to state it flatly.
Curious — CURIOUS_THRESHOLD <= confidence < CONFIDENT_THRESHOLD (65%–85%). A solid guess, but not certain.
Unsure — anything below CURIOUS_THRESHOLD. BonicBot hedges openly.

Each persona pairs a spoken prefix with the same kind of Piper settings you tuned in Lesson 12: length_scale changes speaking speed, volume changes loudness, and swapping voice_name gives some personas a genuinely different voice model rather than just a different pace. Piper doesn’t give us a direct pitch knob, so “tone” here is really an approximation built from speed, volume, and voice choice — not true pitch-shifting.

IGNORED_CLASSES and the two-tier fallback inside get_top_detection() exist for a separate reason: without it, a person filling most of the frame would almost always out-score a smaller object like a phone or bottle, and BonicBot would never get to react to anything else in the room.

Student Challenge

Extend the personality engine with a fourth persona: a "confused" mood that triggers when the top detection’s confidence is below DETECTION_CONFIDENCE_THRESHOLD but a box was still returned — right now those frames are silently ignored by get_top_detection().

Hint

Add a new entry to PERSONAS:


"confused": dict(display_name="Confused", prefix="Hmm, I honestly can't tell — ",
                 voice_name="en_US-lessac-medium", length_scale=1.15, volume=0.75),

Then loosen get_top_detection() so it also returns very low-confidence boxes instead of dropping them, and add a check at the front of choose_persona() — before the “confident” check — that returns "confused" whenever confidence is below some new CONFUSED_THRESHOLD.

Reflection Question

PersonaWorker uses a queue with maxsize=1, so if a new object is confirmed while BonicBot is still finishing its previous sentence, the old request is simply dropped in favor of the new one — BonicBot never queues up a backlog of things to say.

Why do you think that’s the right design here, instead of making every confirmed detection wait in line to be described eventually?