JDR AI started as a simple idea: a desktop assistant that could browse, talk, and actually see what's on my machine, instead of living behind a chat box in a browser tab. What it turned into was a much longer lesson in why desktop AI apps are harder to ship than web ones. This is the architecture I settled on, and the bugs that forced me there.
Most "desktop AI assistant" tutorials point straight to Electron because the web stack is familiar. I went with PyQt6 for two reasons: I wanted native system access (file indexing, process automation, system tray control) without shelling out to a separate backend process, and I wanted a single Python runtime handling both the UI and the AI logic so state stays in one place instead of being synced across an IPC boundary.
The trade-off is real — Qt's signal/slot system and thread model take longer to get right than a React component tree. But for an app that needs to touch the filesystem, spawn background threads for voice recognition, and embed a browser, it paid off.
JDR AI is built around four loosely-coupled modules that communicate through Qt signals rather than direct calls:
QWebEngineView instance that can be driven programmatically (navigate, inject scripts, read page state) so the assistant can act on web pages, not just talk about them.Keeping these on separate threads with signal-based communication meant a slow file scan or a stuck voice loop couldn't freeze the window — which was the single biggest usability win in the whole project.
The first serious bug: TTS worked fine when the app launched, then silently died after the user switched windows a few times. The cause was COM (Component Object Model) initialization scope. Windows' speech APIs are COM-based, and COM has to be initialized per-thread. My TTS calls were firing from whichever thread happened to trigger them — sometimes the main UI thread, sometimes the voice worker thread — and only one of those had properly initialized COM in apartment-threaded mode.
The fix: pin all TTS/STT calls to a single dedicated worker thread, call CoInitializeEx explicitly at that thread's start, and route every speech request through a queue into that thread rather than calling the speech API from wherever convenient.
This is a trap that's easy to fall into with any Windows automation feature exposed through pywin32 or comtypes — if you're not deliberate about which thread owns COM, it degrades intermittently instead of failing loudly, which makes it miserable to debug.
A "listen continuously" voice mode sounds simple until you have to handle: the user talking over the assistant, the assistant needing to interrupt itself mid-sentence when a new command comes in, and the microphone stream needing to pause during TTS playback (otherwise the assistant hears itself). I ended up with a small state machine — idle → listening → processing → speaking → idle — where each transition explicitly closes or reopens the microphone stream, instead of trying to run listening and speaking concurrently and filtering out feedback after the fact. State machines feel like overkill for a "simple" voice loop until you've tried to patch around the alternative.
The most frustrating bug to trace was a crash-on-close that only happened sometimes. It turned out to be a teardown order issue: QWebEngineView holds native Chromium resources, and if the parent window is destroyed before the web view's own cleanup runs, Qt can crash trying to deallocate the wrong thing. The fix was to explicitly call .stop() and schedule the web view's deletion before the main window's own closeEvent finished, rather than trusting Qt's default parent-child cleanup order to handle a component that wraps an entire external browser engine.
Getting a working dev environment into a distributable .exe was its own project. The two issues that cost the most time:
hiddenimports, or the packaged app fails at runtime with no build-time warning.QWebEngineView ships its own resource files (locales, ICU data) that PyInstaller doesn't bundle by default. Skipping this produces an app that runs fine on the dev machine and shows a blank white browser panel on anyone else's.The general lesson: test the packaged build on a clean machine early, not after the feature list is "done." Packaging bugs are invisible in a dev environment where all the dependencies already happen to be present.
None of these bugs were exotic — COM threading rules, Qt object lifecycle, and PyInstaller's static analysis limits are all documented. What made them costly was that each one failed silently or intermittently instead of throwing a clear error. If you're building something similar, the shortcut that would have saved me the most time is this: assume every native/system integration (speech, browser embedding, packaging) has thread- or lifecycle-ownership rules that aren't obvious from the API surface, and go looking for them before you build on top, not after something breaks in production.