Python Learning Chatbot

A desktop app (PyQt5) for learning Python: browse 39 lessons across Beginner, Intermediate, and Advanced levels, safely execute a student's exercise code in a sandbox, grade it by comparing actual program output against a reference answer (not by comparing source text), and get help from an AI tutor that's grounded in the course content and verifies its own code examples before showing them.

Screenshots

Dashboard — level progress, overall completion ring, exercise stats, and a personalized "topics to practice" panel:

Exercise with error feedback — a runtime error is caught by the sandbox and shown with a clean traceback:

Exercise passed — the sidebar checkmarks track completed lessons per level:

Setup & running

python3 -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt
python3 chatbot.py

Requirements: Python 3.9+. On Linux, the copy button also needs xclip or xsel installed (sudo apt install xclip).

AI Tutor setup (optional but recommended)

The "Ask AI" tab needs a free Groq API key (no credit card required): console.groq.com → API Keys → create one.

Copy .env.example to .env and add your key:

GROQ_API_KEY=your_key_here

Semantic search embeddings are computed once and cached locally (ai_tutor/lesson_embeddings.pkl) using a free, fully local sentence-transformers model — no API cost for retrieval, only for the actual chat responses (Groq's free tier is generous enough for normal use).

Architecture

project/
├── .github/
│   └── workflows/
│       └── tests.yml     # Runs the test suite automatically on every push (CI)
├── .gitignore
├── .env.example           # Template for the Groq API key — copy to .env
├── screenshots/           # Images used in this README
├── chatbot.py             # UI only (PyQt5) — no grading/AI logic lives here
├── lessons.py              # Lesson content: 39 lessons (explanation, sample code, exercise, reference answer)
├── grader/
│   ├── sandbox.py          # Isolated execution via subprocess (no external dependency)
│   ├── sandbox_docker.py   # Isolated execution via a Docker container (industry standard)
│   └── grading.py          # Compares actual stdout against the reference answer
├── ai_tutor/
│   ├── embeddings.py        # Builds + caches sentence embeddings for all lessons
│   ├── retriever.py         # Semantic search: finds the most relevant lesson(s) for a question
│   └── qa.py                 # Hybrid RAG + LLM call + code-verification ("Output Validation Layer")
├── progress/
│   └── db.py               # SQLite: records attempts + "which topics are hardest" stats
├── tests/
│   ├── test_sandbox.py
│   ├── test_sandbox_docker.py  # Real tests (need Docker) + a graceful-fallback test
│   ├── test_grading.py     # Includes an integration test against the real lessons.py
│   ├── test_db.py
│   ├── test_retriever.py    # Semantic search ranking (unit) + real-model integration test
│   └── test_qa.py            # Hybrid routing, markdown cleanup, and code verification (~40 tests)
└── requirements.txt

Deliberate separation: chatbot.py contains no grading, SQL, or AI logic — it's UI only. That means grader, progress, and ai_tutor can all be tested without ever starting PyQt (and that's exactly how all the tests work).

Running the tests

python3 -m unittest discover tests -v

No pytest needed — everything runs on the standard-library unittest module. Over 100 tests total, covering the sandbox, grading, progress database, and the AI tutor (retrieval ranking, hybrid routing, and code verification).

Two backends for safe code execution

The project supports two ways to run student code in isolation:

	grader/sandbox.py (default)	grader/sandbox_docker.py
External dependency	None	Requires Docker installed
Fork-bomb prevention	❌ None (see below for why)	via cgroups --pids-limit (enforced even for root)
Network	Not blocked	Fully disabled (--network none)
Filesystem	Only the working directory is isolated	Fully read-only
Good for	Fast development, environments without Docker	Real (production) deployment

To switch to Docker, set this environment variable before running:

docker pull python:3.12-slim   # only needed once
export SANDBOX_BACKEND=docker
python3 chatbot.py

If Docker isn't installed or running, the system returns a clear error message instead of crashing ("Docker is not installed or its daemon isn't running...").

Why grade on output, not on source text?

Two pieces of code that are logically equivalent can look completely different in text (variable names, spacing, range(n) vs. range(0, n)). Instead of comparing text, both the student's code and the reference answer are run in the same sandbox, and their stdout is compared.

Known limitation: exercises with non-deterministic output (e.g. printing elapsed time) need a mask_patterns entry in lessons.py so that part is ignored before comparison (example: the "Decorators" lesson). Purely graphical exercises (like Tkinter) are marked with "auto_gradable": False, since they never produce comparable stdout.

If a student is stuck after two failed attempts, a "Show Solution" button appears (with a confirmation prompt first) that reveals the reference answer — this is deliberately gated behind repeated failure rather than always visible, so it's a last resort, not the first thing a student reaches for.

AI-Powered Tutor (RAG)

Each lesson has an "Ask AI" tab where students can ask free-form questions and get an answer grounded in the course content — not just the model's general training data.

How it works

1. Retrieval: the question is embedded (via sentence-transformers, running fully locally — no API call) and compared against embeddings of all 39 lessons, using cosine similarity.
2. Context selection: the lesson currently open in the UI is always offered as context alongside anything semantic search finds, since "which lesson the student is on" is often a stronger signal than the wording of a short follow-up question (e.g. "show another example of this").
3. Generation: the question, plus any relevant course context, is sent to an LLM (Groq's openai/gpt-oss-120b). A single prompt lets the model itself decide whether to answer from the course material or fall back to general knowledge — clearly labeled either way — rather than using two rigid, separate code paths for these two cases (an earlier, stricter version of this caused off-topic questions asked inside an unrelated lesson to get stuck saying "not covered" without ever actually answering).

Output Validation Layer — never trusting the model's own claims

The same principle grader/sandbox.py uses for students — never trust a claim about what code does, actually run it — is applied to the model's own answers:

• Any code the model writes must be in a fenced (```) Python block (enforced by the prompt). Each block is extracted from the raw response and actually executed through the same sandbox used for grading students.
• All Python blocks in one answer are run as one combined script (each wrapped in its own try/except), not in isolation — so a later block can use a function or variable defined by an earlier one (matching how a student would actually paste a multi-block example into one file), while an error in one block (including a deliberate one, like demonstrating a KeyError) doesn't prevent independent later blocks from still being verified.
• The model's own inline "claimed output" comments (e.g. # -8) are stripped from the displayed code using Python's tokenize module (not a naive regex, so a # inside a string literal is never mistaken for a comment) — an unverified claim has no place once the system already has the means to verify it. Only the real, executed result is shown, labeled plainly as Output:.
• Blocks explicitly tagged with a different language (e.g. ```javascript, used legitimately when a question is genuinely about another language) are excluded from verification — this project's sandbox only runs Python.

This exists because manual testing found real, repeated cases of the model confidently miscalculating expressions (e.g. claiming (x + y) * (x // y) - (x % y) ** 2 evaluates to -8 when it actually evaluates to 3) — LLMs are not reliable calculators, and an AI feature that can silently be wrong about code is worse than no AI feature at all.

Known limitations (honest, not just marketing)

• The grounded label means "context was offered", not "the model necessarily used it." For a genuinely off-topic question asked while viewing an unrelated lesson, the model is still given that lesson as context and decides for itself whether it's relevant — it correctly declines to force-fit an answer, but the UI doesn't currently distinguish "used" from "was offered but set aside."
• AI answers are not persisted per lesson. Switching lessons while a response is still pending discards that response (handled safely via a request-id check in chatbot.py, so it's never shown on the wrong lesson), and returning to a lesson does not restore its previous question/answer, even from earlier in the same session.
• The embeddings cache doesn't auto-invalidate. If lessons.py content changes, ai_tutor/lesson_embeddings.pkl must be deleted (or build_lesson_index() re-run) manually.
• Code verification only covers fenced Python blocks. A block the model writes without triple backticks, or explicitly tagged as another language, isn't checked — the model could still be wrong about something it describes only in prose, or about non-Python code shown for a genuinely off-topic question.
• The "don't show a separate expected-output block" instruction is a prompt request, not a hard guarantee — like any LLM instruction, compliance isn't 100%. If the model does show one anyway, it would be executed as if it were more code and likely fail with a SyntaxError that has nothing to do with whether the actual example was correct.

Security notes (honest, not just marketing)

grader/sandbox.py runs student code in a separate subprocess with CPU, memory, and wall-clock timeout limits. This level of isolation is acceptable for fast development or environments without Docker, but it has known limitations:

• ⚠️ It does not defend against a fork bomb. This was not a design choice made lightly — it went through two real, failed fixes first. A fixed RLIMIT_NPROC of 1 silently broke any exercise using the threading module (since Linux counts threads against this limit too, not just forked processes). Computing the limit dynamically (current process count for the user + a fixed headroom) still failed on GitHub Actions, because RLIMIT_NPROC counts every process/thread for that user ID system-wide, not just this subprocess's own descendants — and that system-wide baseline turned out to vary unpredictably across environments. Neither failure ever showed up in this project's own development environment, because it ran as root, and root is exempt from RLIMIT_NPROC entirely — which is exactly why it took two different real, non-root runs (once crashing the dev environment directly, once failing in CI) to uncover both problems. Rather than guess a third number, this backend now leaves RLIMIT_NPROC alone and relies on sandbox_docker.py for real fork-bomb protection.
• ⚠️ Never run this as root/Administrator anyway. Even without RLIMIT_NPROC, root is exempt from RLIMIT_CORE/RLIMIT_NOFILE too. In production this should run as an unprivileged system user.
• This sandbox does not fully block network or filesystem access.
• On Windows, environment variables passed to the sandboxed subprocess are deliberately minimal (PATH, LANG, and SystemRoot) to avoid leaking secrets like API keys into student code. SystemRoot specifically has to be kept even though it isn't sensitive — Windows networking (and therefore asyncio) fails to initialize without it, which only surfaced once a lesson actually exercised that code path.

grader/sandbox_docker.py closes these gaps: --network none fully disables networking, --read-only makes the filesystem read-only, and — most importantly — --pids-limit provides real fork-bomb protection via cgroups, which (unlike rlimit) is scoped to the container itself rather than the whole machine, so it doesn't inherit the system-wide-baseline problem above. This is the same approach real code-judging platforms use (like the open-source project Judge0). This backend is recommended for real deployments, and is the only one of the two that meaningfully protects against a fork bomb.

• On Windows (with the default backend), the resource module doesn't exist; only the wall-clock timeout remains as protection.

Known UI limitation

Running the sandbox (for grading, or for AI code verification) can take a few seconds. To avoid freezing the UI, this work runs on a separate QThread (GradingWorker for exercises, AIWorker for the AI tutor), not on the main UI thread. Both use a request-id pattern to safely discard a stale response if the student has already moved on to a different exercise or lesson by the time it arrives.

Analytics

progress/db.py records student attempts in SQLite (without storing the full code — just its length and the result), and get_struggling_topics returns the topics with the lowest success rate (with a minimum-attempts threshold, so a single random failure doesn't wrongly get flagged as the "hardest" topic).

Development notes

A selection of real issues found and fixed while building this project — documented here because they reflect genuine engineering problem-solving, not just the finished result. A fuller write-up of each (with root cause, fix, and what it taught) lives in rag-project-challenges.md.

• A deprecated model dependency broke on day one. Groq deprecated llama-3.3-70b-versatile in favor of openai/gpt-oss-120b — a reminder that third-party model availability isn't stable.
• A Windows-specific PyTorch/PyQt import-order bug. Importing torch after PyQt5 caused a DLL initialization failure (pytorch/pytorch#166628). Fixed by importing torch first, at the very top of chatbot.py.
• A retrieval/generation conflict only found through manual, not automated, testing. Making the current lesson always count as valid context (to handle vague follow-ups like "show another example") had a side effect: genuinely off-topic questions got force-fit into a context-only prompt and never actually got answered. A reminder that automated tests catch regressions in logic you've already specified — exploratory manual testing is what finds gaps in the logic itself.
• The AI's own code needed the same skepticism as a student's. See "Output Validation Layer" above — built after manual testing caught the model confidently miscalculating simple expressions.