Vision
Build a VS Code extension that asks developers focused quiz questions derived strictly from official documentation. The extension should feel lightweight and useful for daily learning without turning into a gamified tracking platform.
Goals
- Ask ad-hoc questions about languages and libraries
- Ask one daily question to encourage consistent practice
- Use retrieval-augmented generation (RAG) to ground questions in docs
- Use AI to evaluate answers against the same source material
- Avoid user profiles, scores, and progress tracking
Architecture
The extension can be kept modular with a small set of focused components:
VS Code Extension ├─ Commands ├─ Documentation Retriever ├─ Chunking/Index ├─ RAG Retrieval ├─ AI Question Generator └─ AI Answer Evaluator
Suggested Stack
| Component | Technology |
|---|---|
| Extension | TypeScript |
| AI | OpenAI, Claude, Gemini, or Ollama |
| Parsing | markdown-it, cheerio, pdf-parse |
| Index | Local vector index |
Core Commands
- Developer Quiz: Ask Question
- Developer Quiz: Daily Question
- Developer Quiz: Configure Sources
- Developer Quiz: Refresh Documentation
Workflow
- User selects a library
- Extension retrieves relevant documentation chunks
- AI generates one question from retrieved docs only
- User submits an answer
- AI evaluates the answer against the same documentation
- Extension cites the documentation section used
Prompting Principles
- Never invent APIs
- Use only supplied documentation
- Cite the documentation used
These constraints are critical. They keep the generated quiz content trustworthy and make answer evaluations auditable for learners.
Future Enhancements
- Context-aware questions based on the currently open file
- Support for multiple AI providers through a common adapter
- Expanded documentation support for Markdown, HTML, PDF, and GitHub docs
Conclusion
This concept is a strong fit for developers who want short, documentation-grounded practice inside their editor. By combining RAG retrieval with strict prompting rules, the extension can stay practical, transparent, and focused on learning instead of scoring.