Learning Graphics Programming with AI: From Zero to Game Engine
Didiet Noor demonstrates how to use AI (Gemini, Claude) to learn computer graphics programming by building a bouncing-ball physics engine. Key workflow: prepare documentation first, break tasks into tiers, use AI for code generation with clear context, validate with unit tests, and always read and understand the generated code. Covers WebGPU, scene graphs, FFT visualization, and practical tips for job applications using AI.
Learning Mandarin with AI: Context-First Approach
HSK Curriculum as AI Training Data
HSK (Chinese proficiency test) materials are open-source and free, making them ideal as grounding data for AI language models. By providing HSK grammar and vocabulary lists as context before asking for exercises, the model avoids hallucination and stays within a defined curriculum rather than generating random content.
Domain-Specific Vocabulary Adaptation
Standard language curricula (HSK, TOEFL) teach everyday vocabulary, but professionals need domain-specific terms. Didiet used AI to adapt HSK2 grammar with engineering vocabulary (e.g., 'database' in Taiwanese vs mainland Chinese), creating personalized learning materials that match his actual work context rather than generic textbooks.
Pronunciation Cannot Be Fully Automated
AI cannot effectively teach Mandarin pronunciation and tones because it lacks visual feedback of mouth movements and cannot correct ear training in real time. Didiet still relies on human tutors for this specific skill, using AI only for grammar, writing, and vocabulary where feedback is text-based.
Reading Comprehension as the Bottleneck
Most people skip reading AI-generated content and assume it is correct. True learning requires reading every line to understand the logic. Didiet emphasizes that in the AI era, reading comprehension is the critical skill—without it, you cannot validate or learn from generated code or text.
Graphics Programming Workflow: Documentation-First Method
Always Start with Documentation, Not Code
Before writing any graphics code, Didiet creates a detailed document with formatting, mermaid diagrams, and tier breakdowns. This serves as a learning guide and context for the AI, ensuring generated code is well-structured and documented from the start rather than trying to document messy code afterward.
Tier-Based Task Decomposition
Complex projects are broken into tiers (T0: blank screen with GPU context, T1: clear screen, T2: draw triangle, T3: draw cube, T4: add physics). Each tier has a clear scope and deliverable. This prevents the AI from getting lost in complexity and allows incremental validation at each step.
Unit Testing for Graphics Code
Graphics code is hard to test because output is visual, not console-based. Didiet uses known test values (e.g., FFT input/output pairs from documentation) and validates them with unit tests before running the full visualization. This catches logic errors early without needing to see the rendered output.
Avoid Loops; Use Clear Goals Instead
AI loop features (auto-refinement) often produce worse code over time because the model loses context and drifts. Instead, define a clear goal upfront, generate code once with full context, validate it, and manually iterate if needed. This prevents the 'infinite refinement spiral' that degrades output quality.
AI Model Choice and Code Generation Strategy
Gemini Excels at Script Writing; Claude Better for Coding
Didiet uses Gemini primarily for generating scripts and educational content because it handles narrative structure well. Claude (via Vertex API) is used for complex coding tasks like graphics engines and physics simulations. Neither is universally superior; tool choice depends on task type.
Provide Full Context Before Code Generation
Never ask AI to generate code without context. Always include: the problem statement, existing code, domain knowledge, performance constraints (e.g., 'use 128 MB RAM only'), and the tier scope. With context, the model generates production-quality code; without it, output is generic and often wrong.
Use Indonesian to Save Tokens
Didiet writes documentation and prompts in Indonesian (his native language) rather than English to reduce token usage on multilingual models. This is a practical cost-saving measure when working with paid APIs like Vertex or Claude.
Code and Documentation in Same Directory
Keeping code and mermaid diagrams in the same repository folder makes it easy for AI to reference both during generation and for humans to review later. Separated files (like Java projects) force the AI to lose context and produce worse results.
Practical Graphics Programming Examples
WebGPU as the Modern Graphics API
WebGPU is the newest web graphics standard, replacing WebGL. Didiet chose it because recent AI models have better training data on WebGPU than older APIs, resulting in fewer errors and better code generation. Older APIs like Direct3D require more manual refinement.
FFT for Audio Visualization
Fast Fourier Transform converts audio signals into frequency data for visualization. Didiet used AI to generate FFT code with optimization: skip square root (slow), use log scaling instead, and divide by 2 for the same visual result. This shows how domain knowledge improves AI-generated code.
Scene Graphs for Hierarchical Rendering
Scene graphs organize 3D objects in a tree structure where child transforms are relative to parents. Moving a parent (e.g., a torso) automatically moves all children (arms, legs). Didiet used AI to generate a fast array-based scene graph instead of pointer-based, reducing memory and improving cache performance.
Virtual Lists for UI Rendering
Rendering 100 UI elements when only 25 are visible wastes RAM and CPU. Virtual lists render only visible items and update as the user scrolls. Didiet had AI generate this pattern for a console UI in a graphics engine, preventing RAM overflow.
Resident Evil-Style Game Built in 12 Hours
Didiet built a full 3D game prototype (inventory system, scene graph, real-time and baked lighting, item pickup) in 12 hours using AI code generation. The same project would take 2 weeks if coded manually. The code is 'good enough' for learning and prototyping, not production.
Debugging Graphics Code Without Console Output
Render Debug Information to Screen
Graphics engines have no console output because the GPU is busy rendering. To debug, you must render debug data (logs, values, wireframes) directly to the game screen. Didiet had AI generate a HUD (heads-up display) that shows console-like output overlaid on the 3D viewport.
Screenshot Feedback Loop for AI Debugging
When code produces visual artifacts, Didiet takes a screenshot and sends it to the AI with a description of the problem. The AI can analyze the visual output and suggest fixes. This creates a feedback loop that works even though graphics code has no text-based error messages.
Job Application Strategy Using AI
Tailor Resume to Each Job, Not Generic
Most job seekers use the same resume for all applications. Instead, maintain a large master CV (3–6 pages) with all projects and skills, then use AI to generate a tailored resume for each job posting. Match the job description keywords and required skills to your experience.
Use LaTeX for Resume, Not Word
LaTeX resumes are cleaner, version-controllable, and easier to automate. Didiet uses AI to convert resume text to LaTeX, then compiles it. This allows him to automate resume generation for multiple job applications via scripts.
Automate Job Application Pipeline
Use scripts to scrape job postings, generate tailored resumes via AI, compile LaTeX, and push to GitHub or email. Recruiters use AI too; automating your application process levels the playing field and saves hours of manual work.
Provide Context When Submitting AI-Generated Code
In take-home coding interviews, if you use AI to generate code, explicitly state it in commit messages and explain your role in driving the solution. Interviewers care more about your ability to read, understand, and refactor code than to write it from scratch.
Hiring and Junior Developer Expectations
Trust Is Given, Not Earned (in the AI Era)
In the past, you could assess a junior by assigning a task and seeing if they completed it. Now, everyone uses AI, so you cannot tell who actually understands the code. Didiet's approach: assume everyone uses AI and evaluate based on code review and on-site interviews where they must explain their reasoning.
Reading Comprehension Is the New Bottleneck
Most juniors can generate code with AI but cannot read or understand it. The ability to read documentation, understand generated code, and identify why it works (or doesn't) is now the critical skill. Juniors who skip reading will plateau quickly.
Juniors Must Understand the 'Why', Not Just the 'How'
Many web developers use React hooks (useEffect, useReducer) without understanding when or why to use them. They copy-paste code and it works, but they cannot optimize or debug. Didiet expects juniors to read documentation and ask questions to build conceptual understanding, not just functional code.
The Junior Dilemma: Investment vs. Productivity
Investing in junior developers is slow and expensive, but without investment, there will be no senior developers in the future. Didiet acknowledges this dilemma but has no solution; it depends on company resources and tolerance for slower output.
Notable quotes
Never start promoting without including context. — Didiet Noor
I don't care what they use, what's important is me. I can code so I can just relax. — Didiet Noor
Reading comprehension is the key that Indonesia in general is really weak at. — Imre Nagi
Action items
- Create a master CV (3–6 pages) with all your projects, skills, and experience, then use AI to generate tailored resumes for each job application.
- For graphics programming projects, start with documentation (mermaid diagrams, tier breakdown) before writing any code; use this as context for AI code generation.
- Break complex projects into tiers with clear scope and deliverables; validate each tier with unit tests before moving to the next.
- When using AI to generate code, always provide full context: problem statement, performance constraints, existing code, and domain knowledge.
- Read and understand all AI-generated code before using it; ask the AI to explain any part you don't understand.
- For graphics debugging, render debug information (logs, values) directly to the screen since there is no console output.
- Use LaTeX for resumes and automate resume generation and job application pipelines with scripts.
- In coding interviews, explicitly state if you used AI and focus on explaining your understanding of the code and your role in driving the solution.