Learning-Tool Landscape — AIEWF ’26 Field Course
Building a Personal Learning Tool for Technical AI Content: A Product & Learning-Science Landscape
TL;DR
- The online-learning market splits into ~12 product categories, but almost none serve your profile — an expert who learns by building and teaching. The categories that are evidence-aligned (spaced-repetition tools, interactive coding platforms, PKM/resurfacing) mostly ignore video; the categories built for video (MOOCs, course marketplaces, conference archives) mostly ignore the techniques that actually create durable learning.
- The learning science is unambiguous and directly buildable: retrieval practice, spacing, interleaving, generation/self-explanation, and “desirable difficulties” (Bjork) beat passive re-watching; worked examples help novices but hurt experts (the expertise-reversal effect), so your tool should skew toward retrieval and generation, not explanation. Gamification reliably boosts engagement but its effect on learning outcomes is inconsistent — steal Duolingo’s habit mechanics carefully.
- The highest-leverage thing you can build is a “mnemonic medium for conference talks”: a pipeline that turns AI Engineer World’s Fair talk transcripts into high-quality spaced-repetition prompts (Matuschak’s five-property standard) scheduled with FSRS, paired with a Socratic/Feynman generation loop and a build-a-demo forcing function. Your existing practice (building systems, giving meetup talks) already implements the two most powerful techniques in the literature — the protégé effect and project-based just-in-time learning. Build the tool to feed those, not to replace them.
Key Findings
- No existing product is designed for your use case. Every category optimizes for a market — beginners, credential-seekers, or B2B — that isn’t you. The gap is real and buildable.
- Passive video consumption is the single worst-supported input for durable learning, yet it’s your primary content source (conference talks). This is the core problem your tool should solve: converting watch-time into retrieval/generation events.
- Retrieval practice and spacing are the two best-evidenced techniques (Roediger & Karpicke; Cepeda et al.), and they are exactly what SRS tools (Anki, FSRS) implement — but those tools require manual card authoring, which is the friction an LLM can now remove.
- The expertise-reversal effect (Kalyuga/Sweller) means advanced-learner tools should look different from beginner tools: less explanation, more problem-solving and generation. Most AI-tutor products (Khanmigo) are built for novices and would bore you.
- Duolingo’s mechanics are a masterclass in habit formation but a cautionary tale on learning: they optimize CURR/DAU (engagement), and the gamification literature shows engagement and learning outcomes can diverge.
- You already do the two highest-value things — teaching (protégé effect) and building (project-based/JIT learning). The tool’s job is to make those cheaper and more frequent, and to plug the retention leak on the ~90% of talks you don’t turn into a demo or talk.
- The “AI + learning” space in 2024–2026 has matured enough that the components you need (transcript → LLM prompt generation → FSRS scheduling → Socratic dialogue → resurfacing) all exist as proven parts you can assemble.
Details
AREA 1 — Category Map of Online Learning Products
For each category: core mechanic, best-fit learning, and strengths/weaknesses for an advanced technical learner (you).
MOOCs (Coursera, edX, Udacity). Mechanic: university-style courses (video lectures + auto-graded quizzes + certificates). Best for: structured foundational knowledge and credentials. For you: too slow, too broad, pitched at novices; completion rates are notoriously low. Weak on cutting-edge AI-engineering content, which moves faster than course production cycles.
Cohort-based courses (Maven, On Deck, Reforge). Mechanic: time-boxed live courses with deadlines, expert instructors, and peer cohorts. Maven was co-founded by ex-Udemy president Gagan Biyani, who coined “cohort-based course”; it uses a 10% revenue-share model. Biyani’s own argument: “the accountability of deadlines and time crunch… is the number one thing that drives engagement,” plus live expert help and peer accountability. Best for: skills where accountability and feedback matter (management, writing). For you: the mechanics (deadlines, accountability, live expert unblocking, peer group that outlives the course) are worth stealing, but the format is expensive, timezone-locked, and rarely covers bleeding-edge AI systems topics. A critical academic analysis flagged concerns about transparency, surveillance, and over-reliance on instructors’ social-media status.
Video course marketplaces (Udemy, Pluralsight, Frontend Masters, egghead.io). Mechanic: pre-recorded video courses, sometimes with exercises. Frontend Masters (workshop-style, industry instructors, ~$399/yr) and egghead.io (very short 2–3 min lessons, ~$150–350/yr) are the most respected for intermediate-to-advanced developers. Frontend Masters explicitly targets “intermediate to advanced frontend developers” with an “Expert Learning Path” toward “Lead/Staff Developer.” For you: good for just-in-time skill acquisition on a specific tool; weak on retention (passive video), and catalog freshness lags for the newest AI topics. egghead’s concision is a genuine strength — dense, skimmable at 2×.
Interactive coding platforms (Codecademy, Exercism, DataCamp, Brilliant, executable notebooks). Mechanic: in-browser coding with immediate feedback; Exercism adds human mentorship; Brilliant uses interactive problem-solving. Best for: skill acquisition through doing (implements deliberate practice’s immediate-feedback loop). For you: Exercism’s mentored track and executable/interactive notebooks (Jupyter, observable, marimo) are the closest fit — they support active generation and immediate feedback, the mechanics that matter most. Most are still pitched below your level.
Spaced-repetition tools (Anki, RemNote, Mochi, SuperMemo). Mechanic: algorithmic flashcard scheduling to fight the forgetting curve. Anki (free, open-source, SM-2 historically) adopted FSRS as default scheduler in v23.10 (Nov 2023); FSRS-6 shipped in late 2025, trained (per Migaku’s technical overview) on “roughly 700 million reviews contributed by about 20,000 volunteer” Anki users, modeling three per-card variables (stability, difficulty, retrievability). On the open-spaced-repetition benchmark (~350M reviews across ~10,000 collections), FSRS “produces more accurate recall predictions than SM-2 for 99.5 percent of users tested” and “achieves the same retention as SM-2 with 20–30% fewer reviews.” (Parameter note: FSRS-4.5 used 17 trainable weights; FSRS-6 uses 21.) SuperMemo (Woźniak) remains the research pioneer; SM-20 (2026) computes all parameters via ML. For you: this is the single most evidence-aligned category and the engine your tool should use. The weakness is authoring friction — writing good cards is slow — which is exactly what an LLM removes. RemNote is notable because it embeds cards inside a networked note system and does 1-click YouTube→cards.
Microlearning/gamified apps (Duolingo, Sololearn). Mechanic: bite-sized lessons wrapped in game loops (streaks, XP, leagues). Best for: sustaining daily habits in low-intensity learning. For you: the content depth is far below your needs, but the habit architecture is the most refined in the industry and worth studying (see Area 3).
Documentation/tutorial ecosystems (official docs, MDN, dev.to, READMEs). Mechanic: reference + task-oriented tutorials. Best for: just-in-time problem-solving — the dominant mode for working engineers. For you: primary daily driver, but zero retention scaffolding; you look it up, solve it, forget it.
AI-tutor products (Khanmigo, ChatGPT Study Mode, Claude Learning mode, Socratic). Mechanic: LLM-driven Socratic dialogue that withholds direct answers. Khanmigo (built on GPT-4, fine-tuned on Khan Academy content) never gives answers directly. For you: Khanmigo is built for K-12 novices and would frustrate you; but ChatGPT Study Mode (launched July 29, 2025, built with input from pedagogy experts at 40+ institutions and Stanford’s SCALE Initiative) and Claude’s Learning mode / Claude Code Explanatory mode (2025) are directly relevant — the latter “pauses mid-task, inserting #TODO comments that prompt users to write 5–10 lines of their own code.” Critical caveat: pure Socratic chatbots have a documented engagement cliff (Stanford CEPA documented a ~60% engagement drop after three weeks without teacher facilitation), so dialogue alone won’t sustain a habit — it needs pairing with resurfacing.
Conference talk/video platforms (YouTube, conference archives). Mechanic: passive video, maybe with chapters/transcripts. Best for: exposure to frontier ideas and tacit knowledge (how experts think). For you: your primary content source and the format with the worst retention affordances — the central problem your tool addresses. Andy Matuschak’s “Why Books Don’t Work” argument applies doubly to talks: “neither medium has any explicit theory of how people actually learn things, and as a result, both mediums accidentally… evolved around a theory that’s plainly false” (he names it “transmissionism”).
Newsletter/curation-based learning (Substacks, TLDR, Lenny’s Newsletter, Import AI). Mechanic: expert curation + synthesis delivered on a schedule. Best for: staying current, triage. For you: efficient signal-filtering, but consumption is passive and non-retentive.
Community/Discord-based learning (Discords, Slack groups, meetups). Mechanic: social, real-time, question-driven learning with peers/experts. Best for: unblocking, tacit knowledge, motivation. For you: you already run one of these (your AI meetup) — this is a high-value channel you’re on the supply side of. The social accountability and teaching pressure are doing real pedagogical work.
Note-taking/PKM tools used for learning (Obsidian, Roam, Readwise, Logseq). Mechanic: networked notes; some (Readwise) add spaced resurfacing. Andy Matuschak’s key insight: “Evergreen note maintenance approximates spaced repetition” — regularly rereading and revising notes on active topics is a weak but real form of spacing and leverages the generation effect. Readwise’s Daily Review resurfaces highlights (including from YouTube transcripts) using “a decaying algorithm based on a recall probability half-life,” resurfacing once recall probability decays to ≤50%. For you: the resurfacing pattern (Readwise) and the generation-through-writing pattern (evergreen notes) are both directly stealable.
AREA 2 — What Actually Works for Learning (Evidence-Based)
Retrieval practice / the testing effect (Roediger & Karpicke 2006; Karpicke & Blunt 2011). The single most robust finding: actively recalling information strengthens long-term retention far more than re-studying. Karpicke & Blunt (2011, Science 331:772–775) found that “practicing retrieval produces greater gains in meaningful learning than elaborative studying with concept mapping,” and — critically — “the advantage of retrieval practice occurred even when the criterial test involved creating concept maps.” Production tests (short-answer, free recall) beat recognition (multiple-choice). Implemented by: Anki, RemNote, Brilliant, quiz features. Ignored by: video platforms, MOOCs (mostly), newsletters, docs.
Spaced repetition / the spacing effect (Cepeda et al. 2006; Ebbinghaus). Distributing practice over time beats massing (“cramming”). This is the algorithmic core of SRS. Implemented by: Anki/FSRS, SuperMemo, Readwise resurfacing, Duolingo’s Birdbrain. Ignored by: nearly everything else.
Interleaving (Rohrer; Kornell & Bjork 2008). Mixing problem types/topics beats blocked practice, because it forces discrimination between concepts. Bjork’s lab found the discrimination (“temporal juxtaposition”) is more critical than the spacing it introduces. Implemented by: well-designed problem sets; largely absent from course platforms, which block by module.
Elaboration & self-explanation (Chi; Fiorella & Mayer 2016). Explaining why and connecting to prior knowledge deepens encoding. Bisra, Liu, Nesbit, Salimi & Winne’s 2018 meta-analysis (“Inducing Self-Explanation,” Educational Psychology Review 30:703–725) identified 69 effect sizes from 64 reports (5,917 learners) and found “the overall weighted mean effect size using a random effects model was g = .55.” The effect is strongest when learners explain from memory without looking at source material. Implemented by: Socratic AI tutors (partially), good note-taking. Ignored by: passive media.
Generation effect (Slamecka & Graf 1978). Generating information yourself (vs. reading it) makes it more memorable. Underlies why writing your own notes/cards/explanations beats consuming others’. Implemented by: evergreen notes, Feynman-style writing, teaching.
Desirable difficulties (Bjork & Bjork 1994, 2011). Conditions that slow acquisition but improve long-term retention and transfer: spacing, interleaving, retrieval, generation, varied practice. The key trap is the “fluency illusion” — mistaking the ease of re-reading/re-watching for actual learning. Critical nuance: a difficulty is desirable only if the learner has enough scaffolding to engage it productively; past that line it’s just cognitive overload. This is the master principle for your tool: deliberately make review effortful.
Deliberate practice (Ericsson, Krampe & Tesch-Römer 1993). Expert performance traces to structured practice on specific weaknesses, with immediate feedback, repetition, and effort beyond one’s comfort zone — “effortful and not enjoyable.” Note the replication caveat: a 2019 pre-registered replication (Macnamara & Maitra, Royal Society Open Science) found the effect substantial but smaller than the original, and rejected the original’s “reject any important role for innate ability” claim. Implemented by: coding platforms with feedback, Exercism mentorship. Relevant for you: the immediate-feedback loop of running code / building demos is deliberate practice.
Worked examples & cognitive load theory (Sweller 1988; Kalyuga et al. 2003). Working memory is severely limited; instruction should manage intrinsic load and cut extraneous load. Worked examples help novices by reducing extraneous load. But the expertise-reversal effect is the key finding for you: techniques that help novices (worked examples, detailed guidance) become “redundant or even counter-productive” for experts, who learn more by solving problems themselves. Guidance should fade as expertise grows. Direct implication: your tool should give you problems and prompts, not explanations.
The learning-styles myth. The belief that matching instruction to a “visual/auditory/kinesthetic” style improves learning (“the meshing hypothesis”) is not supported by evidence. Pashler, McDaniel, Rohrer & Bjork (2008, Psychological Science in the Public Interest 9:105–119) concluded verbatim: “we conclude therefore, that at present, there is no adequate evidence base to justify incorporating learning-styles assessments into general educational practice.” The legitimate replacement concepts are: (1) learners do have preferences (which don’t predict better outcomes when matched); (2) prior knowledge is the individual difference that genuinely changes what instruction works (per the expertise-reversal effect); (3) working memory capacity and motivation type also matter. So “what’s my learning style?” is the wrong question; “what’s my current expertise level in this specific topic, and what format lets me generate/retrieve most?” is the right one.
Mapping to product categories. SRS tools (Anki/FSRS), Brilliant, and Exercism implement the evidence best. AI tutors implement self-explanation partially. MOOCs, course marketplaces, newsletters, docs, and conference video mostly ignore the evidence — they’re built around “transmissionism.” Duolingo implements spacing (Birdbrain) and retrieval but wraps them in engagement mechanics that can pull against learning (Area 3).
For expert vs. novice learners specifically: experts benefit from less guidance and more generation/retrieval/interleaving; novices need worked examples and scaffolding first. Since you’re an expert in your domain, your tool should default to retrieval, generation, and desirable difficulties, and reserve explanation for genuinely new subfields where you’re temporarily a novice.
AREA 3 — UX and Behavioral Techniques Used by Learning Products
Duolingo deep case study. Duolingo’s former CPO Jorge Mazal, writing on Lenny’s Newsletter, described the 2018–2021 gamification push: “we were able to increase CURR by 21%, which represents a reduction in the daily churn of our best users by over 40% and… led to an increase in our DAU of 4.5x.” The share of DAU with a 7-day+ streak rose “almost 3 times to more than half of our DAU.” By 2024 DAU grew from ~5M (2020) to over 40M. The mechanics: - Streaks — the most powerful mechanic; users who care about streaks are far more likely to return. Loss aversion is the engine. - Loss-aversion safety valves — Streak Freezes, Weekend Amulets, and repair mechanics prevent “streak-snapping” frustration from making the habit stressful. - Leagues/leaderboards — social competition with promotion/demotion, calibrated so users can realistically win. - XP, gems/currency, quests, badges — layered so each mechanic serves a distinct retention function for a different user segment/tenure. - Notification strategy — personalized, well-timed nudges (the “Duo” guilt-trip memes). - The Duolingo growth model — segments users into activity states (new, current, reactivated, resurrected, at-risk, dormant) and optimizes transition rates, decomposing DAU into movable sub-metrics like Current User Retention Rate (CURR). This lets teams target the “most causally promising” levers. - A/B testing culture — every mechanic is measured and iterated against retention data. - Adaptive difficulty (Birdbrain) — see below.
Onboarding & habit design. Best practice (Duolingo, per multiple analyses): defer sign-up until after a first “I did it!” win; make early levels easy wins to build self-efficacy; invest in sound/animation micro-rewards; aim for time-to-value within minutes.
Behavioral frameworks. - Fogg Behavior Model: B=MAP (Behavior = Motivation × Ability × Prompt). A behavior fires only when motivation, ability, and a prompt coincide. Design implication: make the target action easy and prompted. - Nir Eyal’s Hook Model (Trigger → Action → Variable Reward → Investment) is “essentially Fogg’s model with a feedback loop welded onto the end.” The Variable Reward stage imports Skinner’s variable-ratio reinforcement; the Investment stage adds a commitment/consistency loop that increases future prompt-responsiveness. - Variable rewards (Skinner) — unpredictable rewards drive the strongest engagement (and the strongest addiction concerns).
Adaptive learning / knowledge tracing algorithms. - Item Response Theory (IRT) — models P(correct) as a function of item difficulty and learner ability. - Bayesian Knowledge Tracing (BKT) — Corbett & Anderson (1990s); a hidden Markov model of skill mastery over time. - Duolingo’s Birdbrain (launched 2020, now V2) — uses “a flavor of logistic regression inspired by item response theory,” updating both exercise difficulty and learner proficiency after each exercise, then feeding a session generator that targets “Goldilocks difficulty” (not too easy, not too hard). Duolingo’s earlier Half-Life Regression (HLR) model reduced recall-prediction error by >45% vs. Leitner and produced a ~12% daily-engagement lift in A/B testing. - FSRS (for your tool) is the state-of-the-art open scheduler and the practical choice.
Where gamification helps vs. harms learning. The meta-analytic picture is genuinely mixed. A PRISMA review of 41 studies (2012–2023) found effectiveness “depends on the context” and varies by learning environment and student characteristics. A separate meta-analysis (41 studies, 49 samples, >5,000 participants) found a large overall effect (g = 0.822) on learning outcomes — but other reviews find gamification improves metacognition and engagement with “no substantial variations in learning outcomes,” and warn of undermined intrinsic motivation and unhealthy competition. The core critique — the engagement-vs-outcomes tension / “engagement farming” — is that products optimize the metric they can measure and monetize (DAU, streaks, session count), which is a proxy for learning, not learning itself. Duolingo optimizes CURR because retention drives revenue; whether streak-holders actually become fluent is a different question the business model doesn’t force it to answer. Implication for a personal tool: you have no incentive to engagement-farm yourself. Use habit mechanics (streaks, prompts, a small daily commitment) in service of effortful retrieval, and measure a learning proxy (retrieval success on delayed review), not just time-in-app.
AREA 4 — Understanding Your Personal Learning Taste and Style
Reframe away from “learning styles.” As established, style-matching is a myth. The individual differences that do matter and are worth self-assessing: - Prior knowledge (the big one) — determines whether you need explanation (novice mode) or should skip straight to problems/generation (expert mode). This varies per topic, not globally. - Working memory / cognitive load — manage how much new material you take on per session. - Motivation type (intrinsic vs. extrinsic; Self-Determination Theory) — intrinsic motivation (autonomy, mastery, curiosity) sustains long-term effortful learning far better than extrinsic rewards, and extrinsic gamification can crowd out intrinsic motivation. - Need for cognition — dispositional enjoyment of effortful thinking; high-NFC learners (likely you, given you build systems for fun) thrive on desirable difficulties and resent over-scaffolding. - Interest-driven vs. goal-driven — do you learn best chasing curiosity or shipping toward a deadline?
How advanced practitioners like you actually learn best. - Just-in-time (JIT) over just-in-case. Engineers learn concepts as needed to complete a meaningful project, which maximizes retention because the knowledge is immediately applied and connected. JIT depends on strong fundamentals + “learning how to learn” as a meta-skill. - Project-driven learning. Start building before you fully know how; hit a “stuck point” at the edge of your skills; your brain is then primed to learn the specific thing that unblocks you. You already do this with multi-agent systems and your GPU inference server. - Teaching-to-learn / the protégé effect. Preparing to teach and teaching improve the teacher’s own encoding and retention — the “protégé effect.” The Feynman Technique operationalizes this as a diagnostic: explain a concept in plain language from memory; wherever you stumble is exactly where your understanding is thin. Fiorella & Mayer found the explaining effect is strongest when generated from memory without notes. You already run this loop every time you give a meetup talk or a live code demo.
Reflection practices. Learning journals and the Feynman technique as a diagnostic make gaps visible (metacognition). Writing a précis or evergreen note forces the generation effect.
Concrete self-experiments you can run (N-of-1). You have an ideal setup — you consume talks, build systems, and give talks — so you can instrument yourself:
- Format retention A/B. For the next ~10 AI Engineer World’s Fair talks, randomly assign each to one of two conditions: (A) watch only; (B) watch + write 5–10 retrieval prompts immediately after. Two weeks later, do a free-recall test (write everything you remember) for each talk and score it. Predicted result per the literature: condition B dramatically outperforms. This directly tests whether the retrieval-prompt pipeline is worth building.
- Spacing vs. massing. Take two comparable new subtopics (e.g., two inference-optimization techniques). Study one in a single 90-min block; study the other in three 30-min sessions across a week. Test both after two weeks.
- Generation vs. consumption. For half of new topics, write your own explanation/notes first (generation); for the other half, read/watch a good explanation. Compare delayed recall and — more importantly — whether you can use it in code.
- Teaching forcing-function. Commit to turning one talk per month into a 10-minute meetup segment or live demo. Track subjective fluency and retention for “taught” vs. “untaught” talks — this quantifies your own protégé effect and tells you how much of your learning should route through teaching.
- Desirable-difficulty calibration. Track review-session “success rate” in your SRS. If you’re above ~90% you’re not being challenged enough (too easy, low storage-strength gain); tune toward a lower retention target so retrieval is effortful.
- Interest vs. goal tracking. For a month, log each learning session as curiosity-driven or goal-driven, plus a 1–5 energy/completion rating. See which mode you actually finish and enjoy — then design the tool’s defaults around your dominant mode.
Recommendations
What to build: a “mnemonic medium for conference talks.” The strongest primary-source blueprint is Andy Matuschak & Michael Nielsen’s mnemonic-medium work. Their “Quantum Country” (quantum.country) essay embeds spaced-repetition prompts inline in prose and schedules them by email on expanding intervals; the first essay (“Quantum computing for the very curious”) has 112 questions. Their informal results (Matuschak’s 2022 Patreon letter): “after half an hour of practice most readers can remember the answers to almost all of the essay’s 112 questions across intervals of at least 2 weeks; after an hour, at least 5 weeks; after 1.5 hours, at least 9 weeks” — exponential retention gain per unit practice, at a “35–50% reading time overhead.” (Matuschak explicitly hedges these as informal, self-reported data — worth noting.) His open-source platform Orbit (withorbit.com, github.com/andymatuschak/orbit) is a ready starting point for embedding prompts in text; note his caveat that “Orbit is primarily a vehicle for research; its direction is determined by Andy Matuschak and direct collaborators.”
Staged plan:
Stage 1 (weekend build) — Transcript → prompts → FSRS. Pipeline: pull the talk transcript (YouTube/conference) → prompt an LLM (Claude/GPT) to generate retrieval-practice cards following Matuschak’s five properties of good prompts (focused, precise, consistent, tractable, effortful) → import to Anki (FSRS) or a lightweight custom store. This is the 80/20: it converts your worst-retention input (video) into your best-evidenced technique (spaced retrieval). Off-the-shelf comparables to study/borrow from: AnkiDecks and StudyGlen (YouTube→FSRS cards), Scholarly (transcript-grounded conceptual cards — “questions probe concepts and relationships, not rote recall”), RemNote (1-click YouTube→cards inside a note graph), and Readwise Reader (ingests YouTube transcripts + Ghostreader AI card generation + half-life resurfacing). - Benchmark to proceed to Stage 2: run self-experiment #1; if watch+prompt beats watch-only on your two-week recall test, keep building.
Stage 2 — Add a generation/Socratic layer for depth. For talks that matter, add a Feynman-diagnostic loop: the tool asks you to explain the talk’s core idea from memory, then an LLM (via ChatGPT Study Mode / Claude Learning mode patterns) probes gaps Socratically and only then surfaces the source. This layers the generation effect + self-explanation (g≈0.55) on top of retrieval. Because you’re an expert, bias the LLM toward problems and probing questions, not explanations (expertise-reversal effect).
Stage 3 — Add a build-forcing-function and teaching hook. For the highest-value talks, the tool should prompt: “What’s the smallest demo that would prove you understood this?” and track it. Tie this to your meetup: a monthly “one talk → one demo/segment” commitment operationalizes project-based JIT learning + the protégé effect — the two techniques you’re already best at.
Stage 4 — Add habit mechanics carefully, and resurfacing. Borrow from Duolingo selectively: a daily review prompt (Fogg’s B=MAP: make it easy + prompted), a lightweight streak with a forgiveness mechanic (streak-freeze) to avoid stress, and Readwise-style resurfacing of your best notes/prompts. Do not import leaderboards or variable-reward slot-machine loops — you have no reason to engagement-farm yourself, and extrinsic gamification risks crowding out the intrinsic motivation that actually sustains expert learning. Measure a learning proxy (delayed retrieval success), not time-in-app.
Thresholds that change the plan: - If Stage 1’s card quality from LLMs is poor for dense technical talks (caption/jargon errors are a known failure mode — auto-captions are ~90–95% accurate for clear English and degrade on jargon and accents), invest in better transcription (Whisper-large on your GPU server) before adding features. - If you find you rarely review (habit fails), the bottleneck is behavioral, not content — prioritize Stage 4’s prompt/streak design and time-to-value over more generation features. - If delayed-recall gains are marginal even with prompts, shift weight toward Stage 3 (building/teaching), since for you the protégé effect and JIT project learning may simply dominate flashcards.
What to steal from each product: Anki/FSRS (the scheduler — don’t reinvent it); Matuschak’s Orbit + prompt-writing standards (the intellectual core); Duolingo (streak + forgiveness + B=MAP onboarding, nothing more); Readwise (resurfacing + AI card generation from highlights, YouTube-transcript ingestion); Maven (the accountability/deadline principle — recreate it via your meetup); ChatGPT Study Mode / Claude Learning mode (the Socratic generation layer, tuned for experts).
Caveats
- Some engagement/retention statistics come from company blogs and PM accounts (Duolingo’s 4.5× DAU and CURR/churn figures via ex-CPO Jorge Mazal; Birdbrain’s 45%/12% figures) rather than peer-reviewed sources; treat them as directionally true but self-reported, not independent findings.
- The mnemonic-medium retention numbers are explicitly informal and self-reported (Matuschak’s own hedge). The strongest claim (“if you do the reviews, you’ll remember”) has good evidence; the claim that it generalizes across all domains has, in his words, “no evidence yet.”
- Deliberate practice’s effect size is contested — the 2019 replication found it real but smaller than Ericsson’s original, and the “innate ability doesn’t matter” claim is rejected by later work.
- Gamification’s effect on learning outcomes is genuinely mixed across meta-analyses; large positive effect sizes and null-outcome findings coexist, so don’t assume game mechanics will improve your learning even if they improve your engagement.
- Learning science is largely studied on novices and students, not expert practitioners learning frontier technical material from talks. The expertise-reversal effect is the best-evidenced reason to expect your optimal design differs from mainstream edtech, but the specific N-of-1 experiments above are how you’ll get ground truth for you.
- AI-generated cards/summaries carry hallucination and “cognitive offloading” risk — if the AI does the synthesis, you lose the generation effect. Keep yourself in the loop as the generator, not just the reviewer.