diff --git a/.claude/skills/copy-graham-blog-post/SKILL.md b/.claude/skills/copy-graham-blog-post/SKILL.md new file mode 100644 index 0000000..34e6d5a --- /dev/null +++ b/.claude/skills/copy-graham-blog-post/SKILL.md @@ -0,0 +1,107 @@ +--- +name: copy-graham-blog-post +description: Copies blog posts from Graham Dumpleton's blog (grahamdumpleton.me GitHub repo) to the Educates blog in this repo, rewriting first person singular to first person plural (we, the Educates team). Use when the user wants to copy, migrate, or add a post from Graham's blog to the educates site, or when given a GitHub URL to a post folder under GrahamDumpleton/grahamdumpleton.me. +--- + +# Copy Graham blog post to Educates site + +Copy a post (and its images) from [Graham Dumpleton's blog repo](https://github.com/GrahamDumpleton/grahamdumpleton.me) into this repo's `blog/` folder, and rewrite the text from first person singular (I, my) to first person plural (we, our) as the Educates team. + +## Source and target + +- **Source**: GitHub repo `GrahamDumpleton/grahamdumpleton.me`, path like `src/posts/YYYY/MM/post-slug/` (e.g. `src/posts/2026/02/deploying-educates-yourself`). +- **Target**: This repo's `blog/` folder. Create a folder named `YYYY-MM-DD-/` (e.g. `2026-02-26-deploying-educates-yourself`) containing `index.md` and any image files. Use the post's `date` from frontmatter for the folder date, or the path date if needed. Add `` after first paragraph. If first paragraph is too short, do it after the second. Also, review the tags to align with those in `blog/tags.yaml` and the topic in the blog. + +## Workflow + +### 1. List source files + +Call GitHub Contents API to list files in the post folder: + +``` +GET https://api.github.com/repos/GrahamDumpleton/grahamdumpleton.me/contents/src/posts/YYYY/MM/ +``` + +Or use `curl -sL "https://api.github.com/repos/..."`. From the response, identify `index.md` and any image files (e.g. `.png`, `.jpg`). Ignore `.DS_Store` and non-content files. + +### 2. Fetch post content + +Download the raw markdown: + +``` +https://raw.githubusercontent.com/GrahamDumpleton/grahamdumpleton.me/main/src/posts/YYYY/MM//index.md +``` + +Use `curl -sL ""` if needed (mcp_web_fetch may timeout). + +### 3. Rewrite voice (first person singular → plural) + +In the **body** (not code blocks or image alt text unless they are first-person narrative), replace: + +| Original (Graham, singular) | Rewritten (Educates team, plural) | +|----------------------------|------------------------------------| +| In my [last/previous] post | In our [last/previous] post | +| I showed / I walked / I pointed / I had to / I find / I have / I'm / I noticed / I tried / I could / I couldn't | we showed / we walked / we pointed / we had to / we find / we have / we're / we noticed / we tried / we could / we couldn't | +| my previous post / my last post | our previous post / our last post | +| being the author of the Educates platform | being the team behind the Educates platform | +| as far as I'm aware | as far as we're aware | +| I'd told it | we'd told it | + +Preserve "you" when addressing the reader. Do not change technical wording, code blocks, or link URLs except internal post links. + +### 4. Adapt frontmatter + +- Keep: `title`, `description`, `date`, `tags`, `image`, `draft`. +- Add: `slug: ` (e.g. `deploying-educates-yourself`). +- Add: `authors: [graham]`. +- Remove any frontmatter fields this site does not use (check existing posts in `blog/` for the schema). +- Remove `daft` and `date` frontmatter fields. + + +### 5. Fix internal links + +Replace Graham's post paths with this site's blog paths: + +- `/posts/YYYY/MM/slug/` or `/posts/2026/02/teaching-an-ai-about-educates/` → `/blog/` (e.g. `/blog/teaching-an-ai-about-educates`). +- Use `/blog/` consistently (no trailing slash required; add one if the site expects it). + +### 6. Adjust tags + +- Review tags in frontmatter section to align with the contents of the blog post and tags already defined in `blog/tags.yaml` + +### 6. Create target folder and files + +- Create `blog/YYYY-MM-DD-/` (e.g. `blog/2026-02-26-deploying-educates-yourself/`). +- Write the rewritten content to `blog/YYYY-MM-DD-/index.md`. +- Download each image from `https://raw.githubusercontent.com/GrahamDumpleton/grahamdumpleton.me/main/src/posts/YYYY/MM//` into the same folder. Image references in the markdown are relative (e.g. `training-portal.png`), so they resolve correctly. + +### 7. Verify + +- Confirm all images are present and paths in markdown match filenames. +- Spot-check that first-person singular in the body has been converted to plural and internal links use `/blog/...`. + +## Example + +User provides: `https://github.com/GrahamDumpleton/grahamdumpleton.me/tree/main/src/posts/2026/02/deploying-educates-yourself` + +- Post slug: `deploying-educates-yourself`. +- List contents → get `index.md`, `training-portal.png`, `workshop-dashboard.png`. +- Fetch `index.md`, rewrite I/my → we/our, add `slug` and `authors: [graham]`, change link to previous post to `/blog/teaching-an-ai-about-educates`. +- Create `blog/2026-02-26-deploying-educates-yourself/index.md` and download the two PNGs into that folder. + +## Reference: existing Educates post frontmatter + +```yaml +--- +title: "..." +slug: post-slug +description: "..." +date: YYYY-MM-DD +tags: ["educates", "ai"] +authors: [graham] +image: https://opengraph.githubassets.com/1/educates/educates-training-platform +draft: false +--- +``` + +Authors are defined in `blog/authors.yml`; `graham` exists. Use the same structure for new posts. diff --git a/blog/2026-02-24-fixed-ip-bridge-macos/index.md b/blog/2026-02-23-fixed-ip-bridge-macos/index.md similarity index 100% rename from blog/2026-02-24-fixed-ip-bridge-macos/index.md rename to blog/2026-02-23-fixed-ip-bridge-macos/index.md diff --git a/blog/2026-02-24-fixed-ip-bridge-macos/swiftbar-plugin.png b/blog/2026-02-23-fixed-ip-bridge-macos/swiftbar-plugin.png similarity index 100% rename from blog/2026-02-24-fixed-ip-bridge-macos/swiftbar-plugin.png rename to blog/2026-02-23-fixed-ip-bridge-macos/swiftbar-plugin.png diff --git a/blog/2026-02-24-teaching-an-ai-about-educates/air-framework-workshop.png b/blog/2026-02-24-teaching-an-ai-about-educates/air-framework-workshop.png new file mode 100644 index 0000000..84b1f37 Binary files /dev/null and b/blog/2026-02-24-teaching-an-ai-about-educates/air-framework-workshop.png differ diff --git a/blog/2026-02-24-teaching-an-ai-about-educates/index.md b/blog/2026-02-24-teaching-an-ai-about-educates/index.md new file mode 100644 index 0000000..57aefcc --- /dev/null +++ b/blog/2026-02-24-teaching-an-ai-about-educates/index.md @@ -0,0 +1,117 @@ +--- +title: "Teaching an AI about Educates" +slug: teaching-an-ai-about-educates +description: "Using an AI agent skill to generate interactive Educates workshops." +tags: ["ai", "educates", "authoring", "python"] +authors: [graham] +--- + +The way we direct AI coding agents has changed significantly over the past couple of years. Early on, the interaction was purely conversational. You'd open a chat, explain what you wanted, provide whatever context seemed relevant, and hope the model could work with it. If it got something wrong or went down the wrong path, you'd correct it and try again. It worked, but it was ad hoc. Every session started from scratch. Every conversation required re-establishing context. + +What's happened since then is a steady progression toward giving agents more structured, persistent knowledge to work with. Each step in that progression has made agents meaningfully more capable, to the point where they can now handle tasks that would have been unrealistic even a year ago. We've been putting these capabilities to work on a specific challenge: getting an AI to author interactive workshops for the [Educates](https://github.com/educates/educates-training-platform/) training platform. In our [previous posts](/blog/when-ai-content-isnt-slop/) we talked about why workshop content is actually a good fit for AI generation. Here we want to explain how we've been making that work in practice. + + + +## How agent steering has evolved + +The first real step beyond raw prompting was agent steering files. These are files you place in a project directory that give the agent standing instructions whenever it works in that context. Think of it as a persistent briefing document. You describe the project structure, the conventions to follow, the tools to use, and the agent picks that up automatically each time you interact with it. No need to re-explain the basics of your codebase every session. This was a genuine improvement, but the instructions are necessarily general-purpose. They tell the agent about the project, not about any particular domain of expertise. + +The next step was giving agents access to external tools and data sources through protocols like the Model Context Protocol (MCP). Instead of the agent only being able to read and write files, it could now make API calls, query databases, fetch documentation, and interact with external services. The agent went from being a conversationalist that could edit code to something that could actually do things in the world. That opened up a lot of possibilities, but the agent still needed you to explain what to do and how to approach it. + +Planning modes added another layer. Rather than the agent diving straight into implementation, it could first think through the approach, break a complex task into steps, and present a plan for review before acting. This was especially valuable for larger tasks where getting the overall approach right matters more than any individual step. The agent became more deliberate and less likely to charge off in the wrong direction. + +Skills represent where things stand now, and they're the piece that ties the rest together. A skill is a self-contained package of domain knowledge, workflow guidance, and reference material that an agent can invoke when working on a specific type of task. Rather than the agent relying solely on what it learned during training, a skill gives it authoritative, up-to-date, structured knowledge about a particular domain. The agent knows when to use the skill, what workflow to follow, and which reference material to consult for specific questions. + +With the advances in what LLMs are capable of combined with these structured ways of steering them, agents are genuinely reaching a point where their usefulness is growing in ways that matter for real work. + +## Why model knowledge isn't enough + +Large language models know something about most topics. If you ask an AI about Educates, it will probably have some general awareness of the project. But general awareness is not the same as the detailed, precise knowledge you need to produce correct output for a specialised platform. + +Educates workshops have specific YAML structures for their configuration files. The interactive instructions use a system of clickable actions with particular syntax for each action type. There are conventions around how learners interact with terminals and editors, how dashboard tabs are managed, how Kubernetes resources are configured, and how data variables are used for parameterisation. Getting any of these wrong doesn't just produce suboptimal content, it produces content that simply won't work when someone tries to use it. + +I covered the clickable actions system in detail in our [previous post](/blog/clickable-actions-in-workshops/). There are eight categories of actions covering terminal execution, file viewing and editing, YAML-aware modifications, validation, and more. Each has its own syntax and conventions. An AI that generates workshop content needs to use all of these correctly, not approximately, not most of the time, but reliably. + +This is where skills make the difference. Rather than hoping the model has absorbed enough Educates documentation during its training to get these details right, you give it the specific knowledge it needs. The skill becomes the agent's reference manual for the domain, structured in a way that supports the workflow rather than dumping everything into context at once. + +## The Educates workshop authoring skill + +The obvious approach would be to take the full Educates documentation and load it into the agent's context. But AI agents work within a finite context window, and that window is shared between the knowledge you give the agent and the working space it needs for the actual task. Generating a workshop involves reasoning about structure, producing instruction pages, writing clickable action syntax, and keeping track of what's been created so far. If you consume most of the context with raw documentation, there's not enough room left for the agent to do its real work. You have to be strategic about what goes in. + +The skill we built for Educates workshop authoring is a deliberate distillation. At its core is a main skill definition of around 25 kilobytes that captures the essential workflow an agent follows when creating a workshop. It covers gathering requirements from the user, creating the directory structure, generating the workshop configuration file, writing instruction pages with clickable actions, and running through verification checklists at the end. This isn't a copy of the documentation. It's the key knowledge extracted and organised to drive the workflow correctly. + +Supporting that are 20+ reference files totalling around 300 kilobytes. These cover specific aspects of the platform in the detail needed to get things right: the complete clickable actions system across all eight action categories, Kubernetes access patterns and namespace isolation, data variables for parameterising workshop content, language-specific references for Python and Java workshops, dashboard configuration and tab management, workshop image selection, setup scripts, and more. + +The skill is organised around the workflow rather than being a flat dump of information. The main definition tells the agent what to do at each step, and the reference files are there for it to consult when it needs detail on a particular topic. If it's generating a terminal action, it knows to check the terminal actions reference for the correct syntax. If it's setting up Kubernetes access, it consults the Kubernetes reference for namespace configuration patterns. The agent pulls in the knowledge it needs when it needs it, keeping the active context focused on the task at hand. + +There's also a companion skill for course design that handles the higher-level task of planning multi-workshop courses, breaking topics into individual workshops, and creating detailed plans for each one. But the workshop authoring skill is where the actual content generation happens, and it's the one we want to demonstrate. + +## Putting it to the test with Air + +To show what the skill can do, we decided to use it to generate a workshop for the [Air web framework](https://airwebframework.org/). Air is a Python web framework written by friends in the Python community. It's built on FastAPI, Starlette, and HTMX, with a focus on simplicity and minimal JavaScript. What caught our attention about it as a test case is the claim on their website: "The first web framework designed for AI to write. Every framework claims AI compatibility. Air was architected for it." That's a bold statement, and using Air as the subject for this exercise is partly a way to see how that claim holds up in practice, not just for writing applications with the framework but for creating training material about it. + +There's another reason Air makes for a good test. We haven't used the framework ourselves. We know the people behind it, but we haven't built anything with it. That means we can't fall back on our own knowledge to fill in gaps. The AI needs to research the framework and understand it well enough to teach it to someone, while the skill provides all the Educates platform knowledge needed to structure that understanding into a proper interactive workshop. It's a genuine test of both the skill and the model working together. + +The process starts simply enough. You tell the agent what you want: create me a workshop for the Educates training platform introducing the Air web framework for Python developers. The phrasing matters here. The agent needs enough context in the request to recognise that a relevant skill exists and should be applied. Mentioning Educates in the prompt is what triggers the connection to the workshop authoring skill. Some agents also support invoking a skill directly through a slash command, which removes the ambiguity entirely. Either way, once the skill is activated, its workflow kicks in. It asks clarifying questions about the workshop requirements. Does it need an editor? (Yes, learners will be writing code.) Kubernetes access? (No, this is a web framework workshop, not a Kubernetes one.) What's the target difficulty and duration? + +We'd recommend using the agent's planning mode for this initial step if it supports one. Rather than having the agent jump straight into generating files, planning mode lets it first describe what it intends to put in the workshop: the topics it will cover, the page structure, and the learning progression. You can review that plan and steer it before any files are created. It's a much better starting point than generating everything and then discovering the agent went in a direction you didn't want. + +From those answers and the approved plan, it builds up the workshop configuration and starts generating content. + +``` +lab-python-air-intro/ +├── CLAUDE.md +├── README.md +├── exercises/ +│ ├── README.md +│ ├── pyproject.toml +│ └── app.py +├── resources/ +│ └── workshop.yaml +└── workshop/ + ├── setup.d/ + │ └── 01-install-packages.sh + ├── profile + └── content/ + ├── 00-workshop-overview.md + ├── 01-first-air-app.md + ├── 02-air-tags.md + ├── 03-adding-routes.md + └── 99-workshop-summary.md +``` + +The generated workshop pages cover a natural learning progression: + +1. Overview, introducing Air and its key features +2. Your First Air App, opening the starter `app.py`, running it, and viewing it in the dashboard +3. Building with Air Tags, replacing the simple page with styled headings, lists, and a horizontal rule to demonstrate tag nesting, attributes, and composition +4. Adding Routes, creating an about page with `@app.page`, a dynamic greeting page with path parameters, and navigation links between pages +5. Summary, recapping concepts and pointing to further learning + +What the skill produced is a complete workshop with properly structured instruction pages that follow the guided experience philosophy. Learners progress through the material entirely through clickable actions. Terminal commands are executed by clicking. Files are opened, created, and modified through editor actions. The workshop configuration includes the correct YAML structure, the right session applications are enabled, and data variables are used where content needs to be parameterised for each learner's environment. + +![Workshop dashboard showing the Air framework workshop with instructions and clickable actions](air-framework-workshop.png) + +The generated content covers the progression you'd want in an introductory workshop, starting from the basics and building up to more complete applications. At each step, the explanations provide context for what the learner is about to do before the clickable actions guide them through doing it. That rhythm of explain, show, do, observe, the pattern we described in our earlier posts, is maintained consistently throughout. + +Is the generated workshop perfect and ready to publish as-is? Realistically, no. Although the AI can generate some pretty amazing content, it doesn't always get things exactly right. In this case three changes were needed before the workshop would run correctly. + +The first was removing some unnecessary configuration from the `pyproject.toml`. The generated file included settings that attempted to turn the application into an installable package, which wasn't needed for a simple workshop exercise. This isn't a surprise. AI agents often struggle to generate correct configuration for `uv` because the tooling has changed over time and there's plenty of outdated documentation out there that leads models astray. + +The second was that the AI generated the sample application as `app.py` rather than `main.py`, which meant the `air run` command in the workshop instructions had to be updated to specify the application name explicitly. A small thing, but the kind of inconsistency that would trip up a learner following the steps. + +The third was an unnecessary clickable action. The generated instructions included an action for the learner to click to open the editor on the `app.py` file, but the editor would already have been displayed by a previous action. This one turned out to be a gap in the skill itself. When using clickable actions to manipulate files in the editor, the editor tab is always brought to the foreground as a side effect. The skill didn't make that clear enough, so the AI added a redundant step to explicitly show the editor tab. + +That last issue is a good example of why even small details matter when creating a skill, and also why skills have an advantage over relying purely on model training. Because the skill can be updated at any time, fixing that kind of gap is straightforward. You edit the reference material, and every future workshop generation benefits immediately. You aren't dependent on waiting for some future LLM model release that happens to have seen more up-to-date documentation. + +You can browse the generated files in the [sample repository on GitHub](https://github.com/GrahamDumpleton/lab-python-air-intro). If you check the commit history you'll see how little had to be changed from what was originally generated. + +Even with those fixes, the changes were minor. The overall structure was correct, the clickable actions worked, and the content provided a coherent learning path. What would have taken hours of manual authoring to produce (writing correct clickable action syntax, getting YAML configuration right, maintaining consistent pacing across instruction pages) the skill handles all of that. A domain expert would still want to review the content, verify the technical accuracy of the explanations, and adjust the pacing or emphasis based on what they think matters most for learners. But the job shifts from writing everything from scratch to reviewing and refining what was generated. + +## What this means + +Skills are a way of packaging expertise so that it can be reused. The knowledge we've accumulated about how to author effective Educates workshops over years of building the platform is now encoded in a form that an AI agent can apply. Someone who has never created an Educates workshop before could use this skill and produce content that follows the platform's conventions correctly. They bring the subject matter knowledge (or the AI researches it), and the skill provides the platform expertise. + +That's what makes this different from just asking an AI to "write a workshop." The skill encodes not just facts about the platform but the workflow, the design principles, and the detailed reference material that turn general knowledge into correct, structured output. It's the difference between an AI that knows roughly what a workshop is and one that knows exactly how to build one for this specific platform. + +Both the workshop authoring skill and the course design skill are available now, and we're continuing to refine them as we use them. If the idea of guided, interactive workshops appeals to you, the [Educates documentation](https://docs.educates.dev) is the place to start. And if you're interested in exploring the use of AI to generate workshops for Educates, do reach out to me. diff --git a/blog/2026-02-25-deploying-educates-yourself/index.md b/blog/2026-02-25-deploying-educates-yourself/index.md new file mode 100644 index 0000000..2732927 --- /dev/null +++ b/blog/2026-02-25-deploying-educates-yourself/index.md @@ -0,0 +1,86 @@ +--- +title: "Deploying Educates yourself" +slug: deploying-educates-yourself +description: "Educates is open source, not a SaaS. Deploy it yourself, wherever you want." +date: 2026-02-26 +tags: ["educates", "installation", "local"] +authors: [graham] +image: https://opengraph.githubassets.com/1/educates/educates-training-platform +draft: false +--- + +In our [last post](/blog/teaching-an-ai-about-educates/) we showed how an AI skill can generate a complete interactive workshop for the Educates training platform. The result was a working workshop for the Air Python web framework, and you can browse the source in the [GitHub repository](https://github.com/GrahamDumpleton/lab-python-air-intro). But having workshop source files sitting in a repository is only half the story. The question that naturally follows is: how do you actually deploy it? + +If you've used platforms like Killercoda, Instruqt, or Strigo, the answer would be straightforward. You push your content to the platform, and it handles the rest. But that convenience comes with a trade-off that's easy to overlook until it bites you. + + + +## Not another SaaS platform + +The interactive workshop space has been dominated by SaaS offerings. Katacoda was one such option, but when it shut down in 2022, a lot of people lost workshops they'd invested significant time in creating. Killercoda emerged as a successor, but the fundamental dynamic is the same: you're reliant on a third party. Instruqt and Strigo are commercial services where you pay for access and have no control over how the platform evolves, what it costs next year, or whether it continues to exist at all. + +[Educates](https://github.com/educates/educates-training-platform/) takes a different approach. It's open source and self-hosted. You deploy it on your own infrastructure, which means you decide where it runs, when it runs, and who has access. Whether you're running private internal training for your team or hosting public workshops at a conference, you own the platform. If you want to modify how it works, you can. If you want to run it air-gapped inside a corporate network, you can do that too. That freedom is the point. + +## Deploy it where you want + +Because you're deploying Educates yourself, you get to choose the infrastructure. Educates runs on Kubernetes, and that Kubernetes can live anywhere. You could use a managed cluster from a cloud provider like AWS, GCP, or Azure. You could run your own Kubernetes on virtual machines or physical hardware. Or, if you just want to try things out on your own machine, the Educates CLI can create a local Kubernetes cluster for you using [Kind](https://kind.sigs.k8s.io/) running on Docker. + +That range of options means you can go from personal experimentation to production training without switching platforms. The same workshop content works regardless of where the cluster is running. + +For the rest of this post we'll walk through that local option, since it's the easiest way to get started and doesn't require any cloud infrastructure. + +## Creating a local Educates environment + +The Educates CLI handles the setup. If you have Docker running on your machine, creating a local Educates environment is a single command: + +``` +educates create-cluster +``` + +This creates a Kind-based Kubernetes cluster and installs everything Educates needs on top of it: the platform operators that manage workshop sessions, a local container image registry, and ingress routing so you can access workshops through your browser. It takes a few minutes to complete, but once it's done you have a fully functional Educates installation running locally. + +## Publishing and deploying a workshop + +With the local environment running, you can publish the Air workshop from a local checkout of the GitHub repository. From the workshop directory, run: + +``` +educates publish-workshop +``` + +This builds an OCI image containing the workshop files and pushes it to the local image registry that was set up alongside the cluster. The workshop definition file (which the AI skill created as part of the workshop in our previous post) already contains everything Educates needs to know about how to deploy and configure the workshop, so there's nothing else to set up. + +To deploy the workshop so it's available to use, run: + +``` +educates deploy-workshop +``` + +That's it. The workshop is now running on your local Educates installation and ready to be accessed. + +## Accessing the workshop + +To open the training portal in your browser, run: + +``` +educates browse-workshops +``` + +This opens the Educates training portal, which is where learners go to find and start workshops. + +![Training portal showing the available Air workshop](training-portal.png) + +From the portal, you select the workshop you want to run and click to start it. Educates spins up an isolated session for you, and after a moment you're dropped into the workshop dashboard. + +![Workshop dashboard showing the Air framework workshop with instructions, terminal, and editor](workshop-dashboard.png) + +The dashboard is the full Educates workshop experience. On one side you have the workshop instructions with their clickable actions. On the other side is the integrated environment with a terminal and editor, plus any additional dashboard tabs the workshop has configured (in this case, a browser tab for viewing the running web application). Everything the learner needs is right there, and it's all running on your own machine. + +The workshop we showed being generated in the last post works exactly as expected. You can click through the instructions, run the commands, edit the files, and see the Air web application running in the embedded browser tab. The entire guided experience functions just as it would on a production Educates installation. + +## Beyond Kubernetes + +Running Educates on a local Kubernetes cluster is the most straightforward way to try things out, and it gives you the complete platform experience. But Kubernetes isn't the only option for running a workshop. The same underlying container image used to run the workshop can also be run directly in Docker without any Kubernetes cluster at all. The Educates CLI supports this too. + +That said, running in Docker comes with trade-offs. Some features that depend on Kubernetes won't be available, and the workshop itself may need adjustments to work correctly in a Docker environment. We'll cover that in a future post, including how the AI skill from the previous post can help figure out what needs to change. + +If you want to try any of this yourself, the [Educates documentation](https://docs.educates.dev) has a more detailed quick start guide that covers installation and configuration options beyond what we've shown here. diff --git a/blog/2026-02-25-deploying-educates-yourself/training-portal.png b/blog/2026-02-25-deploying-educates-yourself/training-portal.png new file mode 100644 index 0000000..4cac4b1 Binary files /dev/null and b/blog/2026-02-25-deploying-educates-yourself/training-portal.png differ diff --git a/blog/2026-02-25-deploying-educates-yourself/workshop-dashboard.png b/blog/2026-02-25-deploying-educates-yourself/workshop-dashboard.png new file mode 100644 index 0000000..4042a98 Binary files /dev/null and b/blog/2026-02-25-deploying-educates-yourself/workshop-dashboard.png differ diff --git a/blog/2026-02-26-reviewing-workshops-with-ai/index.md b/blog/2026-02-26-reviewing-workshops-with-ai/index.md new file mode 100644 index 0000000..f120b84 --- /dev/null +++ b/blog/2026-02-26-reviewing-workshops-with-ai/index.md @@ -0,0 +1,68 @@ +--- +title: "Reviewing workshops with AI" +slug: reviewing-workshops-with-ai +description: "Having the Claude browser extension review an Educates workshop as a real user would." +date: 2026-02-28 +tags: ["educates", "ai", "authoring"] +authors: [graham] +image: https://opengraph.githubassets.com/1/educates/educates-training-platform +draft: false +--- + +In our [previous post](/blog/deploying-educates-yourself/) we walked through deploying an AI-generated Educates workshop on a local Kubernetes cluster. The workshop was up and running, accessible through the training portal, and ready to be used. But having a workshop that runs is only the first step. The next question is whether it's actually any good. + +Workshop review is traditionally a manual process. You open the workshop in a browser, click through each page, read the instructions, run the commands, check that everything works, and make notes on what could be improved. It's time-consuming and somewhat tedious, especially when you're the person who wrote the workshop in the first place and already know what it's supposed to do. Even this task, though, is one where AI can help. + + + +## Reviewing the source vs the experience + +One option would be to point Claude at the workshop source files directly. Hand it the Markdown content and the YAML configuration and ask it to review the material. This works to a degree, but it only checks the content in isolation. It doesn't tell you anything about how the workshop actually feels when someone uses it. + +The real test of a workshop is the experience of navigating it as a learner. How do the instructions read when you're looking at them alongside a terminal and editor? Does the flow between pages make sense? Do the clickable actions appear at the right moments, and is it clear what they do? These are things you can only assess by going through the workshop the way a learner would. + +So rather than having Claude review the source files, the better approach is to have it use the workshop in an actual browser. + +## Claude in the browser + +The Claude extension for Chrome can interact with web pages directly. It can read page content, click on elements, scroll, and navigate between pages. That makes it well suited to walking through an Educates workshop end to end. + +We pointed it at the Educates training portal with the Air workshop available, and gave it a prompt asking it to go through the workshop and provide a summary along with recommendations for improvement. + +![Prompting Claude to review the workshop in the browser](initial-request-prompt.png) + +The prompt needed to include some explanation of how Educates workshops work, because the browser extension doesn't appear to support Claude skills (as far as we're aware). That meant we had to tell it that the workshop has clickable action blocks in the instructions panel on the left side, and that it should click on each of those highlighted actions to execute the described tasks, just as a learner would. Without that context, Claude wouldn't know to interact with those elements. + +## Navigating the workshop + +Once Claude had its plan approved, it started working through the workshop. It navigated to the workshop catalog, started the workshop, and began reading through each page of instructions. It scrolled down through longer pages, clicked through to following pages, and because we'd told it about the clickable actions, it clicked on those too, triggering terminal commands and editor actions just as a human learner would. + +![Claude navigating the workshop and triggering clickable actions](trigger-clickable-actions.png) + +Claude worked through the entire workshop from start to finish, experiencing it the same way someone encountering it for the first time would. It isn't as quick as if it were reviewing the raw files directly, but it's still faster than a person doing the same thing. + +## The impartial reviewer + +Once Claude had gone through the whole workshop, it provided its review with a summary of what the workshop covers and recommendations for how it could be improved. + +![Claude's summary and review of the workshop](summary-and-review.png) + +What we find most valuable about this is that Claude is impartial in a way that human reviewers often aren't. We have high standards for workshops, being the team behind the Educates platform itself, and when we're reviewing workshops that others have written, if we feel the quality doesn't meet those standards, there's a social pressure not to be too blunt about it. You don't want to discourage people, so you temper your feedback. Most human reviewers do the same, holding back on the more critical observations to be polite. + +Claude doesn't have that constraint. It gives you the direct feedback, which in practice means you get more useful observations. The feedback covers multiple dimensions too: the quality of the written content, the overall flow of the workshop, how clickable actions are used and whether their usage is effective or confusing. It can even surface shortcomings in the Educates platform itself, pointing out places where the clickable action mechanisms could work better, which is feedback that's genuinely useful for improving the platform and not just the workshop. + +## Model selection matters + +One thing we noticed is that the quality of the review can vary significantly depending on which model you use. We tried both Sonnet and Opus, and somewhat counterintuitively, Sonnet produced better review output for this particular use case. Bigger doesn't always mean better when the task is focused and specific. + +The results can also vary between runs of the same model. Two reviews of the same workshop with the same model and similar prompting can surface different observations or emphasise different aspects. That's not necessarily a problem, but it does mean you should be prepared to iterate. Running the review more than once and comparing what comes back gives you a more complete picture than relying on a single pass. + +## What could be better + +This is a valuable tool, but there are several things that could make it better. + +The most impactful would be skills support in the browser extension. Because the extension doesn't support Claude skills, we had to manually explain how Educates workshops work in the prompt. If it supported skills, we could create a dedicated workshop review skill that encapsulates all the context Claude needs: how the workshop dashboard is laid out, what the different types of clickable actions are, how navigation works, and what to look for when reviewing. That would make the prompting much simpler and the review more consistent and thorough. + +The review output is also trapped in the browser extension's chat pane. It would be far more useful if Claude could save the analysis to a file, with screenshots linked to specific pieces of feedback so you can see exactly what the AI is referring to when it flags an issue. The extension does supposedly support screen recording to produce an animated GIF of the session, but we couldn't get that to work (possibly a macOS permissions issue). Even if it did work, a recording of the whole session is less useful than targeted screenshots tied to individual observations. + +The new Claude remote control feature could also be interesting for this. Claude can work through a workshop faster than a person would, but because the Chrome extension pauses at each step waiting for the browser to fully update, it can still take a while when working through a large workshop, or even a course made up of multiple workshops. During that time you're just watching it work. If you could access the chat remotely, even just with snapshots of what the browser is showing rather than a live view, you could wander away from your desk while you wait and still check on it if you need to. diff --git a/blog/2026-02-26-reviewing-workshops-with-ai/initial-request-prompt.png b/blog/2026-02-26-reviewing-workshops-with-ai/initial-request-prompt.png new file mode 100644 index 0000000..05ea165 Binary files /dev/null and b/blog/2026-02-26-reviewing-workshops-with-ai/initial-request-prompt.png differ diff --git a/blog/2026-02-26-reviewing-workshops-with-ai/summary-and-review.png b/blog/2026-02-26-reviewing-workshops-with-ai/summary-and-review.png new file mode 100644 index 0000000..b480288 Binary files /dev/null and b/blog/2026-02-26-reviewing-workshops-with-ai/summary-and-review.png differ diff --git a/blog/2026-02-26-reviewing-workshops-with-ai/trigger-clickable-actions.png b/blog/2026-02-26-reviewing-workshops-with-ai/trigger-clickable-actions.png new file mode 100644 index 0000000..1123c33 Binary files /dev/null and b/blog/2026-02-26-reviewing-workshops-with-ai/trigger-clickable-actions.png differ diff --git a/blog/tags.yml b/blog/tags.yml index e694f7d..228898f 100644 --- a/blog/tags.yml +++ b/blog/tags.yml @@ -96,4 +96,9 @@ ai: macos: label: macOS permalink: /macos - description: macOS \ No newline at end of file + description: macOS + +python: + label: Python + permalink: /python + description: Python \ No newline at end of file