Ephemeral Intelligence: Running On-Demand Copilot Agents in GitHub Workflows

The Challenge: Agent Configuration at Scale

Over the past few weeks, I hav been thinking about how to assess the architecture and technical health of every repository across an organization — think 100s of repos — and produce a standardized set of outputs: Tech stack summaries. Dependency inventories. Risk flags. Improvement opportunities. The kind of artifacts that, taken together, would give us a coherent picture of the whole landscape and a defensible starting point for modernization planning.

With GitHub Copilot I knew it could accelerate the analysis dramatically. But here was the real problem: every time I would need update the repo with the right set of prompts, skills, and instructions for architecture discover. It was a bit tedious: Copying files over. Tweaking instructions. Not to mentioned the maintainability of these assets.

That friction is what led me to APM — and it changed how I think about agentic workflows entirely.

📦 APM: Think 'npm' for Agent Primitives (custom agents, instructions, prompts, skills, etc.)

APM (Agent Package Manager) is an open-source tool that does for AI agent configuration what npm does for Node.js dependencies, or pip does for Python. Instead of telling every developer to manually set up their Copilot instructions and prompts, you declare them in a manifest file.

Simply run apm install, and APM resolves and installs everything: custom instructions (.instructions.md), skills (.skill.md), prompts (.prompt.md), agents (.agent.md), and more.

But here's the thing that really unlocked the use case for me: 💡agent primitives don't have to live permanently in the target repository. You can bring them in at workflow time, use them, and leave no trace. Ephemeral, intentional, and fully reproducible.

⚙️ The Setup: A Discovery Factory

The architecture I landed on has two moving parts:

1. A central discovery package — I have around 20 primitives I use for the repository discovery, I packaged them all into a repo-metadata-generator pack.

2. A reusable GitHub Actions workflow that can be dispatched against any target repository. It checks out the repo, installs Copilot CLI and APM, pulls in our discovery pack, runs the prompts, and uploads the results as artifacts.

You can check out the full workflow. Here are the interesting bits:

name: Generate Architect Metadata
...
jobs:
  discover:
    ....
    - name: Download repo-metadata-generator pack
      run: |
        curl -L -o repo-metadata-generator-1.0.0.tar.gz \
            "https://github.com/.../repo-metadata-generator-1.0.0.tar.gz"

    - name: Install GitHub Copilot CLI
      run: npm i -g @github/copilot

    - name: Install APM and Unpack Metadata Generator
      run: |
        curl -sSL https://aka.ms/apm-unix | sh
        apm unpack repo-metadata-generator-1.0.0.tar.gz

    - name: Run Documentation
      env:
        COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
      run: |
        set -euo pipefail

        copilot --agent repo-analyzer -p "/fleet Run all the prompts in the .github/prompts"
    ...

We're injecting our own discovery-specific manifest, pulling in exactly the primitives we need for this job, doing the analysis, and uploading the results. The target repo is unchanged. The workflow is stateless. You can run it against a repo today, improve the discovery pack tomorrow, bump the version, and re-run.

🔍 What's Inside the Discovery Pack

If you are interested, you can check out the contents of the repo-metadata-generator pack. But be aware this is only a sample pack.

🧠 The Mental Model Shift That Matters

What excites me about this pattern isn't the specific tools. It's what it represents.

We've spent years building reusable code: shared libraries, internal npm packages, Artifactory registries. We've invested heavily in CI/CD so that quality checks, security scans, and deployments are automated and consistent. But AI agent configuration has been the wild west — per-developer, per-project, copy-pasted, undocumented, and completely non-reproducible.

APM brings the same engineering discipline we apply to code dependencies to agent configuration. And when you combine it with GitHub Copilot CLI and GitHub Actions, you get agentic workflows that are portable, versioned, auditable, and scalable.

The same pattern extends naturally to:

• ✅ Automated PR review checklists — different skill packs for different service types (API, frontend, data pipeline)
• 🔒 Security audit workflows — triggered automatically when a new repo is created
• 📝 Release note generators — with org-specific tone, format, and audience targeting
• 🚀 Developer onboarding agents — that generate a tailored "getting started" guide for each codebase, pulled fresh every time

In every case: define your primitives once, package them, version them, let the workflow do the rest.