A Comment Is a Job Submission: A 9-Test Coding-Agent Admission Controller

Use GitHub's native trust boundary first

GitHub's managed automation boundary is the right default when its constraints fit. On August 3, GitHub added Copilot cloud-agent automations that run when an issue or pull-request comment is created and matches configured comment text. That turns repository conversation into an execution trigger. (GitHub Changelog)

The native path already carries useful controls:

• Automations are available only in private or internal repositories.
• A user needs write access to create an automation.
• Events from users without write access are ignored by default.
• The selected tools limit what the automation can do, and the automation is scoped to one repository.
• Each run consumes GitHub Actions minutes and GitHub AI Credits billed to the automation creator.
• Pull requests and code are attributed to that creator, who cannot approve the resulting pull request.
• Workflows on an agent pull request wait for approval from a user with write access.

Those are product facts, not guarantees that every custom rollout is safe. GitHub also exposes a switch that allows events from users without write access. Once that switch is enabled, or when a team builds its own GitHub App, webhook receiver, or issue_comment workflow, the team owns an additional admission boundary. (About Copilot automations)

The first decision is therefore simple:

Build a custom admission path only when the workflow must accept a broader author set, combine multiple repositories, apply a separate cost ledger, select among tool profiles, or dispatch to an agent runtime outside the managed automation.

A trigger phrase matches text; it does not grant authority

The phrase /agent run answers one question: did a string match? It does not answer whether the sender may spend budget, expose repository context, use a write-capable tool, execute code from a fork, or request this class of task.

Treat the comment as untrusted input even when the sender is trusted. A maintainer account can be compromised. A copied stack trace can contain instructions. A pull-request branch can contain code written by an external contributor. Authorization to comment on a repository is not automatically authorization to run that branch with credentials.

GitHub's Actions security documentation names the dangerous shape directly. An issue_comment workflow that fetches and executes untrusted pull-request code can create the same privileged-code problem as a misused pull_request_target workflow. The trusted workflow and secret-bearing token do not make the checked-out code trusted. (Securely using pull_request_target)

Separate three authorities:

1. Trigger authority: may this actor ask for an agent job in this repository?
2. Runtime authority: which code, tools, network paths, credentials, and budget may this job use?
3. Release authority: who may approve the checks and merge the resulting change?

One account should not silently inherit all three. GitHub's cloud-agent controls preserve part of that separation with a single writable branch, branch protections, required checks, human review before merge, session logs, and audit events. A custom dispatcher should keep those controls, not replace them with a bot token that can push to the default branch. (Cloud-agent risks and mitigations)

Put one admission decision between event and run

An admission controller returns a bounded job specification or a denial. It does not execute the agent, interpret the requested code change, or approve the result.

The decision needs eight inputs:

• Authenticity: validate X-Hub-Signature-256 against the raw request body with HMAC SHA-256 and a constant-time comparison. (Validating webhook deliveries)
• Event semantics: accept only the expected event and action, such as issue_comment with created.
• Actor: query the actor's current effective repository permission after the event is authenticated. GitHub's permission endpoint considers repository, team, organization, and enterprise grants; its legacy permission field maps maintain to write and triage to read. (Collaborator permissions API)
• Repository and context: bind an allowlisted repository, issue or pull-request number, and the exact head commit that a later worker may inspect.
• Idempotency: reserve X-GitHub-Delivery before enqueueing. GitHub preserves that identifier on redelivery, so a replay must not create a second job. (Webhook best practices)
• Capacity: reserve per-actor and fleet concurrency before dispatch, not after the agent starts.
• Cost: choose an estimated credit class from trusted automation configuration, not from the comment, and reserve it against the creator or cost center.
• Authority profile: attach a server-side tool and credential profile plus the required merge policy. Never let comment text select an admin tool or stronger token.

The blue ADMIT stage in the figure is a transaction boundary. In a multi-replica service, the idempotency key, actor counter, concurrency slot, and budget reservation must commit atomically. A process-local Set is suitable for the reference test below, not for production coordination.

Define the policy outside the comment

This illustrative policy is intentionally small. Its numeric values are example operating choices, not GitHub limits or benchmarks.

{
  "repositories": ["acme/payments-api"],
  "trigger": "/agent run",
  "minimum_permission": "write",
  "max_comment_bytes": 4096,
  "max_jobs_per_actor_per_day": 3,
  "max_concurrent_jobs": 2,
  "max_ai_credits_per_job": 5,
  "tool_profile": "coding-read-write-no-admin",
  "merge_policy": "pull-request-review"
}

The important properties are not the example thresholds. They are ownership and direction of trust:

• repository scope comes from policy;
• permission comes from GitHub after signature validation;
• estimated cost comes from trusted job configuration;
• tool and merge profiles come from policy;
• only the task arguments come from the comment.

If a comment can override minimum_permission, tool_profile, max_ai_credits_per_job, or merge_policy, the policy is merely another prompt.

Run the dependency-free admission controller

The reference implementation below uses Node's built-in crypto functions and returns a job specification without dispatching it. The caller must obtain actorPermission from GitHub's permission endpoint and must atomically reserve the returned reservation_key and counters before enqueueing.

import { createHmac, timingSafeEqual } from "node:crypto";

const PERMISSION_RANK = Object.freeze({ none: 0, read: 1, write: 2, admin: 3 });

export function verifyGitHubSignature(secret, rawBody, signature) {
  if (!secret || !signature?.startsWith("sha256=")) return false;
  const expected = Buffer.from(
    `sha256=${createHmac("sha256", secret).update(rawBody).digest("hex")}`,
  );
  const received = Buffer.from(signature);
  return expected.length === received.length && timingSafeEqual(expected, received);
}

function deny(code, details = {}) {
  return { admitted: false, code, ...details };
}

export function decideCommentJob({ headers, payload, policy, state }) {
  const event = headers["x-github-event"];
  const deliveryId = headers["x-github-delivery"];

  if (state.signatureValid !== true) return deny("invalid_signature");
  if (event !== "issue_comment") return deny("wrong_event", { event });
  if (payload.action !== "created") return deny("wrong_action");
  if (!deliveryId) return deny("missing_delivery_id");
  if (state.seenDeliveryIds.has(deliveryId)) return deny("duplicate_delivery");

  const repository = payload.repository?.full_name;
  if (!policy.repositories.includes(repository)) return deny("repository_not_allowed");

  const actor = payload.sender?.login;
  if (!actor || actor === "ghost" || payload.sender?.type !== "User") {
    return deny("actor_not_resolved");
  }

  const permission = state.actorPermission ?? "none";
  if (
    (PERMISSION_RANK[permission] ?? -1) <
    (PERMISSION_RANK[policy.minimum_permission] ?? Number.POSITIVE_INFINITY)
  ) return deny("insufficient_permission", { actor, permission });

  const body = payload.comment?.body?.trim() ?? "";
  if (Buffer.byteLength(body, "utf8") > policy.max_comment_bytes) {
    return deny("comment_too_large");
  }
  if (!(body === policy.trigger || body.startsWith(`${policy.trigger} `))) {
    return deny("trigger_not_matched");
  }

  const jobsToday = state.jobsTodayByActor.get(actor) ?? 0;
  if (jobsToday >= policy.max_jobs_per_actor_per_day) return deny("actor_daily_limit");
  if (state.activeJobs >= policy.max_concurrent_jobs) return deny("concurrency_limit");
  if (state.estimatedAiCredits > policy.max_ai_credits_per_job) {
    return deny("job_credit_limit");
  }

  const issueNumber = payload.issue?.number;
  if (!Number.isInteger(issueNumber)) return deny("missing_issue_number");

  return {
    admitted: true,
    reservation_key: `${repository}:${deliveryId}`,
    job: {
      repository,
      issue_number: issueNumber,
      context_kind: payload.issue?.pull_request ? "pull_request" : "issue",
      actor,
      command: body.slice(policy.trigger.length).trim(),
      tool_profile: policy.tool_profile,
      merge_policy: policy.merge_policy,
      source_delivery_id: deliveryId,
    },
  };
}

Signature validation must operate on the raw request bytes. Parsing and re-serializing JSON before computing the HMAC changes the byte sequence and can invalidate a legitimate delivery. After validation, parse once, query the current permission with a read-only GitHub App installation token, and pass only the calculated base permission into the decision.

The permission lookup is one request:

async function getActorPermission({ owner, repo, actor, token }) {
  const response = await fetch(
    `https://api.github.com/repos/${owner}/${repo}/collaborators/${actor}/permission`,
    {
      headers: {
        Accept: "application/vnd.github+json",
        Authorization: `Bearer ${token}`,
        "X-GitHub-Api-Version": "2026-03-10",
      },
    },
  );
  if (!response.ok) return "none";
  return (await response.json()).permission;
}

Do not rely on a stale team export or comment.author_association when current authorization is the decision. The API response reflects the actor's highest effective base permission across the repository and its parent organization controls.

Test every denial before dispatch

The reference artifact passed nine Node tests: signature validation, one admitted job, and seven named fail-closed cases. The denial matrix covers an invalid signature, wrong event, duplicate delivery, insufficient permission, actor daily limit, concurrency limit, and per-job credit limit.

const denials = [
  ["invalid_signature", state({ signatureValid: false })],
  ["wrong_event", state(), { "x-github-event": "push" }],
  ["duplicate_delivery", state({ seenDeliveryIds: new Set(["delivery-001"]) })],
  ["insufficient_permission", state({ actorPermission: "read" })],
  ["actor_daily_limit", state({ jobsTodayByActor: new Map([["mona", 3]]) })],
  ["concurrency_limit", state({ activeJobs: 2 })],
  ["job_credit_limit", state({ estimatedAiCredits: 6 })],
];

for (const [code, testState, headerOverride] of denials) {
  test(`fails closed: ${code}`, () => {
    const result = decideCommentJob({
      headers: { ...headers, ...headerOverride },
      payload,
      policy,
      state: testState,
    });
    assert.equal(result.admitted, false);
    assert.equal(result.code, code);
  });
}

Add repository-specific negative cases before rollout:

• a former maintainer whose permission was removed;
• an outside contributor when the native untrusted-author opt-in is enabled;
• a redelivered webhook after the original job completed;
• a pull-request comment whose head branch belongs to a fork;
• a command that attempts to name a stronger tool profile;
• a second automation triggered by the first automation's output;
• a job admitted just as the concurrency or budget ceiling is reached.

The last case needs a transactional test against the real store. Two replicas can both observe one remaining slot and both admit unless reservation occurs in the same atomic operation as idempotency and budget.

Choose the native path or own the full controller

Whichever path you choose, preserve the release boundary. The agent works on its constrained branch, CI runs under its normal protections, a separate reviewer evaluates the diff and evidence, and branch policy decides whether merge is possible.

This controller does not replace the runtime boundary described in A Sandbox Is Not a Security Model for AI Coding Agents. It decides whether a job may enter that runtime. The same pattern applies to enterprise admission for portable agent packages: pin the authority profile, test denial, and make the decision repeatable.

Log the admission decision, not only the agent transcript

Keep an append-only record for every accepted and rejected event:

• delivery identifier and signature result;
• event, action, repository, issue or pull-request number, and head commit;
• actor and current effective permission;
• matched policy version and trigger;
• idempotency, actor, concurrency, and cost reservation results;
• selected tool, credential, network, and merge profiles;
• decision code, timestamp, job identifier, and reviewer identity;
• final branch, pull request, checks, approval, merge, or cancellation outcome.

The agent transcript explains what happened after dispatch. The admission record explains why the business allowed the run to exist. You need both when a job spends unexpectedly, reads the wrong context, creates a recursive automation chain, or produces a pull request nobody can safely approve.