AI Coding Agent Guardrails: Finding Gaps in Tool Permissions

I wanted to let a coding agent change an endpoint while keeping application configuration out of reach. The request was fairly simple: update the Java source, adjust the tests, and leave the rest of the project alone. Writing that in the prompt explained the task to the agent, but I also wanted a hard check that could deterministically reject an edit before it hit the file system.

So I added a hook, basically a small program that runs before any editing tool runs. What I saw was that the endpoint and its test changed as requested, and the test passed. When I looked at the tool calls, though, I found that neither edit had passed through my check. I had attached the hook to write_file, while the agent had chosen apply_diff. This is an article about a lesson I had to learn the hard way. 

The agent had followed my request through a tool I had left outside my wonderfully thought-through guardrail. Nothing harmful happened in that session, but I realized designing guardrails requires a different approach. One way to adjust my approach going forward was to check that I really understand how the agent made the change.

Following the Edit Through the Tools

For the example, I used Bob Shell with a generated Quarkus application running on Java 25 and the latest Quarkus. It had a GET /hello endpoint and an HTTP test, which gave me a super small change for my test agent with a clear result that I can check and understand. Forgive the simplistic approach, but I really do want to talk about the guardrails and not how amazing Quarkus is. The policy allowed edits under src/main/java/ and src/test/java/, leaving src/main/resources/application.properties outside the allowed directories.

Bob Shell's PreToolUse hook runs before a tool whose name matches the configured expression. Returning exit code 2 from the hook blocks that call. I put the configuration in .bob/settings.json:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "^write_file$",
        "hooks": [
          {
            "type": "command",
            "command": "python3 .bob/hooks/allow-source-writes.py",
            "timeout": 5
          }
        ]
      }
    ]
  }
}

The path rule never got a chance to make a decision because apply_diff did not match ^write_file$. This was probably my initial wrong assumption. Writing a file and applying a patch are different operations, but both can change the file you are trying to protect.

I expanded the matcher to include four native editing tools:

"matcher": "^(write_file|apply_diff|insert_content|search_and_replace)$"

Now the hook could inspect the patch operation as well as a direct write. Of course, thinking about it now, that still leaves shell commands outside the scope for now. But it's good enough for the little experiment we want to do here.

Getting the Hook and the Agent to Agree

On my next attempt, the hook caught apply_diff and blocked an edit to an allowed Java file. My handler expected the tool arguments inside the documented input object. It could not find that object in the event it received, so it returned exit code 2. Sigh. Documentation is always behind. Let's look into this more deeply.

The session output showed the edit arguments under parameters, although session output and hook input are separate interfaces. That field name alone did not establish the hook's input format, so I extended the handler to accept input, tool_input, and parameters, as well as a direct path object. With that little adapter, the same source edits finally passed:

def tool_input(event):
    for key in ("input", "tool_input", "parameters"):
        value = event.get(key)
        if isinstance(value, dict):
            return value
    return event if "path" in event else None

Once the arguments are available, the policy can check the path. A string starting with src/main/java/ is not quite enough: a symbolic link inside that directory could still point to application.properties. So I had to resolve the path first, and then check where it leads. The simplified excerpt below shows that; the surrounding handler reads the event and returns 2 when it rejects an edit:

from pathlib import Path, PurePosixPath

ALLOWED_PREFIXES = ("src/main/java/", "src/test/java/")

def allows_path(raw_path):
    path = PurePosixPath(raw_path)
    if not path.is_absolute() and ".." in path.parts:
        return False

    workspace = Path.cwd().resolve()
    candidate = Path(raw_path) if path.is_absolute() else workspace / Path(raw_path)
    try:
        normalized = candidate.resolve().relative_to(workspace).as_posix()
    except ValueError:
        return False

    return normalized.startswith(ALLOWED_PREFIXES)

And those were basically the two separate mistakes in the integration I wanted to share with you. First, the hook did not run. Then it ran but could not read the arguments. The path rule could pass its own unit tests in both cases, because those tests supplied the JSON I expected. Sending an actual edit through the agent exposed the difference.

Trying an Allowed Edit and a Blocked One

After I have the guardrails in place, it is finally time to test them. Of course, in a disposable workspace with no really important stuff. But I included secrets/demo.env with a dummy value, so we had another forbidden path for the hooks to check. I also added that directory to .bobignore.

Bob's ignore file covers several normal file operations. It does not restrict arbitrary shell commands, so it cannot provide isolation for the whole workspace.

With the matcher and handler updated, I asked for the endpoint change again:

Change the GET /hello endpoint to return exactly `Hello from policy checked Quarkus`.

Update its existing unit test to assert the new response. Run the relevant Maven test.

Use only Bob's native file-editing tool for source changes. Do not edit configuration, files under secrets/, or the hook policy.

The command starts a new session in the current project. Workspace hooks require a trusted folder, which is why I included --trust with my Bobshell command. And I did specify in the prompt that the agent should use its native editor for the source changes. 

bob run --workspace "$PWD" --trust --format stream-json --mode agent \
  --max-cost 0.20 --max-turns 8 \
  --disable-mcp --disable-subagents \
  --accept-license < prompts/03-allowed-apply-diff.md

This time, two apply_diff calls passed the hook. One changed GreetingResource.java to return Hello from policy checked Quarkus; the other updated the expected response in GreetingResourceTest.java. The Maven test passed as well.

I then tried the same in a new session and asked for a change outside the allowed directories:

Set `quarkus.http.port=8081` in `src/main/resources/application.properties`.

Use only Bob's native file-editing tool. Do not edit Java source, tests, the hook policy, or files under secrets/.

The agent selected insert_content, which was also covered by the expanded matcher. The hook rejected the path with exit code 2, and application.properties stayed empty.

Checking the Policy as Well as the Application

Once a hook decides whether an agent can change a file, it becomes part of a permission system. A mistake can allow an edit that should have been blocked or stop a valid change. A very good reason to give that code its own tests, just as I would test authorization logic in an application. Start with the rule itself. Give it requests that should pass and requests that must be rejected, including input it cannot understand. You can check these decisions without asking a model to produce each request. That makes the tests repeatable and lets you keep a failing example whenever you find a mistake in the policy.

Then check how the hook is connected to the agent. A correct rule cannot stop an action if the runtime never calls it. Send both allowed and forbidden actions through each tool the policy is intended to cover, and inspect what actually changed. This checks whether the hook runs at the right time, receives the input it expects, and has its decision enforced. It is also worth checking what happens when the hook crashes or times out. Rejecting a request and failing to evaluate it may lead to different behavior in the runtime. I want to know which outcome lets an action continue, and repeat these checks when the hook configuration or agent version changes.

The application tests still have their usual job, of course. They check the behavior of the changed code. Hook tests check whether the agent was allowed to make that change in the first place. A passing build gives you no answer to that second question.

Deciding How Much Access to Give the Agent

My example covered a few native editing tools. Shell commands could still change files, and the hook did not restrict network access. The hook itself also ran with my user permissions. Protecting configuration from every possible edit would require controls over those other routes as well. The two requests here checked the native editor policy; they did not show a fully sandboxed or isolated workspace.

This connects to the source-and-sink model in OpenAI's discussion of prompt injection defenses. Content the agent reads can influence it, and a tool call can turn that influence into an action. A check before the action gives the surrounding software a chance to reject it. My endpoint example did not involve an injection attack, but it showed a prerequisite for that check: the action has to reach it.

Anthropic also recommends limiting the tools and permissions given to agents and using defenses at several layers in its guidance on trustworthy agents. For a coding agent, that means deciding which files and commands it needs, applying tool checks, and reviewing what changed afterward. A disposable checkout without credentials keeps the consequences of a mistake smaller; enforced filesystem and network restrictions are separate controls.

I started with a rule about directories and ended up spending more time on the tools that could reach them. Maybe this little article helps you build better hooks for your agents.