Scoping tool access for an AI agent

OpenAI released the Responses API on 11 March with built-in web search, file search and computer use, and the announcement also introduced an Agents SDK with guardrails and tracing. That makes it easy to put all three tools in one workflow. It doesn't tell you what the agent should be allowed to see, change or expose, though. You still have to define that yourself.

I don't want an agent borrowing an administrator's session or a developer's personal token. It should have its own named service identity tied to one job, and you should be able to say which environment it can reach, which file collections it can search, whether it sees every customer or just the current account, whether it can publish or only draft, and how fast you can revoke the credential if it ends up in a log. Use short-lived tokens where you can, keep production and test identities separate, and keep secrets away from the model entirely. The application can validate arguments, add the credential and filter the response behind its own boundary. If a connector can't express the scope you need, I wouldn't grant broad access and hope the prompt behaves.

Each tool needs its own limits. Search results were written by somebody else, so they can be wrong or contain text that looks like an instruction. Keep the source URL and retrieval time, keep page content apart from your instructions, and don't let anything found on the web trigger an internal action without another validation step. Strip private names and identifiers out of queries before they go to an external service. File search looks read-only, but a shared vector store holding several accounts' documents can hand back text from the wrong one, so partition collections and enforce tenant filters in the retrieval layer rather than asking the prompt to "only use files for this customer". Computer use is a research preview, and OpenAI recommends isolation and user confirmation for consequential actions. I'd run it in a sandbox with a clean profile, an allowlist of destinations and downloads switched off unless the job needs them.

Reading a page can run on its own. Sending a message, submitting a form, buying something, changing permissions or deleting a record should stop and show a plain preview: the destination, the affected account, the values going in and anything irreversible. "Continue?" isn't enough. Tie the approval to that exact action, and if the page or the values change, ask again. Schema-valid JSON doesn't make a tool call authorised, so the executor should check every argument against the current identity, resource scope and allowed fields, and expose reads and writes as separate functions. A dry-run mode helps too, so the agent and reviewer look at the same proposed update.

Before release I'd test the paths that should be denied: a search containing private data, a cross-project file query, a write outside the allowed resource and a computer-use action that changed after approval. Each one should be blocked, and the trace should say why, with the agent identity, tool version, validated arguments, target, approval event and result for every call. For the first production workflow I'd start with one identity, a short tool allowlist and a review on every write, then widen a permission when a real task needs it and the logs show the current boundary holding up.

Back to the Responses API, then. Having web search, file search and computer use in one workflow is convenient, but each tool gives the agent a different kind of authority, and I'd want that scoped before any of them is connected. In practice that means the agent has its own identity, each tool has its own limits, and consequential actions stop for an approval tied to that exact action. Before connecting a tool, I'd write down what the agent may see, change or expose through it, and if I can't, I'd leave that tool off for now.