Responsible Engineering

Responsible engineering is the practice of building AI systems whose behaviour can be measured, constrained, explained and reversed. It is a set of engineering disciplines, not a values statement.

Why the phrase needs defending

“Responsible AI” has been used to mean almost anything — a principles page, a committee, a compliance checkbox. We use it to mean four concrete practices that show up as code and artifacts: evaluation, governance, security, and auditability. If a program at FIRE uses the phrase without teaching one of those, it should not be using it.

What does evaluation actually mean?

An evaluation harness is a test suite for non-deterministic systems. It has a golden dataset built from real failures, deterministic assertions where behaviour must be exact, and calibrated judgement where it cannot be. It runs in CI, so a prompt change cannot silently ship a regression.

Most teams skip this because the system appears to work when they try it by hand. That is precisely the failure mode: a system that works on the examples its author thought of. Evaluation is what turns a demo into something you can operate.

What does governance mean in a codebase?

Approval gates where the cost of a wrong action is irreversible. Confidence thresholds that trigger escalation instead of guessing. Audit trails detailed enough to reconstruct why a system did what it did, months later, for someone who was not there.

Governance written after a system is built is documentation. Governance designed into the loop is engineering, and the difference is visible the first time something goes wrong.

Why is security part of this?

Because an agent with tool access is a system with privileges, and privileges get abused. Prompt injection — direct and indirect through retrieved content — is not theoretical. Any content your agent reads is untrusted input, including web pages, documents and email.

We teach engineers to attack their own systems: injection attempts, tool abuse, scope escalation. It is uncomfortable, and it is far cheaper than learning the same lesson in production.

What does auditability require?

That someone outside your team can answer three questions from your logs: what did the system do, on what basis, and who could have intervened. If your tracing cannot answer those, you have monitoring, not auditability.

Where this is taught

Responsible engineering is not a single elective. It runs through the agentic track as four of its twelve weeks, and as a dedicated four-week program for teams that already have systems in production and need to bring them under control.

What we are not claiming

That any of this makes an AI system safe in an absolute sense. It makes behaviour measurable, failures visible, and decisions reversible. That is a large improvement over the norm, and it is a smaller claim than the phrase usually carries.