AI Fundamentals for AI-Assisted Development · Chapter 13

Chapter 13 — Using AI Appropriately in AI-Assisted Development

Type
Book
Edition
Version 1
Language
English
Format
Web
Author
mars70
Chapter 13

Breaking down "it's done"

In AI-assisted development, you may receive a report saying:

it's done

But "done" is not a single state. This book separates the completion states related to development work into the following six:

implemented
tested
PASS
committed
pushed
deployed

This is not an official industry-wide taxonomy — it's this book's organization for Verification, meant to avoid confusing completion reports in AI-assisted development.

implemented

implemented is the state where a change exists as an implementation. An AI reporting "I fixed it" alone does not mean implemented has been confirmed — you confirm the actual code or repository change. Also, even if implemented is confirmed, that does not mean tested, PASS, committed, pushed, or deployed have also been confirmed.

tested

tested is the state where a test was run. What matters is the distinction tested != PASS. For example, if a test was run and a failure occurred, then tested = YES and PASS = NO. That is not a contradiction.

PASS

PASS is the state where a defined scope of success was observed as a result. As seen in Chapter 11, interpret it together with:

  • target
  • version
  • test set
  • conditions
  • observation point

Therefore, PASS != all requirements were met.

committed

committed is the state where a change was recorded as a commit in the local Git repository. But committed != pushed. A commit existing does not mean it was pushed to the remote repository.

pushed

pushed is the state where an intended commit reached an intended remote repository. Freshness matters here — rather than inferring from local information alone that it must also be on the remote by now, check the fresh remote state when necessary.

deployed

deployed relates to the state where a change was actually put into place in the execution environment. An AI writing "I deployed it" alone cannot confirm this — observe runtime or service state as needed.

That said, having run once on a runtime does not mean it will always run correctly in the future:

ran on the runtime once
        !=
always correct going forward

Nor does:

deployed
        !=
all requirements were met

confirmed is not the 7th state

This book does not treat confirmed as a seventh completion state. confirmed is used as a word meaning whether a given state has been confirmed by Evidence — as in:

is "implemented" confirmed?
is "tested" confirmed?
is "pushed" confirmed?

Don't lump the six together

Suppose you get an AI report like:

I implemented it, tested it, committed it, and pushed it.

Rather than treating this as one single "done," check it broken apart:

implemented ?
tested ?
PASS ?
committed ?
pushed ?
deployed ?

This doesn't mean you need to check every single one every time — check the state that's needed for the scope of this particular task.

Verification is not "an act of doubting the AI"

The term Verification might make it feel like "you can't trust AI, so doubt everything." But this book doesn't think of it that way. Verification is work that avoids confusing the AI's explanation, the external state, and the Human's judgment. Even when the AI was correct, performing an independent check gets you to the state where:

you have confirmed, with Evidence, that it was correct

Decide the intent, Scope, and constraints first

Verification isn't something that suddenly starts at the very end. When requesting work, make the following as clear as you can:

  • what you want done
  • how far it should go
  • what must not be done
  • what would need to be visible for it to count as success

For example, just "fix the web thing" doesn't tell you whether it means:

just editing files?
up to commit?
up to push?
up to deploy?

Making the Scope explicit makes it easier to keep the AI from advancing to the next stage on its own.

Authority Boundary

An AI being technically able to do something and it being okay to do it are not the same thing. Even if an AI has a Tool that lets it deploy, that doesn't mean deploy permission has been granted by a Human:

capability != authorization

This distinction matters in AI-assisted development that uses Tools or Agents.

Seven-step Verification Loop

This book organizes the minimal Verification Loop for AI-assisted development into seven steps.

A seven-step Verification Loop from fixing the request and constraints through the Human decision, with a bounded return for another attempt when justified
This book organizes the minimal Verification Loop for AI-assisted development into seven steps.

Step 1 — Fix the request, intent, Scope, and constraints

First, decide:

what to do
how far it goes
what must not be done

Then consider what state would mean this task's purpose has been met.

Step 2 — Receive the AI's output

The AI generates things such as:

  • code
  • configuration
  • an explanation
  • an investigation result
  • a completion report

At this point, this is output received — it is not yet task verified complete.

Step 3 — Find the important Claims

You don't need to verify every sentence with equal weight. Extract the Claims that determine the outcome — for example:

"3 files modified"
"tests PASS"
"remote pushed"

Step 4 — Decide the Source of Truth and expected result

For each Claim, decide:

what you would need to look at to confirm it

For example:

file change → repository
test result → test artifact
remote reflection → remote repository
runtime → runtime observation

At the same time, decide:

what would need to be visible for this to count as success

Step 5 — Confirm the Independent Evidence

Confirm the independent Evidence needed, not just the AI report itself. What matters here is confirming the necessary scope — you don't need to investigate everything unrelated.

Step 6 — Classify the Evidence

Organize the confirmation results as:

VERIFIED / OBSERVED
INFERENCE
UNKNOWN / UNDISCLOSED

When judging the task result, distinguish PASS / FAIL / UNKNOWN as needed. If the Evidence is insufficient, leave it as UNKNOWN rather than forcing it into PASS or FAIL.

Step 7 — The Human decides

Finally, the Human makes a decision among:

accept
correct
stop
escalate

Record that decision, and return to Step 1 only when another bounded iteration is justified.

What matters here is:

technical PASS
        !=
the next action was automatically authorized

A PASS result does not mean:

it's okay to deploy

Crossing the next authority boundary may require a separate Human decision.

This book describes Human approval as the decision to accept a confirmation result, and Human authorization as the decision that a given scope or operation may be executed. The two are related, but neither arises automatically from an AI's capability.

The Loop is not "keep going until it PASSes"

The name "Verification Loop" might make it sound as if AI keeps making fixes forever until it PASSes. But that is not what this book's Loop means. When FAIL or UNKNOWN comes up, options include:

  • checking the cause
  • making a bounded correction
  • stopping
  • returning it to the Human
  • escalating

Even when running a new attempt, do so within a range where you can say why you're doing that next attempt.

Also avoid excessive Verification

More Verification is not always better. Requiring the same full set of checks used for a production deployment on a small docs-only change drives up the cost of the work. On the other hand, work with major impact — runtime, security, public deployment — needs stronger confirmation. In other words:

Verification proportional to risk

is needed. This book does not recommend applying the maximum level of process to every task.

Separate AI's capability from the Human's role

AI can help with a lot — for example:

  • producing candidates
  • writing code
  • reading files
  • searching
  • executing Tools
  • organizing Evidence

But things such as:

what to permit
what to treat as the Source of Truth
where to stop
what to accept

fall to Human authority. AI-assisted development does not mean only:

development where AI executes everything autonomously

Combining AI's capability with Human authority is what matters.

Chapter summary

This book separates completion into the following six:

implemented
tested
PASS
committed
pushed
deployed

And confirmed is a concept for looking at whether each of these has been confirmed by Evidence.

In AI-assisted development, we use the following seven-step Verification Loop:

1. Intent / Scope / Constraints
2. AI Output
3. Important Claims
4. Source of Truth / Expected Result
5. Independent Evidence
6. Evidence Classification
7. Human Decision

What matters is:

  • don't lump the completion states together
  • separate the AI report from the external state
  • separate capability from authorization
  • don't generalize beyond the scope of PASS
  • don't force UNKNOWN to be filled in
  • separate the technical result from Human authorization
  • don't turn the Loop into unlimited retries until PASS
  • perform Verification proportional to task risk

The next chapter, Chapter 14, uses everything learned so far to follow one AI-assisted development task from request through generation, Tool Use, Evidence confirmation, and the Human decision, start to finish.