Chapter 14 — Following One AI-Assisted Development Task from Start to Finish

Connecting everything so far in a single piece of work

In the chapters so far, we separated AI-related concepts one at a time — for example:

AI != LLM
Model != AI service
Generation != Search
Context != Training Data
Tool request != Tool execution
AI report != external fact
PASS != zero UNKNOWNs
capability != authorization

This final chapter uses all of them within one AI-assisted task. It does not introduce new theory or a new development method. It follows the way of thinking learned in Chapters 1 through 13 through one piece of work, from start to finish. The example is a document update, but at the end we also confirm how it maps onto development work.

Today's task

Let's use a simple example. Suppose an organization has an existing procedure document.

A Human asks the AI:

Check the existing procedure document, cross-reference it against
the specified public notice page, and update only the descriptions
that are out of date.
Don't add anything not present in the source material.
Just fix the document — final publication is my call.

Here, we'll call the procedure document operations/application-procedure.md, and the public notice to cross-reference is an already-specified source. It's a small task, but it lets us see, in one pass, the important boundaries in AI-assisted development.

Scene 1 — the Human decides the Intent, Scope, and Constraints

The first thing to check is not what the AI is capable of. It's what it's actually allowed to do, this time.

Organizing the request gives us:

Intent:
Cross-reference against the specified public notice,
and update only the outdated descriptions in the procedure document

Target:
operations/application-procedure.md

Authorized:
reading the procedure document, checking the specified source,
editing the document

Not Authorized:
adding content not present in the source material
final publication of the procedure document

Chapter 13's capability != authorization is relevant here. Even if the AI has a Tool that can publish documents, that doesn't mean it's okay to use it this time.

Scene 2 — the AI checks the current state

For the AI to update the document, it first needs to check the current procedure document and the content of the specified public notice. If the public notice needs to be searched for, Search or Retrieval is involved; if it's just a matter of reading an already-specified notice, reading the document is the main step. If Tool access is available, either kind of information-gathering may take place.

Here, we use the distinction from Chapter 8: the model generated "I'll read the document" != the document was actually read. A Tool request and Tool execution are different things.

If the actual document-read result and information-gathering result can be confirmed, you get observable information about the current state of operations/application-procedure.md and the specified source.

Scene 3 — information enters Current Context

When the content of the procedure document and the specified source is passed to the AI system, that information may become part of the Context available for the current task. Here too, we use the distinction from Chapter 7: the current document content entered Context != it became Training Data. Also, Context != saved Memory.

What this task needs is simply being able to use the current document and the specified source. Beyond that, we don't speculate about the internal processing of saving or Training without Evidence.

Scene 4 — the AI generates a proposed change

The AI generates a proposed change based on the current document, the specified source, and the Human's instruction. This generation involves the model generation covered in Chapter 4.

That said, for this task, there's no need to know things such as:

• what hidden prompt was used
• how many times the model was called on the backend
• what internal routing was used

Anything that can't be confirmed stays UNKNOWN / UNDISCLOSED.

What we want to confirm this time is: does only the description identified as outdated relative to the specified source get properly updated in the target document?

Scene 5 — the AI requests an edit to the document

Suppose the AI is a system that edits the document through Tool Use. Conceptually, we can relate it this way:

Human instruction
      ↓
AI output / tool request
      ↓
document-edit tool
      ↓
document state

The AI deciding on a change, or generating a request to an edit tool, does not by itself mean the external document has changed. Only once the request to the edit tool is actually executed, and the file or document content is actually rewritten, does the external state change.

In practice, a document-edit tool, a file-write operation, or a patch operation may carry out this step. In development work, applying a patch, or running a shell / PowerShell command, may play this role instead. That said, not every AI system uses commands or a shell.

tool request != actual document modification. Even if the AI reports "I made the change," that sentence alone does not confirm that the document was actually changed.

Scene 6 — external state changes

If the document edit actually succeeds, the external state of operations/application-procedure.md changes. Only now is there an external object — document file state — beyond the generated text.

If we want to confirm this task's implemented Claim, the candidate Source of Truth is the actual document file.

Scene 7 — the AI generates a completion report

Suppose the AI reports as follows:

I updated the outdated description in
operations/application-procedure.md.
The work is complete.

This report contains multiple Claims:

Claim 1:
operations/application-procedure.md was changed

Claim 2:
the outdated description in
operations/application-procedure.md was updated

Claim 3:
this task's work is complete

What matters is that an AI report and an external fact are not the same thing:

AI report
        !=
external fact

The report is a starting point for confirmation — it is not by itself independent Evidence that the work is complete.

Scene 8 — deciding the Source of Truth

What we want to confirm this time is whether the procedure document was updated as requested, in light of the specified source:

was the procedure document updated as requested,
in light of the specified source

So, as the source for confirming the document change, we use the actual document file. As the source for confirming whether the description matches the current public notice, we use the specified source.

This is a Source that fits the question:

"what is currently written in the document"

If instead the question were "was it finally published," we'd need to check the state of the publication destination. But this time, final publication itself is Unauthorized.

Which Source of Truth is right depends on the question — that's Chapter 12's idea.

Scene 9 — confirming the Independent Evidence

Suppose that checking the actual document shows that the description of the reception method — which differed from the specified source — has been updated, and no unintended changes are found elsewhere.

This observation, at least within the scope confirmed here, is Evidence supporting the claim that only the descriptions within the requested scope were updated — not because the AI said "I updated it," but because the external state and the specified source were checked separately.

Scene 10 — classifying the Claims

Let's organize what we've confirmed.

Claim 1 — the document was changed

If the change can be confirmed in the actual document file, this can be treated as:

VERIFIED / OBSERVED

Claim 2 — the outdated description was updated

If cross-referencing the specified source against the updated document confirms a match, this can be treated as having confirmed that the outdated description was updated:

VERIFIED / OBSERVED

Claim 3 — this task's work is complete

The word "complete" in the AI's report is a Claim that this task's work is complete. From that word alone, you cannot decide whether or not final publication is included.

The state of final publication is confirmed separately

If you want to know the state of final publication, check the external state at the publication destination. The publication state you observe there is not a Claim extracted from the original AI report — it is a verification result. Because the Human held off final publication this time, whether the authority boundary was respected can also become something to confirm.

Scene 11 — separating the Completion States

Looking at this against Chapter 13's six completion states, what's expected this time is basically only up to:

implemented

That is, today's completion state is as follows:

implemented: YES / confirmed
tested:     NOT REQUIRED for this bounded document update
PASS:       NOT USED as a separate test result
committed:  NOT PART OF THIS TASK
pushed:     NOT PART OF THIS TASK
deployed:   NOT PART OF THIS TASK

What matters here is that it does not proceed as:

implemented
        ↓
automatically finally published
        ↓
automatically the whole piece of work is approved

"I went ahead and did the next part too" is not success

What if the AI had reported:

I updated the procedure document, and also added an explanation
not present in the source material, and published it.

Even if the changed content was technically correct, both the content addition and the final publication are outside this task's authorization. So the request content being correct and the whole piece of work being appropriate are separate things.

This is extremely important in AI-assisted development: a correct result and an authorized process are not the same thing.

correct result
        !=
authorized process

Scope expansion

Let's also consider a case where the AI judges:

Since the procedure document alone is hard to understand,
I've also unified the related documents.

The Human only requested cross-referencing against the specified source and updating the outdated descriptions in the procedure document. Even if the AI judges "this would be better," expanding the Scope on its own from:

a bounded update to the procedure document
        ↓
a full rewrite of the related documents

is not included in this task's authorization. This is where Chapter 13's Intent, Scope, and Constraints come into play.

Don't unilaterally turn "something better" into the goal

In AI-assisted development, ideas like:

easier to read would be better
being unified would be better
being brought up to date would be better

can seem reasonable on their own. But they are not necessarily the goal the Human actually requested. the Human's goal needs to be kept separate from a goal the AI judged would be better to add. The AI can propose improvements — but proposing something and having the authority to carry it out are different things.

An example where UNKNOWN remains

Not everything can be confirmed for every task. For example, if you wanted to know, about the internals of the AI service, how many seconds this document content was retained on the backend, that may remain UNKNOWN / UNDISCLOSED without public information or a way to observe it. But that UNKNOWN existing does not mean the procedure-document update failed. What matters is separating the questions this task's completion actually needs answered from the ones it doesn't.

Match the strength of Verification to the task

This example is a small document update. So there's no need to go as far as:

• production runtime validation
• a security penetration test
• deployment verification
• an infrastructure health check

What's mainly needed this time is whether the specified document was updated in light of the specified source, within the specified scope:

was the specified document,
in light of the specified source,
within the specified scope,
updated

This is the idea covered in Chapter 13: Verification proportional to risk. More Verification isn't automatically better — you confirm the Evidence a task actually needs.

Connecting general work and development work

The distinctions used in this example — document updates, checking information, editing via a Tool — also apply to development work. In development, code changes may replace document editing, applying a patch or running a command may replace editing via a Tool, and running tests may replace confirmation.

general work
document update / checking information / editing via a Tool

development
code change / running commands / test / commit / push / deploy

In a development task where a bug is fixed and tests are run through to completion, you separately confirm implemented, tested, and PASS or FAIL. committed, pushed, and deployed remain separate states, and may require separate authorization. Not every task needs all six states — you separate out and confirm the states a given task actually needs. Even as the task changes, the basic idea of separating generation, execution, external state, confirmation, and authority does not change.

AI with Tools requires thinking about the scope of impact too

There's a difference in impact, when something fails, between an AI that can only read files and one that can also:

write
delete
execute
deploy

As Tool capability broadens, the Human-side judgment of what to permit becomes more important. As we saw in Chapter 10, we also separate read-only from state-changing. Today's document edit is a state-changing action. That's exactly why keeping the Scope limited to only the outdated descriptions in the procedure document has meaning.

Finally, the Human decides

Suppose, having confirmed the Independent Evidence, we've established that:

• only the specified document was changed
• it was updated to content that matches the specified source
• checking the state of the publication destination shows final publication has not happened

The last item is not a Claim extracted from the original AI report — it is a verification result obtained by checking the publication destination.

With that established, the Human can choose:

accept

If there are unwanted changes, it might be:

correct

If there's a serious Scope violation, they might choose:

stop

If a decision beyond their own authority is needed, they may:

escalate

The AI does not make the final call itself.

Bringing it back to the Seven-step Verification Loop

Let's map today's example back onto Chapter 13's seven steps.

1. Intent / Scope / Constraints

Cross-reference against the specified source and update only
the outdated descriptions in the procedure document
Up to fixing the document
Final publication is the Human's call

2. AI Output

The AI generates a proposed change, a tool request, and a completion report.

3. Important Claims

We extract Claims:

the document was updated
the outdated description was updated
this task's work is complete

4. Source of Truth / Expected Result

We set:

Source:
document file state
the specified public notice
the publication destination, if checking the final-publication state

Expected:
operations/application-procedure.md matches the specified source
no content not in the source material, and no unwanted scope expansion

5. Independent Evidence

We confirm the actual state of the document and the specified source. If checking the state of final publication, we also check the publication destination, and keep that observation result separate from the Claims extracted from the AI report.

6. Evidence Classification

We organize what we've confirmed as:

VERIFIED / OBSERVED
INFERENCE
UNKNOWN / UNDISCLOSED

For the necessary task result, we also separate PASS / FAIL / UNKNOWN.

7. Human Decision

The Human decides among:

accept
correct
stop
escalate

They record that decision, and return to Step 1 only when another bounded iteration is justified. This closes one Verification Loop.

What's at the center of AI-assisted development

Throughout this book, we've seen a lot about AI's capability. AI can, at times:

• generate text
• generate code
• be built into a system that can Search
• use Retrieval
• use Tools
• be built into a system that, as an Agent, advances through multiple steps

But what matters in AI-assisted development isn't just the number of capabilities. It's being able to separate:

what was requested
what the AI generated
what was actually executed
what was confirmed by Evidence
what is still UNKNOWN
how far the Human authorized it

What it means to use AI appropriately

This book's goal is neither:

trust AI

nor:

don't trust AI

Using AI appropriately means: making use of AI's capability while not confusing the generated output, external execution, Evidence, Source of Truth, and Human authority with one another.

Sometimes AI is right. Sometimes AI is wrong. Sometimes it can't be confirmed at all. What matters is not deciding that difference based on plausibility alone.

plausibility

The final map of this book

Finally, let's pull the whole book together into a single flow:

This diagram is this book's conceptual map for thinking about AI-assisted development.

Chapter summary

Following one AI-assisted development task through connects all the distinctions covered so far:

• the Human decides the Intent, Scope, and Constraints
• separate the AI's capability from its authorization
• separate Model Generation from Tool execution
• separate Context from Training Data
• separate the AI report from the external fact
• decide a Source of Truth for each Claim
• confirm with Independent Evidence
• separate VERIFIED / OBSERVED, INFERENCE, and UNKNOWN
• read PASS with its scope limited
• separate implemented, tested, PASS, committed, pushed, and deployed
• separate the technical result from Human authorization
• perform Verification proportional to task risk
• in the end, the Human decides among accept, correct, stop, and escalate

Even as AI becomes more advanced, the need to separate:

what was generated
what was executed
what was confirmed
who authorized it

does not go away. If anything, this distinction matters more, the further AI's capabilities extend.

That is the center of what you learn in this book: AI Fundamentals for AI-Assisted Development.