AI Fundamentals for AI-Assisted Development · Chapter 12

Chapter 12 — Who Decides the Source of Truth, and How?

Type
Book
Edition
Version 1
Language
English
Format
Web
Author
mars70
Chapter 12

What is a Source of Truth?

Chapter 11 separated the AI said so from confirmed as an external fact. So when confirming an external fact, what should you look at? That's where Source of Truth becomes important. This book treats Source of Truth as:

the information source treated as the basis for confirmation, for a given question.

There is no single, all-purpose Source of Truth

The term Source of Truth might bring to mind something like "a single database that is absolutely correct about everything." But this book does not treat it that way. Which source is appropriate depends on what you want to know:

what was committed
what was intended to be built
whether it was tested
whether it's currently running
what the configuration is
what happened in the past

Each of these may require a different source.

What Git lets you confirm

If you want to know "what was committed," Git history or a diff serves as confirmation material. For example, if an AI reports "I modified this file and committed it," checking the Git history or the commit contents lets you confirm — separately from the AI's explanation — whether that commit exists and what was changed. However, recorded in Git = the content is absolutely correct does not follow. Git is a source for the questions Git can confirm.

Specification

To confirm "what was intended to be built" or "what requirements should be met," a specification or requirements document becomes important. For example, even if the code runs, if it differs from the requested behavior, simply "it ran" is not enough to call it complete. In that case, you need to compare:

implementation
        ↕
specification

Test Evidence

If you want to know "whether a test was run," a test execution artifact or log serves as confirmation material. Here too, an AI's sentence saying "I ran the tests" and the actual test execution record are different things. In this project, when looking at test Evidence, we also check what the observation is about — things such as:

  • target
  • version
  • scope

CI / Build Artifact

CI or build records can also serve as confirmation material, depending on the conditions. If things such as:

  • target
  • version
  • time

are known, it becomes easier to trace "what build/test event this was actually about." That said, a CI artifact existing by itself does not guarantee complete reproducibility or correctness in every sense.

Configuration

To confirm "what was intended as the configuration," a configuration file is a clue. However, configuration = the current runtime state does not necessarily hold. Even if a config file says A, whether the system actually running is A may need separate confirmation.

Runtime

To confirm "how things currently stand," runtime observation matters — for example:

whether the service is running
how it responds to a request
which version is currently running

But runtime observation, too, is only a result observed for a given object, at a given point in time, using a given method. From that result, you cannot prove this state will hold forever going forward.

Log

To investigate "what happened in the past," a log may be used. However, a log doesn't always contain all the necessary information either. If the time, target, or event isn't sufficiently clear, you may not be able to pin down the past occurrence. In that case, rather than forcing a conclusion, leave it as UNKNOWN.

Matching questions to Sources

To organize this simply:

Question to confirm Candidate Source
what was committed Git history / diff
what should have been built specification / requirements
whether it was tested test artifact / log
what happened during the build CI / build artifact
what was intended as the configuration configuration
how things are currently running runtime observation
what happened in the past log

This is not a universal table. Which Source is appropriate depends on the object.

AI does not get to decide the Source of Truth on its own

You can consult an AI with something like "what should I check?" But an AI freely choosing a source that's convenient for itself and declaring "this is the Source of Truth" doesn't, by itself, make that source the official point of confirmation. Which source is treated as authoritative is a question the project or a Human decides.

Human Authority

This book treats a Human as the final decision-maker for important adoption decisions and authority boundaries. Human authority refers to the side that makes the final decision, and Human authorization refers to the decision that a given scope or operation may be executed. A technical PASS or an AI's recommendation cannot substitute for that decision.

Even if multiple AIs agree that "this source is fine," that agreement does not substitute for Human authority. AI recommendation and Human authorization are different things:

AI recommendation
        !=
Human authorization

When Sources conflict

Suppose a state like this exists:

configuration: A
runtime: B

Rather than asking simply which one is true and eliminating one, consider:

what does configuration indicate
what does runtime indicate

Configuration may indicate "the intended setting," while runtime may indicate "the actual state observed at that point in time." The two may be answering fundamentally different questions.

Freshness also matters

For Source of Truth, it's not just the type of source that matters — when the information was obtained matters too. For example, even if you check local Git state, if the remote has since changed, that does not amount to having confirmed the current remote state. For a Claim about the current state, you need to be conscious of freshness.

Questions for deciding the Source of Truth

In AI-assisted development, it helps to think it through this way:

what do I want to confirm
        ↓
what source can directly confirm that question
        ↓
what target/version/time is that source actually about
        ↓
who decides whether to treat that source as authoritative

Chapter summary

  • Source of Truth is not a single, all-purpose source
  • the appropriate Source changes depending on what you want to confirm
  • Git, specification, test artifacts, CI/build, configuration, runtime, and logs are each used for different questions
  • don't automatically treat configuration and runtime as the same state
  • runtime observation has a boundary defined by time and observation method
  • don't use old information as Evidence for the current state
  • AI does not get to decide the Source of Truth on its own
  • ultimately, authority is decided by the Human/project
  • when Sources conflict, think through what each one is actually indicating

The next chapter connects all of this to the practice of AI-assisted development, organizing how to use AI appropriately using completion states and the Verification LoopThe Verification Loop is the minimal confirmation flow used in this book. It is not a universal SDLC or industry standard — it's this book's model for keeping an AI's output and the state of its target separate..