Chapter 6 — What Can AI Do?

Don't reduce it to "can" versus "cannot"

When thinking about AI's capability, a simple binary like:

AI can do it
AI cannot do it

isn't enough for a real decision about how to use it. Even with the same AI, the information required, the effect on the outside world, and the method of confirmation change depending on the nature of the work. This chapter uses three perspectives to think about "what kind of information or external effect does this piece of work require":

1. Work that mainly uses given information
2. Work that requires current external information
3. Work that requires performing something externally and confirming the result

This is a learning model used in this book to think about "what to delegate to AI, and what to confirm separately" in AI-assisted development.

Work that mainly uses given information

AI can be used for work that processes text or data it has been given. For example:

• Summarization
• Rewriting
• Translation
• Question Answering
• Information Extraction
• Idea / Candidate Generation

There are uses such as condensing long text, rewriting phrasing, extracting information, and producing multiple candidates. In this kind of work, the input given to the AI itself becomes an important piece of material.

Summarization

Summarization — condensing long text — is one of AI's representative use cases. However, being able to summarize = always accurate does not hold. Important conditions may be dropped, or the meaning may change. For important documents, it therefore helps to keep the original in a form you can compare against.

Rewriting

AI can also be used for Rewriting text — for example:

• shortening it
• making it more readable
• making it more polite
• changing it to a different style

However, it is not automatically guaranteed that the rewritten text fully preserves the original meaning. For text where fine differences in meaning matter — legal text, specifications, Evidence records — confirmation is necessary.

Translation

Translation is also a task a model can provide. However, the translated text reads naturally and the technical meaning matches completely are not the same thing. Technical terms, negation, conditions, exceptions, numbers, and legal expressions are all things that need to be checked.

Information Extraction

AI can also be used to extract things like:

• names
• dates
• numbers
• items
• conditions
• candidates

from text. Here too, the important thing is keeping "being able to ask for extraction" separate from "the extraction result being complete."

Candidate Generation

AI can also be used for work that produces ideas or candidates — for example, candidates like:

• title candidates
• design proposals
• test-case candidates
• candidate causes
• items to consider

In Candidate Generation, it can help to treat the AI's output not as a "final answer," but as a set of candidates for a human to review.

Work that requires current external information

On the other hand, there is work that cannot be answered without knowing the current external state — for example:

• today's weather
• the current version of some software
• the latest incident information
• the current Git remote state
• what files currently exist
• the current server state

For work that requires current external information, what's given to the model may not be enough by itself. The mechanisms for obtaining external information are covered in Chapter 8. However, the fact that "the AI answered" alone does not mean the current state has been confirmed.

Work that requires external execution

There is also a separate category of work that acts on the outside world — for example:

• modifying a file
• running a test
• executing a command
• calling an API
• committing to a repository
• changing a service's state

In this case, being able to explain what should be done and it actually being executed are different things. And it actually being executed and the expected result being confirmed are also different things. Tool requests and the mechanism of execution itself are covered in detail in Chapter 8.

This distinction is extremely important in AI-assisted development.

"Work AI is suited for" isn't fixed

It's not the goal of the examples in this chapter to establish a universal classification along the lines of "this kind of work should always be given to AI" or "this kind of work is absolutely impossible for AI." Conditions vary depending on the model, service, tools, input, task, and method of confirmation. What matters is: seeing what information the work needs, and what ultimately has to be confirmed.

Thinking in three axes

It helps to think this way when delegating work to AI:

A. Is the necessary information already given as input?

B. Is current external information needed?

C. Is something that needs to be done externally, with a result to confirm?

For example, summarizing text is mainly about A. Investigating current incident information makes B important. Fixing a program and running its tests all the way through makes C important too.

This three-axis framing is a learning model in this book for thinking about the information and confirmation needed in AI-assisted development.

Chapter summary

• AI can be used for Summarization, Rewriting, Translation, Q&A, Information Extraction, Candidate Generation, and more
• being able to execute a task and the result always being accurate are different things
• for work that requires current external information, what's given to the model may not be enough by itself
• for work that requires performing something externally, separate explanation, execution, and result confirmation
• don't judge AI use with a simple "can / cannot" binary
• think in terms of the three perspectives: input, external information, and external execution / result confirmation

The next chapter organizes how to avoid treating Context, Memory, and Training Data as the same thing when it comes to the information AI works with.