AI overconfidence risks

Introduction

The adoption of AI in projects is sparking heated debate. Caught between technophilia and technophobia, an avalanche of information and the difficulty of processing the ever-more-frequent announcements from AI providers (who must appeal to both users and investors) it is difficult to make sense of it all.

Within our group, the levels of adoption of AI tools vary greatly. We have developers who do not use AI at all, right through to those who have delegated all their coding to agents.

As an architect and member of the IT experts team, I’m wondering what stance to take.

To this end, I’ve conducted a small experiment on an innersource tool that I lead. It enables the management of architectural decisions, roadmaps and other aspects.

Goal of the experience

As not all teams follows best practices, the goal of my experience is to see what could happen to a code base with basic code generation approach:

• evaluate differences between models, at an instant
• see emerging accidental anti-patterns
• evaluate risks on the code base evolution
• identify weak signals to trigger in-depth analyses made by software architects, experts or tech lead

I don't to evaluate the best possible output of AI

Project's context

One of the product’s key features is to facilitate the creation of high-quality ADR (Architecture Decision Records), with a good level of traceability thanks to the use of Git and GitLab. The aim is to enable all stakeholders to contribute to decisions, even without any knowledge of Git (as the tool makes the commits).

Each architect creates GitLab repositories and uses them to facilitate collaboration with the teams with whom they seek to make decisions.

A small community of contributors is responsible for the product’s development.
When I began implementing the tool, I saw it both as a genuine solution to everyday problems and as a testing ground with limited risks.

The project is two and a half years old and comprises multiple technical components, the main one of which handles interaction with end users.

Some information :

• Application is in Python
• The source code is 10 kLoc (Kilo Line of codes)
• The tests are 16 kLoc (Test:Code ratio – 1.6:1)
• 96% coverage; tests run in 15–20 seconds in the IDE
• Based on Clean Architecture, using TDD and BDD (tests drive the specifications)
• Several hundred users in the group (architects, business analysts, product owners, developers, tech leads, etc., ops, etc.)

I’m a moderate user of AI; I don’t "vibe code" but I use mostly LLMs as agentic assistants. For example:

• Claude Opus fixed a performance issue in my unit tests where execution times had skyrocketed following some changes
• GPT Codex fixed my linter configuration in a matter of minutes to catch errors as early as possible

For the sake of experiment, I decided to take it a step further by applying different models to the development of a feature.

Methodology

The test conditions are as follows:

• I used Copilot cli with the --yolo option (XD), i.e. without any human interaction (AI had full access to my terminal)
• I didn't use skills or MCP
• Each AI worked on a separate branch starting from the same commit
• I submitted the same prompt to each AI, along with a specs.md file describing the requirement
• The evaluation has been made in April 2026.

** As a python developer you'll need to implement the following specs**

**You must notify me when you have finished and how many time you spent to implement the solution**
# Project infos

- Current project is a python web app
- it's configured in pyproject.toml
- it's a continuous delivery project used widely internally, be careful
- Sources are in src
- Domain code is in src/adrgen/domain
- Infrastructure code is in src/adrgen/infrastructure
- Entry points code is in src/adrgen/interface
- Tests are in tests
- Domain tests are in tests/domain
- Infrastructure tests are in tests/infrastructure
- Entry points tests are in tests/interface

# Specs

Deleting an ADR is a demanded feature. Today we can only abandon ADR but sometimes we just don't want to see it.

# Acceptance Criteria

* Having complete test stack
* Respect the clean architecture
* ADR model has a remove method checking that only ADR without releases (in progress or abandoned with one decision) can be removed
* An indicative label "Delete ADR" is added
* When ADR is added, all labels related to ADR (maturity, ADR, Fix validation errors) are removed from the issue
* The issue is not deleted, only the snapshot in snapshots directory
* After the deletion a message is added in the issue to notice the user that ADR has been removed
* Label "Delete ADR" is removed after deletion
* An "not-an-adr" label, ADR deleted is added to the issue to specify the success of the operation and to avoid recreating a new ADR (as issue are ADR by default)

For the evaluation criteria, I have selected a range of qualitative and quantitative criteria:

• The speed of the model’s implementation
• Compliance with the architecture
• The quality of the test code
• The quality of the production code
• Whether or not I had to intervene in order to test the MR
• Validation of the implementation in a test environment
• Additivity rate: (% of lines added / % of changes).

Additivity rate

To maintain a codebase over the long term, complexity must be kept at a sufficiently low level.

To achieve this, developers must regularly make changes to the existing codebase (reusing code, adding or modifying tests to make them more readable, renaming variables, etc.).

There is a rule in software craftsmanship: the Boy Scout Rule. Always leave the codebase cleaner after a change than it was before. Thanks to this, we gradually improve the codebase, even though more extensive refactoring phases still take place.

If you never refactor your code, you have a problem.

To easily assess compliance with the ‘Boy Scout Rule’, I suggest using an indicator I name ‘additivity rate’. This is based on a simple ratio of the number of lines added to the total number of lines.

The indicator varies between 0 and 100%.

The baseline for my evaluation:

• Under 50%, it's all about cleaning. The author removed more code than he added
  • Around 5,4% of the Gitlab ADR Generator MR are under this threshold (13MR)
• As developers spends more time adding new stuff than cleaning existing base code, majority of the values must be greater than 50% and lower than 80%
  • Adding features while refactoring
  • Around 77.2% MR of Gitlab ADR Generator (183MR)
• Between 80% and 90%
  • Concernes new modules with a lot of new code segments
  • Around 11.8% of MR of Gitlab ADR Generator (28MR)
• Once you exceed 90%, it’s almost entirely new code; this is a figure that must be very rare in a project with an existing codebase.
  • For example: MR with small changes (add one or two line of configuration), or features at the very beginning of a project
  • Around 5.4% of gitlab ADR generator MR have a ratio below 90%. 10 have a ratio under strictly 0.96. Three (1.2%) have a ratio of 1, but are very small (less than 24 lines of code).

Models

For the experience I evaluated three models:

• GPT 5-4 mini
• GPT 5.3 Codex
• Claude Opus 4.6

GPT 5-4 mini, the challenger

GPT 5.4 took around 25 minutes to implement the merge request (which is quicker than I could have managed for a first draft).

I had very low expectations of GPT 5.4 mini, having seen it produce some truly questionable code during my previous attempts.

It is the only model where I had to intervene to complete my end-to-end test. I had to correct some tests and remove a section of duplicate code. The end-to-end test then worked.

It should also be noted that it is the only one to have modified the user guide.

Architectural compliance

As regards adherence to the application architecture, it is the only one to have correctly placed the code that adds the labels:

def set_labels_after_adr_deletion(self, application_id: ApplicationId, user_input: DecisionUserInput):  

It has adhered to the clean architecture, and the business logic is in the right place.

However, in the command handler, it made a mistake by calling the ‘user input repository’, which should have been handled in an event handler:

if DELETE_ADR_LABEL in user_input.labels_with_indicative_labels:
  deleted_adr = adr.remove()
  self.__adr_domain_repository.save(deleted_adr)
  self.__user_input_repository.set_labels_after_adr_deletion(application_id, user_input)
  return

Test code

As for the test code, the model did a good job; they implemented numerous high-level integration tests. These are very important for the solution, as this is how I specify the application’s expected behaviour.

Although I use an internal API, it used it correctly:

def test_delete_adr_label_should_delete_snapshot_and_mark_issue_as_deleted(self):
  # Given
  container = given_application_up_to_date(
      set_adr_with_one_decision_in_progress(gitlab_issue_labels=[ADR_LABEL, DELETE_ADR_LABEL])
  )

  # When
  run_adr_generator_pipeline(container)

  # Then
  assert_that_file_has_been_deleted_once(container, file_path_of_adr_snapshot(DEFAULT_ADR_UNIVERSAL_ID))
  issue = container.get(ProjectIssueManager).get(DEFAULT_ADR_LOCAL_ID.value)
  self.assertIn(ADR_DELETED_LABEL.full_name, issue.labels)
  self.assertNotIn(DELETE_ADR_LABEL.full_name, issue.labels)
  self.assertNotIn(ADR_LABEL.full_name, issue.labels)

Production code

The model has a real problem with managing code duplication. It has copied existing sections without making any effort to share them.

For example, in this function:

def set_labels_after_adr_deletion(self, application_id: ApplicationId, user_input: DecisionUserInput):
  issue: ProjectIssue = self.__gitlab_project_repository.find_project_issue_by_id(
            to_gitlab_project_id(application_id), GitlabIssueId(user_input.adr_local_id.value)
        )
  issue_labels = list(map(lambda label: LabelId.parse(label), issue.labels))
  labels_to_keep = list(
      filter(
          lambda label: (
              self.__filter_label(label)
              and label != DELETE_ADR_LABEL
              and label != ADR_LABEL
              and label != FIX_VALIDATION_ERRORS_LABEL
          ),
          issue_labels,
      )
  )
  target_issue_labels = set(labels_to_keep).union({ADR_DELETED_LABEL})
  target_issue_labels_full_name = list(map(lambda label: label.full_name, target_issue_labels))
  if set(target_issue_labels_full_name) != set(issue.labels):
      issue.labels = target_issue_labels_full_name
      issue.save()

The lambda that filters the labels already exists in other parts of the application:

filter(
          lambda label: (
              self.__filter_label(label)
              and label != DELETE_ADR_LABEL
              and label != ADR_LABEL
              and label != FIX_VALIDATION_ERRORS_LABEL
          ),
          issue_labels,
      )

Verdict

Processing time: 25 minutes

Compliance with architecture: 3/5

Production code quality: 2.5/5

Test code quality: 3/5

Additivity rate: 92.09%

Human intervention: Yes (incorrect code and test corrections)

End-to-end testing: After intervention, the feature was working

Claude Opus 4.6, a slight disappointment

Before the benchmark, I had high hopes for Claude Opus 4.6, given the model’s reputation. But the results for implementing a feature didn’t really live up to my expectations.

Claude Opus was incredibly quick to deliver its code: 5 minutes.

Yet with an additivity rate of almost 99%, it’s a real machine for generating duplication. I wasn’t expecting that at all.

Architectural compliance

As far as adherence to the architecture is concerned, there are no issues to report.

For example, it created an event handler to update the status of the GitLab issue, which is standard practice in the application:

class ResolveDeleteADREventHandler(EventHandler):
    def __init__(
        self,
        gitlab_project_repository: GitlabProjectRepository,
        all_adrs_ids_repository: AllADRsIdsRepository,
        application_id: ApplicationId,
    ):
      self.__gitlab_project_repository = gitlab_project_repository
      self.__all_adrs_ids_repository = all_adrs_ids_repository
      self.__application_id = application_id

    def handle_events(self, events: List[DomainEvent]):
      removed_events = list(filter(lambda event: isinstance(event, ADRRemovedDomainEvent), events))

However, the code should not be placed directly within the event handler, but in a dedicated repository.

The entire domain logic was in the right place.

Test code

This is by far the most disappointing part of the code generated by Claude.

It only tested the domain and did not include any integration tests:

@pytest.mark.unit
class ADRRemoveTest(unittest.TestCase):
    def test_remove_new_adr_should_emit_removed_event(self):
        # Given
        adr = create_new_adr_data()

        # When
        removed_adr = adr.remove()

        # Then
        assert_events_are_same(
            removed_adr.uncommited_events,
            [
                create_adr_removed_domain_event(
                    header=create_decision_on_adr_event_header(
                        most_recent_decision_maturity=DecisionMaturity.IN_PROGRESS,
                    ),
                    decision_universal_id=adr.decision_universal_id,
                )
            ],
        )

That is far from sufficient in terms of the project criteria.

Production source

The quality of the source code is quite good and readable, although once again there is a real issue with code reuse, with absolutely no effort having been made in this regard.

Verdict

Processing time: 5 minutes

Architectural compliance: 4/5

Source code quality: 3.5/5

Test code quality: 1.5/5

Additivity rate: 98.81%

Human intervention: None required

End-to-end test: Worked straight away

GPT 5.3 CODEX, a pleasant surprise

Just like GPT 5.4 mini, GPT 5.3 Codex took a long time to run (28 minutes) but produced working code.

Architectural compliance

The model has adhered to the architecture and implemented the necessary classes (for example, an event handler to remove labels), but it tends to concentrate too much code in one place.

Test code

In this respect, GPT 5.3 CODEX proved to be both the best and the worst: it used unit tests – which were just as good as Claude’s – as well as integration tests.

@pytest.mark.parametrize(
    "adr,expected",
    [
        (create_new_adr_data(), True),
        (create_abandoned_adr_data(), True),
        (create_released_adr_data(), False),
        (
            create_adr_data(
                decisions=[
                    create_decision_abandoned(adr_version_number=ADRVersionNumber(1)),
                    create_decision_abandoned(adr_version_number=ADRVersionNumber(2)),
                ]
            ),
            False,
        ),
    ],
)
def test_can_be_removed_only_for_eligible_adrs(adr: ADR, expected: bool):
  assert adr.can_be_removed() == expected, f"Expected can_be_removed={expected} for adr {adr}"

Or else :

def test_delete_adr_label_should_remove_snapshot_and_mark_issue_as_not_an_adr(self):
  # Given
  container = given_application_up_to_date(
      set_adr_with_one_decision_in_progress(
          gitlab_issue_labels=[DELETE_ADR_LABEL],
      )
  )

  # When
  run_adr_generator_pipeline(container)

  # Then
  assert_that_adr_domain_event_has_been_sent(
      container, adr_universal_id=DEFAULT_ADR_UNIVERSAL_ID, event_type=DecisionRemovedOnADRDomainEvent
  )
  issue = container.get(ProjectIssueManager).get(DEFAULT_ADR_LOCAL_ID.value)
  self.assertNotIn(DELETE_ADR_LABEL.full_name, issue.labels)
  self.assertIn(ADR_DELETED_NOT_AN_ADR_LABEL.full_name, issue.labels)

However, it generated a test (which, incidentally, passed incorrectly), contrary to the application’s specifications.

In fact, when creating an ADR, the ADR or decision maturity labels must be present (this is even a feature of the latest version of the tool).

I don’t understand why it generated this test, as it was not within the scope of the merge request.

def test_creating_adr_should_remove_indicative_adr_labels_from_issue(self):
  # Given
  container = given_application_up_to_date(
      set_adr_candidate_with_modification_in_progress(
          labels=[
              ADR_LABEL,
              decision_maturity_label_id(DecisionMaturity.IN_PROGRESS),
              FIX_VALIDATION_ERRORS_LABEL,
          ]
      )
  )
  user_input_repository = container.get(UserInputRepository)

  # When
  run_adr_generator_pipeline(container)

  # Then
  adr_id = get_universal_id(container)
  user_input = user_input_repository.load_decision_user_input(DEFAULT_APPLICATION_ID, adr_id)
  self.assertNotIn(ADR_LABEL, user_input.labels_with_indicative_labels)
  self.assertNotIn(
      decision_maturity_label_id(DecisionMaturity.IN_PROGRESS), user_input.labels_with_indicative_labels
  )
  self.assertNotIn(FIX_VALIDATION_ERRORS_LABEL, user_input.labels_with_indicative_labels)

Production code

As with GPT-5, some sections of code need to be rewritten.

For example, here, these two sections of code are simply one below the other :

for event in created_events:
    created_event: ADRCreatedFromADRCandidateEvent = event
    user_input = self.__user_input_repository.load_decision_user_input(
        self.__application_id, created_event.header.aggregate_id
    )
    if user_input is None:
        continue

    issue = self.__gitlab_project_repository.find_project_issue_by_id(
        to_gitlab_project_id(self.__application_id), GitlabIssueId(user_input.adr_local_id.value)
    )
    issue_label_ids = set(map(lambda label: LabelId.parse(label), issue.labels))
    issue_label_ids = set(
        filter(
            lambda label: (
                label.category != DECISION_MATURITY_CATEGORY
                and label != ADR_LABEL
                and label != FIX_VALIDATION_ERRORS_LABEL
            ),
            issue_label_ids,
        )
    )
    issue.labels = sorted(list(map(lambda label: label.full_name, issue_label_ids)))
    issue.save()

for event in removed_events:
    removed_event: DecisionRemovedOnADRDomainEvent = event
    user_input = self.__user_input_repository.load_decision_user_input(
        self.__application_id, removed_event.header.aggregate_id
    )
    if user_input is None:
        continue

    issue = self.__gitlab_project_repository.find_project_issue_by_id(
        to_gitlab_project_id(self.__application_id), GitlabIssueId(user_input.adr_local_id.value)
    )
    issue_label_ids = set(map(lambda label: LabelId.parse(label), issue.labels))
    issue_label_ids = set(
        filter(
            lambda label: (
                label.category != DECISION_MATURITY_CATEGORY
                and label != ADR_LABEL
                and label != FIX_VALIDATION_ERRORS_LABEL
                and label != DELETE_ADR_LABEL
            ),
            issue_label_ids,
        )
    )
    issue_label_ids.add(ADR_DELETED_NOT_AN_ADR_LABEL)
    issue.labels = sorted(list(map(lambda label: label.full_name, issue_label_ids)))
    issue.save()

LLMs are capable of causing significant duplication issues.

Verdict

Processing time: 28 minutes

Architectural compliance: 4/5

Code quality: 2.5/5

Test code quality: 2,5/5

Additivity rate: 94.44%

A careful reading of the code was required to spot the error in the test generation. It’s a shame, as it had generated a lot of tests that were correct (which increases the risk of failing to spot errors due to overconfidence).

Results analysis

Each model succeeded (with only minor adjustments) in producing code that worked straight away and met the specifications.

Speed

The speed of the models – without human interaction – was truly impressive, particularly Claude Opus.

I wouldn’t have been able to code so many changes in five minutes, especially as they were functional.

But it's necessary to add the time used to carefully review the changes.

That’s why it’s important to look closely at the total implementation time, not just the generation time.

Code review is a complex and costly exercise. As AI tends to be very verbose, this cost increases compared to a human code.

Specifications

I wrote the specification very quickly (~10 minutes), expecting the result to fall short of my expectations. But the AI systems produced working code just with short description.

To achieve a function that works ‘right first time’, I'll need to invest much more time in the specifications and adopt a more precise format (such as the Given-When-Then style, for example).

Indeed, it is very difficult to anticipate all edge cases in advance. The development phase allowed these to be identified, but when entrusting everything to the AI without interaction, this work must absolutely be carried out beforehand.

Code quality

The code is generally good and adheres to the project’s coding style. It is readable and clear. However, there are a few areas that are substandard and need to be revised.

In the tests, the results were mixed, and the strengths and weaknesses varied considerably from one model to another.

When analysing the code, I noticed that the AI exhibited mimetic behaviour, which can cause problems.

Mimicry

The example below illustrates this phenomenon well: to implement the change, the models had to modify two configuration files.

Here, I used the value ‘True’ in the file, with a capital letter immediately above it; the models did the same:

    immutable: True
    description: "Fix validation errors of the decision to validate it."

  - name: "Delete ADR"
    color: Light Gray
    indicative: True
    immutable: True
    description: "Delete the ADR snapshot and keep the issue as a regular non-ADR issue."

And in this other file, I used ‘true’ in lower case. The code added by the templates mimics the nearby ‘true’ :

    immutable: True
    indicative: true

  - name: "ADR deleted"
    color: Light Gray
    description: "The ADR snapshot has been deleted and this issue is no longer considered as an ADR."
    not-an-adr: true
    immutable: true
    indicative: true

This means that the immediate environment serves as inspiration when writing code (which explains why some tests are written exactly as if I had written them myself).

Sometimes practices evolve and it becomes necessary to standardise usage. Developers should be aware of the current standard and give specific instructions to model (in skills for example).

However, I question myself for problematic code. This point warrants further investigation.

Additionally, I found an interesting indicator: the additivity rate.

Additivity rate & Code Bloat

Only 5.4% of 237 previous MR were beyond 90% of additivity rate and only 1.2% beyond 95%.

But AI vibe coded MR additivity rates were greater than 90% and one was even ~99%. I did not have such a case in the past for an evolution.

Whilst this is relatively logical for a brand-new section of code, such a rate should remain rare, as there are always instances of code reuse and adjustments to existing code to improve its structure.

But that is not the case here, and models has missed several opportunities to reuse code.

In my three previous changes, which I carried out with the AI as an assistant, the additivity rate was 73%, 72% and 66%.

Between the last minor release and the current major release, the rate stands at 64.5% (as a result of refactoring the codebase).

If left unchecked, a rate greater than 90% risks triggering a ‘Code Bloat’ – an insidious form of degradation that is gradual and imperceptible in an isolated MR. This poses a real risk to a project, potentially leading to degradation or even making it impossible to develop the application further in the future. The symptoms include a dramatic increase in debugging and fix times, and an inability to deliver without introducing regressions.

Conclusion

AI is often presented as inevitable. Whilst it is true that its adoption and use have skyrocketed, this characterisation is dangerous. It makes us forget that, like any tool, it has its limitations, it makes mistakes, and it must be monitored.

Using AI is a choice, and as every choice, risks and benefits must be evaluated.

Of course, you might challenge my prompt, saying that human agents can also handle code review. However, my aim in this post is not to establish the best possible use of AI, but to remind readers that this tool risks derailing projects without careful monitoring.

That even with safeguards in place, AI is not deterministic and can introduce flaws and regressions. That maintaining a human-maintained codebase, particularly for critical code, remains essential. I believe it is important to be able to switch it off (the recent economical model switch from subscription to token based pricing is a good example).

Has AI saved me time? For some tasks, yes, but for others, such as writing features, I’m far more ambivalent. The effort required for quality control is significant, and does not draw on the same cognitive skills. Being a good coder does not mean being a good quality controller.

To avoid code bloat, I recommend monitoring the evolution of the codebase to ensure that refactoring is carried out properly in a project that makes intensive use of AI.

This is how I use AI for coding:

• Vibe Coding for proofs of concept and very narrowly scoped technical functions, with strict oversight
• AI as an assistant for features; I prefer to continue writing the tests myself, as they serve as my specifications

What articles shines:

• Some weird code pattern, generated by LLMs
• Raised risks of AI without control (imitation of existing code, code bloat).
  • To improve the output of the AI many good practices emerged like use markdown file proposed in this repository https://github.com/multica-ai/andrej-karpathy-skills
• Additivity rate as a potential signal to detect excessive complexity in code base to help you keeping complexity low

In my next article, I will dive deep into our codebase to see if additivity indicators can act as an early warning system for projects at risk of AI-induced code bloat. Stay tuned