Fundamentals of Logic Hallucinations in AI-Generated Code

Tools like GitHub Copilot, ChatGPT, Cursor, and other AI coding assistants can generate boilerplate, suggest algorithms, and even create full test suites within seconds. This accelerates development cycles and reduces repetitive coding work.

Hallucinations, however, are a common problem of AI-generated code. There are several types of hallucinations, and in this article, I will focus on some basic logical hallucinations.

AI is not guaranteed to understand the problem domain, business requirements, or architectural constraints. It generates outputs that appear syntactically correct and logically plausible but may conceal contradictions or omissions. These issues can be subtle, often passing unit tests or static analysis, yet surfacing later in integration, production, or customer-facing scenarios.

This article focuses on three key areas of logic hallucinations. Development code logic, testing logic, and architectural logic. For each area, we will explore examples and detection strategies.

Development Code Logic

A logic hallucination in development code is an AI-generated (or AI-influenced) artifact that may look syntactically sound and credible. However, it can be internally contradictory or misaligned with its stated purpose, surrounding system, or domain rules. Unlike syntax errors, these issues often compile, run, and pass tests.

Impossible Conditions/Unreachable Code

AI might generate code where a condition is always false, or a block of code can never be executed. This indicates a fundamental misunderstanding of the program's flow or the data's properties.

Example 1:

if (user.status == 'active' and user.status == 'inactive'):
    send_alert("Contradictory status detected!") # This line will never execute

Example 2:

if (isActive && !isActive) {
    sendNotification();
}

Example 3:

def process_age(age):
    if age > 0 and age < 0:  # Impossible condition
        return "valid"
    return "invalid"

Example 4:

def validate_input(data):
    if data is None:
        return False
        print("Data is None")  # Unreachable
    
    if len(data) == 0:
        return False
    else:
        return True
        cleanup_data(data)  # Unreachable

What to look for:

1. Boolean expressions that are always true/false
2. Nested if-statements that can never be reached
3. Conditions using and that contradict each other
4. Static analysis (unreachable branch checks)
5. Branch coverage reports (branch never covered)
6. Code after return statements
7. Code in else blocks when the if always returns
8. Exception handling that can never be triggered

9. Require branch justifications in PRs for complex conditions

Conflicting Loops/Circular Logic

Conflicts and contradictions in loops may arise in many ways. Self-contradictory loops may exist, for example, modifying the iteration variable in a way that prevents the loop from progressing as intended. Infinite loops may exist due to flawed termination conditions. There may be recursive functions without a proper base case, leading to stack overflows.

Example 1:

for i in range(10):
    # AI's attempt to optimize or add an unrelated feature
    # that inadvertently resets or modifies 'i' in a way that prevents normal iteration
    if some_condition:
        i = 0 # This could lead to an infinite loop or drastically alter intended sums
    total += data[i]

Example 2:

def calculate_total_with_tax(price):
    tax = price * 0.1
    price_with_tax = price + tax
    final_price = calculate_total_with_tax(price_with_tax)  # Infinite recursion
    return final_price

What to look for:

1. Functions calling themselves without proper base cases
2. Dependency chains that loop back
3. Order of operations that don't make sense
4. while loops where the termination condition is never met.

Contradictory State Changes

This occurs when the AI generates code that sets an object or variable to a specific state, only to immediately contradict that state. This is often due to a misunderstanding of if/else logic or business rules.

Example 1:

def update_user_status(user):
    user.is_active = True
    if user.subscription_expired():
        # Hallucination: AI correctly identifies the 'expired' condition
        # but assigns the *same* value, not the contradictory one.
        user.is_active = True  # Should be False
    return user

Example 2:

public Cart addItem(Item item) {
    this.items.add(item);
    this.totalPrice += item.getPrice();
    this.isEmpty = false;

    if (this.items.size() > 0) {
        // Redundant and potentially contradictory if logic was more complex
        this.isEmpty = false;
    } else {
        // This branch is now impossible because we just added an item
        this.isEmpty = true;
    }
    return this;
}

What to look for:

1. Variables being set to conflicting values in the same logical path.
2. Sequential assignments to the same state variable without intervening logic.
3. State changes that contradict business intent. State machine violations (e.g., setting status to 'Closed' then 'In Progress').
4. Missing else clauses that lead to default states being incorrectly applied.

Return/Test Contract Mismatch

Here, the AI generates a function whose implementation does not match its name, its documentation (docstring), or its implied contract. 

Example 1:

def get_active_user_count(users):
    """
    Finds all active users from a list and returns them.
    """
    active_list = [u for u in users if u.is_active]
    
    # Hallucination: The docstring says it "returns them" (a list),
    # but the code returns a number.
    return len(active_list)

Example 2:

/**
 * Retrieves a user from the database by ID.
 * Returns null if not found.
 */
public User getUserById(String id) {
    User user = database.find(id);
    if (user == null) {
        // Hallucination: The contract says return null, but the AI
        // decides to create a new user, violating the "get" premise.
        return new User(id, "default-guest");
    }
    return user;
}

What to look for:

1. Mismatches between function/method names and their bodies (e.g., a "get" function that modifies data).
2. Inconsistencies between docstrings/comments and the return statement.
3. Functions that have unexpected side effects (e.g., a calculate_ function that also saves to the database).
4. Unit tests that test for the wrong return type (e.g., assert count > 0 instead of assert isinstance(users, list)).

Test Code Logic

Logic hallucinations in test code are particularly dangerous because they undermine the primary safety net for catching other bugs. An AI-generated test that passes can create a false sense of confidence, allowing flawed application code to be merged and deployed.

Assertions Ignoring Setup

This is when a test meticulously sets up a specific scenario, but the assert statement fails to validate the outcome of that scenario. Instead, it asserts something trivial, a tautology, or a value that was true before the action was even performed.

Example:

def test_add_item_to_cart():
    cart = Cart()
    item = Item(name="Apple", price=1.50)
    
    # Action: The code under test
    cart.add_item(item)
    
    # Hallucination: The assertion checks the input data,
    # not the result of the 'add_item' action on the 'cart' object.
    # This test will pass even if 'cart.add_item' is empty.
    assert item.price == 1.50 
    
    # A correct assertion would be:
    # assert cart.get_total_items() == 1
    # assert cart.get_total_price() == 1.50

What to look for:

1. Assertions that check constants (e.g., assert 1 == 1).
2. Tests that assert the state of input variables rather than the output or mutated state of the system under test.
3. Tests that pass even if the main logic being tested is commented out.

Test Coverage Gaps

AI assistants are often optimistic. They may excel at generating tests for the "happy path"—where all inputs are valid and everything works as expected. They may omit, however, to generate tests for edge cases, error conditions, or invalid inputs.

Example: A test for calculate_shipping(weight):

def test_calculate_shipping_standard():
    # Happy path
    assert calculate_shipping(weight=10) == 5.00

The hallucination here is that this test is deemed to be sufficient. We are missing basic edge cases like for instance:

• test_calculate_shipping_zero_weight() (Should it be free or an error?)
• test_calculate_shipping_negative_weight() (Should raise ValueError)
• test_calculate_shipping_max_weight() (Test the boundary)
• test_calculate_shipping_non_numeric() (Should raise TypeError)

What to look for:

1. A lack of tests for null, None, empty lists, or zero-value inputs.
2. Missing assertions for expected exceptions (e.g., pytest.raises, assertThrows).
3. Test suites where all tests are positive assertions, with no negative test cases.
4. Relying on "lines covered" metrics, which don't show branch or condition coverage.

Incompatible Mocking

Mocks and stubs are used to isolate tests. An AI can generate a mock that is syntactically correct but does not match the real interface or behavior of the object it's replacing. This leads to tests that pass in isolation but fail dramatically during integration.

Example: The real DatabaseService returns a User object: User(id=1, name="Alice").

def test_get_user_name_display():
    # Hallucination: The AI mocks the service to return a simple string.
    mock_db = Mock()
    mock_db.get_user.return_value = "Alice" # Real service returns User(id=1, name="Alice")
    
    # This code expects a User object, so it will fail:
    # service.get_user_display_name(mock_db, 1) -> "Logged in as: Alice.name" (AttributeError)
    
    # But the AI writes a test that works with its own flawed mock:
    username = mock_db.get_user(1)
    assert username == "Alice" # This test passes, but it tests nothing.

What to look for:

1. Mocks that return simple types (strings, ints) when complex objects are expected.
2. Mocks that don't match the argument signature of the real method.
3. Lack of "autospeccing" (like Python's create_autospec) which forces mocks to conform to the real object's interface.

Context Consistency Failure

An AI may generate a series of tests in a file that are not properly isolated. One test may "pollute" a global or static state (like a database connection or a singleton), causing subsequent tests to fail or, worse, to pass for the wrong reasons.

// Global static list to "mock" a database
static List<String> userDb = new ArrayList<>();

@Test
public void testAddUser() {
    userDb.clear(); // This test clears, good.
    userDb.add("testUser");
    assertEquals(1, userDb.size());
}

@Test
public void testUserCount() {
    // Hallucination: This test assumes an empty DB, but 'testAddUser'
    // might have run before it, leaving "testUser" in the list.
    // This test is "flaky"—it depends on execution order.
    assertEquals(0, userDb.size()); 
}

What to look for:

1. Tests that fail when run in a different order or in parallel.
2. Lack of proper setup() and teardown() methods (or fixtures) to reset state between each test.
3. Use of global or static variables in test files.

Architectural Logic Hallucinations

These are high-level, systemic hallucinations. The generated code is functionally correct in isolation but violates the fundamental design principles, patterns, or constraints of the larger application.

Architectural Contradictions/Violations

This occurs when the AI, often focusing on a single function, generates code that breaks established architectural rules, such as layer separation (e.g., MVC, 3-Tier).

Example: In a strict 3-tier architecture (Controller -> Service -> Repository), the AI is asked to "add an endpoint to get active users."

# In Controller.py (The wrong layer)

@app.route('/active_users')
def get_active_users():
    # Hallucination: The AI bypasses the Service and Repository layers
    # and directly queries the database from the Controller.
    # This is a major architectural violation.
    db_conn = get_db_connection()
    users = db_conn.execute("SELECT * FROM users WHERE status = 'active'")
    return jsonify(users)

What to look for:

1. import statements that cross architectural boundaries (e.g., a View or Controller file importing a Database or ORM library).
2. Business logic (calculations, complex rules) appearing in the UI or Controller layers.
3. Data access code (SQL, ORM calls) appearing anywhere except the Repository or Data Access Layer.

Context Window Limitations

An AI's "memory" (context window) is finite. It can't see your entire codebase. This leads to it "forgetting" a crucial constraint, custom utility, or design pattern that was defined in another file or earlier in a long conversation.

Example: Your project has a custom structured_logger.py that must be used for all logging. You ask the AI to add error handling to a function.

# The AI, unaware of or forgetting 'structured_logger.py',
# falls back to a generic (and forbidden) built-in.
def process_data(data):
    try:
        # ... complex logic ...
    except Exception as e:
        # Hallucination: Violates the project's logging standard.
        print(f"An error occurred: {e}") 
        
        # Correct implementation:
        # from my_app.structured_logger import logger
        # logger.error("data_processing_failed", error=str(e), data_id=data.id)

What to look for:

1. Inconsistent use of helper functions, custom utilities, or constants.
2. Re-implementation of logic that already exists in a utility module elsewhere in the project.
3. Use of generic, built-in functions when custom, project-specific ones are required.

Meta-Logic Inconsistency

This is when the AI contradicts its own high-level advice. It might suggest one design pattern ("We should use the Factory pattern here"), but then generate code that implements a different, simpler pattern (like a basic switch statement), or vice versa.

Example: 

Developer: "How should I handle notifications for email, SMS, and push?" 

AI: "You should use a message queue like RabbitMQ to decouple these services. The main app will publish a 'notification_request' event." 

Developer: "Okay, generate the code for the user profile service to send a password reset notification." 

AI:

# Hallucination: The AI ignores its own advice about message queues
# and generates a direct, synchronous call.
class UserProfileService:
    def __init__(self):
        self.email_service = EmailService() # Direct coupling

    def request_password_reset(self, user):
        token = generate_token()
        # This blocks the main thread and couples the services.
        self.email_service.send_reset_email(user.email, token)

What to look for:

1. Code that violates a design pattern that was just discussed or agreed upon.
2. Suggestions for one pattern followed by the implementation of another.
3. Mixing architectural styles (e.g., synchronous and asynchronous logic, polling and eventing) without clear reasoning.

Wrapping Up

AI-assisted development marks a new era of productivity. However, it also creates a new species of failure: the plausible illusion. The code runs. The tests pass. The architecture seems compliant. And yet, the business may fail.

AI-code assistants appear to be overconfident junior developers. We should treat every AI suggestion as code from a brand-new, brilliant, but dangerously naive intern. Always assume it's missing the context. Implement a second opinion rule. After getting a suggestion, always ask a follow-up: "Is this code thread-safe?" "What are the performance implications of this lock?" "Refactor this code to be idempotent." Towards this end, this article has explained a number of logic hallucinations in AI-generated code. For each case presented, I also proposed what to look for in a second opinion rule.

AI doesn't replace expertise; it demands more of it. Our job is no longer just to write code. We should skillfully manage — and rigorously question — a team of infinitely fast, infinitely confident, and occasionally nonsensical digital interns. The only defense is a human-guided QA immune system — a layered verification process that tests not only what the AI wrote, but whether the logic, rules, and architecture still make sense together.