DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Related

  • Build a Simple REST API Using Python Flask and SQLite (With Tests)
  • Chat Completion Models vs OpenAI Assistants API
  • Idea to Running: One Minute
  • Model-Driven Development and Testing

Trending

  • MCP Servers: The Technical Debt That Is Coming
  • Optimizing Integration Workflows With Spark Structured Streaming and Cloud Services
  • Can You Run a MariaDB Cluster on a $150 Kubernetes Lab? I Gave It a Shot
  • Unlocking Data with Language: Real-World Applications of Text-to-SQL Interfaces
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. Leveraging LLMs for Software Testing

Leveraging LLMs for Software Testing

Explore how LLMs enhance Python testing by automating test case generation, improving test coverage, reducing maintenance, and supporting efficient workflows.

By 
Pradeesh Ashokan user avatar
Pradeesh Ashokan
·
Mar. 17, 25 · Analysis
Likes (2)
Comment
Save
Tweet
Share
2.5K Views

Join the DZone community and get the full member experience.

Join For Free

As software systems become more complex, traditional testing methods often fall short in keeping up with the fast-paced development cycles and changing user needs. Fortunately, advancements in machine learning and generative AI are bringing intelligent and adaptive testing strategies that improve test coverage and decrease maintenance efforts to speed up the entire testing process. 

This article details using large language models (LLMs) to test a Python codebase project.

Benefits of LLMs in Test Automation

Increased Efficiency and Speed

LLMs can greatly accelerate the testing process by automating tasks such as test case generation, execution, and analysis. This automation allows testing teams to concentrate on more strategic activities, such as exploratory testing and test planning.

Improved Test Coverage

LLMs can enhance test coverage by identifying edge cases and generating test scenarios that manual testing might overlook. This results in a more comprehensive and robust testing process, reducing the risk of defects being released into production.

Reducing Test Script Maintenance

LLMs can analyze code changes and automatically update test scripts. This process significantly minimizes manual effort and the potential for errors.

Test Case Generation for a User Story Using ChatGPT

The following Python code uses OpenAI's API to generate test cases for a given user story. It sets an API key, defines a generate_test_case.py function that creates a prompt using the user story and interacts with the GPT-4 model to generate test cases. 

This helps with the bootstrapping required for creating manual test cases.

Python
 
from openai import ChatCompletion

openai.api_key = "your-api-key"

def generate_test_case(user_story):
    prompt = f"Write test cases for: '{user_story}'"
    response = ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message['content']

user_story = "As a user, I want to reset my password to regain account access."
print(generate_test_case(user_story))


Pytest Case Generation for a Python Function Using ChatGPT

Pytest is Python's popular testing framework. The following Python code uses OpenAI's GPT API to generate pytest test cases for a calculate_bmi function, that calculates BMI based on weight and height and categorizes it ("Underweight," "Normal weight," etc.). 

Using the inspect module, the script extracts the function's source code and prepares a prompt asking GPT to generate parameterized test cases, including edge cases like invalid or zero inputs. This method is very effective in generating automated pytest cases, which can then be included as a separate test file in the project.

Python
 
import openai
import inspect

# Set up your API key
openai.api_key = "your-api-key-here"

# Calculate BMI function part of your app's feature
def calculate_bmi(weight, height):
    if height <= 0 or weight <= 0:
        raise ValueError("Height and weight must be greater than zero.")
    bmi = weight / (height ** 2)
    if bmi < 18.5:
        return "Underweight"
    elif 18.5 <= bmi < 24.9:
        return "Normal weight"
    elif 25 <= bmi < 29.9:
        return "Overweight"
    else:
        return "Obesity"


# Get the function source code as a string
function_code = inspect.getsource(calculate_bmi) 

# Define the prompt
prompt = f"""
Generate pytest test cases for the following Python function:
{function_code}
Include edge cases such as invalid inputs (zero or negative values), and use parameterized tests where possible.
"""

# Make the API call
response = openai.ChatCompletion.create(
    model="gpt-4",     
    messages=[
        {"role": "system", "content": "You are a Python testing assistant."},
        {"role": "user", "content": prompt}
    ],
    max_tokens=500,
    temperature=0.7
)

# Print the response
print(response['choices'][0]['message']['content'])


Pytest Cases Generation Without Passing Function Code

In some situations, sharing code directly may not be feasible due to privacy or security concerns. However, ChatGPT can still assist in these cases.

  • Function signatures. Using only the function's name, parameters, and return type, ChatGPT can infer its purpose and create relevant test cases.
  • Code descriptions. By providing a detailed description of the code's functionality, developers can guide ChatGPT to generate appropriate tests.
Python
 
import openai
import inspect
openai.api_key = "your-api-key-here"

signature = inspect.signature(calculate_bmi)
docstring = inspect.getdoc(calculate_bmi)

prompt = f"""
Generate pytest test cases for the following Python signature and docstring:
signature - {signature}
docstring - {docstring}
Include edge cases such as invalid inputs (zero or negative values), and use parameterized tests where possible.
"""

# Make the API call
response = openai.ChatCompletion.create(
    model="gpt-4",     
    messages=[
        {"role": "system", "content": "You are a Python testing assistant."},
        {"role": "user", "content": prompt}
    ],
    max_tokens=500,
    temperature=0.7
)

# Print the response
print(response['choices'][0]['message']['content'])


Conclusion

LLM-powered test automation represents a significant advancement in software quality assurance. While challenges exist, organizations that successfully implement these technologies gain substantial advantages in testing efficiency, coverage, and reliability. 

As AI capabilities evolve, we can expect even more sophisticated testing approaches that will further improve software quality and reduce manual effort.

API Software testing Python (language)

Opinions expressed by DZone contributors are their own.

Related

  • Build a Simple REST API Using Python Flask and SQLite (With Tests)
  • Chat Completion Models vs OpenAI Assistants API
  • Idea to Running: One Minute
  • Model-Driven Development and Testing

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!