Connecting ChatGPT to Code Review Made Easy

The era of artificial intelligence is already already in bloom. Everyone working in IT is already familiar with our "new best friend" for development — AI. Working as a DevOps Engineer at Innovecs, I’d like to share one of my latest findings.

Concept

• Would you like every pull/merge request to be checked by ChatGPT-4 first and then by you?
• Do you want instant feedback on code changes before your colleagues see them?
• How about detecting who committed confidential data or API keys and where with the ability to tag the "culprit" for correction immediately?

We’re perfectly aware that GPT can generate code quite well. . . but it turns out it can review it just as smoothly! I will immediately show how this works in practice (parts of the code are blurred to avoid showing too much).

Step 1

I created a test merge request in which I added a JSON file with slightly broken formatting and plaintext passwords, recreating a security leak:

As we can see, AI not only detected the sensitive information but also masked it in its comment + tagged me to remove it.

Or here's another review:

Much more powerful analytics compared to tools like GitLeaks, which perform simple searches statically using regex and wildcards.

Step 2

Examples from frontend merge requests (colleague's screenshots), highlighting only certain points:

The GPT-4o model shows quite a good knowledge of React. Everyone knows this design with the key, but they always forget it.

Or here, it is like I’ve read a book:

I can give many examples, but the point is that its recommendations make sense! Of course, not all comments are worth attention, but a large part is invaluable.

Implementation

I'll explain how you can program an automated review process in Python in just one hour. This code should be added immediately to the CI/CD flow for maximum effect and process automation. It’s perfect to wrap it into one CI/CD job and execute it on each MR/PR (just make an exception for changes from renovate/dependabot).

I would love to share the ready-made code, but unfortunately, I can't (due to company restrictions). So, I'll share the idea with architectural blocks for you to piece it together like a constructor.

Let's get to work, take Python, and write the code.

Step 1: Connect To the AI Model

We need a connection to the AI model.

import openai

It could be GPT-4o from OpenAI. But I recommend Azure OpenAI, as Azure promises not to transfer code and not to train on it.

from openai import AzureOpenAI

Ask GPT how to do this if it's your first time.

Step 2: Code Changes and Comments

Get the code changes and comments on these changes. We use GitLab, so I'll immediately provide the endpoints as an example:

• Code changes using the MR changes API: /api/v4/projects/{gitlab_project_id}/merge_requests/{gitlab_request_id}/changes?access_raw_diffs=true
• Comments using the MR notes API: /api/v4/projects/{gitlab_project_id}/merge_requests/{gitlab_request_id}/notes?order_by=created_at&sort=asc

Where:

gitlab_request_id = os.getenv("CI_MERGE_REQUEST_IID")

gitlab_project_id = os.getenv("CI_PROJECT_ID")

For aesthetic purposes, the JSON response from GitLab needs to be parsed.

Step 3: Send Code for AI Review

Do not forget to add a prompt at the beginning with explanations on what to do, and put it all into one request:

review_request=f"{prompt}\n\n{notes}\n\n{changes}"

In the prompt, you need to politely ask the AI to analyze your code changes based on criteria — something like this (a very simplified version compared to what we use):

“As a Developer, I want to ask you to perform a GitLab Merge Request review.
Consider previous comments noted below and avoid repeating similar recommendations.
If you spot a recurring issue, skip it.

For security issues or sensitive information leaks, mention the assignee's username with @.

Make your feedback clear, concise, and actionable, with specific improvement recommendations.

Review the code snippet below based on these criteria:

• Syntax and Style: Look for syntax errors and deviations from conventions.
• Performance Optimization: Suggest changes to improve efficiency.
• Security Practices: Check for vulnerabilities and hard-coded secrets (mask half the info).
• Error Handling: Identify unhandled exceptions or errors.
• Code Quality: Look for code smells, unnecessary complexity, or redundant code.
• Bug Detection: Find potential bugs or logical errors.”

Step 4: Post Response as Comment

That’s it! Simply post the received response as a comment. 

Create a GitLab PAT token for this with the name AI MR review and direct POST to the MR notes API: /api/v4/projects/{gitlab_project_id}/merge_requests/{gitlab_request_id}/notes

Conclusions

Advantages

Implementing this solution will bring improvements for:

• DevSecOps: Significantly adds security
• Senior+ level: Now finding errors and shortcomings in MR/PR can be done without reading through the code — AI will do it for you.
• Development/QA: You immediately get surprisingly useful comments and recommendations.
• Business: Gets slightly better code at the output.

Disadvantages

• Cost: Difficult to predict; It all depends on how much you plan to submit for review and which model will conduct the review.

Risks

• Continuous improvement: Ideas for enhancing such a tool can be endless; more context does not always mean better review. Iterative testing of changes requires some time.
• “Improvements”: Ee may become even lazier =), but laziness is the engine of progress, right?

Prospects

Imagine the opportunities: it will be your script, so you can, for example:

• Add task context from Jira
• Create a summary for PM
• Write release notes/release changes
• Search for vulnerabilities

So let's make our code better and life easier. Welcome to the AI era, dear colleagues!