Black Box Tester in Python

Brief Problem Description

Imagine the situation: you (a Python developer) start a new job or join a new project, and you are told that the documentation is not up to date or is even absent, and those, who wrote the code, resigned a long time ago. Moreover, the code is written in a language that you are not familiar with (or “that you do not know”). You open the code, start examining it, and realize that there are no tests either. Also, the service has been working on Prod for so long that you are afraid to change something.

I am not talking about any particular project or company. I have experienced this at least three times.

Black Box

Well, you have a black box that has API methods (judging by the code) and you know that it pulls something and writes to a database. There is also documentation for those services that are receiving requests.

The advantages include the fact that it starts there is documentation on the API that it pulls, and the service code is quite readable. As for the disadvantages, it wants to get something via API. Something can be run in a container, and something can be used from a developer environment, but not everything.

Another problem is that requests to the black box are encrypted and signed as well as requests from it to some other services.

At the same time, you need to change something in this service and not break what is working.

In such cases, Postman or cURL is inconvenient to use. You need to prepare each request in each specific case since there are dynamic input data and signatures that depend on the time of the request.
There are almost no ready-made tests, and it is difficult to write them if you do not know the language very well.

The market offers solutions that allow you to run tests in such a service. However, I have never used them, so trying to understand them would be more difficult and would take much more time than creating my own solution.

Created Solution

I have come up with a simple and convenient option. I have written a simple script in Python that will pull this very application.

I used requests and a simple signature that I created very quickly for the requests prepared in advance.
Next, I needed to mock backends.

First Option

To do this, I just ran a mock service in Python. In my case, Django turned out to be the fastest and easiest tool for this.

I decided to implement everything as simply and quickly as possible and used the latest version of Django. The result was quite good, but it was only one method and it took me several hours to use despite the fact that I wanted to save time. There are dozens of such methods.

Examples of Configuration Files

In the end, I got rid of everything I did not need and simply generated JSON with requests and responses. I described each request from the front end of my application, the expected response of the service to which requests were sent, as well as the rules for checking the response to the main request.
For each method, I wrote a separate URL.

However, manually changing the responses of one method from correct to incorrect and vice versa and then pulling each method is difficult and time-consuming.

{
  "id": 308,
  "front": {
    "method": "/method1",
    "request": {
      "method": "POST",
      "data": {
        "from_date": "dfsdsf",
        "some_type": "dfsdsf",
        "goods": [
          {
            "price": "112323",
            "name": "123123",
            "quantity": 1
          }
        ],
        "total_amount": "2113213"
      }
    },
    "response": {
      "code": 200,
      "body": {
        "status": "OK",
        "data": {
          "uniq_id": "sdfsdfsdf",
          "data": [
            {
              "number": "12223",
              "order_id": "12223",
              "status": "active",
              "code": "12223",
              "url": "12223",
              "op_id": "12223"
            }
          ]
        }
      }
    }
  },
  "backend": {
    "response": {
      "code": 200,
      "method": "POST",
      "data": {
        "body": {
          "status": 1,
          "data": {
            "uniq_id": "sdfsdfsdf",
            "data": [
              {
                "number": "12223",
                "order_id": "12223",
                "status": "active",
                "code": "12223",
                "url": "12223",
                "op_id": "12223"
              }
            ]
          }
        }
      }
    }
  }
}

Second Option

Then I linked mock objects to the script. As a result, it appeared that there is a script call that pulls my application and there is a mock object that responds to all its requests. The script saves the ID of the selected request, and the mock object generates a response based on this ID.

Thus, I collected all requests in different options: correct and with errors.

What I Got

As a result, I got a simple view with one function for all URLs. This function takes a certain request identifier and, based on it, looks for the response rules — a mock object. In the meantime, the script that pulls the service before the request writes this very request identifier to the storage.

This script simply takes each case in turn, writes an identifier, and makes the correct request, then it checks if the response is correct, and that's it.

Intermediate Connections

However, I needed not only to generate responses to these requests but also to test requests to mock objects. After all, the service could send an incorrect request, so it was necessary to check them too. As a result, there was a huge number of configuration files, and my several API methods turned into hundreds of large configuration files for checking.

Connecting Database

I decided to transfer everything to a database.

My service began to write not only to the console but also to the database so that it would be possible to generate reports. That appeared to be more convenient: each case had its own entry in the database.

Cases are combined into projects and have flags that allow you to disable irrelevant options. In the settings, I added request and response modifiers, which should be applied to each request and response at all levels.

To simplify this as much as possible, I use SQLite. Django has it by default. I have transferred all configuration files to the database and saved all testing results in it.

Algorithm

Therefore, I found a very simple and flexible solution. It already works as an external integration test for three microservices, but I am the only one who uses it. It certainly does not override unit tests, but it complements them well. When I need to validate services, I use this Django tester to do that.

Configuration File Example

The settings have become simpler and are managed with Django Admin. I can easily turn them off, change, and watch history. I could go further and make a full-fledged UI, but this is more than enough for me for now.

Request Body JSON

{
  "from_date": "dfsdsf",
  "some_type": "dfsdsf",
  "goods": [
    {
      "price": "112323",
      "name": "123123",
      "quantity": 1
    }
  ],
  "total_amount": "2113213"
}

Response Body JSON

{
  "uniq_id": "sdfsdfsdf",
  "data": [
    {
      "number": "12223",
      "order_id": "12223",
      "status": "active",
      "code": "12223",
      "url": "12223",
      "op_id": "12223"
    }
  ]
}

Backend Response Body JSON

{
  "status": 1,
  "data": {
    "uniq_id": "sdfsdfsdf",
    "data": [
      {
        "number": "12223",
        "order_id": "12223",
        "status": "active",
        "code": "12223",
        "url": "12223",
        "op_id": "12223"
      }
    ]
  }
}

What It Gives You

In what way can this service be useful? Sometimes, even with tests, you need to pull services from the outside, or several services in a chain. Services can also be black boxes. A database can be run in Docker. As for an API...an API can be run in Docker as well. You need to set a host, port, and configuration files and run it.

Why the Unusual Solution?

Some may say that you can use third-party tools integration tests or some other tests. Of course, you can! But, with limited resources, there is often no time to apply all this, and quick and effective solutions are needed. And here comes the simplest Django service that meets all requirements.