Packaging Python Classes as a Wheel

Introduction

Recently, while working on a project where we were using Python, I realized how easy the Java package mechanism makes it, to organize and reuse common classes and functions.

Due to my Java hangover, I organized the common classes and functions in a directory hierarchy like com.<company>.<project>.util. The util package contained a few classes for UUID creation as well as classes for date and time manipulation. After placing the files in the above-mentioned directory, I started developing the application as a micro-service. I placed the micro-service code in the directory com.<company>.<project>.<service>. While trying to use the date and time classes in the micro-service, I used the syntax from com.<company>.<project>.util.mydatetime import MyDateTime.

This syntax threw a compilation error, as the Python environment was not able to find the com/company/project/util directory. I was puzzled by this and after some time realized that the Python environment was expecting the MyDateTime class in the com/company/project/service/com/company/project/util directory. Compared to this, when the package path is mentioned in each Java class, the Java environment resolves the reference to the class.

After some exploration, I settled on packaging the common classes as a wfheel. Once a package is made into a wheel, it can be installed like any other Python package, using the pip command. Upon installation, the package is installed in the same place where external site packages are installed using pip.

The advantage of a wheel is that after installation, we can refer to the class as com.<company>.<project>.util from anywhere in the code and it will be resolved. The limitation is that each time the common code is updated, we have to repackage the wheel and distribute it to all users. Each user has to uninstall the existing wheel and then install the updated wheel.

As we were in the development phase, we had to frequently update the wheel and make it available to other members of the development team. To make the task of building and managing the wheel, I wrote a few Windows batch files. In the remaining part of the article, I am sharing the batch files. I also have a few helper batch files that make my task of file management a bit easier.

Note: Though the code snippet mentions PowerShell, the commands have to be executed on Windows Command Prompt by saving each one into a .bat file

Delete Backup Files - del-bak.bat

xxxxxxxxxx

@echo off

echo Deleting .bak files

del /S *.bak

Delete Dist Directory - del-dist.bat

Before creating the actual wheel, I thought it better to delete the 'dist' directory that is referred by Python during wheel creation. This ensures that files that are no longer part of the project do not continue to be included in the wheel.

xxxxxxxxxx

@echo off

call set-package-name.bat # rem sets the value for the PACKAGE_NAME variables

echo Cleaning old files and directories

del /Q /S dist

rd /Q /S dist

del /Q /S build

rd /Q /S build

del /Q /S %PACKAGE_NAME%.egg-info

rd /Q /S %PACKAGE_NAME%.egg-info

echo Done

Delete Log Files - del-log.bat

xxxxxxxxxx

@echo off

echo Deleting .log files

del /S *.log *.log.*

echo Done

Make Wheel - make-wheel.bat

Before building the wheel, delete the 'dist' directory, set the package name as environment variable, and then invoke a command to make the wheel.

xxxxxxxxxx

@echo off

call del-dist.bat

call set-package-name.bat

echo Creating new distribution

python setup.py bdist_wheel

echo Done

pause

Install the Wheel - wheel-install.bat

xxxxxxxxxx

@echo off

call set-package-name.bat

echo Installing %PACKAGE_NAME% package

pip install dist\%PACKAGE_NAME%.whl

echo Done

pause

Uninstall the wheel - wheel-uninstall.bat

xxxxxxxxxx

@echo off

call set-package-name.bat

echo Uninstalling %PACKAGE_NAME% package

pip uninstall dist\%PACKAGE_NAME%.whl

echo Done

pause

Run the Steps in Sequence - refresh.bat

I packaged the various steps related to wheels into a helper batch file. The file uninstalls the existing package and makes the new wheel. Once the wheel is created, it is installed.

xxxxxxxxxx

call package-uninstall.bat

call del-dist.bat

call make-wheel.bat

call package-install.bat

Set Package Name - set-package-name.bat

Set the name of the package for use by other batch files.

xxxxxxxxxx

@echo off

set PACKAGE_NAME=my_package

set INSTALL_NAME=%PACKAGE_NAME%-1.0.0

setup.py

While the batch files make the task of creating and installing the wheel easy, the key to making the wheel is the setup file.

xxxxxxxxxx

#setup(name="my_package", version="1.0", packages=find_packages())

with open("README.md", "r") as fh:

    long_description = fh.read()

setuptools.setup(

    name="my_package",

    version="1.0",

    author="myself",

    author_email="[email protected]",

    description="my package",

    long_description=long_description,

    long_description_content_type="text/markdown",

    url="https://myself.myself/my_package",

    packages=setuptools.find_packages(),

    classifiers=[

        "Programming Language :: Python :: 3",

        "License :: OSI Approved :: MIT",

        "Operating System :: OS Independent"

    ],

    python_requires=">=3.7"

)

Conclusion

By creating these helper files, each time I make changes to the code and test it, I simply run refresh.bat and all the work is done one by one. I only need to keep pressing the 'Enter' key a couple of times - while it is not required, I use it to keep track of the steps.