Python SDKs Package Management in GCP Artifact Registry
This article provides tested and implemented best practices for managing Python SDKs in the GCP Artifact Registry, which allows for code reuse and security.
Join the DZone community and get the full member experience.Join For Free
Using a centralized, private repository to host SDK as a package not only enables code reuse but also simplifies and secures the existing software delivery pipeline. By using the same formats and tools as you would in the open-source ecosystem, you can leverage the same advantages, simplify building, and keep business logic and applications secure.
Storing SDK packages in Google Cloud Artifact Registry not only enables SDK code reuse but also simplifies and secures your existing build pipeline. In addition to bringing your internal packages to a managed repository, using Artifact Registry also allows you to take additional steps to improve the security of your software delivery pipeline.
In this tutorial, we will learn how to:
- Set up an Artifact Registry for Python SDK repository
- Upload an SDK package
- Install the SDK package
1. Set Up an Artifact Registry for Python SDK Repository
As a part of setting up an Artifact Registry for Python SDK repository, let's first set up a virtual environment called "pythnsdk" and then install Twine and the Artifact Registry keyring backend in the virtual environment.
python3 -m venv pythnsdk source pythnsdk/bin/activate python -m pip install -U pip pip install twine keyrings.google-artifactregistry-auth keyring --list-backends
Log in to Google Cloud CLI with service account credentials.
gcloud auth activate-service-account [SERVICE_ACCOUNT] --key-file=/path/key.json --project=PROJECT_ID
Now, create a new Python package repository in the current project named "python-sdk-repo" in the location us-west1.
gcloud artifacts repositories create python-sdk-repo \ --repository-format=python \ --location=us-west1 \ --description="Python package repository"
To set up authentication with the Artifact Registry keyring backend, run the following command to print the repository configuration to add to your Python project.
gcloud artifacts print-settings python
Update the .pypirc file that specifies repositories for publishing Python packages. Verify
$VIRTUAL_ENV/pip.conf files and check whether repository settings are matching to the intended Artifact Registry repository and other GCP project details that were set up earlier.
cat $HOME/.pypirc [distutils] index-servers = python-sdk-repo [python-sdk-repo] repository: https://us-west11-python.pkg.dev/PROJECT_ID/python-sdk-repo/
cat $VIRTUAL_ENV/pip.conf [global] extra-index-url = https://us-west1-python.pkg.dev/PROJECT_ID/python-sdk-repo/simple/
mkdir python-sdk cd python-sdk
Now download the whole Python SDK folder structure from the project Git repository (this folder structure contains a Python SDK that is already packaged in dist/ folder ) to the python-sdk folder
2. Upload an SDK package
Upload the packages to the Artifact Registry Python SDK repository from your dist directory.
twine upload --repository-url https://us-west1-python.pkg.dev/PROJECT_ID/python-sdk-repo/ dist/*
Now view the package in the repository.
gcloud artifacts packages list --repository=python-sdk-repo
To view versions of a package, run the following command:
gcloud artifacts versions list --package=PACKAGE
PACKAGE is the package ID.
3. Install the SDK package.
Run the following command to install the Python SDK package uploaded to Artifact Registry in local.
pip install --index-url https://us-west1-python.pkg.dev/PROJECT_ID/python-sdk-repo/simple/ projectsdk
Note: Use Cloud Identity and Access Management (IAM) for repository access control.
The above procedure has been successfully implemented and tested on the GCP client environment, verified in the below screenshot on what messages are seen after successful installation of a SDK package as reference.
Opinions expressed by DZone contributors are their own.