Dynamically generate version and apply python package from DevOps

 

Dynamically versioning for packages from DevOps

This is a side-note / microblog to our blog series on Applying DevOps for data engineering topics.

 

In our use-case, considering Python as the programming language used in Databricks.

 

Our source code is written as python class files and committed to repos in Azure DevOps.

 

To build, package and deploy these classes as a library to Databricks cluster, DevOps pipelines is used to automate this task.

 

Every time we make some changes, the build package version needs incrementing.

 

To make this versioning process automatic, versioneer  is one such versioning solution among other options. 

 

Jacob has done wonderful job clearly explaining step-by-step instructions on how to use versioneer in this blog.

 

https://jacobtomlinson.dev/posts/2020/versioning-and-formatting-your-python-code/

 

With versioneer configured, custom packages built from DevOps pipeline gets a dynamic version based on based on "git tag <version>" tag string.

Screenshot showing artifacts from DevOps pipeline 

.whl package built and versioned using Versioneer