Python and Automation: A Perfect Combo

Python is one of the most popular languages among data scientists, researchers, and academics. It systematically tops the ladder on TIOBE, Stack Overflow, and GitHub, being the rockstar among other coding languages. It has a rich library of tools that can be applied to solve various problems in the field of automation. Indeed, Python compares favorably in its ability to automate virtually any process. 

Python also has a lot of built-in libraries. Since many services provide their data via APIs, you will have the ability to write scripts to solve tasks without deep programming knowledge.

But although it seems basic compared to such giants as Java, C, C++, C#, and others, it can still present challenges for learners. 

Automate With Python Like a Hero 

Automation is a very important topic in the software development world and everyday life. It is also an arena where Python has some distinct advantages over other programming languages thanks to a wide range of automation tools. Just think of the tasks you do on a daily basis like summing up numbers, sending emails, copying texts, etc. All these can be automated with Python tools in mere minutes (provided you know the secret words, of course). And I will prove this point further in the post.

So let's imagine you need to help someone automate the process — it happens a lot, I guess. But as is often the case, the person asking for help doesn’t have any programming background or they don’t have the right to put Python on a Windows-based work computer. Again, it’s often the case.

Let's also imagine the task as a shared file open to multiple users. They can all store data here except for executable files (e.g., .bat and .exe). And of course, everyone tends to break this rule.

Let’s type a path: search_path = "/Users/physboy/Article" 

We’ll use the os library.

The walk() function of the os module generates file names in the directory tree, traversing the tree from top to bottom or bottom to top. It also generates a 3-tuple (dirpath, dirnames, and filenames) for each directory in the tree with the root at the top of the directory, including the top itself.

• dirpath is a string, the path to the directory.
• dirnames is a list of subdirectory names in dirpath, excluding the special characters "." and "..".
• filenames is a list of filenames in the dirpath (not directories).
• os.path.join is a function that joins path components. It is also a nested module in the os module, and implements some useful functions for working with paths (hence, you can play around with it by adding extensions like joining one).

The main advantage of using the os.path module is that it allows the code to remain compatible with all operating systems since it uses a delimiter that matches the platform it runs on. 

It also lets you access the file system in a very easy way. In general, you can use the os module to get information about files and directories and to change the current directory.

Then a function that returns all the files in the directory will look like this:

Let's write an auxiliary function (meaning it does not provide functionality on its own) that converts datetime to str:

Here’s a function that defines the time a file was created (Windows) and the time when a file was last modified (Unix):

Here’s a function that defines the time of the last file change, in seconds:

And the function that defines the file extension:

According to network share protection rules, each team creates a subdirectory with their team name.

We will define it by using the function that defines the team name. It assumes that the subdirectory has a team folder.

os.path.relpath defines the path relative to the "start" directory.

os.path.getsize defines the size in KB.

The function that returns the file size in MB:

Let's put all the files in a DataFrame for easier workflow:

Let's calculate the information for each file for which we have written the functions:

Since we have some restricted files, we will let you remove them automatically to prevent manual removal.

Now let’s count them and remove them using the function os.remove:

Now, we need Python installed to get off the ground. Let’s run the python run.py command at the command line.

Now let’s say you’ve overcome your fear of programming and boom… There is another problem — you don’t have the right to install the software.

But each problem has its solution. In this case, our magic wand is the PyInstaller library. The PyInstaller bundles the Python application and all its dependencies into a single executable package. The user can run the application without installing the Python interpreter or any modules. PyInstaller supports Python 2.7 and Python 3.3+ and libraries like PyQt, Django, wxPython, and others.

So, here, we can convert a Python file into an .exe that will run on its own — "pyinstaller -F run.py"

Now all we have to do is to combine it all into one run.py file.

The Bottom Line

Python is a great language for automating and it's one I bet on for a whole range of tasks, including automation. To demonstrate the breadth of its powers, Python can be used to automate email sorting, replying to, sending HTTP requests, and among other things. Even if you're a newbie, this coding language comes complete with a rich library of packages and ready-to-use automation components. Overall, Python will help you automate the mundane stuff while saving you time and effort.