DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Building Micro-Frontends With Vue and Reusable Components
  • How To Get Closer to Consistency in Microservice Architecture
  • Modeling Saga as a State Machine
  • Practical Transaction Handling in Microservice Architecture

Trending

  • Medallion Architecture: Why You Need It and How To Implement It With ClickHouse
  • Top Book Picks for Site Reliability Engineers
  • Event-Driven Architectures: Designing Scalable and Resilient Cloud Solutions
  • A Developer's Guide to Mastering Agentic AI: From Theory to Practice
  1. DZone
  2. Data Engineering
  3. Data
  4. What Does Synchronization With Asyncio Look Like

What Does Synchronization With Asyncio Look Like

Here is how you build up an architectural style, by creating the asynchronous development process on Python with the use of Asyncio.

By 
Tetiana Stoyko user avatar
Tetiana Stoyko
·
Updated Nov. 02, 22 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
5.9K Views

Join the DZone community and get the full member experience.

Join For Free

The most general view of program development approaches claims we have two basic coding options. The first one, synchronous code architecture, works step-by-step, and all the processes run the same way. For instance, each process performs data input and receives output from the server. Consequently, if we have operation one and process two, as well as input/output one and input/output two, input two starts only after output one. 

In asynchronous code, the architecture is not that strict and hierarchic. Our example with two operations looks different, as both operations can be performed simultaneously and remain independent. It is also called concurrent code. 

The main advantage of asynchronous coding is increased application performance. Response rates are higher due to independence in processes. The main drawback is that every request is on its own, and with too many requests, the CPU core won't be able to deal with them.

Python Asynchronous Programming Explained

A few working approaches to discuss in Python asynchronous programming are Multiprocessing, Multitasking, and Concurrency.

Briefly About Multiprocessing and Multitasking 

Multiprocessing is a common processing type when each task is performed independently yet simultaneously. Multiprocessing requires an additional CPU to handle multiple processes. 

Multitasking allows running more than one process simultaneously on the same processor. Multiprocessing supports switching between processes or context switching. 

What Is Concurrency?

Two previous processing types are quite easy to understand. Let's answer the question of what concurrency is. Concurrent code supports the processing of multiple tasks at the same time. They are executed in sequence, yet this sequence is flexible. So, in a way, processes communicate or interrupt each other. These sequences are called threads. Therefore, a single-threaded program executes one sequence/thread. 

On the other hand, the multithreading program executes more than one thread simultaneously. Python asynchronous function such as multithreading means threads regularly interrupt each other and are executed. For a resource-intensive task, the thread might be lowlily prioritized and frozen for some time. Speaking, Python multithreading allows changing the order of performing various tasks or requests. 

Asyncio Applications

Asyncio is a library to write concurrent code using the async/await syntax. In the applications that can benefit from exploiting concurrency, Asyncio is added to simplify the syntax necessary to consume promise-based APIs. It is a foundation for many Python asynchronous frameworks to provide high-performance network and web servers, database connection libraries, distributed task queues, etc. 

Asyncio requires a Python event loop feature to routinely measure the progress of tasks. In case the system sees a task in progress, it can automatically start the next one, avoiding unnecessary waiting. The Python event loop makes it possible to automate the queue, starting the tasks in a different order and not one by one. It is beneficial in projects based on microservice architecture and the ones that deal with tons of data: databases, files, and large amounts of information.  

 When discussing benefits in a microservice architecture, Asyncio allows better communication. Communicating between microservices is usually over REST or gRPC protocol. To talk to many microservices at once, we can run requests concurrently. Another plus for microservice architecture is error handling of the Asyncio APIs, such as wait and gather. They allow handling exceptions from groups of tasks and coroutines.

Asynchronous Python Code Samples

Python asynchronous programming works great with databases and with the use of coroutines. Coroutines are used for cooperative multitasking, where a process voluntarily gives away control periodically to enable multiple applications to be run simultaneously.

Let's look at the example. We included need all the necessary components to show a small case study. We used ArrangoDB —  a free graph database; RabbitMQ —  a free message broker; Asyncio library; and coroutines. First things first, let's connect the database and declare the coroutine:

 
from aioarango import ArangoClient
async def arango_context(db_name, host, username, password):
client = ArangoClient(hosts=host)
       return await client.db(name=db_name, username=username, password=password)

The "arango_context" function here is a coroutine. Next, we need an "await" keyword when we call it:
Class Integration:

 
async def db(self):
return await arango_context(
db_name=AS.DATABASE, host=AS.HOST,
username=AS.USER, password=AS.PASSWORD)

Now let’s create a sample data class that retries running a callback function. For instance, we want to test if the data was transformed and saved to the database. Saving data can take time, so we will try to get data after sleeping for three seconds. Only after five failed attempts will an exception arise:

 
class Retry:

async def run(self, callback_func, *args):
result = None
retry = 0
while True:
if not result and retry >= 5:
raise Exception('Retry exhausted')

try:
result = await callback_func(*args)
if result:
return result

except Exception as exc:
key = f'for {args[0].key}' if args else ''
logger.info('Got an error %s, will retry. Exception: %s', key, exc)

retry += 1
           await asyncio.sleep(3)


Setting up a Python event loop comes next, as it’s the most important aspect of asynchronous programming. The following Python code sample shows how to consume multiple queues concurrently using Python and Asyncio. The first step is to create a Python event loop. The connection to the RabbitMQ will be closed when the service is stopped. Loop will run nonstop, waiting for a RabbitMQ message to the specific processor queue: async def main(loop):

 
client = Client(settings=rabbit_settings)
await client.build_client()
processors = [
CompanyUpdatedProcessor,
CompanyInvalidatedProcessor,
CompanySyncProcessor,
]
await client.subscribe(processors)
return client.connection


if __name__ == '__main__':
logger.info('Service sync started.')
loop = asyncio.get_event_loop()
connection = loop.run_until_complete(main(loop=loop))
try:
loop.run_forever()
except KeyboardInterrupt:
logger.info('Service sync stopped.')
except Exception as exc:
logger.exception(exc)
finally:
       loop.run_until_complete(connection.close())

Asyncio library has a set of methods you can use in different cases. The following Python code samples show a method that allows one to wait until the created task completes:
 
 async def consume(self, context, channel, message):
task = asyncio.create_task(self.send_message(context, channel, message))
       done, pending = await asyncio.wait({task}, timeout=None, return_when=ALL_COMPLETED)

The "create_task" method shown above submits the coroutine to run "in the background," i.e., with the current task and all other actions, changing them at awaiting points. It returns an awaitable object that might be used to cancel the execution of the coroutine. It's one of the main Asyncio principles, similar to starting a thread. When you call "create_task," you submit a coroutine for implementation and receive back an object. Now when you add this feature, you can wait for the result or let it be. We’ve gone through step-by-step synchronization of the Asyncio library with a graph database and coroutines, which will benefit high-level structured network code.

Short Sum Up

Python asynchronous programming becomes useful when the project involves microservice architecture, is heavy on processes and needs Multiprocessing, Multitasking, or Concurrency code. In the case of the last one, Asyncio applications are of great use, along with databases, message brokers, and coroutines. Async/await syntax in Python allows multiple promises to be initiated and resolved for values when required.

Let’s summarize what we discussed: 

  1. Difference between synchronous and asynchronous programming works
  2. Main notions about the asynchronous mechanism. 
  3. General overview of Asyncio applications. 
  4. Python code samples that show what synchronization with Asyncio looks like. 

You are ready to write web apps that can handle multiple processes.

Architecture Database Database connection Event loop Message broker Web apps Data (computing) microservice Python (language) Task (computing)

Opinions expressed by DZone contributors are their own.

Related

  • Building Micro-Frontends With Vue and Reusable Components
  • How To Get Closer to Consistency in Microservice Architecture
  • Modeling Saga as a State Machine
  • Practical Transaction Handling in Microservice Architecture

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!