Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Announcing Celery-S3

DZone's Guide to

Announcing Celery-S3

· Cloud Zone
Free Resource

Learn how our document data model can map directly to how you program your app, and native database features like secondary indexes, geospatial and text search give you full access to your data. Brought to you in partnership with MongoDB.

TL;DR: Check out celery-s3, it lets you store Celery task results in S3.

Celery has good support for a variety of different message brokers – RabbitMQ, Redis, SQS, etc. – but support for result storage is somewhat more limited.

This is particularly apparent if you’re running Celery on an EC2 server and wish to take advantage of the distributed services AWS provides. SQS works as a message broker, but there’s nowhere to store results (and using the AMQP result store with SQS results in a queue for each result).

Celery-S3 lets you store Celery results in an S3 bucket, which means you can run a fully-functioning Celery installation on AWS with nothing but a Python install.

Discover when your data grows or your application performance demands increase, MongoDB Atlas allows you to scale out your deployment with an automated sharding process that ensures zero application downtime. Brought to you in partnership with MongoDB.

Topics:

Published at DZone with permission of Rob Golding, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}