Complex Cloud API Integrations Made Easy
This article shows how, in just a few Python code lines, Zato makes their usage easy, convenient, and extremely effective.
Join the DZone community and get the full member experience.Join For Free
Cloud-based connections are a staple of modern API integrations — this article shows how, in just a few Python code lines, Zato makes their usage easy, convenient, and extremely effective.
As per the diagram, in this article, we will integrate REST and FTP based resources with Dropbox, but it needs to be emphasized that the same code would work with other protocols.
REST and FTP are just the most popular ways to upload data that can be delivered to Dropbox, but if the source files were made available via AMQP, SAP, or other applications — everything would the same.
Similarly, in place of Dropbox, we could use services based on AWS, Azure, OpenStack, or other cloud providers — the same logic, approach, and patterns would continue to work.
Also, speaking of Dropbox, for simplicity, we will focus on file uploads here, but the full Drobpox API is available to Zato services, so anything that Dropbox offers is available in Zato too.
We will use two layers in the solution:
- Channel services — receive data from external resources (here, REST and FTP) and deliver it to the Dropbox connector.
- Dropbox connector — receives data from channels and uploads it to Dropbox.
The separation of concerns lets us easily add new channels as needs arise without modifying the layer that connects to Dropbox.
In this way, the solution can be extended at any time — if one day we need to add SFTP, no changes to an already existing part will be required.
First, we will create a Dropbox connector into which we can plug channel services.
Dropbox Connector — Web-admin
Let's create a new Dropbox connection definition in Zato web-admin first. Click Cloud -> Dropbox and fill out the form below, remembering to click "Change token" afterward.
Note that the "User-agent field" is required — this is part of metadata that Dropbox will accept. You can use it, for instance, to indicate whether you are connecting to Dropbox from a test vs. production environment.
Dropbox Connector — Python
And here is the Python code that acts as the Dropbox connector. Note a few interesting points:
It does not care where its input comes from. It just receives data. This is crucial because it means we can add any channel, and the actual connector will continue to work without any interruptions.
The connector focuses on business functionality only — it is only web-admin that specifies what the connection details are, e.g., the connector itself sends data and does not even deal with details as low-level as security tokens.
The underlying client object is an instance of dropbox. Dropbox from the official Python SDK
After hot-deploying the file, the service will be available as API.connector.dropbox.
Now, let's add a service to accept REST-based file transfers — it will be a thin layer that will extract data from the HTTP payload to hand it over to the already existing connector service. The service can be added in the same file as the connector, or it can be a separate file; it is up to you; it will work in the same. Below, it is in its own file.
Having uploaded the REST channel service, we need to create an actual REST channel for it. In web-admin, go to Connections -> Channels -> REST and fill out the form.
At this point, we can already test it all out. Let's use curl to POST data to Zato. Afterward, we confirm in Dropbox that a new file was created as expected.
Note that we use POST to send the input file, so we need the file_name query parameter.
$ curl -XPOST --data-binary \
FTP Connection Definition
Having made sure that the connector delivers its files through REST, let's focus on FTP, first creating a new FTP connection definition in web-admin.
We need some Python code now — it will connect to the FTP server, list all files in a specific directory, and send them all to the Dropbox connector.
The connector will not even notice that the files do not come from REST this time; it will simply accept them on input.
We have an FTP connection, and we have a service — after uploading some files to the FTP server, we can test the new service now using web-admin. Navigate to Services -> List services -> Choose "API.channel.scheduler.upload" -> Invoker and click Submit.
This will invoke the service directly, without a need for any channel. Next, we can list recent additions in Dropbox to confirm that the file we were uploaded, which means that the FTP's service-connected files, the files were downloaded, and the Dropbox connector delivered them successfully.
Invoking the service from web-admin is good, but we would like to automate the process of transferring data from FTP to Dropbox — for that, we will create a new job in the scheduler. In web-admin, go to Scheduler and create a new interval-based job.
However, note that the job's start date will be sent to the scheduler using your user's preferred timezone. By default, it is set to UTC, so ensure that you set it to another if your current timezone is not UTC — go to Settings and pick the correct timezone.
Now, on to the creation of a new job. Note that if this were a real integration project, the interval would probably be set to a more realistic one, e.g., if you batch transfer PDF invoices, then doing it once a minute or twice an hour would probably suffice.
Going Even Further
We have just concluded the process — in a few steps, we connected REST, FTP, and Dropbox. Moreover, it was done in an extensible way. Should a business need arise, nothing is preventing us from adding more data sources.
Not only that, if one day we need to add more data sinks, e.g., S3 or SQL, we could do it in the same easy way. Or we could publish messages to topics and guaranteed delivery queues for Zato to manage the whole delivery life-cycle; there are no limits whatsoever.
This integration example was but a tiny part of what Zato is capable of. To learn more about the platform — do visit the documentation and read the friendly tutorial that will get you started in no time.
Opinions expressed by DZone contributors are their own.