While loading chunks of data from one system to another system, it is always difficult to process each individual record separately. It is always recommended to process the data in batches; most systems also support this. Similarly, Salesforce provides SalesforceBulkAPI. To send bulk data to SalesforcBulkAPI as an input, the Batch Commit scope in a Batch Step can be used. Here is a brief introduction to the SalesforceBulkAPI and Batch Commit.
Salesforce provides SalesforceBulkAPI to handle very large sets of data. It is a REST-based Bulk API used to process large amounts of data. It is optimized for inserting, updating, upserting, and deleting large numbers of records asynchronously by submitting them in batches to Force.com for being processed in the background.
Mule also has the ability to process messages in Batches using Batch Processing. The BatchCommit scope in a Batch Step is used to process subsets of records within a batch for bulk operations to an external source or service.
Fetch the Employee records from HR Database and create Contacts for Employees in Salesforce.com. Follow these steps:
1. Create an Application in Mulesoft.
2. Drag and drop the HTTP connector.
3. Create an HTTP connector configuration by clicking on the + icon.
4. Leave all the defaults and click OK. Under BasicSettings, set the Path as /getEmp and AllowedMethods as GET.
5. Drag and drop the Database Connector.
6. Configure OracleConfiguration, give the database details, and add the dependency JAR file.
7. Write the select query to fetch employee details from the database.
8. Drag and drop Batch Scope to the flow and Batch Step into the Batch Scope.
9. Drag and drop BatchCommit into BatchStep, set the Commit Size property of BatchCommit to 100.
This allows Mule to send sets of records of size 100 to the target system.
10. Drag and drop the Salesforce component into the BatchCommit scope.
11. Configure SalesforceConnector configurations and test the connection.
12. Select CreateBulk as Operation and Object Type as Contact.
13. BatchCommit writes a transformation message to convert the EmployeeRecord fetched from Database to Salesforce in an acceptable format for the creation of contacts in Salesforce.
14. This completes Batch Configuration! Drag and drop Batch Execute after the Database component and refer to the Batch Module created above.
The whole application should look like this:
Run the application and invoke the service from Postman or any browser with the configured URI.
After the successful run of the application, log in to SalesForce. Go to Setup and click on Bulk Data Load Jobs. Verify the jobs created.
Click on the JobId to see the details of a job.
You can view input requests and results by clicking on ViewRequest and ViewResult.