Unit Testing Named Queues: Spring 3+maven2+Google App Engine
Join the DZone community and get the full member experience.
Join For FreeProblem, you have a task that you know can take more than 30 seconds to
complete, what do you do? What if this task needs to be triggered every
day at a specific time? Google provides several mechanisms to to solve
just this problem, queues and scheduled task, respectively.
Queues
First, let's explore the "queue" system implemented in GAE. Note, at
this time, the "queue" is still experimental which means that the API
can change around some, so make sure you have strong unit testing, but
don't worry, we'll cover that a bit further down the page. To implement
queues, you need to let the web app know that your expecting queues to
be used, and to do this, you need to create a file named queue.xml in your WEB-INF folder (example: /src/main/webapp/WEB-INF/queue.xml).
The queue.xml file has a specific layout and that will not be covered
in depth in this post. Please read up on the queue.xml structure on Google's documentation. We will expand on the knowledge since the documentation is not complete.
Scheduled Task
Scheduled task are task that will trigger by GAE at repeatable times, emulating linux cron, or Quartz application. To implement a cron task, simply create a file called cron.xml and place it in your WEB-INF folder (example:/src/main/webapp/WEB-INF/cron.xml). This post will not go into detail on the implementation of scheduled task so please check Google's documentation
to get a firm understanding before reading on. In order to not pack too
much into one post, Scheduled Task will be covered in the next post.
Our Problem
The scenario we're going to explain is: You have an birthday tracking
software and you want a user's friends to be notified, via email, that
their friend is having a birthday. Assume that the code for the services named are tested and fully working.
To solve this we will use a queue and the scheduled task system provided by Google App Engine.
First, let's tackle the queue portion, then we will create the scheduled
task. Google's documentation on unit testing queues is very limited
and only shows how to retrieve the "default" queue. Since we could have
multiple task in our system that need to be run at different rates, we
will want to create a new Queue. Let's open (or create) the queue.xml file and insert the following:
<?xml version="1.0" encoding="UTF-8"?> <queue-entries> <queue> <name>birthdayemail</name> <rate>10/s</rate> <bucket-size>10</bucket-size> </queue> </queue-entries>
Next, let's create a simple service (Note: assume that the Autowired resources are working and imports are included at the top of the file, this has been reduced to emphasize the solution)
@Service("birthdayService") public class BirthdayServiceImpl implements BirthdayService { // this is the same name as in your queue.xml file private static final String EVENT_REMINDER_TASK_NAME = "birthdayemail"; @Autowired private BirthdayDAO birthdayDAO; public List handleGuestEmailsForBirthday(Date date) { if (date == null) { throw new IllegalArgumentException("Date can not be null"); } List<Guest> guestBirthdays = birthdayDAO.getByDate(date); for (Guest guest : guestBirthdays) { String TASK_URL = "/queue/birthday-emails/" + guest.getId() final TaskOptions taskOptions = TaskOptions.Builder.url(TASK_URL); // create a unique task name, note, must conform to [a-z] regex taskOptions.taskName(guest.getName() + "Task"); taskOptions.method(TaskOptions.Method.POST); Queue queue = QueueFactory.getQueue(EVENT_REMINDER_TASK_NAME); // commented out to show the default queue shown in the docs //Queue queue = QueueFactory.getDefaultQueue(); queue.add(taskOptions); } return events; } }
Let's look at what's here:
First, we have to create a URL that will be run when the task is
implemented. This URL will handle the unit of work specified, and should
run in under 30 seconds per GAE's restrictions. We will not cover what
this servlet does, just assume it uses the ID given in the URL path,
pulls up the Guest, then uses that to get the Guest's friends' emails;
then uses that to construct an email and pass to an EmailService. (HINT:
the EmailService could also implement a queue so each email is
separated out to run individually)
Next, notice that we are implementing a "Task Name". The this is
optional, but does help when debugging to figure out what task is
failing or what is running.
Now we're on to the TaskOptions. TaskOptions are a helper function used
to combine URL and model data to pass on to the servlet handling the
queue. Just a suggestion, but to follow the REST ideals, setting the
RequestMethod to POST or PUT is advisable, depending on what you are
trying to do. All Task should be idempotent, meaning that if the task
fails or is interrupted, the task can be run again and not harm the
data.
Lastly, we have the QueueFactory retrieving the Named Queue (the same
name in the queue.xml file). Simply add your taskOptions object to the
queue.add and move on.
Unit Testing Named Queues
Next, let's setup a unit test to test our named queue. The Google
documentation does not explicitly explain how to properly setup a test
case for one, datastore aware context and named queries. Below is an
example of a test case written to test our Named Queries. (Note,
as before, imports have been removed and the services used have been
used before. This is not intended to be the end-all-be-all for unit
testing GAE, please refer to previous post for more help, and yes, you
might be able to use JMock or Mockito instead of persisting data, but I
have not explored the option).
@RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = {"/testApplicationContext.xml"}) public class QueueBirthdayServiceTest extends AbstractJUnit4SpringContextTests { @Autowired private BirthdayService birthdayService; private final LocalServiceTestConfig[] configs = { new LocalDatastoreServiceTestConfig(), /* NOTE: THE QUEUE XML PATH RELATIVE TO WEB APP ROOT, More info below */ new LocalTaskQueueTestConfig() .setQueueXmlPath("src/test/resources/test-queue.xml")}; private final LocalServiceTestHelper helper = new LocalServiceTestHelper(configs); @Before public void setUp() { helper.setUp(); } @After public void tearDown() { helper.tearDown(); } @Test public void testHandleBirthdayQueueEmails() throws InterruptedException { // setup data Date birtdayDate = DateUtils.parse("10/16/1900"); // assume this builds the data for guest/guest's friends int guestWithBD = 5; // # of birthdays on date int gPerBirthday = 10; // # of emails being sent per birthday setupDataForBirthdays(birthdayDate, guestWithBD, gPerBirthday); List<Guest> birthdayOnDay = birthdayService. handleGuestEmailsForBirthday(birthdayDate); assertEquals(guestWithBD, birthdayOnDay.size()); // pause for a moment to allow queue to fill from previous statment Thread.sleep(1000); // verify # of birthdays with that day's expire date LocalTaskQueue ltq = LocalTaskQueueTestConfig.getLocalTaskQueue(); final Queue queue = QueueFactory .getQueue(BirthdayService.EVENT_REMINDER_TASK_NAME); //final Queue queue = QueueFactory.getDefaultQueue(); QueueStateInfo qsi = ltq.getQueueStateInfo() .get(queue.getQueueName()); assertNotNull(qsi); int expectedTaskCount = guestWithBD*gPerBirthday; assertEquals(expectedTaskCount, qsi.getTaskInfo().size()); assertEquals(birthdayOnDay.get(0).getID() + "Task", qsi.getTaskInfo().get(0).getTaskName()); } }
NOTE: If the syntax or unit testing is very foreign to you, please visit my previous post on Unit Testing on Google App Engine.
Let's explore more
The first half is not too exiting, just setting up data and calling the
service that creates the queue. We are creating 5 guest with birthdays
on the given date, and each of those 5 guest have 10 friends who we
intend to email. This service is already "written" out (above)
What you need to pay attention to is at the top of the file, during the
setup of the "helper", we have constructed it with a
LocalDatastoreServiceTestConfig and LocalTaskQueueTestConfig objects.
The second, the "LocalTaskQueueTestConfig" is the important one to add
when using a queue. If you are not using the DefaultQueue, you will
need to explicitly state where the queue.xml file is. I suggest that
you create a test-queue.xml file and place it in your /src/test/resources folder, so as to not mix production data and testing. NOTE: This file is loaded based relative to the ROOT application folder. The rest of the test should be pretty self explanatory.
In the next post, we will uncover how to wire the service up to a
Scheduled Task so you can automate the emails being sent. Stay tuned!
From http://www.ensor.cc/2010/11/unit-testing-named-queues-spring.html
Opinions expressed by DZone contributors are their own.
Comments