Over a million developers have joined DZone.

Intelligent Agents or Web Crawlers: Where is My Automation Level?

DZone's Guide to

Intelligent Agents or Web Crawlers: Where is My Automation Level?

As automation testing engineers, asking about automation levels can help identify the need for multi-agent systems and intelligent agents to improve automation.

· DevOps Zone ·
Free Resource

Is the concept of adopting a continuous everything model a daunting task for your fast moving business? Read this whitepaper to break down and understand one of the key pillars of this model in Continuous Governance: The Guardrails for Continuous Everything.

 As Automation testing engineers we must ask this question ourselves frequently. And it's not only new technologies, frameworks, or tools that we can master. It's more abstract - how we automate our workflow. A very good answer will be - 'I automate my repeatable tasks'. But it's still not quite enough. Why?

One can be our daily workflow activities. Let's take for example simple process as checking your email. If you do it manually, many times a day and analyse it - does this ring a bell?

If yes, than it's time to talk about Multi-agent systems and Intelligent agents. If we stick to it by definition, we can refer to more specific ones - User_agent. So what they can do for us, we can find in their classification:

"User agents, or personal agents, are intelligent agents that take action on your behalf. In this category belong those intelligent agents that already perform, or will shortly perform, the following tasks:

  • Check your e-mail, sort it according to the user's order of preference, and alert you when important emails arrive.
  • Assemble customized news reports for you. There are several versions of these, including CNN.
  • Find information for you on the subject of your choice.
  • Fill out forms on the Web automatically for you, storing your information for future reference
  • Scan Web pages looking for and highlighting text that constitutes the "important" part of the information there
  • Facilitate with online job search duties by scanning known job boards and sending the resume to opportunities who meet the desired criteria

So if you already found your 'guy', lets discuss how to 'build' it and put 'him' to work. For the reason of good programming I've divide my agents according to their tasks and capabilities.

First let's start with the Mail_User_Agent. If Your company already uses Outlook, You can setup MUA to use the Web Access API and automate this task via any script or tool You know. In my examples good old Powershell (as we know already - task automation and configuration management framework from Microsoft) with it's headless IE capabilities (will make it work as a Daemon and allow it to communicate with us only if preconditions are true) is able to support all this. 

And to be more precise if You have to Login and use Firewall  Web Access APIs - the implementations is the same. So with one stone we get two birds - every morning You'll be behind-the-wall and knowing your mail content. Here You can see sample solution.

As next we can continue with Assembled and customized news reports & Find information for you on the subject of your choice (Hint- You can combine all scrips into single start-up.ps1 and they'll read all news for You). This one will search for all jobs in such Bulgarian Web Portal according to Your predefined criteria and will give you a Html report. Using this one You will be able to get Google & Youtube search-resources results again with predefined criteria and Html report. And a little extra one who can convert any Text-To-Speech file. In case You are asking - why? I'm using it to listen my lectures when I'm in the Subway. 

And my personal favorite -  online job search duties by scanning known job boards and sending the resume to opportunities who meet the desired criteria. Again the Jobs Web Portal is Bulgarian, but to adopt the logic and reuse it will be trivial task. And as a Syntactic sugar there is a "Apply via Linkedin" suport.  But be careful with this  little baby -because it can generate enormous amount of apply records on Your account and You don't want to get a call when someone is asking why have send 20 times your CV and Cover letter for this position. The algorithm is smart (but greedy) and 'apply' only to jobs given by your criteria and since Your last Login, but still it's better to monitor it from time to time. 

So can use them all - Web_Crawlers or User Agents, but in the end they do what they are designed to do - automate our daily lives.

Are you looking for greater insight into your software development value stream? Check out this whitepaper: DevOps Performance: The Importance of Measuring Throughput and Stability to see how CloudBees DevOptics can give you the visibility to improve your continuous delivery process.

devops ,automation ,testing

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}