DZone
AI Zone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
  • Refcardz
  • Trend Reports
  • Webinars
  • Zones
  • |
    • Agile
    • AI
    • Big Data
    • Cloud
    • Database
    • DevOps
    • Integration
    • IoT
    • Java
    • Microservices
    • Open Source
    • Performance
    • Security
    • Web Dev
DZone > AI Zone > Simplifying AI With the 3 Ms Approach

Simplifying AI With the 3 Ms Approach

I've been dealing with data for several years now, and the 3 Ms approach to AI that I recently tried has proven to be reliable, easy to maintain, robust, and optimized. What do you think they are?

Deepak Kumar user avatar by
Deepak Kumar
·
Oct. 31, 17 · AI Zone · Opinion
Like (3)
Save
Tweet
2.19K Views

Join the DZone community and get the full member experience.

Join For Free

With the growing need for and popularity of AI, crawling data from the web and summarizing this data based business requirements is one of the most common problems many people have to deal with. The problem becomes more challenging when the data sources can be any website or even a large set of websites, as providing a generic solution that meets all business requirements is very difficult.

I have been dealing with such problems in the last couple of years and recently, I tried an approach that I found to be very reliable, easy to maintain, robust, and optimized.

The approach is the 3 M approach, where the 3 Ms mean following:

  1. Microservice

  2. Message queue

  3. Multithreading

Microservice

I used the microservices approach for separation of concern. I segregated the AI algorithms and business logic from engineering work to make it easier for the team to maintain the application. There are two main reasons behind this segregation:

  1. Usually, data scientists/AI-ML researchers and Product engineers have different specializations. Former  are expert in AI/ML while later later are expert in infrastructure and engineering works.

  2. For most data scientists and ML researchers, the favorable programming language are Python and R because they contains a huge list of libraries for AI and is pretty easy compared to other programming languages. On the other hand, for enterprise server-side applications, Java is the most preferable, as it has very robust and powerful frameworks like Spring, Hibernate, etc., which have plenty of features to make developers' jobs easy and simple.

Message Queue

Data crawling from the web is a tedious and unreliable process, as every website is different from any other website. The time consumed in crawling data from one website to another also varies significantly and depends on many factors. To make this process reliable, I used RabbitMQ to queue the requests and process asynchronously. This approach helped me in processing the request in a controlled way and it helped the user not to wait for a long time while the request was being processed.

Multithreading

Multithreading is used in the application for parallel processing of the request queued in the message queue. Use a thread pool with configurable size. The MessageQ listener continuously observes the MessageQ and calls the thread manager to start a thread whenever it is available in the pool.

The entire application is divided into the three components mentioned below, along with the  technologies used in them:

  1. Advisor-app: Java, Spring Boot, Spring Data, MySQL,and  MongoDB

  2. Advisor-Msg: Spring Boot, and RabbitMQ

  3. Advisor-AI: Python, Django, NLP, NER, and various AI algorithms

AI Data science

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • How Does the Database Understand and Execute Your Query?
  • Upload Files to AWS S3 in JMeter Using Groovy
  • How to Handle Early Startup Technical Debt (Or Just Avoid it Entirely)
  • A Guide to Understanding Vue Lifecycle Hooks

Comments

AI Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • MVB Program
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends:

DZone.com is powered by 

AnswerHub logo