AI/ML

Neural network embeddings are a low-dimensional representation of input data that gives rise to various applications. Embeddings have some interesting capabilities, as they can capture the semantics of the data points. This is especially useful for unstructured data like images and videos so that you can encode not only pixel similarities but also some more complex relationships. Embeddings from the BDD100K dataset were visualized using FiftyOne and Plotly. Performing searches over these embeddings gives rise to many use cases like classification, building up recommendation systems, or even anomaly detection. One of the primary benefits of performing a nearest-neighbor search on embeddings to accomplish these tasks is that there is no need to create a custom network for every new problem; you can often use pre-trained models. Using the embeddings generated by some publicly available models is possiblewithout any further finetuning. While there are a lot of powerful use cases that involve embeddings, there are several challenges in workflows performing searches over embeddings. Specifically, performing a nearest neighbor search on a large dataset and then effectively acting on the results of the search, for example, performing workflows like auto-labeling data, are both technical and tooling challenges. To that end, Qdrant and FiftyOne can help make these workflows effortless. Qdrant is an open-source vector database designed to perform an approximate nearest neighbor search (ANN) on dense neural embeddings, which is necessary for any production-ready system that is expected to scale to large amounts of data. FiftyOne is an open-source dataset curation and model evaluation tool that allows you to manage and visualize your dataset effectively, generate embeddings and improve your model results. In this article, we will load the MNIST dataset into FiftyOne and perform the classification based on ANN. The data points will be classified by selecting the most common ground truth label among the K nearest points from our training dataset. In other words, for each test example, we’re going to select its K nearest neighbors, using a chosen distance function, and then select the best label by voting. All that search in the vector space will be done with Qdrant to speed things up. We will then evaluate the results of this classification in FiftyOne. Installation If you want to start using the semantic search with Qdrant, you need to run an instance of it, as this tool works in a client-server manner. The easiest way to do this is to use an official Docker image and start Qdrant with just a single command: docker run -p “6333:6333” -p “6334:6334” -d qdrant/qdrant After running the command, we’ll have the Qdrant server running, with HTTP API exposed at port 6333 and the gRPC interface at 6334. We will also need to install a few Python packages. We will use FiftyOne to visualize the data, their ground truth labels, and those predicted by our embedding similarity model. The embeddings will be created by MobileNet v2, available in torchvision. Of course, we need to communicate to the Qdrant server somehow as well, and since we’re going to use Python, qdrant_client is a preferred way of doing that. pip install fiftyone pip install torchvision pip install qdrant_client Processing Pipeline Loading the dataset Generating embeddings Loading embeddings into Qdrant Nearest neighbor classification Evaluation in FiftyOne Loading the Dataset There are several steps we need to take to get things running smoothly. First, we need to load the MNIST dataset and extract the train examples from it, as we will use them in our search operations. To make everything even faster, we’re not going to use all the examples but just 2500 samples. We can use the FiftyOne Dataset Zoo to load the subset of MNIST we want in just one line of code. import fiftyone as fo import fiftyone.zoo as foz # Load the data dataset = foz.load_zoo_dataset("mnist", max_samples=2500) # Get all training samples train_view = dataset.match_tags(tags=["train"]) Let’s start by looking at the dataset in the FiftyOne App. # Visualize the dataset in FiftyOne session = fo.launch_app(train_view) Generating Embeddings The next step is to generate embeddings on the samples in the dataset. This can always be done outside of FiftyOne, with your custom models. However, FiftyOne also provides various models in the FiftyOne Model Zoo that can be used right out of the box to generate embeddings. In this example, we use MobileNetv2 trained on ImageNet to compute an embedding for each image. # Compute embeddings model = foz.load_zoo_model("mobilenet-v2-imagenet-torch") train_embeddings = train_view.compute_embeddings(model) Loading Embeddings Into Qdrant Qdrant allows storing not only vectors but also some corresponding attributes — each data point has a related vector and, optionally, a JSON payload attached to it. We want to use this to pass in the ground truth label to make sure we can make our prediction later on. ground_truth_labels = train_view.values("ground_truth.label") train_payload = [ {"ground_truth": gt} for gt in ground_truth_labels ] Having the embedding created, we can start communicating with the Qdrant server. An instance of QdrantClient is helpful, as it encloses all the required methods. Let’s connect and create a collection of points called “mnist.” The vector size depends on the model output, so if we want to experiment with a different model another day, we will need to import a different one, but the rest will be kept the same. Eventually, after making sure the collection exists, we can send all the vectors and their payloads containing their true labels. import qdrant_client as qc from qdrant_client.http.models import Distance # Load the train embeddings into Qdrant def create_and_upload_collection( embeddings, payload, collection_name="mnist" ): client = qc.QdrantClient(host="localhost") client.recreate_collection( collection_name=collection_name, vector_size=embeddings.shape[1], distance=Distance.COSINE, ) client.upload_collection( collection_name=collection_name, vectors=embeddings, payload=payload, ) return client client = create_and_upload_collection(train_embeddings, train_payload) Nearest Neighbor Classification Now to perform inference on the dataset. We can create the embeddings for our test dataset but ignore the ground truth and try to find it using ANN, then compare if both matches. Let’s take one step at a time and start with creating the embeddings. # Assign the labels to test embeddings by selecting # the most common label among the neighbours of each sample test_view = dataset.match_tags(tags=["test"]) test_embeddings = test_view.compute_embeddings(model) Time for some magic. Let’s iterate through the test dataset’s samples and corresponding embeddings and use the search operation to find the 15 closest embeddings from the training set. We’ll also need to select the payloads, as they contain the ground truth labels required to find the most common label in the neighborhood of a particular point. Python’s Counter class will be helpful to avoid any boilerplate code. The most common label will be stored as an “ann_prediction” on each test sample in FiftyOne. This is encompassed in the function below, which takes an embedding vector as input, uses the Qdrant search capability to find the nearest neighbors to the test embedding, generates a class prediction, and returns a FiftyOne Classification object that we can store in our FiftyOne dataset. import collections from tqdm import tqdm def generate_fiftyone_classification( embedding, collection_name="mnist" ): search_results = client.search( collection_name=collection_name, query_vector=embedding, with_payload=True, top=15, ) # Count the occurrences of each class and select the most common label # with the confidence estimated as the number of occurrences of # the most common label divided by a total number of results. counter = collections.Counter( [point.payload["ground_truth"] for point in search_results] ) predicted_class, occurences_num = counter.most_common(1)[0] confidence = occurences_num / sum(counter.values()) prediction = fo.Classification( label=predicted_class, confidence=confidence ) return prediction predictions = [] # Call Qdrant to find the closest data points for embedding in tqdm(test_embeddings): prediction = generate_fiftyone_classification(embedding) predictions.append(prediction) test_view.set_values("ann_prediction", predictions) We estimated the confidence by calculating the fraction of samples belonging to the most common label. That gives us an intuition of how sure we were while predicting the label for each case and can be used in FiftyOne to spot confusing examples easily. Evaluation in FiftyOne It’s high time for some results! Let’s start by visualizing how this classifier has performed. We can easily launch the FiftyOne App to view the ground truth, predictions, and images. session = fo.launch_app(test_view) FiftyOne provides a variety of built-in methods for evaluating your model predictions, including regressions, classifications, detections, polygons, instances, and semantic segmentation, on both image and video datasets. In two lines of code, we can compute and print an evaluation report of our classifier. # Evaluate the ANN predictions, with respect to the values in ground_truth results = test_view.evaluate_classifications( "ann_prediction", gt_field="ground_truth", eval_key="eval_simple" ) # Display the classification metrics results.print_report() precision recall f1-score support 0 - zero 0.87 0.98 0.92 219 1 - one 0.94 0.98 0.96 287 2 - two 0.87 0.72 0.79 276 3 - three 0.81 0.87 0.84 254 4 - four 0.84 0.92 0.88 275 5 - five 0.76 0.77 0.77 221 6 - six 0.94 0.91 0.93 225 7 - seven 0.83 0.81 0.82 257 8 - eight 0.95 0.91 0.93 242 9 - nine 0.94 0.87 0.90 244 accuracy 0.87 2500 macro avg 0.88 0.87 0.87 2500 weighted avg 0.88 0.87 0.87 2500 After evaluating FiftyOne, we can use the results object to generate an interactive confusion matrix allowing us to click on cells and automatically update the App to show the corresponding samples. plot = results.plot_confusion_matrix() plot.show() Let’s dig in a bit further. We can use the sophisticated query language of FiftyOne to easily find all predictions that did not match the ground truth yet were predicted with high confidence. These will generally be the most confusing samples for the dataset and the ones from which we can gather the most insight. from fiftyone import ViewField as F # Display FiftyOne app, but include only the wrong predictions that # were predicted with high confidence false_view = ( test_view .match(F("eval_simple") == False) .filter_labels("ann_prediction", F("confidence") > 0.7) ) session.view = false_view These are the most confusing samples for the model, and, as you can see, they are fairly irregular compared to other images in the dataset. A next step we could take to improve the model's performance could be to use FiftyOne to additional curate samples similar to these. Those samples can then be annotated through the integrations between FiftyOne and tools like CVAT and Labelbox. Additionally, we could use some more vectors for training or perform a fine-tuning of the model with similarity learning, for example, using the triplet loss. But now, this example of using FiftyOne and Quadrant for vector similarity classification is already working well. And that’s it! As simple as that, we created an ANN classification model using FiftyOne with Qdrant as an embedding backend, so finding the similarity between vectors can stop being a bottleneck as it would in the case of a traditional k-NN. Try It Yourself The notebook contains the source code of what you saw in this. Additionally, it includes a realistic use case of this process to perform pre-annotation of night and day attributes on the BDD100K road-scene dataset. FiftyOne and Qdrant can be used together to efficiently perform a nearest-neighbor search on embeddings and act on the results on your image and video datasets. The beauty of this process lies in its flexibility and repeatability. You can easily load additional ground truth labels for new fields into both FiftyOne and Qdrant and repeat this pre-annotation process using the existing embeddings. This can quickly cut down on annotation costs and result in higher-quality datasets faster.

Messengers, network services, and other software cannot go without bots. In software, a bot is an application designed to automate (or execute, according to a preset script) an action created at a user’s request. In this article, Daniil Mikhov, .NET Developer at NIX United, will look at an example of creating a chatbot using Microsoft's Azure Bot Service. The article will be helpful for anyone who wants to develop bots using the service. Why Azure Bot Service? The advantage of developing bots on Azure Bot Services is the high level of Microsoft support for its products. The company's experts actively communicate with the community and promptly identify and fix bugs in their services. In addition, Microsoft provides the ability to create custom JSON files to work with some messengers’ API, allowing you many possibilities when creating bots. It is also important to remember the other advantages: The Azure Bot Service allows you to use open-source SDK tools (Software Development Kit) to create, test, and deploy the bot. Integration with Cognitive Services is services that use machine learning tools in their work to solve typical tasks. CS ensures a better process of interaction between the bot and the user. Multi-platform is the ability to connect the bot to multiple channels without having to change the original code. A large number of open source examples to facilitate the development process and quick start (a lot of examples of ready-made code on GitHub). You can expand your bot infrastructure on Azure by adding new functionality. For example, you can add more channels and test with each of them. You can use the Cosmos DB service to store dialog states and information entered by the user. To train the bot, you can add LUIS (Language Understanding). This uses machine learning algorithms for better communication with the user. However, LUIS is not free, and not every one of your customers will want to allocate additional funding. Anatomy of a Bot in Azure Bot Services The structure of the functioning of the bot created on Azure can be represented as follows: You can see a list of possible channels to connect to the bot on the right. This list is constantly updated with new platforms. At the bottom are the available Microsoft Cognitive Services that the Azure platform can work with. These services allow you to communicate with the bot through voice requests, facial expressions, gestures, and more. The Bot Builder SDK is used to develop a bot on Azure. The product is in the public domain, and its main advantage is the constant support of developers. In a separate branch on GitHub, you can always get the latest information about the service or ask questions to its developers. Moving on to the Сreation of the Bot Before writing the code, let's analyze the nuances you should consider before creating a bot on Azure Bot Service: Updates that disrupt functionality. Microsoft is constantly updating its products. New updates often break parts of the code that used to work. So always read the patch lists of new Bot Builder SDK builds. Someone else’s manuals you will use to develop your bot may become irrelevant. Unobvious solutions. When using the Bot Builder SDK, you should always be open to experimentation and be willing to do things differently than you are used to. Versatility. The same bot can be uploaded to different channels (Telegram, Skype, Slack, and so on) without changing its source code. However, when developing a bot, you should keep in mind that each of these platforms has nuances which will require different approaches from you as a developer when creating the working logic of the application. Does the Bot Understand You Correctly? Communication with the bot is through the user interface. The user interface allows you to communicate with the bot in a language it understands. For this purpose, Azure uses a system of dialogs that follow a specific hierarchy: Here you can see three basic ways to build a dialog with the bot: Prompt—the bot interacts with the user through hints and answers. For example, you give the bot information as a numerical hint. Prompt checks whether the user correctly answered the cue. If it is successful, the dialog with the bot continues. If an incorrect answer was received from the user, he would be prompted to enter valid data. Waterfall is a way of collecting information from the user by going through a series of sequential tasks/questions. Each step of the waterfall dialog is implemented as an asynchronous function. At each stage, the bot asks the user for input data, waits for a response, and then passes the results to the next step. The result of the first function is passed to the following function as an argument, and so on until the whole cycle of questions has been passed. Component is a method that uses the breakdown of a large and voluminous dialog into smaller, easily manageable parts. Component allows you to create a reusable dialog and use it later for various independent scenarios. For example, you can use it to make a dialog that will sequentially ask the user for the street name/address/postal code. In the bottom row, you can see the allowed ways to create a custom request to the bot: Text query (Text) Number query (Number) Date/time request (DateTime) Confirm request (Confirm) Choice request (Choice) Attachment request (Attachment) In essence, queries are phased dialogs: in the first phase, the bot asks for input data, and in the second phase, it returns valid values to the user or starts the data query cycle anew if invalid values are received. Controllers and Templates Let's look at the code on the example of the Remind me Later chatbot I created—its main task was to remind me of any actions I need to take in the future. To create the bot, I used the Empty Template provided by Visual Studio, which includes several types of controllers: BotController and NotifyController. The BotController accepts messages for the bot and passes them to the bot framework. The bot also consists of several Deployment Templates, which are used to more easily deploy the application to the Azure platform. The NotifyController determines when to send a message to the user. I will talk about this in more detail later on. Startup Features and Filling the ToDoDialog Tab Let's go to the Startup.cs tab and look at its content. Here you can see the registered error handler, AdapterWithErrorHandler. It is necessary for the reaction of the application to errors in case they occur in the program. Pay attention to the registration ConversationState—we use it to make the bot understand with which user he is communicating and at what stage of the dialog. Let's look at the contents of the ToDoDialog.cs tab. I've declared waterfallSteps, a set of steps for the waterfall-dialog, which we've already mentioned above. In waterfallSteps we specify which asynchronous functions will be used to build a dialog between the user and the bot in each step. Below you can see what types of input Prompts our bot will use. The content here is pretty standard: the bot will ask us a few questions about the event and then offer to schedule a reminder. Now let's run the chatbot and test its operation using the Bot Framework Emulator interface. First Launch and Testing in the Bot Framework Emulator When you run the app, a link to the URL where the bot will wait for a message from the user appears. Before we start testing, let's specify this link in the Bot Framework Emulator: In the first communication step, the bot asks the user to enter the event name for which you want a reminder. To do this, call the following code: Now, when you call the bot, it will return the following text: Please enter event description. After declaring the event for which you want to make a reminder (for example: Buy Milk), call the code of the second step—here the bot will offer to choose one of the three options of the reminder time: Note the use of stepContext. It saves all the information about the dialog, recording intermediate values. To implement the list of possible reminder times, I used ChoicePrompt. This method will offer the user to choose 3 options with a possible reminder time (in 2 minutes, in 5 minutes, or the next day at the same time). There could be more choices, but I settled on three. We denote each new selection time by Choice and obtain: In the Bot Framework Emulator, this code will be rendered like this: The result can be parsed using Parse. Let me remind you that parsing is the process of automatically collecting data and then structuring it. The bot will then ask the user if they are sure about the selected reminder time, using ConfirmPrompt to confirm the agreement: Visually, this method will look as follows: The last step is to take the previously filled information out of stepContext and generate a SavedNotificationModel, to which we must add a conversationReference. Without it, the bot won't be able to restore the dialog with the user and won't figure out which user specifically addressed it. I used the dictionary method as a temporary repository for these events, thanks to which the bot assigns its unique instanceId to each specific dialog: This will end the dialog with the bot. You can display text to the user, indicating the end of the dialog and creating a corresponding reminder request: Thanks. Notification has been successfully saved. How the Bot Navigates Through Time To orient the bot in time, I created the NotifiedController method NotifyTimeCheck(). This method allows us to poll our application systematically, and if the time of an event is about to occur, the bot will retrieve this event from the dictionary and send a notification to the user. To get the notification, I will call the ContinueConversationAsync() method of the BotAdapter and pass the ConversationReference to it. The first parameter of ContinueConversationAsync() must always be the appId (Application ID) of our bot service, otherwise, it won't work. In addition, the bot needs to be reminded that when a certain time arrives, the event must be reminded to a particular user. You can use Azure Function (BotTimerFunction), which will be triggered by a time trigger (TimerTrigger). Every minute, the function will send a request to this Endpoint and start checking for the prescribed events. If it hits the correct time frame, the bot will notify the user that the scheduled event is imminent. Today WhatsApp, Facebook Messenger, Telegram, and other messengers are not only platforms for communication but also for doing business. Chatbots help companies effectively sell and promote goods and services online. Automating routine processes, promptly providing customers with necessary product information, and receiving and processing requests — all these features of an appropriately configured chatbot will help convert a regular user into a regular customer. So you, as developers, should remember how popular this tool is nowadays and how cool it is to be able to create such things and thus be a sought-after specialist.

Since Gartner coined the term AIOps in 2016, artificial intelligence has become a buzzword in the advanced technological world. The goal of AIOps is to automate complex IT systems resolution while simplifying their operations. Simply put, AIOps is the transformational approach that uses machine learning and AI technologies to run operations such as event correlation, monitoring, service management, observability, and automation. With AIOps, you can collect and aggregate ever-increasing data generated from observability and monitoring systems, different applications, or infrastructure, filter the noise to identify events and patterns for system performance and availability issues, and determine root causes and often resolve them automatically or send the alert to the IT team. If you aren’t using AIOps to complete the process, then it will become difficult to run alongside technology innovation taking place at a rapid pace. Besides, if you depend on traditional knowledge and old systems, your IT operations are more likely to become unpredictable and unscalable. As predicted by Gartner, 40% of the DevOps team is likely to implement AIOps in their applications and infrastructure monitoring tools for better platform performance and capabilities by 2023. AIOps Architecture The AIOps architecture provides methods and technologies that help in seamless integration for enterprise monitoring, service management, and automation to provide a complete AIOps solution. AIOps Architecture Enabling Insights Across Operation Monitoring. As shown in the image above, AIOps has three key areas when it comes to IT operations, namely Monitor (Observe), Engage, and Act. Unlike traditional event management and monitoring tools, in observability, machine-learning-based functions are used to ensure there aren’t gaps or blindspots left while serving the organizations’ monitoring needs regardless of their architecture. In the observability stage, primary processes that take place include data ingestion, data integration, event suppression, event deduplication, rule-based correlation, machine learning correlation (including anomaly detection, event correlation, root cause analysis, and predictive analytics), visualization, collaboration, and feedback. The Engage section of AIOps architecture is related to IT Service Management (ITSM) and its functions that deal with processes and their execution through different metrics and functions. As the Engage part deals with the data of service management, it acts as a repository for all the activities or actions occurring in ITSM, including problem management, configuration management, incident management, change management, capacity management, availability, and service-level agreements. While in Observability events, metrics, traces, and logs act as the primary data; in Engage, the primary data remains around the execution of actions in different processes where the data is a blend of on-demand and real-time analytics. The major phases in Engage consist of Incident Creation, Task Assignment, Task Analytics, Agent Analytics, Change Analytics, Process Analytics, Visualization, Collaboration, and Feedback. Finally, in the Act stage, the actual technical task execution takes place. The act is the final phase that executes all the technical tasks such as change execution, incident resolution, service request fulfillment, etc. It is here that all the incidents discovered are resolved, and the system gets back to its normal condition. How AIOps Works? You can simply understand the working of AIOps by looking at the technology components supporting its processes — machine learning, big data, and automation. AIOps work best when deployed independently and provide a centralized system to collaborate for collecting and analyzing data from multiple monitoring sources. Note: The data can consist of streaming real-time events, network data, historical performance events, system logs, and metrics, incident-related or ticketing. After collecting the data, AIOps implement machine learning and analytics capabilities to: Identifying and separating significant abnormal event alerts from tons of data. Detects the root cause of the abnormal events and proposes solutions. Automates alerts to the operation analysts along with the proposed solution. Create remedies for abnormal events based on the nature of the problem and address problems in real time. Finally, based on the analytics results, AIOps’ machine learning helps adapt algorithms and even creates new ones to determine problems at earlier stages and propose highly impactful solutions. Simply put, the AIOps model continues to improve, given the previous results. Core Elements of AIOps By now, you must know that the core elements behind AIOps are Big Data and Machine Learning. To understand these two terms, we will take a better look at each of them here. 1. Big Data As AIOps ingests data from numerous resources, it is essential to build the AIOps platform on Big Data technology. Big data refers to complex and large data sets that cannot be dealt with using traditional software for data processing. The data it contains comes in greater variety, increasing volumes, and high velocity, also known as the three V’s of Big Data. As AIOps integrate large, complex, variant data sets from different sources into a data warehouse, the velocity of processing so much data volume can become unmanageable in case one doesn’t use Big Data platforms. 2. Machine Learning The second yet most important part of AIOps is machine learning, a pivotal aspect of artificial intelligence. Machine Learning is centered on studying human behavior to replicate them using algorithms and data. When ML is implemented after gaining the information to solve a task, it can provide better accuracy in results than humans themselves. Similarly, ML helps AIOps platforms to leverage their power to analyze data and detect patterns and anomalies while monitoring events and entities. The analyzed data is then used to offer insights and reach the root-cause alerts. Benefits and Challenges of AIOps The Major Benefits of AIOps Are as Follows: Higher System Availability: As AIOps ensures maximum application availability for the modern hybrid infrastructure, It has become a potential game changer. Better SLA compliance in the meantime to repair: Integrated with IT Service Management functionalities, AIOps can find patterns in events, identify useful insights, and allow automation solutions. All of that reduces the mean time to repair while exceeding the SLA compliance. Minimum Human Errors: As AIOps automates most of the mundane and iterative tasks of the operations handled by IT teams, it reduces human errors simultaneously. Better Automated Incident Detection: A lot of time is saved by AIOps as it reduces the noise created due to pseudo incidents by leading through event analysis to verify the incident. Prediction and Outrage Prevention: AIOps use essential KPIs to measure the performance of operations, creating intelligent suggestions to help IT operations complete their goal. Cost Optimization: A matured AIOps system can impactfully bring down the costs of operations by offloading tasks from humans to algorithms, leading human resources to spend their time on other important tasks. Better Environment Visibility: Using AIOps, businesses can identify opportunities, make strategic decisions, and identify inefficiencies in IT operations. Some of the Challenges That AIOps Entail Are: Difficult Organizational Change Management. Mismatched Expectations. Rigid Processes. Difficulty in Data Availability and Monitoring. Lack of Domain Inputs. Inaccurate Predictive Analysis. Minimum Accuracy on Historical Data due to Data Drift. Difficulty in Understanding Machine Learning. Use Cases of AIOps As we know, AIOps is designed to gather and analyze IT operational data. Some of the popular use cases of AIOps are: Anomaly Detection AIOps continuously analyze and compare data to its historical events that help in detecting potential problems. Incident Event Correlation You can use AIOps for incident event correlation as it quickly processes and analyzes incident data while giving solutions to the problem before it gets out of control. Predictive Analytics Apart from early error detection, AIOps with data gathering and analyzing features can help machine learning algorithms understand current and historical data trends while offering actionable insights into future outcomes. Digital Transformation As AIOps removes the complexity of new technologies from ITOps, a new space for unrestricted transformation is created. It helps organizations to leverage flexibility to new advancements to deal with their strategic goals. Root Cause Analysis One can also use AIOps in analyzing root causes by correlating numerous data points, tracking patterns of events, and more. The root cause analysis of AIOps helps businesses as well as their users in identifying and resolving issues more effectively, making the customer experience better. Cloud Adoption/Migration With AIOps comes a clear understanding of cloud adoption and migrations’ transforming interdependence, minimizing the risks related to such shifting. Future of AIOps Given the advancements in technologies, most organizations are moving from traditional infrastructure to a dynamic one running on virtualized environments that can be reconfigured and scaled as required. But, as we know, these systems tend to generate an enormous volume of data endlessly. Even Gartner has suggested that IT infrastructures are more likely to create two to three times more operational data every year. Needless to say, traditional solutions can’t keep up with such data volume, sort events from the surrounding environment, or correlate data to provide real-time analysis and insights on IT operations to meet customer needs. However, with AIOps providing visibility into dependencies and performance throughout the infrastructure while analyzing data, extracting abnormal events, or automating alerts to the IT team, it becomes the best solution for modern organizations. Undoubtedly, AIOps are platforms utilizing modern machine learning and big data along with other advanced analytics technologies to improve IT operations with dynamic, proactive, and personalized insights by finding the root cause of problems and providing recommended solutions.

Any specialized software in artificial Intelligence, self-iterated data analysis, supervised learning, and other Machine Learning algorithms are considered machine learning software. Machine learning can be used in many software applications, including email classification or human-computer interaction. Machine learning software is available for modeling, designing, recruitment, and accounting. It can make all the difference between a useless bot and an AI system that is fully functional. Knowing which software package you should use can help you choose. Machine Learning Software Key Features There are many pattern recognition techniques, including classification, regression, and pattern recognition. Predictive analytics for image and text retrieval. Functionality for reducing the dimension. Assistance is provided by vector machines. Collaboration with machine learning libraries such as Apache SparkMLlib. Uses popular programming languages like Scala, Java, and C++. Machine learning with full-stack open source. 1. AmazonML Amazon Machine Learning (AML), a cloud-based, comprehensive machine learning tool that is accessible to all skill levels and online app developers, can be used by any level of developer. This managed service provides machine learning models and forecasting. It also integrates data from multiple sources such as Redshift, Amazon S3, RDS, and Amazon S3. Amazon Machine Learning provides visualization and wizard tools. Three types of models are supported: binary classification, multi-class classification, and regression. This tool allows users to create a data source object using a MySQL database. It also allows users to create data source objects from Amazon Redshift data. 2. Google ML Kit Mobile Google's Android Team created an ML KIT for mobile app developers that combines machine learning and technical knowledge to create more resilient and optimized apps to run on smartphones. This machine-learning software package can be used to perform tasks like face detection, text recognition, and landmark detection. It also helps with picture labeling and barcode scanning. You can access powerful technologies through it. It can be run on the device or in the cloud, depending on your needs. It can use either pre-made models or off-the-shelf solutions for software development. This kit includes Google's Firebase mobile-development platform. 3. Apple CoreML Apple Core ML, a platform that uses machine learning to assist you in integrating machine-learning models into your mobile apps, is available from Apple. Drop the file from machine learning into your project, and Xcode will instantly generate a Swift wrapper or Objective-C code. This method is easy to use and will make use of all CPUs and GPUs. CoreML supports Computer Vision to accurately analyze images, GameplayKit to assess learned decision trees, and Natural Language to perform natural language processing quickly. It is optimized for optimal performance on-device. 4. Apache Spark MLlib It's a machine learning library that can scale on Apache Mesos and Hadoop. It can also retrieve data from multiple data sources. There are several techniques that can be used to classify data, including naive Bayes and logistic regression. Regression: General Linear regression is also available. Clustering: K-means is another option. Its workflow tools include ML Pipeline Creation, Feature Transformations, ML Persistence, and so on. You can access Hadoop data sources like HDFS, HBase, or local files. It is easy to integrate with Hadoop operations because it has the ability to access Hadoop data sources such as HDFS, HBase, or local files. MLlib also integrates with Spark APIs and works well with NumPy in Python libraries and R libraries. It has superior algorithms to MapReduce. 5. Apache Singa This program was developed by the DB System Group of the National University of Singapore in collaboration with Zhejiang University's databases group. This artificial intelligence system aids in picture identification as well as natural language processing. It supports many well-known deep-learning models. It consists of three main parts: IO Core, Model, and Core. Tensor abstraction can be used to create more complex machine-learning models. This application provides improved IO classes to write, read, encode, and decode files and data. This application can be used to train synchronously, asynchronously, or in a combination of both. 6. Apache Mahout Apache Mahout is Scala's distributed linear algebra framework and Scala DSL. It is mathematically expressive. Apache Software Foundation's free and open-source project. This framework was created to rapidly develop algorithms for statisticians, mathematicians, and data scientists. It provides machine learning techniques like a recommendation, classification, clustering, and classification, as well as a framework to create scalable algorithms that can be expanded. It includes vector and matrix libraries and runs on Apache Hadoop using the MapReduce paradigm. 7. Accord.NET It integrates.Net machine-learning foundation with C# audio and image processing APIs. It has many libraries that can be used for various purposes, such as pattern recognition, data processing, and linear algebra. It also contains the Accord. Statistics, Accord.Math, and Accord.MachineLearning classes. Features of Accord.Net There are more than 40 estimates of statistical distributions that can be used to estimate non-parametric or parametric statistics. High-quality computer programs for computer vision, computer hearing, signal processing, and statistics. There are more than 35 hypotheses tests available, including one-way and two-way ANOVA tests. It supports more than 38 kernel functions. 8. Shogun It is an open-source and free machine learning library. It was developed by Gunnar Raetsch & Soeren Sonnenburg in 1999. This software can be implemented in C++. This software actually offers methods and data structures that can be used to solve machine-learning problems. It supports a variety of programming languages, including R, Python and Java, Octave as well as C#, Ruby, Lua, Lua, Ruby, C#, Ruby, and many others. Shogun is primarily focused on kernel machines such as regression issues and support vector machines to classify. You can connect to machine learning libraries like LibLinear and LibSVM. 9. TensorFlow It is an open-source machine learning library that allows you to build ML models. Google created Tensorflow. It offers a wide range of libraries, tools, and resources that allow researchers and developers to develop and deploy machine-learning systems. It helps you develop and train your models. TensorFlow.js is a tool that converts models to html. It is an open-source software program that can be used to compute numerically using data flow graphs. It can be used on both CPUs and GPUs as well as a range of mobile computing devices. 10. Google Cloud ML Engine Google Cloud ML Engine is a great tool to help you if you have billions or millions of training data points or if the algorithm takes a lot of time to execute correctly. It's a cloud-based platform that allows machine learning app developers as well as data scientists to create and execute high-quality models. All available options for machine learning models training, construction, deep-learning, predictive modeling, and even prediction are available. This application is used by many businesses for a variety of purposes. Businesses can use it to recognize clouds in satellite images or respond faster to customer emails. It can train a complex model in many ways. 11. IBM Machine Learning IBM Machine Learning Services allow you to combine and mix technologies such as IBM Watson Studio and IBM Watson OpenScale. Open source software is available to build AI models, integrate Models into your applications, and test them. IBM Machine Learning offers a free light plan that includes a cap of 20 CPUH and two simultaneous optimizations of batch tasks. 12. Oryx 2 It is built on Apache Spark and Apache Kafka, and it is an example of lambda architecture. It is used for large-scale, real-time algorithms. Orxy2 software development platform includes end-to-end applications for filtering and packaging, regression, classification, clustering, and classifying. Oryx 2.8.0 is the latest version of this utility. Oryx 2 refers to a more advanced version of the Oryx 1 project. It has three layers of cooperative work: the speed layer and the batch layer. The serving layer is the third. Also included is a data transport layer that transports data across different levels and receives input from external sources. 13. Neural Designer Neural Designer, a machine learning service on the rise, allows you to skip coding and create block diagrams using drag-and-drop and point-and-click capabilities. They offer a better average GPU training performance with a 417K+ sample rate than many other systems. Neural Designer is written entirely in C++. This compromises some usability benefits for faster performance. Excellent memory management is required for large data loads. Optimizing CPU and GPU performance allows for fast computations. 14. Azure Machine Learning Azure Machine Learning by Microsoft allows customers to quickly and easily build, train, deploy, and maintain machine learning models. QA managers will love the ability to quickly identify and test relevant methods using automated machine learning. It offers many enhancing features such as event processing, app services, and automation for up 500 minutes of task duration. You get a robust selection of add-ons, a long trial period, and monetary credits. 15. Anaconda Anaconda, a framework that supports the MLOps cycle, is used by American National Bank and AT&T, as well as Toyota and Goldman Sachs. The basic components of Conda include a Conda package administrator, unlimited corporate products and connectivity, as well as a replicable or cloud repository, and an environment administrator. Freelancing is easy with personal subscriptions. They are available for anyone to use, and they include hundreds of open-source frameworks and tools as well as 7500+ Conda Packages. Summary While some machine learning algorithms can be pre-designed to focus on a specific area, others are designed to allow users to create their own models using any data. There are different types of application software available in the market, and we have discussed here among those best software tools for machine learning technology. We looked at some of the most widely used machine learning tools and how they can be used for different purposes. There are many other machine learning libraries out there that don't make it onto the list, as the field of machine learning is growing.

Python is the most popular language for developing machine learning applications. According to one survey, 69% of developers that create machine learning programs use Python. There are many reasons machine learning developers use Python. One of the biggest benefits is that the language is platform independent. It is also simple, flexible, and has a large community that developers can call on for support. However, there is still a learning curve for Python developers creating machine learning programs. Fortunately, they can find clever ways to make the programs more efficiently. They can use great artificial intelligence tools and learn different coding hacks to simplify the coding process and work with large datasets more easily. One of the best ways to create more versatile machine learning applications in Python is to use lambda functions to create DataFrames. You will need to learn what lambda functions are and how to use them in the Pandas package to get the most benefit from them. Using Lambda Functions to Create Machine Learning Applications A growing number of developers are taking advantage of lambda functions to create powerful machine learning applications. Lloyd Hamilton of Towards Data Science talked about some of the benefits of using lambda functions to create more efficient machine learning applications. Some of these benefits include: Training classifiers for K-Nearest neighbors Initializing S3 Buckets for AWS Organizing and calling important Docker functions However, he didn’t discuss one of the biggest benefits, which is using them to create DataFrames by using the Pandas package. Lambda functions are gamechangers for machine learning development. However, developers that haven’t worked with them before might need to spend some time familiarizing themselves with their syntax. What Are Lambda Functions? Lambda or anonymous functions are Python functions that are typically defined in one line. They can be executed with very little code. Python lambdas are only a shorthand notation if you're too lazy to define a function. Here is an example of a Lambda function that adds two numbers: def sum(a, b): return a+b It could be expressed in the form of a lambda function as follows. lambda a, b : a + b The first difference is that a lambda function does not have a name. Therefore, unless it is assigned to a variable, it is totally useless. You can assign it with the following syntax: sum = lambda a, b: a + b Once we have the function, it is possible to call it as if it were a normal function. sum(2, 4) If it is a function that you only want to use once, it may not make sense to store it in a variable. It is possible to declare the function and call it on the same line. (lambda a, b: a + b)(2, 4) ## Examples A lambda function can be the input to a normal function. def my_function(lambda_func): return lambda_func(2,4). my_function(lambda a, b: a + b) And a normal function can also be the input of a lambda function. Note that these are didactic examples without much practical use per se. def my_other_function(a, b): return a + b (lambda a, b: my_other_function(a, b))(2, 4) Although lambda functions have many limitations compared to normal functions, they share a lot of functionality. It is possible to have arguments with a default value assigned. (lambda a, b, c=3: a + b + c)(1, 2) # 6 It is also possible to pass parameters indicating their name. (lambda a, b, c: a + b + c)(a=1, b=2, c=3) # 6 As in the functions, you can have a variable number of arguments by using *, known as tuple unpacking. (lambda *args: sum(args))(1, 2, 3) # 6 If you have the input parameters stored in the form of key and value as if it were a dictionary, it is also possible to call the function. (lambda **kwargs: sum(kwargs.values()))(a=1, b=2, c=3) # 6 Finally, it is possible to return more than one value. x = lambda a, b: (b, a) print(x(3, 9)) # Output (9,3) Using Lambda Functions to Create DataFrames With the Pandas Package Once you understand the basics of lambda functions, you can find creative ways to implement them while creating machine learning applications. Tonichi Edeza of Towards Data Science has emphasized a few of the benefits of using them for this purpose. One of the biggest benefits is that you can use lambda functions to create and add data to DataFrames. You can look at the code in the example in his blog post, which shows how you can create new columns in your DataFrame with only two lines of code. DataFrames are very versatile elements in Python and are crucial for many machine learning applications. They are part of the Pandas package, which is a very important toolkit for machine learning development. However, as Pavan Kalyan points out in his article Must know Pandas Functions for Machine Learning Journey, they can be difficult to work with due to the challenges they face with storing large data sets. This underscores the benefits of using lambda functions to work with them much more easily.

Deep learning is growing at a tremendous pace in terms of models and their datasets. In terms of applications, the deep learning market is dominated by image recognition followed by optical character recognition, and facial and object recognition. According to Allied Market Research, the global deep learning market was valued at$ 6.85 billion in 2020, and it is predicted to reach $179.96 billion by 2030, with a CAGR of 39.2% from 2021 to 2030. At one point in time it was believed that large and complex models perform better, but now it’s almost a myth. With the evolution of edge AI, more and more techniques came in to convert a large and complex model into a simple model that can be run on edge and all these techniques combine to perform model compression. What Is Model Compression? Model compression is a process of deploying SOTA (state of the art) deep learning models on edge devices that have low computing power and memory without compromising on models’ performance in terms of accuracy, precision, recall, and so on. Model compression broadly reduces two things in the model viz. size and latency. Size reduction focuses on making the model simpler by reducing model parameters, thereby reducing RAM requirements in execution and storage requirements in memory. Latency reduction refers to decreasing the time taken by a model to make a prediction or infer a result. Model size and latency often go together, and most techniques reduce both. Popular Model Compression Techniques Pruning Pruning is the most popular technique for model compression which works by removing redundant and inconsequential parameters. These parameters in a neural network can be connectors, neurons, channels, or even layers. It is popular because it simultaneously decreases models’ size and improves latency. Pruning Pruning can be done while we train the model or even post-training. There are different types of pruning techniques, including weight/connection pruning, neuron pruning, filter pruning, and layer pruning. Quantization As we remove neurons, connections, filters, layers, etc. in pruning to lower the number of weighted parameters, the size of the weights is decreased during quantization. Values from a large set are mapped to values in a smaller set in this process. In comparison to the input network, the output network has a narrower range of values but retains most of the information. For further details on this method, you may read our in-depth article regarding model quantization here. Knowledge Distillation In the knowledge distillation process, we train a complex and large model on a very large dataset. After fine-tuning the large model, it works well on unseen data. Once achieved, this knowledge is transferred to smaller Neural Networks or models. Both, the teacher network (a larger model) and the student network (a smaller model) are used. There exist two aspects here which is, knowledge distillation in which we don’t tweak the teacher model whereas in transfer learning we use the exact model and weight, alter the model to some extent, and adjust it for the related task. knowledge distillation system The knowledge, the distillation algorithm, and the teacher-student architecture models are the three main parts of a typical knowledge distillation system, as shown in the diagram above. Low Matrix Factorization Matrices form the bulk of most deep neural architectures. This technique aims to identify redundant parameters by applying matrix or tensor decomposition and making them into smaller matrices. This technique when applied on dense DNN (Deep Neural Networks) decreases the storage requirements and factorization of CNN (Convolutional Neural Network) layers and improves inference time. A weight matrix A with two dimensions and having a rank r can be decomposed into smaller matrices as below. Low Matrix Factorization Model accuracy and performance highly depend on proper factorization and rank selection. The main challenge in the low-rank factorization process is harder implementation and it is computationally intensive. Overall, factorization of the dense layer matrices results in a smaller model and faster performance when compared to full-rank matrix representation. Due to edge AI, model compression strategies have become incredibly important. These methods are complementary to one another and can be used across stages of the entire AI pipeline. Popular frameworks like TensorFlow and Pytorch now include techniques like pruning and quantization. Eventually, there will be an increase in the number of techniques used in this area.

NVIDIA NeMo is a toolkit for building new state-of-the-art conversational AI models. NeMo has separate collections for Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and Text-to-Speech (TTS) models. Each collection consists of prebuilt modules that include everything needed to train on your data. Every module can easily be customized, extended, and composed to create new conversational AI model architectures. So let's explain briefly what is the ASR, NLP, and TTS models. ASR, short for Automatic Speech Recognition, is a subfield of machine learning that focuses on techniques used to translate spoken languages into text formats. The main benefit behind these methods is to allow for better word searchability within a given voice message. ASR models are used commonly in today’s world; the most well-known practical usage does include Apple’s famous voice interaction system known as Siri. Natural Language Processing, or NLP, is an artificial intelligence subfield that is capable of understanding and processing different language formats to extract unique insights and patterns. Most common uses include recommendation systems, in which natural language processing techniques can categorize different items, objects, or services into different categories. Further on, if a user likes certain item categories, recommendation systems can suggest items in the same or similar categories. TTS, short for Text-to-Speech, is the exact opposite of the previously mentioned Automatic speech recognition field. In ASR, the machine learning model takes in a voice message and converts it into text format. Meanwhile, the Text-to-Speech model takes in data in text format and converts it into speech format. As the previously mentioned Siri model, an ASR system is used to convert a user's voice into text for Siri to understand then a TTS model is used for Siri to answer in a speech format as well. In the following sections of this article, we will show you how to run an actual natural language processing model using NVIDIA Nemo. Prerequisites to Using NVIDIA NeMo Before using NeMo, it’s assumed you meet the following prerequisites. You have Python version 3.6, 3.7, or 3.8. You have Pytorch version 1.8.1. You have access to an NVIDIA GPU for training. Using NLP to Build a Machine Learning Model Using natural language processing (NLP), we will build a machine learning model that is capable of detecting the sentiment of a given sentence. A sentiment is meant to categorize a given sentence as either being emotionally positive or negative. For example, a positive sentiment would be “he worked so hard and achieved great things.” On the opposite side of the spectrum, a negative sentiment would be “his performance was not good enough, and he failed to reach his goal.” Dataset The text classification model used in this tutorial requires the data set to be stored in a TAB-separated file (.tsv) format. The following is the format in which the data should be stored. [WORD][SPACE][WORD][SPACE][WORD][TAB][LABEL] To even better explain the given format, let's take the first line of our dataset as an example: The first data point is: hide new secretions from the parental units: 0 Thus the data must be stored in the format given below: [hide][SPACE][new][SPACE][secretions][SPACE][from][SPACE][the][SPACE][parental][SPACE][units][TAB][0] Our data set contains only 2 columns which are the Sentence and Label columns. The sentence column (independent variable) will be used as the input value of our model. This column contains the actual movie comment written by a given user. For the output variable, we will use the label column (dependent variable), which states the actual sentiment (positive or negative) of the previously written message. The label column is usually manually filled by human users. NVIDIA NeMo NLP Tutorial View the test classification sentiment analysis example used in this tutorial. 1. Choosing to Run the Model Either Locally or on Google Colab """ You can run either this notebook locally (if you have all the dependencies and a GPU) or on Google Colab. Instructions for setting up Colab are as follows: Open a new Python 3 notebook. Import this notebook from GitHub (File -> Upload Notebook -> "GITHUB" tab -> copy/paste GitHub URL) Connect to an instance with a GPU (Runtime -> Change runtime type -> select "GPU" for hardware accelerator) Run this cell to set up dependencies. """ # If you're using Google Colab and not running locally, run this cell # install NeMo BRANCH = 'r1.10.0' !python -m pip install git+https://github.com/NVIDIA/NeMo...;BRANCH#egg=nemo_toolkit[nlp] 2. Install ipywidgets and Upgrade the Jupyter Notebook If you're not using Colab, you might need to upgrade Jupyter notebook to avoid the following error: # 'ImportError: IProgress not found. Please update jupyter and ipywidgets.' Install the ipywidgets extension ! pip install ipywidgets ! jupyter nbextension enable --py widgetsnbextension # Please restart the kernel after running this cell Note that the above 2 steps will always be necessary to follow when you use Google Colab to run your models. 3. Import the Needed Libraries As the actual first step of training our model, we need to import all the needed libraries and packages required. Some necessary libraries that you may have worked with before might include the torch and pytorch libraries. In this model, we will also use a couple of NeMo packages, such as the nemo.collections and nemo.utils.exp_manager packages. from nemo.collections import nlp as nemo_nlp from nemo.utils.exp_manager import exp_manager import os import wget import torch import pytorch_lightning as pl from omegaconf import OmegaConf 4. Create the Data and Working Directory In this step, we will save our work by creating 2 new directories. You can choose not to create or to create and rename your own directories, but always make sure to replace the old directory names with the new ones in the code. The last two line commands start with the '!' will install the dataset automatically and then unzip it in the specified Data directory file. Saving you a bit of unnecessary work. DATA_DIR = "DATA_DIR" WORK_DIR = "WORK_DIR" os.environ['DATA_DIR'] = DATA_DIR os.makedirs(WORK_DIR, exist_ok=True) os.makedirs(DATA_DIR, exist_ok=True) ! wget <a href="https://dl.fbaipublicfiles.com/glue/data/SST-2.zip" target="_self">https://dl.fbaipublicfiles.com/glue/data/SST-2.zip </a>! unzip -o SST-2.zip -d {DATA_DIR} 5. Filter and Clean the Data Since the Nemo data set has its own unique format, any data set used should be inserted or modified to fit this required format. In our data set, we are required to remove the extra header line. This is done using the sed 1d command, which removes the first column from the specified data set. Note that we removed the header line from the training and validation data sets only as the testing data set does not have this problem. ! sed 1d {DATA_DIR}/SST-2/train.tsv > {DATA_DIR}/SST-2/train_nemo_format.tsv ! sed 1d {DATA_DIR}/SST-2/dev.tsv > {DATA_DIR}/SST-2/dev_nemo_format.tsv ! ls -l {DATA_DIR}/SST-2 To print the first 5 lines of each data set: print('Contents (first 5 lines) of train.tsv:') ! head -n 5 {DATA_DIR}/SST-2/train_nemo_format.tsv print('\nContents (first 5 lines) of test.tsv:') ! head -n 5 {DATA_DIR}/SST-2/test.tsv 6. Download the Configuration File The NVIDIA Nemo models work using a configuration system, in which a configuration file containing all the necessary data to run our model is saved and utilized. Most of the code written in this tutorial will update and add new data to this file. To begin with, NVIDIA provides you with a basic YAML configuration file that is ready to be used immediately. Of course, some modifications and tweaks will be needed from model to model. The configuration file will be downloaded to the specified directory. # download the model's configuration file MODEL_CONFIG = "text_classification_config.yaml" CONFIG_DIR = WORK_DIR + '/configs/' os.makedirs(CONFIG_DIR, exist_ok=True) if not os.path.exists(CONFIG_DIR + MODEL_CONFIG): print('Downloading config file...') wget.download(f'<a href="https://raw.githubusercontent.com/NVIDIA/NeMo/{BRANCH}/examples/nlp/text_classification/conf/">https://raw.githubusercontent.com/NVIDIA/NeMo/{BRANCH}/examples/nlp/text_classification/conf/</a>' + MODEL_CONFIG, CONFIG_DIR) print('Config file downloaded!') else: print ('config file already exists') config_path = f'{WORK_DIR}/configs/{MODEL_CONFIG}' print(config_path) config = OmegaConf.load(config_path) 7. Configure the Number of Classes in the Given Model The dataset.num_classes number indicates the different variable values that are used in our data set. In this step, we chose our value to be equal to 2 as our label column can have either a 0(negative) or a 1(positive) value. In cases where there is also a neutral value, for example, then we should have the num.classes=3. config.model.dataset.num_classes=2 8. Pass the Training and Validation File Paths to the Configuration File. After downloading and modifying ( removing the header line ) our data set files, we will pass their given path into our configuration file. This is a bit similar to the more known pd.read_csv(“file path”) function. config.model.train_ds.file_path = os.path.join(DATA_DIR, 'SST-2/train_nemo_format.tsv') config.model.validation_ds.file_path = os.path.join(DATA_DIR, 'SST-2/dev_nemo_format.tsv') # Name of the .nemo file where trained model will be saved. config.save_to = 'trained-model.nemo' config.export_to = 'trained-model.onnx' 9. Configuring Trainer String in the Configuration File Check if there's a GPU provided and if there is, then we will use it. config.trainer.accelerator = 'gpu' if torch.cuda.is_available() else 'cpu' config.trainer.devices = 1 Disable distributed training when using Colab to prevent errors. config.trainer.strategy = None Set up the max number of steps to reduce training time. The training stops when the max_epochs are reached. config.trainer.max_epochs = 1 Instantiates a PT Trainer object by using the trainer section of the config. trainer = pl.Trainer(**config.trainer) 10. Handling the Logging and Saving Checkpoints Nemo offers its users the ability to handle the logging and saving checkpoint tasks automatically. Since the experiment manager of a trainer object can not be set twice. We repeat the trainer creation code here to prevent errors when this cell is executed more than once. trainer = pl.Trainer(**config.trainer) The exp_dir specifies the path to store the checkpoints and also the logs; its default is "./nemo_experiments". The exp_dir provides a path to the current experiment for easy access. exp_dir = exp_manager(trainer, config.exp_manager) 11. Specify the BERT-Like Model and Create It After specifying the BERT-base-uncased model name, we will pass the configuration and the trainer parameters to our final model. BERT stands for Bidirectional Encoder Representations from Transformers, which is a transformer-based machine learning technique for natural language processing pre-training developed by Google. Set the ‘model.language_model.pretrained_model_name' parameter in the config to the bert-base-uncased model. config.model.language_model.pretrained_model_name = "bert-base-uncased" Creating the actual model. model = nemo_nlp.models.TextClassificationModel(cfg=config.model, trainer=trainer) 12. Run the Model Using the trainer.fit() function, the actual training of the model starts. trainer.fit(model) model.save_to(config.save_to) 13. Model Evaluation Extract the path of the best checkpoint from the training; you may update it to any checkpoint. checkpoint_path = trainer.checkpoint_callback.best_model_path Create an evaluation model and load the checkpoint. eval_model = nemo_nlp.models.TextClassificationModel.load_from_checkpoint(checkpoint_path=checkpoint_path) Create a data loader config for evaluation; the same data file provided in validation_ds is used here. eval_config = OmegaConf.create({'file_path': config.model.validation_ds.file_path, 'batch_size': 64, 'shuffle': False, 'num_samples': -1}) eval_model.setup_test_data(test_data_config=eval_config) A new trainer is created to show how to evaluate a checkpoint from an already trained model. Then we are going to create a copy of the trainer config and update it to be used for the final evaluation. eval_trainer_cfg = config.trainer.copy() eval_trainer_cfg.accelerator = 'gpu' if torch.cuda.is_available() else 'cpu' # it is safer to perform evaluation on single GPU as PT is buggy with the last batch on multi-GPUs eval_trainer_cfg.strategy = None # 'ddp' is buggy with test process in the current PT, it looks like it has been fixed in the latest master eval_trainer = pl.Trainer(**eval_trainer_cfg) eval_trainer.test(model=eval_model, verbose=False) # test_dataloaders=eval_dataloader, Extract the path of the checkpoint. checkpoint_path = trainer.checkpoint_callback.best_model_path Create an evaluation model and load the checkpoint. infer_model = nemo_nlp.models.TextClassificationModel.load_from_checkpoint(checkpoint_path=checkpoint_path) Move the model to the desired device for inference. We move the model to "cuda" if available otherwise, "cpu" would be used. if torch.cuda.is_available(): infer_model.to("cuda") else: infer_model.to("cpu") Define the list of queries for inference. queries = ['by the end of no such thing the audience , like beatrice , has a watchful affection for the monster .', 'director rob marshall went out gunning to make a great one .', 'uneasy mishmash of styles and genres .'] # max_seq_length=512 is the maximum length BERT supports. results = infer_model.classifytext(queries=queries, batch_size=3, max_seq_length=512) print('The prediction results of some sample queries with the trained model:') for query, result in zip(queries, results): print(f'Query : {query}') print(f'Predicted label: {result}') The following are the prediction results of the model: The prediction results of some sample queries with the trained model: Query : by the end of no such thing the audience , like beatrice , has a watchful affection for the monster . Predicted label: 1 Query : director rob marshall went out gunning to make a great one . Predicted label: 1 Query: uneasy mishmash of styles and genres . Predicted label: 0 For our 3 examples, the model predicted the results correctly, where 1 means that the sentiment of the sentence was positive and 0 means that it was negative. Using NVIDIA NeMo In this tutorial, we explained what NVIDIA NeMo is and its main focus on enhancing the machine learning field. We also explained some of its modules, such as Automatic Speech Recognition, Natural Language Processing, and Text-to-Speech , which are smaller subfields in text classification. Lastly, we used NVIDIA NeMo to build a natural language processing model that is capable of predicting a given text’s sentimental meaning. With conversational artificial intelligence being on the rise, machine learning toolkits such as NVIDIA’s NeMo are becoming more and more dependable by the day. By allowing its users to train their models with great ease, it might just be worth a try.

What Is Computer Vision? Computer vision (CV) is a major task for modern Artificial Intelligence (AI) and Machine Learning (ML) systems. It is accelerating almost every domain in the industry, enabling organizations to revolutionize the way machines and business systems work. Academically, it is a well-established area of computer science, and many decades' worth of research work have gone into this field to make it rich. The use of deep neural networks has recently revolutionized the field and given it new life. There is a diverse array of application areas for computer vision, such as: Autonomous driving Medical imaging analysis and diagnostics Detection of manufacturing defects Image and video analysis for surveillance footage Facial Recognition for security systems There are, of course, plenty of challenges associated with CV systems. For example, autonomous driving doesn’t just use object detection but also object classification, segmentation, motion detection, etc. On top of it, these systems are expected to process CV information in a fraction of a second and produce a high-probability decision. The higher-level supervisory control system has to make the decision, responsible for the ultimate driving task. Furthermore, multiple CV systems/algorithms are often at play in any respectable autonomous driving system. The demand for parallel processing is high in those situations, and this leads to high stress on the underlying computing machinery. If multiple neural networks are used at the same time, they may be sharing the common system storage and competing with each other for a common pool of resources. In the case of medical imaging, the performance of the computer vision system is judged by highly experienced radiologists and clinical professionals who understand the pathology behind an image. Moreover, in most cases, the task involves identifying rare diseases with very low prevalence rates. This makes the training data sparse and rarefied, i.e., not enough training images can be found. Consequently, the Deep Learning (DL) architecture has to compensate for this by adding clever processing and architectural complexity. Why TensorFlow for CV? TensorFlow is a widely used and highly regarded open-source Python package from Google that makes building computer vision deep learning models straightforward and easy. From its official website: “It has a comprehensive, flexible ecosystem of tools, libraries, and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML-powered applications.” With the release of TensorFlow 2.0 and Keras library integration as the high-level API, it is easy to stack layers of neurons and build and train deep learning architectures of sufficient complexity. Easy Model BuildingBuild and train ML models easily using intuitive high-level APIs like Keras for immediate model iteration and easy debugging Robust ML Production AnywhereEasily train and deploy models in the cloud, on-prem, on on-device, no matter what programing language you use. Powerful for ResearchSimple and flexible architecture to take new ideas from concept to code. Publish state-of-the-art models with confidence Now, of course, TensorFlow can be used to build deep learning models for all kinds of applications, including, Object detection Scene segmentation Generative adversarial network for synthetic images Autoencoders for image compression Recommender systems However, in this article, we focus on code and hands-on examples of building a simple object classification task with Convolutional Neural Network (CNN) using TensorFlow. This covers all the essential components of TensorFlow, such as layers, optimizers, error functions, training options, hyperparameter tuning, etc. Hands-on Example of an Object Classification Task Deep learning tasks and model training benefit extensively from specialized hardware like Gaming Processing Units (GPU). General-purpose CPUs struggle when operating on a large amount of data, e.g., performing linear algebra operations on matrices with tens or hundreds of thousand floating-point numbers. Under the hood, deep neural networks are mostly composed of operations like matrix multiplications and vector additions.GPUs were developed (primarily catering to the video gaming industry) to handle a massive degree of parallel computations using thousands of tiny computing cores. They also feature large memory bandwidth to deal with the rapid dataflow (processing unit to cache to the slower main memory and back) needed for these computations when the neural network is training through hundreds of epochs. This makes them the ideal commodity hardware to deal with the computation load of computer vision tasks. Using and Checking for GPU There are various avenues to use a GPU for deep learning tasks. Purchasing a bare metal server or workstation can be beneficial for those looking for peak customizability. Renting GPU computing resources on Cloud from AWS or GCP is great for those running short sessions with limited data. Or use free (but limited) resources like Google Colaboratory. The larger the dataset, the better the model, but in this example, we use the last option Google Colab. We first test whether a GPU is present for training, import tensorflow as tf device_name = tf.test.gpu_device_name() if device_name != '/device:GPU:0': raise SystemError('GPU device not found') print('Found GPU at: {}'.format(device_name)) Getting the Dataset We directly download the dataset into the local environment (of Google Colab) with the following code. If you are working on your local machine (and saved the file already), then modify the code accordingly. !wget --no-check-certificate \ https://storage.googleapis.com/laurencemoroney-blog.appspot.com/horse-or-human.zip \ -O /tmp/horse-or-human.zip Accessing the zipfile Contents The following python code will use the OS library to use Operating System libraries, giving you access to the file system and the zipfile library, allowing you to unzip the data. import os import zipfile local_zip = '/tmp/horse-or-human.zip' zip_ref = zipfile.ZipFile(local_zip, 'r') zip_ref.extractall('/tmp/horse-or-human') zip_ref.close() The contents of the .zip are extracted to the base directory /tmp/horse-or-human, which in turn each contains horses and humans subdirectories. In short, the training set is the data that is used to tell the neural network model that 'this is what a horse looks like,' ' this is what a human looks like,' etc. Note that we do not explicitly label the images as horses or humans. Later we will see something called an ImageGenerator being used -- and this is coded to read images from subdirectories and automatically label them from the name of that subdirectory. So, for example, we will have a 'training' directory containing a 'horses' directory and a 'humans' one. ImageGenerator will label the images appropriately for you, reducing the coding step. Let's define each of these directories. # Directory with our training horse pictures train_horse_dir = os.path.join('/tmp/horse-or-human/horses') # Directory with our training human pictures train_human_dir = os.path.join('/tmp/horse-or-human/humans') The following code will let us see what the filenames look like in the horses and humans training directories. train_horse_names = os.listdir(train_horse_dir) print(train_horse_names[:10]) train_human_names = os.listdir(train_human_dir) print(train_human_names[:10]) We can find out the total number of horse and human images in the directories. print('total training horse images:', len(os.listdir(train_horse_dir))) print('total training human images:', len(os.listdir(train_human_dir))) Now let's take a look at a few pictures to get a better sense of what they look like. We first configure the matplot parameters, <a data-re-name="format" data-dropdown="true" href="https://app.contentstack.com/#" alt="Format" rel="format" role="button" aria-label="Format" tabindex="-1" data-re-icon="true"></a> import matplotlib.pyplot as plt import matplotlib.image as mpimg # Parameters for our graph; we'll output images in a 4x4 configuration nrows = 4 ncols = 4 # Index for iterating over images pic_index = 0 Now, display a batch of 8 horse and 8 human pictures. We can rerun the following code in a Jupyter notebook cell to see a fresh batch each time. # Set up matplotlib fig, and size it to fit 4x4 pics fig = plt.gcf() fig.set_size_inches(ncols * 4, nrows * 4) pic_index += 8 next_horse_pix = [os.path.join(train_horse_dir, fname) for fname in train_horse_names[pic_index-8:pic_index]] next_human_pix = [os.path.join(train_human_dir, fname) for fname in train_human_names[pic_index-8:pic_index]] for i, img_path in enumerate(next_horse_pix+next_human_pix): # Set up subplot; subplot indices start at 1 sp = plt.subplot(nrows, ncols, i + 1) sp.axis('Off') # Don't show axes (or gridlines) img = mpimg.imread(img_path) plt.imshow(img) plt.show() Building the Architecture From Scratch First, import the TensorFlow library, import tensorflow as tf We then add convolutional layers and flatten the final result to feed into the densely connected layers. Finally, we add the densely connected layers. Note: We are facing a two-class classification problem, i.e., a binary classification problem; we will end our network with a sigmoid activation so that the output of our network will be a single scalar between 0 and 1, encoding the probability that the current image is class 1 (as opposed to class 0). model = tf.keras.models.Sequential([ # Note the input shape is the desired size of the image 300x300 with 3 bytes color # This is the first convolution tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(300, 300, 3)), tf.keras.layers.MaxPooling2D(2, 2), # The second convolution tf.keras.layers.Conv2D(32, (3,3), activation='relu'), tf.keras.layers.MaxPooling2D(2,2), # The third convolution tf.keras.layers.Conv2D(64, (3,3), activation='relu'), tf.keras.layers.MaxPooling2D(2,2), # The fourth convolution tf.keras.layers.Conv2D(64, (3,3), activation='relu'), tf.keras.layers.MaxPooling2D(2,2), # The fifth convolution tf.keras.layers.Conv2D(64, (3,3), activation='relu'), tf.keras.layers.MaxPooling2D(2,2), # Flatten the results to feed into a DNN tf.keras.layers.Flatten(), # 512 neuron hidden layer tf.keras.layers.Dense(512, activation='relu'), # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans') tf.keras.layers.Dense(1, activation='sigmoid') ]) Note the use of tf.keras API for building the model. The model.summary() method call prints a summary of the neural network. The "output shape" column shows how the size of your feature map evolves in each successive layer. The convolution layers reduce the size of the feature maps by a bit due to padding, and each pooling layer halves the dimensions. Next, we'll configure the specifications for model training. We will train our model with the binary_crossentropy loss because it's a binary classification problem and our final activation is a sigmoid.We will use the rmsprop optimizer with a learning rate of 0.001. During training, we will want to monitor classification accuracy. NOTE: In this case, using the RMSprop optimization algorithm is preferable to stochastic gradient descent (SGD), because rmsprop automates learning-rate tuning for us. Other optimizers, such as Adam and Adagrad, also automatically adapt the learning rate during training and would work equally well here. from tensorflow.keras.optimizers import RMSprop model.compile(loss='binary_crossentropy', optimizer=RMSprop(lr=0.001), metrics=['acc']) Data Preprocessing Let's set up data generators that will read pictures in our source folders, convert them to float32 tensors, and feed them (with their labels) to our network. We'll have one generator for the training images and one for the validation images. Our generators will yield batches of images of size 300x300 and their labels (binary). As you may already know, data that goes into neural networks should usually be normalized in some way to make it more amenable to processing by the network. (It is uncommon to feed raw pixels into a convent.) In our case, we will preprocess our images by normalizing the pixel values to be in the [0, 1] range (originally all values are in the [0, 255] range). In Keras, this can be done using the rescale parameter: keras.preprocessing.image.ImageDataGenerator This ImageDataGenerator class allows you to instantiate generators of augmented image batches (and their labels) via .flow(data, labels) or .flow_from_directory(directory). These generators can then be used with the Keras model methods that accept data generators as inputs: fit_generator, evaluate_generator, and predict_generator. from tensorflow.keras.preprocessing.image import ImageDataGenerator # All images will be rescaled by 1./255 train_datagen = ImageDataGenerator(rescale=1/255) # Flow training images in batches of 128 using train_datagen generator train_generator = train_datagen.flow_from_directory( '/tmp/horse-or-human/', # This is the source directory for training images target_size=(300, 300), # All images will be resized to 300x300 batch_size=128, # Since we use binary_crossentropy loss, we need binary labels class_mode='binary') Training and Accuracy Graph Let's train for 15 epochs -- this may take a few minutes to run on a typical Google Colab hardware. The Loss and Accuracy are a great indication of the progress of training. It's making a guess as to the classification of the training data and then measuring it against the known label, calculating the result. Accuracy is the fraction/percentage of correct predictions. history = model.fit_generator( train_generator, steps_per_epoch=8, epochs=15, verbose=2) The following code will help us plot the accuracy graph i.e., how the accuracy has improved with repeated training (passing the images through the network back and forth) epochs. plt.plot(history.history['acc'],c='k',lw=2) plt.grid(True) plt.title("Training accuracy with epochs\n",fontsize=18) plt.xlabel("Training epochs",fontsize=15) plt.ylabel("Training accuracy",fontsize=15) plt.show() Prediction Let's now take a look at actually running a prediction using the model. This code will allow you to choose 1 or more files from your file system, it will then upload them and run them through the model, indicating whether the object is a horse or a human. For this purpose, we use files.upload() method from the google.colab class. NOTE: Please download some images of horses and humans from the internet and keep them on your local hard drive as you will need to upload them for testing the model. You can upload multiple images at a time. import numpy as np from google.colab import files from keras.preprocessing import image uploaded = files.upload() for fn in uploaded.keys(): # predicting images path = '/content/' + fn img = image.load_img(path, target_size=(300, 300)) plt.imshow(img) plt.show() x = image.img_to_array(img) x = np.expand_dims(x, axis=0) images = np.vstack([x]) classes = model.predict(images, batch_size=10) if classes[0]>0.5: print(fn + " shows a human image") else: print(fn + " shows a horse image") Here are some example test images. If trained properly, the network will predict ‘human’ class for the first one and ‘horse’ class for the other three. The model got some right but some wrong. The model has a high chance of making a mistake in the last image. You are encouraged to test with your images and check the performance of the model. Computer Vision With TensorFlow This article covered building a computer vision model with convolutional neural net architecture using the TensorFlow and Keras framework for an object classification task. Hands-on code was provided and all the important components of TensorFlow - layers, model compile, optimizers, loss functions, and even Image generator class - were covered in this one example. Take this as a starting point and explore many more CV tasks using TensorFlow, such as object detection, semantic segmentation, etc.

What Is Computational Aesthetics? Computational aesthetics studies the design and appearance of things created using computers. It is relatively new and still in its infancy. However, it has already had a significant impact on the way we think about robotics and automation. Computer aesthetics focuses on how computers can create perfect-looking designs without human interference or input. That means it can allow us to design robots without worrying about how they will look or function once produced. That allows designers the freedom and flexibility in creating their products, as they don't have to worry about how things will look or function once they're finished. It also makes it easier for them to experiment with new ideas and styles since these won't have any immediate consequence on how the final product looks or functions. There are several branches of computational aesthetics, all of which involve studying and evaluating the graphical and symbolic aspects of computer-generated images and animations. They include: Formalism: This approach focuses on the formal structure of images and animations and their aesthetic qualities. Poststructuralism: This approach studies how social contexts shape images and animations, which can influence their meaning and value. Deconstructionism: This approach examines images and animations from a critical perspective, looking at how they undermine traditional notions of reality or truth. Structuralism: This perspective focuses on how images and animations are constructed based on specific structural rules. What Does Computational Aesthetics Bring to Robotics Automation? Robotics automation has revolutionized manufacturing and logistics. At the same time, it has spawned a new branch of computational aesthetics known as robotic informatics. Robotic informatics studies how robots interact with their surroundings and how this affects the aesthetics of their design. It considers such questions as how to create visually appealing robots and how to ensure that they behave in an aesthetically pleasing way. Though research is still ongoing in this area, there are already some promising results. For example, a study shows that people respond more favorably to visually engaging robots with well-designed features. That may help increase efficiency and productivity in the industry while creating a more positive image for robotics technology. How Do Computational Aesthetics Impact Robotics Programming? One of the main ways computational aesthetics impacts robotics programming is by providing new methods for designing and improving robots. Such new methods are artificial intelligence (AI), machine learning (ML), and deep learning (DL). All these methods involve teaching computers to do things that were once impossible for them. ML, in particular, is a form of AI that allows computers to learn from experience and improve their performance over time based on this experience. DL is similar to ML. The difference is that it enables robots to gain knowledge from data and other machines. With DL, robots can become more creative — they can learn how to create designs that no human could ever come up with on their own, at least not without much effort! How Does Programming Improve Computational Aesthetics? Robotics automation has demonstrated that it has the potential to improve the efficiency and accuracy of factory operations, especially as robots become more and more popular in the world of design. However, it can also lead to the creation of aesthetically displeasing objects. To avoid this, we must learn how to program robots properly. That will allow us to create objects that look good and function effectively. It will also afford us a level of precision and detail that wasn’t possible before. There are many robots with advanced computational aesthetics, and they are becoming increasingly common in the fields of manufacturing, healthcare, and security. There are a few principles to follow when programming robots to ensure computational aesthetics: Keep programs simple and easy to understand. Ensure that every program is legible and unambiguous. Use consistent terminology throughout your programs. Design programs so they can be ported easily from one environment to another. Examples of Aesthetically Pleasing Robots Some famous examples of aesthetically pleasing robots include Google's self-driving cars and Five9's tactile vacuum cleaners. Both technologies have been met with overwhelming approval by consumers, thanks to their innovative design and impressive functionality. Other robots with advanced computational aesthetics that you may be familiar with include DARPA's Intelligent Robotic Vehicle (IRV) program and SoftBank Robotics' Pepper humanoid robot. These two technologies are particularly notable for their ability to interact naturally with humans, making them potentially invaluable in future workplaces. There are other robots with advanced computational aesthetics, and they're becoming more common each year. Here are some examples: The humanoid robot, Sophia, is designed to look and feel like a human being and learn and understand human emotions. The robots developed by Hanson Robotics can navigate their surroundings using natural language processing, which allows them to interact with humans in a more human-like way. The HAL project was a collaboration between scientists at the University of Tokyo and NEC Corp. HAL was a humanoid robot that can exhibit realistic facial expressions and movements. Importance of Computational Aesthetics in Robotics Automation One of the key benefits of using robots is that we can design them to look aesthetic without having to program them explicitly with aesthetics in mind. That is because the design process for robots depends on a different set of principles than traditional design processes. Computational aesthetics plays a crucial role in creating visually appealing robots. It does this by enhancing the user's experience with robots to be likable and easy to employ. That allows people to focus on a task rather than feeling intimidated or uncomfortable around robots. In addition, computational aesthetics also help us understand how humans interact with objects and systems, which can help designers to create more intuitive designs for humans to use. That is particularly important as we move towards ever-more-complex machines and systems. Besides looking good, an effective and efficient design should be easy to use. By studying the usability of the design process, designers can ensure that their robots are easy to understand and use, regardless of what language or interface they are using. That will make it easier for people to interact and share information with them. Where Next Is Computational Aesthetics Going? Current research on computational aesthetics focuses on how we can design robots and style their appearance after human perception and cognition. It investigates how the design of a robotic can achieve aesthetic goals, such as reducing noise levels or improving occupant comfort. It also examines how visual information processing contributes to our understanding of robotics aesthetics. Overall, this research is helping us to understand how robots can achieve desirable outcomes, both within specific application contexts and, more generally, across different areas of life. It may even empower us to develop new aesthetic paradigms that can apply across diverse disciplines.

For context, artificial intelligence in this article refers to its modern state and not the ideal goal. We live in a world of narrow or weak AI, which beats humans at individual tasks such as trying out basic troubleshooting options faster than a developer would. We’re still years or decades away from truly strong AI that would do almost anything a human could. It means that artificial intelligence tests won’t happen without human input, but you can minimize the effort that much. How Does AI Implementation Improve the Software Testing Process? Artificial intelligence in software testing is the natural evolution of automated QA. AI test automation goes a step further than emulating manual work. “The machine” also decides when and how to run the tests in the first place. Innovation doesn’t end here. Artificial intelligence tests are already a thing. Depending on the implementation, tests will be modified and/or created from scratch without any human input. This is a wonderful solution if project complexity makes you wonder how to test — AI could very well be the answer. Benefits of AI This section alone warrants a series of articles depending on the definition, among other factors. Let’s stick to the benefits of AI tests and other uses of artificial intelligence for QA. AI automated testing is a time saver. We’ve covered using test automation tools to achieve scheduling miracles, but let’s take things up a notch. What if you could also maintain useful tests only? For instance, you can automatically sunset or suspend tests that never fail to investigate if they are indeed a waste of time. Test consistency can be increased quite a bit. It’s natural to occasionally run into flaky tests that fail for no apparent reason. It is possible to automatically flag such tests for artificial intelligence’s review that will identify a coding issue or point you to a conceptual flaw found across several tests. Test maintenance becomes much less cumbersome. It is especially relevant for B2C solutions that are often tweaking the user interface daily (if not more frequently) for A/B purposes. Such small changes could still be disruptive for tests imitating user journeys, e.g., a button is simply not there anymore. Combining artificial intelligence + test automation means that your tests are adjusted for the UI changes without human input. Best Practices Here is some advice coming from the trial and error of companies at the bleeding edge of artificial intelligence testing. Know what you’re getting into. Pushing for test automation without adequate preparation is a huge time sink. Just like with automated tests, lacking a senior specialist that will lead the way is catastrophic. Get your test suite in order. Missing or incorrect labels, typos, and legacy databases may all skew data that will be used by artificial intelligence to improve your testing. Write down goals for implementing AI. This includes business goals that you hope to tackle (e.g., measurable improvements in retention through smoother UX), QA goals that will verify your AI endeavor was worth the effort, and some AI testing benchmarks to see if you’re on the right track. Give your colleagues a heads up. Incorporating artificial intelligence into testing is a lengthy process, and it may affect the availability of QA specialists and their output in the least short term. Your Project Manager, Product Owner, and upper management will appreciate advance notice of such a drastic change. Naturally, developers should also be in the know, especially if they handle unit testing for the project. Make sure your test management is just as innovative. AI tests have little use if your team is still stuck doing QA on Excel. You need a dedicated test management solution that is friendly to third-party artificial intelligence tools. Methods for AI-Based Software Test Automation Methods for incorporating artificial intelligence into software testing mainly come from the most popular AI techniques. They are Machine Learning, Natural Language Processing (NLP), Automation/Robotics, and Computer Vision. Below are some examples of how these techniques are used for QA. Pattern recognition employs machine learning to find patterns in test and/or test execution that can be turned into actionable insights. If issues with one and the same class make multiple tests fail, your AI solution will ask the team to have another look at the potentially problematic code. Pattern recognition can also be used for your software’s code itself to spot and predict potential vulnerabilities. Self-healing corrects automated tests if they start to become a headache. Flaky tests can eventually be traced to the route of the problem. Seemingly unreproducible defects will be caught and resolved. Tests that fix themselves are a real game changer as your project gets bigger. Visual regression testing keeps both your software and tests for it in working order. This is where the UI tweaking example from earlier slots in. Good self-healing eliminates a lot of redundant work, empowers the product team to be more ambitious about A/B testing, and helps them respond to something trending really fast. Data generation is really useful alongside a primary software test tool. Artificial intelligence can be employed to parametrize tests on a bigger scale, e.g., generating tons of profile pictures with rare resolution and metadata to see if users can upload them fine. Best Testing Tools for AI Software Testing Let’s look at some tools employing the methods described above. Launchable Launchable uses pattern recognition to see how likely a test will fail. This information can be used to cut through the testing suite and eliminate some clear redundancies. Also, you can group tests and, for instance, run only the most problematic ones before deploying a hotfix. Launchable’s most recognized client is BMW. Percy Percy is a visual regression testing tool. It is great for keeping your UI tests relevant and helps you maintain consistency of user interface across different browsers and devices. Google, Shopify, and Canva are all in Percy’s client portfolio. mabl mabl is a neat test automation platform with self-healing functionality. It preaches a low-code approach yet can be used perfectly fine in the traditional way. Riot Games, JetBlue, and fellow IT companies like Stack Overflow and Splunk are featured on mabl’s website as clients. Avo Test Data Management Avo has a dedicated tool for managing test data, and the functionality includes AI data generation as well. The solution claims to mimic real-world data at a large scale with some data discovery on top. Avo is used by Sony, PwC, and also one of the aqua’s clients — Tech Mahindra. Conclusion Artificial intelligence methods in software testing are a truly powerful tool that pushes efficiency even further than regular automation does. Some subsets may seem a little excessive (e.g., data generation was a thing before people started labeling everything “AI”), but self-healing tests and pattern recognition are no small feats. Implementing AI in your quality assurance routine is certainly worth the effort as long as you formulate adequate goals and get the right people. However, introducing AI into your software testing is meaningless without a good test management solution. You need a solid test organization to dabble with AI, and any serious effort adds the complexity of juggling multiple artificial intelligence QA tools. Make sure that you find a good all-in-one test management solution before you set on a software testing AI journey.