DZone Spotlight

The subject of Web resource caching is as old as the World Wide Web itself. However, I'd like to offer an as-exhaustive-as-possible catalogue of how one can improve performance by caching. Web resource caching can happen in two different places: client-side - on the browser and server-side. This post is dedicated to the former; the next post will focus on the latter. Caching 101 The idea behind caching is simple: if a resource is time- or resource-consuming to compute, do it once and store the result. When somebody requests the resource afterwards, return the stored result instead of computing it a second time. It looks simple - and it is, but the devil is in the detail, as they say. The problem is that a "computation" is not a mathematical one. In mathematics, the result of a computation is constant over time. On the Web, the resource you requested yesterday may be different if you request it today. Think about the weather forecast, for example. It all boils down to two related concepts: freshness and staleness. A fresh response is one whose age has not yet exceeded its freshness lifetime. Conversely, a stale response is one where it has. A response's freshness lifetime is the length of time between its generation by the origin server and its expiration time. An explicit expiration time is the time at which the origin server intends that a stored response can no longer be used by a Cache without further validation, whereas a heuristic expiration time is assigned by a Cache when no explicit expiration time is available. A response's age is the time that has passed since it was generated by, or successfully validated with, the origin server. When a response is "fresh" in the cache, it can be used to satisfy subsequent requests without contacting the origin server, thereby improving efficiency. -- RFC 7234 - 4.2. Freshness Early Web Resource Caching Remember that the WWW was relatively simple at its beginning compared to nowadays. The client would send a request, and the server would return the requested resource. When the resource was a page, whether it was a static page or a server-rendered page was unimportant. Hence, early client-side caching was pretty "rustic". The first specification of Web caching is defined in RFC 7234, aka HTTP/1.1 Caching, in 2014. Note that it has been superseded by RFC 9111 since 2022. I won't talk here about the Pragma HTTP header since it's deprecated. The most straightforward cache management is through the Expire response header. When the server returns the resource, it specifies after which timestamp the cache is stale. The browser has two options when a cached resource is requested: Either the current time is before the expiry timestamp: the resource is considered fresh, and the browser serves it from the local cache Or it's after: the resource is considered stale, and the browser requires the resource from the server as it was not cached The benefit of Expire is that it's a purely local decision. It doesn't need to send a request to the server. However, it has two main issues: The decision to use the locally cached resource (or not) is based on heuristics. The resource may have changed server-side despite the Expiry value being in the future, so the browser serves as an out-of-date resource. Conversely, the browser may send a request because the time has expired, but the resource hasn't changed. Moreover, Expire is pretty basic. A resource is either fresh or stale; either return it Cache or send the request again. We may want to have more control. Cache-Control to the Rescue The Cache-Control header aims to address the following requirements: Never cache a resource at all Validate if a resource should be served from the cache before serving it Can intermediate caches (proxies) cache the resource? Cache-Control is an HTTP header used on the request and the response? The header can contain different directives separated by commas. Exact directives vary depending on whether they're part of the request or the response. All in all, Cache-Control is quite complex. It might be well the subject of a dedicated post; I won't paraphrase the specification. However, here's a visual help on how to configure Cache-Control response headers. The Cache-Control page of Mozilla Developer Network has some significant use cases of Cache-Control, complete with configuration. As Expire, Cache-Control is also local: the browser serves the resource from its cache, if needed, without any request to the server. Last-Modified and ETag To avoid the risk of serving an out-of-date resource, the browser must send a request to the server. Enters the Last-Modified response header. Last-Modified works in conjunction with the If-Modified-Since request header: The If-Modified-Since request HTTP header makes the request conditional: the server sends back the requested resource, with a 200 status, only if it has been last modified after the given date. If the resource has not been modified since, the response is a 304 without any body; the Last-Modified response header of a previous request contains the date of last modification. Unlike If-Unmodified-Since, If-Modified-Since can only be used with a GET or HEAD. -- If-Modified-Since Let's use a diagram to make clear how they interact: Note: the If-Unmodified-Since has the opposite function for POST and other non-idempotent methods. It returns a 412 Precondition Failed HTTP error to avoid overwriting resources that have changed. The problem with timestamps in distributed systems is that it's impossible to guarantee that all clocks in the system have the same time. Clocks drift at different paces and need to synchronize to the same time at regular intervals. Hence, if the server that generated the Last-Modified header and the one that receives the If-Modified-Since header is different, the results could be unexpected depending on their drift. Note that it also applies to the Expire header. Etags are an alternative to timestamps to avoid the above issue. The server computes the hash of the served resource and sends the ETag header containing the value along with the resource. When a new request comes in If-None-Match containing the hash value, the server compares it with the current hash. If they match, it returns an 304 as above. It has the slight overhead of computing the hash vs. just handing the timestamp, but it's nowadays considered a good practice. The Cache API The most recent way to cache on the client side is via the Cache API. It offers a general cache interface: you can think of it as a local key value provided by the browser. Here are the provided methods: Cache.match(request, options) Returns a Promise that resolves the response associated with the first matching request in the Cache object. Cache.matchAll(request, options) Returns a Promise that resolves to an array of all matching responses in the Cache object. Cache.add(request) Takes a URL, retrieves it and adds the resulting response object to the given cache. This is functionally equivalent to calling fetch(), then using put() to add the results to the cache. Cache.addAll(requests) Takes an array of URLs, retrieves them, and adds the resulting response objects to the given cache. Cache.put(request, response) Takes both a request and its response and adds it to the given cache. Cache.delete(request, options) Finds the Cache entry whose key is the request, returning a Promise that resolves to true if a matching Cache entry is found and deleted. If no Cache entry is found, the Promise resolves to false. Cache.keys(request, options) Returns a Promise that resolves to an array of Cache keys. The Cache API works in conjunction with Service Workers. The flow is simple: You register a service worker on a URL The browser calls the worker before the URL fetch call From the worker, you can return resources from the cache and avoid any request to the server It allows us to put resources in the cache after the initial load so that the client can work offline - depending on the use case. Summary Here's a summary of the above alternatives to cache resources client-side. Order Alternative Managed by Local Pros Cons 1 Service worker + Cache API You Yes Flexible Requires JavaScript coding skills Coding and maintenance time 2 Expire Browser Yes Easy configuration Guess-based Simplistic Cache-Control Browser Yes Fine-grained control Guess-based Complex configuration 3 Last-Modified Browser No Just works Sensible to clock drift ETag Browser No Just works Slightly more resource-sensitive to compute the hash Note that those alternatives aren't exclusive. You may have a short Expire header and rely on ETag. You should probably use both a level 2 alternative and a level 3 one. A Bit of Practice Let's put the theory that we have seen above into practice. I'll set up a two-tiered HTTP cache: The first tier caches resources locally for 10 seconds using Cache-Control The second tier uses ETag to avoid optimizing the data load over the network I'll use Apache APISIX. APISIX sits on the shoulder of giants, namely NGINX. NGINX adds ETag response headers by default. We only need to add the Cache-Control response header. We achieve it with the response-rewrite plugin: YAML upstreams: - id: 1 type: roundrobin nodes: "content:8080": 1 routes: - uri: /* upstream_id: 1 plugins: response-rewrite: headers: set: Cache-Control: "max-age=10" Let's do it without a browser first. Shell curl -v localhost:9080 Plain Text HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 Content-Length: 147 Connection: keep-alive Date: Thu, 24 Nov 2022 08:21:36 GMT Accept-Ranges: bytes Last-Modified: Wed, 23 Nov 2022 13:58:55 GMT ETag: "637e271f-93" Server: APISIX/3.0.0 Cache-Control: max-age=10 To prevent the server from sending the same resource, we can use the ETag value in an If-None-Match request header: Shell curl -H 'If-None-Match: "637e271f-93"' -v localhost:9080 The result is a 304 Not Modified as expected: Plain Text HTTP/1.1 304 Not Modified Content-Type: text/html; charset=utf-8 Content-Length: 147 Connection: keep-alive Date: Thu, 24 Nov 2022 08:26:17 GMT Accept-Ranges: bytes Last-Modified: Wed, 23 Nov 2022 13:58:55 GMT ETag: "637e271f-93" Server: APISIX/3.0.0 Cache-Control: max-age=10 Now, we can do the same inside a browser. If we use the resend feature a second time before 10 seconds have passed, the browser returns the resource from the cache without sending the request to the server. Conclusion In this post, I described several alternatives to cache web resources: Expiry and Cache-Control, Last-Modified and ETag, and the Cache API and web workers. You can easily set the HTTP response headers via a reverse proxy or an API Gateway. With Apache APISIX, ETags are enabled by default, and other headers are easily set up. In the next post, I will describe caching server-side. You can find the source code for this post on GitHub. To go further: RFC 7234: HTTP/1.1: Caching (obsolete) RFC 9111: HTTP Caching HTTP caching Cache-Control Cache API

Using YOLOv5 in PyTorch YOLO, an acronym for 'You only look once,' is an open-source software tool utilized for its efficient capability of detecting objects in a given image in real time. The YOLO algorithm uses convolutional neural network (CNN) models to detect objects in an image. The algorithm requires only one forward propagation through a given neural network to detect all objects in the image. This gives the YOLO algorithm an edge in speed over others, making it one of the most well-known detection algorithms to date. What Is YOLO Object Detection? An object detection algorithm is an algorithm that is capable of detecting certain objects or shapes in a given frame. For example, simple detection algorithms may be capable of detecting and identifying shapes in an image, such as circles or squares, while more advanced detection algorithms can detect more complex objects, such as humans, bicycles, cars, etc. Not only does the YOLO algorithm offer high detection speed and performance through its one-forward propagation capability, but it also detects them with great accuracy and precision. In this tutorial, we will focus on YOLOv5, which is the fifth and latest version of the YOLO software. It was originally released on the 18th of May, 2020. The YOLO open-source code can be found on GitHub. We will be using YOLO with the well-known PyTorch library. PyTorch is a deep learning open-source package that is based on the well-known Torch library. It's also a Python-based library that is more commonly used for natural language processing and computer vision. How Does the YOLO Algorithm Work? Step 1: Residual Blocks (Dividing the Image Into Smaller, Grid-Like Boxes) In this step, the complete (whole) frame is divided into smaller boxes or grids. All the grids are drawn over the original image sharing the exact shape and size. The idea behind these divisions is that each grid box will detect the different objects inside it. Step 2: Bounding Box Regression (Identifying the Object Inside a Bounding Box) After detecting a given object in an image, a bounding box is drawn surrounding it. The bounding box has parameters such as the center point, height, width, and class (object type detected). Step 3: Intersection Over Union (IOU) The IOU, short for intersection over union, is used to calculate our model's accuracy. This is achieved by quantifying the degree of intersection of two boxes: the real value box (red box in image) and the box returned from our result (blue box in image). In the tutorial portion of this article, we identified our IOU value as 40 percent, meaning that if the intersection of the two boxes is below 40 percent, then this prediction should not be taken into consideration. This is done to help us calculate the accuracy of our predictions. Below is an image showing the complete process of the YOLO detection algorithm For additional information on how the YOLO algorithm works, view the Introduction to YOLO algorithm. What Are We Trying to Achieve With Our Model? The main goal of the example in this tutorial is to use the YOLO algorithm to detect a list of chest diseases in a given image. As with any machine learning model, we will run ours using thousands of chest-scanned images. The goal is for the YOLO algorithm to successfully detect all lesions in the given image. Data Set The VinBigData 512 image Dataset used in this tutorial can be found on Kaggle. The data set is divided into two parts, the training, and the testing data sets. The training data set contains 15,000 images, while the testing data set contains 3,000. This division of data between the training and the testing is somehow optimal as the training data set is usually 4 to 5 times the size of the testing data set. The other part of the data set contains the label for all the images. Inside this data set, each image is labeled with a class name (chest disease found), along with the class ID, width and height of the image, etc. Check the below image to view all the columns available. YOLOv5 Tutorial Note: You can view the original code used in this example on Kaggle. Step 1: Importing the Necessary Libraries To start with, we will import the required libraries and packages at the very beginning of our code. First, let's explain some of the more common libraries that we just imported. NumPy is an open-source numerical Python library that allows users to create matrices and perform a number of mathematical operations on them. import pandas as pd import os import numpy as np import shutil import ast from sklearn import model_selection from tqdm import tqdm import wandb from sklearn.model_selection import GroupKFold\ from IPython.display import Image, clear_output # to display images from os import listdir from os.path import isfile from glob import glob import yaml # clear_output() Step 2: Defining Our Paths To make our life easier, we will start by defining the direct paths to the labels and the images of the training and testing data sets. TRAIN_LABELS_PATH = './vinbigdata/labels/train' VAL_LABELS_PATH = './vinbigdata/labels/val' TRAIN_IMAGES_PATH = './vinbigdata/images/train' #12000 VAL_IMAGES_PATH = './vinbigdata/images/val' #3000 External_DIR = '../input/vinbigdata-512-image-dataset/vinbigdata/train' # 15000 os.makedirs(TRAIN_LABELS_PATH, exist_ok = True) os.makedirs(VAL_LABELS_PATH, exist_ok = True) os.makedirs(TRAIN_IMAGES_PATH, exist_ok = True) os.makedirs(VAL_IMAGES_PATH, exist_ok = True) size = 51 Step 3: Importing and Reading the Textual Dataset Here we will import and read the textual data set. This data is stored as rows and columns in a CSV file format. df = pd.read_csv('../input/vinbigdata-512-image-dataset/vinbigdata/train.csv') df.head() Note: the df.head() function prints the first 5 rows of the given data set. Step 4: Filtering and Cleaning the Data Set As no data set is perfect, most of the time, a filtering process is necessary to optimize a data set, thus optimizing our model’s performance. In this step, we would drop any row with a class id that is equal to 14. This class id stands for a no finding in the disease class. The reason we dropped this class is that it may confuse our model. Moreover, it will slow it down because our data set will be slightly bigger. df = df[df.class_id!=14].reset_index(drop = True) Step 5: Calculating the Coordinates of the Bounding Box for YOLO As mentioned previously in the 'How does the YOLO algorithm work section' (particularly steps 1 and 2), the YOLO algorithm expects the dataset to be in a certain format. Here we will be going through the dataframe and applying a few transformations. The end goal of the below code is to calculate the new x-mid, y-mid, width, and height dimensions for each data point. df['x_min'] = df.apply(lambda row: (row.x_min)/row.width, axis = 1)*float(size) df['y_min'] = df.apply(lambda row: (row.y_min)/row.height, axis = 1)*float(size) df['x_max'] = df.apply(lambda row: (row.x_max)/row.width, axis =1)*float(size) df['y_max'] = df.apply(lambda row: (row.y_max)/row.height, axis =1)*float(size) df['x_mid'] = df.apply(lambda row: (row.x_max+row.x_min)/2, axis =1) df['y_mid'] = df.apply(lambda row: (row.y_max+row.y_min)/2, axis =1) df['w'] = df.apply(lambda row: (row.x_max-row.x_min), axis =1) df['h'] = df.apply(lambda row: (row.y_max-row.y_min), axis =1) df['x_mid'] /= float(size) df['y_mid'] /= float(size) df['w'] /= float(size) df['h'] /= float(size) Step 6: Changing the Provided Data Format In this part of the code, we will change the given data format of all rows in the dataset into the following columns; <class> <x_center> <y_center> <width> <height>. This is necessary since the YOLOv5 algorithm can only read the data in this specific format. # <class> <x_center> <y_center> <width> <height> def preproccess_data(df, labels_path, images_path): for column, row in tqdm(df.iterrows(), total=len(df)): attributes = row[['class_id','x_mid','y_mid','w','h']].values attributes = np.array(attributes) np.savetxt(os.path.join(labels_path, f"{row['image_id']}.txt"), [attributes], fmt = ['%d', '%f', '%f', '%f', '%f']) shutil.copy(os.path.join('/kaggle/input/vinbigdata-512-image-dataset/vinbigdata/train', f"{row['image_id']}.png"),images_path) We will then run the preproccess_data function two times, once with the training data set and its images and the second with the testing data set and its images. preproccess_data(df, TRAIN_LABELS_PATH, TRAIN_IMAGES_PATH) preproccess_data(val_df, VAL_LABELS_PATH, VAL_IMAGES_PATH) Using the line below, we will clone the YOLOv5 algorithm into our model. !git clone https://github.com/ultralytics/yolov5.git Step 7: Defining our Model’s Classes Here we will define the available 14 chest diseases in our models as classes. These are the actual diseases that can be identified in the data set’s images. classes = [ 'Aortic enlargement', 'Atelectasis', 'Calcification', 'Cardiomegaly', 'Consolidation', 'ILD', 'Infiltration', 'Lung Opacity', 'Nodule/Mass', 'Other lesion', 'Pleural effusion', 'Pleural thickening', 'Pneumothorax', 'Pulmonary fibrosis'] data = dict( train = '../vinbigdata/images/train', val = '../vinbigdata/images/val', nc = 14, names = classes ) with open('./yolov5/vinbigdata.yaml', 'w') as outfile: yaml.dump(data, outfile, default_flow_style=False) f = open('./yolov5/vinbigdata.yaml', 'r') print('\nyaml:') print(f.read()) Step 8: Training the Model To start, we will open the YOLOv5 directory. Then we will use pip in order to install all the libraries written inside the requirements file. The requirements file contains all the required libraries that the code base needs to work. We will also install other libraries such as pycocotools, seaborn, and pandas. %cd ./yolov5 !pip install -U -r requirements.txt !pip install pycocotools>=2.0 seaborn>=0.11.0 pandas thop clear_output() Wandb, short for weights and biases, allows us to monitor a given neural network model. # b39dd18eed49a73a53fccd7b684ea7ecaed75b08 wandb.login() Now we will train the YOLOv5 on the vinbigdata set provided for 100 epochs. We’ll also pass some other flags, such as --img 512, which tells the model that our image size is 512 pixels, --batch 16 will allow our model to take 16 images per batch. Using the --data ./vinbigdata.yaml flag, we will pass our dataset, which is the vinbigdata.yaml data set. !python train.py --img 512 --batch 16 --epochs 100 --data ./vinbigdata.yaml --cfg models/yolov5x.yaml --weights yolov5x.pt --cache --name vin Step 9: Evaluating the Model First, we will identify the testing data set directory along with the weights directory. test_dir = f'/kaggle/input/vinbigdata-{size}-image-dataset/vinbigdata/test' weights_dir = './runs/train/vin3/weights/best.pt' os.listdir('./runs/train/vin3/weights') In this part, we will use the detect.py as our inference to check the accuracy of our predictions. We will also pass some flags, such as --conf 0.15\, which is the model's confidence threshold. If the confidence rate of a detected object is under 15 percent, remove it from our output. The --iou 0.4\ flag informs our model that if the intersection over the union of two boxes is below 40 percent, it should be removed. !python detect.py --weights $weights_dir\ --img 512\ --conf 0.15\ --iou 0.4\ --source $test_dir\ --save-txt --save-conf --exist-ok Final Thoughts on Using YOLOv5 in PyTorch In this article, we explained what YOLOv5 is and how the basic YOLO algorithm works. Next, we went on to briefly explain PyTorch. Then we covered a couple of reasons why you should use YOLO over other similar detection algorithms. Finally, we walked you through a machine-learning model that is capable of detecting chest diseases in x-ray images. In this example, we used YOLO as our main detection algorithm to find and locate chest lesions. We then classified each lesion into a given class or disease. If you are interested in machine learning and building your own models, especially models that require the detection of multiple objects in a given image or video representation, then YOLOv5 is definitely worth a try.

With this design support, new query parameters will be as easy as adding a new query parameter as a property in the application's configuration file. Implemented this using Spring Boot framework and MongoDB as the backend database. HTTP GET is two types of path parameters and query parameters. In this article, I’m discussing query parameters. Query parameter GET call is a set of parameters that are part of the URL. Query parameters are appended in the URL after the HTTP resource with ? symbol and multiple parameters are concatenated by & symbol. Implementation Details The important part of this design is supporting the query parameters as they are referred to in the JSON path. With this, we can simply include the JSON path as the key and values in the Mongo Query. From below JSON example, forPerson resourceId for the JSON path is items.forPerson.resourceId. JSON { "_id": UUID('92f4b597-467a-4094-b796-27d1124c2fbd'), "forName": { "firstName": "Hemanth", "lastName": "Atluri" }, "invoiceNumber": "387373872", "items":[ "resourceId": UUID('bad40e6b-4d17-433c-9c56-b61ea5bef06f'), "forPerson": { "resourceId": "9714a305-a9ec-4a3f-9ba6–93766b2c77b7" } ] } In the implementation, all the query parameters provided are applied in AND fashion to filter the resources. Some of the query parameters have to be supported with exact values, greater than this value, less than this value, and this element exists. To support this, I used keywords from, to, after, before, and exists and added these to request parameters with a delimiter. Now let’s see the code to get a better understanding. Let’s start with the Controller code: Controller class takes the help of RequestParamValidatorAndConvertor and MongoHelper to give a response back to the client. The core logic is in these two classes. Below is the code snippet: Java @RestController @RequestMapping("/v1/orders") public class MongoRestController { Logger log = LoggerFactory.getLogger(MongoRestController.class); @Autowired private RequestParamValidatorAndConvertor requestParamValidatorAndConvertor; @Autowired private MongoHelper mongoHelper; @GetMapping public ResponseEntity search(@RequestParam MultiValueMap<String, String> requestParams) { log.info(requestParams.toString()); ResponseEntity responseEntity = null; try { List<QueryParameter> queryParameters = requestParamValidatorAndConvertor.validateAndConvertToQueryParameters(requestParams); List<Order> orders = mongoHelper.getOrders(queryParameters); responseEntity = ResponseEntity.status(HttpStatus.OK).body(orders); } catch (IllegalArgumentException ex) { responseEntity = ResponseEntity.status(HttpStatus.BAD_REQUEST).body("UnSpported Query Parameter"); } return responseEntity; } } RequestParamValidatorAndConvertor validates the provided query parameter and converts them into a List of CustomCriteria.Below is the code snippet: Java public enum MongoOperator { IN, EXISTS, LT,LTE,GT,GTE; } Java @NoArgsConstructor @AllArgsConstructor @Data public class QueryParameter { private String attributeName; private String value; private MongoOperator mongoOperator; } Java @Component public class RequestParamValidatorAndConvertor { @Value("#{'${supportedQueryParamters}'.split(',')}") private List<String> supportedQueryParamters; public List<QueryParameter> validateAndConvertToQueryParameters(MultiValueMap<String, String> requestParams) { List<QueryParameter> queryParameters = new ArrayList<>(); requestParams.forEach((key, value) -> { QueryParameter qp = new QueryParameter(); if (key.contains(":")) { String[] keySplits = key.split(":"); MongoOperator mongoOperator = null; String queryParameter = keySplits[0]; String queryParamOperator = keySplits[1]; if (!supportedQueryParamters.contains(queryParameter)) { throw new IllegalArgumentException("Unsupported Query Parameter"); } if (queryParamOperator.endsWith("from")) { mongoOperator = MongoOperator.GTE; } if (queryParamOperator.endsWith("to")) { mongoOperator = MongoOperator.LTE; } if (queryParamOperator.endsWith("after")) { mongoOperator = MongoOperator.GT; } if (queryParamOperator.endsWith("before")) { mongoOperator = MongoOperator.LT; } if (queryParamOperator.endsWith("exists")) { mongoOperator = MongoOperator.EXISTS; } qp.setAttributeName(queryParameter); qp.setMongoOperator(mongoOperator); } else { if (!supportedQueryParamters.contains(key)) { throw new IllegalArgumentException("Unsupported Query Parameter"); } qp.setAttributeName(key); qp.setMongoOperator(MongoOperator.IN); } qp.setValue(String.valueOf(value)); queryParameters.add(qp); }); return queryParameters; } } MongoHelper class encapsulates the building of the criteria and querying the MongoDB. Below is the code snippet: Java @Component @AllArgsConstructor public class MongoHelper { private MongoCriteriaBuilder mongoCriteriaBuilder; private MongoCustomRepository mongoCustomRepository; public List<Order> getOrders(List<QueryParameter> queryParameters) { Criteria criteria = mongoCriteriaBuilder.build(queryParameters); return mongoCustomRepository.findOrders(criteria); } } MongoCriteriaBuilder takes the QueryParameters for each query parameter; it creates Criteria and chains individual criteria with AND operator. @Component public class MongoCriteriaBuilder { public Criteria build(List<QueryParameter> queryParameters) { List<Criteria> criterias = new ArrayList<>(); queryParameters.forEach(queryParameter -> { Criteria criteria = null; switch (queryParameter.getMongoOperator()) { case EXISTS: criteria = Criteria.where(queryParameter.getAttributeName()).exists(Boolean.getBoolean(queryParameter.getValue())); case GT: criteria = Criteria.where(queryParameter.getAttributeName()).gt(queryParameter.getValue()); case GTE: criteria = Criteria.where(queryParameter.getAttributeName()).gte(queryParameter.getValue()); case LT: criteria = Criteria.where(queryParameter.getAttributeName()).lt(queryParameter.getValue()); case LTE: criteria = Criteria.where(queryParameter.getAttributeName()).lte(queryParameter.getValue()); default: List<String> values = Arrays.stream(queryParameter.getValue().split(",")).collect(Collectors.toList()); criteria = Criteria.where(queryParameter.getAttributeName()).in(values); } criterias.add(criteria); }); Criteria criteria = new Criteria().andOperator(criterias.toArray(new Criteria[criterias.size()])); return criteria; } } MongoCustomRepository build’s the mongo query with the criteria from the MongoCriteriaBuilder and passes it to the MongoTemplate to get the List of documents back. Java @Component @AllArgsConstructor public class MongoCustomRepository { private MongoTemplate mongoTemplate; public List<Order> findOrders(Criteria criteria){ Query query = new Query(); query.addCriteria(criteria); return mongoTemplate.find(query, Order.class); } } Conclusion With the above implementation, new query parameters can be supported through configurations. This article shares only the talks about querying; this doesn’t include the performance of the query. To achieve performance in the query, one must index the field and understand MongoDB indexing.

Edge computing is an emerging paradigm leading to major transformation in the networking world. The prime advantages that edge computing offers are reduced latency, bandwidth optimizations, and faster processing of data which leads to a better user experience. All depends on how time-critical the application in question is. With COVID-19 and the way the work from home culture got popular — streaming applications used for edutech, the collaborative tools, online healthcare, live trainings, and of course the OTT platforms which were a savior for people socially disconnected due to being in home isolation or taking general COVID precautions — the criticality of edge could be very well seen. All these applications could survive these tough times since the networking infrastructures across the world were mature enough to offer really low latency to these applications. Because of this, edge computing played a pivotal role getting content closer to the real user. Edge use cases were no longer confined to public safety, military uses, or manufacturing sectors — edge had a way bigger role to play in the daily lives of people. This wasn't necessarily evident to a consumer directly, but the service providers offering them network or real-world applications had a huge dependency on these emerging networking techniques. Edge Infra: "What All Is Involved?" Customers Ask Edge offered a lot of monetization opportunities to the small and the big players. There are equipment manufacturers who offer low footprint edge devices — be it edge in a box solution or devices embedded with small sensor SoCs. And then there are big players like Amazon, Google, and Microsoft who started offering a lot of edge infra as IaaS across the world — for the edge application developers as well as the service providers who could leverage their edge infra to offer an immersive and interactive experience to the end users. These edge solutions heavily use virtualization techniques — containerized apps, VMs, and so on, which helps one have more scalability based on the load and more reliability with redundant infra getting activated only on failures. This is way more cost-effective. Of course, for all of this to work as a whole, most of the edge solutions bank a lot on two things — automation and orchestration. Edge infra is generally spread out — you don’t expect someone to go and configure boxes every now and then. It is important to have automated deployments controlled and orchestrated from some central points across the globe. The right resource distribution and configuration is critical to get the best out of these edge deployments. These edge infra might be placed in the customer premise or in the communication service provider’s network — it might be a private cloud deployment based on OpenStack or Kubernetes, or it might be hybrid cloud deployments. Edge Infra: Commercial Offerings The whole idea of edge is a resource constrained environment — the push was toward low footprint data centers, densification of immense processing power in miniaturized form factors, moving toward containerized platforms, compact and self-contained data center building blocks which are optimized in terms of CPU or GPUs, memory, and so on. There have been offerings like edge in the box. All of these offer cost optimizations when one is planning an edge deployment. There are products from vendors like Qualcomm, AWS, and so on — AWS offers portable edge devices like Snowball and Snowcone. All these solutions have an emerging market in the time to come. When it comes to market offerings, AWS Wavelength is one popular solution for media streaming or real-time streaming. It offers wavelength zones, and the apps deployed in these zones can connect to rest of the application running somewhere in the cloud — so these zones directly connect to the AWS infrastructure across the globe. There are about 70+ zones across 20+ AWS regions. Another big offering is from Google Cloud. They have started offering the Media CDN platform, which is completely oriented toward media applications. This allows Google Cloud customers to reuse the existing infra for edge streaming applications. The Google network is spread across more than 200 countries and has more than 1000 cities covered. The Media CDN platform also has APIs and automation tools which aid the edge application developers to get more info and offer easy deployments with respect to their applications. Edge Infra: Open Source With the industry bending toward open-source solutions, edge infra and the edge platform market is also augmented by open source. The Linux Foundation offers StarlingX, which is a complete private cloud-based edge platform. Another option is Akraino, which offers blueprints for various edge scenarios. These blueprints can be used to deploy and maintain edge platforms over COTS infrastructure. These blueprints have specific target markets though — for instance, there is a radio edge cloud which offers 5G RAN edge. There is an edge automation blueprint, and the vertical edge applications blueprint which can be leveraged for the various real-time streaming applications. There are some more open-source options like Edge Foundry, applicable to the IoT edge focusing on the interoperability between heterogenous devices and applications, and EVE (Edge Virtualization Engine), which can offer orchestration for the cloud-native real-time streaming applications running in context of containers, VMs, or unikernels. With the kind of involvement and interest across industry, in the coming five years, we expect these open-source edge solutions to have more real world deployments — thus helping on cost and a vaster range of use cases.

Recently I had a task to migrate some data from an old monolith’s Oracle database to a microservice with a PostgreSQL database. The problem was that the data needed for migration had a parent table with around 2 million records with 150 columns and on top of that, everything was brought into view with a payload column aggregating data from various tables in XML. As you can imagine, the SELECT from that view was pretty slow, and by pretty, I mean insanely slow which was not going to work very well for the connector. So, in this article we’ll take a look at a similar simplified use case and how can we deal with it. Use Case We have a course-catalogue application with a PostgreSQL database that deals with instructors and their courses. Now we need to migrate some legacy instructors from another PostgreSQL database that soon is going to be decommissioned. So we have instructors-legacy-db and the course-catalog-db. In our case, both databases won’t be that overwhelmed with records, with just about 200 records for the instructors-legacy-db, but for the sake of the example, just imagine that instructors-legacy-db is that table with 2 million cumbersome records. Right, here is the docker-compose.yml YAML version: '3' services: course-catalog-operational-db: image: postgres:13.3 container_name: course-catalog-operational-db command: - "postgres" - "-c" - "wal_level=logical" environment: POSTGRES_PASSWORD: 123456 POSTGRES_DB: course-catalog-db ports: - "5433:5432" instructors-legacy-db: image: postgres:13.3 container_name: instructors-legacy-db command: - "postgres" - "-c" - "wal_level=logical" environment: POSTGRES_PASSWORD: 123456 POSTGRES_DB: instructors-db ports: - "5434:5432" volumes: - ./init.sql:/docker-entrypoint-initdb.d/init.sql Here is the course-catalog-db/instructors table: SQL create table instructors ( id integer not null primary key, created_at timestamp not null, updated_at timestamp not null, description varchar(3000), name varchar(255), summary varchar(3000) ); And here is the instructors-legacy-db/instructors table: SQL create table instructors ( id integer not null primary key, created_at timestamp not null, updated_at timestamp not null, first_name varchar(255), last_name varchar(255), title varchar(255) ); Also if you’ve noticed for the instructors-legacy-db container I’m using a init.sql script to create the table and do some inserts on the startup, so we’ll have some data to play around with. There is nothing special in that script, just 200 randomly generated inserts, here are some of them (the rest of them can be viewed on the repo) : SQL INSERT INTO public.instructors (id, created_at, updated_at, first_name, last_name, title) VALUES (0, '2022-08-28 21:00:19.315552', '2022-08-28 21:00:19.315552', 'Liam', 'Martinez', 'Mr.'); INSERT INTO public.instructors (id, created_at, updated_at, first_name, last_name, title) VALUES (1, '2022-08-28 21:00:19.315552', '2022-08-28 21:00:19.315552', 'Thomas', 'Williams', 'Mr.'); INSERT INTO public.instructors (id, created_at, updated_at, first_name, last_name, title) VALUES (2, '2022-08-28 21:00:19.315552', '2022-08-28 21:00:19.315552', 'Mateo', 'Martinez', 'Mr.'); INSERT INTO public.instructors (id, created_at, updated_at, first_name, last_name, title) VALUES (3, '2022-08-28 21:00:19.315552', '2022-08-28 21:00:19.315552', 'Ciro', 'Smith', 'Mr.'); And here is the view that we are going to address: SQL create view instructor_aggregate_vw(id, created_at, updated_at, name) as SELECT instructors.id, instructors.created_at, instructors.updated_at, (((instructors.title::text || ' '::text) || instructors.first_name::text) || ' '::text) || instructors.last_name::text AS name FROM instructors; Okay, not that we have everything in place, how do we get our data queried and published? Kafka Connect in Action That’s right, we are going to use io.confluent.connect.jdbc.JdbcSourceConnector for that. The Kafka Connect JDBC Source connector allows you to import data from any relational database with a JDBC driver into an Apache Kafka® topic. This connector can support a wide variety of databases. But first, we need to set up our Kafka Environment. For this, I am going to use the landoop/fast-data-dev’s docker image, since it comes with almost everything properly configured starting from the zookeeper, schema registry, kafka-connectand the broker and ending with a nice UI provided by Landoop for managing everything Kafka-related. Here is what we are going to add to our docker-compose.yml YAML course-catalog-kafka-cluster: container_name: course-catalog-kafka-cluster image: landoop/fast-data-dev environment: ADV_HOST: 127.0.0.1 RUNTESTS: 0 ports: - 2181:2181 - 3030:3030 - 8081-8083:8081-8083 - 9581-9585:9581-9585 - 9092:9092 volumes: # Specify an absolute path mapping - C:\Users\ionpa\Projects\course-catalog\infra:/my-data Okay, docker-compose up, and our entire environment is up. You can go ahead and check the UI on http://localhost:3030/ and explore it a bit. Now about the connector. Connectors can be added both through the UI on localhost here following a property-based or JSON-based configuration or by executing an HTTP request http://localhost:8083/connectors/ with the PUT method and a JSON body containing your connector’s config. Let’s take a look at the configuration that we are going to use for our connector: Properties files name=legacy.instructors.connector topic.prefix=legacy-instructors connector.class=io.confluent.connect.jdbc.JdbcSourceConnector table.types=VIEW connection.url=jdbc:postgresql://instructors-legacy-db:5432/instructors-db connection.user=postgres connection.password=123456 connection.attempts=100 connection.backoff.ms=300000 poll.interval.ms=100 transforms=AddNamespace,createKey,AddKeyNamespace transforms.AddNamespace.type=org.apache.kafka.connect.transforms.SetSchemaMetadata$Value transforms.AddNamespace.schema.name=inc.evil.coursecatalog.InstructorAggregate transforms.AddKeyNamespace.type=org.apache.kafka.connect.transforms.SetSchemaMetadata$Key transforms.AddKeyNamespace.schema.name=inc.evil.coursecatalog.Key transforms.createKey.type=org.apache.kafka.connect.transforms.ValueToKey transforms.createKey.fields=id mode=timestamp+incrementing timestamp.column.name=updated_at timestamp.delay.interval.ms=3000 incrementing.column.name=id numeric.mapping=best_fit query=select * from(select * from instructor_aggregate_vw where "updated_at" < ? AND (("updated_at" = ? AND "id" > ?) OR "updated_at" > ?) order by "updated_at","id" ASC) limit 50 -- First of all, hats off to Confluent, for doing such a great job of documenting everything. You can check the meaning of each property here. But, anyway, I am going to give a short description of what we have here: name – obviously, is the connector’s name. topic.prefix – because we are using a custom query this is the name of the topic that we are going to publish our records. connector.class – is the implementation of the connector that we are going to use, in our case it is the io.confluent.connect.jdbc.JdbcSourceConnector table.types – since we are going to query a custom view, the type is going to be VIEW. connection.* - are connection-related properties, obviously the connection URL, user, password to our DB and also some configuration related to the number of attempts and back off. poll.interval.ms – it is basically how frequently the connector should poll the table for new records. transforms.* - in my case configuration properties related to converting/serializing both the payload and the key to AVRO. mode- this is basically one of the most important properties, and it can have the following values: bulk – queries the entire table every time incrementing – will detect new rows based on an id column timestamp – will detect new/updated rows based on an automatically updated timestamp column timestamp+incrementing – considered to be the most robust and accurate mode, since it combines 2 modes mentioned above, and having both the timestamp column and id column in place allows us to identify both the new and updated rows uniquely timestamp.* - defines the required column and delay in milliseconds for the timestamp mode, in our case update_at incrementing.column.name – defines the required column for the incrementing mode, in our case id numeric.mapping – decides how are we going to treat, NUMERIC values, and in our case, it is best_fit which tells those numeric columns should be cast to INT or FLOAT based upon the column’s precision and scale query– this is the most essential property regarding to this article, so let us dive a bit deeper here. Basically, this property defines the query that is going to be used to address a table/view, and for our case, this should be enough, right? SQL select * from instructor_aggregate_vw Well not really, since we want this query to poll in batches, our query needs to be modified. Let’s say we want to query in batches of 50 records, this should be easily implemented with a simple LIMIT like this SQL select * from instructor_aggregate_vw limit 50 Yes and no, yes that is the correct implementation, and this won’t work for the Kafka Connector. And it won’t work because the query should be in such a form that WHERE clause should be "appendable", and why is that? Because of our specified mode, the connector will append a WHERE clause based on our defined timestamp/incrementing columns and the query, in the end, will look like this SQL select * from instructor_aggregate_vw WHERE "updated_at" < ? AND (("updated_at" = ? AND "id" > ?) OR "updated_at" > ?) ORDER BY "updated_at","id" ASC And if we add the LIMIT 50, that will break, as WHERE cannot be after LIMIT in PostgreSQL. So how do we deal with this? The trick is to take everything in our hands, comment on the generated WHERE clause by the connector, use a subquery to handle the WHERE clause ourselves and perform the LIMITing like this (pay attention to them --) SQL select * from(select * from instructor_aggregate_vw where "updated_at" < ? AND (("updated_at" = ? AND "id" > ?) OR "updated_at" > ?) order by "updated_at","id" ASC) limit 50 -- And this should work for any database, for example in Oracle you’ll do a WHERE rownum <= 50. This should be it, if you create this connector, you won’t see much since our view is very light on the data that it provides, and the streaming should happen really fast, but if you’re to use a heavy view/table you’ll notice how data appears in your queue in batches of 50 messages. Consuming If we are to go even further to consume this data and bring it in course-catalog-db/instructors, it is a matter of defining a sink connector since we don’t do any transformation or use a simple consumer. In Spring Boot with Kotlin, a Kafka listener would look like this Java @KafkaListener( topics = ["\${kafka.consumer.legacyInstructorsTopic}"], containerFactory = KafkaConfiguration.LEGACY_INSTRUCTORS_CONTAINER_FACTORY ) fun handle( @Header(KafkaHeaders.RECEIVED_MESSAGE_KEY) key: Key, @Payload instructorAggregate: InstructorAggregate ) { log.info("Received old instructor [${instructorAggregate}]") instructorService.upsert(instructorAggregate) } For the AVRO deserialization, I got the generated AVRO schemas from the schema registry here and saved the .avsc files. And with the help of this plugin id("com.github.davidmc24.gradle.plugin.avro") configured like this Groovy tasks.withType<com.github.davidmc24.gradle.plugin.avro.GenerateAvroJavaTask> { source(file("${projectDir}\\src\\main\\resources\\avro")) setOutputDir(file("${projectDir}\\src\\main\\kotlin")) } I obtained the POJOs, used in the listener. Conclusion We’ve seen how a connector can be configured to work in batches and track both new and updated records in the most robust way. You can check all the mentioned codes and files here. That’s all folks, I hope that you’ve learned something. Happy coding!

Artipie is an open-source binary artefact management tool under an MIT license. It supports the following types of repositories: Docker, Maven, NPM, PyPI, Anaconda, RPM, Gem, Go, and so on. If you are eager to learn about Artipie in detail, you can read the DZone article An Easy Way to Get Your Own Binary Repository. Let’s focus on the Maven repository. Maven is a tool that can be used for building and managing any Java-based project. Maven has two types of repositories: The local repository is a directory on the computer where Maven runs. Maven caches downloads from the remote maven repositories in ‘{user.home}/.m2’ folder. The local repository contains temporary build artefacts. Remote repositories are repositories accessed by using a variety of protocols. These repositories can be public, for example, repo.maven.apache.org or private, they are accessed only inside an organization, sharing private artefacts between development teams. Local and remote maven repositories have the same structure and are built, extended and supported according to the same “maven repository rules”. Thus, it’s possible to expose anyone’s local maven repository as a remote maven repository. We will set up a private remote maven repository. First, we have to get the Artipie jar file, which you can find on this page. In the next step, we should define an Artipie main configuration that is represented by a ‘yml’ file. So, let’s create a folder for Artipie config ‘{user.home}/artipie/config’ with the ‘my-artipie.yml’ file, which has the following content: YAML meta: storage: type: fs path: {user.home}/artipie/repo Where: Field ‘type’ describes which type of storage Artipie will use to get the configuration of repositories. In our case, it’s ‘fs’— the file system storage. Field ‘path’ is the absolute path to the directory in the file system where repository config files will be stored. Now, we are ready to define a maven repository config that is also represented by a ‘yml’ file: YAML repo: type: maven storage: type: fs path: {user.home}/artipie/data Where: Field ‘type’ describes a repository type; in our case, it’s ‘maven’. Field ‘storage’ defines the parameters of the file storage where artefacts are placed. We call this ‘my-maven.yml’ file and place it in the folder ‘{user.home}/artipie/repo’. As you will see later, Artipie uses the filename as the maven repository name. Now everything is done, and we are ready to launch Artipie. Execute the following command in a terminal: Shell java -jar {path-to-jar}/artipie-latest-jar-with-dependencies.jar \ --config-file={user.home}/artipie/config/my-artipie.yml \ --port=8085 Where: ‘--config-file’ parameter points to the Artipie main configuration file. ‘--port’ parameter defines the port to start the service. That is all. Now, we have our maven repository! Let’s try to deploy an artefact to our Artipie’s maven repository. We use the ‘hamcrest-2.2.jar’ file of the well-known Hamcrest library. Having this file downloaded on the local computer, I just copy it from the local maven repository ‘{user.home}/.m2/repository/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar’ to a temp folder, for example, ‘{user.home}/temp’. Next, to deploy a 3rd party JAR we will use the ‘deploy:deploy-file’ goal under maven-deploy-plugin. The deployment command can be the following: Shell mvn deploy:deploy-file -DgroupId=org.hamcrest \ -DartifactId=hamcrest \ -Dversion=2.2 \ -Dfile={user.home}/temp/hamcrest-2.2.jar \ -Durl=http://localhost:8085/my-maven/ You can learn more about the ‘deploy:deploy-file’ goal here. The console output should be: Plain Text Uploading to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar Uploaded to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.2/hamcrest-2.2.jar (123 kB at 851 kB/s) Uploading to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.2/hamcrest-2.2.pom Uploaded to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.2/hamcrest-2.2.pom (392 B at 21 kB/s) Downloading from remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/maven-metadata.xml Uploading to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/maven-metadata.xml Uploaded to remote-repository: http://localhost:8085/my-maven/org/hamcrest/hamcrest/maven-metadata.xml (296 B at 2.1 kB/s) Here we can see that the ‘hamcrest-2.2.jar’ artefact was successfully deployed to ‘http://localhost:8085/my-maven/’ repository. Artipie stores artefacts using the same folder structure as it does maven, so now the folder ‘{user.home}/artipie/data’ looks like in the following image: If you already have artefacts cached in a local maven repository and do not want to load them again, just copy the folder ‘{user.home}/.m2/repository’ to a directory where you are going to store your repository artefacts. In the example, we could copy the local maven repository content to the ‘{user.home}/artipie/data’ folder and these artefacts would be accessible by Artipie’s repository URL. This feature allows us to quickly and easily deploy Artipie’s maven repository based on the existing maven-compatible storage. Frequently, we want to use private and public maven repositories simultaneously. With Artipie, this requirement can be reached using the ‘maven-proxy’ repository type. The maven-proxy repository redirects the requests to the remote repositories. Let’s turn our existing maven repository into the maven-proxy repository. We have to change the ‘my-maven.yml’ file in the following way: YAML repo: type: maven-proxy remotes: - url: https://repo.maven.apache.org/maven2 - url: {your_private_remote_maven_repository} Where: The field ‘type’ is the ‘maven-proxy’ which means using the Artipie maven repository in proxy mode. The field ‘remotes’ defines the list of remote maven repositories which Artipie will request for needed artefacts. Note that you do not have to shut down Artipie to change the repository configuration. All repository configuration changes will be applied on the fly. Artipie will try to get the artefact from the list of remote repositories one by one until the artefact is found. To check to download, we will use the maven dependency plugin and any artefact, for example, the same ‘org.hamcrest:hamcrest’ artefact, but 2.1 version. The maven dependency plugin first tries to get an artefact from the local repository and only then from remote repositories. If you already have the folder ‘{user.home}/.m2/repository/org/hamcrest/hamcrest/2.1’, just delete it to force the maven plugin to request the artefact from the remote repository. Then you may add the repository’s description to ‘{user.home}/.m2/settings.xml’ or create a new maven settings file. Not to affect other projects, we will go with the second option — create maven settings ‘my-maven-settings.xml’ in the folder ‘{user.home}/artipie/’ and define the profile describing our remote repository: XML <?xml version="1.0"?> <settings> <profiles> <profile> <id>artipie</id> <repositories> <repository> <id>my-maven</id> <url>http://localhost:8085/my-maven/</url> </repository> </repositories> </profile> </profiles> <activeProfiles> <activeProfile>artipie</activeProfile> </activeProfiles> </settings> Now, we have to execute the Maven command: Shell mvn -s {user.home}/artipie/my-maven-settings.xml \ dependency:get -Dartifact=org.hamcrest:hamcrest:2.1:jar You can see the following piece of console output: Plain Text Downloading from my-maven: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.1/hamcrest-2.1.pom Downloaded from my-maven: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.1/hamcrest-2.1.pom (1.1 kB at 2.0 kB/s) Downloading from my-maven: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar Downloaded from my-maven: http://localhost:8085/my-maven/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar (123 kB at 676 kB/s) Artipie found the ‘org.hamcrest:hamcrest:2.1:jar’ artefact in the ‘https://repo.maven.apache.org/maven2’ repository and downloaded it. If someone else requests this artefact again, Artipie will try to get it from one of the repositories listed in the configuration file. However, to reduce internet traffic, it is useful to cache the downloaded artefacts in the maven-proxy storage. Let’s add the cache option to the ‘my-maven.yml’ file: YAML repo: type: maven-proxy remotes: - url: https://repo.maven.apache.org/maven2 cache: storage: type: fs path: {user.home}/artipie/data - url: {your_private_remote_maven_repository} Where: The ‘storage’ field defines the storage where artefacts are cached. Before trying to download the ‘hamcrest-2.1.jar’ file one more time, do not forget to delete the ‘{user.home}/.m2/repository/org/hamcrest/hamcrest/2.1’ folder. The console output should be the same as above, but now the folder ‘{user.home}/artipie/data’ has the following structure: For the next requests for the ‘org.hamcrest:hamcrest:2.1’ artefact, Artipie will get it in file storage rather than download one from external repositories. This behaviour is precisely what we wanted. Artipie is an open-source binary artefact management tool that allows you to create any repository from a bunch of supported repositories simply and fast. In this article, we don’t touch on some Artipie features, such as security, the ability to host data on a custom storage, etc. But you may find this information in Artipie’s wiki documentation. To encourage our efforts in growing Artipie, please give us a star on GitHub.

Quality assurance (QA) is the act or process of certifying that a company’s quality criteria are met. Planning, completing, and monitoring tasks are all part of managing quality in production. The monitoring aspect is the essence of what quality assurance is. The techniques and procedures that systematically monitor multiple aspects of a service or facility make up the quality assurance process. Quality assurance activities uncover and fix faults or deviations from defined standards or requirements through audits and other forms of assessment. In other words, quality assurance ensures high standards during the development of products or services. Any startup — from packaged food manufacturers to software development firms — and almost every company uses some type of quality assurance in their production. Some businesses even have a quality assurance department with people entirely dedicated to the task. How QA Works Rather than going through an unmonitored production process and trying to “inspect the quality” of a finished product, quality assurance test methodologies focus on developing good procedures to generate quality goods. Today, there are different ways to carry out quality assurance, depending on the industry. Because product development for a healthcare company is unique to a SaaS company that is creating a customer onboarding template, their methods to ensure quality will differ. A healthcare company's practices could be guided by active pharmaceutical ingredient criteria, but a SaaS firm’s processes will be guided by different standards. In this article, we’ll focus on QA for software companies. Here are five vital steps in setting up your startup’s QA process. 1. Define Your Testing Goals Quality assurance is frequently carried out before the software has been created. The documentation and requirements for the final product are checked by QA professionals to ensure that they are clear, consistent, and testable. Repairing problems during the testing stage is up to 15 times more expensive than fixing faults during the design and development stages; therefore, these preventative practices can help businesses save money. At this point, you should decide which areas and functionalities will be the focus of your testing. You could be looking at the present performance of your connector, or the features users are looking for based on the reviews they left on PCMag or Capterra. Both will offer good insights into what area you can start testing. Whichever that may be, QA specialists usually go over the program documentation and write a plan that specifies the scope of the testing, whether it should be automated testing, and which aspects will be examined. Software is generally tested for the following: Consistency Stability Executability Clarity Completeness Performance accuracy Your specific testing requirements will be determined by your product, how it will be distributed, your target audience, and your post-launch software support strategy. 2. Identify the Best Startup QA Testing Approach for You To begin, select a software testing technique that will help your QA team to complete your software development QA process as efficiently as possible. Employing a software testing technique that elevates your project management style can help you overcome difficulties that arise during testing cycles. Which software testing approach is best for your in-house team and startup QA outsourcing? Take into consideration the following methodologies for your quality assurance process: Agile Methodology QA teams can manage a project using exploratory testing in an agile approach by dividing it down into sprints. When preparing each sprint stage — including planning, requirements analysis, and testing — the QA team may consider using an Agile QA process to expedite the process. Each sprint concludes with a review meeting in which the team discusses their success and plans for future testing sprints. The QA team uses their previous knowledge to reduce risks and improve efficiency with each new iteration. Waterfall Methodology This software development quality assurance process is based on QA teams’ step-by-step testing progress. Before developing sequential steps for QA testers to follow, a QA team documents and plans the entire project. The team must follow the steps in sequence with no deviations. Many testing teams like the waterfall methodology — including QA outsourcing for startups — because it is simple and straightforward to execute within the development cycle. But, because of the strict structure, teams are unable to make quick changes to the testing process. Verification and Validation Methodology Like the waterfall methodology, this method takes a step-by-step approach to software testing. However, the development process runs concurrently with the testing phase, which makes it unique. When a development stage is completed, startup QA testing for that product element begins immediately. This approach to QA for startups allows teams to implement product improvements sooner, which frees up time and resources for other projects. Incremental Methodology This software testing process comprises various iterations, each of which has several stages. Each iteration tries to improve the product’s value by adding new features or improving quality. The incremental methodology involves the following three stages: Design and development Testing Final implementation This approach to QA testing is adaptable, allowing QA teams to modify the testing process in real-time. Regardless of Your Chosen Methodology... These four parameters should be the foundation of your tests: Component tests: to assist your development team and their processes. Business scenarios: to test the end-to-end business systemization and processes, starting with the customer’s perspective and ending with the end user’s perspective. User tests: to test the product from the user’s point of view. These are largely manual tests, which demand a full-time QA engineer or the use of QA outsourcing for startups. Technical tests: to assess the product’s performance, load, and security. Connectivity, performance, and recovery testing are all of these tests, which focus on the product’s infrastructure and platform. 3. Acquire QA at the Best Time Regarding acquiring QA for startups, it all comes down to how much risk you can tolerate. Getting your first paying client, having your first serious complaint, or noticing a lot of churn with your product are all good reasons to start developing your QA staff right now. When resources are limited, many startups benefit from outsourcing QA. So, how can you know when it’s time to recruit a quality assurance partner? When you are going through one (or all) of these problems: Your time to market is down You’re trying to integrate new tools and technologies Your team is constantly missing deadlines and pushing back launch dates to fix product flaws that may have been detected earlier by QA testers You’re implementing an agile methodology You’re receiving poor customer reviews 4. Analyze the Results After testing results have been obtained and any defects have been rectified, QA specialists rerun the identical test cases to ensure that the issues have been resolved effectively. The QA process is regressive, which means that the bulk of tests is repeated. Always incorporate regression testing into your QA process to ensure that one fix does not affect the normal operation of other features. It’s also crucial to retest your application using the integration testing approach because bug fixes might occasionally affect the behavior of several capabilities rather than merely breaking them. When you reach the deployment stage, the QA team creates and executes tests in line with the development team’s release documentation. For instance, the Azure data lake documentation contains all established functionalities, all conducted repairs, and software constraints that took place. This stage is referred to as smoke testing. During smoke testing, test case sets are used to confirm that the final build is functional, stable, and ready to deploy. 5. Make Adjustments and Tell Your Team Adjustments should be made when needed after listening to feedback and reviewing initial aims. While many people recommend doing a plan audit every two to three months, it can be done sooner if necessary. If a mistake is discovered or issues arise, implementing changes as quickly as possible is beneficial in reaching the company’s aim — particularly if it is an SMB — and saving or growing sales by introducing a better product or service. Also, employees want to know if what they’re doing is effective. Let them know if the QA approach has helped the company. Positive or constructive feedback helps to build a stronger team and shows employees that they are contributing to the company’s success. The fact that you are empowering employees to make a difference motivates them to keep doing a good job and provides valuable input into the QA process. As a result, have employees stick to the plan, provide both positive and negative feedback, and listen so that changes may be implemented. Conclusion As they say, preparation is key, like how reviewing the qualities of your standard operating procedure can make a difference to your business. The same goes for planning for quality assurance, as it saves your firm both time and money. Quality assurance is vital to maintaining high product quality standards and continued business success. Software is a rapidly evolving industry, and there are always efficiencies to be made and processes to be optimized. Follow these steps, and don’t forget to customize them for your business.

If you’ve followed along with the Flow series so far, you already know that the Flow Blockchain excels in handling digital assets, such as NFTs. It was built from the ground up as a better alternative to Ethereum’s network congestion and high fee issues. In addition, the Cadence smart contract language is a first-of-its-kind resource-oriented programming language that makes creating and managing digital assets easy and efficient. Although Solidity is excellent at facilitating Web3 through smart contracts, there are drawbacks. Cadence improves upon Solidity’s flaws by providing the ability to upgrade smart contracts and features that reduce the risk of human error, among other improvements. And finally, the list of tools and libraries available to developers looking to get started is extensive. So let’s put it all together and build something on Flow. This article is a tutorial about creating a full-fledged NFT-minting dApp for the Flow Blockchain. Let’s Get To It For the rest of this article, we will walk through the process of creating an NFT minting dApp on the Flow blockchain. We will start with setting up and deploying a Cadence smart contract. Then, we will build a front end to connect to our smart contract and mint an NFT into the user’s account. The functionality we build will allow users to connect their Flow account, create an account if they don’t already have one, then select from one of three images to mint into an NFT. Then, the dApp will display the NFTs from our collection that is in the user’s account. It will be an excellent project to highlight how easy and efficient creating NFTs is on Flow and how effective the Flow Client Library (FCL) is for interacting with the blockchain. To follow along with this tutorial, you’ll need the following things: Node.js and npm The Flow Command Line Interface (Flow CLI) Your favorite IDE With all of these installed, let’s get started! 1. Set Up Flow Account Before we start building, we’ll need to set up an account on the Flow blockchain so we can deploy our smart contract. Run the following command to generate a new public and private key pair: flow keys generate Be sure to write down the values your console outputs, as we’ll need them in the following steps. Next, we’ll head over to the Flow Faucet to create a new address based on our keys and fund our account with some test tokens. Complete the following steps to create your account: Paste in your public key in the specified input field Keep the Signature and Hash Algorithms set to default Complete the captcha Click on Create Account With a successful account generation, we get a dialogue with our new Flow address containing 1,000 FLOW tokens. Copy the address for use in the next step. 2. Set Up The Smart Contract Before we build the project frontend, let’s create the smart contract we will interact with later. In the command terminal, navigate to the folder you would like to work from and type the following command to initiate a project: flow init This command creates a flow.json file inside the folder, where we’ll place all the information we need to deploy our smart contract. Open the flow.json file in your code editor and we’ll set up a testnet account. Inside the accounts section, we’ll add a new entry called testnet-account, which contains our new address and the private key generated in the flow keys generate command earlier. { "emulators": { "default": { "port": 3569, "serviceAccount": "emulator-account" } }, "contracts": {}, "networks": { "emulator": "127.0.0.1:3569", "mainnet": "access.mainnet.nodes.onflow.org:9000", "testnet": "access.devnet.nodes.onflow.org:9000" }, "accounts": { "emulator-account": { "address": "f8d6e0586b0a20c7", "key": "2becfbede2fb89796ab68df3ec2a23c3627235ec250a3e5da41df850a8dd4349" }, "testnet-account": { "address": "0x8e0dac5df6e8489e", "key": "c91f4716a51a66683ccb090ca3eb3e213b90e9f9ae2b1edd12defffe06c57edc" } }, "deployments": {} } Next, we’ll create a new file to write our smart contract. When writing out the code, you may notice some differences in how Cadence handles NFT creation compared to Solidity. For example, NFTs in Cadence are created as a resource and minted directly into the account of the user. In contrast, Solidity NFTs are essentially just an ID number referenced in a mapping to a specific address on the digital ledger. So with that in mind, in the same folder as the flow.json file, create a new file called FlowTutorialMint.cdc and type the following code: /* * * This is an example implementation of a Flow Non-Fungible Token. * This contract does not implement any sophisticated classification * system for its NFTs. It defines a simple NFT with minimal metadata. * */ import NonFungibleToken from 0x631e88ae7f1d7c20 import MetadataViews from 0x631e88ae7f1d7c20 pub contract FlowTutorialMint: NonFungibleToken { pub var totalSupply: UInt64 pub event ContractInitialized() pub event Withdraw(id: UInt64, from: Address?) pub event Deposit(id: UInt64, to: Address?) pub let CollectionStoragePath: StoragePath pub let CollectionPublicPath: PublicPath pub let MinterStoragePath: StoragePath pub struct FlowTutorialMintData{ pub let id: UInt64 pub let type: String pub let url: String init(_id: UInt64, _type: String, _url: String){ self.id = _id self.type = _type self.url = _url } } pub resource NFT: NonFungibleToken.INFT, MetadataViews.Resolver { pub let id: UInt64 pub let type: String pub let url: String init( id: UInt64, type: String, url: String, ) { self.id = id self.type = type self.url = url } pub fun getViews(): [Type] { return [ Type<FlowTutorialMintData>() ] } pub fun resolveView(_ view: Type): AnyStruct? { switch view { case Type<FlowTutorialMintData>(): return FlowTutorialMintData( _id: self.id, _type: self.type, _url: self.url ) } return nil } } pub resource interface FlowTutorialMintCollectionPublic { pub fun deposit(token: @NonFungibleToken.NFT) pub fun getIDs(): [UInt64] pub fun borrowNFT(id: UInt64): &NonFungibleToken.NFT pub fun borrowFlowTutorialMint(id: UInt64): &FlowTutorialMint.NFT? { post { (result == nil) || (result?.id == id): "Cannot borrow FlowTutorialMint reference: the ID of the returned reference is incorrect" } } } pub resource Collection: FlowTutorialMintCollectionPublic, NonFungibleToken.Provider, NonFungibleToken.Receiver, NonFungibleToken.CollectionPublic, MetadataViews.ResolverCollection { // dictionary of NFT conforming tokens // NFT is a resource type with an `UInt64` ID field pub var ownedNFTs: @{UInt64: NonFungibleToken.NFT} init () { self.ownedNFTs <- {} } // withdraw removes an NFT from the collection and moves it to the caller pub fun withdraw(withdrawID: UInt64): @NonFungibleToken.NFT { let token <- self.ownedNFTs.remove(key: withdrawID) ?? panic("missing NFT") emit Withdraw(id: token.id, from: self.owner?.address) return <-token } // deposit takes an NFT and adds it to the collections dictionary // and adds the ID to the id array pub fun deposit(token: @NonFungibleToken.NFT) { let token <- token as! @FlowTutorialMint.NFT let id: UInt64 = token.id // add the new token to the dictionary which removes the old one let oldToken <- self.ownedNFTs[id] <- token emit Deposit(id: id, to: self.owner?.address) destroy oldToken } // getIDs returns an array of the IDs that are in the collection pub fun getIDs(): [UInt64] { return self.ownedNFTs.keys } // borrowNFT gets a reference to an NFT in the collection // so that the caller can read its metadata and call its methods pub fun borrowNFT(id: UInt64): &NonFungibleToken.NFT { return (&self.ownedNFTs[id] as &NonFungibleToken.NFT?)! } pub fun borrowFlowTutorialMint(id: UInt64): &FlowTutorialMint.NFT? { if self.ownedNFTs[id] != nil { // Create an authorized reference to allow downcasting let ref = (&self.ownedNFTs[id] as auth &NonFungibleToken.NFT?)! return ref as! &FlowTutorialMint.NFT } return nil } pub fun borrowViewResolver(id: UInt64): &AnyResource{MetadataViews.Resolver} { let nft = (&self.ownedNFTs[id] as auth &NonFungibleToken.NFT?)! let flowTutorialMintNFT = nft as! &FlowTutorialMint.NFT return flowTutorialMintNFT as &AnyResource{MetadataViews.Resolver} } destroy() { destroy self.ownedNFTs } } // public function that anyone can call to create a new empty collection pub fun createEmptyCollection(): @NonFungibleToken.Collection { return <- create Collection() } pub fun mintNFT( recipient: &{NonFungibleToken.CollectionPublic}, type: String, url: String, ) { // create a new NFT var newNFT <- create NFT( id: FlowTutorialMint.totalSupply, type: type, url: url ) // deposit it in the recipient's account using their reference recipient.deposit(token: <-newNFT) FlowTutorialMint.totalSupply = FlowTutorialMint.totalSupply + UInt64(1) } init() { // Initialize the total supply self.totalSupply = 0 // Set the named paths self.CollectionStoragePath = /storage/flowTutorialMintCollection self.CollectionPublicPath = /public/flowTutorialMintCollection self.MinterStoragePath = /storage/flowTutorialMintMinter // Create a Collection resource and save it to storage let collection <- create Collection() self.account.save(<-collection, to: self.CollectionStoragePath) // create a public capability for the collection self.account.link<&FlowTutorialMint.Collection{NonFungibleToken.CollectionPublic, FlowTutorialMint.FlowTutorialMintCollectionPublic, MetadataViews.ResolverCollection}>( self.CollectionPublicPath, target: self.CollectionStoragePath ) emit ContractInitialized() } } Important things to note in the smart contract above: We are importing the NonFungibleToken and MetadataViews contracts to create our NFTs using Flow standards. We define our NFT resource in the pub resource NFT function. The mintNFT function mints an NFT into the account that calls the function. Now we need to go back into our flow.json file to add a few things: In the contracts section, add the contract and its path. In the deployments section add the network (testnet), the account that we will use to perform the deployment (testnet-account), and the contract name (FlowTutorialMint). { "emulators": { "default": { "port": 3569, "serviceAccount": "emulator-account" } }, "contracts": { "FlowTutorialMint": "./FlowTutorialMint.cdc" }, "networks": { "emulator": "127.0.0.1:3569", "mainnet": "access.mainnet.nodes.onflow.org:9000", "testnet": "access.devnet.nodes.onflow.org:9000" }, "accounts": { "emulator-account": { "address": "f8d6e0586b0a20c7", "key": "2becfbede2fb89796ab68df3ec2a23c3627235ec250a3e5da41df850a8dd4349" }, "testnet-account": { "address": "0x8e0dac5df6e8489e", "key": "c91f4716a51a66683ccb090ca3eb3e213b90e9f9ae2b1edd12defffe06c57edc" } }, "deployments": { "testnet": { "testnet-account": [ "FlowTutorialMint" ] } } } The final step in setting up the smart contract is to deploy it to the testnet. To do that, type the following command in the project folder in your terminal: flow project deploy -n=testnet We should receive an output stating the contract was deployed successfully: It’s important to note here that Cadence smart contracts exist in the storage of the account that deploys them, whereas, with Solidity, the smart contract exists at its own address on the blockchain. Although there are limits to the account’s storage capacity, these are relative to the amount of FLOW tokens reserved in the account. You can learn more about account storage in the Flow Developer Portal. Awesome! Now let’s build a simple frontend to interact with our contract. 3. Creating The Frontend For the frontend of this project, we will be using React. First, navigate to a new folder and run the following command to create a React project: npx create-react-app flow-tutorial Next, navigate into the flow-tutorial folder and install the Flow Client Library (FCL): npm i -S @onflow/fcl The FCL will allow us to communicate with the Flow blockchain, call transactions, and integrate all other FCL-compatible wallets without needing to add custom integrations. Once that finishes, we will install a few additional dependencies: npm i elliptic sha3 styled-components After installing all our dependencies, we are ready to start working on the dApp frontend. 3.1 Configure the FCL Before we begin structuring and styling things, let’s create an FCL configuration file where we’ll define important settings, such as whether we will interact with testnet or mainnet. In the src directory, create a new folder named flow. Within this new folder, create a file called config.js. In this config.js file, we will import the FCL, call the fcl.config function and create some settings for our dApp, such as: app.detail.title accessNode.api discovery.wallet Open the config.js file and fill it with the following code: const fcl = require("@onflow/fcl"); fcl.config({ "app.detail.title": "Flow Mint Page Tutorial", // this adds a custom name to our wallet "accessNode.api": "https://rest-testnet.onflow.org", // this is for the local emulator "discovery.wallet": "https://fcl-discovery.onflow.org/testnet/authn", // this is for the local dev wallet }) There are additional settings we can configure for our dApp, but for now, this is all we will need. With the configuration out of the way, let’s move on to building! 3.2 The Initial Structure First, navigate to the App.js file in the src folder and replace the code with this: import './App.css'; function App() { return ( <div className="App"> <h1>Mint Your Dog!</h1> </div> ); } export default App; This will give us the initial structure of our dApp, from which we will expand upon. Next, we’ll style this structure. Open the index.css file and replace the code with the following: @import url('https://fonts.googleapis.com/css2?family=Michroma&family=Montserrat:wght@200;300;600;700&display=swap'); body { margin: 0; font-family: 'Montserrat', -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', sans-serif; -webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; } code { font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New', monospace; } If you run npm start, you will see a blank page with the title Mint Your Dog! Next, let’s create some components! 3.3 The Nav Component Inside the src directory, create a new folder called components, where we will build all of our custom React components. The first component we will create is the Navbar, which will show the Login button if the user isn’t connected, or the Logout button next to the user’s address and the number of FLOW tokens the account has if they are connected. Create a file called Navbar.jsx and fill it with the following code: import * as fcl from "@onflow/fcl"; import styled from "styled-components"; import { useState, useEffect } from "react"; import "../flow/config"; const Wrapper = styled.nav` width: -webkit-fill-available; background-color: #8dfe89; position: fixed; top: 0; display: flex; justify-content: space-between; align-items: center; padding: 10px 50px; button { background-color: white; padding: 5px 40px; max-width: 200px; border: none; border-radius: 20px; font-size: 18px; height: 50px; &:hover { color: white; background-color: black; cursor: pointer; } } div { display: flex; gap: 15px; } box { display: flex; flex-direction: column; gap: 10px; } `; function Navbar() { const [user, setUser] = useState({ loggedIn: false, addr: undefined }); const [flow, setFlow] = useState(0); useEffect(() => { fcl.currentUser.subscribe(setUser); if (user.addr !== "") getFlow(user.addr); }, [user.addr]); const logOut = async () => { await fcl.unauthenticate(); setUser({ addr: undefined, loggedIn: false }); }; const logIn = async () => { await fcl.authenticate(); }; async function getFlow(address) { try { const res = await fcl.query({ cadence: ` import FlowToken from 0x7e60df042a9c0868 import FungibleToken from 0x9a0766d93b6608b7 pub fun main(address: Address): UFix64{ let balanceVault = getAccount(address).getCapability(/public/flowTokenBalance).borrow<&FlowToken.Vault{FungibleToken.Balance}>()! return balanceVault.balance }`, args: (arg, t) => [arg(address, t.Address)], }); setFlow(res); } catch (error) { console.log("err:", error); } } return ( <Wrapper> <h1>Flow Tutorial Mint</h1> {user.loggedIn ? ( <div> <button onClick={() => logOut()}>Logout</button> <box> <span>Address - {user.addr}</span> <span>Flow Balance - {flow}</span> </box> </div> ) : ( <button onClick={() => logIn()}>Login</button> )} </Wrapper> ); } export default Navbar; Let’s walk through the code to see what’s going on here. First, we are importing the Flow Client Library, which will provide us with functions to authenticate, unauthenticate, and determine the currentUser. Next, we import the other dependencies we need and then use styled-components to create the basic styling of our Navbar inside the Wrapper variable. Then, we define some React state variables (user and flow). Next is the functionality of the dApp, such as logOut, logIn, and getFlow (get the connected account’s FLOW balance). After that, we return the html for the Navbar wrapped in our styling. With a complete Navbar component, we can now import it into the App.js file: import './App.css'; import Navbar from './components/Navbar.jsx'; function App() { return ( <div className="App"> <Navbar /> <h1>Mint your Dog!</h1> </div> ); } export default App; Now, if we run the project with npm start, we see our Navbar gives us the functionality we defined in our code. Awesome! Next, let’s build our NFT minting component! 3.4 The NFT Minting Component Inside the components folder, create a new file called MintComponent.jsx, then copy the following code: import styled from "styled-components"; import * as fcl from "@onflow/fcl"; const Wrapper = styled.div` display: flex; flex-direction: column; gap: 10px; align-items: center; justify-content: center; margin-top: 80px; padding: 100px; main{ display: flex; } div{ width: 300px; display: flex; flex-direction: column; align-items: center; justify-content: center; gap: 5px; } button{ width: 100px; padding: 10px; border: none; background-color: #8dfe89; border-radius: 20px; font-weight: 500; &:hover { color: white; background-color: black; cursor: pointer; } } img{ width: 200px; } `; function MintComponent() { async function mintNFT(type, url) { try { const res = await fcl.mutate({ cadence: ` import FlowTutorialMint from 0x8e0dac5df6e8489e import NonFungibleToken from 0x631e88ae7f1d7c20 import MetadataViews from 0x631e88ae7f1d7c20 transaction(type: String, url: String){ let recipientCollection: &FlowTutorialMint.Collection{NonFungibleToken.CollectionPublic} prepare(signer: AuthAccount){ if signer.borrow<&FlowTutorialMint.Collection>(from: FlowTutorialMint.CollectionStoragePath) == nil { signer.save(<- FlowTutorialMint.createEmptyCollection(), to: FlowTutorialMint.CollectionStoragePath) signer.link<&FlowTutorialMint.Collection{NonFungibleToken.CollectionPublic, MetadataViews.ResolverCollection}>(FlowTutorialMint.CollectionPublicPath, target: FlowTutorialMint.CollectionStoragePath) } self.recipientCollection = signer.getCapability(FlowTutorialMint.CollectionPublicPath) .borrow<&FlowTutorialMint.Collection{NonFungibleToken.CollectionPublic}>()! } execute{ FlowTutorialMint.mintNFT(recipient: self.recipientCollection, type: type, url: url) } } `, args: (arg, t) => [arg(type, t.String), arg(url, t.String)], limit: 9999, }); fcl.tx(res).subscribe((res) => { if (res.status === 4 && res.errorMessage === "") { window.alert("NFT Minted!") window.location.reload(false); } }); console.log("txid", res); } catch (error) { console.log("err", error); } } return ( <Wrapper> <h1>Mint your Dog!</h1> <main> <div> <img src="https://images.unsplash.com/photo-1517849845537-4d257902454a" alt="Mad Dog"/> <h3>Mad Dog</h3> <button onClick={() => mintNFT("Mad Dog", "https://images.unsplash.com/photo-1517849845537-4d257902454a")}>Mint</button> </div> <div> <img src="https://images.unsplash.com/photo-1517423568366-8b83523034fd" alt="Swag Dog"/> <h3>Swag Dog</h3> <button onClick={() => mintNFT("Swag Dog", "https://images.unsplash.com/photo-1517423568366-8b83523034fd")}>Mint</button> </div> <div> <img src="https://images.unsplash.com/photo-1517519014922-8fc06b814a0e" alt="French Dog"/> <h3>French Dog</h3> <button onClick={() => mintNFT("French Dog", "https://images.unsplash.com/photo-1517519014922-8fc06b814a0e")}>Mint</button> </div> </main> </Wrapper> ) } export default MintComponent; Again, let’s walk through the code to ensure we understand what’s going on. We need to import the FCL in this component to gain access to the function that will let us mint our NFT. Again, we use styled-components to add some styling. The mintNFT function uses the fcl.mutate function to perform the actual mint by: Validating whether the user has a Flow Tutorial Mint NFT collection in their account and creating one if not. Calling the existing mint function inside the FlowTutorialMint contract and passing the parameters. The function returns the resource (NFT), which we deposit into the user’s account. In the fcl.mutate function, we are importing the smart contract we deployed with the line: import FlowTutorialMint from 0x8e0dac5df6e8489e We also import the NonFngibleToken and MetadataViews standards. In the transaction, we specify the NFT type and url of the image. Cadence transactions have two phases: prepare and execute prepare – we ask for the user’s signature to access their account and perform private functions. In this case, creating a new FlowTutorial Mint collection if they don’t already have one. We also initialize a public Capability restricted to NonFungibleToken.CollectionPublic. For more context on Capabilities, check out this link. execute – call the mintNFT function inside of our contract on the testnet. In the html portion of the code, we display three images from which the user can mint an NFT. With our MintComponent complete, we can import it into the App.js file: import './App.css'; import Navbar from './components/Navbar.jsx'; import MintComponent from './components/MintComponent.jsx'; function App() { return ( <div className="App"> <Navbar /> <h1>Mint your Dog!</h1> <MintComponent /> </div> ); } export default App; Now the user can log into the dApp and mint an NFT to their account! The final piece of the puzzle is to create a component that will fetch the user’s NFTs and display them. 3.5 Showing the User’s NFTs In the components folder, create a new file called ShowNfts.jsx, and we will use the following code: import * as fcl from "@onflow/fcl"; import { useState, useEffect } from "react"; import styled from "styled-components"; const Wrapper = styled.div` background-color: #e5e5e5; display: flex; flex-direction: column; gap: 10px; align-items: center; justify-content: center; padding: 50px; button { width: 100px; padding: 10px; border: none; background-color: #8dfe89; border-radius: 10px; font-weight: 700; &:hover { color: white; background-color: black; cursor: pointer; } } section { display: flex; justify-content: center; align-items: center; flex-wrap: wrap; gap: 30px; padding: 10%; } .nftDiv{ padding: 10px; background-color: #141414; border-radius: 20px; color: white; box-shadow: 4px 4px 8px rgba(0, 0, 0, 0.25); img{ width: 140px; border-radius: 10px; } p{ font-size: 14px; } } `; export default function ShowNfts() { const [nfts, setNfts] = useState([]); const [user, setUser] = useState({ loggedIn: false, addr: undefined }); useEffect(() => { fcl.currentUser.subscribe(setUser); getNFTs(user.addr) }, [user.addr]); async function getNFTs(addr) { try { const result = await fcl.query({ cadence: ` import FlowTutorialMint from 0x8e0dac5df6e8489e import MetadataViews from 0x631e88ae7f1d7c20 pub fun main(address: Address): [FlowTutorialMint.FlowTutorialMintData] { let collection = getAccount(address).getCapability(FlowTutorialMint.CollectionPublicPath) .borrow<&{MetadataViews.ResolverCollection}>() ?? panic("Could not borrow a reference to the nft collection") let ids = collection.getIDs() let answer: [FlowTutorialMint.FlowTutorialMintData] = [] for id in ids { let nft = collection.borrowViewResolver(id: id) let view = nft.resolveView(Type<FlowTutorialMint.FlowTutorialMintData>())! let display = view as! FlowTutorialMint.FlowTutorialMintData answer.append(display) } return answer } `, args: (arg, t) => [arg(addr, t.Address)], }); setNfts(result); } catch (error) { console.log("err", error); } } return ( <Wrapper> <h1>My NFTs</h1> <main> <button onClick={() => getNFTs(user.addr)}>Get NFTs</button> <section> {nfts.map((nft, index) => { return ( <div key={index} className="nftDiv"> <img src={nft.url} alt="nft" /> <p>Type: {nft.type}</p> <p>Id: {nft.id}</p> </div> ); })} </section> </main> </Wrapper> ); } Essentially what we are doing in this code is querying the Flow Blockchain using the FCL, and gathering the NFTs in the connected account that are from our FlowTutorialMint collection. We just need to add this component to our App.js, and we are good to go! import './App.css'; import Navbar from './components/Navbar.jsx'; import MintComponent from './components/MintComponent.jsx'; import ShowNfts from './components/ShowNfts'; function App() { return ( <div className="App"> <Navbar /> <h1>Mint your Dog!</h1> <MintComponent /> <ShowNfts /> </div> ); } export default App; That’s everything! Now let’s test our dApp and make sure we can mint some NFTs. 4. Let’s Mint Some NFTs! So first, let’s start the app with npm start and then open our browser to http://localhost:3000/. If everything goes well, your screen should look like this: The beautiful thing about using the FCL in our Login sequence is that it gives our users easy access to making an account right on the spot using just an email address. Let's walk through the process to make sure it works properly. By clicking the Login button, a dialogue will pop up, giving us two options to login with. We will choose Blocto. Blocto will prompt us to enter an email address and, upon doing so, gives us the ability to Register a new account. Then, once we input the code emailed to our address, Blocto sets us up with a shiny, new Flow address! From here, we can choose which dog image we want to mint as an NFT. I chose the Swag Dog because it reminds me a bit of myself! Pressing the Mint button will pop up another dialogue telling us about the transaction we are about to perform. We can see that Blocto is graciously covering the minting fees, and if we want to look at the script we are calling, we can do so. Several seconds after hitting Approve, we should receive a message that our mint was successful, and our newly minted Swag Dog will display under the My NFTs section of our dApp. Here’s a link to our dApp in action: https://s1.gifyu.com/images/flow_tutorial-min.gif The entire source code for this project can be found in this repository. Conclusion As you can see, building an NFT minting dApp on the Flow Blockchain is straightforward once you understand how it all works together. Additionally, the Flow Client Library is a powerful tool at our disposal that gives us access to extensive built-in functionality and helps give our dApp a better user experience. In contrast to Ethereum, Flow handles NFT creation and management much more efficiently and securely. This is achieved by deploying smart contracts and minting the NFTs directly into the user’s account, rather than creating a reference to addresses or mappings stored on the digital ledger. For more information about building on Flow, check out the Flow Developer Portal. Have a really great day!

Source code is one of the most valuable assets for any company. So if it gets stolen or leaked, this can result in colossal damages to your business. Source code theft can cause financial and reputational losses in the long term perspective. Even the giants in the industry are at risk, as the source code of Windows 2000 was stolen from Microsoft in 2004, and a recent data breach cost Capital One around $300 million. That's why securing your source code and protecting it from theft should be one of your top priorities. Why Is Securing Source Code Important? Recently, attempts to compromise services, software, apps, and devices have risen as the rewards for hackers can be quite profitable. Source code includes a lot of valuable information since it plays a crucial role in creating and building software, application, extensions, web services, and so much more. Unfortunately, source code security doesn't get enough attention in many cases. By stealing your source code, hackers can get ahold of important and sensitive information such as technical specifications, passwords, OAuth tokens, encryption keys, users' personal data, and so on. Without proper security, they can easily copy, modify, or distribute this data. Besides that, hackers can use information from the source code to exploit vulnerabilities and cause data breaches. You could lose a lead in the development of innovational products to your opponents if stolen source code gets sold to competitors. How to Secure Source Code There are several ways to secure and protect your source code from theft. Creating a dedicated policy, encryption, and securing endpoint devices are just some of the precautions you should implement. Also, it is best to use a layered approach instead of point protection when using technologies to mitigate source code theft. Implement Source Code Security Policy The policy should include rules and requirements to protect source code during the development lifecycle. It should cover the source code development process and all employees involved in the development. Every regulation you want to implement has to be clearly outlined in the policy. This includes encryption and security protocols, application shielding and hardening process, use of repositories, and access controls. You should also add documentation, training, and recommendations on best secure coding practices. Manage Access Control Defining who has access to the source code is fundamental in securing it from theft. Usually, there is no need to give a developer access to the whole code. Global access to source code brings potential security risks, so always avoid giving it unless absolutely necessary. You should use the "least privilege model" rule when managing source code access control. In addition, you can define and isolate source code areas when giving access and use the Zero Trust model when possible. Using multifactor authentication will also prevent access to course codes from unauthorized users. Implement two-factor authentication to ensure that only users with appropriate privileges can obtain company data and source code. Secure Your Endpoint Devices Securing your entry point is a crucial step in protecting source code from theft. Endpoint devices such as laptops and desktops are prone to malware attacks and other malicious activities. You should use Data Loss Prevention tools should be used to prevent source code leaks and exfiltration. In addition, protecting your devices with anti-malware, VPN, and antivirus solutions such as Avast, Clario, and Norton brings an additional level of security. While endpoint security is important, you should also remember to secure all other devices. Any phone or tablet connected to the same network as the entry-point device can bring potential risks to the source code security. If someone accesses these devices, they can get ahold of passwords and other sensitive information to further attack your source code. Use special code to check if your phone is hacked, regularly check it for malware and install the latest updates to keep your device safe. Avoid Insecure Source Code Usage It is easy to miss some security flaws during the development process. Analysis and testing tools are a great way to check source code security. Static Application Security Testing software can scan your code for security flaws, vulnerabilities, violations of coding standards, and other development issues. The tool works in real-time, providing updated information on an early stage of the software development life cycle. You should also use Dynamic Application Security Testing to find vulnerabilities and flaws outside the code. The tool can find issues with third-party interfaces and prevent SQL injection, cross-site scripting, insecure server configuration, and other types of attacks. Protect Source Code With Patents and Copyright Copyright laws and patents can't prevent course code theft, but they can confirm the ownership of the code. Source code can include new concepts and technological inventions, so it should be protected as intellectual property. That is why securing the idea, and written code with patents and copyright is crucial to state your ownership in case of source code theft or legal dispute. Conclusion Securing source code and protecting it from theft is crucial for every business. Applying multiple security layers to your code hardens and bulletproofs your development against attacks. It can take one data breach to bring colossal financial and reputational damages to your business. So creating protection policies, managing access, and securing your endpoint devices are just some of the approaches you should take to protect your source code.

A loosely coupled architecture is a software application development model wherein multiple components are connected with one another but are not heavily dependent on each other. Together, these components create a general network or system, despite each service being an independent entity created to perform a single task. The primary purpose of a loosely coupled architecture is to create a system that doesn’t fail due to the failure of a single component. Service-oriented architectures (SOAs) normally comprise a loosely coupled architecture. Example of a Loosely Coupled Architecture Consider an instance wherein you have created two classes in a program: A and B. When a method of Class A calls the method of Class B or uses variable instances defined in Class B, both classes are tightly coupled. However, when Class A depends on the interface of Class B instead of the methods defined in Class B, both classes are loosely coupled. Above is an example of a food ordering app that uses a loosely coupled system. The app contains different services such as order service, restaurant service, delivery service, and customer service. When a customer orders a food item, the order service takes care of processing it. The restaurant service receives this data and prepares the food while the delivery service handles the delivery part. Customer service assists customers when need be. In this case, each individual service is not heavily dependent on any of the other services. All services communicate via APIs in order to send and receive the required information. Therefore, when one service crashes, it can be instantly replaced without disturbing the other components of the app. Similarly, the order service can be automatically scaled during peak hours or seasons. Loosely Coupled Architecture vs. Tightly Coupled Architecture Loosely coupled vs. tightly coupled architecture is an important point to ponder. In the software development world, coupling refers to the dependency of an object on another. The degree of knowledge an object has about another or the level of dependency of one object on another determines whether the system is loosely coupled or tightly coupled. When the dependency level is low, we say it’s loosely coupled. Basically, this means one object knows what the other objects expose through an API. Similarly, two systems are loosely coupled when changes to one system don’t force a change in the other. Each service in a loosely coupled system is designed with a single purpose or responsibility and can be independently deployed, managed, scaled, and removed. Each service contains an interface (REST API) for communication. Seeing as each service can use a different programming language, platform, technology, and framework, it’s easy to make changes to individual services, perform tests, scale services and remove them without affecting the system. The loosely coupled architecture accommodates extensive APIs and interoperable resource schemas decoupled from the interface. On the other hand, objects in a tightly coupled architecture heavily depend on the other components of the system. Each object must collaborate and coordinate with other objects or components in order to complete a function. In this case, the logic of one class is called by another. This means one object or class requires another object or class to complete a task. Therefore, each class takes on multiple responsibilities, a change made to one class also forces a change in one or more other classes. The Importance of Microservices in a Loosely Coupled Architecture Microservices is an architectural design that facilitates the development of an application as small and independent services that run their own processes and communicate using lightweight protocols and messaging systems, making them platform-agnostic and language-agnostic. Each service is independent, testable, highly maintainable, independently deployable, and implements a business capability. Most importantly, each service is maintained, deployed, created, and owned by a small team. Microservices is a notable variant of a service-oriented architecture that leverages the loosely coupled architecture approach. In fact, loose coupling is the key component of microservices architecture. Microservices architecture is highly important to create a loosely coupled system, as the coupling is inevitable in a software system design. For instance, in an e-commerce portal, the order service should communicate with the customer service or delivery service using APIs. That said, microservices allow you to keep this coupling as minimal as possible. Microservices comprise a set of collaborating services that work together using different coupling methods. Runtime Coupling: The degree to which the availability of one service impacts the availability of another service. Design-time Coupling: The degree to which a change in a service triggers a change in another service. Infrastructure Coupling: The degree to which one service’s resource consumption impacts another service’s resource consumption. Each coupling method brings its own challenges to the table. For instance, runtime coupling reduces availability when a change occurs. As such, you’ll experience a long queue of calls in a microservices architecture. When you implement design-time coupling, a small change in an API requires all the services that interact with that API to be changed. Therefore, all of the teams involved with those services must collaborate and discuss the change. As such, it’s important to keep design-time coupling as loose as possible, especially when different collaborating teams are involved. Microservices make it easy to implement loose coupling in a design-time coupling model while enabling you to efficiently handle coupling challenges using lean development and DevOps CI/CD best practices. Which AWS Services Can Be Used for Loosely Coupled Architectures? Amazon Simple Queuing Service (SQS) and Simple Notification Service (SNS) are powerful mechanisms that can help you build highly scalable and loosely coupled architectures, with coupling at the platform, network, and operation levels. Amazon Simple Queuing Service (SQS): When a client makes an order, the app that receives that order sends it as a message to the SQS. The message is queued and consumed by a function such as AWS Lambda. Once the message is processed, it’s automatically deleted. Amazon Simple Notification Service (SNS): Works as a “publish-subscribe” model wherein the application sends order messages to an SNS topic which is then replicated across various endpoints in order to run processes in parallel. AWS Lambda: Serverless compute engine. AWS Fargate: Container as a Service for serverless computing. AWS AutoScaling: Autoscaling service. Amazon S3: Storage service. Amazon API Gateway: API management service. AWS CloudWatch: Used to manage messages in SQS. Loosely Coupled Architecture Scenarios Unlike tightly coupled architecture which runs thousands of parallel processes or cores that are dependent on one another to perform a calculation, loosely coupled architecture runs smaller jobs in large numbers on a single node with parallelization occurring within the node. As processes are not highly dependent on each other, losing one node doesn’t affect overall functionality. The job is either processed later or deleted. It’s important to consider the following aspects when choosing a loosely coupled architecture. Compute: The underlying EC2 instance type should be determined based on the memory-to-compute ratio of an application. Network: Bandwidth and latency issues are not a concern in a loosely coupled scenario, as processes that run in parallel don’t need to interact much. Storage: Each loosely coupled app comes with a different storage requirement. Therefore, consider the dataset size and data transfer requirements. Deployment: Seeing as loosely coupled apps run on cores installed in different availability zones, they can be efficiently deployed using AWS ParallelCluster or AWS Batch. AWS Serverless Scenario With Loosely Coupled Architecture Serverless computing is a cloud-native software development approach that helps organizations easily and seamlessly build and run applications without the burdens of provisioning and managing servers. Servers are still used but are managed by the cloud vendor. The catch here is that the charge is not based on the number of servers or the bandwidth but rather on usage. It enables organizations to purchase back-end servers on a pay-as-you-go basis and only pay for the services used. Serverless computing comes in two models: 1. Function as a Service: A service that allows you to run code based on events without the complexities of infrastructure management. Applications run in a stateless mode wherein the server and client are loosely coupled and hence are independent, flexible, scalable, and fault-tolerant. Regarding compute engines, AWS Lambda is a popular serverless FaaS tool from AWS. For container-based workloads, AWS Fargate is available. For event triggering, we recommend using Amazon SQS, AWS SNS, and Eventbridge. 2. Backend-as-a-Service: The task of running the server-side logic is outsourced to a cloud vendor. AWS BaaS tools include Amazon Elasticsearch Service for logging, AWS Kinesis for analytics, and AWS Cognito for IAM. There are three types of serverless invocation methods: Synchronous Invocation: Used when a client request requires an immediate response. Asynchronous Invocation: Used when you don’t want the client to wait for a response and instead alert the client through a notification once the process is completed. Steaming Invocation: Used to upstream, process, and downstream data at the same pace. When it comes to serverless computing, loosely coupled architecture with an asynchronous method is a good choice. That said, don’t forget to ensure it is fail-fast and fans out properly. Conclusion Software technologies rapidly change, forcing organizations to adapt to newer technologies and trends continually. When it comes to loosely coupled vs. tightly coupled architecture, information flow and coordination between services are better in a tightly coupled architecture. However, they constrain you regarding flexibility when making changes to your apps on the go. A loosely coupled architecture is the need of the hour. Not only does it allow you to swap or scale components instantly, but it also helps you add new features without affecting the availability and performance of the existing system. With microservices, lean development, and DevOps practices, a loosely coupled architecture allows you to stay in the competition or even ahead.