{{announcement.body}}
{{announcement.title}}

Content-Based Image Classification and Moderation With AWS Rekognition

DZone 's Guide to

Content-Based Image Classification and Moderation With AWS Rekognition

Dive into this tutorial of AWS Rekognition and see how, with some Python and serverless functions, you can set up an image classification application.

· AI Zone ·
Free Resource

First things first! Rekognition is not a typo...

Ever wondered how social networking sites like Facebook, Instagram, and Twitter pulls this off

These are some common ‘sensitive media’ messages that warn the users before they can view these offensive content. These contents include Graphic Violence Or Gore, Physical Violence, Weapons, Self Injury, and more. But how do they know that these images have some offensive content?

One more Question

How do sites like Unsplash, Pixabay, and GettyImages return images based on your search queries?



Thousands of users upload images on these websites but when we searched for a Cat, we got only images related to the cat. How does this happen? Is it magically classifying images according to their contents?

I know you might be thinking that we are halfway through our blog and still hanging with a lot of questions ❔

“Judge a man by his questions rather than by his answers.”
Voltaire

Hey, I am not running away without the answers though… Let’s jump straight to the answers.

These techniques fall under the concept of Deep Learning. With the help of Deep Learning, a system can search for objects in a given image.

Deep learning is a technique that teaches machines to do a task which humans naturally possesses — LEARN FROM EXAMPLES

Deep Learning is the pivotal technology behind Face Unlock feature, Lane detection for driverless cars or to differentiate a residential area from the terrorists’ camp. But the applications of deep learning doesn’t stop here as it has now been used in a lot of fields ranging from Medical Research to the Home assistant devices like Google Home.

Deep Learning Models are trained using tons of labeled images transfused with neural network frameworks.

One of the most common Deep Learning based cloud services available today is AWS Rekognition. It is capable of detecting objects from an image, recognizing you from a group of people, detecting offensive or unsafe content, extracting text from an image, etc.

AWS provides a GUI based console, SDKs, and even a Command Line Interface (CLI) tool.

Without any further delay, let’s start looking at the solution for the above problems — Classification and Moderation.

Explaining the Architecture

  1. User Uploads an image to S3 Bucket.
  2. As soon as the image is uploaded successfully, the lambda function is triggered.
  3. Lambda function sends the image to the Rekognition service.
  4. Rekognition service responds with the labels(object) detected in the image with the confidence percentage.
  5. Classified images are stored in another S3 Bucket.

Put on Your Coding Hats

Python
 




x
51


1
from boto3 import client,resource
2
from botocore.exceptions import ClientError
3
from urllib.parse import unquote_plus
4
import logging
5
 
            
6
logger = logging.getLogger()
7
logger.setLevel(logging.INFO)
8
 
            
9
def lambda_handler(event, context):
10
    image_key=unquote_plus(event['Records'][0]['s3']['object']['key'])
11
    logger.info('Image key is '+image_key)
12
    try:
13
        rekognition = client('rekognition')
14
        s3 = resource('s3')
15
    except ClientError as client_error:
16
        logger.error('AWS Exception')
17
        
18
    source_bucket='all-uploaded-images'
19
    target_bucket='filtered-images-bucket'
20
    
21
    s3_image = Image={
22
        'S3Object': {
23
            'Bucket': source_bucket,
24
            'Name': image_key
25
        }
26
    }
27
    
28
    source_image = {
29
        'Bucket': source_bucket,
30
        'Key': image_key
31
    }
32
    
33
    try:
34
        detected_moderated_labels = rekognition.detect_moderation_labels(Image=s3_image, MinConfidence=80)
35
        detected_labels = rekognition.detect_labels(Image=s3_image, MinConfidence=80)
36
    except rekognition.exceptions.InvalidImageFormatException as invalid_image_format_exception:
37
        logger.error('Not a valid image format detected. Deleting uploaded image.')
38
        s3.Object(source_bucket, image_key).delete()
39
    else:
40
        if not detected_moderated_labels['ModerationLabels']:
41
            logger.info('No unsafe content detected')
42
            for label in detected_labels['Labels']:
43
                logger.info(label['Name'] + ' detected with confidence percentage of ' + str(label['Confidence']))
44
                s3.Object(target_bucket, label['Name'] + '/' + image_key).copy_from(CopySource=source_bucket + '/' + image_key)
45
        else:
46
            logger.info('Unsafe content detected')
47
            s3.Object(target_bucket, 'Unsafe/' + image_key).copy_from(CopySource=source_bucket + '/' + image_key)
48
            
49
    return {
50
        'statusCode': 200
51
    }



The above Lambda function would be triggered when a file is uploaded to source S3 Bucket-'all-uploaded-images'. If the uploaded file is not a valid image file for AWS Rekognition, the file would be deleted else the image file would be processed further to extract the labels.

On the same lines, if the image contains some sensitive labels, it is stored in the target S3 Bucket-'filtered-images-bucket' under the directory named 'Unsafe'. Other images are sorted according to the labels detected in the image and stored in the target S3 Bucket under the directory named by the detected label.

As soon as the files are uploaded in the source bucket, the target bucket is populated accordingly.

A Memory-Efficient Approach

The lambda function seems to be working just fine, but the approach used is not memory efficient as the same image is being copied multiple times. We can think of another technique, in which we can save just the key(name) of the image instead of the whole image file itself. Here's the updated Lambda function.

Python
 




xxxxxxxxxx
1
63


 
1
from boto3 import client,resource
2
from botocore.exceptions import ClientError
3
from urllib.parse import unquote_plus
4
import logging
5
 
            
6
logger = logging.getLogger()
7
logger.setLevel(logging.INFO)
8
 
            
9
def lambda_handler(event, context):
10
    image_key=unquote_plus(event['Records'][0]['s3']['object']['key'])
11
    logger.info('Image key is '+image_key)
12
    try:
13
        rekognition = client('rekognition')
14
        s3 = resource('s3')
15
    except ClientError as client_error:
16
        logger.error('AWS Exception')
17
        
18
    source_bucket='all-uploaded-images'
19
    target_bucket='filtered-images-bucket'
20
    
21
    s3_image = Image={
22
        'S3Object': {
23
            'Bucket': source_bucket,
24
            'Name': image_key
25
        }
26
    }
27
    
28
    source_image = {
29
        'Bucket': source_bucket,
30
        'Key': image_key
31
    }
32
    
33
    bucket = s3.Bucket(target_bucket)
34
    
35
    try:
36
        detected_moderated_labels = rekognition.detect_moderation_labels(Image=s3_image, MinConfidence=80)
37
        detected_labels = rekognition.detect_labels(Image=s3_image, MinConfidence=80)
38
    except rekognition.exceptions.InvalidImageFormatException as invalid_image_format_exception:
39
        logger.error('Not a valid image format detected. Deleting uploaded image.')
40
        s3.Object(source_bucket, image_key).delete()
41
    else:
42
        if not detected_moderated_labels['ModerationLabels']:
43
            logger.info('No unsafe content detected')
44
            for label in detected_labels['Labels']:
45
                logger.info(label['Name'] + ' detected with confidence percentage of ' + str(label['Confidence']))
46
                list_of_keys = list(bucket.objects.filter(Prefix=label['Name']+'.txt'))
47
                if list_of_keys:
48
                    object=s3.Object(target_bucket, label['Name'] + '.txt')
49
                    object.put(Body=object.get()['Body'].read().decode('utf8')+'\n'+image_key)
50
                else:
51
                    s3.Object(target_bucket, label['Name'] + '.txt').put(Body=image_key)
52
        else:
53
            logger.info('Unsafe content detected')
54
            list_of_keys = list(bucket.objects.filter(Prefix='Unsafe.txt'))
55
            if list_of_keys:
56
                object=s3.Object(target_bucket, 'Unsafe.txt')
57
                object.put(Body=object.get()['Body'].read().decode('utf8')+'\n'+image_key)
58
            else:
59
                s3.Object(target_bucket, 'Unsafe.txt').put(Body=str.encode(image_key))
60
            
61
    return {
62
        'statusCode': 200
63
    }



This would save space and would make the process faster than the previous approach. 

Now, we don't have any image files stored in the target bucket. Instead, we just have the text files which store the key(name) of the image files stored in the source bucket.



Closing Thoughts

There can be a number of approaches to these problem statements including saving of keys of the Images in the Databases like AWS RDS or DynamoDB.

You can try this example by just pasting the above function in the AWS Lambda in-line code editor. Just make sure that the bucket names are valid and you have enough permissions for the lambda function to work. Don't forget to add an S3 PUT Trigger in the Lambda function.

That's all for this blog, I will be back with some more Tech blogs. Till then, happy blogging.

Topics:
ai, aws lambda, aws lambda function, aws rekognition, classifcation, moderation, python, tutorial

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}