DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
The Latest "Software Integration: The Intersection of APIs, Microservices, and Cloud-Based Systems" Trend Report
Get the report
  1. DZone
  2. Data Engineering
  3. AI/ML
  4. GPU-Accelerated IoT Workloads at the Edge

GPU-Accelerated IoT Workloads at the Edge

Bring IoT to the Edge!

Paul DeCarlo user avatar by
Paul DeCarlo
·
Oct. 25, 19 · Presentation
Like (5)
Save
Tweet
Share
15.76K Views

Join the DZone community and get the full member experience.

Join For Free

Bring IoT to the Edge!

Bring IoT to the Edge!

Introduction

In this article, we will look at how to create GPU accelerated IoT Edge workloads targeting the NVIDIA Jetson line of IoT devices. Portions of this content are planned to be delivered during an interactive workshop session as part of Microsoft Ignite 2019.

If you are unable to attend the conference in person, we will give you all of the details that you need to know to run this content on your own, or in a public workshop environment. Let's get started.

You may also like: IoT on the Edge

Background

NVIDIA has recently produced a line of small-form-factor IoT devices under the brand name 'Jetson' that come equipped with onboard GPUs to enable complex video, machine learning, and AI-powered applications. This is a paradigm shift for IoT as it enables the dawning of the Artificial Intelligence of Things, or AIOT, in light and heavy edge scenarios. To accommodate various workloads, these devices come in a range of offerings, including the beefy 512-Core Jetson AGX Xavier, mid-range 256-Core Jetson TX2, and the entry-level $99 128-Core Jetson Nano.

The $99 Nvidia Jetson NanoArtificial Intelligence and Internet of Things applications aren't entirely new; however, these solutions typically rely on off-site processing of AI workloads in a heavy compute environment that then communicates results downstream for eventual use. AIOT changes this by allowing AI workloads to run on-site, potentially on a device in the field that is also producing telemetry in the form of sensor readings or video capture.

When we are able to reduce our reliance on external compute for AI processing by taking advantage of accelerated workloads on-device, we are able to obtain and make use of results immediately, even if we are disconnected from the Internet. This is highly advantageous in scenarios that involve mission-critical computer vision workloads for producing insights in environments where outbound network connectivity isn't reliable or even possible, for example, in offshore vessels or off-network factory floors.

Solution

Microsoft's IoT Edge platform is designed specifically for these types of disconnected and intermittent network environments. The IoT Edge solution accomplishes this through the safe deployment of code to IoT devices using containerized modules for workload processing.

These modules can leverage the wide variety of public containers that you can find on DockerHub or your own custom container images that would typically deploy as cloud applications. You can also run popular Microsoft Azure Services as containerized modules; these include Serverless Functions, Stream Analytics, Machine Learning modules, Custom Vision AI services, and local storage with SQL Server.

In both cases, you can leverage containerization to optimize where you want your workloads to run (cloud or edge), based on your scenario and constraints. These modules can even be designed to take advantage of onboard capabilities that may be available on a given device, including access to GPU hardware.

In the remainder of this article, we will show how we can build out containerized software packages as modules for these devices using Azure IoT Edge. We will leverage dedicated tooling in Visual Studio Code to configure those modules, and then deploy and manage them through Azure IoT Hub. This will culminate in an Intelligent Edge Hands-On-Lab that walks through the process of deploying an IoT Edge module to an Nvidia Jetson Nano device to allow for the detection of objects in YouTube videos, RTSP streams, HoloLens Mixed Reality Capture, or an attached webcam.


Getting Started

This lab requires that you have access to the following materials:

Hardware:

  • Nvidia Jetson Nano Device
  • A cooling fan installed on or pointed at the NVIDIA Jetson Nano device
  • USB Webcam (Optional)
    • Note: The power consumption will require that your device is configured to use a 5V/4A barrel adapter as mentioned here with an Open-CV compatible camera.

Development Environment:

  • Visual Studio Code (VSCode)
  • VSCode Extensions
    • Azure Account Extension
    • Azure IoT Edge Extension
    • Docker Extension
    • Azure IoT Toolkit Extension
  • Git tool(s) Git command line

Workshop Content

Once you have verified that you have the appropriate materials listed above in the "Getting Started" section, you are ready to proceed in following along with the workshop content.

Image title

The workshop goes by the title "Intelligent Edge Hands-On Lab," or IntelligentEdgeHOL, and walks through the process of deploying an IoT Edge module to an Nvidia Jetson Nano device to allow for the detection of objects in YouTube videos, RTSP streams, Hololens Mixed Reality Capture, or an attached webcam. It achieves a performance of around 10 frames per second for most video data.

Image title

The module ships as a fully self-contained Docker image totaling around 5.5GB. This image contains all necessary dependencies including the NVIDIA Linux for Tegra Drivers for Jetson Nano, CUDA Toolkit, NVIDIA CUDA Deep Neural Network library (CUDNN), OpenCV, and Darknet.

Object Detection is accomplished using YOLOv3-tiny with Darknet, which supports detection of the following:

person, bicycle, car, motorbike, aeroplane, bus, train, truck, boat, traffic light, fire hydrant, stop sign, parking meter, bench, bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe, backpack, umbrella, handbag, tie, suitcase, frisbee, skis, snowboard, sports ball, kite, baseball bat, baseball glove, skateboard, surfboard, tennis racket, bottle, wine glass, cup, fork, knife, spoon, bowl, banana, apple, sandwich, orange, broccoli, carrot, hot dog, pizza, donut, cake, chair, sofa, potted plant, bed, dining table, toilet, tv monitor, laptop, mouse, remote, keyboard, cell phone, microwave, oven toaster, sink, refrigerator, book, clock, vase, scissors, teddy bear, hair drier, toothbrush

It works with HoloLens too!

You can follow along with the content to create your very own GPU accelerated IoT Edge workload by going through the project's README. The workshop content is available in its entirety on GitHub at http://aka.ms/IntelligentEdgeHOL.

However, if you would like to follow along using the content the way, it is intended to be delivered in a workshop environment. There is also a video presentation and walkthrough of the content available at http://aka.ms/IntelligentEdgeHOLVideo.


The video content can be followed along on your own, used as a "train the trainer" video for running your own workshop, or used as-is to run a workshop with minimal effort. A link to the presentation deck used in this video can be found at http://aka.ms/IntelligentEdgeHOLDeck.

Conclusion

This article is intended to give interested developers the chance to walk through the workshop content presented in the WRK3031 session at Ignite 2019. If you have ideas to run this content at a future event or interest in running it by yourself, please feel free to reach out to me on Twitter @pjdecarlo for any assistance.

If you are able to successfully reproduce the project and/or have ideas to improve it, I would love to hear about it. Please let us know in the comments how the overall experience was, we look forward to hearing from you!

Happy Hacking!

Further Reading

IoT on the Edge

Why Edge Computing Is Transforming IoT 

IoT azure AI Visual Studio Code Machine learning

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • How To Handle Secrets in Docker
  • The Path From APIs to Containers
  • Fargate vs. Lambda: The Battle of the Future
  • Cloud Performance Engineering

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: