Building a Real-Time Device Management System

the number of connected devices in the home is projected to expand dramatically, even as other devices (iot) are also expected to explode in number and variety. two of the constraints on success in this space is effective monitoring and automated device management.

fortunately, a significant step forward in remote monitoring and device management was provided by the broadband forum (link), with the tr-069 standard. now, we aren’t limited to the snmp used for so many years in network ip-focused devices. tr-069 is an application layer http protocol, focused on provisioning and communications between on-premise devices (cpe) and auto-configuration servers. the range of devices serviceable with tr-069 include routers, gateways, set top boxes, voip phones, and others.

this article describes how device management systems are transformed from being reactive and remedial, to a proactive real-time device management system.

challenges and architecture of device management

customer premise device management is challenging with millions of devices, from varied manufacturers, and the mandate for low-cost always-on service. an in-home device that requires extensive service, or requires the end-user to continually reboot the device won’t fly.

tr-069 allows the automated configuration server (acs) to understand the device type, manufacturer, serial number, and upload device logs. tr-069 allows us to understand the online status of the device, with a “heartbeat,” the last configuration change and reboot.  the acs can also request the device to download updated configuration or new firmware. tr-069 is designed to restore a device to operation after it has failed.

unfortunately, customer premise devices are capable of widely ranging uses and configurations, which lead to many failures. routers and residential gateways are good examples, supporting wired and wireless lans with wep or wpa2 encryption, dhcp or static ip assignment, with variable dhcp lease terms, ipv4 and ipv6, with varied routing protocols and bridges. environmental factors also play a large role, with power quality, rf interference, overheating, and simple customer-driven loading with services like bittorrent.

customer experience with most in-home devices is a story of failure, with frequent customer-initiated reboots of routers. to enable the vision of the smart home, or iot more generally, we need to do dramatically better in device management.

enhancing acs services with scenarios and actions

simply put, tr-069 is a feedback-oriented design, as it waits for devices to go offline prior to taking action. this approach generates maximum failures, and ultimately higher costs. what is needed is the addition of proactive pattern recognition and action, detecting patterns that lead to device failure, with proactive steps taken to preserve device uptime.

the causes for devices going offline are many and include software quality (memory leaks), device configuration, device loading, power quality, device overheating, and many other issues.

the model used for proactive event management is one that has been developed over the past decade and is widely used today for real-time event management for fraud detection in banking, real-time offer management for retailers, and wireless service management for carriers worldwide. the design is low-risk, works on premise or on the major public clouds, and is economical. a recent design for managing 3 million end customer devices on a wireless network was implemented on a single 8 core linux host with 64 gb of ram.

in this design, scenarios are defined that recognize sequences or patterns of events that precede a device failure. the system recognizes when the events combine to fulfill a scenario — and corresponding actions are taken. for example, devices may require specific configurations. when a configuration log indicates the customer has enabled unsupported features, the service provider may email the customer recommending a change to the supported configuration.

the architecture of such a system is shown below, with real-time event collection occurring between the acs server and the real time event engine (illustrated here as evam). in step #1, the event engine collects and recognizes events in real time. when a scenario is fulfilled, the resulting actions can go either back to the acs server (step 2), such as requesting the device to download a new configuration file, or in step 3, an action is directed via separate channels such as email, outbound calls, or other customer directed communication.

a critical aspect of this system architecture is surfacing the patterns that lead to failure, that once recognized, can be added to scenarios for action.

conclusions

it’s easy to see that smart home device management will be enhanced with proactive event detection and device management. the design outlined in this article is proven across many industries, is affordable, and helps drive the actions in device management from purely reactive to a mix that is closer to 50:50 mix of proactive-reactive actions.