Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Twitter Builds the Ability to Detect What’s in a Live Stream

DZone's Guide to

Twitter Builds the Ability to Detect What’s in a Live Stream

In a remarkable step up from image recognition in photography that Google and Facebook have been pioneering, Twitter has now developed the ability to detect what's happening in a live stream.

· Big Data Zone
Free Resource

Effortlessly power IoT, predictive analytics, and machine learning applications with an elastic, resilient data infrastructure. Learn how with Mesosphere DC/OS.

periscope-live-streamsI wrote earlier this year about a fascinating project by a team of Google researchers to automatically derive context from images. Rather than relying on meta information, the project was able to infer knowledge directly from the images themselves via a smart machine learning based approach.

Suffice to say, doing the same thing with video is a similarly challenging proposition, especially when the video is being streamed live. Nevertheless, it’s the challenge that the AI team at Twitter are tackling head on.

Real-Time Tagging

Now, it should be said that this is no small undertaking, and the team has had to build their own supercomputer to handle the heavy graphics processing that’s required to do this sort of thing. Their machine is particularly adept at deep learning, which is fundamental to the approach they’re taking.

It also requires a significant shift in how videos are usually tagged, which involves something known as collaborative filtering to gauge the interest in a video and how similar people may like similar videos.

This broadly works for recorded clips, but is much less effective at categorizing live streams. The Cortex researchers have attempted to improve this recommendation system to better filter clips based upon our previous viewing habits.

They’ve utilized deep learning to train its algorithm to understand inputs from a vast range of examples that have been fed the system by a paid network of helpers that have tagged up videos with a rich range of keywords.

Suffice to say, the technology is not yet available in any of Twitter’s live products, but it’s apparently been tested significantly behind the scenes on platforms such as Periscope.

Given the fledgling state of the project, things are in a very early stage right now, and it isn’t clear quite how Twitter might utilize the technology, but it’s certainly possible that it might be used to better offer relevant adverts on live streams.

It’s certainly a fascinating approach and it will be interesting to see just where Twitter take it going forward.

Learn to design and build better data-rich applications with this free eBook from O’Reilly. Brought to you by Mesosphere DC/OS.

Topics:
google ,twitter ,learning ,deep learning ,processing

Published at DZone with permission of Adi Gaskell, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}