Over a million developers have joined DZone.

Twitter Builds the Ability to Detect What’s in a Live Stream

In a remarkable step up from image recognition in photography that Google and Facebook have been pioneering, Twitter has now developed the ability to detect what's happening in a live stream.

· Big Data Zone

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

periscope-live-streamsI wrote earlier this year about a fascinating project by a team of Google researchers to automatically derive context from images. Rather than relying on meta information, the project was able to infer knowledge directly from the images themselves via a smart machine learning based approach.

Suffice to say, doing the same thing with video is a similarly challenging proposition, especially when the video is being streamed live. Nevertheless, it’s the challenge that the AI team at Twitter are tackling head on.

Real-Time Tagging

Now, it should be said that this is no small undertaking, and the team has had to build their own supercomputer to handle the heavy graphics processing that’s required to do this sort of thing. Their machine is particularly adept at deep learning, which is fundamental to the approach they’re taking.

It also requires a significant shift in how videos are usually tagged, which involves something known as collaborative filtering to gauge the interest in a video and how similar people may like similar videos.

This broadly works for recorded clips, but is much less effective at categorizing live streams. The Cortex researchers have attempted to improve this recommendation system to better filter clips based upon our previous viewing habits.

They’ve utilized deep learning to train its algorithm to understand inputs from a vast range of examples that have been fed the system by a paid network of helpers that have tagged up videos with a rich range of keywords.

Suffice to say, the technology is not yet available in any of Twitter’s live products, but it’s apparently been tested significantly behind the scenes on platforms such as Periscope.

Given the fledgling state of the project, things are in a very early stage right now, and it isn’t clear quite how Twitter might utilize the technology, but it’s certainly possible that it might be used to better offer relevant adverts on live streams.

It’s certainly a fascinating approach and it will be interesting to see just where Twitter take it going forward.

Hortonworks Sandbox is a personal, portable Apache Hadoop® environment that comes with dozens of interactive Hadoop and it's ecosystem tutorials and the most exciting developments from the latest HDP distribution, brought to you in partnership with Hortonworks.

google,twitter,learning,deep learning,processing

Published at DZone with permission of Adi Gaskell, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}