Building “The House of Data”

this article was originally written by asami novak for the new relic blog .

who knew that working at a media company could be harder than being a rocket scientist? but that’s what colin coleman, senior director of analytics product strategy and data governance at turner broadcasting system , claims—and he spent almost a decade working at the nasa ames research center.

the comparison came up as coleman sat down with mike dauber of battery ventures at the gigaom structure data conference in nyc this week to talk about turner’s experiences transitioning to a big data environment, and the challenges it’s facing to collect and try to make sense of growing amounts of new audience and advertiser-related data.

as coleman explained, the real challenge at tbs—a division of time warner–is trying to find a way to address both its business-to-consumer (b2c) audience and business-to-business (b2b) customers. on the consumer side, turner needs to target an incredibly diverse demographic—from preteens who watch the cartoon network to more mature viewers tuning into cnn or turner classic movies. the company needs to stitch together data from more than 100 branded channels, each running across multiple platforms. meanwhile, the company also has to meet the needs of its advertisers looking to gain insights about turner’s consumer audiences, especially as the company’s engagement models expand to include digital, mobile, and so on.

“the physics of engagement are shifting and we’re trying to solve it all at the same time,” said coleman. the solution, for turner, was to “build the house of data” and for the time being, that means using hadoop for what it calls internally, “hadumping.” but the technical changes at the company also bring the need for an organizational shift. critically, it’s not just the data aspects coming into play, but governance and security issues as well.

although coleman admits turner’s big data transition is still a work in progress, the ultimate goal is to better understand the physics of engagement, and to link consumer engagement patterns with the ad side of the house in real time. “getting the insight—that’s the holy grail,” he said.

you can learn more about turner’s data story in this gigaom article and the video below: