Over a million developers have joined DZone.

Perl 6 Is Remarkably Robust at Handling Big Data Sets

DZone's Guide to

Perl 6 Is Remarkably Robust at Handling Big Data Sets

Perl has undergone a massive overhaul over the last year. Check it out!

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

Perl has undergone a massive overhaul over the last year.

Perl 6 was released last December. It's virtually unrecognizable from previous versions of the 15-year-old programming language.

There are a number of updates Perl programmers should be aware of. Among the most significant is the emphasis on big data. Brian Kelly, The main developer of FullAuto said these applications will be tremendously useful in many verticals.

“Perl has a huge community of avid users that continues to thrive in spite of detractors,” said Brian Kelly, the main developer behind a configuration management tool written with Perl 5. “This community — like communities for every language in computing, is being pulled into the Big Data world, like it or not.”

Tom Radcliffe, the Director of Engineering at ActiveState, concurs that Perl 6 will lead the way with big data analytics. He said the new language will be particularly valuable for his clients in the financial industry.

“Many of our clients in the finance industry are using ActivePerl to pull data from various databases and process it,” said Radcliffe. “It’s being used as ‘big-data lite’ or as a way to load up big-data Hadoop systems.”

There are also more mundane, but equally important applications that rely on Perl and big Country office is a website that connects people with thousands of government offices across the country. This website can now take advantage of the most advanced features Perl 6 has to offer, since the language is now compatible with big data.

The big data capabilities of Perl 6 are even disrupting industries that are rarely associated with technology, such as personal injury attorneys. Bernard Marr recently stated that attorneys throughout the world will need to prepare for these changes.

"So it is surprising that until recently there has been little innovation in the way that the legal profession uses Big Data. But some believe that is all about to change with the arrival of a new breed of data savvy lawyers and legal professionals. The first Big Data tools to be made available to lawyers generally focused on billing, time management, marketing and customer relations functions – in line with their incursions into many other industries. Now, lawyers and those developing tools for the profession are starting to think about how this technology could be applied to the fundamental research and case preparation which is the core of their job."

How Does Perl 6 Handle Big Data?

Perl 6 is capable of handling data queries much more efficiently. The language relies on lightweight arrays, which don’t require much attention from the programmer.

These arrays don’t have properties, which limits their applications. However, they save considerable memory for applications where such properties are not needed. This isn’t really a shortcoming, though, because programmers usually work with data that doesn’t have to exactly match built-in data types.

However, the real benefits of big data are being realized in consumer businesses, such as food service, insurance and B2C marketing. The National Restaurant Association has previously discussed the growing importance of big data for food service businesses, which will play a significant role in the future of restaurant software solutions.

“Maybe you’ve heard the term, but you don’t see how big data could possibly apply to your restaurant. It seems so big and so complex… The data from your POS, marketing, accounting, inventory and scheduling systems is a vein of gold just waiting to be mined. Now your establishment can benefit from the same predictive analytics and business insights previously reserved for the likes of Amazon or Google.”

Perl 6 is Late to the Party, But May Be a New Leader in Big Data

A couple years ago, if you asked data engineers which programming language they relied on, they would claim Python or R. R was preferred by programmers that enjoyed a robust ecosystem and visualized data, while Python programmers enjoyed the short learning curve and general purpose applications.

Perl wouldn’t even be on their list of languages for big data applications until Perl 5 was released. Perl 5 was able to handle large data sets much better than its predecessors. However, the functionality still left a lot to be desired. Programmers needed to work with very resource intensive arrays to process such data, which consumed a lot of memory.

In 2012, Perl developers predicted that they would be one of the pioneers of big data. Here is what Martin Holste said during YAPC::NA 2012:

“Hadoop is overrated. Come see what modern Perl can do with map/reduce on terabytes of data with an extremely simple, maintainable architecture by orchestrating the inserting and querying of data on enormous scales. This talk will deconstruct the Enterprise Log Search and Archive (ELSA) project which is fully written in asynchronous, object-oriented Perl and provides a framework for Big Data analytics in a modular, pluggable architecture with the flexibility and customization that only Perl can provide.”

Despite these promises, Perl was a late adopter of big data. The developers realized they couldn't ignore it any longer. The language was quickly eroding as an industry standard, largely due to its inability to handle large data sets. This shortcoming threatened its very existence, as Conor Myhrvold pointed out a couple years ago.

Here is an excerpt from Myhrvold’s post, discussing why he chose to transition from Perl to Python.

“Why Python, and not Perl? Perhaps an illustrative example of what happened to Perl is my own experience with the language.

In college, I still stuck to the contained environments of Matlab and Mathematica, but my programming perspective changed dramatically in 2012. I realized lacking knowledge of structured computer code outside the "walled garden" of a desktop application prevented me from fully simulating hypotheses about the natural world, let alone analyzing data sets using the web, which was also becoming an increasingly intellectual and financially lucrative skill set.”In 2013, Gregory Piatesky discussed the top languages for analytics, data mining, and data science. He noted that most data mining applications were written in R, SQL and python. The usage of Perl for data mining decline 50% that year.

The launch or Perl 6 was intended to stave off the language’s otherwise inevitable obsolescence. Considering how robust many of the new methods are for handling large data queries, Perl could very well be a leader in the big data revolution in the coming years.

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.

perl 6 ,big data ,software app

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}