The survey also suggests that organizations find the software more useful the longer they use it. 65% of the organizations who used Hadoop for more than a year wrote down more than three reasons for using it. New users of Hadoop obviously had less knowledge of its usefulness.
These were the top three reasons mentioned for using Hadoop
- Mining data for improved Business Intelligence
- Reduces the cost of data analysis
- Log Analysis
Here were some of the main challenges respondents listed for using Hadoop:
- Steep learning curve
- Hiring qualified people
- Low availability of good products and tooling
- Not enough information on how to get started
The programming-related challenges included:
- Debugging Hadoop jobs
- Monitoring Hadoop jobs
- Insufficient information about Hadoop
- Availability of useful algorithms
- Writing Hadoop Jobs
Other areas of the survey found that Hadoop is usually introduced from the developers in an organization rather than management.
Based on certain survey questions, LaunchPad projects a 50-60% growth in Hadoop developers for organizations already using Hadoop. They also expect Java to remain the primary language for Hadoop. Usage of streaming and the Pig sub-project should remain constant. Usage of Hive/SQL and Mahout are expected to jump. Since all of the respondents were current Hadoop users, we can't be sure how many organizations try Hadoop and give up.