Link Details

Link 875679 thumbnail
User 448255 avatar

By dotCore
via blog.repustate.com
Published: Nov 10 2012 / 10:28

Our sentiment analysis relies on various models to do feature extraction for the purposes of determining sentiment. These models are stored in the form of Python dictionaries and they’re big – about 40MB each, and there are 14 models in total. Doing the math, that’s 560MB of data that must be available to a process. Now, not all of the data is used for each request; only some of the keys in these dictionaries are used for a particular piece of text that needs to be analyzed.
  • 6
  • 0
  • 888
  • 905

Add your comment


Html tags not supported. Reply is editable for 5 minutes. Use [code lang="java|ruby|sql|css|xml"][/code] to post code snippets.

Apache Hadoop
Written by: Piotr Krewski
Featured Refcardz: Top Refcardz:
  1. Play
  2. Akka
  3. Design Patterns
  4. OO JS
  5. Cont. Delivery
  1. Play
  2. Java Performance
  3. Akka
  4. REST
  5. Java