DZone
Java Zone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
  • Refcardz
  • Trend Reports
  • Webinars
  • Zones
  • |
    • Agile
    • AI
    • Big Data
    • Cloud
    • Database
    • DevOps
    • Integration
    • IoT
    • Java
    • Microservices
    • Open Source
    • Performance
    • Security
    • Web Dev
DZone > Java Zone > Fixing Solr Java Heap OOM with “OmitNorms=true”

Fixing Solr Java Heap OOM with “OmitNorms=true”

Mitch Pronschinske user avatar by
Mitch Pronschinske
·
Oct. 27, 11 · Java Zone · Interview
Like (0)
Save
Tweet
7.01K Views

Join the DZone community and get the full member experience.

Join For Free
I found another post today from the blogger who decided to clone Wikipedia and index it with Solr.  This time he's got a short commentary which can serve as useful advice to you search indexer's  out there.

Solr has been running out of heap memory while trying to add a *small* number of documents to my 11,000,000 document Wikipedia index. So, diving bravely into the world of Java heap memory …

Because I am indexing diverse types of data (Wikipedia and Nutch to begin with), I have a lot of fields: I count 31 without omitNorms values, which is false by default.

11,000,000 * 1 * 31 = 31 x 10M = 310MB RAM all by itself. So time to fire up the schema editor!  -- Fred Zimmerman


I would also check out Fred's posts on Indexing Nutch Crawls and DataImportHandler Commands.

Source:  http://business.zimzaz.com/wordpress/2011/10/fixing-solr-heap-oom-with-omitnormstrue/
Java (programming language)

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Selenium vs. Protractor: What's the Difference?
  • Creating a Spring Boot Project With Eclipse and Maven
  • Data Lakes, Warehouses and Lakehouses. Which is Best?
  • 30 Common CI/CD Interview Questions (With Answers)

Comments

Java Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • MVB Program
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends:

DZone.com is powered by 

AnswerHub logo