Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Generating Millions of Objects with Random Values [Snippet]

DZone's Guide to

Generating Millions of Objects with Random Values [Snippet]

When it comes to indexing with Solr, Java's Reflection API might be the solution to your problem when you need to generate objects and marshal them.

· Java Zone ·
Free Resource

Get the Edge with a Professional Java IDE. 30-day free trial.

In my previous article, I mentioned using Apache Flume and Apache Solr together. While working with Solr Cloud, I felt the need to test the Solr indexing with distributed nodes.

So, I created 4 Solr nodes, with 4 shards and 2 replicas each. My aim was very simple: run a Solr index on 100k XML files every 15 minutes. So, to test this, I needed 100,000 XML files every 15 minutes to run my indexing.

The problem was how to generate so many Objects and marshal them. Then a very simple idea struck my mind, which many of you might have already tried at least once in your life.

Yes, I used the Java Reflection API. And it solved my problem in a minute (actually five minutes — it took a little time to debug). So here is the code snippet from the method, 

public static Object generateRandomObjects(Class clazz) throws InstantiationException, IllegalAccessException,
    IllegalArgumentException, InvocationTargetException, ClassNotFoundException {

        Object obj = clazz.newInstance();

        for (Method method: clazz.getMethods()) {
            if (method.getName().startsWith("set")) {
                for (Type subType: method.getGenericParameterTypes()) {
                    Class parameterClass = Class.forName(subType.getTypeName());
                    if (parameterClass.isAssignableFrom(String.class)) {
                        method.setAccessible(true);
                        method.invoke(obj, getRandomValues());
                    } else if (parameterClass.isAssignableFrom(Date.class)) {
                        method.setAccessible(true);
                        method.invoke(obj, new Date());
                    } else {
                        method.invoke(obj, generateRandomObjects(parameterClass));
                    }
                }
            }

        }
        return obj;
    }


Just pass in the Class of which you want to create an object, and this method will return a completely filled object. I know this method can be optimized and also tweaked, but you get the idea, right? 

I am currently working with Solr indexing on large volume sets, and my next article will definitely be about Solr indexing.

Get the Java IDE that understands code & makes developing enjoyable. Level up your code with IntelliJ IDEA. Download the free trial.

Topics:
solr cloud ,reflection api ,java ,indexing

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}