Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Generating Millions of Objects with Random Values [Snippet]

DZone's Guide to

Generating Millions of Objects with Random Values [Snippet]

When it comes to indexing with Solr, Java's Reflection API might be the solution to your problem when you need to generate objects and marshal them.

· Java Zone ·
Free Resource

Secure your Java app or API service quickly and easily with Okta's user authentication and authorization libraries. Developer accounts are free forever. Try Okta Instead.

In my previous article, I mentioned using Apache Flume and Apache Solr together. While working with Solr Cloud, I felt the need to test the Solr indexing with distributed nodes.

So, I created 4 Solr nodes, with 4 shards and 2 replicas each. My aim was very simple: run a Solr index on 100k XML files every 15 minutes. So, to test this, I needed 100,000 XML files every 15 minutes to run my indexing.

The problem was how to generate so many Objects and marshal them. Then a very simple idea struck my mind, which many of you might have already tried at least once in your life.

Yes, I used the Java Reflection API. And it solved my problem in a minute (actually five minutes — it took a little time to debug). So here is the code snippet from the method, 

public static Object generateRandomObjects(Class clazz) throws InstantiationException, IllegalAccessException,
    IllegalArgumentException, InvocationTargetException, ClassNotFoundException {

        Object obj = clazz.newInstance();

        for (Method method: clazz.getMethods()) {
            if (method.getName().startsWith("set")) {
                for (Type subType: method.getGenericParameterTypes()) {
                    Class parameterClass = Class.forName(subType.getTypeName());
                    if (parameterClass.isAssignableFrom(String.class)) {
                        method.setAccessible(true);
                        method.invoke(obj, getRandomValues());
                    } else if (parameterClass.isAssignableFrom(Date.class)) {
                        method.setAccessible(true);
                        method.invoke(obj, new Date());
                    } else {
                        method.invoke(obj, generateRandomObjects(parameterClass));
                    }
                }
            }

        }
        return obj;
    }


Just pass in the Class of which you want to create an object, and this method will return a completely filled object. I know this method can be optimized and also tweaked, but you get the idea, right? 

I am currently working with Solr indexing on large volume sets, and my next article will definitely be about Solr indexing.

Secure your Java app or API service quickly and easily with Okta's user authentication and authorization libraries. Developer accounts are free forever. Try Okta Instead.

Topics:
solr cloud ,reflection api ,java ,indexing

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}