/ Big Data Zone
Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

VTD XML Parser

DZone's Guide to

VTD XML Parser

In this article, the author is going to talk about VTD parser which used XPath to navigate through XML documents.

by
·
Aug. 26, 16 · Big Data Zone ·
Free Resource

Comment (2)

Save
Tweet
{{ articles[0].views | formatCount}} Views

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

As we know that XML parsing is a very important for most of the software development, I am going to talk about VTD parser which used XPath to navigate through XML documents. This is very powerful tool and it's getting used widely nowadays.

Let us see the working example below:

Jar required : vtd-xml-2.11.jar

 College.xml :-
<?xml version = '1.0' encoding = 'Cp1252'?>
<college>
      <staff id="101" dep_name="Admin">
         <employee id="101-01" name="ashish"/>
         <employee id="101-02" name="amit"/>
         <employee id="101-03" name="nupur"/>
         <salary id="101-sal">
               <basic>20000</basic>
               <special-allowance>50000</special-allowance>
               <medical>10000</medical>
               <provident-fund>10000</provident-fund>      
         </salary>
      </staff>
      <staff id="102" dep_name="HR">
         <employee id="102-01" name="shikhar"/>
         <employee id="102-02" name="sanjay"/>
         <employee id="102-03" name="ani"/>
         <salary id="102-sal">
               <basic>25000</basic>
               <special-allowance>60000</special-allowance>
               <medical>10000</medical>
               <provident-fund>12000</provident-fund>      
         </salary> 
      </staff>
      <staff id="103" dep_name="IT">
         <employee id="103-01" name="suman"/>
         <employee id="103-02" name="salil"/>
         <employee id="103-03" name="amar"/>
         <salary id="103-sal">
               <basic>35000</basic>
               <special-allowance>70000</special-allowance>
               <medical>12000</medical>
               <provident-fund>15000</provident-fund>      
         </salary>
      </staff>
</college>


 VTDXmlTest.java:-
import com.ximpleware.AutoPilot;
import com.ximpleware.NavException;
import com.ximpleware.VTDGen;
import com.ximpleware.VTDNav;
import com.ximpleware.XPathEvalException;
import com.ximpleware.XPathParseException;

public class VTDXmlTest {

      public static void main(String[] args) throws XPathParseException, XPathEvalException, NavException {
            parseXmlUisngVtd("college.xml");
      }

      public static void parseXmlUisngVtd(String xmlFilePath) throws XPathParseException, XPathEvalException, NavException{

            final VTDGen vg = new VTDGen();
            vg.parseFile(xmlFilePath, false);
            final VTDNav vn = vg.getNav();
            final AutoPilot ap = new AutoPilot(vn);

            ap.selectXPath("/college/staff");

            while (ap.evalXPath() != -1) {

                  toNormalizedStringAttr("id", vn);
                  toNormalizedStringAttr("dep_name", vn);

                  if(vn.toElement(VTDNav.FIRST_CHILD,"employee")){
                        do{
                              toNormalizedStringAttr("id", vn);
                              toNormalizedStringAttr("name", vn);
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"employee"));
                  }

                  if(vn.toElement(VTDNav.FIRST_CHILD,"salary")){
                        do{
                              toNormalizedStringAttr("id", vn);

                              toNormalizedStringText("basic", vn);
                              toNormalizedStringText("special-allowance", vn);
                              toNormalizedStringText("medical", vn);
                              toNormalizedStringText("provident-fund", vn);
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"salary"));
                  }
                  System.out.println("========================================================================");
                  vn.toElement(VTDNav.PARENT);
            }
      }

      private static String toNormalizedStringAttr(String attrbName, final VTDNav vn) throws NavException{
            return toNormalizedString(attrbName, vn.getAttrVal(attrbName), vn);
      }

      private static String toNormalizedStringText(String tagName, final VTDNav vn) throws NavException{
            return toNormalizedString(tagName, vn.getText(), vn);
      }

      private static String toNormalizedString(String name, int val, final VTDNav vn) throws NavException{

            String strValue = null;
            if(val != -1){
                  strValue = vn.toNormalizedString(val);
                  System.out.println(name + ":: " + strValue);
            }
            return strValue;
      }
}

O/P of the above program:

id:: 101
dep_name:: Admin
id:: 101-01
name:: ashish
id:: 101-02
name:: amit
id:: 101-03
name:: nupur
========================================================================
id:: 102
dep_name:: HR
id:: 102-01
name:: shikhar
id:: 102-02
name:: sanjay
id:: 102-03
name:: ani
========================================================================
id:: 103
dep_name:: IT
id:: 103-01
name:: suman
id:: 103-02
name:: salil
id:: 103-03
name:: amar
========================================================================

Now let is look at the XML traversal using push-pop mechanism of VTD.

XML (same above XML) traversal using push-pop mechanism with VTD-XML and XPATH:

import com.ximpleware.AutoPilot;
import com.ximpleware.NavException;
import com.ximpleware.VTDGen;
import com.ximpleware.VTDNav;
import com.ximpleware.XPathEvalException;
import com.ximpleware.XPathParseException;

/**
 * XML traversal using push-pop mechanism with VTD-XML and XPATH
 *
 */
public class VTDXmlTest {

      public static void main(String[] args) throws XPathParseException, XPathEvalException, NavException {
            parseXmlUisngVtd("college.xml");
      }

      public static void parseXmlUisngVtd(String xmlFilePath) throws XPathParseException,
                  XPathEvalException, NavException{

            final VTDGen vg = new VTDGen();
            vg.parseFile(xmlFilePath, false);
            final VTDNav vn = vg.getNav();
            final AutoPilot ap = new AutoPilot(vn);

            ap.selectXPath("/college/staff");
            int val;

            while(ap.evalXPath() != -1) {

                  vn.push();
                  if(vn.toElement(VTDNav.FIRST_CHILD,"employee")){
                        do{
                              if((val = vn.getAttrVal("id")) != -1){
                                    System.out.println("Employee Id:" + vn.toNormalizedString(val));
                              }
                              if((val = vn.getAttrVal("name")) != -1){
                                    System.out.println("Employee name:" + vn.toNormalizedString(val));
                              }
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"employee"));
                  }
                  vn.pop();

                  if(vn.toElement(VTDNav.FIRST_CHILD,"salary")){
                        do{
                              vn.push();
                              if((val = vn.getAttrVal("id")) != -1){
                                    System.out.println("Employee Id:" + vn.toNormalizedString(val));
                              }

                              //XML traversal using push-pop mechanism
                              getXpathValue(vn, ".//basic/text()");
                              getXpathValue(vn, ".//special-allowance/text()");
                              getXpathValue(vn, ".//medical/text()");
                              getXpathValue(vn, ".//provident-fund/text()");
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"salary"));
                  }
                  System.out.println("========================================================================");
                  vn.toElement(VTDNav.PARENT);
            }
      }

      static void getXpathValue(final VTDNav vn, String xpath) throws NavException,
                  XPathParseException, XPathEvalException{

            //push to save the pointer state and traverse anywhere
            vn.push();
            AutoPilot apSal = new AutoPilot(vn);
            apSal.selectXPath(xpath);

            if(apSal.evalXPath() != -1){
                  System.out.println(xpath + " = " + vn.toNormalizedString(vn.getText()));
            }
            //pop to get the saved pointer state and reach the same saved place
            vn.pop();
      }
}

O/P of the above program as below:

Employee Id:101-01
Employee name:ashish
Employee Id:101-02
Employee name:amit
Employee Id:101-03
Employee name:nupur
Employee Id:101-sal
.//basic/text() = 20000
.//special-allowance/text() = 50000
.//medical/text() = 10000
.//provident-fund/text() = 10000
========================================================================
Employee Id:102-01
Employee name:shikhar
Employee Id:102-02
Employee name:sanjay
Employee Id:102-03
Employee name:ani
Employee Id:102-sal
.//basic/text() = 25000
.//special-allowance/text() = 60000
.//medical/text() = 10000
.//provident-fund/text() = 12000
========================================================================
Employee Id:103-01
Employee name:suman
Employee Id:103-02
Employee name:salil
Employee Id:103-03
Employee name:amar
Employee Id:103-sal
.//basic/text() = 35000
.//special-allowance/text() = 70000
.//medical/text() = 12000
.//provident-fund/text() = 15000

Let us see another way of VTD parsing using Namespaces, here, in this case, the XML document will have the namespace.

VTD –XML with Namespaces:

XML file with Namespace:-

<?xml version = '1.0' encoding="UTF-8"?>

<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope">
      <env:body>
            <college>
                  <staff id="101" dep_name="Admin">
                        <employee id="101-01" name="ashish" />
                        <employee id="101-02" name="amit" />
                        <employee id="101-03" name="nupur" />
                        <salary id="101-sal">
                              <basic>20000</basic>
                              <special-allowance>50000</special-allowance>
                              <medical>10000</medical>
                              <provident-fund>10000</provident-fund>
                        </salary>
                  </staff>
                  <staff id="102" dep_name="HR">
                        <employee id="102-01" name="shikhar" />
                        <employee id="102-02" name="sanjay" />
                        <employee id="102-03" name="ani" />
                        <salary id="102-sal">
                              <basic>25000</basic>
                              <special-allowance>60000</special-allowance>
                              <medical>10000</medical>
                              <provident-fund>12000</provident-fund>
                        </salary>
                  </staff>
                  <staff id="103" dep_name="IT">
                        <employee id="103-01" name="suman" />
                        <employee id="103-02" name="salil" />
                        <employee id="103-03" name="amar" />
                        <salary id="103-sal">
                              <basic>35000</basic>
                              <special-allowance>70000</special-allowance>
                              <medical>12000</medical>
                              <provident-fund>15000</provident-fund>
                        </salary>
                  </staff>
            </college>
      </env:body>
</env:Envelope>
import com.ximpleware.AutoPilot;
import com.ximpleware.NavException;
import com.ximpleware.VTDGen;
import com.ximpleware.VTDNav;
import com.ximpleware.XPathEvalException;
import com.ximpleware.XPathParseException;

/**
 * XML traversal using push-pop mechanism with VTD-XML and XPATH and NameSpace
 */
public class VTDXmlTest {

      /** NameSpace **/
      private static final String NAMESPACE_URL = "http://www.w3.org/2003/05/soap-envelope";
      private static final String NAMESPACE = "env";

      public static void main(String[] args) throws XPathParseException, XPathEvalException, NavException {
            parseXmlUisngVtd("college.xml");
      }

      public static void parseXmlUisngVtd(String xmlFilePath) throws XPathParseException,
                  XPathEvalException, NavException{

            final VTDGen vg = new VTDGen();
            vg.parseFile(xmlFilePath, true);
            final VTDNav vn = vg.getNav();
            final AutoPilot ap = new AutoPilot(vn);

            ap.declareXPathNameSpace(NAMESPACE, NAMESPACE_URL);
            ap.selectXPath("/env:Envelope/env:body/college/staff");
            int val;

            while(ap.evalXPath() != -1) {
                  vn.push();
                  if(vn.toElement(VTDNav.FIRST_CHILD,"employee")){
                        do{
                              if((val = vn.getAttrVal("id")) != -1){
                                    System.out.println("Employee Id:" + vn.toNormalizedString(val));
                              }
                              if((val = vn.getAttrVal("name")) != -1){
                                    System.out.println("Employee name:" + vn.toNormalizedString(val));
                              }
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"employee"));
                  }
                  vn.pop();

                  if(vn.toElement(VTDNav.FIRST_CHILD,"salary")){
                        do{
                              vn.push();
                              if((val = vn.getAttrVal("id")) != -1){
                                    System.out.println("Employee Id:" + vn.toNormalizedString(val));
                              }

                              //XML traversal using push-pop mechanism
                              getXpathValue(vn, ".//basic/text()");
                              getXpathValue(vn, ".//special-allowance/text()");
                              getXpathValue(vn, ".//medical/text()");
                              getXpathValue(vn, ".//provident-fund/text()");
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"salary"));
                  }
                  System.out.println("========================================================================");
                  vn.toElement(VTDNav.PARENT);
            }
      }

      static void getXpathValue(final VTDNav vn, String xpath) throws NavException,
                  XPathParseException, XPathEvalException{

            //push to save the pointer state and traverse anywhere
            vn.push();
            AutoPilot ap = new AutoPilot(vn);
            ap.declareXPathNameSpace(NAMESPACE, NAMESPACE_URL);
            ap.selectXPath(xpath);

            if(ap.evalXPath() != -1){
                  System.out.println(xpath + " = " + vn.toNormalizedString(vn.getText()));
            }
            //pop to get the saved pointer state and reach the same saved place
            vn.pop();
      }
}

AutoPilot's XPath-related methods are:    

  1. declareXPathNameSpace(...): This method binds a namespace prefix (used in an XPath expression) to a URL    
  2. selectXPath(...): This method compiles an XPath expression into internal representation.
  3. evalXPath(...): This method moves the cursor to a qualified node in the node set.
  4. evalXPathToBoolean(...), evalXPathToNumber(...), evalXpathToString(...): These three methods evaluates an XPath expression respectively to a boolean, a double and a string.
  5. resetXPath(): Reset the internal state so the XPath can be re-used.    
  6. getExprString(): Call this method to verify the correctness of the compiled expression.In addition, two more exception classes are introduced for XPath:       
  7. XPathParseException: It is thrown when there is a syntax error in the XPath expression.
  8. XPathEvalException: It is thrown when there is an exception condition during XPath evaluation

I hope this will be helpful for understanding the power of VTD parser.

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.

Like This Article? Read More From DZone

Working With XML in Scala
Cast Your Votes in the Open-Source Showdown Contest [Round 1]
Continuous Testing Live: What's Driving the Quality Engineering Evolution? [Podcast]

Free DZone Refcard

Understanding Stream Processing
DOWNLOAD
Topics:
xml parsing

Comment (2)

Save
Tweet
{{ articles[0].views | formatCount}} Views

Opinions expressed by DZone contributors are their own.

Big Data Partner Resources

Apache NiFi for Dummies, Download the Free eBook
Industry Solution Guide for Using Apache NiFi and Attunity Replicate
Data Lake vs. Data Warehouse

Big Data Partner Resources

Apache NiFi for Dummies, Download the Free eBook
Industry Solution Guide for Using Apache NiFi and Attunity Replicate
Data Lake vs. Data Warehouse

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.linkDescription }}

{{ parent.urlSource.name }}
by {{ parent.authors[0].realName || parent.author}}
· {{ parent.articleDate | date:'MMM. dd, yyyy' }} {{ parent.linkDate | date:'MMM. dd, yyyy' }}
· {{ parent.portal.name }} Zone