/ Big Data Zone
Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

VTD XML Parser

DZone's Guide to

VTD XML Parser

In this article, the author is going to talk about VTD parser which used XPath to navigate through XML documents.

by
·
Aug. 26, 16 · Big Data Zone ·
Free Resource

Comment (2)

Save
Tweet
{{ articles[0].views | formatCount}} Views

The open source HPCC Systems platform is a proven, easy to use solution for managing data at scale. Visit our Easy Guide to learn more about this completely free platform, test drive some code in the online Playground, and get started today.

As we know that XML parsing is a very important for most of the software development, I am going to talk about VTD parser which used XPath to navigate through XML documents. This is very powerful tool and it's getting used widely nowadays.

Let us see the working example below:

Jar required : vtd-xml-2.11.jar

 College.xml :-
<?xml version = '1.0' encoding = 'Cp1252'?>
<college>
      <staff id="101" dep_name="Admin">
         <employee id="101-01" name="ashish"/>
         <employee id="101-02" name="amit"/>
         <employee id="101-03" name="nupur"/>
         <salary id="101-sal">
               <basic>20000</basic>
               <special-allowance>50000</special-allowance>
               <medical>10000</medical>
               <provident-fund>10000</provident-fund>      
         </salary>
      </staff>
      <staff id="102" dep_name="HR">
         <employee id="102-01" name="shikhar"/>
         <employee id="102-02" name="sanjay"/>
         <employee id="102-03" name="ani"/>
         <salary id="102-sal">
               <basic>25000</basic>
               <special-allowance>60000</special-allowance>
               <medical>10000</medical>
               <provident-fund>12000</provident-fund>      
         </salary> 
      </staff>
      <staff id="103" dep_name="IT">
         <employee id="103-01" name="suman"/>
         <employee id="103-02" name="salil"/>
         <employee id="103-03" name="amar"/>
         <salary id="103-sal">
               <basic>35000</basic>
               <special-allowance>70000</special-allowance>
               <medical>12000</medical>
               <provident-fund>15000</provident-fund>      
         </salary>
      </staff>
</college>


 VTDXmlTest.java:-
import com.ximpleware.AutoPilot;
import com.ximpleware.NavException;
import com.ximpleware.VTDGen;
import com.ximpleware.VTDNav;
import com.ximpleware.XPathEvalException;
import com.ximpleware.XPathParseException;

public class VTDXmlTest {

      public static void main(String[] args) throws XPathParseException, XPathEvalException, NavException {
            parseXmlUisngVtd("college.xml");
      }

      public static void parseXmlUisngVtd(String xmlFilePath) throws XPathParseException, XPathEvalException, NavException{

            final VTDGen vg = new VTDGen();
            vg.parseFile(xmlFilePath, false);
            final VTDNav vn = vg.getNav();
            final AutoPilot ap = new AutoPilot(vn);

            ap.selectXPath("/college/staff");

            while (ap.evalXPath() != -1) {

                  toNormalizedStringAttr("id", vn);
                  toNormalizedStringAttr("dep_name", vn);

                  if(vn.toElement(VTDNav.FIRST_CHILD,"employee")){
                        do{
                              toNormalizedStringAttr("id", vn);
                              toNormalizedStringAttr("name", vn);
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"employee"));
                  }

                  if(vn.toElement(VTDNav.FIRST_CHILD,"salary")){
                        do{
                              toNormalizedStringAttr("id", vn);

                              toNormalizedStringText("basic", vn);
                              toNormalizedStringText("special-allowance", vn);
                              toNormalizedStringText("medical", vn);
                              toNormalizedStringText("provident-fund", vn);
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"salary"));
                  }
                  System.out.println("========================================================================");
                  vn.toElement(VTDNav.PARENT);
            }
      }

      private static String toNormalizedStringAttr(String attrbName, final VTDNav vn) throws NavException{
            return toNormalizedString(attrbName, vn.getAttrVal(attrbName), vn);
      }

      private static String toNormalizedStringText(String tagName, final VTDNav vn) throws NavException{
            return toNormalizedString(tagName, vn.getText(), vn);
      }

      private static String toNormalizedString(String name, int val, final VTDNav vn) throws NavException{

            String strValue = null;
            if(val != -1){
                  strValue = vn.toNormalizedString(val);
                  System.out.println(name + ":: " + strValue);
            }
            return strValue;
      }
}

O/P of the above program:

id:: 101
dep_name:: Admin
id:: 101-01
name:: ashish
id:: 101-02
name:: amit
id:: 101-03
name:: nupur
========================================================================
id:: 102
dep_name:: HR
id:: 102-01
name:: shikhar
id:: 102-02
name:: sanjay
id:: 102-03
name:: ani
========================================================================
id:: 103
dep_name:: IT
id:: 103-01
name:: suman
id:: 103-02
name:: salil
id:: 103-03
name:: amar
========================================================================

Now let is look at the XML traversal using push-pop mechanism of VTD.

XML (same above XML) traversal using push-pop mechanism with VTD-XML and XPATH:

import com.ximpleware.AutoPilot;
import com.ximpleware.NavException;
import com.ximpleware.VTDGen;
import com.ximpleware.VTDNav;
import com.ximpleware.XPathEvalException;
import com.ximpleware.XPathParseException;

/**
 * XML traversal using push-pop mechanism with VTD-XML and XPATH
 *
 */
public class VTDXmlTest {

      public static void main(String[] args) throws XPathParseException, XPathEvalException, NavException {
            parseXmlUisngVtd("college.xml");
      }

      public static void parseXmlUisngVtd(String xmlFilePath) throws XPathParseException,
                  XPathEvalException, NavException{

            final VTDGen vg = new VTDGen();
            vg.parseFile(xmlFilePath, false);
            final VTDNav vn = vg.getNav();
            final AutoPilot ap = new AutoPilot(vn);

            ap.selectXPath("/college/staff");
            int val;

            while(ap.evalXPath() != -1) {

                  vn.push();
                  if(vn.toElement(VTDNav.FIRST_CHILD,"employee")){
                        do{
                              if((val = vn.getAttrVal("id")) != -1){
                                    System.out.println("Employee Id:" + vn.toNormalizedString(val));
                              }
                              if((val = vn.getAttrVal("name")) != -1){
                                    System.out.println("Employee name:" + vn.toNormalizedString(val));
                              }
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"employee"));
                  }
                  vn.pop();

                  if(vn.toElement(VTDNav.FIRST_CHILD,"salary")){
                        do{
                              vn.push();
                              if((val = vn.getAttrVal("id")) != -1){
                                    System.out.println("Employee Id:" + vn.toNormalizedString(val));
                              }

                              //XML traversal using push-pop mechanism
                              getXpathValue(vn, ".//basic/text()");
                              getXpathValue(vn, ".//special-allowance/text()");
                              getXpathValue(vn, ".//medical/text()");
                              getXpathValue(vn, ".//provident-fund/text()");
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"salary"));
                  }
                  System.out.println("========================================================================");
                  vn.toElement(VTDNav.PARENT);
            }
      }

      static void getXpathValue(final VTDNav vn, String xpath) throws NavException,
                  XPathParseException, XPathEvalException{

            //push to save the pointer state and traverse anywhere
            vn.push();
            AutoPilot apSal = new AutoPilot(vn);
            apSal.selectXPath(xpath);

            if(apSal.evalXPath() != -1){
                  System.out.println(xpath + " = " + vn.toNormalizedString(vn.getText()));
            }
            //pop to get the saved pointer state and reach the same saved place
            vn.pop();
      }
}

O/P of the above program as below:

Employee Id:101-01
Employee name:ashish
Employee Id:101-02
Employee name:amit
Employee Id:101-03
Employee name:nupur
Employee Id:101-sal
.//basic/text() = 20000
.//special-allowance/text() = 50000
.//medical/text() = 10000
.//provident-fund/text() = 10000
========================================================================
Employee Id:102-01
Employee name:shikhar
Employee Id:102-02
Employee name:sanjay
Employee Id:102-03
Employee name:ani
Employee Id:102-sal
.//basic/text() = 25000
.//special-allowance/text() = 60000
.//medical/text() = 10000
.//provident-fund/text() = 12000
========================================================================
Employee Id:103-01
Employee name:suman
Employee Id:103-02
Employee name:salil
Employee Id:103-03
Employee name:amar
Employee Id:103-sal
.//basic/text() = 35000
.//special-allowance/text() = 70000
.//medical/text() = 12000
.//provident-fund/text() = 15000

Let us see another way of VTD parsing using Namespaces, here, in this case, the XML document will have the namespace.

VTD –XML with Namespaces:

XML file with Namespace:-

<?xml version = '1.0' encoding="UTF-8"?>

<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope">
      <env:body>
            <college>
                  <staff id="101" dep_name="Admin">
                        <employee id="101-01" name="ashish" />
                        <employee id="101-02" name="amit" />
                        <employee id="101-03" name="nupur" />
                        <salary id="101-sal">
                              <basic>20000</basic>
                              <special-allowance>50000</special-allowance>
                              <medical>10000</medical>
                              <provident-fund>10000</provident-fund>
                        </salary>
                  </staff>
                  <staff id="102" dep_name="HR">
                        <employee id="102-01" name="shikhar" />
                        <employee id="102-02" name="sanjay" />
                        <employee id="102-03" name="ani" />
                        <salary id="102-sal">
                              <basic>25000</basic>
                              <special-allowance>60000</special-allowance>
                              <medical>10000</medical>
                              <provident-fund>12000</provident-fund>
                        </salary>
                  </staff>
                  <staff id="103" dep_name="IT">
                        <employee id="103-01" name="suman" />
                        <employee id="103-02" name="salil" />
                        <employee id="103-03" name="amar" />
                        <salary id="103-sal">
                              <basic>35000</basic>
                              <special-allowance>70000</special-allowance>
                              <medical>12000</medical>
                              <provident-fund>15000</provident-fund>
                        </salary>
                  </staff>
            </college>
      </env:body>
</env:Envelope>
import com.ximpleware.AutoPilot;
import com.ximpleware.NavException;
import com.ximpleware.VTDGen;
import com.ximpleware.VTDNav;
import com.ximpleware.XPathEvalException;
import com.ximpleware.XPathParseException;

/**
 * XML traversal using push-pop mechanism with VTD-XML and XPATH and NameSpace
 */
public class VTDXmlTest {

      /** NameSpace **/
      private static final String NAMESPACE_URL = "http://www.w3.org/2003/05/soap-envelope";
      private static final String NAMESPACE = "env";

      public static void main(String[] args) throws XPathParseException, XPathEvalException, NavException {
            parseXmlUisngVtd("college.xml");
      }

      public static void parseXmlUisngVtd(String xmlFilePath) throws XPathParseException,
                  XPathEvalException, NavException{

            final VTDGen vg = new VTDGen();
            vg.parseFile(xmlFilePath, true);
            final VTDNav vn = vg.getNav();
            final AutoPilot ap = new AutoPilot(vn);

            ap.declareXPathNameSpace(NAMESPACE, NAMESPACE_URL);
            ap.selectXPath("/env:Envelope/env:body/college/staff");
            int val;

            while(ap.evalXPath() != -1) {
                  vn.push();
                  if(vn.toElement(VTDNav.FIRST_CHILD,"employee")){
                        do{
                              if((val = vn.getAttrVal("id")) != -1){
                                    System.out.println("Employee Id:" + vn.toNormalizedString(val));
                              }
                              if((val = vn.getAttrVal("name")) != -1){
                                    System.out.println("Employee name:" + vn.toNormalizedString(val));
                              }
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"employee"));
                  }
                  vn.pop();

                  if(vn.toElement(VTDNav.FIRST_CHILD,"salary")){
                        do{
                              vn.push();
                              if((val = vn.getAttrVal("id")) != -1){
                                    System.out.println("Employee Id:" + vn.toNormalizedString(val));
                              }

                              //XML traversal using push-pop mechanism
                              getXpathValue(vn, ".//basic/text()");
                              getXpathValue(vn, ".//special-allowance/text()");
                              getXpathValue(vn, ".//medical/text()");
                              getXpathValue(vn, ".//provident-fund/text()");
                        }while(vn.toElement(VTDNav.NEXT_SIBLING,"salary"));
                  }
                  System.out.println("========================================================================");
                  vn.toElement(VTDNav.PARENT);
            }
      }

      static void getXpathValue(final VTDNav vn, String xpath) throws NavException,
                  XPathParseException, XPathEvalException{

            //push to save the pointer state and traverse anywhere
            vn.push();
            AutoPilot ap = new AutoPilot(vn);
            ap.declareXPathNameSpace(NAMESPACE, NAMESPACE_URL);
            ap.selectXPath(xpath);

            if(ap.evalXPath() != -1){
                  System.out.println(xpath + " = " + vn.toNormalizedString(vn.getText()));
            }
            //pop to get the saved pointer state and reach the same saved place
            vn.pop();
      }
}

AutoPilot's XPath-related methods are:    

  1. declareXPathNameSpace(...): This method binds a namespace prefix (used in an XPath expression) to a URL    
  2. selectXPath(...): This method compiles an XPath expression into internal representation.
  3. evalXPath(...): This method moves the cursor to a qualified node in the node set.
  4. evalXPathToBoolean(...), evalXPathToNumber(...), evalXpathToString(...): These three methods evaluates an XPath expression respectively to a boolean, a double and a string.
  5. resetXPath(): Reset the internal state so the XPath can be re-used.    
  6. getExprString(): Call this method to verify the correctness of the compiled expression.In addition, two more exception classes are introduced for XPath:       
  7. XPathParseException: It is thrown when there is a syntax error in the XPath expression.
  8. XPathEvalException: It is thrown when there is an exception condition during XPath evaluation

I hope this will be helpful for understanding the power of VTD parser.

Managing data at scale doesn’t have to be hard. Find out how the completely free, open source HPCC Systems platform makes it easier to update, easier to program, easier to integrate data, and easier to manage clusters. Download and get started today.

Like This Article? Read More From DZone

Working With XML in Scala
Tutorial: Bare-Metal Shell for Kinetis
Domain-Driven Design in JavaScript

Free DZone Refcard

Data Warehousing
DOWNLOAD
Topics:
xml parsing

Comment (2)

Save
Tweet
{{ articles[0].views | formatCount}} Views

Opinions expressed by DZone contributors are their own.

Big Data Partner Resources

Data Lakes: A Serious Consideration for Serious Data Science
Industry Solution Guide for Using Apache NiFi and Attunity Replicate
Data Lake vs. Data Warehouse
eBook: Simplify Apache Kafka
White Paper: 6 Simple Steps for Replatforming in the Age of the Data Lake
White Paper: 12 Best Practices for Modern Data Ingestion
Mastering Data Lakes with HPCC Systems
Whitepaper: Scaling Capabilities to Build a Homogeneous Big Data Ecosystem

Big Data Partner Resources

Data Lakes: A Serious Consideration for Serious Data Science
Industry Solution Guide for Using Apache NiFi and Attunity Replicate
Data Lake vs. Data Warehouse
eBook: Simplify Apache Kafka

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.linkDescription }}

{{ parent.urlSource.name }}
by {{ parent.authors[0].realName || parent.author}}
· {{ parent.articleDate | date:'MMM. dd, yyyy' }} {{ parent.linkDate | date:'MMM. dd, yyyy' }}
· {{ parent.portal.name }} Zone