How to Configure Infinispan With Transactions, Backed by Relational DB on JBoss AS 7 vs. Tomcat 7

Configuring transactions and JDBC cache store on JBoss AS 7

Infinispan will be configured via new fluent API using builders, hence the call to  .build() method at the end. We need to configure aspects related to  transactions and cache loaders. The configuration API for cache loaders  is likely going to be changed in not-so-far future. It should be fluent  and more intuitive, generally easier to use than current one. 

I purposely do not show XML configuration. Configuration examples can be found at https://github.com/infinispan/infinispan/blob/master/core/src/main/resources/config-samples/sample.xml. In order to configure transactions and cache loaders, look for tags called  <transaction> and <loaders> and modify that sample file according to below configuration. Tag names and attribute names are very similar for both XML and Java configuration. If that is not enough, there is always a schema in Infinispan distribution.

The configuration of Infinispan is as follows:

GlobalConfiguration glob = new GlobalConfigurationBuilder()
   .nonClusteredDefault().build();
 
Configuration loc = new ConfigurationBuilder()
   .clustering().cacheMode(CacheMode.LOCAL) 
   .transaction().transactionMode(TransactionMode.TRANSACTIONAL)
   .autoCommit(false)
   .transactionManagerLookup(new GenericTransactionManagerLookup())
   .loaders().passivation(false).preload(false).shared(false)
   .addCacheLoader().cacheLoader(new JdbcStringBasedCacheStore())
   .fetchPersistentState(false).purgeOnStartup(true)
   .addProperty("stringsTableNamePrefix", "carmart_table")
   .addProperty("idColumnName", "ID_COLUMN")
   .addProperty("dataColumnName", "DATA_COLUMN")
   .addProperty("timestampColumnName", "TIMESTAMP_COLUMN")
    //for different DB, use different type
   .addProperty("timestampColumnType", "BIGINT") 
   .addProperty("connectionFactoryClass", 
        "org.infinispan.loaders.jdbc.connectionfactory.ManagedConnectionFactory")
   .addProperty("connectionUrl", "jdbc:mysql://localhost:3306/carmartdb")
   .addProperty("driverClass", "com.mysql.jdbc.Driver")
    //for different DB, use different type 
   .addProperty("idColumnType", "VARCHAR(255)") 
    //for different DB, use different type 
   .addProperty("dataColumnType", "VARBINARY(1000)") 
   .addProperty("dropTableOnExit", "false")
   .addProperty("createTableOnStart", "true")
   .addProperty("databaseType", "MYSQL")
   .addProperty("datasourceJndiLocation", "java:jboss/datasources/ExampleDS")
   .build();

BasicCacheContainer manager = new DefaultCacheManager(glob, loc, true); 
.... = manager.getCache() 

Lines marked with red are different in other containers/configurations, as you'll see in a minute. The code above implies that we need to specify proper TransactionManagerLookup implementation which is, in this case, GenericTransactionManagerLookup. We  also need to say: "Hey, I wanna use ManagedConnectionFactory as a connectionFactoryClass". OK, here we go. I should, as well, explain how to configure a datasource properly, right? In JBoss AS 7, this is configured as a subsystem in $JBOSS_HOME/standalone/configuration/standalone.xml:

<subsystem xmlns="urn:jboss:domain:datasources:1.0">
   <datasources>
      <datasource jndi-name="java:jboss/datasources/ExampleDS"    
                  pool-name="ExampleDS" enabled="true" 
                  use-java-context="true">
         <connection-url>jdbc:mysql://localhost:3306/carmartdb</connection-url>
         <driver>mysql-connector-java-5.1.17-bin.jar</driver>
         <security>
            <user-name>carmart</user-name>
            <password>carmart</password>
         </security>
      </datasource>
   </datasources> 
</subsystem>

The usage of transactions is very simple as we can obtain a transaction object by injection.

@Inject 
private javax.transaction.UserTransaction utx;
     
try {
    utx.begin();
    cache.put(...) //store/load keys to/from the cache
    utx.commit(); 
} catch (Exception e) {
    if (utx != null) {
        try {
            utx.rollback();
        } catch (Exception e1) {}
    } 
}

Sources: https://github.com/mgencur/infinispan-examples/blob/master/carmart-tx-jdbc/src/jbossas/java/org/infinispan/examples/carmart/session/CarManager.java

Quite easy, isn't it ...if you know how to do it. The only problem is that it does not work (at least not completely) :-) If you deploy the app, you find out that when storing a key-value pair in  the cache, an exception is thrown. This exception indicates that the operation with DB (and JDBC cache store) failed. The exception says:

Error while processing a commit in a two-phase transaction:
org.infinispan.CacheException:
org.infinispan.loaders.CacheLoaderException:
This might be related to https://jira.jboss.org/browse/ISPN-604

A complete stack trace looks similar to https://gist.github.com/2777348
There's still an open issue in JIRA (ISPN-604) and it is being worked on. 

Configuring transactions and JDBC cache store on JBoss AS 7 - c3p0

But how do we cope with this inconvenience for now... By not using a managed datasource but rather a third party library called c3p0 (JDBC3  Connection and Statement Pooling, more information at http://www.mchange.com/projects/c3p0/index.html) Infinispan allows you to use this library for connecting to the database. If you really want to use it, you need to choose a different connectionFactoryClass which is, in this case, PooledConnectionFactory.

Infinispan configuration looks like this:

GlobalConfiguration glob = new GlobalConfigurationBuilder()
    .nonClusteredDefault().build(); 

Configuration loc = new ConfigurationBuilder()
    .clustering().cacheMode(CacheMode.LOCAL)
      .transaction().transactionMode(TransactionMode.TRANSACTIONAL)
    .autoCommit(false)
    .transactionManagerLookup(new GenericTransactionManagerLookup())
    .loaders().passivation(false).preload(false).shared(false)
    .addCacheLoader().cacheLoader(new JdbcStringBasedCacheStore())
    .fetchPersistentState(false).purgeOnStartup(true)
    .addProperty("stringsTableNamePrefix", "carmart_table")
    .addProperty("idColumnName", "ID_COLUMN")
    .addProperty("dataColumnName", "DATA_COLUMN")
    .addProperty("timestampColumnName", "TIMESTAMP_COLUMN")
    .addProperty("timestampColumnType", "BIGINT")
    .addProperty("connectionFactoryClass",
        "org.infinispan.loaders.jdbc.connectionfactory.PooledConnectionFactory")
    .addProperty("connectionUrl", "jdbc:mysql://localhost:3306/carmartdb")
    .addProperty("userName", "carmart")   //we do not have a managed datasource 
     -> specify credentials here
    .addProperty("password", "carmart")
    .addProperty("driverClass", "com.mysql.jdbc.Driver")
    .addProperty("idColumnType", "VARCHAR(255)")
    .addProperty("dataColumnType", "VARBINARY(1000)")
    .addProperty("dropTableOnExit", "false")
    .addProperty("createTableOnStart", "true")
    .addProperty("databaseType", "MYSQL")
    //.addProperty("datasourceJndiLocation", "java:jboss/datasources/ExampleDS") 
    //oh, yes, we do not use JNDI now     
    .build();

Transactions are accessible in the same way as in the previous use case. Now let's look at configuration for Tomcat servlet container. 

Configuring transactions and JDBC cache store on Tomcat 7

Tomcat does not have any Transaction Manager in it so we have to bundle one with the application. For the purpose of this exercise, we choose JBoss Transactions (http://www.jboss.org/jbosstm). See dependencies at the end.

Cache manager and cache configuration is in this form:

GlobalConfiguration glob = new GlobalConfigurationBuilder()
    .nonClusteredDefault().build();

Configuration loc = new ConfigurationBuilder()
    .clustering().cacheMode(CacheMode.LOCAL) 
    .transaction().transactionMode(TransactionMode.TRANSACTIONAL)
    .autoCommit(false)
    .transactionManagerLookup(new JBossStandaloneJTAManagerLookup()) 
    .loaders().passivation(false).preload(false).shared(false)
    .addCacheLoader().cacheLoader(new JdbcStringBasedCacheStore())
    .fetchPersistentState(false).purgeOnStartup(true)
    .addProperty("stringsTableNamePrefix", "carmart_table")
    .addProperty("idColumnName", "ID_COLUMN")
    .addProperty("dataColumnName", "DATA_COLUMN")
    .addProperty("timestampColumnName", "TIMESTAMP_COLUMN")
    .addProperty("timestampColumnType", "BIGINT")
    .addProperty("connectionFactoryClass", 
        "org.infinispan.loaders.jdbc.connectionfactory.ManagedConnectionFactory")
    .addProperty("connectionUrl", "jdbc:mysql://localhost:3306/carmartdb")
    .addProperty("userName", "carmart")
    .addProperty("driverClass", "com.mysql.jdbc.Driver")
    .addProperty("idColumnType", "VARCHAR(255)") 
    .addProperty("dataColumnType", "VARBINARY(1000)")
    .addProperty("dropTableOnExit", "false")
    .addProperty("createTableOnStart", "true")
    .addProperty("databaseType", "MYSQL")
    .addProperty("datasourceJndiLocation", "java:comp/env/jdbc/ExampleDB")
    .build();

For Tomcat, we need to specify a different transactionManagerLookup implementation and datasourceJndiLocation. Tomcat simply places objects  under a bit different JNDI locations. The datasource is defined in context.xml file which has to be on classpath. This file might look like this:

<Context>
    <Resource name="jdbc/ExampleDB"
        auth="Container"
        type="javax.sql.DataSource"
        factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
        maxActive="100"
        minIdle="30"
        maxWait="10000"
        jmxEnabled="true"
        username="carmart"
        password="carmart"
        driverClassName="com.mysql.jdbc.Driver"
        url="jdbc:mysql://localhost:3306/carmartdb"/> 
</Context>

How do we get the transaction manager in the application then? Lets obtain  it directly from a cache. 

Infinispan knows how to find the manager and we need to know how to obtain it from Infinispan.

javax.transaction.TransactionManager tm = 
    ((CacheImpl) myCache).getAdvancedCache().getTransactionManager(); 
try {
    tm.begin();
    myCache.put(...) 
    tm.commit(); 
} catch (Exception e) {
    if (tm != null) {
        try {
            tm.rollback();
        } catch (Exception e1) {}
    } 
}

Sources: https://github.com/mgencur/infinispan-examples/blob/master/carmart-tx-jdbc/src/tomcat/java/org/infinispan/examples/carmart/session/CarManager.java The transaction manager provides standard methods for transactions, such as begin(), commit() and rollback(). 

Now is the time for dependencies

So...which dependencies do we always need when using Infinispan with JDBC cache stores and transactions? These are infinspan-core, infinispan-cachestore-jdbc and javax.transaction.jta. The scope for jta dependency, as defined in Maven, is different for JBossAS and Tomcat.

Common dependencies for JBossAS and Tomcat

<dependency>
   <groupId>org.infinispan</groupId>
   <artifactId>infinispan-core</artifactId>
   <version>5.1.4.FINAL</version>
</dependency>
    
<dependency>
   <groupId>org.infinispan</groupId>
   <artifactId>infinispan-cachestore-jdbc</artifactId>
   <version>5.1.4.FINAL</version>
</dependency>

Of course, our application needs a few more dependencies but these are not directly related to Infinispan. Let's ignore them in this article. JBoss AS 7 provides managed datasource that is accessible from Infinispan. The only specific dependency (related to transactions or Infinispan) is JTA.

Dependencies specific to JBossAS - using managed Datasource (managed by the server)

<dependency>
   <groupId>javax.transaction</groupId>
   <artifactId>jta</artifactId>
   <version>1.1</version>
   <scope>provided</scope>
</dependency>

Dependencies specific to JBossAS - using c3p0

<dependency>
   <groupId>javax.transaction</groupId>
   <artifactId>jta</artifactId>
   <version>1.1</version>
   <scope>provided</scope>
</dependency>
    
<dependency>
   <groupId>c3p0</groupId>
   <artifactId>c3p0</artifactId>
   <version>0.9.1.2</version>
</dependency>
    
<dependency>
   <groupId>mysql</groupId>
   <artifactId>mysql-connector-java</artifactId>
   <version>5.1.17</version>
</dependency>

Yes, you need to bundle also MySQL connector. On the other hand, for Tomcat use case and JBossAS with managed datasource, this jar file needs do be deployed to the server separately. For Tomcat, do this simply by copying the jar file to $TOMCAT_HOME/lib.  For JBoss AS 7, copy the jar file into $JBOSS_HOME/standalone/deployments.

Dependencies specific to Tomcat - using JBoss Transactions

<dependency>
   <groupId>javax.transaction</groupId>
   <artifactId>jta</artifactId>
   <version>1.1</version>
   <scope>runtime</scope>
</dependency>
    
<dependency>
   <groupId>org.jboss.jbossts</groupId>
   <artifactId>jbossjta</artifactId>
   <version>4.16.4.Final</version>
</dependency>