Integrating Lucene into a Transactional XML Database
In this talk we will present an integration of the Lucene search engine with EMC Documentum xDB database (native XML database). We will introduce a new approach implemented in xDB 10.3 which integrates Lucene index (used for XQuery queries optimization) into transactional xDB engine on the storage level. That is, Lucene files are stored to the XDB data pages instead of the file system as in earlier releases, Lucene accesses all the files through xDB buffer pool instead of the just the Operating system buffer cache. This approach allows us to simplify the implementation of traditional database features for Lucene within xDB like transactions isolation, rollbacks, recovery after database crashes, snapshots construction , replication, hot backups, buffer management, etc. We cover performance analysis of new approach for queries and ingest operations, performance tuning tips and future optimization techniques in the area. The presentation is intended as a description of an implementation and performance analysis.