17 March 2017

Lightning Memory-mapped Database on OSGi

Now and then i get what i call a niche requirement, something that does not hold to any standard or good practice. These kind of requirements are usually very development intensive, time consuming and great fun, or a real PITA when having to deal with a deadline.

I am in need of a fast, local, in process atomic counter for daily file name creation, which is thread and crash save, runs within a Java/OSGi environment and can participate in a XA transactions. The files being published are getting consumed and modified by other applications as soon as they are marked with a done file, making a stateless file counting solution impossible. A RDBMS springs to mind thinking of state, but the overhead in production is unproportional as the data generated is only valid for a short period of time and in limited scope.

After turning my toolbox upside down, finding nothing even remotely fitting this job description, I heated up the search engines ready to burn time. My data structure is simple and the amount of data generated is small so a simple key/value store should be just right. This results in a long hit list, but only few libraries support file persistence, and only one made a near perfect match: lmdb. Let me emphase, near perfect match, since it is no Java, no OSGi and has no transactional integration.

The Symas page lists wrappers, and sure enough, java is listed with a JNI implementation called lmdbjni. Strange thing is that this binary has no more Maven support for Windows, MacOS and Android from 0.4.7 onward, bummer. While browsing the github repo i stumbled (by accident via RxLMDB) upon lmdbjava supporting the latest and greatest lmdb version.

So, Java is tackled with a big thanks to the people of lmdbjava. OSGi brings its own mechanism for handeling runtime files and since version 0.0.6 of  lmdbjava the path to the lmdb binary, extracted by OSGi, can be set manually to allow the JNI part to link to it. A example bundle is on github and i leave the JCA integration as exercise.

Happy coding.

No comments:

Post a Comment