This article presents the first demo of my session about NoSQL databases at the German Entwicklercamp conference in March 2012. It demonstrates how the document-oriented NoSQL database MongoDB can be used in an XPages web application for the IBM Lotus Domino server.
The rise of location based services
Location based services have become quite popular in the past years: Most of the smartphones carry a GPS sensor, and there are a lot of popular apps out there (e.g. Foursquare, Yelp) that use cloud services to find people, restaurants and places nearby.
For a number of reasons (mostly scalability/performance related), the NSF database of Lotus Notes/Domino is not the best choice to implement these kind of applications/services. MongoDB is much more suited for this use case and the article shows how you can integrate MongoDB into Notes/Domino to combine both worlds and get the best from both of them.
The Dojo based web application that I am going to discuss in detail uses MongoDB'sgeospatial indexing feature to easily find the nearest points of interest for a mobile web user.
Here is how our application looks like:
The "Administration" tab provides a user interface to add/remove places to/from the MongoDB database. A place is stored as a document in a MongoDB document collection and consists of a name, a type (e.g. shop/gas station) and the position information as [longitude, latitude] value pair.
The database can be queried on the "Search" tab by entering an address, a distance in kilometers and an optional type. The address will automatically get converted to [longitude, latitude] coordinates by using the Google Geocoding API, which is then used to find places within the specified distance.
Our MongoDB document collection is indexed in a way so that we can quickly query the database for places that surround the current user's position and sort them by ascending distance between both points.
Since geospatial indexing is a built-in feature of MongoDB, the solution is very easy to implement.
Limitations of NSF
While it is not impossible to build this application in pure Lotus Notes/Domino technology with an NSF database, the missing geo index and limited scalability of NSF would make it difficult to keep response times low with large data sets, e.g. millions or even billions of places worldwide. MongoDB instead provides a sharding feature to distribute the data evenly to several servers and reduce server load.
Additional indexer like Apache Lucene in combination with the OSGi tasklet service plugin's extension hooks might help to improve Domino's limited indexing support (we already used a Lucene index with Domino data in a big customer project) , but using MongoDB for this use case is just so much easier.
Architecture
The application consists of client-side and server-side code: Client-side code is written in JavaScript language as a Dojo class using the class declaration system of Dojo 1.6. This Dojo class handles all user interface operations.
Server-side code is written in Java as a servlet and is exposed to the client-side via REST API's (here: GET/POST requests sent via dojo.xhrGet / dojo.xhrPost).
We developed two servlets for this application. The first servlet handles init/add/remove/query operations in MongoDB. The second servlet simply acts as a proxy to access the Google Geocoding API from client-side Javascript in order to convert between addresses and geo coordinates (longitude/latitude).
Since the REST protocol is completely stateless, the server-side code is very easy to test. With a bit of refactoring, it's even possible to run the whole application in alternative servlet containers like Jetty, Tomcat or directly within the Eclipse IDE instead of the Domino server.
Personally, I really like this REST API based approach because it's clean, transparent and keeps technologies separated. It enables you to replace the server environment, the database system and the client-side UI toolkit at any time. You can even build multiple UIs (e.g. based on Dojo 1.6, 1.7 and Sencha's ExtJS) or mobile clients that work with the same REST API.
Diving into the code: server side Our sample comes as an Eclipse plugin "com.mindoo.mongo.test" to be run on Domino's OSGi framework (tested with Domino 8.5.3 GA without any fixpacks or XPages extension library):
As you can see above, I added the Mongo API classes to the plugin's classpath. Adding the Mongo API classes to an NSF and using them directly from SSJS code might also be possible, but I prefer the plugin solution, because the MongoDB access is supposed to be a central service on the Domino server. In addition, by using a plugin we don't have any issues with Domino's restricted security manager that prevents operations to run properly in an XPages context (e.g. a lot of drivers are using Log4J for logging, which does not run well within an XPages application).
Our two servlets are defined in the plugin.xml file of the plugin:
That's all you need to declare your own servlet on Domino. After that, the servlet can either be accessed on the URL base level (http://server/mongogeo) or in the context of a Domino database (http://server/path/to/db.nsf/mongotest).
If called in the context of a database, the Domino HTTP server first makes sure that the current user is allowed to access the database and if not, it displays a login prompt. A utility class (com.ibm.domino.osgi.core.context.ContextInfo) can be used to get session/database objects for the current HTTP request.
In our sample, we use the first format for our Geo API query, because it's an open service of the server without access restriction. Anyone can use it and the URL is simple to remember.
The second URL format (/path/to/db.nsf/mongotest) is used to access MongoDB data. This enables us to check the user against the Notes database ACL to see if he is allowed to read or write MongoDB data.
Here is the method of our MongoDB access servlet that handles POST requests:
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { Session session=ContextInfo.getUserSession(); Database dbContext=ContextInfo.getUserDatabase(); if (dbContext==null) { ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN, "Servlet must be run in the context of a Domino database", null); return; }
try { String pathInfo=req.getPathInfo(); if ("/deleteplaces".equals(pathInfo)) { if (!isUserAuthenticated(MongoAccessMode.Write, session, dbContext)) { ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN, "Current user is not allowed to write data", null); return; } doDeleteGridData(session, dbContext, req, resp); } else if ("/addplaces".equals(pathInfo)) { if (!isUserAuthenticated(MongoAccessMode.Write, session, dbContext)) { ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN, "Current user is not allowed to write data", null); return; } doAddGridData(session, dbContext, req, resp); } else if ("/queryplaces".equals(pathInfo)) { if (!isUserAuthenticated(MongoAccessMode.Read, session, dbContext)) { ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN, "Current user is not allowed to read data", null); return; } doQueryPlaces(session, dbContext, req, resp); } else if ("/initdb".equals(pathInfo)) { if (!isUserAuthenticated(MongoAccessMode.Write, session, dbContext)) { ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN, "Current user is not allowed to write data", null); return; } doInitDbWithValues(session, dbContext, req, resp); } else if ("/cleardb".equals(pathInfo)) { if (!isUserAuthenticated(MongoAccessMode.Write, session, dbContext)) { ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN, "Current user is not allowed to write data", null); return; } doClearDb(session, dbContext, req, resp); } else { ServletUtils.sendHttpError(resp, 500, "Unsupported command "+pathInfo, null); } } catch (Throwable e1) { e1.printStackTrace(); //avoid posting the full stacktrace to the user (for security reasons) ServletUtils.sendHttpError(resp, 500, "An error occurred processing the request. The error has been logged. Please refer to the server log files/console for details", null); return; } }
The code is pretty simple: We check if the user is allowed to call the specific service (read operations require the role [MongoDBRead], write operations the role [MongoDBWrite] ) and then forward the request to helper methods.
Note that I am not going into full detail in this blog article how we actually access data in MongoDB. You can find that in the provided download archive and there is a great tutorial at the MongoDB website about using the Java API.
My EntwicklerCamp 2013 slides: Dojo 1.8 and AMD (German)
Wed, Mar 13th 2013 8:12a Karsten Lehmann The closing session of EntwicklerCamp 2013 has just finished. Here are the slides for my EntwicklerCamp 2013 session about "Dojo 1.8 and AMD": Download archive: Slides as ZIP-Archive To all English speaking readers: Google Translate is your friend - the slides are in German :-) My plan is to translate them to English though and publish my demos as soon as possible. [read] Keywords: archive
dojo
google
288
Quick tip: Fixing Dojo drag and drop issues in a Windows 8 VMWare on the Mac
Wed, Mar 6th 2013 11:59p Karsten Lehmann I am currently setting up a new dev environment with Windows 8 and Notes/Domino 9 to work on demos for my Dojo 1.8/AMD session at Entwicklercamp next week. To my surprise, I noticed yesterday, that drag and drop operations on Dojo widgets did not work as expected. For example, I could not drag the splitters of a BorderContainer layout widget and the columns of a LazyTreeGrid could not get resized. It seemed as if mouse events got lost, but I only got that effect in Firefox and Chrome. In IE, [read] Keywords: domino
notes
dojo
firefox
mac
vmware
widget
widgets
768
Download links for IBM Connect 2013 session slides
Thu, Jan 31st 2013 7:23a Karsten Lehmann As in previous years, I copied the download URLs of all the already available session slides from the socialbizonline.com website. Unfortunately, a lot of slides are still missing and some are only provided in black and white mode with 2-on-1 page. Use your preferred download manager to download the files. I use DownThemAll for this purpose. You need to be logged in to the website to access the files. Here are the download links: Connect2013_PDFs.html And here is the spreadsheet I u [read] Keywords: ibm
lotusphere
ods
firefox
Session accepted for Entwicklercamp 2013: Dojo 1.8 and AMD
Sun, Sep 30th 2012 1:57p Karsten Lehmann I just received confirmation that my session submission for next years developer conference Entwicklercamp 2013 in Gelsenkirchen (11th - 13th of March) got accepted. The session is about the new Dojo toolkit 1.8 that we have already used in a customer project since its release in August 2012. Sessions at Entwicklercamp are held in German language, so here is the translated abstract and the original one in German. Dojo 1.8 and AMD The session provides an introduction to web application [read] Keywords: application
applications
development
dojo
mobile
New address of Mindoo GmbH in Karlsruhe, Germany
Sat, Jun 30th 2012 6:55a Karsten Lehmann Please note that we have moved. As of 1st of July 2012, our new company address in Karlsruhe, Germany is Mindoo GmbH Heid-und-Neu-Strasse 7 76135 Karlsruhe Germany [read] Keywords:
