In this post, I'd like to go over another main component of the NSF ODP Tooling project, the ODP Exporter. The exporter is significantly simpler than the compiler, but still had a surprising number of gotchas of its own.

My goal in writing the exporter was to replace the need to use Designer to create an on-disk project out of an NSF - in one of my projects, in addition to the primary NSF we use, there are also a dozen or so secondary NSFs inheriting from templates and being modified by people not using Git (I know.), and keeping them all in sync is a giant PITA. Previously, I had a dedicated VM just to open the DBs periodically to sync them, but even that took a long time and got error-prone when Designer would miss a change or generally trip over itself.

So I set out to create a compatible replacement, so that I could run a script and update the ODPs en masse.

The Basics

At its core, the exporter does what you might expect: it reads through each design note and sends them through a DXL exporter. For its work, it makes use of the aforementioned design collection and IBM's NAPI. I went with IBM's variant in this case for one class: com.ibm.designer.domino.napi.design.FileAccess. Though this class let me down when it came to importing, it has just enough encapsulated composite-data reader methods to save me a ton of work here, though I had to cheat to access one of them.

For each design note, it determines its type, which contains behavioral information for each type, including whether it should be included in the normal export process at all, where it's placed in the ODP, and the type of export treatment it should get. The main categories of exported note types line up with what the compiler had to know about, with some special knowledge of whether a note is one of many in a folder (e.g. an XPage) or one-per-database (like the icon note).

The Gotchas

Unsurprisingly, things aren't quite as simple as a basic loop. For elements that are just DXL files, it is that easy, but the ones that exist either as just file data (e.g. "plugin.xml") or file data plus metadata require special handling.

Skipped Items

The first thing to note with split data/metadata items isn't complicated, but bears mentioning: the metadata file is generated by exporting the design note as DXL but ignoring data items. Some of these are common among all types, but others (like indexed "$ClassData0", etc. fields) are best matched and excluded with a regex.

LotusScript, Again

LotusScript libraries threw me for an unexpected loop this time. Their storage format is actually not even composite data: the script itself is stored in plain non-summary text items, multiple ones with the same name. However, the NAPI's DXL exporter doesn't actually export the full script test properly, instead only outputting the content of the first item. Additionally, the legacy Java API shows the presence of multiple items with the same name, but also only gives you the value for the first.

So I ended up having to use the raw note format in memory, which DOES include all of the items, and then stitch the script content together onto the filesystem.

Other File Data

The other data types aren't too complicated, but need special cases for each composite data structure, which is where the FileAccess class comes in. Without the convenience methods there, I would have had to write CD iterators to read the data based on the appropriate structures - not terribly difficult, but it's all the better to have the work already done for me. Especially so since FileAccess pleasantly writes directly to a java.io.OutputStream, just like I'd want if I wrote it myself.

Special-Case Files

There are three final special cases that the exporter handles:

1. The icon note is specially-exported not once, but twice. It's exported using an NAPI-specific special method to create the "database.properties" file, which includes the ACL and and formatted settings alongside the icon note, and then also exported specially after the main loop as "Resources/IconNote". I've always appreciated how much Lotus wildly overloaded the icon note.

   • There's also a distinct "$DBIcon" note that houses the 32-bit icon introduced in R8 (if I recall correctly), but that's just a normal old image resource with a special name and not related to the icon note.

2. The "META-INF/MANIFEST.MF" file resource became important in 9.0.1 FP10, but Designer's handling of it is a little schizophrenic. FP10+ will usually fill it in with plugin information when it rebuilds an NSF, but nonetheless exports it as a zero-byte file. It's important for it to exist, so I create a blank file if it doesn't exist.

3. The Eclipse ".project" file is also not present in older NSFs, but is critical for ODPs. If it wasn't exported, I create a generic stub version.

Swiper

When dealing with on-disk projects, Swiper is a mandatory tool, cleaning up the generated XML and (critically) removing extraneous items that change too frequently to be source-control-friendly.

The core of Swiper is an XSLT stylesheet to do the transformation, and I incorporated this wholesale, with a minor modification to retain the ACL that's stripped out by stock Swiper. I then created an OutputStream implementation that passes DXL output through Swiper if configured. As a small note, I think there was a specific reason why I have the Swiper path buffer the DXL into an in-memory ByteArrayOutputStream first instead of just wrapping the file output stream, but I don't remember what that was.

Final Steps

With this post, I think I've covered the big topics I set out to with the two main components of the Tooling. I plan on having at least one final post in the series to cover some potential future additions and enhancements, since I have a lot of ideas in mind for it. Unfortunately, a lot of the most-useful ideas would also be tremendous amounts of work, but the payoff may eventually be worth it.