|Latest 7 Posts
| Bundle Inspector – Diagnosing XPages Plugin Resolution problems|
Wed, Aug 2nd 2017 0
| Swiper Official Version 2 Release|
Fri, Jul 21st 2017 4
| Auto-width Bootstrap Column XPages Controls|
Tue, Mar 21st 2017 1
| Swiper FP8 Version Beta Release|
Thu, Mar 16th 2017 4
| Swiper FP8 Integration Rollout|
Wed, Mar 8th 2017 4
| Markdown XPages UIControl|
Sun, Mar 5th 2017 3
| TextDiff XPages control – For visual comparison of text|
Tue, Feb 28th 2017 2
| Inspect Component Properties using Component Binding|
Tue, Mar 8th 2016 12
| Pasting Images into XPages CKEditor|
Sun, Feb 12th 2017 7
| Build System for XPages and OSGi plugins|
Fri, Aug 8th 2014 6
| Tips for Creating a Webmail UI with XPages|
Tue, Apr 19th 2016 5
| Extended Messages Control available as an XspLibrary|
Tue, Jul 12th 2016 5
| XPages webmail – Using Mime Inspector to debug Mime|
Tue, Feb 14th 2017 5
| Domino Debug Plugin - Fix for Eclipse Luna|
Tue, Sep 23rd 2014 4
| Select themeIds from plugins in Designer Xsp Properties Editor|
Mon, Sep 8th 2014 4
| Email Address Validator for XPages|
Wed, Feb 11th 2015 4
| Uploading Plugins Headlessly to Open Eclipse Update Site|
Sat, Jul 18th 2015 4
||Webmail UI – You must learn about MIME
If you were like me, you spent many years developing classic Notes applications before making the switch to XPages. If this was the case, you were no doubt comfortable with the notion of a RichText field.
You probably even occasionally did some RichText manipulation in LotusScript, adding Paragraphs and formatting using RichTextStyles and RichTextNavigators, attaching Files using EmbeddedObjects.
And then XPages comes along, and says “If you want to edit any RichText through XPages, it is going to be saved as MIME”. And I’m like “Ok whatever you say, although, I don’t know what your’e talking about and I’m too busy trying to learn all these other new things to also learn about MIME”
And then later on, you come along and write an agent or a function that is meant to deal with a RichText Item and you don’t know why it doesn’t work and why everything looks funny after you do it, and it all comes down to the difference between RichText and MIME but you didn’t even realise there was a difference.
If you can understand some core principles of this Mime / Rich Text ecosystem, then you will be on a much better footing to solve the problems which inevitably appear whenever you are getting involved with Mime / Rich Text creating and manipulation.
I think one origin of confusion is that the phrase ‘Rich Text’ is often used to describe more than one thing.
To demonstrate what I mean, let’s think about ‘Audio files’. Imagine you are a programmer and your problem is that you need to record and playback some sound. Well what file format do you use? Maybe you decide to come up with your own format like Microsoft did e.g. Windows Media Audio (.wma) or alternatively, maybe you decide to use a format that everybody else uses e.g. MPEG layer-3 (.mp3).
The end result for both these ‘Audio Files’ is that you hear sound coming out your speaker just the same, but they are store in different underlying formats. Additionally, if I talk to you about the above situation and I say the word ‘Audio File‘, you know that I am talking about the CONCEPT of an audio file, not any specific format.
In comparison, IBM came up with there own way to save ‘Pretty words, pictures and files’, and the rest of the world came up with another way ‘MIME’, but in Notes/Domino we tend use the word RichText to describe both the CONCEPT of ‘Pretty words, pictures and files’ and also we use RichText to describe the specific format IBM created to store these items.
So you can see the confusion here, someone says “I am storing it as RichText” well what do you mean? are you talking about the format or the concept???
A less confusing way to think about it is maybe:
- Pretty words, pictures and files
- RichText format
- MIME format
Learning about MIME
If you don’t know about MIME
, this is one of the first things you should try to get an understanding of because you will need to know what’s going on. MIME stands for Multipart Internet Mail Extensions,
When they first invented email, RFC822 was the standard which defined how a computer could send a text message to another. This was great, but then people wanted to send messages with using different character sets, they wanted to include images and attachments (Look how excited Steve Ballmer is about putting a picture of a Ferrari in a ‘Microsoft Write’ document). So MIME was invented, which was a new standard of transferring this more complicated information over the same RFC822 compliant system.
In a crude nutshell, a MIME message is just a big long text string/file, which is formatted in a precise way in accordance to the MIME standard.
The Wikipedia article on MIME is a good overview, but if you are ever in doubt about exactly how it works, read the RFC’s that are linked there as these are from the horse’s mouth and they form the actual standard that software would hopefully adhere to.
Inspecting some MIME
Let’s have a quick look at a MIME message. Many email clients / webmail clients will have some way for you to inspect the original ‘MIME’ version of your email. To demonstrate, I will send myself an Email in Gmail and have a look.
Here is a draft Email that I am sending myself, it has an embedded Image, and it also has an Attachment.
So now I send the email to myself, and when I open it in Gmail, I can choose option to ‘Show Original’.
This then opens up the Original MIME representation that was received for Gmail
So you can see it is like a big long text file, but hidden in all that is my image, my attachment, my Html message (and also a plain text version so that anyone on an old school messaging client can also read it).
Let’s break down the Message:
The Message Starts with some SMTP headers which help understand how the message came to be in our inbox! If you are sending receiving MIME in domino, this header information will not be stored in the Body ‘RichTextItem’ but is usually stored as separate items on the Document, e.g. the ‘Subject’ item, SendTo etc.
Then we get to the actual message body, which is made up of a bunch of Mime Parts. Each mime Part has some headers of it’s own which describe some things about that mime part, it then has a blank line, and then it has the Content of the Mime Part. Let’s look at an easy to understand MIME part, the Html message:
So the Mime Part starts with a ‘Content-Type’ Header which describes the purpose of this mime part, in our case it is saying ‘Please treat this as Html’, it also says that our charset is UTF-8. Then it has the blank line which indicates we are finished with headers, and now comes the content. The content then begins and you can see the Html content of our email message.
Note in our html content the <img> tag does not use a http URL, but instead uses ‘cid’. This is a mechanism that is used for inline images. The cid refers to a Content-ID of another Mime Part, let’s have a look at that part:
This mime part represents the little image of the G, it begins with the headers that say “Treat me like a PNG image, I like to be called ‘image.png’. I am meant to be shown ‘inline’ (e.g. embedded, not as a separate attachment), my content is encoded using base64. Some other mime parts refer to me as ‘ii_154338a9bb6d446c’.”
It then has the blank line, and then the content, which is just the image data in a base64 encoded string.
Multipart Content Types
So we have seen 2 simple MIME Parts, but for a complicated message it also needs to be specified how all our Mime parts fit together, and this is usually done by using the ‘multipart’ mime types. Multipart content types are special parts which contain child mime parts.
Multipart types must specify a ‘Boundary’. The boundary is just a text string that is used as a marker to signify when each Child MIME Part begins and ends. This boundary could be anything you like, however if you were to choose a boundary that might also appear in the content, there might be a clash, so the boundary is usually some type of garbled text that is unlikely to appear in a content block.
In our example message we have:
- multipart/mixed – usually used to put an attachment, alongside a message body
- multipart/related – usually used to put an inline image alongside a message body
- multipart/alternative – usually used to say ‘hey you can used this or that’ e.g. html or plaintext
Let’s look at our multipart/alternative mime part
So we can see at the start, the multipart/alternative says “I will use 047d7b414d2a112e990530e95d88 as the boundary, and we can see it has used this boundary to define the start and end of the 2 child mime parts (text/plain and text/html) which are to be treated as ‘alternatives’ i.e. you could read one or the other. Note also, the very last boundary must have 2 hypens at the end.
Put it all together
So if we were to think of our entire message in terms of a tree like child/sibling structure, we could represent it as such:
- image/png (encoding in base64 display inline)
- text/plain (encoded in base64, treat as an attachment)
Learning how to Inspect/Manipulate MIME in Domino
In Domino, mime parts are referred to as MIMEEntities, and the MIMEEntity is the class used to represent one of these parts. If this message (above) was stored in a Domino document as a RichTextItem using MIME format, you could access it using:
MIMEEntity me = document.getMIMEEntity();
But, what you would actually get is the just the first MimeEntity (in our case the multipart/mixed part). You would then have to traverse through child/sibling to do what you want to do. This topic warrants another post so I won’t go into depth right now.
IMPORTANT NOTE: If you wish to do some MIME processing on a document you must tell your NotesSession NOT to convert MIME to RichText when it opens a document. This must be done before you retrieve your document. e.g.
Document doc = db.getDocumentByUNID(someunid);
// YOU CAN NOW HAVE FUN WITH MIME
MIMEEntity me = doc.getMIMEEntity();
If you don’t do this, it will look to you as if the MIME was actually stored as RichText (the format) but what is really happened is that Notes is doing a sneaky conversion when you open the document.
This is a NotesSession-wide setting, so it is good practice to check what this setting was before you modify it. That way you can set it back to what it was before you changed it. e.g.
boolean wasConvertMime = session.isConvertMIME();
// Make sure it won't convert for us
//DO YOUR MIME THING
// Set it back to whatever it was
One last point, is that if you do any manipulation of MIME Entities (we will cover in another post), when you are finished you should use the closeMIMEEntities method to tell the document you are finished mucking around. This method has a ‘saveChanges’ parameter which indicates whether you want to keep what you have done, or discard whatever you were doing with MIMEEntities. By default it is ‘false’ or discard. This ‘savechanges’ is an in-memory process, e.g. it just means apply the changes to the document you have open. You still need to save the document to make the changes permanent.
MIMEEntity me = document.getMIMEEntity();
//DO YOUR MIME PROCESSING
//Apply the changes to the document
// Save changed document for good
I feel MIME is one of those things that is complicated and simple at the same time. Whenever I have to solve some MIME related problem, I know the answer will be simple, it is usually one line of code in the wrong place, or one bad MIME part header. But, the steps involved to debug that ONE change, are often complicated, and require a good understanding of what is actually going on both in MIME-land and in Domino-land. I hope this post has helped clarify at least some of it for you, please let me know if you have any questions (or if I have made a mistake somewhere)!
Apr 20, 2016
| Recent Blog Posts
Bundle Inspector – Diagnosing XPages Plugin Resolution problems|
Wed, Aug 2nd 2017 12:26p Cameron Gregor
A common task with XPages development is to installing some plugins that provide additional functionality.
If you do any plugin development of your own, you end up doing this task a lot more as you create new plugins, install new versions etc.
It is also common to encounter some problems after installing plugins to your Domino Server! Maybe your expected plugin doesn’t load up at all? maybe the wrong version is loading?
The standard method of diagnosing these problems is to use the OSGi c
Swiper Official Version 2 Release|
Fri, Jul 21st 2017 1:52p Cameron Gregor
So I have finally posted Swiper version 2 to OpenNTF! (and Github of course)
Swiper OpenNTF Project Page
Swiper Github Releases
The latest version is 2.0.1, and is the same as 2.0.0beta but with a bug fix for the toolbar buttons.
Swiper 2.0.0 beta has been available on the Github project site for a few months, and the core functionality of it works as planned. There was however a bug within the shortcut buttons that I added to the menu bar, and this is a bit of a nasty bug that can cause del
Auto-width Bootstrap Column XPages Controls|
Tue, Mar 21st 2017 1:13p Cameron Gregor
I’ve been stuck working with OneUI Version 3 for the past couple of years, due to a regretful decision made at the beginning of my major project. OneUI was better than nothing but very frustrating at times.
Finally, I have moved on to my next project and I am now using bootstrap (version 3)
A common task when laying out a page using bootstrap is to divide sections up into rows and columns, and use the appropriate css styles to do so. I’m going to assume you are familiar with bo
Swiper FP8 Version Beta Release|
Thu, Mar 16th 2017 12:50p Cameron Gregor
Last week I released the ‘alpha’ version of Swiper which was untested on FP8 but presumed to be ok. So far I have only had good reports from the pioneers who have gone ahead and installed FP8 + the alpha version.
I have since managed to ugrade my home office setup to FP8 which unfortunately has broken my ability to launch designer from eclipse but I am seeking some advice on fixing this up.
In the meantime I have to test the slow way of building plugins, import plugins, restart R
Swiper FP8 Integration Rollout|
Wed, Mar 8th 2017 12:43p Cameron Gregor
Notes Domino 9.0.1 FP8 is finally here and as far as I know (I have yet to download it) it includes the necessary changes which will allow Swiper to swipe whatever it wants, whenever it wants, which is good news for people who don’t like to have ‘Build Automatically’ turned on.
Plan of Attack for release of Swiper version 2.0.0
So, I haven’t actually downloaded FP8 yet, so I can’t say for sure that the updated version works perfectly. Here is a bit of backgrou
Markdown XPages UIControl|
Sun, Mar 5th 2017 11:44a Cameron Gregor
Often when I’m designing an xpage, there might be a section of the page in which I want to explain some instructions to the user. Some options here are to:
write the Instructions using html and embed directly in the xpage markup
write the Instructions directly in the design pane and format using designer’s ui e.g. bold, color, size etc
use some native xpage controls to achieve the desired result.
Write the instructions in a richtext field on a notes document that is loaded to dis
TextDiff XPages control – For visual comparison of text|
Tue, Feb 28th 2017 12:00p Cameron Gregor
A few years back I stumbled across Google’s diff-match-patch project which provides some handy algorithms for text manipulation. At the time of discovery I was doing ‘classic’ notes development. Although I probably could have implemented something that worked in lotuscript with RichText or Mime, it wasn’t a priority at the time and I never bothered.
Since then, I have been doing mainly XPages, and now that I have been also doing a bit of XPages Control development. I was
XPages webmail – Using Mime Inspector to debug Mime|
Tue, Feb 14th 2017 11:12a Cameron Gregor
In a previous post in this series I did a bit of an overview on how MIME works. We also did a little bit about how MIME works in XPages + Domino land. With this knowledge in hand we can now start to analyse the different ways a ‘Pretty words, pictures and attachments’ can be stored in the document.
During development of the ‘XPages Webmail’ interface, I encountered many problems which could only be solved by investigating the MIME content in detail.
To help me do this, I
Pasting Images into XPages CKEditor|
Sun, Feb 12th 2017 10:00p Cameron Gregor
Programs like ‘Snipping Tool’ on Windows, are super useful for users to make a quick snapshot, do some quick markup on the image, paste into chat/email and send.
Unfortunately when using the default configuration of CKEditor in XPages (the inputRichText control), support for pasting images is not available for all browsers, and even for the ones that do support it, the images are only pasted as a PNG data URI. I have explained data URI images in a previous post, so check that out if
Preventing pasting of remotely hosted images in CKEditor|
Mon, Nov 14th 2016 11:21p Cameron Gregor
In the previous post, I showed how to prevent a user from pasting Images from the Clipboard into CKEditor. This post is of a similar nature but is designed to ensure that users don’t paste images with URLs to external / internal applications.
This post is part of my XPages webmail tips series, and addresses a problem where, a user copies and pastes some HTML that includes images, from a webpage and pastes it into CKEditor for a message that is then sent via email. The recipient is then una