Have you ever had to write an agent that looks at ALL of the documents in a database? As is usual with LotusScript/Java and the back-end classes in Notes, there's more than one way to do that.
Actually, there are (at least) 3 common techniques: 1) Database.AllDocuments 2) Get the "All Documents" view and iterate 3) Database.search("@all")
I used to always use (and recommend) Database.AllDocuments, which returns a DocumentCollection instance. It's very efficient, because it maps to an optimized C API call that retrieves a list of all NOTEIDs (as an IDTable) in the database. This translates to a LotusScript DocumentCollection instance very easily, and none of the document objects is actually "opened" until you access it from the DocumentCollection,
Then, somewhere along the way, I discovered that a change had been made to the behavior of this property. Originally (Notes 4.x), the code in the back-end classes implementation filtered out any deletion stubs in the IDTable resulting from the C API call. Again, this was an efficient operation, made possible by the fact that the NOTEIDs of deletion stubs all have a high bit set (0x80000000L). So you could tell the API to remove all values in the IDTable greater than 0x80000000L, and what was left would be all valid NOTEIDs.
At some point (not sure when, or why), this final filtering operation was removed. So now, if you use Database.AllDocuments, you might find that as you iterate through the document objects in the result set, some of those documents will be "invalid" or deleted. It can be hard to work around this in your code, as the DocumentCollection.getNextDocument() call does not usually work if the "current" document object is invalid. See here and here for some of the details.
I recently worked on a project where the speed of acquiring all the document objects in the NSF and iterating through them was really important. The AllDocuments/GetNth technique was too slow. So I thought I'd try iterating through the "All Documents" view instead (this particular project was processing dbs derived from the standard mail template). I thought it would be pretty fast because all the indexing would have already been done, and it was fast. The problem, however, was that it didn't really retrieve ALL of the documents in the database. Why? Here's the mail template "AllDocuments" view selection formula (the "($All)" view):
SELECT @IsNotMember("A"; ExcludeFromView) & IsMailStationery != 1 & Form != "Group" & Form != "Person"
So any document that has a field "ExcludeFromView" containing an "A", or any document with a Form field containing "Group" or "Person", or any mail stationery document would never show up in the "All" view, and your agent would never see it. For some purposes this would be fine, but not for the project I was doing at the time, I really needed ALL of the documents.
So I fell back on Plan C: do a Database.search using "@all" as the selection criterion. I thought it would be slow, because the search() function visits every document in the database to apply the @function selection formula. But I knew that it would also explicitly exclude any deletion stubs from the result set, and that was a major concern. It turned out that the search was not particularly slow, certainly it's faster than using DocumentCollection.GetNthDocument() on a very large (100K) result set. The performance overall was actually pretty reasonable, and since I know that the resulting DocumentCollection will not contain any deleted documents, I can avoid GetNthDocument and just use GetFirst/NextDocument.
So, there you have it. All 3 techniques "work" (with variations in behavior), you need to think carefully about which one you should use.
Sun, Jun 8th 2014 11:10a Bob Balaban Greetings, geeks! I came across this problem on a project I'm doing: given a RESTful API on a web application server, how do you "throttle" calls so that a given authenticated user can only make a certain number of calls (N) per time interval (W), where both N and W are configurable. The solution I came up with is basically a “Token Bucket” with a sliding time window. What surprised me when I got something working was how easy it really was to implement. The context in whi [read] Keywords: application
"Employee of the Quarter" (not)
Fri, Dec 27th 2013 2:20p Bob Balaban Greetings, geeks! This one is not a technical topic, so feel free to move on. In my work email today, I received a shotgun message announcing the current "Employee of the Quarter". The lucky designee receives preferential parking (near the building entrance) for 3 months. There was a long-ish description of how great the EotQ is, and why they were selected for this honor. Not to take away from someone who is obviously (from the email description) a very hard worker, get-it-done typ [read] Keywords: email
Some Interesting Tech I’ve Been Involved With Lately
Sat, Apr 20th 2013 8:00p Bob Balaban Greetings, geeks! Some of you have been wondering what I've been up to lately. No, not a stealth-mode startup, not a new API for Notes/Domino, nothing like that. In fact, I'm at about the 6-month mark in a whole new (to me) tech area: storage and storage management. It's one of those things that's a whole lot more complicated than you'd ever think, until you start to dig into it. Me, I thought a "disk" started and ended with that box on the end of a USB cable that you plug into a la [read] Keywords: domino
The Interesting Difference Between "Exception" and "RuntimeException" (Java)
Sat, Feb 2nd 2013 6:30p Bob Balaban Greetings Geeks! I hope everyone who attended Lotusphere IBMConnect '13 had a good time and has returned home safely. I didn't attend (again, I stopped going after the first 17), but I've been hearing that it was pretty good. I learned something new about certain types of exceptions in Java programming the other day. Here's the context: I had an Enum class with a few items in it I had a String that came from outside the program that I wanted to convert into an enum'ed constant Thi [read] Keywords: lotusphere
"Reflections on Reflecting Light" - Java program
Wed, May 9th 2012 6:10p Bob Balaban Greetings Geeks! Today it is my very great pleasure to introduce to you a new guest blogger: my son David. He has graciously allowed me to prevail upon him to write about his recent science fair project, titled "Reflections on Reflecteing Light". That may sound innocuous, but to quote one of my favorite movies, "That's no ordinary rabbit!". David took this project through our local high school's science fair, was invited to the regional fair, and then to the Massachusetts state science [read] Keywords: blogger
Happy 5th? Yeah, 5th
Tue, Apr 10th 2012 8:40p Bob Balaban Greetings Geeks! So. March 30, 2007 was my first blog entry. Evidently I missed my own 5th Blanniversary! :-( Better late than never, I suppose. In theory, anyway :-) I know I don't post here often, but I do hope that when I do post, it's worth reading. For some people, anyway. I'll quote from a nice person who commented on that first post: "energetic,funny,cute--that's what I feel about you from your special explanation about "In Theory".I think you must be very content while wr [read] Keywords: lotus
"Programming Domino With Java" - Final Flush
Mon, Jan 2nd 2012 3:10a Bob Balaban Happy New Year, Geeks! Here's the 7th (and final) installment of the book. Thanks again for all the positive feedback. The first installment can be found here The 2nd installment is here The 3rd is here The 4th is here The 5th is here and the 6th is here All of the book content (as is all of the content on this blog) is Copyright 1998 and 2011 by Looseleaf Software, Inc. You may not reproduce or distribute the book's content without permission from me. Some Caveats and ex [read] Keywords: domino