354 Lotus blogs updated hourly. Who will post next? Home | Downloads | Events | Pods | Blogs | Search | myPL | About 
 
Latest 7 Posts
IBM Connections wsadmin commands for newcomers
Wed, Mar 5th 2014 401
Writing command line scripts with node.js
Mon, Feb 17th 2014 252
Year in review 2012 (not a typo)
Tue, Dec 31st 2013 157
My IBM Connect 2014 sessions
Tue, Dec 31st 2013 157
Artikel om mine session på ISBG i maj
Sun, Dec 1st 2013 102
Dreaming of Japan
Fri, Nov 29th 2013 119
Terminology is the most important thing to know when when starting out with WebSphere Application Server
Fri, Nov 29th 2013 310
Top 10
IBM Connections wsadmin commands for newcomers
Wed, Mar 5th 2014 401
Terminology is the most important thing to know when when starting out with WebSphere Application Server
Fri, Nov 29th 2013 310
Writing command line scripts with node.js
Mon, Feb 17th 2014 252
My IBM Connect 2014 sessions
Tue, Dec 31st 2013 157
Year in review 2012 (not a typo)
Tue, Dec 31st 2013 157
Java in Notes/Domino Explained: On Java Security and how it relates to Notes/Domino
Thu, Jun 20th 2013 129
List of OpenSocial fixes in 9.0.1?
Tue, Oct 29th 2013 119
Dreaming of Japan
Fri, Nov 29th 2013 119
Trusting certificates in WebSphere Application Server
Fri, Nov 29th 2013 116
New plugin appdev values for Notes 9.0.1
Tue, Oct 29th 2013 114


A tale from a customer reaching (and exceeding) the 64 gb limit
   

As I've tweeted I have spent the last couple of days (and the weekend) helping out a customer that exceeded the hard 64 gb database size limit in Lotus Domino. Before discussing how we solved the problem and got the customer back in business I would like you to think about how situations like this could be avoided. And avoiding it is key as once you exceed the size you're doomed.

First --- how and why database platform would EVER allow a database to cross a file size that makes it break. Why doesn't Domino start to complain at 50gb and make the warnings progressively harder to ignore as the database gets closer to 64gb. Why doesn't it refuse data once it reaches 60gb? I find it totally unacceptable that a software product allows a database to exceed a size it knows it cannot handle.

Now I know that there are considerations for such a warning and that it could be done in application code (e.g. database script, QueryOpen event) but it really isn't something an application developer should think about. Also it should be applied to backend logic as well and really doesn't lend itself to a UI computation. I also know that DDM or similar could warn about it but it still doesn't change my stance. The 64gb limit is a hard limit and reaching, and exceeding it, shouldn't depend on me configuring a specific piece of functionality.

Second -- having the option of keeping the view index in another location/file than the database would have helped. This has been brought up a number of times including at Lotusphere Ask-The-Developers sessions. One could argue that externalizing the view index from the database would just have postponed the problem but the view index takes up a substantial amount of disk for databases of this size.

Now on to how we saved the data.

The bottom line in this is that the customer was lucky. VERY lucky. The customer uses Cisco IP telephones and keeps a replica of the database in question on a secondary server for phone number lookup using a Java servlet. Due to the way the way the servlet is written only as single, very small, view was built on the secondary server. This is turn meant that the database that had exceeded 64 gb on the primary server was "only" 55 gb on the secondary server. The database on the primary server was toast and gave out very interesting messages if attempting the access or fixup the database:

**** DbMarkCorruptAgain(Both SB copies are corrupt)
So thank God they had the secondary server otherwise the outcome of the story would have been less pleasant because using the secondary server we were able to:
  1. Take the database offline (restrict access using ACL)
  2. Purge all view indexes (using Ytria ViewEZ)
  3. Create a database design only copy to hold archived documents
  4. Delete all views to avoid them accidentally being built
  5. Build a very simple view to prepare for data archiving
  6. Write a LotusScript to archive documents (copy then delete) from the database
  7. Use Ytria ScanEZ to delete deletion stubs from the database (this works for them because the database isn't replicated to user workstations or laptops)
  8. Do a compact to reclaim unused space
  9. Make the database available on the primary server
Whew! They are now back in business after building views in the database. They were lucky - VERY lucky. If they hadn't had that secondary replica the data would probably have been lost to much distress. To them and me.

So what are the main take aways from this?

  1. UI check -- in the future all databases that I develop will have a database script check on the database size to try and prevent situations like this
  2. DAOS -- enable DAOS for databases to keep attachments out of the database and keep the size down
  3. Monitoring -- monitor databases either using DDM or other tools to try and prevent sitations like this

And so concludes a story from the field. 4 days later where my hair have turned gray from watching copy/fixup/compact progress indicators the customer is back in and happy once again. Whew!!



---------------------
http://lekkimworld.com/2011/05/18/a_tale_from_a_customer_reaching_and_exceeding_the_64_gb_limit.html
May 18, 2011
45 hits



Recent Blog Posts
401


IBM Connections wsadmin commands for newcomers
Wed, Mar 5th 2014 3:05a   Mikkel Heisterberg
I've updated my IBM Connections wsadmin commands for newcomers page for IBM Connections 4+ and added a couple of new commands. I've also added information on how to easily work with wsadmin from the command line on *nix. Comments are very welcome. [read] Keywords: connections ibm
252


Writing command line scripts with node.js
Mon, Feb 17th 2014 11:20p   Mikkel Heisterberg
Found this little tip this morning to make it easier to use command line scripts written in node.js. Instead of having your node.js file(s) and invoking it using "node myfile.js" on the Mac you can simply do the following: At the top of the file as the first line add: #!/bin/usr/env node Make the file executable using chmod +x myfile.js Invoke away Now the file is usable by simply using myfile.js. [read] Keywords: mac
157


Year in review 2012 (not a typo)
Tue, Dec 31st 2013 5:12a   Mikkel Heisterberg
Boy 2013 was a busy year. In fact it's been so busy and I have been so bad at blogging that I never got around to finish my year end review for 2012. In a draft blog post I had the following: "2012 was a busy year - maybe the busiest year I've had in a long time. Besides numerous customer projects here in Denmark I've also been involved in a number of international projects and traveled more than ever before. I went to the US twice, Japan twice, Australia once, and to too many European cou [read] Keywords: connections domino ibm lotus lotusphere notes blogging
157


My IBM Connect 2014 sessions
Tue, Dec 31st 2013 4:54a   Mikkel Heisterberg
It's been a very busy fall and christmas for me so I haven't bragged about being chosen to speak at IBM Connect 2014 on my blog besides creating a new static page for the event that will - eventually - sum up what I'm up to at the event. I am fortunate enough be have been selected to speak in two sessions - one with my good buddy Mat Newman (aka Yellow Man) and one solo. Below are the session IDs (probably subject to change), the session titles and the abstracts. BP301 An Introduction to W [read] Keywords: connections ibm xpages applications development java javascript
102


Artikel om mine session på ISBG i maj
Sun, Dec 1st 2013 11:53a   Mikkel Heisterberg
Activity Stream og Widgets på ISBG [read] Keywords: widgets
119


Dreaming of Japan
Fri, Nov 29th 2013 1:51p   Mikkel Heisterberg
[read] Keywords:
310


Terminology is the most important thing to know when when starting out with WebSphere Application Server
Fri, Nov 29th 2013 4:41a   Mikkel Heisterberg
Over the last few weeks I've done a fair amount of consulting on IBM Connections - not so much the install and technical stuff but more simply talking to customers about WebSphere Application Server (WAS) and how it works. The single thing that people new to WAS seems to struggle the most with is the terminology and getting the overall architecture in place. Once that's done most people actually like the platform and find it nice to work with. A while back I linked to a PDF containing a nice g [read] Keywords: administration connections ibm lotus application applications consulting java linux profile server websphere
116


Trusting certificates in WebSphere Application Server
Fri, Nov 29th 2013 3:33a   Mikkel Heisterberg
If you make SSL connections from a WebSphere Application Server based application the server (or rather the cell) needs to trust the certificate of the server you are connecting to. This is very easy to do in WAS and is easily done using the Integrated Solutions Console (ISC). The way to establish the trust is as follows: Log into the WebSphere Application Server Integrated Solutions Console (ISC) From the lefthand navigator select Security/SSL certificate and key management In the list of rela [read] Keywords: connections application properties security server websphere
92


IBM Notes and Domino 9.0 Application Development Elective Exam is now available
Wed, Nov 20th 2013 3:00a   Mikkel Heisterberg
"General availability of the following exam has been announced: LOT-411: XPages Mobile Advanced Topics. Follow the link to read the announcement for this test, which includes links to the exam overview, competency, and preparation pages. Information on the corresponding accreditation - IBM Certified Advanced Application Developer - Notes and Domino 9.0 - including recommended prerequisite skills can be found on the certification page. " LOT-411: XPages Mobile Advanced Topics [read] Keywords: domino ibm notes xpages application development mobile




108


Servers upgraded to 9.0.1 and what a difference a point release makes
Thu, Nov 14th 2013 2:04a   Mikkel Heisterberg
Yesterday night I upgraded two of our internal Domino servers to Domino 9.0.1 incl. Traveler and the new Social Edition components. The entire process for two servers took just about 15 minutes over remote desktop and VPN incl. the restart of the server which strictly speaking was superfluous. Pretty impressive but hey it's Domino!! The upgrade was from Domino 9.0.0 and what a difference a point release makes. Not for the Traveler server but for the Social Edition server that runs the embe [read] Keywords: connections domino ibm traveler desktop email office server twitter




Created and Maintained by Yancy Lent - About - Blog Submission - Suggestions - Change Log - Blog Widget - Advertising - Mobile Edition