nsftools.com - The March 2006 Blog

Following up on yesterday's blog entry about programming languages and the need to have them available to you as tools, here's an excerpt from Stephen King's excellent book On Writing:

And when Fazza retired, it was Uncle Oren who inherited the old fellow's toolbox... The toolbox was what we called a big 'un. It had three levels, the top two removable, all three containing little drawers as cunning as Chinese boxes. It was handmade, of course. Dark wooden slats were bound together by tiny nails and strips of brass. The lid was held down by big latches; to my child's eye they looked like the latches on a giant's lunchbox...
One summer day I helped Uncle Oren replace a broken screen on the far side of the house. I might have been eight or nine at the time. I remember following him with the replacement screen balanced on my head, like a native bearer in a Tarzan movie. He had the toolbox by the grabhandles, horsing it along at thigh level...
We finally reached the window with the broken screen and he set the toolbox down with an audible sigh of relief. When Dave and I tried to lift it from its place on the garage floor, each of us holding one of the handles, we could barely budge it. Of course, we were just little kids back then, but even so I'd guess that Fazza's fully loaded toolbox weighed between eighty and a hundred and twenty pounds.
Uncle Oren let me undo the big latches. The common tools were all on the top layer of the box. There was a hammer, a saw, the pliers, a couple of sized wrenches and an adjustable; there was a level with that mystic yellow window in the middle, a drill (the various bits were neatly drawered farther down in the depths), and two screwdrivers. Uncle Oren asked me for a screwdriver.
"Which one?" I asked.
"Either-or," he replied.
The broken screen was held on by loophead screws, and it really didn't matter whether he used a regular screwdriver or the Phillips on them; with loopheads you just stuck the screwdriver's barrel through the hole at the top of the screw and then spun it the way you spin a tire iron once you've got the lugnuts loose.
Uncle Oren took the screws out -- there were eight, which he handed to me for safekeeping -- and then removed the old screen. He set it against the house and held up the new one. The holes in the screen's frame mated up neatly with the holes in the window-frame. Uncle Oren grunted with approval when he saw this. He took the loophead screws back from me, one after the other, got them started with his fingers, then tightened them down just as he'd loosened them, by inserting the screwdriver's barrel through the loops and turning them.
When the screen was secure, Uncle Oren gave me the screwdriver and told me to put it back in the toolbox and "latch her up." I did, but I was puzzled. I asked him why he'd lugged Fazza's toolbox all the way around the house, if all he'd needed was that one screwdriver. He could have carried a screwdriver in the back pocket of his khakis.
"Yeah, but Stevie," he said, bending to grasp one of the handles, "I didn't know what else I might find to do once I got out here, did I? It's best to have your tools with you. If you don't, you're apt to find something you didn't expect and get discouraged."
I want to suggest to you that to write to the best of your abilities, it behooves you to construct your own toolbox and then build up enough muscle so you can carry it with you. Then, instead of looking at a hard job and getting discouraged, you will perhaps seize the correct tool and get immediately to work.

Even though he's using the toolbox as a metaphor for the art of writing, it applies equally well to the art of programming.

And the On Writing book is an excellent overall read, by the way. It's not the terrifying fiction that Stephen King is known for; rather, it's a memoir of the things in his life that formed him as a writer, and some great discussions on how to write a story. Definitely recommended reading.

Boy oh boy, this whole "why should Lotus Notes developers learn Java" meme has been interesting to read. The quick recap so far:

Jack Dausman asks Is IBM Ready to Get Us Ready for Hannover? and says "I haven't read anyone talking about what it will take to upgrade the skills of Domino professionals into Eclipse."
Mikkel Heisterberg followed up with his own question: Is the lack of Java skills in the Notes/Domino developer community the Achilles' heel of IBM?
Then Ed Brill picked up the theme and got about a bazillion comments on the subject, with all sorts of opinions represented.
Follow up blog entries by Mikkel, Ben Poole, Brendon Upson, and probably lots of others I'm not thinking of right now

Sadly, I was on vacation while all this was going on, and I'm just now starting to catch up on the action.

It looks like a lot of the discussion took some strange turns and went way off-road, with talk of J2EE and bytecode generation and general "my platform is so much better than your platform" arguments.

Here's my take on using Java in Lotus Notes:

You should use Java when it's the right tool for the job. You should learn it so you know how to use it when you need it.

That's it in a nutshell. No esoteric arguments about the beauty of Object Oriented programming or unit testing or whatever. Java is a tool, just like any programming language. When you're comparing LotusScript to Java, you're not comparing two brands of hammers to figure out which one is best. You're comparing a hammer and a wrench. There are certain nails you can hammer into a wall with a wrench, and certain bolts you can pry out with a hammer, but in general you should be using the right tool for the job.

For another way to put it, I'll shamelessly requote something that Jeff Atwood posted recently, which is from Alex Papadimoulis' blog:

"A client has asked me to build and install a custom shelving system. I'm at the point where I need to nail it, but I'm not sure what to use to pound the nails in. Should I use an old shoe or a glass bottle?"
a) It depends. If you are looking to pound a small (20lb) nail in something like drywall, you'll find it much easier to use the bottle, especially if the shoe is dirty. However, if you are trying to drive a heavy nail into some wood, go with the shoe: the bottle with shatter in your hand.
b) There is something fundamentally wrong with the way you are building; you need to use real tools. Yes, it may involve a trip to the toolbox (or even to the hardware store), but doing it the right way is going to save a lot of time, money, and aggravation through the lifecycle of your product. You need to stop building things for money until you understand the basics of construction.

I really don't want this to degrade into a discussion of "Lotus Notes itself is the wrong tool, because you should be using .NET/J2EE/Ruby/whatever" either. Because then you're getting into a discussion of frameworks, not languages. Languages are what we're talking about here, and every language has strengths and weaknesses. If your language happens to be weak in a situation where another language is strong, you should consider using the stronger language -- as long as your framework allows it. And that's one of the nice things about Lotus Notes, that you can choose between LotusScript and Java and JavaScript and Formula language and even C or C++. Use the one that best suits your needs.

As far as determining the situations where Java is a better choice than LotusScript, I outlined several cases over 3 years ago: times when you need to use threads or sockets or the vast array of external libraries that have been written in Java, for instance. I recently put some working examples in the Java Jumpstart samples database from my and Tom's session at Lotusphere. Take a look at the modal dialog box with a progress bar or the 3-D pie chart, and tell me how you would reasonably do that with LotusScript.

Of course, I'm completely ignoring the argument that we should be learning Java and Eclipse to get ready for Hannover. I think it's a great point and it's absolutely true, but unfortunately it's one that's lost on most people who are already swamped in their current jobs and have a hard enough time keeping up with the here and now, rather than planning even 2 months down the road. I can appreciate that; I think we all can.

Do I think that IBM needs to do a better job at actively pushing Notes/Domino developers into Java and Eclipse (which was Jack Dausman's original point)? Absolutely! And not the "why don't you learn Workplace?" kind of pushing, because that's jumping way too far forward. Maybe more official training classes on using Java and/or Eclipse with Notes, some webinars on cool stuff you can do right now in your current Notes environment, or -- if IBM wants to keep Workplace on the screen -- even good examples of how to integrate Notes and Workplace today. Heck, maybe they could make the Lotusphere Java Jumpstart session that Tom and I did a freely downloadable video! ;-)

On the other hand, I also think that Notes developers need to take their own initiative and plan for the future that they can see coming. See Rich Schwartz's latest blog entry for some perspective on accepting change, including his great Bob Dylan quote, "admit that the waters around you have grown... you better start swimmin' or you'll sink like a stone."

Yes, you'll be able to natively use LotusScript in Hannover, just like you can now. Just like you can still go to Blockbuster and rent VHS tapes instead of DVDs. The old technology is still there, and it still works, but shouldn't you at least start looking at some of the new technology? DVD players are cheap, and you don't have to get rid of your VCR or old tapes, because you actually can have both. And if you have both, then you can borrow your co-worker's DVDs as well as your Mom's old tapes and watch either one.

It's not an either-or question, it's a matter of choice. But you don't really have choice if only know how to use one tool.

Via the Datamation IT News Blog, I see that the IE7 team is setting up a Bugzilla-type site for reporting problems with IE7. They're actually calling it the Internet Explorer Feedback program, but it's the same concept as Bugzilla. Or maybe BugzillIE.
;-)

Not that I have a problem with that. I think it's a great idea to have a way for people to report and track bugs in a product. The Bugzilla model is a good one.

If you're not familiar with the Bugzilla product, here's the "About" page that you can click through to browse around. Two working implementations I can think of off the top of my head are at Mozilla and WineHQ.

Bugzilla normally has kind of a plain, ugly interface though. It'll be interesting to see how Microsoft ends up managing their look and feel (since they've got some good UI designers in-house) and whether a nice-looking Microsoft interface might spur a more active community of Bugzilla skinners who can make the pages and forms seem a little less raw.

Then again, maybe there already is an active Bugzilla skinner community out there. A quick Google search didn't turn up any promising leads, but that doesn't mean anything.

UPDATE: slightly feisty discussion about all this on Asa Dotzler's mozillazine blog.

I've mentioned before that I think Daylight Saving Time is weird, especially when you look at how it's implemented (or not) across various parts of the world. But then, saying DST is weird is about as revolutionary a thought as saying that traffic is annoying, so I'm not going to rant about it here.

Rather, I'd like to point out that the state of Indiana is finally trying to straighten out it's extra-weird DST situation. Come April 2nd, the entire state will be on Daylight Saving Time.

That may sound unbelievably trivial, but the current/old setup was that part of the state is in the Central time zone and part is in the Eastern time zone and some of the state recognized DST but not all of it. I had a friend who was a salesperson in the Indianapolis area, and he said it was really hard to keep appointments straight if you had clients all over the state.

The new setup still has the time zone split, but everyone will finally be on DST together. Here's some information from a recent IBM technote (and it's as good a summary as any I've seen):

State of Indiana time zone changes beginning April 2, 2006
http://www-1.ibm.com/support/docview.wss?rs=463&uid=swg21232886
Prior to April 2, 2006, Indiana has had a complex time zone fabric consisting of three different time zones, with Daylight Saving Time observed in some, but not all areas.

The northwestern corner of the state, close to Chicago, Illinois, follows Central Standard Time (CST) and observes Central Daylight Time (CDT) in the summer months. The same is true of the southwestern tip of the state, near St. Louis, Missouri.

A few counties in the southeastern part of the state follow Eastern Standard Time (EST) and observe Eastern Daylight Time (EDT) in the summer months.

The majority of the state, including the capital, Indianapolis, follows what the Windows operating system calls the "Indiana (East)" time zone, which is GMT -5 (Eastern Time Zone) but does not observe DST. This means that in the winter months, Indianapolis has the same time as the states to the east of it (Michigan, Ohio, and the US East Coast), while in the summer months, Indianapolis has the same time as the state to the west of it (Illinois), so that they are an hour behind the East Coast.

Upcoming changes to Indiana time zones and DST:
In April 2006, the central part of the state, here referred to as "Indianapolis", will change permanently from the "Indiana (East)" time zone to Eastern Time. This portion of the state will remain in GMT-5, but will start observing DST in the same way as the eastern states and a few southeastern Indiana counties already do. They will join DST for the first time on April 2, 2006, when most of the US changes to DST. The change in Indiana is a permanent change of time zone.
For more information, refer to the following map of Indiana counties depicting the time zones as of April 2, 2006:
http://www.in.gov/sic/pdfs/TimeZones.pdf

Of course, in 2007 the DST schedule for the entire USA will change, so we Americans will have plenty more fun with this stuff next year.

Oh, and on a related note Australia is delaying the ending of their Daylight Saving Time schedule by a week this year to accomodate the Commonwealth Games, but if that affects you and you didn't know it then you've probably already experienced any problems you're going to experience (because this week is the extra week).

A few days ago I posted about the fear that LSI_Info wasn't a safe call, which was of specific importance to me because my OpenLog database uses this function to return information about LotusScript method names and stack traces.

I've sent a few e-mails back and forth to the author of the LDD forum post that started this whole thing -- he was very nice, and gave me a lot of good information -- and I found the following:

The server in question was crashing a few times a week, somehow related to a web-enabled Notes database
DECS, LC LSX, Oracle, Domino.Doc, and Lotus Workflow were potentially involved
After looking at some log files, IBM noticed the use of the LSI_Info function and recommended removing it
They removed the OpenLog code and applied some LC LSX hotfixes, and the problem went away

So... even though I realize there are a LOT of people using the OpenLog code without problems, if anyone is still concerned about the LSI_Info function, I am making available a modified version of the OpenLog template:

DOWNLOAD: OpenLog1_1Beta.zip

I have removed most of the calls to LSI_Info (where they could be replaced with calls to GetThreadInfo), and added a global flag called NoLSIStackTrace that you can set to "True" if you want to disable the remaining instance of LSI_Info (the one that gets the stack trace). I've also added some code that should allow OpenLog to log errors to Domino Domain Monitor (DDM) in Notes 7.

There are a lot of other changes I have on my list, but those are likely to make it to a 2.0 release because they'll require more testing -- and I really want to make sure I don't break anything, because I know that a lot of people are relying on this in production environments. I've had some great suggestions from users of the database, but backwards-compatibility is kind of tricky sometimes. This release is really about getting rid of LSI_Info for people who are concerned.

I'm going to float the new template out there as a 1.1 BETA version, if anyone is immediately worried about the LSI_Info calls (or if you just want to play with the DDM logging). There are also a few minor bug fixes. After I've had a little more chance to test it and get some feedback, I'll post a non-beta version to the OpenNTF site as an "official" download.

Please note that this is a copy of the current version of OpenLog, not a replica, so it shouldn't overwrite anything if you move it to your server. Also, I've changed the template name to "OpenLog 1.1 BETA", so you'll need to make that adjustment as well. Most likely, your upgrade process will be:

Copy the database to your server
Sign everything with an appropriate ID
Set your current OpenLog database to inherit from the template "OpenLog 1.1 BETA"
Manually refresh the design on your current OpenLog database, and then run the Design task on the server to upgrade all the downstream databases

Good luck, and please test this on a development server first!!! If you have any questions I'll be out of pocket for much of next week, but I'll check e-mail and comments on this entry as much as possible.

This week's Show and Tell Thursday installment discusses secure connections to your Domino server.

Beyond the strong out-of-the-box aspects of Lotus Notes security -- databases ACLs, Readers and Authors fields, built-in PKI and encryption options, etc. -- there are a lot of interesting things you can do to secure your interaction with "the outside world". Here are a few:

To start out with, you can encrypt your Notes client traffic, end-to-end. This might be useful if you're accessing your server from outside the network and you can't use a VPN for whatever reason. All you have to do is enable network encryption on the server, and on your client go to File - Preferences - User Preferences, then the "Ports" tab, choose TCPIP and click the "Encrypt network data" box (see screenshot below - click for a larger image).
For web browser connections, you can use SSL/HTTPS. The Domino Admin Help file has good step-by-step instructions. It looks kind of complicated, but it's really not.
You can either self-certify your server (which means the users will get prompted with an "are you sure you trust this certificate?") or you can buy one from a registrar whose root certificates are already included with modern browsers. Some of these (like Verisign and Thawte) are very expensive, but you can also get a GoDaddy certificate for very little money -- it's recognized by all modern browsers and IBM even has a Technote describing how to use a GoDaddy SSL certificate. I've used one, and it worked perfectly.
For server-to-server SMTP connections, you can enable TLS. TLS is essentially SSL for SMTP traffic, so no one can sniff your SMTP transactions. It even uses the same SSL certificate you use for HTTPS. Chris Linfoot recently wrote a terrific article on how to set up TLS on a Domino server, and that will give you all the steps you need.
For encrypting external e-mails, you can set up S/MIME. S/MIME is a standard way to encrypt message contents, and it's a public/private key technique much like PGP. However, it's built-in to many e-mail clients (like Notes) so you don't necessarily need to buy or install any additional software.
There are several steps involved in the setup process, but it's well documented in a few places. For example, here's a short set of instructions, and here are some longer instructions with a lot more detail. In Notes 7 you can even use S/MIME from iNotes/DWA, and there's an API interface so you can call the encryption routines from your own programs.

Those are the things I can think of off the top of my head. I'm sure there are more, but that will at least whet your whistle.

UPDATE: Per Henrik Lausten mentioned that you can also have SSL-encrypted IIOP and LDAP connections. I never knew that. So then I checked the Domino Admin Help again, and saw that all of these protocols can be configured for SSL:

Web server and Web Navigator (HTTP)
Internet Inter-ORB Protocol (IIOP)
Internet Message Access Protocol (IMAP)
Lightweight Directory Access Protocol (LDAP)
Post Office Protocol 3 (POP3)
Simple Authentication and Security Layer (SASL)
Simple Mail Transport Protocol (SMTP)

Coolness. And again, it's all built-in to the product.

technorati tag: Show-n-tell thursday, SnTT

In a message on the OpenNTF site today, I was referred to a recent LDD post where a person said that IBM told him that LSI_Info was an unsafe LotusScript call, and more specifically that he should stop using the OpenLog database.

Here's the whole post, as it exists in this moment in time:

For those who use OpenLog or have written code with LSI_Info:
IBM has recently told us that our open source database OpenLog that calls the function LSI_Info is not thread safe. LSI_Info is an internal debug code and cause memory issues such as heap corruption.
We have had to remove the OpenLog code from our Domino application to ensure its reliability.
I am hoping that IBM/Lotus will provide a thread-safe solution because we do rely on this code.

All I can say is that I've talked to a lot of people who have included the OpenLog code in a lot of databases, and I've never heard of any kind of memory issues. Which isn't to say that they don't exist (they may and they may not), simply that it doesn't correspond with any anecdotal evidence that I'm aware of.

I did a search in the Lotus Technotes and the Domino Fix List and saw nothing related to the possible problem.

Also, the LDD post never said whether there was actually a memory problem in the first place. It only says that they removed the OpenLog code "to ensure [the database's] reliability". Which may mean that this was a theoretical problem rather than a real one. Because Normunds Kalnberzins asked for more specifics in a follow-up post and didn't get any, I went ahead and e-mailed the author of the post to try to get some additional information.

If any of my IBM buddies out there could ask around for me, I'd appreciate it. If this really is a problem, I'd like to try to work around it. If not, I'd rather not mess around with code that's not broken.

In other words, I want to make sure it's not like when Lotus Support told Mikkel Heisterberg that there was a 100MB recommended limit on Notes mail databases (which Ed Brill quickly determined was completely incorrect, as experience told all of us it was).

A few days ago, Bill Buchan asked for some Show and Tell Thursday success stories. Just today, I had one of my own.

I was asked about TLS (think https/SSL for e-mail), and whether or not Domino supports it. I thought to myself, "Self, didn't Chris Linfoot just post something about that?". Why yes he did. In fact, Chris Linfoot actually posted step by step instructions for setting up SMTP TLS on Domino for his most recent SnTT entry!

That's some good stuff right there.

I've seen a few other people link to this, but I'll throw my hat in the ring too.

John Dvorak just wrote a piece called Web 2.0 Baloney. Even if you don't read the whole thing for some reason (it's really not that long), at least read the last page, where he writes:

As I see it, the impetus behind Web 2.0 is the drive to make the do-it-yourself Internet more efficient. And this is what is mostly overlooked by the folks who hope to create a new dot-com boom by promoting excitement about Web 2.0... They don't get it. The latest buzzwords just capture an idea that has been passed along for decades: "If you want something done right, do it yourself." Welcome to Web 2.0. You're on your own!

While I wouldn't agree that this sums up my vague understanding of Web 2.0, I think it's a point that's very well said. Greasemonkey, Web APIs, mashups, aggregation, tagging -- essentially a user-controlled web experience. Bringing all the distributed data together in whatever way makes sense for you. That ends up being a very powerful idea.

Bruce finished his post-production work on the Taking Notes podcast with Ed Brill this weekend, and it really turned out well.

It did end up being a little long, but for any of you ADD-types out there who may have trouble focusing on anything that's more than about 5 minutes in duration, it really is worth listening all the way to the end. In the last 15 minutes or so (maybe more), Bruce and I went through some of the "questions from the audience", and there's good information there. A few little tidbits I'll point out for you:

There should be a Hannover beta (maybe public, maybe not) sometime this year
There will be a non-Eclipse-based Notes client released in conjunction with the Hannover client (first I had heard of this)
JSR-170 will be very important as a means of accessing server-based document stores from the Workplace Rich Client

Otherwise, please download and listen at your leisure.

This week's Show and Tell Thursday installment deals with reading CSV (comma-delimited) files in LotusScript.

There are 3 common ways to deal with reading CSV files:

Use the LotusScript Input function to read each line as a variable length record, as long as you know exactly what format all the data elements are in, and what data types they all are, and how many elements are in each line
Treat it as an ODBC data source with the Microsoft Text Driver, as long as all the machines you're running on are Windows, and they all have that driver installed, and you've created a System DSN with that driver (or are using ADO calls and DSN-less connection strings)
Parse each line of text manually and pull out what you need

While all of these techniques work, I'd like to offer up a fourth option. About a year and a half ago, without much announcement, I put the CSVReader class on my Notes Tips page. It was sort of a side note to accompany some code that converts CSV files to XML, but I've actually found that I've used the CSVReader code a fair bit (CSV to XML was something I played with, but in practice the XML parsing was too much overhead).

Here's some example code, to let you see how it works:

Use "CSVReader Script Library"

Dim fileName As String
Dim headerArray As Variant
Dim dataArray As Variant
Dim reader As CSVReader

fileName = "C:\testdata.csv"
Set reader = New CSVReader(fileName)

'** for our example file, the first non-blank line is a set of
'** column headers
headerArray = reader.getNextLine
Do Until reader.isEOF
	dataArray = reader.getNextLine
	If Not Isnull(dataArray) Then
		Print headerArray(0) & " = " & dataArray(0)
	End If
Loop

That's pretty much it. Normally you'd be doing some ForAll loops or something, but that's the basic structure. By default the delimiter is a comma and the quoted string character is a quote, but you can change those at runtime if you need to (like if it's a tab-delimited file). Blank lines are skipped and delimiter characters within quoted strings are handled properly.

In my experience, comma-delimited files are at least as common as XML for a "neutral" text data format, sometimes more common. So maybe this will help you out a little if you need to accept data from an external non-database system (like an Excel spreadsheet :-).

technorati tag: Show-n-tell thursday