nsftools.com - The January 2004 Blog

Apparently the dbActivity command line program doesn't always work properly in ND6 and 6.5. Jerry Carter had told me before that he was getting some errors and crashes running it, but I could never figure out what was going on. Then yesterday Tony Campbell-Cooke started reporting some problems where he was getting no output at all.

I did a little more research, and it appears that the NSFSearch API function has problems in ND6 and 6.5. I'm using NSFSearch in that program to provide wildcard support for database names, so what's happening is that the database list that I'm searching against is empty, so you never get any matches, so you get no output. On other machines, this is probably what's causing the errors that Jerry is seeing.

There were a few posts in the LDD forum from a while ago describing the same problem, but apparently it hasn't been fixed yet. SPR THUR5LPPW9 in the Fix List database says that there's a problem with NSFSearch that's being fixed in 6.5.1, but I'm not sure if it's the same one. I also tried to run the old find_dbs program from the API toolkit on a machine with a 6.5 client, and that didn't work either (I got information about one database returned, but that was all).

For now, here's a link to an old version of dbActivity that doesn't use NSFSearch. This might work better for you as an interim solution. It doesn't allow you to use wildcards in database names, but it might be a temporary fix until I can try to figure out a workaround...

Like everyone else in the world, I get a lot of spam. My ISP has an upstream spam filter, and I've got a spam filter on my local machine, but my inbox is still littered with crap. As a typical example, my spam folder has 196 messages just from yesterday, and a large number of those I had to delete by hand.

Without getting into a discussion about spam filtering techniques or which tool is better than another, let me just say this: I have to delete enough junk e-mail by hand every day that I do it with rather wide swings of my blade -- which means I'm doing the manual filtering based on a quick look at the sender's name and the subject line, not by looking at all of the hundreds of messages individually.

As a result, I'm sure that I occasionally delete legitimate messages. If you've ever sent me an e-mail that I've never responded to, that's almost certainly what happened, and I apologize profusely. I'm often pretty slow at responding to e-mail, but I usually get around to it eventually. If you don't hear from me in a few weeks, just try sending me another message.

I thought about this because I happened to look at a message today that I normally would have deleted with reckless abandon. I'm not sure why I decided to look at it before I deleted it, but I did, and it was something I needed to respond to. The message was from someone who I've never had contact with before in my life, and it was someone whose name I did not recognize, and the subject line was simply: "Question".

Now, to the untrained eye, many people would say, "Oh my gosh, you thought of deleting that? That's so obvious that it's a legitimate question from a user who needs your help!" However, take a look at a sampling of subject lines from real spam messages that I received (again, this is just from yesterday):

"Hi"
"Nice"
"It works"
"Best site of the month"
"DO NOT DELETE - READ THIS"

Now think of 100 of those messages, duplicated randomly and packed into my inbox, along with the one that says "Question", none of which are from people with familiar names, and you can start seeing the picture through my eyes. Granted, my local spam filter caught many of these, but you can't count on that.

So I just want to say that I truly do enjoy hearing from all of you, and I really do like getting e-mail from people who have come across this site and share my interests. I think I'm a pretty friendly person and I try very hard not to be rude.

If you're going to send me an unsolicited e-mail though, please do your best to use a subject line that is descriptive. Please don't use subjects like "Hello" or "Help" or anything short and vague like that. If you can use a specific technical term in the subject (like "Java" or "LotusScript"), you will greatly increase the chances of getting a response to your message.

Again, I'm really not trying to be mean here. It's just that I have a job and a family and I don't have a lot of time to spend on this site most days, so I need a little help separating the wheat from the chaff. And if I've ever accidentally deleted one of your e-mails as a "false positive", then I'm really sorry. Please write again soon.

With all the Lotusphere talk that's going on, I forgot to tell you how the R5 to 6.5 server migration went 2 weekends ago (by the way, I know that version 5 was "R5", and version 6 was "ND6", but what's version 6.5 -- ND6.5?).

Everything went quite smoothly. I wasn't in charge of the server migrations themselves (we have a competent administrator who does such things), but I did have to make sure that the apps would continue to run. In retrospect, I've got a few things to add to my earlier comments about what to look for before you upgrade:

My initial plan included using DDSearch to find obsoleted bits of code, but it didn't occur to me that databases with hidden designs or code that %Includes LSS files wouldn't be searchable. As always, nothing beats a healthy dose of testing.
I found a Notes/Domino 6 Migration Knowledge Collection in the Lotus Knowledgebase that has a few extra things to watch out for.
I also found a technote describing the problem with uploads greater than 10 MB, which is important to be aware of. Rocky mentioned this recently too.
In ND6, the permissions of the signer of a script library will filter down to agents that use that script library, so make sure that you have everything signed properly.
And the biggest gotcha of all: beware the new security settings that restrict agent operations. By default, the runtime security level on some agents seemed to be set to "1. Do not allow restricted operations", which caused some scheduled agents to fail. After resetting the runtime security level to "2. Allow restricted operations", they worked fine. One thing I noticed though is that the CreateObject and GetObject functions seem to be restricted operations, although they are not listed as such in the Notes Help.

As I said, the biggest thing to watch out for after you migrate to ND6 and higher is the agent runtime security, because your agents can fail somewhat silently if this isn't set properly in ND6 and 6.5. All you'll see is that the agents don't seem to be running, and there might be some errors like this in the Console Log:

Error 201: Operation is disallowed in this session

Programmer that I am, when I discovered this, I knew I had to write an agent that checks the runtime security settings. So I wrote a LotusScript agent that checks all the agents in all the databases on a server (the ones you have access to, anyway) and spits out a comma-delimited file with information about the $Restricted field for the agents ($Restricted = 1 is level 1, $Restricted = 0 is level 2, $Restricted = 2 is level 3 -- if the field doesn't exist, I think it's treated as level 1). This code is available as my latest tip: Listing the ND6 Restriction Level for All Agents.

One nice thing about the code in this tip is that it can serve as a template for any other agent you want to write that searches though agents and script libraries in a database. The procedure is that it gets the NoteIDs of all the agents and script libraries and returns them in a List, so you can access them as regular Notes documents (and read and/or set their fields). Granted, you can also do this using Damien Katz's DatabaseDesign library, ~~but I think that his technique requires Designer access to the databases you're looking at, and I'm pretty sure that mine only requires Reader access~~ (see update below for a correction to that statement).

If you're feeling a bit wacky, you could easily modify this agent to set the $Restricted field to 0 for all scheduled agents, but that could be dangerous. Script hacker beware.

UPDATE: Okay, here are two clarifications to what I was saying about Damien Katz's DatabaseDesign library:

1. When I was remembering the library, I was thinking of the PlatformIndependentDatabaseDesign class, which creates temporary views and uses some interesting $FormulaClass manipulations to get collections of design documents. That particular method requires Editor access (with the ability to create public views), not Designer access as I previously said.

2. The API_DBDesign class in Mr. Katz's library uses exactly the same technique that I was demonstrating (using IDScan to get the NoteIDs of a class of Notes), so I'm not really presenting a different method at all. It's really the same technique, only he uses a class and offers a much more finished and flexible piece of code, and I've got a function with more specific functionality. In either case, that way of doing it does only require Reader access to a database.

I think that covers it. Sorry to put such false and misleading information on my site. I really should have actually looked at the DatabaseDesign library before I talked about it, instead of relying on my increasingly faulty memory. Measure twice, cut once and all that.

In preparation for Lotusphere, here are a few things for you:

a LotusScript agent that will produce an RSS feed of the Lotusphere sessions for you (also a text version, in case you don't like the HTML). Just plop this in the latest version of Ben Langhinrichs' Lotusphere Sessions DB and you can export the sessions to an RSS feed -- either all the sessions or just the ones you've indicated an interest in with the Session Interest view. I'm still not entirely sure that you could do anything useful with an RSS feed of Lotusphere sessions, but it seemed like an interesting idea...
a big beefy version of the "official" i.blog logo, in case you want to do something strange with it ;-)
a link to The Britonary (via Spuggy). I'll need to study up on this one to keep up with some of the conversations. Likewise, those of you from Europe can check the Southernese Dictionary to go the other way (or if you want to focus on other American regional accents, there are plenty of American Dialect pages out there).

Update: Okay Volker, I succumb to the pressure. Here's the official i.blog t-shirt (and other stylish apparel). Order fast, and you might get it next week!

Okay, I've actually heard a lot of interest in a blogger get-together at Lotusphere, from both bloggers and non-bloggers (members of the studio audience?) alike. Unfortunately, I'm probably the least qualified to put something like that together, because I really don't know anything about the logistics of Lotusphere, Disney, or Orlando in general.

Even if I knew where everything was, I'm pretty reluctant to try to say that we should all try to rearrange our schedules and be at a certain place at a certain time, because that's pretty impossible too. So here's what I propose:

Volker already said that he's going to be hanging out by the round pool at the Sunday night party (see the map on his page if you don't think you can find it). I say that we just take advantage of that, and all us bloggers can hang out in that general viscinity too. That way we don't have to worry about a special "blogger table" at a bar or a "blogger mosh pit" in front of the Dolphin or a secret meeting at 8:18 PM in front of the Banana or whatever.

And what I'll do to make it easier for everyone is print up a bunch of little stickers that look like this:

Nothing fancy, but I'll just run them off on a few pages of Avery labels or something, and as I meet each of you I'll hand them out. If you want to, you can stick one on your shirt, and your peers and adoring fans can recognize you and talk to you. If you think it's a gay and stupid idea, you can stick them all to jonvon's hair and go to the bar and buy another drink.

How's that?

(Please Note: jonvon's name was chosen completely at random in a double-blind test. I was not trying to encourage any kind of jonvon-related abuse in any way. Please be nice you your friendly neighborhood jonvon.)

Less than two weeks 'til Lotusphere. Man, I can't wait. I'll be arriving on Saturday, and if I'm lucky they'll let me in at the Gonzo party at ESPN that night. If not...well, I guess I'll just find some other place to drink.

Also, if anyone is curious to meet me for any odd reason, here's a recent picture that may help you find me. That's how I look when people are taking pictures.

Also (and this may be a bit nombrilistic of me), I'm wondering how many of the Lotus/Notes/Domino bloggers are going to be at Lotusphere. Here's the list of people I know about so far (in no particular order -- don't get offended if you're at a lower position than someone else):

Me :-)
Ed Brill
Volker Weber
The Turtle (of course)
Jake Howlett
Bruce Elgort
Joe Litton
Tom Duff
John Vaughn
Tom Nichols
Richard Schwartz
Declan Lynch
Matt White
Bill Buchan
Ben Langhinrichs
Rocky Oliver
Henry Newberry
Libby Schwarz
~~Bob Congdon~~ (oops, no Bob this year)
Rob Novak
Andrew Pollack
Dan Frederiksen
Melissa Gena
Erwin van Hunen
John Roling (Greyhawk68)
Ray J. Bilyk (The Lion King)
Lance Spellman

There are others who I think are going, but I'm doing this based on people who have announced their intentions in one way or another. If you know of anyone else (or if I'm wrong about anyone), please tell me and I'll update the list.

It's always bugged me that there's no method to programmatically convert a Notes document or a rich text field to HTML. It bugs me not because HTML conversion is a feature that I think should be added to the product, but because it's a feature that's already included in the product, and it just hasn't been exposed to us as developers. Lotus Notes has performed native HTML conversion since at least version 4.5, when the Domino web engine was introduced (that was in 1996!) -- even earlier if you count the InterNotes web server addin from version 4.0. But here we are in 2004 on version 6.5 of the product, and that's still one of those hidden secrets that lurks beneath the covers.

I know that you can buy third party products that do the conversion for you (Midas is the first one that comes to mind), but I just think that it should be available in the Notes product itself.

Well, it occurred to me recently that you might be able to hack something together using the feature introduced in R5 that lets you set a rich text field to store its contents as MIME. I toyed around with a few different techniques, and I finally found something that worked. So my latest tip for you is: Convert Notes Documents and Rich Text Fields to HTML.

The download is a database with a LotusScript library that demonstrates the technique, along with some wrapper functions that make it very easy to get an HTML representation of either a single rich text field or an entire Notes document. Included are functions that will send the HTML directly to a file.

There are a few limitations to this technique, notably that it uses NotesUIWorkspace and NotesUIDocument, so you have to run it in the front end. It also exposes a bug in R5 that truncates large multi-part MIME sections, so you may have problems with graphics or large documents in R5 (it seems to work fine in 6.5 though -- and probably 6.0x). And, the result you get is the HTML representation of the document or rich text field if it was sent as an e-mail, which may be slightly different than the representation you would get if it was displayed on the web with Domino.

All in all though, it may be a useful technique if you're as frustrated as I am with the inability to do this with a single native function call in script.

I have to admit, I'm a sucker for any post on the LDD forums that has more than about 15 responses. I get drawn to them like a moth to the light, because I know that somewhere in that thread someone has made a comment that really irked the other readers of the forum, resulting in a battle of some sort. It's like watching a train wreck...I just can't seem to avert my eyes.

Alternately, I can just look for any post that Nathan Freeman has responded to, since he usually gets drawn out by those sorts of things. :-)

Anyway, the latest such thread was one regarding securing local databases, which drew some pretty good responses about why you really can't "secure" local data that has already been accessible to the user at some point in time. I thought the discussion was actually going pretty well until someone jumped in and said that the discussion had "lost site [sic] of the question". Then, well...the discussion got really off-track (which is a bit ironic, if you think about it).

Without getting too much into the message thread itself, because you can just go there and read it if you're curious, I think it's important to realize that sometimes when someone asks a question, a valid answer can (and maybe should) be, "You're asking the wrong question."

For example, let's say your Mom calls you up and asks how to stop getting spam in her AOL inbox. The only true answer to this question is, "Don't have an e-mail account", much like the only true answer to "How do I make my server totally secure" is, "Unplug it."

However, just giving her that answer and hanging up is sure to make her mad enough to forget your birthday this year. What you need to understand is that she's just not asking the right question. The right question is, "How do I control the spam in my inbox." Knowing that (and clarifying for her), you can help her understand what the real issue is, which is really why she called you in the first place.

As developers, we get these kind of misguided questions all the time. Sometimes from our boss ("How can I archive all my data forever?") and sometimes from our customers or co-workers ("Can you help me print the contents of my entire hard drive?"). When we get these sorts of questions, we need to have the presence of mind to try to figure out what the person is really asking, or what they really need to know.

And after all that, sometimes the answer/explanation/alternative isn't what the person wanted to hear. But that's just life. Either you can be honest and have the person be mad about the answer, or you can lie and have the person be mad about the faulty solution that you provided. Which would you prefer?

Update: via a comment on Volker's site, here's good related reference: How To Ask Questions The Smart Way, by Eric Raymond.

I was playing around with the Notes logo (which I swiped from one of the desktop wallpaper images I downloaded from David Stephens' Quickplace -- thanks for the link, Ed), and I decided to make my own desktop wallpaper with it. Unfortunately, I have the graphic art abilities of a slug, so nothing really turned out too well.

If you want to try, here's a copy of the logo I was using (in gif and jpg format). I was just playing around with some of the stock images that came with the CorelXara CD I have from long ago, but if you have access to more/better clipart, you might fare better. Below are a few of the images I ended up with:

Okay, so I'm starting off the new year looking at all my website projects that are anywhere from halfway done to just a glimmer in my eye, and I think I need your help. What is it you would like to see? What exactly is important to you?

For example, I've spent a decent amount of time recently getting the dbactivity and ntimeset code cleaned up for release as open source projects. I could very easily keep going down that road and release some more of that sort of code, but I also realize that programming command-line utilities using the Notes C-API is a bit of a dark art, and most people really don't care. So it might be a huge waste of time to keep doing that, because most of my readers will just yawn and skip to the next entry in their RSS reader.

So without going into a list of specific projects I've got on the back burner, tell me what's interesting to you:

Notes API programming using C
Notes API programming using LotusScript
general LotusScript programming
Java programming for Notes
general Java programming
XML and related technologies (XSLT, RSS, etc.)
SOAP and web services

Other suggestions are obviously welcome, especially if there are things that I've promised on this site in the past and failed to deliver on. Feel free to leave comments on the blog or send me e-mail.

And if no one has any suggestions, that's fine too...I'll just keep doing whatever seems interesting to me on a given day. Heck, I might do that anyway, no matter how many suggestions I get. But it's always nice to ask.

A long time ago, I published a database that showed you how to export Rich Text Fields to Microsoft Word-compatible RTF format. As my first tip for the new year, I updated that database to show you how to export an entire Notes document in RTF Format.

This may not seem like a big deal, because you may already be doing this using @Command([FileExport]) on the client, but the neat things about my technique are:

it's in LotusScript
it can run against multiple selected docs in a view
it uses only the backend classes, so you can use it on a server
it allows you to specify an alternate form for exporting
it's much, much faster than using OLE to manually create your own Word documents

The only thing you have to watch out for is that Notes doesn't do a good job of exporting tables to RTF format, so you may have to tweak your form design to get everything to look good. However, I did provide functionality for specifying a different form to export the document with, so you can always just design an export form that the user never sees.