Bug in CFHTTP's FirstRowAsHeaders when COLUMNS not used

There appears to be a bug in the function of the new FirstRowAsHeaders attribute for CFHTTP. It's not working when the COLUMNS attribute is not specified, which is counter to the indication of how it should work in the docs.

CFHTTP adds a new firstrowasheaders attribute for working with reading in of text files. This is useful in case the firstrow of a text file (such as a CSV or comma-delimited file) doesn't have a list of column names. By adding the NAME attribute, CFHTTP will read the text file and turn it into a CF query result set. (The ability to read in text files and turn them into queries isn't new, of course: just the ability to indicate FirstRowAsHeaders.)

But the docs (CFML Reference) say that if it's set to "no" then 'If the columns attribute is not specified: processes as data, and generates column names; for example, "column_1"'

That would suggest that CF would create some column names automatically, which would be cool when one has a very large number of columns in the file being read. But I just tried it and simply got the error "The column name "1" is invalid." because that was the data in the first column of data in the first row.

It's as if it's ignoring the firstrowasheaders="no" attribute and trying to read the first row data as the column names. That attribute is indeed working if I add a COLUMNS attribute and give it column names, but again the point is that one shouldn't have to according to the docs.

I just reported this on the cfbughunt.org site and added a comment about it to the livedocs.macromedia.com site for that page in the reference manual.

BTW: I've found a work-around. If you know the number of columns in the file being read, you can just run a little loop to create the column names automatically, as in:

<cfset colnames="">
<cfloop from="1" to="7" index="i">
<cfset colnames = listappend(colnames,"column_"&i)>
</cfloop>

Then use that created "colnames" value in the CFHTTP as COLUMNS="#colnames#". Unfortunately, you have to know exactly how many columns to use or a failure to specify the right amount will get an error when CF tries to read the file and matches the number it finds against the number of columns you generated. It would be nice if it just did itself as it seems it's supposed to.

New Java Blueprints Application: Adventure Builder

Some may know that Sun came out some time ago with a "blueprint" application for demonstrating J2EE practices, called the Java PetStore. Many have taken it to be a model J2EE app (perhaps exceeding its real purpose as just an example), and some have even poked at it for not being scalable, etc. (which never really was its intention.) It was just an example app showing several J2EE practices. Anyway, Macromedia modelled their PetMarket after it to show how Rich Internet Apps could be built to create a better user experience for providing the same kind of pet store.

Well now Sun's coming out with another one, called the Adventure Builder. Available for download from the Java BluePrints Program, the Java Adventure Builder Sample Application v1.0 is the first release of the new Java BluePrints application. This early access release is a native J2EE 1.4 application that runs on the J2EE 1.4 Beta SDK. It showcases Servlet 2.4, JSP 2.0, JSTL, and web services. Of course, the Servlet 2.4 and JSP 2.0 specs are the next ones to come out and many servlet engines (including JRun 4.0 and therefore CFMX) don't yet support them. Also, it should be noted that the app itself is an "early access" version.

The new app is designed to be easy to understand and includes a new mini J2EE application that is even smaller and simpler, for folks really just getting started with J2EE. Download it from http://developer.java.sun.com/developer/releases/adventure/. And read the documentation at http://java.sun.com/blueprints/code/adventure/1.0/docs/index.html.

Will someone be coming up with an RIA of the Adventure Builder or the "mini application"? We shall see.

$10 computer books from APress (overstock)

Folks, some of you may know APress as a small but feisty publisher of computer books. I just noticed they're having a $10 overstock sale on 17 items, including some SQL Server, Linux, and other topics that may interest CFers. Check it out. If the link fails and/or the sale is over, still check them out at apress.com. They have some interesting books and often off-beat titles covering topics that other publishers may not.

Expecting custom tag queries in the DWMX Bindings tab

Someone recently asked me:

Do you know of any way to make cfquery/recordsets contained in a custom tag show up in the bindings panel on the calling page in DWMX?

The bindings tab in DWMX lists any queries on your current page, among other things, because DWMX looks at your CF code and pulls out the CFQUERY tags and contents and renders them in the bindings tab for various purposes.

Sadly, though, the idea of expecting it to find such queries in a custom tag won't work and if you think about it, it's technically correct that it not do so on a couple of levels.

First, the bindings are for things available in the current page. Technically, because custom tags have isolated scopes, a query in there would not really be available to this page. One could argue that if the custom tag set the query to the caller scope, it would indeed then be available to it, but that's a bit much to expect DW to figure out. :-)

But there's still more to it than that. As further proof of the challenge for DWMX, note that it also can't offer the "Open Document" option as it might if you right-clicked on a CFINCLUDE or CFMODULE. In those cases, at least you could open the file in question and see the bindings for it rather readily. But again DWMX can't even know what custom tag file to point to.

With the others, it can deduce the file name because it's provided as either a relative or absolute path/name. With a custom tag, it's really a run-time directive and so can't be determined at author time. The custom tag could exist in the current directory, the cfusion\customtags directory, or a subdirectory of that (or perhaps still another directory if the admin has setup alternative custom tag directories).

It's just impossible to expect DWMX to be able to figure it out.

PS Long time, no see to my blog fans. Just a busy holiday season with some pressing work challenges as well. Hope to be back in the swing of things now.

OK, one more time correcting the precompile.bat. Finally right?

Folks, I've got some bad/good news. The news I shared here on 12/5 was wrong. Just flat wrong. Here, finally, is the correct code for precompiling code that a) works with the -f directive if you add it and b) works against code in directories outside the default cfusionmx\wwwroot.

What a long strange trip it's been. If you're new to the discussion, just know that the ultimate (ok, "ultimate" for now) script is as follows:

set MX_INSTALL=c:\CFusionMX
set PATH=%MX_INSTALL%\runtime\bin;%MX_INSTALL%\runtime\jre\bin;%PATH%
java -classpath %MX_INSTALL%\lib\cfusion.jar coldfusion.tools.Compiler
   -webroot %1 -webinf %MX_INSTALL%\wwwroot\WEB-INF %1

which again, should be only 3 lines, with the first modified to be the location of your CFMX install.

You execute it by specifying:

precompile pathname

where pathname is the full path to your CF templates (or a specific filename). If it's just a directory, it compiles all templates in that directory and below.

See previous blog entries for more details:

12/5
11/28
6/27

The big point is that I was grossly mistaken in my offered solution of 12/5 (here) where I suggested the problem (with the -f directive failing after updater 1) was related to an issue of jvm versions, and to change the script to look in cfusionmx\runtime\bin, etc. Just forget all that. Mea culpa, mea culpa. Maxima mea culpa. (Latin for I really screwed up and am very sorry)

Someone pointed out that there was no Java.exe in the runtime\bin. I'm so disappointed that I posted incorrect information. What worked for me didn't for others for reasons I won't bother you with. But his pointing it out got me to investigating and I've figured things out. Very interesting discoveries. The bottom line is that if you'll change the bat file to what I offer above, it will work.

It turns out I was mistaken about several things and it was confusion from someone suggesting shortening the 3 lines to 2 by dropping the path line and prefacing the Java command with the path to the jvm. That was the original mistake. We had changed it to runtime\jre\bin because that was the location of the Java command in CFMX.

But while that worked ok if one left off the -f directive to the cfusion.jar compiler, it failed when one added it. And I can now see that the error of our ways was that removal of line 2. It's purpose wasn't to point to the location of the Java command. It was pointing at the jikesw.exe which is called during the compiler (with the -f switch), and it's THAT (jikesw.exe) that's in the cfusionmx\runtime\bin and for which the path is needed.

I won't belabor you the details of how I got so confused--or how much I spun my wheels once I was informed of the bad code fragment I gave before finally realizing the source of my mistake.

Anyway, to conclude, there is no Java.exe in the cfusionmx/runtime/bin. This all has nothing to do with the version of the jvm (1.4 or otherwise). That updated batch file should work and all will be right with the world now. :-) And we're all reminded how a single line of code can make or break a program!

Still want to buy CF 5? You can

Some have complained that they have clients who wish they could buy CF 5 while they wait to move up to CFMX. You can. Macromedia is currently still selling the professional and enterprise editions. Each is available with a subscription (only $320 more in the Pro edition) giving you upgrades to MX and whatever may come in the next 2 years.

Precompile problem solved

Folks, some of you may have been following a saga of where I had found that using the -f directive in the precompile.bat was failing as of updater 1. No one could seem to put a finger on it. I did a little poking around and finally uncovered the issue:

Update

I offered some info here on 12/5 about this being a 1.4 jvm problem, and suggesting you look to the cfusion\runtime\bin to find that version. After further review, that was just flat wrong and some mistaken assumptions on my part. I did a lot of digging and offer the correction above on a 12/10 entry. Let's just forget I brought this other stuff up. I've deleted it.

New look for my blog

After something in the BlogSpot or Blogger software caused my blog to lose its archive links, the search feature, and some other features, I decided to just ditch the old template/look and picked this new one. The new look has a couple of useful features, including the ability for you to choose whether to open links in a new window as well as to increase/decrease the font size. Better still, the new template/look is better suited to the larger amount of text typical in my blog entries. Hope it meets favor. I'm open to feedback.

Update: After some comments against the first updated version's changing colors (something it came built-in with and I couldn't control without changing a lot of code), I just decided to change templates. One thing I notice with this one, and perhaps it was an issue in the old one as well, is that it doesn't show the list of archive links for Netscape users. That's too bad. I've reported it to the blogger folks for them to track down. In the meantime, Netscape users can find a list of the links at http://www.systemanage.com/cff/myblogarchives.cfm.

Managing mailing list messages with folders and rules--solving problems that may keep you from trying it

People often find reading list messages challenging, especially if they've not set up rules and folders for them. Or if they have tried, it's a hassle. In this tip, I'll show how to handle this in Outlook. The steps may reasonably apply to other mail programs.

First, let me set the motivation for bothering (again, once you learn the tricks, it's no bother at all).

If you just let all the mail messages come into your inbox rather than have rules and folders, it gets messy real fast. You may find that a list has a sudden flurry of notes on a topic you don't care about and wish you could "not be bothered", or you may find can't easily identify messages from a particular list because the subject line doesn't say it's from the list. Or it may just be challenging keeping up with a large flow of mail. Wouldn't it be nice to leave the messages from a particular list to tackle all at once (in a dedicated folder). That would also make it easy to see if a question's already been answered?

Some may think folders and rules they don't work well because they can't see them, or they're a hassle to set up. A couple of tricks for working with them are:

1) They don't have to become a black hole where you don't see them unless you look in the folders. The thing is, some people don't see the folders listed all the time, so they fear that by putting the messages in a folder, they'll never see them. You can choose to have all your folders displayed all the time in a panel on the left. In Outlook, choose View>Folder List.
2) They don't have to become one long list of dozens of folders. You can create folders within folders.
3) Indeed, I create a folder of _lists-important and _lists-less important, and then create a folder in either for each list according to its importance to me to keep an eye on. 4) Notice too that I used an underscore in the names so that they show up at the top of the list of folders, just under the inbox. That way, when I return to the main inbox to view email ctrl-shift-i is a shortcut for that), I see the list of list folders right there on the left al the time.
5) Finally, Outlook highlights folders in bold if they have unread messages in them, so it's easy to tell at a glance which have messages to be read.

This really helps me to manage my multiple lists. I'm sure others may have still better ways. I just can imagine that we who use them wonder why folks who don't never bother. I wonder if it's just not being aware of a couple of these small things that make all the difference.

Now, the next step is to get the messages into the folder. It may seem trivial to some, because they know some tricks, but if you don't it can seem a hassle. Here are a couple more tricks for creating folders and rules for lists.

1) Open a message in your mail box that is of the sort you want to put in a folder.
2) Choose File>Move to Folder (ctrl-shift-v), and in that dialogue find your _lists (or _lists-less important) folder, select it. Choose "new" to create a new folder. Select that. Select ok. That will move the message to that folder. You've now created the folder and moved the message to it. Now create a rule for it, but there's a useful trick to make doing that easier.
3) Go to that folder, open the message you just moved. Choose Actions>Create Rule to create a rule for this message to place it in this folder. I recommend creating the rule this way because the "which conditions you want to check" will have the characteristics for this particular message, and "what do you want to do with this message" will already be filled with the name of this folder in the "move messages to folder" message. If you use Tools>Rules Wizard from the main window, you have to type all these in yourself. When creating the rule, just use whatever "which conditions you want to check" option that seems to uniquely identify this list's messages. Look at the choices pre-selected with the values for this message you're looking at. Sometimes a list will show its name in the "from" field, sometimes in the "to" field. The rest of the dialogue is pretty self-explanatory, or you can read the help.

From now on, any messages from the list will go to that folder. Now, the last step is to go grab all the list messages out of your inbox.

1) Go back to the main inbox (ctrl-shift-i). Open the Tools>Find (ctrl-shift-f) to find all the messages that meet whatever criteria you chose for the rule (messages to or from the list, whichever works for that particular list). In the find dialogue, you'll probably want to select the Browse button and choose to disable the "Search subfolders", so it just looks in the inbox and not any subfolders of it (unless that's what you really want).
2) Once you find all the messages, select them all (place the cursor on one of them and use Ctrl-A) and then choose Edit>Move to Folder and move them to the desired folder. It will still have it selected from the earlier step.

Now you've got a folder and rule, moved all the lists messages to that folder, and have the outlook display making it easy to keep an eye on the lists. A great benefit of this approach is that now you can ignore a list for hours, days, or longer as suits your interest, and then when you want to you can easily blow through all the messages at once. Great also for being able to easily look at more recent messages in the folder to see if perhaps a question has already been answered. That alone is a time-saver for you and the list respondents, as it will help eliminate redundant answers.

Ever want to open Studio, Notepad, CF Admin or some other program in a single keystroke in Windows

If you want to open a program like CF Studio or the CF Administrator or just Windows Notepad in a single keystroke, it's remarkably easy. For all my years of experience with Windows I'm surprised I never noticed this before.

You just need to edit the properties for the shortcut to the program (a shortcut is simply what you choose to run the program on your Start>Programs menu, or your desktop, or in my computer/explorer, etc.). Just right click on the shortcut/icon, choose properties, and in the "shortcut key" field shown, press any desired keystroke combination. It will be reflected in that field (such as "Control + Alt + A" if you press those three at once). Press "ok" and you're set. That easy. Now when you press those keys, the program will open. Cool. (I know that there are jaded folks among you wondering what I'm so excited about, but those who prefer the keyboard over the mouse will revel in this if it's news to you too).

Just notice that it doesn't warn you if you're overwriting another shortcut. Indeed, if you press a shortcut that's already taken, it will simply execute that shortcut, even while the focus is in that field for choosing a new one. (At least CF Studio's own keyboard shortcut tool, under Tools>Customize>Keyboard Shortcuts shows you if a key you're choosing is already assigned to something else).

And these shortcuts seem to work from within any other program (even overriding those set for that program, it seems) which is a two-edged sword.

Specifying HTTP headers in CFHTTPPARAM

If you find yourself needing to "spoof" different headers in a CFHTTP operation, such as to change the referer, accept encoding value, or other header you may know that you need to use CFHTTPPARAM and its available TYPE="CGI" attribute. But you may have challenges using that if you try to simply fill the corresponding NAME Attribute with the name of the equivalent CGI variables that CF exposes.

For instance, the Referer variable in the HTTP header becomes CGI.HTTP_REFERER. You might be lulled into thinking that the appropriate way to override/spoof that with a new value in CFHTTPPARAM would be:

<CFHTTPPARAM TYPE="CGI" NAME="HTTP_REFERER" VALUE="SomeRefererValue" >

but that will not produce the desired result. Indeed, what it will do is cause a CGI.HTTP_HTTP_REFERER variable to be created on the page being called.

So you may easily deduce that you should leave off the HTTP_ in the NAME attribute, and that is correct.

But what about the CGI.HTTP_ACCEPT_ENCODING? This is the value that indicates whether the browser you're simulating (with CFHTTP) should accept GZIP compressed output. (More about that in a later blog to come on doing GZip filtering with CFMX). The point is, if you want to set that on a CFHTTPPARAM and try to it as follows:

<CFHTTPPARAM TYPE="CGI" NAME="ACCEPT_ENCODING" VALUE="gzip,deflate" >

that will still not produce the desired result. It will NOT pass create the specified value. Why not? Because though the docs don't clarify it, you need to specify the header names by their standard actual names as defined in the HTTP standard. In the case of this header, it's accept-encoding, using a dash rather than the underscore that CF uses in the name it converts to.

So how'd I know the correct name to use? It's documented at this W3C page.

But how did I know what to value to use for the VALUE attribute for the accept-encoding? I wanted in the CFHTTP to send the same value that a browser would send, so I looked at what the value was in a a dump of CGI.HTTP_ACCEPT_ENCODING while browsing a page in a regular browser, and "gzip,deflate" was that value.

As a somewhat interesting side note, if you don't override the value this way using CFHTTPPARAM, then the value would be null on a page called by CFHTTP. Again, this will be more important in regard to a later blog entry I'll offer on gzip filtering and setting up tests of that using CFHTTP.

One last comment: you don't need to set the user-agent header using CFHTTPPARAM, though. The CFHTTP tag has a special USERAGENT attribute just for this specific requirement.

Some interesting Oreilly articles about DW, Flash for developers

I try to make my blog more about new discoveries rather than simply point to others articles, but I came across a couple at the O'Reilly site that seem pretty interesting reading for CF developers wondering (or having to debate folks) about RIA and Flash's place in the big picture of web app development, and another about DW for developers.

The first is a digest of an email conversation between Tim O'Reilly and some of his editors at http://www.oreilly.com/editors/ discussing Rich Internet Apps and Flash MX. Nice to see Tim generally defending the Flash/RIA vision. It doesn't end on the nicest of notes because of who they let have the last word, but overall an interesting read. Also see the comments from readers (just a few so far) at http://www.oreilly.com/cgi-bin/comments.

(Sadly, these links are to generic "editor"and "comments" page and don't incorporate any date, so I suppose in a few weeks they will no longer point to the same article. Here is the Google cached version.)

As for DW for developers, one of the participants in that article, Bruce Epstein, had written "Dreamweaver: A Visual Tool for Serious Web Coders", which while it's about DW 4 and not WMX, it's still rather useful. The first sentences are enticing enough, "With over 20 years of programming experience, I am skeptical of visual Web development environments. Dreamweaver has made me a convert."

I've pointed out before that there are also several articles of that sort about DWMX, and getting into it as a developer who may feel challenged by it compared to Studio/HomeSite, at the MM DesDev site for DWMX. Check those out, too.

Alert: when CF will fail to auto-compile--might be big news to some

Have you heard about people complaining that they find the automatic compile process is sometimes not detecting changed code? Did you wonder how that could ever be? Or had a hard time recreating it?

I can now offer a couple of interesting scenarios that are recreatable and do explain it, and seem worthy of further investigation. The problem has to do with copying old code onto a server when there's already compiled code for a newer version. This is the sort of thing one may do when backing out code versions on a prod or test server.

Of course, the automatic compile process works fine if the "new" code has a date/time stamp greater than the old compiled class file. That's the obvious time when it should recompile, but I've learned that CF has a little harder time dealing with when the date/time stamp of the "new" file is older than that already compiled, such as that described above in a version backout process.

The problems I've found are quite serious and very surprising.

The first problem is that if you copy the old file onto the server while CF is down, it WILL NOT detect the changed old code on restart. Wow.

Second, if you copy such old code onto the server while CF is up and running, it again WILL NOT detect the changed old code if--and this is a weird if--the file has not yet been browsed in the run when the copy is made. Further, even if you copy the old code while the server is running and you HAVE previously browsed the file, it's still not enough. You need to execute it after the copy, otherwise even after a restart it will still not see that new "older" code.

I know it may sound a little confusing, but here are some scenarios with more detail. First, the process of copying an old file while the server is down.

In the following, t1 and t0 are used to represent states of the file at points in time, with t0 being before t1:

filea is created/newly edited
filea (t1) is browsed
- auto-compiled into class under the covers
- is executed
- results show output of filea (t1)

server is stopped
old copy of filea (t0) is copied on to server
server is started

filea (t0) is browsed
- auto-compile does NOT create new class under the covers --- this seems the root of the problem 

- is executed 

- results show output of filea (t1), NOT the results of filea (t0)! -- even though t0 is the current code 

Also, note carefully that I indicated the stopping and starting of the server during which time the old file was copied. This is certainly a reasonable scenario, again as in when a production or testing server has code rolled back to a previous release. It's reasonable to expect that upon restarting the server, the old code should be newly compiled and executed, yet it's not. That's clearly a problem that needs to be solved. 

Now, perhaps more curious is what happens when the old file is copied onto the server while CF is up and running. It's not obvious. Indeed, only sometimes will it detect the changed "older" code. Sometimes it will not. No wonder people are pulling their hair out, both experiencing it and trying to recreate the problem. (If this is already known in support, sorry. I hadn't seen it explained anywhere before.) 

I've found that it depends on whether the file being replaced had been browsed/executed already at least once during the run of the server. If it HAD already been browsed/executed since the server was started and then an old version copied in, then it does detect the change--as long as you execute the page in that run after the copy. Otherwise, even after a restart, that new "older" code is not detected and executed. 

Further, if instead you make the copy during the run when it had NOT yet been browsed/executed since the server was started, then again it will NOT detect the change in that run or on restart. This is all very interesting. Here's a scenario: 

Run 1: 

filea (t1) is newly edited 

filea (t1) is browsed, autocompiled, executed, shows new (t1) output 

Run 2: 

filea (t1) is browsed/executed during this run 

- autocompiled, executed, shows (t1) output 

filea (t0) (older version predating t1) is copied onto server 

filea (t0) is browsed, autocompiled, executed, shows new (correct, t0) output 

That's what we'd all expect (or hope for), of course. 

But if we had not performed that last step of browsing the file after the copy, then even on a restart we'd still see t1! 

Further, if in run 2, you did NOT browse t1 during the run before copying in t0, again you will NOT see the output of t0 on a refresh of the page. You'll still see t1 from the previous compilation. And again, a restart of the server also won't fix this. 

So this is ugly. Really ugly. It seems old code copied in will only be executed a) if the server is up and b) the file has been executed in that run before the copy is made and c) the file if browsed/executed after being copied in. 

Has this been recognized? I'm running on the built-in web server, if that's significant, and running with the updater. 

I'd think this warrants a KB, and hopefully a fix. 

In the meantime, for those needing a workaround, there are a couple: 

First is that while the -f fails to work you could instead delete the corresponding class files from the server (in wwwroot\WEB-INF\cfclasses) before making the copy in of the old code (or at least before starting the server after making the copy). Then when you run the code there will be no class file there and CF will auto-compile it. 

Unfortunately, it's not trivial mapping a given filename (and its directory location) into the equivalent filename used in the cfclasses, to be able to delete just the files you'd be interested in. There's a hashing algorithm that I've seen explained but not well enough that I was able to devise (nor have I seen) a routine to do that conversion for us. If anyone offers it, I'll post it here. 

Another alternative may seem (and some have done this) to delete ALL the files in the cfclasses directory. Sadly, this is way overkill. The directory holds all the classes for all files in all directories (even virtually mapped ones outside the wwwroot), so deleting them all to just effect a change for one or even a few is a pretty high price to pay. It will force recompilation of everything, thus slowing down the server and the end-user experience of the first person hitting each page. While the precompile.bat may help if you then recompiled everything before restarting, this all just seems like hitting a gnat with a hammer. 

Another solution is simply to take advantage of the observation I made that if you start the server, then browse the templates, then make the copies while the server is up, and then browse them, then indeed the changes will be reflected. This is arduous if you need to automate the process (such as overnight), to automatically run the templates and then copy them and execute them again. Of course, you could use CFHTTP to execute the pages--and have it look at some list of files you are going to be copying. And you could use CFFILE to do the copying also from that list, and then use CFHTTP again to re-execute the newly copied files. 

Update:

I had said originally here that

One other idea (again, if we had the agorithm to map source dir/file names to their compiled cfclasses counterparts) would be to automate the process and have it CF automatically run a check at startup to find any files that have dates older than their previously compiled counterparts (to catch such copied older files) and run them so that if they were copied in during the run they'd "take" immediately on their next run. CFHTTP could be used to execute the code.

Now that I think about it, that's no solution at all. All that matters is that anytime old code is copied in during a run, the code needs to have been run once and then be run again after the copy in, in order for the change to be respected (for a new class to be created based on that older code). The bottom line is that there's no automation at startup to do. You simply need to manually run the code, copy it in, then run it again.

That still won't solve the problem of old code copied in while the server was down. It's not enough to execute it in the next run, you need to browse it in a new run, copy it during that run, and browse it again during that run). Otherwise the server will just never see that new "older" code that was copied in.

There are so many permutations and variables here that it's possible I've missed or left something out. Open to thoughts.

Hopefully, MM will recognize this and either has a fix coming or can add one soon. I know some updaters are planned to be released in the very near future.

BTW, my testing has been on the built-in web server. Don't know if that will have an impact. And again I do have the first updater applied. Don't know if things are different pre-updater.

update:
Steve Ringo, manager of the South Africa CFUG, made this suggestion:

I was thinking about an easy workaround - how about this? Use a file "touch" utility to change all dates/times to "now()". This should be fine for a production server, as the date and time modified will still be intact for the development server, where the original date and time may be useful to have - for source control programs for instance.

I found a great freeware one at http://stevemiller.net/apps/ (See win32 Console ToolBox 1.0). It also can recurse subdirectories. Just pop this line into your precompile batch file:
touch /s *.cfm

You can add /q to "quieten" the output if you dont want a status report on each file - hence touch /s/q *.cfm

I haven't had the chance to try it yet in CFMX, but the utility works fine testing with some arbitrary files on my hard drive (with subdirs). AFAIK, the touch utlity is standard with most flavours of Unix (I dont think it has subdir recursion though).

Sounds like it could work. Thanks for that, Steve. One negative is that it would change the date/time of the file on the server which would make it out of synch with the place from which it was copied (whether another server, a develoepr's workstation, or a version control system). It's not a show-stopper, especially if it solve the problem, but it may annoy some as much as the problem. But otherwise worth a look. Thanks

OT: Some ways to avoid abuse via HTML Email in Outlook

OK, I try to avoid being too off-topic in my blog entries, but this may be helpful info for others. Someone was lamenting how sad it was that Outlook users are so open to abuse via HTML email. I put together this reply and thought that perhaps my blog readers might appreciate some of the ideas:

As for outlook being a haven for spammers, well, it can be. But there are steps one can take. It's no longer enough to merely "not open attachments", as you note. Here are some steps:

As for outlook being a haven for spammers, well, it can be. But there are steps one can take. It's no longer enough to merely "not open attachments", as you note. Here are some steps:

1) Turn off View>Preview Pane, otherwise when you're looking at email in your inbox, simply selecting a message will cause it to be "previewed" in whole and will execute HTML in the page, thus triggering not only these IMG SRC tags you describe but also possibly executing code via <object> and other tags.

2) Turn on View>AutoPreview, so that you can (if you like) at least see a few lines of a message. Then, even if it's HTML it's not executed.

3) If a message looks suspicious, rather than open it (which will execute that HTML), do a couple of things. First, if the name listed in the "from" is curious, right click on the message and choose "options" to see the "internet headers" and scroll down to learn more about who sent it, etc.

4) If you're tempted to open it but don't want to risk the HTML "executing", there's one last trick. Use File>Save As. I do this all the time. By saving it off (choosing "text" for "save as type") you can then open that text file, and even if it had HTML, you could now look at the file without risk. Saving it as text, though, will cause it to be stripped of HTML as well. If you want to see if the message was an HTML message, you need to use "html" for "save as type". Just be sure then not to open it with IE or another
browser. Open it with Notepad, Studio, or another editor.

It's a real shame that Outlook (2000) at least doesn't make this last step easier.

Doing all these things will greatly reduce the risk of your being caught off guard. And, as Jorgen points out, by not even reading "spam" messages that might trigger those <IMG SRC> tags sending a toggle back to the server, you may lead them to think that they've reached a dead email address.

Ratings in from my DevCon 2002 Presentation

I try not to do too much obvious self-promotion on the blog, but the evaluations are in from the DevCon and of the 71 attendees of my "Incorporating JSP Custom Tags in CFMX" presentation, I received an average overall rating of 4.6 out of 5 and several nice comments. More detail in the associated press release available on my site. Some of the kind comments included:

• Very well done! Good information clearly presented
  
• Fantastic. Presenter was very knowledgeable. Great overview - a bit fast presenting though!
  
• Managed to increase speed without losing quality (started late [due to earlier session running long])
  
• Charlie presents the material in a manner appropriate for all levels of experience
  
• Charlie is an excellent speaker. great presentation
  
• Charlie Arehart is always a good speaker no matter what he speaks on
  

This is my third year presenting at DevCon and I'm grateful to Macromedia for allowing me another avenue to share with the community.

Nice, simple, low-cost Version Control for Windows

Looking to add source code control to your application, but perhaps put off by the high cost of Microsoft Visual SourceSafe (VSS), or the complications of open source tools like CVS? If you're in a windows environment (only), take a look at QumaSoft's QVCS. It's quite easy to set up and use (includes a Windows interface for managing revisions), and also integrates with Studio/HomeSite (by way of their "project" feature).

If you've never used the projects feature or not set one up to work with Version Control, see Chapter 9 of the "Using HomeSite" book available online. It's the same process in Studio and HomeSite+.

If you want to use QVCS from within Studio/HomeSite, just be sure to first enable the "IDE Integration" feature in QVCS (under the "Admin" menu command). It doesn't seem to matter if you set it as what QVCS calls the "default" version control tool option.

Note also that in creating a "project" in QVCS (not technically the same as a Studio/HomeSite project, but they can both point at the same source code directory), QVCS offers a feature called a "reference copy" location. This seems similar to what VSS calls a "shadow" directory. If you're working in a team, this is a place where whenever you check-in files, they are copied both to the source code repository (called the "archives" in QVCS) in binary form as well as to this "reference" directory in text form, such as to act as the testing/integration directory for a team of developers.

I may put together a couple of movies showing how all this works, but besides the Studio/HomeSite help reference I gave you, there's also decent help in QVCS (including a couple of tutorial chapters in the Help as well as an available PDF of the entire help file), so most people should be able to take it from here. Enjoy!

(BTW, if you're wondering whether you can use this with Dreamweaver MX, sadly, it seems no. The feature in DWMX for working with VSS--in the "remote info" feature of a site--is really a direct connection to the VSS database, not to the SCC API integration such as is used by QVCS, VSS, and other source code tools. There seems to be no SCC API support in DWMX.

This isn't the end of the world, however. If you use DWMX, you can still benefit from using this or any other source code control tool. It, like others, offers its own interface for performing checkin, checkout, comparing files, reporting and lots more. Indeed, if DWMX is setup to be the default application to open the files you want to control--if double-clicking the file in Windows Explorer would open Dreamweaver--then double-clicking it in the QVCS interface will offer to check-out the file and then open it within DWMX. That's a reasonable compromise. You just then would go back to the QVCS interface to check the file/s in when done after saving it in DWMX.)

Enabling CFMX Metrics Reporting and Service Debugging

*** Updated 2/28/03 to change filename reference from jrun-web.xml to jrun.xml. Thanks to Rob Rusher for pointing it out. Originally posted 11/17/02 ***

Interested in logging how many threads are running within CFMX, or how many sessions, or how much memory (in KBs) is being used? There is a set of logging information that you can enable (it's disabled by default) so that it's written to the default-event.log file in CFusionMX\runtime\logs. To enable it, change jrun.xml in CFusionMX\runtime\servers\default\SERVER-INF, setting the <service class="jrunx.logger.LoggerService" name="LoggerService"> element's:

  <attribute name="metricsEnabled">false</attribute>

to true. Another setting next to is is:

  <attribute name="debugEnabled">false</attribute>

Setting that to true will add new lines to that log file, indicating (with a prefix of "debug") major events that happen in the establishment of the CFMX environment.

Of course, you want to think twice about enabling this sort of metric/debug reporting in production, as it will add some overhead. Still, it can be informative.

In the case of the metric logging, you will see that there are other lines that follow these in the jrun.xml that control the reporting frequency and details. You can learn more about these settings not in the CFMX docs but instead in the JRun docs, in particular the JRun Administrator's Guide, Chapter 7 on Connection Monitoring, available online at livedocs.macromedia.com, specifically at http://livedocs.macromedia.com/jrun4docs/JRun_Administrators_Guide/netmon2.jsp#1096147.

You'll learn there the various variables you can monitor about the CFMX server as a whole (memory and session tracking) as well as web server connection status (such as idle, busy, listening threads, and more). Unfortunately, while the mechanism for monitoring such threads with an external web server connection (like IIS or Apache) work as explained, doing the same for the built-in web server (saying to use the web. prefix for the listed variables) does not work in CFMX.

I'd welcome any insights from anyone in or out of MM with more info.

Setting up connections to Access in CFMX

Folks who are using Microsoft Access with CFMX may find that there are problems when trying to configure the datasource in the administrator, perhaps because the default settings aren't appropriate to the kind of security used with their database (whether that's no security, security by a single password, or by use of usernames/passwords--what Access calls user-level security, by use of an MDW file).

There is a MM knowledge base article (http://www.macromedia.com/v1/Handlers/index.cfm?ID=23381) that addresses how to properly configure a DSN in the CFMX Admin for either of these 3 forms of security.

Problems on Linux? Check out new MM KB article

Are you having problems running CFMX on Linux? Interested in learning a little more about some undocumented performance tweak possibilities (that it seems may also have value outside of Linux), check out the new Macromedia KB article, "ColdFusion MX support on Linux", at http://www.macromedia.com/v1/Handlers/index.cfm?ID=23524.

The coverage of threads and thread configuration is not limited to Linux, including the explanations of activeHandlerThreads, minHandlerThreads, and maxHandlerThreads (of which only the first is reflected in the Admin as "Simultaneous Requests" (separate settings for each of the built-in and external web server support), and it seems the recommendations about setting them could apply to outside of Linux, though that's not stated and therefore can't be relied upon.

Another issue that can challenge DWMX speed for CFers

Continuing the discussion of DWMX speedup opportunities from earlier today, I've learned something interesting. After looking at a sample from someone reporting sever delays, I observed something that definitely explain things for some people. She had reported that a very large file (approx 1000 lines) was taking a long time to load. But anyone opening such a file of HTML content would say it was no problem at all.

The thing is, this was a CF template and the code had several CFQUERY's in it, run conditionally so it's not about doing several at runtime, but still this causes issues for DWMX that one might not anticipate. It tries to load up the bindings tab with each CFQUERY to create what DWMX calls a recordset. It also loads up CFPARAM tags and FORM/CFFORM tags there as well, among other things.

So DWMX is clearly processing the page when it opens it, and with a dynamic page like CF it's going to be looking at the code to figure out things like this. (Indeed, she mentioned having inherited one code sample that had 36 queries in CFSWITCH/CFCASE statements!).

Many will want to point out that such code is itself a source of problems in its design (and she realizes that it's code that will benefit from redesign, especially in CFMX with components), but the bottom line is that there are code examples out there that will indeed suffer during opening in DWMX because of this issue. I didn't find any undue time spent while switching from one file to another, though.

I don't see any feature to get DWMX to no perform this preprocessing. If anyone knows of one, please do share. :-)

More Speedup Tips for DWMX

In my 10/16 entry, I pointed out one way to speed up DreamWeaver MX. I have a little more detail and a couple more ideas now.

I had mentioned how you could see Edit Site, then on the "Advanced" tab, in "Local Info" turn off "Refresh Local File List Automatically". I didn't point out that the same setting exists in the "Testing Server" field there as well. These will be a big help with respect to the speed of switching in and out of DWMX. This feature is intentional as it's trying to detect if while you were out of DWMX you were perhaps creating new files (such as when folks do indeed switch out to Studio/HomeSite). But if you don't make such file additions/deletions, or if mostly you're switching out to read email, etc, then this "auto refresh" is an expensive operation. Turn it off and much slowness will go away.

Similarly, the "Remote Info" section of the Advanced tab has a checkbox "automatically upload files to server on save", again to keep your remote machine in sync. Maybe you won't really want that feature enabled (and it will certainly slow down all saves). This feature is available in all the forms of remote access (ftp, local/network, RDS, SourceSafe and WebDAV).

Some have also suggested turning off Virus checking of the DWMX "Config" directory, though I've not experience that doing much good (but I may have gained so much with these other changes that it didn't add much).

I'd welcome anyone else's thoughts. This seems an important problem for so many that it would be useful to gather some ideas.

Date processing with Access in CF

Some folks have had problems doing date processing within SQL against an Access database. (Let's not get into whether anyone should be choosing Access as a DB. For some low volume sites, it works adequately.)

Part of the date handling problem stems from the fact that Access may need help determining that what you're passing it is a date (versus a number). There are two ways to do this. You could use the CreateODBCDate() function to convert a date to a format that will be passed from CF to SQL as an ODBC Formatted date.

Another is to take advantage of the fact that Access can use pound signs around a date to indicate that the value is a date. Of course, those are the same pound signs that CF uses for variables, which can complicate things.

First, just know that one way to format a SQL statement WITHIN Access to process, for instance, records between two dates (leave CF out of the picture for a moment) would be to render it as:

SELECT StartDate FROM Employee WHERE StartDate between #01/01/97# and #01/01/98#

If you wanted to do that within a CFQUERY statement, then you'd need to escape those pound signs (double them) so that CF didn't think you were trying to refer to variables, as in:

<CFQUERY NAME="test" DATASOURCE="CompanyInfo">
SELECT StartDate FROM Employee WHERE StartDate between ##01/01/97## and ##01/01/98##
</CFQUERY>

Of course, you could also convert this to use the ODBCDateFormat function, as in:

<CFQUERY NAME="test" DATASOURCE="CompanyInfo">
SELECT StartDate FROM Employee WHERE StartDate between #ODBCDateFormat("01/01/97")# and #ODBCDateFormat("01/01/98")#
</CFQUERY>

Notice again that the use of this function doesn't require the use of escaped pound signs because the function creates the date in a format Access understands. But you'll usually have the date coming in as a variable. You could then easily change the ODBCDateFormat as in:

<cfset date="01/01/98">
<CFQUERY NAME="test" DATASOURCE="CompanyInfo">
SELECT StartDate FROM Employee WHERE StartDate between #ODBCDateFormat("01/01/97")# and #ODBCDateFormat(date)#
</CFQUERY>

But if you tried to use the Access form of passing pound-delimited dates, such as in the first two examples above, and you wanted to use a variable for one of them, you'd then need 3 pound signs around that (two for the escaped pound for Access and one for the pound needed to refer to the variable/function), as in:

<cfset date="01/01/98">
<CFQUERY NAME="test" DATASOURCE="CompanyInfo">
SELECT StartDate FROM Employee WHERE StartDate between ##01/01/97## and ###date###
</CFQUERY>

Finally, note that if you wanted to use the now() function in that last example for the date (which returns the current date AND time), or if your incoming "date" variable had minutes and seconds in it, you'd need to wrap it in a dateformat function before using the "pound sign" approach to date formatting in Access because that doesn't like anything but a date to be passed in, as in:.

<CFQUERY NAME="test" DATASOURCE="CompanyInfo">
SELECT StartDate FROM Employee WHERE StartDate between ##01/01/97## and ###dateformat(now())###
</CFQUERY>

Note, too, that I've not bothered with any date formatting "mask". In my testing, Access is happy with the default dateformat form of date layout.

So when should you use the pound sign approach? Well, really, it's more something that's needed within Access itself to be able to detect that a string of numbers (and dashes or slashes) are in fact representing a date. The thing is, with the ODBCDateFormat function (and CF's Now() returning a date in ODBCDateTime Format), it's perhaps not as important to use the pound-surrounded approach to dates in Access.

Wow. A Contribute DesDev Site Already

Wow, there's already a DesDev site devoted to Contribute, with 23 articles. Check it out at http://www.macromedia.com/desdev/contribute/.

More Thoughts on Contributor's value for CFers

There's a lot of talk today in various forums about the value and impact of Contributor for CF (and other web app developers). As I started to say in my previous entry, I think it will take time to sort out the impact not only for the clearly intended audience (folks needing simple content management systems, and developers who previously did updates for that audience), as well as the impact on the content management business (don't know that it will be a category killer, but it could in some segments).

But there's also the question of the impact for general CF (and other web app) developers.

While it may seem that the product is geared toward static sites, it does indeed support dynamic sites (it will automatically lock such server-side code to prevent the user editing it). And they suggested in the announcement conference call that it will change even more regarding that sort of development environment.

But I've been thinking about it in a way that perhaps may seem unorthodox to some. Many may even disagree. I'd also argue that it's possible that many sites that are currently dynamic could be switched to being more static. For instance, many use server-side coding simply to facilitate reusable/moderately changeable aspects of interface design (nav bars, page layout, reusable content segments). While Dreamweaver MX's template feature could be used for some of this, many web app developers prefer to do such things the old fashioned way (in CF, using CFINCLUDE and coordinating the upload of that code themselves). With DWMX, they would need to "republish"
the site whenever they'd change the template, rather than simply changing the included file. And while you could maybe put that power into the client's hands, they would have to have DWMX.

Contribute could change that sort of scenario, bringing the power of DWMX's templating feature to alleviate the page being dynamic just for these reusable components, without the user having to have DWMX.

This doesn't diminish CF's place (or whatever server-side coding one may use). Instead, it limits it to being used only for things that truly are dynamic (database, component, or web server-generated info, for instance). This could have a performance impact by allowing pages that are currently CF (or ASP/JSP/PHP) templates (just for the reusable interface aspects) to be reset to static pages.

But another reason server-side coding is often used is simply to facilitate creation/editing of content in a portion of a page. This, of course, is another thing that Contribute would alleviate entirely.

Still, the current lack of an approval/workflow system will keep some from jumping at the opportunity. It's seems the challenge is how to solve that adequately for all preferences. I can see how that would be a dilemma.

Again, time will tell both in how people find ways to best use it as well as how the product may evolve.

Look into Macromedia's new tool announced today, Contributor

Could I be the first to blog the link to Macromedia's new product announced today, Contributor. Check it out at http://www.macromedia.com/software/contribute/. I'm sure other bloggers will have much more to say. Of course, some will argue that it's not really something related to CF since it's really just a way to facilitate end-user updating of predominantly static sites.

But it could be used with CF sites, and more important there may be aspects of some sites you build in CF that could perhaps be set back to "static" by enabling update access in Contributor. Also, it has built-in ability to lock out dynamic code like CF (which can be disabled if you desire), so in time it will become more clear how indeed even CFers may find it quite appropriate to use.

One thing to note: the price listed on the FAQ page off that site shows it being $99, but it's not clarified that it's $99 for a client installation of Contribute (in other words, for a contributor), but according to a conference call announcing the product this morning, you can "contribute" to any number of sites from that one license. But this does mean that if you want dozens of people to be contributors, they'd each need a license. See the site for more details (which will hopefully answer any questions we'd all have over time)

Error in precompile.bat printed in my CFDJ article

Gads! Somehow the version of the precompile.bat file that was printed in my October CFDJ article (http://www.sys-con.com/coldfusion/article.cfm?id=519) is missing some important code! :-(

It should be:

set MX_INSTALL=d:\cfusionMX
%MX_INSTALL%\runtime\jre\bin\java -classpath
    %MX_INSTALL%\lib\cfusion.jar
    coldfusion.tools.Compiler -webroot %1
    -webinf %MX_INSTALL%\wwwroot\WEB-INF %1

As the article indicates, it's just 2 lines, the first being the SET and the second starting with %MX_INSTALL%.

But the printed version was somehow missing the beginning of the second line:

%MX_INSTALL%\runtime\jre\bin\java -classpath

So sorry for that printing mistake!

One other thing I've learned since writing the article is that it seems to fail to work if a directory name or file name being precompiled has spaces in the name.

Update: I've since found another issue with the version offered in the CFDJ. In certain instances it may fail to work, and it turns out that it was the setting the second line above to:

%MX_INSTALL%\runtime\jre\bin\java -classpath

as was shown in the article. It should instead be what I've set it to above:

%MX_INSTALL%\runtime\bin\java -classpath

Turns out that the runtime\jre\bin calls a 1.3 version of the java interpreter, and as of the updater 1 of CFMX, parts of this compile process (when using the -f directive) need the 1.4 interpreter. The corrected line calls the correct one. See my entry of 12/6/02 for more info.

/charlie

Speeding Up Dreamweaver MX: Disable Auto Refresh

Some have noticed that Dreamweaver can perform sluggishly.

One reason is that, by default, it's checking all files in a site (and all subdirectories) to determine if they have changed--and it does this every time you leave and return to DWMX, such as to go read email, edit a word processing document, etc. The thinking is that you may have created or edited a file while outside DWMX. Unfortunately, that "lookup" can cause a painful delay if you leave and return to DWMX often, especially if the site has many files (and subdirectories).

The option is controlled by the feature "refresh local files list automatically", a checkbox in Edit Site>Advanced>Local Info. Turn it off and see if that helps.

You can use the "refresh" button in the site tab to refresh the list. Also, if you expand a directory for he first time since opening the site, that will also obtain a refreshed list of what's in the directory. Once it's opened, though, the file list is cached and only using the "refresh" or restarting DWMX will obtain an updated list (with the "refresh automatically" option disabled).

It's worth clarifying that this is a site-specific feature. You can leave it enabled on one site and disable it on another.

One other comment about poor performance in DWMX in general: I mentioned in my entry of 9/27 ("Getting into Dreamweaver for Studio users") that your machine's performance may have an impact, and in an entry on 8/10 ("Time for developers to consider a hardware upgrade?") I suggested other benefits of considering a machine upgrade.

Running the updater

Well, I've waited a few weeks to install the updater and while I've heard of some problems none seem so grave so I've installed the updater (http://dynamic.macromedia.com/bin/MM/software/trial/hwswrec.jsp?product=cfmx_updater.

It went well, though while it reminds you to stop the CFMX service and the release notes tell you to also stop the CFMX ODBC Services as well as restart all of them after the updater is done, it's easy to forget to start those ODBC Services which could lead you to think something went wrong with the install (I did, and none of the datasources were working correctly until I figured that out).

I was disappointed to see that the udpater did not fix the problem I raised in my June 2002 article on the new server-side redirect, with the getpagecontext().forward() method. It still does not work if a forward is attempted from a form action page. Grr.

/charlie

Looking for Some Verity Docs?

Are you looking for more info to understand how CF's underlying Verity Search engine works? Whether for creating or searching them, there's lots more to it than is discussed in the CFML Reference or Developing CF Apps books in the CF docs.

I've come across some sites with interesting extended documentation. If you've not seen these they may be useful.

Before that, though, some may have noticed that in CFMX there is a new manual "Working With Verity Tools". That's certainly worth a look, particularly for the coverage of using Verity Spider searching. See it at livedocs.macromedia.com.

But beyond that, there is also a complete set of HTM-based Verity docs at http://support.exln.com/doc/full/dm/vdk/doc/. This is just the docs at a particular site that's posted them. There are other places that have posted them, in case these every disappear. Do a google search for "Collection Building Guide" to find such links. This particular link offers several docs related to Verity processing.

New Search Feature for My Blog

You can now search my blog. Been wondering where I talked about how to edit the web.xml file? Or how to make the most of Studio? Or whatever? Now you can find the topic of interest to you. Notice the available "Search" form on the right above the archive links. Go to town!

BTW, I enabled this using Google's feature allowing creation of a search form on my site using their indexed data about my site. See http://services.google.com/cobrand/free_select to sign up to allow searching of your own site.

For bloggers, you may wonder how to get your site content searched by google. It won't just find it on its own. I found the solution was to create a page on my systemanage.com site (which is often spidered by Google) that linked back to all archive pages on my blog site. After a few days, Google found that page and began indexing the blog pages.

Get CFMX Version of the CFML Reference to Studio and HomeSite+ Help Tab

If you're running CF4.5 or 5 of Studio, you may be interested in having the CFMX version of the CFML Reference available in the Help Reference Tab in Studio (Help>Help References Window). Also, folks running HomeSite+ may have noticed that there, too, the help reference doesn't include any CF manuals at all. Grr.

(I'm not referring here to the help shown when you press F1 on a tag or function, but instead the actual help tab listing several available books.)

Macromedia has solved the problem, at least a little. You can go to http://download.macromedia.com/pub/homesite/updates/cfml_ref.zip and download a zip file. Follow the instructions in the CFMLRef_install.txt file on where to install it and optionally how to improve its appearance inside the help tab once installed (by editing the booktree.xml file it talks about). This will cause you to have the CFMX version of the CFML Reference manual available in the help, but not the other CFMX manuals.

While the instructions apply to HomeSite+, they're equally applicable to previous releases of HomeSite and Studio. Try it!

BTW, if you're still using Studio 4.5 or 5 and want to update those, such as to add patches that came out after the product's release or to get tag help, insight and completion for CF 5 in Studio 4.5, those are all separate updates. See my blog entry of August 1 for more details.

How CFMX creates the filename for the compiled version of your CF code

In previous bog entries I've talked about compilation and precompilation of CFMX code into Java. And I've shared how all CF templates get compiled into the one [cfusionmx]\wwwroot\WEB-INF\cfclasses directory. 

Ever wonder how CF creates that file name? A CF template named Setsession.cfm might lead to a class file named cfsetsession2ecfm1011928409.class.  The "cf" is always there, while the next part is the file name and extension, but with "2e" where the period in the filename was. The number that follows represents a hash of the file/folder name.

Here are the details from Mike Nimer at Macromedia:

How CFMX creates the .class name, or how you can find the name of the .class file you want to delete

The formula is: "cf" + filename + hashCode

The fully canonicalized filename is used, with "/" as the directory separator. The following substitutions are made in the filename:
1.) / becomes __
2.) any other character which would be illegal in a Java identifier is represented as 2 hex digits

hashCode is the hashCode() of the File object which represents this file as documented at http://java.sun.com/j2se/1.3/docs/api/java/lang/Object.html#hashCode()

If the hashCode() is negative, it is exclusive-OR'd with 0xFFFFFFFF to get the value used by ColdFusion.

It seems that hashcode call uses the filename AND the folder, as even two files in the same directory could end up with hashcode values that are different. 

You can prove this by creating two templates in the same folder, with distinctive names that you can readily identify, and then save and execute those pages, and view the latest class files created in the cfclasses folder. Those distinctively named files in the same folder may show having the same or different hashcodes. 

So sadly, this is like a one-way hash: cf can figure out what file exists in the cfclasses for a given filename/folder it wants to run, but we can't look at that hashcode and figure out what folder that file was in. At least we can see the name: though again you may well see many classes with the same name, for different folders in which a given filename may live. 

And note that for components (CFCs), note that there is a class for each method in that cfc (technically there is also a class for each cffunction in a cfm, as well). So you could have far more class files than you may expect if you somehow had an idea of how many cfm templates you had. :-)

Pausing your CFMX code

Folks have often wanted to pause their CFcode, particularly when wanting to simulate some sort of long-running task. Usually they use a CFLOOP, but now that CMX is built upon Java, there is a better and less resource intensive way. Java has a "sleep" method in the java.lang.Thread object. You can get a wait by doing the equivalent:

<cfobject type="JAVA" name="obj" class="java.lang.Thread" action="CREATE">

<cfset obj.sleep(10000)>

This will pause 10 seconds.

CF_SLEEP

To make it even easier, I've created a custom tag to handle this, called cf_sleep. It's available both at the DevExchange and via my site. You would use it in the form:

<cf_sleep seconds="nn">

SLEEP UDF

It's also available as a UDF, in which you might use it in the form:

<cfinclude template="sleep_udf.cfm">
<cfset tmp=sleep(2)>

Believe it or not, this approach works in CF5 (and 4.5) as well as MX, since the ability to call Java objects was introduced back then.

Getting into Dreamweaver for Studio users

I know a lot of Studio uses are unhappy with Dreamweaver and inclined to just stay put, but maybe you're being forced by licensing issues or organization policy to make the move. Or maybe you just realize that the writing is on the wall so you might as well suck it up and make the most of it. I want to offer a couple thoughts that might make things a little more palatable. Perhaps these won't be news to you, but maybe they'll be useful resources.

First, Macromedia has put together a couple of resources to help this very audience. Four particular articles at the Dreamweaver Development Center deserve your attention:

• Migrating from Macromedia ColdFusion Studio to Macromedia Dreamweaver MX , by Charles Nadeau
  
• Three Reasons to Migrate From ColdFusion Studio/HomeSite to Dreamweaver MX, by Tom Muck
  
• Migrating from Macromedia HomeSite to Macromedia Dreamweaver MX, by Jennifer Rowe
  
• Code Editing in Dreamweaver MX, by Mark Fletcher
  

Beyond that, there are several other articles at that site which show how to apply DW to various tasks that CF developers may be interested in (developing, using components and web services, for instance). There are are also tutorial examples (like the Trio Motor Company app) where they include source code and steps to finish the projects. All these are great ways to become familiar with really leveraging DWMX, rather than simply opening it up and lamenting "where is this, why can't I do that". OK, some things from Studio are missing or hard to find, but there are also things that DWMX adds that Studio doesn't have.

And DW is much more than the WYSIWYG html editor that basic web page developers liked so much for years. If you don't like it "mucking with your code", don't go into design mode. You can still leverage the database tools, site management features, and so much more.

Still, I know it's not perfect. Some people have reported real problems using it. Some factors relate to using certain features, or working in certain environments, or problems with resources and speed (a faster machine with more memory seems to temper those concerns). Anyway, Macromedia is listening and future updates will surely address our most grievous complaints.

In the meantime, Macromedia really didn't really leave us high and dry. For died in the wool Studio fans (or when things just don't work as you need them to in DWMX), did you know that the DWMX CD comes with a separately installable "HomeSite +"? HomeSite Plus is really nearly entirely the same as Studio 5. Even the CFML debugger is included (for those that leveraged that). Just install it and use either one. Indeed, there are easy means to go back and forth between them.

I recommend you install both and just try to work more and more in DW as you grow comfortable with it. I know many people tried a "cold turkey" approach to force themselves into DWMX. Why suffer the hassle? Use each while you become more familiar and comfortable with DWMX. You may even find that you grow to find it perfectly acceptable, even preferable.

Sure, for some this has been like being forced to write with their other hand. But we humans are amazingly capable of adapting. And Macromedia really is working to make it a more effective transition. At least give these resources 15 minutes reading before you conclude it just won't work for you, and for the things that just flat don't work, drop back to HomeSite+.

Not a sermon. Just a thought. :-)

Building XML dynamically with CFXML

Some may know that they can create XML dynamically in CF, just like creating HTML, by outputting the elements such as within query loops, etc. Here's an example:
<CFQUERY NAME="getemps" DATASOURCE="cfsnippets">
   SELECT *
   FROM Employees
</CFQUERY>

<cfxml variable="employees">
<Employees>
<cfloop query="getemps">
   <Employee>
      <cfoutput>
      <Emp_ID>#getemps.emp_id#</Emp_ID>
      <FirstName>#getemps.firstname#</FirstName>
      </cfoutput>
   </Employee>
</cfloop>
</Employees>
</cfxml>

<cfdump var="#employees#">

While you could do the same sort of thing in CF5, I've added above something that's specific to CFMX. Note that the creation of the XML elements is done inside a CFXML tag. This causes CFMX to create the XML as an internal XML object, stored in the named variable ("employees", above). Doing this allows us to treat the XML as a first class XML object with a variety of internal variables and controls built-into CFMX. See the chapter on working with XML in the "Developing CFML Apps with CFMX" manual.

When building XML this way, however, it can be challenging if you make the slightest mistake. :-) I have a few generic tips:

• make sure that matching opening and closing xml elements are the same case
  
• when building XML dynamically, it's possible that the source of the data may provide values that are inappropriate for use within XML (special characters, or worse, possibly including brackets that are interpreted as elements themselves. You should wrap such dynamic data in an XMLFormat function.
  
• when stumped, you might want to replace the CFXML tags with CFSAVECONTENT (no other changes needed) and try outputting that for debugging purposes. While it would seem that using tostring against the xml object would provide the same result, since the SAVECONTENT doesn't really invoke the underlying XML processing, you might see something in outputting that which would cause an error in CFXML processing. This may be especially valuable if you're building the actual XML element names dynamically
  
• while minimizing whitespace has some value, it might also help to render the source code with normal indenting (as I have above) to help give visual cues as to whether things are coded as expected
  
• try stripping out parts of the process to minimize it and make sure that outer portions of the XML generation process are indeed what you think it to be
  
  Finally, someone may wonder how to simply dump all the contents of a table into XML without needing to spell out the column names. The concepts are again similar to outputting HTML:
  
  <cfxml variable="employees">
  <Employees>
  <cfloop query="getemps">
     <Employee>
     <cfloop list="#getemps.ColumnList#" index="field">
        <cfoutput>
        <#field#>
           #xmlformat(getemps[field][currentrow])#
        </#field#>
        </cfoutput>
     </cfloop>
     </Employee>
  </cfloop>
  </Employees>
  </cfxml>
  
  <cfdump var="#employees#">
  
  
  

Resetting the "other" ip address in the Developer Edition of CFMX

Are you running the developer edition of CFMX? If so, you may know that beyond running code yourself (as localhost or 127.0.0.1) you can also let up to one other IP address connect to the server. This is great for showing the results of some development to a client, etc.

But you can't just expect that IP address to be "removed" when the server is restarted. It's stored in a file and will remain there seemingly locking you in to only allowing that one IP address to visit your site.

The trick is just knowing where to change the value. In your [cfusionmx]/lib/license.properties file, delete the line:

allowedIP=nnn.nnn.nnn.nnn

Then restart the server.

Getting to the Command Line in All Windows OS's

Ever need to run some command or batch file in Windows from the command line window, where you can see the response and enter still more commands? 

It's pretty easy to get to the command line window in Nt 4, Win2k, and XP. Use Start>Run (or Windows key+r) and type "cmd". That opens a new window in which you can both enter commands and view their results. 

But that approach doesn't work on Windows 95, 98, or ME. In those cases, you may feel you have to tell them to use Start>Programs>Command? or might it be MS-Dos Command? Who can keep track of these things, darn it. Take comfort. They can use the Start>Run approach.  Here's the solution on Win 95/98/ME: it's just that they need to type "command" rather than "cmd". 

Oh, and if you may be new to this stuff yourself, note that when you open the command window and are done, you can type "exit" to close the window. 

By the way, if you haven't used this command line window to run programs and might instead type the command name in on the start>run prompt, just be aware that doing that won't let you see the response from the command, and may even make some other aspects of working with the command challenging. Consider using the "cmd" or "command" approach instead. 

 It's easy to presume that everyone knows these things, but often they don't.

Creating JSPs and Servlets in CFMX

Many may have missed that one of the cool new things in CFMX is that it allows integration of JSPs and servlets, meaning (for
one thing) that you can actually place them within the CFMX environment and run them, as well as pass control to them, share variables, andmore.

For instance, (in the Enterprise Edition only), you can place a JSP page in your CFMX web application directories right alongside your CF templates and calling it with a URL just like CFM page (using .jsp for the extension of course in the file name and the URL).

And servlets can be run as well, by placing them in the WEB-INF\classes directory and then calling them with:

http://yourdomain[:port]/servlet/ServletClassName

Those familiar with Servlets may know that sometimes you don't want to use the /servlet/ approach. You can create a virtual mapping instead by editing the file
[CFusionMXhome]\runtime\servers\default\SERVER-INF\default-web.xml.

To create a mapping for a servlet named SimpleServlet residing in the cfusionmx\wwwwroot\WEB-INF\classes directory, for instance, so that it could be called as /simple, just put the following in between the <web-app> element tags of that file:

<servlet>
<servlet-name>SimpleServlet</servlet-name>
<servlet-class>SimpleServlet</servlet-class>
</servlet>

<servlet-mapping>
<servlet-name>SimpleServlet</servlet-name>
<url-pattern>/Simple</url-pattern>
</servlet-mapping>

For more details on this sort of configuration, as well as on how to write and compile JSPs and servlets, see any good JSP/servlet book (including the fine JRun docs at livedocs.macromedia.com).

Those familiar with mapping virtual paths for servlets may also know that you can disabling the "servletinvoker". That's done by editing the file cfusionmx\runtime\servers\default\SERVER-INF\default-web.xml and commenting out the servlet definition for the ServletInvoker servlet.

One interesting aside is that while the Professional edition of CFMX specifically prevents execution of JSPs, it does not prevent execution of Servlets (of course, under the covers, CFML templates get converted to servlets in CFMX, so it may seem like there's no choice but that it should support them). But my reading of a few CFMX documents and the "edition comparison" suggests that running JSPs and servlets is an Enterprise-only feature. Be aware if running on Pro. Don't make any design/architecture decisions based solely on the fact that it "works", if it's against the license. I've reported this observation to Macromedia and am awaiting word.

Beware of case in calling Java CFX custom tags

When calling a Java CFX custom tag in CFMX, it is now critical that the case of the tagname in the call to the tag (as in ) match the actual case of the tag as defined in the CF Administrator. This is a change of behavior from CF5 and is not mentioned in the docs on calling Java CFX custom tags (Chapter 12 of the "Developing ColdFusion MX Applications with CFML" book).

While it has indeed always been true in 4.52, 5, and MX that the case of the classname as specified in the Administrator had to match the name of the actual .class file, you could use any case for the call to the custom tag as in . Now, the case of the call must match the name specified in the Administrator as well. This could cause a problem for old code during migration, especially for any situations where the same tag is called using different "case" in multiple templates.

Updates on problems with J2EE Sessions

I mentioned recently that I'd done an article in the CFDJ on using J2EE sessions. Since then, I've learned of a few challenges that can make working with them difficult in certain circumstances.

First, when you enable J2EE sessions, CFMX no longer creates the variables session.cfid and session.cftoken. If you're using those to attempt to persist sessions when (or in case) cookies aren't supported, those values are no longer available once you enable J2EE sessions.

You might think to try to use the available session.URLToken variable instead (or cookie.jsessionid, which does exist on the server while the page is being executed). Unfortunately, nether of these will work.

CFMX can only receive a jsessionid on a URL if it's appended after the filename and extension as ;jsessionid=nn, a format that was discussed in the article. This is the reason for the URLSessionFormat function, which was also discussed in the article. But that has problems as well.

I had mentioned in the article that the new URLSessionFormat would, when when using J2EE sessions (and when the browser does not present the JsessioID cooke), cause the resulting URL it generates to appear in the format templatename.cfm;JSESSIONID=8030949691025145133137. Note that the JSessionID is appended to the filename separated by a semicolon, rather than to the query string.

I added that "Still, when that's happened, the URL has functioned as expected."

Well, I was doing testing at the time on the built-in CFMX web server. I've since discovered that if you're doing this on an IIS web server, the use of the ;jsessionid after the filename causes a 404 file not found. This is disappointing.

If you use J2EE session variables and are working under an IIS web server, don't use the URLSessionFormat, at least until this problem is resolved by MM (or MS).

And that's not all. If you enable J2EE sessions and then use CFLOCATION, and a browser executes the page but doesn't support cookies or doesn't present the JsessionID such as on the visitor's first visit, CFMX will append the ;jsessionid after the filename and extension as discussed in the article. That's great for the built-in CFMX web server.

Unfortunately, if the redirection is to an IIS server (whether yours or another), the request will fail. Even using the ADDToken="no" won't stop it doing this.

Either MM needs to address this, or Microsoft needs to change IIS to allow this sort of URL.

And for final clarification, in case you missed it above, this issue of how URLSessionFormat or CFLOCATION determines whether to append the needed sessionid info is done not based on whether the browser supports cookies or not (CF can't really tell that), but it's simply a question of whether the expected sessionid cookie is passed: if J2EE sessions are enabled, it looks for a JSessionID cookie, if they're not, then it looks for the CFID and CFTOKEN cookies.

Not only will they not be present in a browser that doesn't support cookies but also they won't be there on the very first page that the user visits even if they do support cookies. These issues above could bit you then.

Restarting the CFMX Service

Many may know that when starting and stopping services in Win2k and WinXP, there are a set of buttons at the top of the Services window similar to stop (square) and play (sideways triangle) buttons on a VCR. Some maybe hadn't noticed that there's also an icon that combines them both, effecting a "restart service" in a single step, with a combination of a square and a triangle as its icon.

But as nifty as that "restart" button is, I've found that sometimes trying to use it with CFMX has caused problems. Sometimes the process of stopping and then starting the service takes quite a while, and It seems that perhaps there's a time limit that that restart interface is willing to wait before complaining that it took too long. I could be wrong. But I've gone to using the stop and then start buttons instead and it seems to have helped. Still, it's occasionally still taking too long to start the service and still the OS complains that it couldn't start it.

I've recently switched to using the command line "net stop" and "net start" commands instead, in the hopes that there's no time limit in their responding. For those not familiar with this, you can run the following from the command line, or create a shortcut or batch file, executing the following lines:

net start "ColdFusion MX Application Server"

and

net stop "ColdFusion MX Application Server"

I'll see if this stops the problem of having the other start/stop approach seemingly timeout.

If anyone has more information about this apparent timeout challenge, let me know.

Wondering about your java configuration running under CFMX?

If you've wondered about various settings of the Java environment configuration you've got running under CFMX--such as what the classpath is, what version of the JVM is running, you can easily see that using the CF Administrator. No, I don't mean by going to the "Java and JVM" settings page. That's where you can modify these things. I mean the "version information" link that's at the top of the administrator.

There's a wealth of info there, about both the Java and CF environments (such as CF Version and license info). Of course, the JVM info there is info you can get from the JVM itself if you know how to do so, but it's nice to know you don't have to.

And though you may think that the "Java and JVM" settings page in the admin tells the same thing about the JVM and the classpath, well, first there's a lot more on this "version" page than just that. Second, the "Java and JVM" page may show the location of the JVM, but it doesn't tell you what version of the JVM it is. Finally, though it lets you enter info in the "class path" field, what you may not realize is that there is already info in the classpath even before you enter a value there.

What if you don't have access to the CF Administrator? Again, if you know how to, you can find some of the JVM info on the command line in that the JVM's directory (using "Java -version" for instance).

But you can also find things out if you know how to use the Java system library API. If that sounds foreign, don't worry. Just save the following script as getversion.cfm in a web directory of CFMX and run it:

<cfset sys = createObject("Java", "Java.lang.System")>
<cfoutput>
JVM name: #sys.getproperty("Java.vm.name")#<br>
JVM version (build): #sys.getproperty("Java.version")# (#sys.getproperty("Java.vm.version")#)<br>
JVM class path: #sys.getproperty("Java.class.path")#<br>
</cfoutput>

When I ran this on my machine, I was surprised to see that CFMX's built-in JVM is running the HotSpot compiler. That's good to see.

And the classpath had much more than just what I may have entered in the JVM Path in the administrator (actually, this is not new. CF 4.5 and 5 did that as well.)

More details on configuring the built-in web server in CFMX

In an Aug 15 entry I wrote about how you could configure the built-in web server in CFMX to support your accessing code outside the built-in wwwroot. There's a very interesting article in the MM support center, Configuring the Macromedia ColdFusion MX built-in web server, that goes into much more detail on this and more. Definitely worth checking out.

Indeed, note that they show an example of configuring a virtual mapping for \* which can effect a replacement of the location of the default webroot. I had discussed it just for adding additional paths outside the default root. Clearly, both uses have their place.

Understanding and Setting up Resource/Sandbox Security

If you've wondered about CFMX's new Resource/Sandbox security feature--or especially if you have no idea what it is, or think it doesn't apply to you, you should check out my two articles just posted in the new Macromedia Security Development Center. I think every developer and administrator ought to understand these features, as well as know the difference between Resource Security (in Professional) and Sandbox Security (in Enterprise).

It's all much more straightforward than Advanced Security was in CF4 and 5. Plus, it extends greatly what Basic Security offered in those releases.

Plus, beware that if you had setting for Basic Security in CF4 or 5, those are NOT automatically migrated into the new security controls in CFMX. You have to create the controls anew in CFMX. The articles fully explain how to understand and setup these new features.

The articles titles are:

• ColdFusion Security, Part One: Understanding Sandbox/Resource Security, at http://www.macromedia.com/desdev/security/articles/sandbox_01.html
  
• ColdFusion Security, Part Two: Sandbox/Resource Basics, at http://www.macromedia.com/desdev/security/articles/sandbox_02.html
  

Fun with Filters and Listeners

Among the cooler new things in CFMX, which becomes available to us by way of the underlying J2EE engine, may be the concept of filters and listeners.

In the JSP/servlet world, these are discussed as a means to provide some sort of global functionality to your applications. Now we can use them with ColdFusion as well.

This first discussion will focus on filters. These could allow every request to your application (or a part of the application) to cause something to be done before running the requested page. In a way, this may seem much like CF's application.cfm.

The filter might make a decision about what template to execute as a result of analyzing the characteristics of the request (the ip address or user agent of the requester), or it might even modify the incoming request data.

And just as we have onrequestend.cfm which can cause some code to execute before a page is returned to the user, filters in jsp/servlets can also be used to do post-processing for them. It might modify the output before sending to the requester, or it might block it entirely.

Now with CFMX, we can use filters with CF templates.

Examples of how you might use them include authentication, logging and auditing, image conversion, data compression, localization,
XSLT transformations of XML content, encryption, caching and more.

You can learn more about filters at http://java.sun.com/products/servlet/Filters.html. There are also articles on listeners O'Reilly and Java Boutique, as well as a chapter in More Servlets and JSP by Marty Hall.

While the focus of most discussions in JSP/servlet articles and books is on how to create such filters, the fact is that you may find one created by someone and available on a public repository of them that already does something interesting for you.

And we as CF developers don't really need to know any Java (or even understand servlets or JSP) to be able to leverage a filter that someone else has written.

There are repositories of filters at various places. One of them is the ColdJava suite. This really has nothing to do with ColdFusion, despite the name.

In a later article, I'll walk through the process of using one of these, but with this pointer to where to learn how to work with them, and where to find them, many will have enough to figure things out on their own.

Before concluding, I had mentioned that another new JSP/servlet concept available to us was "listeners". There are two kinds of listeners, application and session, which can act as triggers that can cause execution of a program when an event takes place in the application or session contexts.

Now, the application and session contexts, as well as the programs that can be fired in response to events, would all be Java ones. What does that mean to us? Well, again, Java code may already exist. Further, we can setup the CFMX environment to use listeners for the application and session contexts just as with JSPs or servlets.

In a later blog entry, I'll show some practical applications of using filters and listeners in CFMX.

For now, you can learn more about listeners and the "servlet life cycle" on your own at O'Reilly and Sun.

Using JSP Custom Tags in CFMX: Why You Should Care

In ColdFusion MX, it's now possible to use JSP custom tags. But what does that mean to you? Well, you don't need to know Java (nor understand JSPs) in order to use tags. They're very easy to use, and libraries with hundreds of them are available to the JSP world.

There may be solutions that you could use that have been created as JSP custom tags, and they're very easily made available to you in CFMX in just a few simple steps.

I explained this in my May 2002 CFDJ article, "Using JSP Custom Tags in CFMX: What, Why, and How ". You can read it, and all my articles, online at http://www.systemanage.com/cff/articles.cfm.

I also will be covering the topic in my Macromedia DevCon talk, "Incorporating JSP Custom Tags Into Your ColdFusion Apps" (http://www.macromedia.com/v1/conference/ConferenceProgramDetails.cfm#SS314W.

I'll also post a couple of specific and interesting examples in later blog entries. Finally, one of those will also leverage a couple of very unheralded features in CFMX which comes to us by way of the underlying J2EE server: filters and session event handlers.