I was "Stumbling" around the internet the other day and came across this web site which has an application called "PortableApps.
the site is
portableapps.com
You should really check it out. I installed the suite on my usb stick and I now have portable (i.e. no install) versions of Open Office, Firefox, Gaim/Pidgin, Gimp and others. I even put Sunbird on to handle my portable calendar.
Firefox
My personal favourite web browser, installed on a usb stick with no install necessary on the host machine.
Open Office
This would be great for kids at school, or when visiting.
7-Zip
I know that windows has built in zip functionality, but 7-Zip does more
Gimp
I sometimes travel (without my laptop) and when I'm staying over if I have some graphics editing to do, a nice portable version of the Gimp helps out.
Nvu
A nice little html editor all portable on the stick
Notepad++
If you're like me, you can't stand editing text files with notepad. I generally use gvim, but this is a nice little programmers editor (with syntax highlighting, all installed on the usb stick).
Gaim/Pidgin
Now, this one would appeal to teens. I don't know how many times I've had teenagers visit, only to find that they've installed some new version of MSN on my machine...or left themselves logged in. A nice portable version on their own memory stick would be helpful.
ClamWin
A portable and free virus scanner. Perfect for when you're doing a little "tech support" at a friends house.
WinSCP Portable
Portable FTP, SFTP and SCP client.
FileZilla Portable
Now we're talking, a portable version of my favourite GUI ftp client. Now I can take all my sites with me.
There are other apps available beyond the original suite too and more being added all the time. Note that this is a windows only deal

Still, it's free and very cool.