Portable Software

Usually when you install a program, it will also create files and leave traces of itself outside of the installation folder, for example in the Registry and in the "Users" folders.

A 'portable' program will keep all the files it uses inside its own folder. The settings are usually stored in one or more files (mostly with the extension ini). This not only keeps the OS partiton cleaner, but it also makes backing up said program easier, since you can simply copy the folder to a new PC or reuse it on the same PC after a clean install. All your settings will already be there, which can save a lot of time you might otherwise spend configuring the program.

Several programs come with "official" portable versions. Sometimes the portable version is provided by the installer—for example, with Foobar or Opera—where you can select whether it will be installed normally or in a portable way to your preferred location. Others offer the portable version as a separate download, usually in a zip file. In that case you just extract that into a folder and run it. For many more programs that don’t have an official portable version, there are 3rd party portable versions. The best known example is PortableApps.com, which hosts a lot of free programs. These portable versions still often receive official updates. Firefox Portable and Thunderbird Portable are some good examples.

Portable programs typically don’t create a shortcut on the desktop and don’t make file associations (so that double clicking on a specific file type opens with a specific program), but both of these can easily be done manually through the right-click context menu in Windows.
If you want to have a certain extension associated with a portable program, the fastest way is to simply right-click on a file with a specific extension and then select "Open with...", "Choose default program...", point to the portable program’s .exe and check "Always use the selected program to open this kind of file".