Linux Neophyte

Due to a problem I’ve been having with centerim and (I think) my MSN account, I’ve decided to give Finch a shot. For the uninitiated, Finch is the console client that comes with the Linux Pidgin package. Since the Debian Etch machine I run my IM client on has no graphical environment, Pidgin isn’t an option but Finch should fill in nicely.

I can’t install using the package manager because Finch doesn’t exist in the default Etch repos (the price of running the stable distro). In reality, I could install it from etch-backports, but it has a lot of dependencies that I don’t want on my system (pidgin & x11 libs among others). This leaves me one option, compile it myself.

• I’ll be compiling Finch 2.3.1 (current version as of this writing). I won’t be compiling Pidgin, so I won’t need to install the development packages to support that program.
• I’m performing a default install, this will put Finch and it’s files into the /usr/local path
• I’m running all of these commands as my regular user, the root password will be required for two steps.

Install everything needed to build the Finch client

su -c 'apt-get install build-essential linux-headers-2.6-686 libxml2-dev intltool libglib2.0-dev gettext libncurses5-dev libgnutls-dev'


Download the source files and extract them (~/src is where I keep my source and build files):

mkdir -p ~/src ; cd ~/src

wget http://downloads.sourceforge.net/pidgin/pidgin-2.3.1.tar.bz2

tar xvjf pidgin-2.3.1.tar.bz2

cd pidgin-2.3.1

Run configure; Feeding it the --disable-gtkui argument tells it that I only want to build the Ncurses client (Finch)

./configure --disable-gtkui

Once the configure script completes, it’ll output Finch’s configuration information (kinda neat to read). I can review it later by reading ~/src/pidgin-2.3.1/config.log.

Now I build the binaries and install, the make command will take a while to do it’s job and output all sorts of stuff to the screen. It looks awesome if you’re doing this from your desktop with transparent terminal windows and a dope background. I like to run top in one terminal and compile in another, then I toss on some 90’s techno, run a small screen visualization and imagine myself doing something cool like hacking the Gibson.

make

su -c 'make install'

I now have finch running and ready to go. When the next version of Pidgin/Finch is released, all I need to do is go back into the directory I built in (~/src/pidgin-2.3.1) and run the command su -c 'make uninstall' to remove the program’s files. Following that, rm -rf ~/src/pidgin-2.3.1, and follow the whole process again substituting the filename/path where applicable.

written by M@ \\ tags: Apps, Compile, Debian, Etch, Finch, Pidgin

While working out a strategy for DVD backups on my server I found some handy tools for working with ISO files. I used some of these to prevent wasting media on test burns (sure, re-writable media would be good, but why not live on the edge and toast all of your WORM media?).

isoinfo:

This is the one that saved me from wasting (a lot of) media so I’ll start with it. Since I need to create DVDs that can be read both in Windows and Linux I create my media utilizing both the MS Joliet specification and Rock Ridge standard. Basically, both of these extensions to the ISO9660 standard allow for additional file information to be stored on CD/DVD (long filenames, permissions and so on).

Initially I was attempting to create an ISO containing files with names exceeding 64 characters. Joliet technically only allows for 64 and was doing awful things to my file names (umm hello, truncating the extension?). Isoinfo (isoinfo -i <iso> -lJ) allowed me to see that the Joliet filenames were broken. Had I burned that DVD it would have appeared fine on my Linux machines (as they would be referring the RR data on the DVD) but would have been been a problem on my Windows boxes. Just that command saved me at least two discs

Handy isoinfo commands:

isoinfo -i <iso> -lJ: View info from Joliet extensions
isoinfo -i <iso> -lR: View info from RR extensions
isoinfo -i <iso> -d: View ISO’s properties

isovfy:

Isovfy’s man page states: isovfy is a utility to verify the integrity of an iso9660 image. Most of the tests in isovfy were added after bugs were discovered in early versions of genisoimage. It isn’t all that clear how useful this is anymore, but it doesn’t hurt to have this around.

Lack luster description, I personally don’t know how useful it is either but I figure that if you download an ISO, it couldn’t hurt to run this proggie on it for a quick sanity check.

isovfy <iso>

isodump:

Yeah, this one is a mystery to me. I personally couldn’t find a human use for this one at all. isodump is a crude utility to interactively display the contents of iso9660 images in order to verify directory integrity. The initial screen is a display of the first part of the root directory, and the prompt shows you the extent number and offset in the extent.

Feel free to try it on your ISO, just keep in mind that the man page is accurate, it’s crude. Personally, I get what I need from the other two utils so I’ll just forget about this one.

That’s all I have on ISOs for now, check out the wikipedia links throughout the post, if you’re a little geeky it’s interesting reading

written by M@ \\ tags: Apps, Archive, CD, CLI, Console, Debian, DVD, ISO, isoinfo, isovfy

I tend to create DVD archives of data. I’ve always done this in Windows using Nero to burn my archives and then I store them in a cool dry place. Since I tend to keep everything on my headless Etch server in the basement now, I need a new way to burn off/save my data. I spent some time this afternoon reading up on creating ISOs and burning them.

Note: I did all of this as root

First, creating an ISO image. What I need to do is create an image from a directory of goodies that I need to back up. The directory I’m backing up contains 4.4G (or one single layer DVD) of random MP3s and I want them written to the root of the DVD when it’s finalized.

I started by reading the man page for mkisofs. That got me started, and then a quick read of http://wiki.linuxquestions.org/wiki/Mkisofs finished me off. My first few attempts just gave me ISOs containing MP3s with short filenames or short filenames in all CAPS. Not what I wanted, these are all descriptively named files and I want to retain the names. The command below worked perfectly.

mkisofs -r -J -l -d -joliet-long -allow-multidot -V Mp3_003 -o ./Mp3_003.iso _Archive/Mp3_003/*

All of those options are important, -r makes the files publicly readable (among other things), -J preserves case (upper/lower), -l allows long filenames, -d doesn’t add a period to the end of files that don’t have them, -joliet-long allows up to 103 characters (vs 64), -allow-multidot allows files to contain multiple periods and -o specifies the output file name, . Optionally, (but why wouldn’t you) -V sets the DVD’s Volume name (up to 32 characters). As is typical, there are many more options available to you, spend a little time reading the man page for more info.

Note: -joliet-long was key for me. I found that when I attempted to read files with filenames longer than 64 characters on Windows, they would be truncated and NOT in a good way. The filename would be truncated in reverse, starting with the file extension. A 65 character file with a .MP3 extension would end up with a .MP extension. Ugly. That said, the man page warns that this breaks the joliet specification and to use with caution. You’ve been warned.

After a few moments you’ll have an ISO image containing all of the files you specified. In my case, all of the MP3’s I’ve collected since I burned my last backup.

Now to write it to my media:

cdrecord -v -sao dev=/dev/dvd Mp3_003.iso

-v sets verbose mode, -sao enables session at once (no multi-disc), dev=/dev/dvd specifies the device to use and finally, the input file; Mp3_003.iso. There are A LOT of options available to you, you’ll want to spend a little time with man cdrecord.

Note: You don’t need to specify the device if you have a properly configured /etc/wodim.conf. If you’re wondering why it isn’t /etc/cdrecord.conf, the answer is that cdrecord is nothing more than a sym link to /usr/bin/wodim on Debian (and possibly other distros). You can verify this on your box with which cdrecord | file -f -.

My first write failed but it was because I have two VMware virtual machines running on this box and they were attempting to poll the CD-ROM drive for media. The fix was to disconnect the drives in the VMs and then burn.

The resulting CD should be readable in any modern OS. I have no problem accessing the files from Linux or Windows and feel confident that I have a solid solution for creating DVD archives.

written by M@ \\ tags: Apps, Archive, CD, cdrecord, CLI, Console, Debian, DVD, ISO, mkisofs, wodim

Now that I’ve got my physical machine up and running, I can get started with my virtuals. I’m going to go into some detail on the Linux one and not so much on the Windows one.

The Linux VM (referred to from here out as Ka) is going to be running my fully configured postfix server (acting as a relay for other machines on my lan), my wiki, my email (I use mutt) and the config files that go along with it, my IRC client and my IM client. This is where I want to keep my always on services. My Windows VM is named Peeves. It’ll be managing my online address book and my FTP server, not much else.

• You may be wondering why my Samba server is running on the physical machine and not this one. It comes down to this; if I store all my media on the VM it becomes much harder to move it to a new machine and get it running if something ugly happens to the physical server. So, if something goes awry, I’ll be without my media for a while but I won’t be without mail and communications.

I performed the same installation/configuration I covered on the physical host with some changes when it came to installing my packages.

apt-get install ssh vim build-essential linux-headers-2.6-686 bzip2 p7zip-full less lsof lshw psmisc w3m lynx nload telnet host dnsutils ftp lftp vnstat file screen irssi unrar cksfv rsync smbfs samba postfix wakeonlan apache2 php5 libapache2-mod-php5 rcs libcgi-session-perl libdigest-sha1-perl libarchive-tar-perl libsasl2-modules mailx mutt procmail fetchmail urlview abook libgpgme11-dev libgpgme11 automake libgnutls-dev libgnutls13 autoconf libncurses5 libncurses5-dev gettext cogito liblzo1 liblzo-dev libcurl3-openssl-dev

• Some of the applications (samba and postfix for example) will require that you provide some setup information. You’ll want to configure them for yourself.
• If you don’t know what these packages are, you can do an apt-cache show <package> on Debian or Ubuntu.
• It’s not obvious why I need all of these packages, many of them are required by Twiki and other applications I plan on compiling/running.

I also installed VMWare tools, I want the host to be able to start and stop the VMs in an orderly fashion and the tools allow that.

Now that everything is installed, I download and install Twiki (this will be the subject of another post). I use it for everything I need to document. That includes records of upgrades/changes to the house, complete and incomplete projects, info on TV shows we watch, side work I’ve done, fishing stuff (knots, laws, etc), gardening tips and schedules, and just about everything else I ever write down. Now that I have a wiki, I can’t imagine NOT having one… End rant on wiki, moving on.

Next I download and compile centerim. They have a list of dependencies and fixes for gotchas on the site. It was simple to do. It was between this and Pidgin, when it comes to console IM, I just prefer the look and feel of centerim.

Now I take a few minutes to configure Samba, I’m only sharing home directories. This makes it easy for me to save attachments from my email and have them quickly available to GUI desktops and Windows machines.

The next step was to configure postfix. I’ll be honest. I’m NOT a pro at ALL. I’ve been able to get postfix to act as a relay server for my LAN (and only my LAN) and to use SSL to authenticate with my ISP’s SMTP server for mail relay. I hand outgoing mail off to my ISP’s server to help prevent mail from me being marked as spam by pretty much every mail server in the world. It works and now that I have it that way, I’m just not touching it. I have a book on postfix that I hope to read (eventually). Maybe I’ll be a little more confident following the read and I can elaborate on this.

With that complete, I move on to building the Windows VM and shutting down my older hardware. This is the best part. I’ve shut down the two machines in the basement (albus & voldemort) and moved cerberus (formerly nix) to the dungeon. I can now keep darktower in standby as it’s no longer acting as a server and all of my documents and pictures live on the Samba server. Come the end of this month I’m going to (attempt to) use the Gparted Live CD to re-partition darktower and install Gutsy Gibbon, converting the machine to a dual boot.

My next step is to develop a backup routine and some scripts to automate it. I’ll be keeping my data on a remote Debian machine that I manage and I’ll be using SSH to transfer my data to it.

written by M@ \\ tags: Apt, CenterIM, Debian, Gutsy Gibbon, Hardware, How-To, IM, IRC, IRSSI, Linux, Migration Project, Mutt, Postfix, Samba, Twiki, Ubuntu, Virtualization, VMware, Wiki

Before getting into the virtual machines, I should probably address the physical machine that’s going to run them. For hardware info, check out this page under the heading Nix/Cerberus. I’m going to be installing Debian Etch (stable) as the OS, and I’ll be doing it in the most minimal way. Read on…

During the installation of Debian, you’re asked several questions, for the most part you need to answer them for yourself. The important question, the one that gets you a basic install, is package selection. By default the installer wants to include “Desktop Environment” and “Standard System”, I chose to omit both. This leaves you with just enough to get the machine running (which IMO is a very good thing).

After the installer does all it needs to do, it’ll reboot the machine and leave you at the login prompt. If you’ve never sat in front of a GUI-less machine, then you’re in for a treat. Unplug your mouse, chuck it in the bin and get used to virtual terminals, screen sessions and tired fingers

First things first, I need to get this machine going and I don’t have enough software to do all I need. I log in as root and make a few changes.

Remove the Debian CDROM as a source from /etc/apt/sources.list and Add contrib and non-free to the same file; it should look like:


deb http://ftp.us.debian.org/debian/ etch main contrib non-free
deb-src http://ftp.us.debian.org/debian/ etch main contrib non-free
deb http://security.debian.org/ etch/updates main contrib non-free
deb-src http://security.debian.org/ etch/updates main contrib non-free

Then update the Apt cache (apt-get update) and install my applications…

apt-get install ssh vim build-essential linux-headers-2.6-686 bzip2 p7zip-full less lsof lshw psmisc w3m lynx nload telnet host dnsutils ntp ftp lftp vnstat file screen unrar cksfv rsync hellanzb samba smbfs wakeonlan postfix mailx

• Some of the applications (samba and postfix for example) will require that you provide some setup information. You’ll want to configure them for yourself.
• I actually installed hellanzb from Debian testing as it’s a more current version of the app.
• If you don’t know what these packages are, you can do an apt-cache show <package name> on Debian or Ubuntu.

This machine is not only going to host some virtual machines, but it’s going to act as a samba server, a shell server and the machine I use for various Usenet tasks and some other goodies. Given that, and the fact that I like to know what’s going on, I’ll be keeping an eye on my bandwidth usage. To do that, I’ll be using vnstat. Since it’s installed already, I need to create a database for my Ethernet adapter.

vnstat -u -i eth0


After that and some time, I can issue the vnstat [-<flag>] command to get some really handy network traffic statistics.

Because there are so many ways to configure a shell server and a samba server, I’m not going to get into the editing of the conf files. Basically, authentication with the shell server will be done via public/private keys only and I’ll be running it on an alternate port. Samba will be configured as a stand alone server and I will be required to enter a valid user/pass to view the shares.

Once all of those applications are installed and configured, one needs only to download the latest version of VMware Server and install that. I responded with all of the default answers while installing VMWare and was up and running in almost no time.

After a few other tweaks, preventing users from entering /root and other user’s homes, adjusting umask for my account and installing ClamAV from the Debian Volatile repos (to scan samba shares), I’m up and running.

There you have it, the quick setup for the physical server that’s taking over @ the flat. Next I’ll start working up the VMs.

written by M@ \\ tags: Apt, ClamAV, Debian, Debian Volatile, Hardware, How-To, Linux, Migration Project, Samba, Virtualization, VMware, Vnstat

Last time I wrote, I gave the overview of a project I’ve begun at the house. Here’s the hardware I’m working with and the roles the machines will be performing (or not, as the case may be).

These first 2 guys are the older noisy machines that I need to replace. They have loud fans and louder hard disks. I don’t notice the noise so much since I moved them to the basement, but I *know* they’re down there making noise…

Voldemort:

A Pentium III (Coppermine) 1GHz running on a SuperMicro P3TSSA motherboard. This baby is maxed out with 512MB PC133 SDRAM. This was a great machine 7+ years ago. I had a lot of fun on it. I built it to learn/play with Windows 2000, it had a whopping 256MB of RAM back then and was moderate to high tech when I built it. This isn’t the original mobo, it had an Intel branded board that went kaput some time after I installed XP on it. I’ve loved this machine for MANY years and while it won’t be running 24/7 anymore, it’ll still be there for me to play with. It’s currently running Debian 4.0 and hasn’t had a real role other than for testing for a while. Most recently, it’s been host to a VM or two that I’ve been playing with.

Albus:

A Pentium III (Coppermine) 1GHz running on a Dell GX150 motherboard. This one is maxed with 512 MB as well and used to be my work desktop many years ago. We were able to pick up our machines before they were refreshed for a modest price so I opted to. I don’t have any really fond memories with this machine. It’s always been on and I’ve never once had a problem with it. Not a bad little box. Currently it’s running Debian 4.0 (upgraded from Debian 3.1) and its primary role is a backup server (I’m using an offsite backup server after this whole project is complete) and shell server. it’s the single door into my home network.

This next machine is the physical box replacing both Albus and Voldemort.

Nix/Cerberus:

Credit: Cerberus was given his name by none other than Sarah Conti. Rock on with your mythology Miss Conti!

A Pentium 4, 3.00GHz (Hyperthreading) running on an Intel D945GTP motherboard. It’s currently rocking 2GB of DDR2/667MHz memory. This is a great machine for running a standard desktop. I built it in March 2006 to be my Ubuntu desktop, and I’ve never had a problem with it. Currently it’s name is Nix but will soon become Cerberus. It will be running Debian 4.0, VMware server and will be providing shell access (replacing albus) and will act as a Samba server for my music and whatever else I need to share.

…and without further delay, the virtual machines!

Ka:

Ka is a Debian 4.0 Virtual Machine. His role is to be an always on machine for my wiki, email, IRC & IM clients. He’s already alive and well and I’ve moved those services over to him. For the moment he’s hosted on Voldemort and will be moved to Cerberus once he lives.

Peeves:

Peeves is Windows Server 2003 Enterprise. Yep, I need to be able to virtualize Windows because I have a few apps that I need/want to have on at all times. This includes my FTP server, Handy Address Book server and WhereIsIt which is a cataloging program that just rocks. I decided to go with Windows 2k3 because I had a license for it and didn’t have one for another copy of XP. It’s currently being hosted by Darktower (you’ll meet him below)

Finally, the desktop machine/VM to get me through this whole mess.

Nix2/Darktower:

Nix2 is just an Ubuntu VM so that I can continue to use my Linux apps while converting Nix to Cerberus. Nix2 will be hosted on Darktower. This is a physical machine similar in make-up to Nix (only diff is that it’s a full ATX board, not a Micro and it’s in a taller case). Darktower runs XP at the moment but will eventually be a dual boot with Ubuntu and XP or Vista. I plan on keeping all data off of this machine, it will be nothing but a client as all of my data will be accessible via the servers. It’s final name will remain Darktower. Once I’ve got my data off of Nix2 and onto the new Darktower, Nix2 will get the boot and all will be good in the world.

So to summarize, both Albus and Voldemort die (seems fitting), Cerberus, Ka and Peeves rule the roost and Darktower just hangs out getting poked and prodded when needed. Since I won’t need it to be on at all times anymore (for IRC and what not) it’ll be spending the majority of it’s time in sleep mode.It’ll be nice to eliminate the 2 older machines and keep a third sleeping most of the time. That will leave only 2 always on machines in the house, that’s Cerberus and HTPC (which is a Windows machine running some home theater software).

Once complete, I’ll not only have infrastructure that’s really easy to backup/re-create but I’ll have stopped wasting so much energy.

Next, I’ll start going over what’s involved in setting up the VMs, things from Debian installation options to installed applications.

written by M@ \\ tags: Apt, ClamAV, Debian, Debian Volatile, Hardware, How-To, Linux, Migration Project, Samba, Ubuntu, Virtualization, VMware, Vnstat