Just to say…

http://imgur.com/SfSSk.jpg

RHEL4 -> RHEL5 upgrade script

So, this has been done for months, but I haven’t gotten around to posting it. So, here it is. Upgrade RHEL4 to RHEL5 with an LVM snapshot of the running system and a backout script (you need to have free space in the VG to do this, really — pure LVM snapshots work, but it’ll fill up to 100% when you try to back out and leave it unbootable if your snapshot isn’t the same size as the regular partitions).

It assumes HP’s packages for Proliants are installed, and a tarball of a yum root with the packages you need to install are present. It rips out any package not on our list of “needed” packages (yours may be different). If you’re actually using a RHEL CD or a full repo (rather than just pushing what you need to remote sites), change the repo definition accordingly.

Variable names are odd, but those are easy enough to fix if you want something more descriptive of what it does:

#!/usr/bin/env perl
 
use Sys::Hostname;
my $host = hostname;
system("mkdir /var/upgrade");
open(my $output, ">/var/upgrade/upgrade.log");
system("cd /boot; tar jcf bootback.tbz *; mv bootback.tbz /root");
system("mkdir /root/bootsave");
system('tail -n 8 /boot/grub/menu.lst |head -n 4| sed -e \'s/LogVol00/ssLogVol00/\' | sed -e \'s/Red.*/Rollback to EL4/\' > /root/bootsave/menu.lst');
system("cp -v /boot/*2.6* /root/bootsave");
system("cp /etc/fstab /etc/fstab.original");
system("cp /etc/fstab /etc/fstab.rollback");
system('sed -i -e \'s/\(LogVol[0-9]*\) /ss\1 /\' /etc/fstab.rollback');
system("cp /etc/lvm/lvm.conf /etc/lvm/lvm.conf.save");
system('sed -i -e \'s/\(.*[^#]filter.*sdb.*\)/#\1/\' /etc/lvm/lvm.conf');
system('sed -i -e \'s:\("a/.*/"\):\1, "r|/dev/cdrom|":\' /etc/lvm/lvm.conf');
 
#I'm not happy about it, but our lastlog is huge, and there's no space for it anywhere in the snapshot. Save off a copy, at least.
#system("cp /var/log/lastlog /root && gzip /root/lastlog && cat /dev/null > /var/log/lastlog");
system("cat /dev/null > /var/log/lastlog");
 
system("cp -v /var/adm/sw/products/backout.pl /usr/bin/backout");
#Try to take snapshots.  Die if it fails, since we're not committed yet, and we want to ensure we can
#abort if necessary
my @lvs = `lvs --noheading --units k| awk '{print \$1, \$4}' | grep -v ss`;
system("mkdir /mnt/snap");
chomp(@lvs);
for my $lv (@lvs) {
        my @args = split(' ', $lv);
        my ($size) = $args[1] =~ m/^(\d+)/;
        $size = int($size * .3);
        #Create new LVM filesystems to back up to
        my $cmd =  "lvcreate --name ss$args[0] --size $size\K VolGroup00 >> /var/upgrade/upgrade.log";
        system($cmd) == 0 or die "Error: $? (probably insufficient free extents)\n";
        $cmd = "mkfs.ext3 /dev/VolGroup00/ss$args[0] >> /var/upgrade/upgrade.log";
        system($cmd);
        $cmd = "mount /dev/VolGroup00/ss$args[0] /mnt/snap";
        system($cmd) == 0 or die "Can't mount snapshot!\n";
        my $mount = `grep $args[0] /etc/fstab | awk '{print \$2}'`;
        chomp($mount);
        $cmd = "cd $mount; find . -xdev | cpio -pvdaum /mnt/snap >> /var/upgrade/upgrade.log";
        system($cmd) == 0 or die "Error in cpio: $? (probably not enough free space)\n";
        $cmd = "umount /mnt/snap";
        system($cmd);
}
 
#Save off fstab
system("mount /dev/VolGroup00/ssLogVol00 /mnt/snap") == 0 or die "Can't mount snapshot!\n";
system("cp /etc/fstab.rollback /mnt/snap/etc/fstab");
system("umount /mnt/snap");
 
#Remove packages before the upgrade from a known-good list
#Anything that differs, kill it
open(my $file, "/var/adm/sw/products/stillinstalled.fs");
my @good = <$file>;
chomp(@good);
close($file);
 
my @current = `rpm -qa --queryformat "%{NAME}.%{ARCH}\n" | grep -v none | grep -v svt`;
chomp(@current);
 
my %known = map { $_ => 1 } @good;
my %present = map {$_ => 1 } @current;
 
my @killrpms;
 
foreach my $rpm (keys %present) {
        unless ($known{$rpm}) {
                push(@killrpms, $rpm);
        }
}
 
my $rpmkillcmd = "rpm -e --nodeps --allmatches " . join(' ',@killrpms);
print "$rpmkillcmd\n";
rpmrun($rpmkillcmd);
 
#Extract the upgrade files
system("mkdir -p /tmp/upgrade");
system("tar -xzvf /var/adm/sw/products/rhel53upgrade.tgz -C /tmp/upgrade");
 
my @rpms = `rpm -qa --queryformat "\%\{NAME\}.\%\{ARCH\}\n"`;
chomp(@rpms);
 
my @specific = `rpm -qa --queryformat "%{NAME}-%{VERSION}.%{ARCH}\n"| egrep "elfutils|elfutils-libelf|pwlib|rpm-libs"`;
 
chomp(@specific);
my $specdel = join(' ', @specific);
 
my @nuke = `rpm -qa --queryformat "%{NAME}-%{VERSION}.%{ARCH}\n" | egrep "libselinux|util-linux|e2fsprogs"`;
chomp(@nuke);
my @cracklib = `rpm -qa --queryformat "%{NAME}-%{VERSION}.%{ARCH}\n" | egrep "cracklib.*i386"`;
chomp(@cracklib);
my @nukers = (@nuke, @cracklib);
 
my @findkernels = `rpm -qa |grep kernel`;
chomp(@findkernels);
 
my @deleters;
my @oldkernels;
my %del = map {$_ => 1} qw{VFlib2 caching-nameserver autofs kudzu pcmcia-cs xorg-x11 newt-perl hal crypto-utils gdm openjade docbook-dtds scrollkeeper pilot-link octave kdemultimedia kdebase xorg-x11-xdm system-config-printer system-config-printer-gui hal-cups-utils linuxwacom OPC OPCCOMM dce redhat-logos redhat-menus redhat-artwork system-config-securitylevel system-config-securitylevel-tui HPOvEaAgt HPOvLcja HPOvPCO HPOvBbc HPOvPacc HPOvLczC HPOvEaAes HPOvEaAja HPOvSecCo HPOvAgtEx HPOvLcko HPOvCtrl HPOvEaAzC HPOvSecCC HPOvXercesA HPOvLces HPOvDepl HPOvXpl HPOvPerlA HPOvXalanA HPOvEaAko HPOvConf};
 
my %postdel = map {$_ => 1} qw{lha bluez-bluefw desktop-backgrounds-extra gtk-engines ttfprint gsl lam-libs ash krbafs tclx fribidi Omni-foomatic Omni mysqlclient10 FreeWnn-libs w3c-libwww};
my @postdeleters;
 
foreach my $rpm (@rpms) {
        my ($shortname) = $rpm =~ m/(.*)\..*/;
        if ($del{$shortname}) {
                push(@deleters, $rpm);
        }
        if ($postdel{$shortname}) {
                push(@postdeleters, $rpm);
        }
}
 
foreach my $kernel (@findkernels) {
        push(@oldkernels, $kernel);
}
 
my $removelist = join(' ', @deleters);
my $removekernels = join(' ', @oldkernels);
my $nukelist = join(' ',@nukers);
 
my %version = map {$_ => 1} qw{redhat-release-notes-5Server.x86_64.rpm redhat-release-5Server.x86_64.rpm};
 
my %prepinstall = map {$_ => 1} qw(rpm-libs.x86_64.rpm rpm.x86_64.rpm yum.noarch.rpm rpm-python.x86_64.rpm popt.x86_64.rpm glibc.x86_64.rpm glibc-common.x86_64.rpm beecrypt.x86_64.rpm glibc-headers.x86_64.rpm glibc-devel.x86_64.rpm binutils.x86_64.rpm elfutils.x86_64.rpm elfutils-libelf.x86_64.rpm elfutils-libs.x86_64.rpm beecrypt-python.x86_64.rpm python.x86_64.rpm python-devel.x86_64.rpm python-elementtree.x86_64.rpm python-sqlite.x86_64.rpm sqlite.x86_64.rpm sqlite-devel.x86_64.rpm python-urlgrabber.noarch.rpm neon.x86_64.rpm libxml2.x86_64.rpm libxml2-python.x86_64.rpm db4.x86_64.rpm libselinux.x86_64.rpm libsepol.x86_64.rpm mcstrans.x86_64.rpm m2crypto.x86_64.rpm krb5-libs.x86_64.rpm openssl.x86_64.rpm readline.x86_64.rpm python-iniparse.noarch.rpm yum-metadata-parser.x86_64.rpm nss.x86_64.rpm glib2.x86_64.rpm e2fsprogs.x86_64.rpm e2fsprogs-libs.x86_64.rpm nspr.x86_64.rpm);
 
my @instlist;
my $packdir = "/tmp/upgrade/base/";
opendir(my $dir, $packdir);
my %cdromfiles = map {$_ => 1} readdir($dir);
closedir($dir);
my @versions;
my @installers;
my $kernel;
foreach my $entry (keys %cdromfiles) {
        my ($packname, $packarch) = $entry =~ m/(.*?)-\d[^S].*(\..*?\.rpm)/;
        my $package = $packname . $packarch;
        if ($version{$package}) {
                $entry = $packdir . $entry;
                push (@versions, $entry);
        }
        if ($prepinstall{$package}) {
                $entry = $packdir . $entry;
                push (@installers, $entry);
        }
        if ($entry =~ /kernel-\d.*/) {
                $kernel = $packdir . $entry;
        }
}
 
my $installlist = join(' ', @installers);
my $versionlist = join(' ', @versions);
 
print "rpm -Uvh $versionlist\n";
rpmrun("rpm -Uvh $versionlist");
print "rpm --import /tmp/upgrade/base/RPM-GPG-KEY-redhat-release\n";
rpmrun("rpm --import /tmp/upgrade/base/RPM-GPG-KEY-redhat-release");
print "rpm -Uvh --nodeps --replacefiles $installlist\n";
rpmrun("rpm -Uvh --nodeps --replacefiles $installlist");
print "rm -f /var/lib/rpm/__* && rpm --rebuilddb\n";
system("rm -f /var/lib/rpm/__* && rpm --rebuilddb");
if ($removelist != "") {
        print "rpm -e --allmatches --nodeps $removelist\n";
        rpmrun("rpm -e --allmatches --nodeps $removelist");
}
if ($specdel != "") {
        print "rpm -e --allmatches $specdel\n";
        rpmrun("rpm -e --allmatches $specdel");
}
open(my $yumconf, "> /etc/yum.repos.d/iso.repo");
print $yumconf "[rh53]\nbaseurl=file:///tmp/upgrade/base\nenabled=1\ngpgcheck=0";
close($yumconf);
system("yum clean all");
rpmrun("rpm -ivh --nodeps $kernel");
print "rpm -e --nodeps $removekernels\n";
rpmrun("rpm -e --nodeps $removekernels");
rpmrun("rpm -ivh --nodeps --replacefiles /var/adm/sw/products/openssl097a-0.9.7a-9.el5_2.1.i386.rpm");
if ($nukelist != "") {
        print "rpm -e --nodeps $nukelist\n";
        rpmrun("rpm -e --nodeps $nukelist");
}
rpmrun("rpm -e hpasm hprsm hponcfg; yum -y --exclude=expect upgrade && rpm -e --nodeps e2fsprogs.i386; yum -y install kernel-devel bind bind-chroot compat-openldap openmotif22 e2fsprogs-libs nss.i386 swig numactl enscript guile rpm-libs.i386 libtool-ltdl nspr wdaemon sqlite.i386 libgcj.i386 elfutils-libelf.i386");
my $postdeletelist = join(' ', @postdeleters);
if ($postdeletelist != "") {
        rpmrun("rpm -e --allmatches $postdeletelist");
}
rpmrun("mkdir -p /usr/bin/X11 && ln -s /usr/bin/mwm /usr/bin/X11/mwm && ln -s /usr/bin/xauth /usr/bin/X11/xauth && ln -s /usr/bin/xmodmap /usr/bin/X11/xmodmap && ln -s /usr/bin/xset /usr/bin/X11/xset && ln -s /usr/bin/xsetroot /usr/bin/X11/xsetroot && ln -s /usr/bin/xterm /usr/bin/X11/xterm");
system("cp -v /var/adm/sw/products/postinstall.pl /usr/bin/postinstall");
system("chmod 777 /tmp");
system("echo 'nohup /usr/bin/postinstall &' >> /etc/rc.local");
system("grub-install /dev/your/root");
finished("success");
 
 
 
sub finished {
        my ($status) = @_;
        system("cp -v /root/bootsave/*2.6* /boot");
        system('sed -i -e \'s/2.6.*ELsmp/2.6.18-128.el5/g\' /boot/grub/menu.lst');
        system('sed -i -e \'s/Red Hat.*up/RHEL5/g\' /boot/grub/menu.lst');
        system('sed -i -e \'s/2.6.*EL/2.6.18-128.el5/g\' /boot/grub/menu.lst');
        system("cat /root/bootsave/menu.lst >> /boot/grub/menu.lst");
        if ($status =~ /failed/) {
                my $msg = "Upgrade failed!  Please boot into the snapshot and run /usr/bin/backout.";
                notify($msg);
                die "Upgrade on $host failed!  Please boot into the snapshot and run /usr/bin/backout.\n";
        }
        else {
                my $msg = "Upgrade on $host successful.  Rebooting...";
                notify($msg);
                print "Upgrade successful.  Rebooting...\n";
                system("chmod +x /etc/rc.d/rc.local");
                system("reboot -t now");
        }
}
 
sub rpmrun {
        my ($cmd) = @_;
        print "$cmd\n";
        #We're probably past the point of no return here, so I'm not going to die(), but I'll still print
        system("$cmd >> /var/upgrade/upgrade.log 2>&1") == 0 or failed($_);
 
}
 
sub failed {
        my ($message) = @_;
        print "Failed on $message packages\n";
        finished("failed");
}
 
sub notify {
        my ($message) = @_;
        my @admins = ('youraddress@whatever.domain');
        foreach my $admin (@admins) {
                system("echo $message \| mailx -s '$host upgrade' $admin");
        }
}

List of packages:

acl.x86_64
acpid.x86_64
alchemist.i386
alchemist.x86_64
alsa-lib.i386
alsa-lib.x86_64
anacron.x86_64
aspell-en.x86_64
aspell.i386
aspell.x86_64
atk.i386
atk.x86_64
attr.x86_64
at.x86_64
audiofile.i386
audiofile.x86_64
audit-libs.i386
audit-libs.x86_64
audit.x86_64
authconfig.x86_64
basesystem.noarch
bash.x86_64
bc.x86_64
beecrypt.i386
beecrypt.x86_64
bind-libs.i386
bind-libs.x86_64
bind-utils.x86_64
binutils.x86_64
bitmap-fonts.noarch
bitstream-vera-fonts.noarch
bzip2-libs.i386
bzip2-libs.x86_64
bzip2.x86_64
chkconfig.x86_64
chkfontpath.x86_64
compat-dapl-1.2.5.x86_64
compat-db.i386
compat-libgcc-296.i386
compat-libstdc++-296.i386
compat-libstdc++-33.i386
comps-extras.noarch
coreutils.x86_64
cpio.x86_64
cpp.x86_64
cracklib-dicts.x86_64
cracklib.x86_64
crash.x86_64
crontabs.noarch
cups-libs.i386
cups-libs.x86_64
cups.x86_64
curl.i386
cyrus-sasl.i386
cyrus-sasl-md5.i386
cyrus-sasl-md5.x86_64
cyrus-sasl-plain.i386
cyrus-sasl-plain.x86_64
cyrus-sasl.x86_64
db4.i386
db4.x86_64
dbus-glib.i386
dbus-glib.x86_64
dbus.i386
dbus-python.x86_64
dbus.x86_64
desktop-backgrounds-basic.noarch
desktop-backgrounds-extra.noarch
desktop-file-utils.x86_64
device-mapper.i386
device-mapper.x86_64
diffutils.x86_64
dmraid.i386
docbook-dtds.noarch
dos2unix.x86_64
dosfstools.x86_64
e2fsprogs.x86_64
ed.x86_64
eject.x86_64
elfutils-libelf.x86_64
elfutils.x86_64
esound.i386
esound.x86_64
ethtool.x86_64
expat.i386
expat.x86_64
expect.i386
fbset.x86_64
filesystem.x86_64
file.x86_64
findutils.x86_64
finger.x86_64
fontconfig.i386
fontconfig.x86_64
fonts-xorg-base.noarch
foomatic.x86_64
freetype.i386
freetype.x86_64
ftp.x86_64
gail.i386
gail.x86_64
gamin.i386
gamin.x86_64
gawk.x86_64
gcc.x86_64
GConf2.i386
GConf2.x86_64
gdbm.i386
gdbm.x86_64
gd.i386
gd.x86_64
gettext.x86_64
ghostscript-fonts.noarch
ghostscript.i386
ghostscript.x86_64
glib2.i386
glib2.x86_64
glibc-common.x86_64
glibc-devel.x86_64
glibc-headers.x86_64
glibc.i686
glibc-kernheaders.x86_64
glibc.x86_64
glib.i386
glib.x86_64
gmp.i386
gmp.x86_64
gnome-keyring.i386
gnome-keyring.x86_64
gnome-mime-data.x86_64
gnome-python2-bonobo.x86_64
gnome-python2-canvas.x86_64
gnome-python2-gtkhtml2.x86_64
gnome-python2.x86_64
gnome-vfs2.i386
gnome-vfs2.x86_64
gnupg.x86_64
gnutls.i386
gnutls.x86_64
gpm.i386
gpm.x86_64
grep.x86_64
groff.x86_64
grub.x86_64
gtk2.i386
gtk2.x86_64
gtkhtml2.i386
gtkhtml2.x86_64
guile.i386
gzip.x86_64
hal.i386
hal.x86_64
hdparm.x86_64
hesiod.i386
hesiod.x86_64
hotplug.x86_64
htmlview.noarch
hwdata.noarch
info.x86_64
initscripts.x86_64
intltool.x86_64
iproute.x86_64
ipsec-tools.x86_64
iptables.x86_64
iptstate.x86_64
iputils.x86_64
jpackage-utils.noarch
jwhois.x86_64
kbd.x86_64
kernel-devel.x86_64
kernel-ib.x86_64
kernel-smp-devel.x86_64
kernel-smp.x86_64
kernel-utils.x86_64
kernel.x86_64
keyutils-libs.x86_64
keyutils.x86_64
krb5-auth-dialog.x86_64
krb5-libs.i386
krb5-libs.x86_64
krb5-workstation.x86_64
krbafs.i386
kudzu.x86_64
less.x86_64
lftp.x86_64
lha.x86_64
libacl.i386
libacl.x86_64
libart_lgpl.i386
libart_lgpl.x86_64
libattr.i386
libattr.x86_64
libbonobo.i386
libbonoboui.x86_64
libbonobo.x86_64
libcap.i386
libcap.x86_64
libcroco.i386
libcroco.x86_64
libf2c.i386
libgcc.i386
libgcc.x86_64
libgcrypt.i386
libgcrypt.x86_64
libgfortran.i386
libglade2.i386
libglade2.x86_64
libgnomecanvas.i386
libgnomecanvas.x86_64
libgnomecups.i386
libgnomecups.x86_64
libgnome.i386
libgnomeprint22.i386
libgnomeprint22.x86_64
libgnomeprintui22.i386
libgnomeprintui22.x86_64
libgnomeui.x86_64
libgnome.x86_64
libgpg-error.i386
libgpg-error.x86_64
libgsf.x86_64
libibverbs.i386
libibverbs.x86_64
libIDL.i386
libIDL.x86_64
libidn.i386
libidn.x86_64
libieee1284.i386
libjpeg.i386
libjpeg.x86_64
libmng.i386
libmng.x86_64
libobjc.i386
libogg.i386
libogg.x86_64
libpcap.i386
libpcap.x86_64
libpng10.i386
libpng.i386
libpng.x86_64
libraw1394.i386
libraw1394.x86_64
librdmacm.i386
librdmacm.x86_64
librsvg2.x86_64
libselinux.i386
libselinux.x86_64
libsepol.i386
libsepol.x86_64
libstdc++.i386
libstdc++.x86_64
libtermcap.i386
libtermcap.x86_64
libtiff.i386
libtiff.x86_64
libusb.i386
libusb.x86_64
libuser.i386
libuser.x86_64
libvorbis.i386
libvorbis.x86_64
libxml2.i386
libxml2-python.x86_64
libxml2.x86_64
libxslt.i386
libxslt.x86_64
logrotate.x86_64
logwatch.noarch
lshw.x86_64
lsof.x86_64
lvm2.x86_64
m4.x86_64
mailcap.noarch
mailx.x86_64
MAKEDEV.x86_64
make.x86_64
man-pages.noarch
man.x86_64
mcelog.x86_64
mdadm.x86_64
metacity.x86_64
mgetty.x86_64
mikmod.i386
mingetty.x86_64
mkinitrd.x86_64
mktemp.x86_64
module-init-tools.x86_64
mpage.x86_64
mtools.x86_64
mtr.x86_64
mt-st.x86_64
nano.x86_64
ncurses.i386
ncurses.x86_64
nc.x86_64
netconfig.x86_64
netdump.x86_64
net-snmp-libs.i386
net-snmp-libs.x86_64
net-tools.x86_64
newt.i386
newt.x86_64
nss_db.i386
nss_db.x86_64
ntp.x86_64
ntsysv.x86_64
numactl.x86_64
Omni-foomatic.x86_64
Omni.i386
Omni.x86_64
OpenIPMI-libs.x86_64
OpenIPMI.x86_64
openldap.i386
openldap.x86_64
openmotif21.i386
openmotif.i386
openssh-askpass-gnome.x86_64
openssh-askpass.x86_64
openssh-clients.x86_64
openssh-server.x86_64
openssh.x86_64
openssl096b.i386
openssl.i686
openssl.x86_64
ORBit2.i386
ORBit2.x86_64
pam_ccreds.i386
pam_ccreds.x86_64
pam.i386
pam_krb5.i386
pam_krb5.x86_64
pam_passwdqc.i386
pam_passwdqc.x86_64
pam_smb.i386
pam_smb.x86_64
pam.x86_64
pango.i386
pango.x86_64
parted.x86_64
passwd.x86_64
patch.x86_64
pax.x86_64
pciutils.x86_64
pcre.i386
pcre.x86_64
pdksh.x86_64
perl-DateManip.noarch
perl-DBI.x86_64
perl-Filter.x86_64
perl-HTML-Parser.x86_64
perl-HTML-Tagset.noarch
perl-libwww-perl.noarch
perl-libxml-enno.noarch
perl-libxml-perl.noarch
perl-Parse-Yapp.noarch
perl-URI.noarch
perl.x86_64
perl-XML-Dumper.noarch
perl-XML-Encoding.noarch
perl-XML-Parser.x86_64
pinfo.x86_64
pnm2ppa.x86_64
popt.i386
popt.x86_64
ppp.x86_64
procmail.x86_64
procps.x86_64
psacct.x86_64
psmisc.x86_64
psutils.x86_64
pygtk2-libglade.x86_64
pygtk2.x86_64
pyOpenSSL.x86_64
pyorbit.x86_64
python.x86_64
pyxf86config.x86_64
PyXML.x86_64
qt.i386
qt.x86_64
quota.x86_64
rdate.x86_64
rdist.x86_64
readline.i386
readline.x86_64
redhat-logos.noarch
redhat-lsb.i386
redhat-lsb.x86_64
redhat-menus.noarch
redhat-release.x86_64
rhgb.x86_64
rhnlib.noarch
rhpl.x86_64
rootfiles.noarch
rpm-libs.i386
rpm-libs.x86_64
rpm-python.x86_64
rpm.x86_64
rp-pppoe.x86_64
rsync.x86_64
scrollkeeper.i386
scrollkeeper.x86_64
SDL.i386
sed.x86_64
sendmail.x86_64
setarch.x86_64
setserial.x86_64
setup.noarch
setuptool.x86_64
sgml-common.noarch
shadow-utils.x86_64
shared-mime-info.x86_64
sharutils.x86_64
slang.i386
slang.x86_64
specspo.noarch
speex.i386
speex.x86_64
startup-notification.i386
startup-notification.x86_64
statserial.x86_64
stunnel.x86_64
sudo.x86_64
switchdesk.noarch
symlinks.x86_64
sysfsutils.x86_64
sysklogd.x86_64
sysstat.x86_64
system-config-date.noarch
system-config-display.noarch
system-config-keyboard.noarch
system-config-language.noarch
system-config-lvm.noarch
system-config-mouse.noarch
system-config-rootpassword.noarch
system-config-securitylevel-tui.x86_64
system-config-securitylevel.x86_64
system-config-services.noarch
system-config-users.noarch
system-logviewer.noarch
SysVinit.x86_64
talk.x86_64
tar.x86_64
tcl.i386
tcl.x86_64
tclx.i386
tcpdump.x86_64
tcp_wrappers.i386
tcp_wrappers.x86_64
tcsh.x86_64
telnet.x86_64
termcap.noarch
time.x86_64
tk.i386
tk.x86_64
tmpwatch.x86_64
traceroute.x86_64
ttmkfdir.x86_64
tzdata.noarch
udev.x86_64
umb-scheme.x86_64
unix2dos.x86_64
unzip.x86_64
urw-fonts.noarch
usbutils.x86_64
usermode-gtk.x86_64
usermode.x86_64
utempter.i386
utempter.x86_64
util-linux.x86_64
vasclnts.x86_64
vasclnts.x86_64
vas.x86_64
vconfig.x86_64
vim-common.x86_64
vim-enhanced.x86_64
vim-minimal.x86_64
vixie-cron.x86_64
vnc-server.x86_64
vte.i386
vte.x86_64
WFBwfssh-maint.x86_64
WFBwfssh.x86_64
wget.x86_64
which.x86_64
words.noarch
Xaw3d.i386
xinetd.x86_64
xinitrc.noarch
xml-common.noarch
xmlsec1.i386
xmlsec1-openssl.i386
xmlsec1-openssl.x86_64
xmlsec1.x86_64
xorg-x11-deprecated-libs.i386
xorg-x11-deprecated-libs.x86_64
xorg-x11-font-utils.x86_64
xorg-x11-libs.i386
xorg-x11-libs.x86_64
xorg-x11-Mesa-libGL.i386
xorg-x11-Mesa-libGLU.i386
xorg-x11-Mesa-libGLU.x86_64
xorg-x11-Mesa-libGL.x86_64
xorg-x11-tools.x86_64
xorg-x11-twm.x86_64
xorg-x11.x86_64
xorg-x11-xauth.x86_64
xorg-x11-xfs.x86_64
xsri.x86_64
xterm.x86_64
zip.x86_64
zlib.i386
zlib.x86_64

Backout script:

#!/usr/bin/perl
open(my $fstab, "/etc/fstab.original");
my @lines = <$fstab>;
close($fstab);
chomp(@lines);
 
#Check if we're on the live partitions
my $test = `mount`;
if ($test =~ /-LogVol\d+/) {
        die "Please ensure you are in the snapshot before running the backout!\n";
}
 
my %mapping;
for my $line (@lines) {
        if ($line =~ /Vol/) {
                my @fields = split(/\s+/, $line);
                $mapping{$fields[0]} = $fields[1];
        }
}
 
system("mkdir /mnt/backout");
foreach my $vol (keys %mapping) {
        print "$vol -> $mapping{$vol}\n";
        print "mount $vol /mnt/backout\n";
        system("mount $vol /mnt/backout");
 
        #Clean it out
        system("rm -rf /mnt/backout/*");
 
        #Step into whatever directory matches and cpio it to the appropriate filesystem
        #Copy special nodes and don't cross filesystems
        print "cd $mapping{$vol}; find . -xdev | cpio -pvdaum /mnt/backout\n";
        system("cd $mapping{$vol}; find . -xdev | cpio -pvdaum /mnt/backout") == 0 or die "Error in cpiO: $? (probably not enough free space)\n";
        print "umount /mnt/backout\n";
        system("umount /mnt/backout");
}
#restore fstab
system("mount /dev/VolGroup00/LogVol00 /mnt/backout");
system("cp -v /etc/fstab.original /mnt/backout/etc/fstab");
system("cp -v /etc/fstab.original /etc/fstab");
 
#restore /boot
my $cmd = "tar -xjvf /root/bootback.tbz -C /boot";
print "$cmd\n";
system($cmd);
system("reboot -t now");

Postinstall script (for anything you need done after the reboot, emails you to let you know it’s done):

#!/usr/bin/perl
 
use Sys::Hostname;
my $host = hostname;
 
 
system("rpm -e hpasm hprsm; yum -y install hponcfg hpacucli hp-health >> /var/log/upgrade.log 2>&1");
 
my @admins = ('youraddress@whatever.domain');
foreach my $admin (@admins) {
        system("cat /etc/motd | sed -e 's/~//g' | sed -e '/^\\s*\$/d' | mailx -s '$host upgraded' $admin");
}
 
#Take us out of rc.local
system("sed -i -e 's/.*postinstall//' /etc/rc.local");

Now here’s a power list…

Dark Elves Roster – 2250 pts

Highborn (1#, 264 Pts)
1 Dreadlord
General; Hand Weapon; Sea Dragon Cloak; Shield; Eternal Hatred
1 Cold One
Thick-skinned; Causes Fear; Stupid
1 Deathpiercer
1 Armor of Eternal Servitude
1 Ring of Hotek

Hag Queen (1#, 205 Pts)
1 Death Hag (Battle Standard Bearer)
Khainite; Hand Weapon; Extra Hand Weapon; Battle Standard
Bearer; Eternal Hatred; Frenzy; Poisoned Attacks
1 Witchbrew
1 Touch of Death
1 Banner of Hag Graef
Always Strike First

Hag Queen (4#, 200 Pts)
1 Death Hag
Khainite; Hand Weapon; Extra Hand Weapon; Eternal Hatred;
Frenzy; Poisoned Attacks
1 Cauldron of Blood
Altar of Khaine; Attendents; Blessings of Khaine; Khainite;
War Machine; Causes Terror; Eternal Hatred; Frenzy; Magic Resistance
(1); Poisoned Attacks
2 Hags
Khainite; Hand Weapon; Extra Hand Weapon; Eternal Hatred;
Frenzy; Poisoned Attacks

Repeater Crossbowmen (10#, 100 Pts)
10 Dark Elf Repeater Crossbowmen
Hand Weapon; Repeater Crossbow; Light Armour; Eternal Hatred

Repeater Crossbowmen (10#, 100 Pts)
10 Dark Elf Repeater Crossbowmen
Hand Weapon; Repeater Crossbow; Light Armour; Eternal Hatred

Dark Riders (5#, 85 Pts)
5 Dark Riders
Hand Weapon; Spear; Light Armour; Eternal Hatred; Fast Cavalry
5 Dark Steed

Dark Riders (5#, 85 Pts)
5 Dark Riders
Hand Weapon; Spear; Light Armour; Eternal Hatred; Fast Cavalry
5 Dark Steed

Executioners (23#, 455 Pts)
21 Har Ganeth Executioners
Khainite; Musician Mus; Standard Bearer Std; Draich; Heavy
Armour; Eternal Hatred; Killing Blow
1 Assassin
A Killer Not a Leader; Always Strike First; Hidden; Khainite;
Hand Weapon; Extra Hand Weapon; Eternal Hatred; Poisoned Attacks;
Scouts
1 Dark Venom
1 Touch of Death
1 Rune of Khaine
1 Draich-master
Khainite; Draich; Heavy Armour; Eternal Hatred

Executioners (10#, 120 Pts)
10 Har Ganeth Executioners
Khainite; Draich; Heavy Armour; Eternal Hatred; Killing Blow

Executioners (10#, 120 Pts)
10 Har Ganeth Executioners
Khainite; Draich; Heavy Armour; Eternal Hatred; Killing Blow

War Hydra (3#, 175 Pts)
1 War Hydra
Fiery Breath; Monsters and Handlers; Causes Terror; Eternal
Hatred; Large Target; Regenerate; Scaly Skin
2 Beastmaster
Hand Weapon; Eternal Hatred
2 Beastmasters Scourge

War Hydra (3#, 175 Pts)
1 War Hydra
Fiery Breath; Monsters and Handlers; Causes Terror; Eternal
Hatred; Large Target; Regenerate; Scaly Skin
2 Beastmaster
Hand Weapon; Eternal Hatred
2 Beastmasters Scourge

Cold One Knights (5#, 166 Pts)
5 Cold One Knights
Standard Bearer Std; Hand Weapon; Lance; Heavy Armour; Shield;
Eternal Hatred
1 Banner of Cold Blood
5 Cold One
Thick-skinned; Causes Fear; Stupid

It’s unlikely to be the sort of list I’d use (Witch elf BSB, no; big unit of Executioners probably replaced with Black Guard; turn Dreadlord into Supreme Sorceress; drop assassin for highborn), but holy gods. Other than core, the whole damn list it Khainite. so all the Executioners would be stubborn. Fun.

Here’s Matt Taibbi’s article on GS. and more fucked up shit on Reddit.

Dear internet, please shut up

Honestly, the fervor over Iran? Enough. While it’s certainly an important event, it merely serves to illustrate that the people of Iran have a nominal choice over their president. That is all. There’s some feeling of solidarity with Iran’s youth (about 30% of the populace) in the US, but people seem to have deluded themselves into thinking it’s another Iranian Revolution or a harbinger of Western-style democracy. First, I guess, a little background on Iran’s government structure:

• Supreme Leader:

Controls everything, really, as we’ll see in a bit. Personally has the power to declare war and peace, command army/intelligence/police forces.

• Assembly of Experts:

Clerics who determine who the next Supreme Leader will be, though they capitulate to his desires in every recorded instance(see: Montazeri). Elected to 8 year terms by direct public vote, but the list of candidates is screened anyway, so it’s not like a “free election.”

• Guardian Council:

Easily the most powerful force in Iranian politics. Twelve members, 6 picked directly by the Supreme Leader, 6 picked by the head of the judiciary (who is also picked by the Supreme Leader). Confirmed by Parliament, but it’s mostly for show. Functions as a combination Supreme Court and presidential veto (except Parliament cannot actually override their veto, it just gets sent back for revision). Screens candidates for every imaginable branch of government and arbitrarily excludes them (this has been happening for a while to keep reform candidates out — the 2004, 2006, and 2008 electiosn were particularly bad). May as well have total control of the Revolutionary Guard (which is not a branch of the normal army).

• Expendiency Council:

Nominally there to function as a screen between the Guardian Council and parliament. In reality, just advises the Supreme Leader

• Majlis (parliament):

Does the things you’d normally expect parliament to do (approve budgets, treaties, drafts legislation), only neutered by the Guardian Council. As with everything else, candidates are screened.

• President/VPs/Ministries

Executive branch stuff — day-to-day running of the state. Nominally right below the Supreme Leader. In truth, Guardian Council wields more power. Candidates are elected, but are screened by Guardian Council beforehand, ensuring real reformers never even make it to the vote. Best described as a figurehead.

There is very little debate in Iran about the importance of religion. They’re not going to turn into a secular nation. There’s hardly any protest (even now) against the Supreme Leader, though some resentment against the Guardian Council for the behavior of the Basij. No matter how the protests go, it’s unlikely to be another revolution — very few revolutions succeed without support from another sovereign nation, which they are not getting. The nature of the screening pretty much ensures that it’ll be another conservative candidate even if it does succeed, but Ahmadinejad and the Guardian Council control the complete apparatus of the state. It’s likely to be brutally repressed (granted, I’ve seen very little from outside Tehran, which I consider to be an indication that it’s relatively calm there).

Mousavi’s campaign? Move control of the police from Khamenei to the president. Make government a little more transparent. Allow non-state run television. Not exactly Kucinich here, or even Obama. Maybe Ron Paul is an apt comparison.

Karroubi? Actual reform! Freedom of the press, women’s rights, nationalize the oil industry and distribute the profits to the people, etc. As you’ve maybe seen, he was not the winner.

With that out of the way (and the background may have been necessary for those thinking about Iran becoming South Korea or post-imperial Japan), can we stop the damn Twittering? Changing your timezone to Tehran? Ahh… I’m not sure what you think that’s accomplishing. Under the assumption that the Iranian government can track individual people Twittering (and really? They’ve got more important things to worry about — just that people don’t post updates after going to rallies), it’s because they control the feeds out of the country. Your timezone isn’t fooling them, unless you’re routing it through Iran, which is doubtful. Open proxies? Let’s say they can track Twitterers — that means they’re using Deep Packet Inspection and, well, proxies don’t exactly help against that. They get around blackholing IP blocks (Facebook/whatever), and the headers change a little, but the packet contents are still eminently sniffable.

Tor exit nodes are a better option, but there’s the possibility the Iranian government is running an exit node themselves as a honeypot (extremely unlikely). VPNs are much better, and not all that hard to set up, as long as you keep the address of the server relatively private. For the sake of argument, we’ll say OpenVPN — which is SSL based — running on port 443. It should be utterly indistinguishable from HTTPS traffic, since the key negotiation is also encrypted, and DPI isn’t going to help them here. For all they know, people would be conducting internet banking or one of the million other things SSL is used for.

Of course, doing this requires more effort than changing your Twitter timezone or giving yourself a green background for “solidarity.” Worse than that, the technical issues should have been obvious to a lot of people by now (or at least the contradiction in assuming they can track what IP addresses are using Twitter to individuals, yet thinking that packets which are never routed through Iran will somehow fool them). The irony of the “solidarity” is that our elections in 2000 and 2004 were filled with questionable voter purges, yet we didn’t rise up. The candidate they elected (assuming the leaked results are true) is more comparable to Reagan or one of the Bushes than Johnson or Carter — a man who the people in support of “#iranelection” would never, ever vote for, but they don’t realize that as he’s a “reform” candidate. We arrested twice as many journalists in the 4 days of the RNC in Saint Paul than Iran has in a week of street riots.

Until you can drop the hypocrisy, technical incompetence (and don’t think most of the people wanted to think about that, since they might have to stop patting themselves on the back for “helping Iranians” with a fucking website that has the same limitations as SMS), and utter lack of understanding about Iran’s politics (ahem, even if Mousavi is inducted, not much is likely to change in Iran until the Guardian Council stops culling candidates who want actual change), you can shut the fuck up and keep your self-righteousness to yourself.

Resizing / on ext3 remotely

This is a feature request we had at work, and ignoring the RedHat consultant’s “If all you have is a hammer…” mentality (he suggested Anaconda again, no matter how unsuitable it is for updating 4,000 remote servers), it’s not actually that difficult of a task. The prime difficult here, of course, is that you cannot shrink a live filesystem, and good luck unmounting / while the system is running. Additionally, it seems like nobody else on the internet has had to do it (or documented the process and changes needed), other than one guy whose documentation is useless (he put up a frakking binary of resize2fs, took it down because it was “a binary without source” [which the GPL abhors], and said he “didn’t have the sources anymore”. You’ll see further down exactly how large the necessary change to publicly available sources is [one line, and not a complicated one]).

In case you’ve never look at it before, the initramfs (and initrd before it, though RHEL still calls the initramfs an initrd, despite the fact that you cannot stuff an ext2 filesystem in there anymore to my knowledge) is a gzipped cpio archive, which makes mucking with it dead simple. Just make yourself a directory somewhere (I used ~/initrd), and issue:

gzip -d --suffix=".img" | cpio -id < /path/to/initrd

You'll find a directory structure resembling:

drwxr-xr-x 2 root root 4096 May  8 13:03 bin
drwxr-xr-x 2 root root 4096 Mar  6 08:54 dev
drwxr-xr-x 4 root root 4096 Mar  6 08:54 etc
-rwxr-xr-x 1 root root 2007 May 11 04:58 init
drwxr-xr-x 2 root root 4096 May  8 09:52 lib
lrwxrwxrwx 1 root root    3 May  8 12:09 lib64 -> lib
drwxr-xr-x 2 root root 4096 Mar  6 08:54 loopfs
drwxr-xr-x 2 root root 4096 Mar  6 08:54 proc
lrwxrwxrwx 1 root root    3 Mar  6 08:54 sbin -> bin
drwxr-xr-x 2 root root 4096 Mar  6 08:54 sys
drwxr-xr-x 2 root root 4096 Mar  6 08:54 sysroot

The bulk of the work is done in ./init, which calls out to nash (an ridiculously limited shell with almost zero constructs for doing useful stuff other than booting). It mounts /proc and /sys, makes device nodes for the consoles, null, zero, and shared memory, loads drivers (LVM/SCSI), then finds the root filesystem, mounts it, and pivots the root to continue normal bootup. The only real caveat here is that every binary in there is statically linked -- and that's not necessarily a bad thing, since portability is a lot easier.

To resize, we'll need resize2fs built statically, and e2fsck wouldn't be a bad idea either. There are make targets for both of them in e2fsprogs (either an SRPM or just sources, your pick). All the routines in e2fsprogs check to see whether or not the filesystem is mounted before they do anything by checking /etc/mtab or /proc/mounts (and nash's mount doesn't put an entry in /proc/mounts for reasons that I haven't dug into). That file does not exist during the init process, and while e2fsck will spit out a warning and continue to run, resize2fs isn't so lucky. From resize/main.c:

/*
* Figure out whether or not the device is mounted, and if it is
* where it is mounted.
*/
len=80;
while (1) {
  mtpt = malloc(len);
  if (!mtpt)
     return ENOMEM;
  mtpt[len-1] = 0;
  retval = ext2fs_check_mount_point(device_name, &mount_flags,
                                                  mtpt, len);
  if (retval) {
    com_err("ext2fs_check_mount_point", retval,
                _("while determining whether %s is mounted."),
                 device_name);
                exit(1);
  }
  if (!(mount_flags & EXT2_MF_MOUNTED) || (mtpt[len-1] == 0))
     break;
  free(mtpt);
  len = 2 * len;
}

That comes from lib/ext2fs/ismounted.c, which is 50% comments from Theodore T'so bitching about the HURD, and I don't feel like editing it (or reposting it verbatim, since the sources are easy enough to get). The gist is that it's checking for a non-zero return value on retval and bailing if it find one (which it will). The decision of modifying ismounted.c or just making a quick hack to resize/main.c is up to you (I opted for the quick hack option, which could be commenting out the if statement, deleting the block, or just setting retval to zero, which involves the least typing).

As an aside, I know braces aren't necessary in lots of languages if there's only one statement in the block (C being one of them), but it makes it seriously ugly without them. Let's hope he never has to add another statement and forgets to add braces.

With that modified (I opted not to modify e2fsck, since it runs anyway), we have handy-dandy make targets for the utilities we need.

make -C resize resize2fs.static && make -C e2fsck e2fsck.static

It should be obvious, but, ah... Don't do a `make install` on this. resize2fs is dangerous now.

Move the new binaries (located in $src/{resize,e2fsck}/*.static) to $initrd/bin, and `$EDITOR $initrd/init`.

At least on CentOS/RHEL, we're running an LVM root. If you're not, the correct place may be slightly different, but not too much. Find `lvm vgchange -ay...`, and tack the following lines after it:

echo Creating root device node...
#RHEL/CentOS ignore the 'defaults' parameter, but other systems may not
mkrootdev -t ext3 -o defaults,ro /dev/your/device/here
echo Resizing filesystem...
e2fsck -fy /dev/your/device/here
/bin/resize2fs -p /dev/your/device/here $newsize
#If you're running lvm, the initrd will contain a static lvm binary, too
#but not of the aliases to `lvmresize` and such, which is basically `lvm $argv[1] $@`
#Caveat: lvm will not resize volumes with snapshots yet, but if you don't have any
#you can resize now if you like.  You don't have to, since lvm will shrink online volumes
#once you're into the actual system
lvm lvresize -L /dev/your/device/here

I would probably recommend putting this into a different initrd than the one you usually use, setting it as the default in grub, and having a one-off script change grub back and delete itself (rc.local would be a good place for this, since knocking itself out of there would be trivial).

find . -print | cpio -o -c | gzip -c9 >/boot/$newinitrd

The new initrd will be ~1.5MB larger than the old one (less if you stripped the binaries), but that's a trivial size different compared to a rescue image or (gods forbid) Anaconda simply to resize. Remember: TMTOWTDI, and what OS vendors/consultants recommend is not always ideal for your situation (it was not for ours).

XMonad and xmobar on OpenSolaris with functional monitoring (mutt to boot)

I’ve been having carpal tunnel flareups lately, so I went looking around for things I could do to use the mouse less and the keyboard more (as counter-intuitive as that may sound, I find that holding a mouse for hours irritates it far more than any amount of typing, YMMV). Vimperator is an obvious first step, but, well, I love vi, so I already had that running.

Tiling window managers came to mind. I’ve already used wmii and dwm once upon a time, but they’re hardly state of the art (as state of the art as tiling window managers get, anyway), and hacking together a reasonable workspace status bar in dzen/Perl didn’t appeal to me. Awesome3 (the window manager) does appeal to me, but getting it running on Solaris looked like a little more work than I wanted to invest, and I’m sick of working with moving targets (Awesome3 looks like they’re almost as “break your config” happy as Rails).

Mind you, I still love Openbox, but given that 99% of my time is spent in terminals (irssi, mutt, ssh, vim, mcabber, slash’em), I see no reason why I should even bother with having window decorations and manually arranging them at all.

Had I known what I was getting into, I probably would have just used Awesome. I mean, it needed two libraries I didn’t have, and some dzen hacking. Not… this. Not that I’m unhappy with XMonad, but…

Firstly, there’s no build of GHC (Glaskow Haskell Compiler) in the OpenSolaris repositories. There’s a pre-compiled version of GHC 6.10, but only for SPARC. Pre-compiled version of GHC 6.8 for x86/amd64, but that ain’t helping me (a scary amount of stuff from Hackage, Haskell’s version of CPAN/rubygems/whathaveyou doesn’t want to run in GHC 6.8, and the recommended fix for some bugs is “upgrade to 6.10″).

No GCC 4 in the repos either, and GCC 4.1.2 is the recommended version for building GHC. So, onto the magic. Don’t even try with SunStudio. GNU-isms in the code stopped me dead.

I was bitching to Dan about how ridiculous this process was a few weeks ago. Maybe it’ll help somebody.

Firstly, either install readline from the OpenSolaris Pending repositories or compile it yourself.

Next, we need to bootstrap gcc. For that, we’ll need gmp from GNU and mpfr. Grab the precompiled version of GHC 6.8.2 while you’re at it. You should also get the newest versions of ghc and ghc-$version and ghc-$version-src-extralibs to get running later.

Complaint #1: GNU automake is braindead. It, I assume, just checks `uname` and not `isainfo`, so I can’t tell when we’re running 64 bit. Either use Solaris libtools or do the following for gmp and MPFR

./configure ABI=32 && gmake && pfexec gmake install
make distclean
configure --prefix=/usr/local/lib64 && gmake && pfexec gmake install

If you don’t specify another prefix, it’ll stomp all over the 32 bit libraries it just installed on your 64 bit box.

Complain #2: gcc is even more braindead. It’ll build a 64 bit binary but link it against 32 bit libraries, then eat itself during stage 2 bootstrap. You’d think the FSF would be smarter, but no. It just finds the wrong ELFCLASS down the line. To correct:

export LDFLAGS=-L/opt/local/lib -L/opt/local/lib/64 \
-R/usr/local/lib:/usr/local/lib/64
export LD_OPTIONS=-L/opt/local/lib -L/opt/local/lib/64 \
-R/usr/local/lib:/usr/local/lib/64
./configure && gmake -j4 && pfexec gmake install

Complaint #3: The precompiled ghc-6.8.2 we got? It sucks. The rts library is broken (check it with ldd). It would be nice to avoid this, but, well… Bootstrapping ghc from C sources and no Haskell compiler involves another goddamn system which DOES have Haskell installed, AND knowledge of what registers your CPU uses. Whoever thought up that notion of bootstrapping? Well… The “goal” is to have Haskell self-bootstrap (it currently does not, since, ironically, Haskell is too “pure” to actually be written in a “pure” language, since we need dirty things like actually doing something useful with “impure” data, like user input or stuff sucked in from a file).

cd ../ghc-6.8.2
./configure && pfexec gmake install
ghc-pkg describe rts > rts.pkg
vim rts.pkg
#add -R/usr/local/lib to the end of the ldoptions field, or ghc bombs bootstrapping the new version in stage 2
ghc-pkg update rts.pkg

Complaint #4: GHC is even stupider than GCC, if possible. Not only do we have to prepend /usr/local/bin to $PATH so GHC can find our shiny new gcc-4.1.2, we have to pass ridiculous amounts of config flags (including one which tells it where GCC is — if the $PATH OR –with-gcc is wrong, it won’t bootstrap. Don’t ask, because I don’t know why).

export PATH=/usr/local/bin:$PATH
./configure --with-gcc=/usr/local/bin/gcc --with-gmp-libraries=/usr/local/lib --with-gmp-include=/usr/local/include --with-readline-libraries=/usr/local/bin --with-readline-include=/usr/local/include
gmake -j4 && pfexec gmake install

Yay! Working GHC. Sadly, if you want to reclaim the 350MB or so the GHC 6.8 install is taking up, you have to go remove it yourself (apparently the GHC team does not believe in `make uninstall`). This means we can install cabal, which requires nothing special, other than you grabbing the tarball and installing it as normal.

Next, `cabal install xmonad xmonad-contrib` I said we were going to install xmobar, and we are, but it’s a little tricker. You see, even though Xmobar mostly reads a pipe to give us a title and workspace listing, the plugins are not optional. They also depend on libnotify, which is only present on Linux. Good job, xmobar developer! Fortunately, this is easily corrected, and xmobar (mostly) works. Caveats explained later.

You can’t just `cabal install` xmobar. No-go since hinotify will not install, and there’s not a clear explanation as to why from the output. As noted, it doesn’t really depend on it, just that the developer can’t be bothered to use Haskell’s typing system to throw messages at you when you try to use features that are not implemented. So… edit ~/.cabal/packages/hackage.haskell.org/xmobar/$version/xmobar-$version/xmonad.cabal

Take out the lines referring to hinotify. Then `cabal build && cabal install` from the directory xmonad.cabal was in. Ooh and aah, but don’t try to use, well… anything. BatteryReader, CpuReader, MemReader, Net, Swap, all broken. Thankfully, we have Dtrace and Python to replace it with, since xmobar’s PipeReader still works.

Memory usage?

#pragma D option quiet
#pragma D option bufsize=16k
 
dtrace:::BEGIN
{
}
 
profile:::tick-1sec
{
	/* RAM stats */
	this->ram_total = `physinstalled;
	this->unusable  = `physinstalled - `physmem;
	this->locked    = `pages_locked;
	this->ram_used  = `availrmem - `freemem;
	this->freemem   = `freemem;
	this->kernel    = `physmem - `pages_locked - `availrmem;
 
	this->ram_total	*= `_pagesize;  this->ram_total	/= 1048576;
	this->unusable	*= `_pagesize;  this->unusable	/= 1048576;
	this->kernel	*= `_pagesize;  this->kernel	/= 1048576;
	this->locked	*= `_pagesize;  this->locked	/= 1048576;
	this->ram_used	*= `_pagesize;  this->ram_used	/= 1048576;
	this->freemem	*= `_pagesize;  this->freemem	/= 1048576;
	printf("RAM: %2d%%\n", ((this->ram_total - this->freemem) * 100 / this->ram_total));
}

Network speeds?

#!/usr/sbin/dtrace -s
#pragma D option quiet
dtrace:::BEGIN
{
	TCP_out = 0; TCP_in = 0;
}
 
 
mib:::tcpOutDataBytes		{ TCP_out += arg0;   }
mib:::tcpInDataInorderBytes	{ TCP_in += arg0;    }
 
profile:::tick-1sec
{
	OUT_print = TCP_out/1024; IN_print = TCP_in/1024;
	printf("Out:%3d|In:%3d", OUT_print, IN_print);
	TCP_out = 0;
	TCP_in = 0;
 
}

.xmobarrc

Config { font = "-*-terminus-*-*-*-*-12-*-*-*-*-*-*-u"
       , bgColor = "#000000"
       , fgColor = "#AFAF87"
       , position = Top 
       , lowerOnStart = True
       , commands = [ Run Date "%a %b %_d %Y %H:%M:%S" "date" 10 
                    , Run Weather "KSTP" ["-t","<tempF>F","-L","64","-H","77","--normal","green","--high","red","--low","lightblue"] 36000
		    , Run PipeReader "/export/home/ryan/dtrace/net" "wireless"
		    , Run PipeReader "/export/home/ryan/dtrace/netspeed" "speed"
		    , Run PipeReader "/export/home/ryan/dtrace/psr" "cpui"
		    , Run PipeReader "/export/home/ryan/dtrace/ram" "mem"
                    , Run StdinReader
                    ]
       , sepChar = "%"
       , alignSep = "}{"
       , template = " %StdinReader% } { %cpui% | %mem% | %wireless% %speed% | %date% | %KSTP%"
       }

A script to feed those pipes. If you don’t have python2.6, pexpect on python2.4 (the Solaris/OpenSolaris default) works. Just install pexpect with easy_install, an .egg, or whatever your poison may be.

#!/usr/bin/python2.6
import math
import os
import platform
import re
import stat
import sys
import time
 
import pexpect
 
#Get the directory we're running from to create the fifos rather than the $pwd of whatever called us
path = os.path.dirname( os.path.realpath(__file__)) + "/"
 
wificonfig = ""
 
#Y'know, I haven't actually written the iwconfig thing.  It's here for posterity and possible later use.
osystem = platform.system()
if osystem == 'SunOS':
  wificonfig = 'wificonfig'
elif osystem == 'Linux':
  wificonfig = 'iwconfig'
 
 
#Kill off any old instances which may be running.  Poor man's pkill, but guaranteed to work pretty much anywhere.
pexpect.run('bash -c "ps -ef |grep mpstat |grep -v python| awk \'{print $2}\' | xargs kill -9"')
pexpect.run('bash -c "ps -ef | grep speed.d | awk \'{print $2}\' | xargs kill -9')
pexpect.run('bash -c "ps -ef | grep meminfo.d | awk \'{print $2}\' | xargs kill -9')
 
def checkfifo(path):
  #If it ain't there, make it
  if not os.path.exists(path):
    os.mkfifo(path)
    handle = open(path, "r+")
    return handle
  #If it is, just return it
  elif stat.S_ISFIFO(os.stat(path).st_mode):
    handle = open(path, "r+")
    return handle
  else:
    if os.path.isfile(path):
      #Not a FIFO, and it needs to be
      os.unlink(path)
    os.mkfifo(path)
    handle = open(path, "r+")
    return handle
 
#Set up our fifos
psrfifo = checkfifo(path + "psr")
netfifo = checkfifo(path + "net")
nspdfifo = checkfifo(path + "netspeed")
ramfifo = checkfifo(path + "ram")
 
#Fire off the processes we'll be reading from.  Using pexpect seems like overkill, but mpstat is apparently smart enough to tell when it's being read from a pipe, and it'll buffer no matter what you do.  pexpect/expect fake being interactive, so it happily runs without buffering.
mpstat = pexpect.spawn('bash -c "mpstat 1 | grep -v CPU"')
ramstats = pexpect.spawn(path + "meminfo.d")
nspeed = pexpect.spawn(path + "speed.d")
 
#Regular expressions to use later.  Since it's a long-runnign script, they may as well be compiled
mpre = re.compile(r'^\s+?(?P<cpu>\d+).*?(?P<idle>\d+)$')
solwifire = re.compile(r'.*?linkstatus: (?P<status>\w+).*essid: (?P<essid>\w+).*strength: \w+\((?P<strength>\d+)\).*', re.DOTALL)
 
def prstat():
  #Yay for awk/sed abuse, but it's concise and I'm already forking.  Basically getting a list of CPUs to check later, so this script should perform its duty no matter if you have 1 CPU or 128 (T2 users)
  psrinfo = pexpect.run('bash -c "psrinfo -v |grep MHz | awk \'{print $6,$7}\' | sed -e \'s/,//\'"').rstrip().split('\r\n')
  output = ""
  for cpu in psrinfo:
    line = prmatch(cpu)
    output = output + line
  psrfifo.write(output + "\n")
  psrfifo.flush()
 
def prmatch(cpu):
  line = mpstat.readline().rstrip()
  m = mpre.match(line)
  #Match it against our earlier regex and subtract the idle value from 100 to get the actual used percentage, which isn't wholly accurate (IOWAIT and whatnot), but it's good enough for me
  usage = 100 - int(m.group('idle'))
  #Padding the string seems stupid, and it is, but xmobar arbitrarily decides spots that it's not going to refresh even if text shows up there, leaving it (black in my case) when text slides.  Padding fixes that.  Also, if your CPU goes to 100%, you probably shouldn't have a script which reads dtrace probes running.  Just sayin'.
  return "Cpu%s: %2d%% (%s) " % (m.group('cpu'), usage, cpu)
 
def memory():
  #I haven't found any swap information from dtrace probes as easy to manipulate as thi sis
  swap = pexpect.run('bash -c "/usr/sbin/swap -l |tail -n 1 | awk \'{print $4, $5}\'"').rstrip().split(' ')
  #Ugly?  You bet.  Cast the subtract free swap from total swap blocks, divide it by free swap blocks * 100 cast to an int to give us an actual percentage, floor that, then cast THAT to an int
  usedswap = int(math.floor(((int(swap[1])-int(swap[0]))/int(swap[1])*100)))
  ram = ramstats.readline().rstrip()
  output =  "%s Swap: %2d%%" % (ram, usedswap)
  ramfifo.write(output + "\n")
  ramfifo.flush()
 
def network():
  #Filter out interface which aren't up, which are vnics, which only point to localhost to get the running interface.  I'm assuming you only have one at a time, but if you have more modify this to suit.
  iface = pexpect.run('bash -c "ifconfig -a | grep UP |grep RUNNING| grep -v IPv6 |grep -v lo | grep -v -E \':[0-9]: \' | awk \'{print $1}\' | sed -e \'s/://\'i"').rstrip()
  if osystem == 'SunOS':
    command = "wificonfig -i " + iface + " showstatus"
    status = pexpect.run(command).rstrip()
    output = ""
    if solwifire.match(status):
      #Beauty of regexes.  If it doesn't match, it's not wireless (or not connected).  It if is, give us values.
      m = solwifire.match(status)
      strength = math.floor((int(m.group('strength')) / 15.) * 100)
      output =  "%s: %s(%s) %3d%%" % (iface, m.group('status'), m.group('essid'), strength)
    else:
      #Probably not wireless
      output = iface + ":"
    netfifo.write(output + "\n") 
    netfifo.flush()
 
def netspeed():
  #Is this method really necessary?  Couldn't the dtrace probe just write to the fifo itself?  Probably, but if you (or I) want to colorize it at some point, it may as well get sucked in here.
  speed = nspeed.readline()
  nspdfifo.write(speed + "\n")
  nspdfifo.flush()
 
while 1:
 
  prstat()
  memory()
  network()
  netspeed()
  time.sleep(1)

It’s not the prettiest python. I should probably move those repetitive fifo flushes/etc to a method, but I didn’t honestly expect that I’d need to replace this much XMobar functionality. Notably, XMobar can colorize things with <span> attributes setting colors in case somebody wanted to pretty up the usages (really, to make it look more like XMobar’s [colorized] defaults for CPU/net usage). I don’t care, personally. I didn’t implement a battery monitor either, but hey, you can if you want to.

Complaint #5 (did I lose track?): SUNWmutt doesn’t have support for header caching, which is a real bitch when I have 12,000 emails. It also doesn’t support SMTP over SSL, making it pretty well worthless for Gmail. I have other accounts I use mutt for, but Gmail’s an important one.

This isn’t that tough, really. You need some kind of a database for the mutt config script, and gdbm is trivially easy to get running (normal ./configure && make && make install). On the other hand, we run into two hiccups. The configure file depends on ncurses, which is just a link to plain ol’ curses on lots of Solaris boxes. Secondly, (and I don’t really begrudge the Mutt guys for this, since there’s actually a commit to fix this, unlike GCC and GHC, whose response is “too fucking bad” [GHC actually posts the recommendation for fixing rts.pkg and the configure flags on their own site rather than FIXING THE BUILD]), configure.ac does some things wrong with libidn.

Find $with_idn, and replace the block which follows it with this (–with-idn doesn’t seem to build properly).:

if test "$with_idn" != "no" ; then
  if test "$with_idn" != "yes" ; then
    AC_CHECK_HEADERS([idn/idn-int.h],
      [AC_CHECK_HEADERS([idn/idna.h], [],
        [CPPFLAGS="$CPPFLAGS -I/usr/include/idn"])])
  fi
fi

If you don’t have or want ncurses (or it’s a symlink on your system), fix configure. `sed -i -e ’s/-lncurses/-lcurses/’ configure`.

./configure –with-regex –with-gnutls –enable-hcache –enable-smtp –enable-imap –enable-pop –enable-mailtool –with-sasl –with-idn=/usr/include/idn

Congratulations!

Next up, re-implementing htop for Solaris with dtrace probes, python, and ncurses.

Bored

So bored it’s unreal, actually. Other than a strong desire find a Haskell developer and treat him like a punching bag (but that’s another rant, probably this weekend, now that I’m finally done getting a working Xmonad/xmobar/Mutt/zsh combination on OpenSolaris with proper statistics), I haven’t had anything to do. I haven’t logged into WoW in a month or so, though I may do that this weekend also. I’m sort of stuck in a rut here.

Half-Price Books had their warehouse sale last weekend, so I picked up some new books, but those aren’t going to keep me occupied for very long. Heather’s website has to be rewritten at some point, but I’m still debating technologies to use for that (I like Rails, I really do, but the core team doesn’t give a shit if they break features during minor version releases, so almost all the documentation you find is out of date without going back $n versions and applying all the errata). Django is seeming appealing. I do not, however, have any idea how well that’s supported on Dreamhost. Sure, there are entries for it on their wiki, but Rails wasn’t exactly stable when they first added support for that. For a personal site, m’kay. For a site for her business? I’d rather have something reliable. Not that Django isn’t reliable, just that I don’t know how reliable it is on Dreamhost. Maybe I’ll set up a Django site on my VPS and just point her DNS servers (which are not hosted by Dreamhost) there.

I am, in truth, spending an inordinate amount of time questing for the Amulet of Yendor. I forgot how much I loved NetHack, and Slash’em just makes it better. On the downside, valkyries don’t appear to start with a longsword in Slash, so I’m not going to get any free Excaliburs at level 5. Monks aren’t bad at all, but food is kind of a pain in the ass. I’d also guess that blessed scrolls of genocide drop less frequently, leading to me getting killed by liches a few times. Awesome.

I’m sort of considering implementing Vassal in PyGame. Vassal 40k is a really neat system. Dice rolling isn’t handled all that well, it doesn’t import AB files, and the Vassal engine doesn’t do reforms/wheels well at all. I’m not going to hack at the Java to fix Vassal (thought the developers have started work on Vassal Fantasy Battles), and I sort of want to learn PyGame anyway, which seems like a good motivation to do it. Dan and I never did finish the C# implementation (in fact, I never really did anything other than the XML parser, and I have no idea how far he got). I don’t imagine it taking me all that long to do it in Python though, with PyGame doing the heavy lifting of the GDI crap C# needs for me. Thoughs (take a look at Vassal 40k, for instance)? I don’t picture it being difficult to render sprites from images of models I find on Google Image Search, but I guess you never know.

I do have lots of time to read news, though, and I’ve come to the following conclusions:

• Slashdot: My first, and probably still my favorite. The Firehose is worthless, but the community moderation system is incredibly effective. It is the only place I visit where I can regularly expect to see somebody with a post-graduate degree in whatever subject it is posting. People like NYCountryLawyer (who is actually a lawyer) make the site. There are, of course, some libertarian “omg government never solves any problems” trolls (who don’t think they are trolls), but that goes across the spectrum (generally people who say this in response to anything have never looked at numbers comparing government spending to private spending in the same field — Medicare being a great example — and mistakenly believe that corporations wouldn’t rape everybody). Increasingly dominated by people who have never worked worked in the real world (with user ids near 1.5 million) and an astounding amount of GPL trolls (who think it’s the greatest license since sliced bread and slander any company which does not abide by it — including Sun’s CDDL, which was devised because they don’t own the IP on some of their drivers — while simultaneously whining that they can’t steal code [ZFS, dtrace, etc] from alternatively-licensed projects). These, though, are generally modded into oblivion also.
• (Total)Fark: Biting headlines, little regard for the average human being (which I sort of appreciate, actually, since most of the people in their stories deserved what they got). Effective moderation, but the comment threading system sucks (not that I really need to read the comments with TotalFark having ~1 story per minute, most of which are worth a read). JonSnow is awesome.
• Reddit: Reddit seems to be a mixed bag. I love /r/programming, and some of the other SubReddits are great. At the same time, the average Redditor believes that they are much more intelligent than they actually are. I often see people talking about how they were “too smart” to be in school which, while possibly true, isn’t really relevant given that school is primarily a social tool. People spew nonsense about how the public education system was designed to make for complacent workers who will follow commands and obey the bell (this is true, but that was 130 years ago, and I’d hardly say school is psychological prep for working in a meat processing plant these days). There’s a lot of talk (on Reddit and Slashdot) about home schooling by parents who believe their children are beautiful, brilliant, unique snowflakes. Their child is doing $randomthing (3d rendering, programming, whatever) or is in “7th grade” at age 9, so it must be better, though they have no unbiased metrics to use for comparison.

  This particular aspect of Reddit comes out in shit like this (which is, by the way, the 2nd time that subject has been on AskReddit in the last few months). I’m all for being inclusive, but seriously? Not everybody deserves it. Reddit’s comment threading system is good, even if I don’t like it as much as Slashdot’s (in particular, if a parent post is downvoted too many times in Reddit, it hides the entire tree, even if one of the replies has 1,000,000 upvotes), but I don’t know where the fuck the moderation is sometimes. To give this guy a platform where he can answer selected questions to rationalize his behavior and make it seem as if he doesn’t need help is, well… Ephebophilia isn’t that uncommon, but this guy states he’s attracted to 2 year olds, FFS, and that he “admires them” at playgrounds and whatnot (he says he’s never touched them or seen images/videos, but that’s extremely dubious, and it’s virtually assured that he will act upon his urges at some point). I’m not easily sickened but, well, read the comments. Virtually nobody calls him out on it. They just stroke his ego.

Somebody give me a suggestion. If not, I’m just going start writing a frontend for a Vassal clone (while that may sound stupid, I suspect that the biggest obstacle WILL end up in unit reformation/wheeling/etc rather than back-end logic, so I’ll write it first. Integration tests can cover everything else when I write it).

Control structures, in my functional language?

In other words, Scala — first impressions (though a lot of this also applies to F#). Don’t get me wrong, the idea of a functional language on the JVM appeals to me (and not Clojure, due to Lisp hate). It’s a major problem for me, though, that the JVM doesn’t support tail packing at all (the CLR might with F# becoming a first-class language, but if so, I’ve heard nothing about it), which makes recursion the way it’s normally used sort of a problem. While somewhat offset by the fact that Scala can utilize pretty much any Java library out there. Groovy and JRuby can, too, and they feel more natural. JRuby, at least, doesn’t clutter the code with Java-isms (for lack of a better term). I don’t want to type MyClass extends SomeOtherClass in a functional language.

I want first-class support for map instead of it being a metaphor for for. Function composition is neat and all, but it’s there in every functional language (and other than map and foldl/r, I haven’t found a lot of use for it). I don’t want to import scala.collection.mutable.Hashmap. I don’t want mutable variables at all. Lists and tuples are enough. I want pattern matching to function the way it does in Haskell, OCaml, SML, F#, Erlang, and probably more rather than using a gussied up case statement. I’d rather not see for, while, foreach, et al in there. If I wanted them, I’d write them myself:

-module(for)
-export([for/3])
 
for(Maxval, Maxval, Function) -> [Function(Maxval)]
for(I, Maxval, Function) -> [Function(I) | for(I+1, Maxval, Function)]

Tough.

Those, though, are really just complaints about the way Scala does functional programming. What I like? Actors. However, actors are a first-class feature in Erlang. In Scala, they ride on top of locking/thread semantics in the JVM. In Scala, objects which don’t implement .receive() silently discard messages. In Scala, I can’t swap out modules without bringing down the service, and it’s nontrivial (unlike Erlang) to pass messages between identical codebases, say, Solaris, AIX, Linux, and OSX. Sure, Erlang isn’t as fast as the JVM, but it scales out a hell of a lot better without assloads of XML and clusters.

I also really like the fact that unlike some languages (-cough- Erlang -cough-), string handling doesn’t, well, suck. I like the fact that I can use JDBC bindings and Swing. I like that it can be compiled to a .war and deployed. Frankly though, in that case, I’d rather just use JRuby.

I don’t like the ludicrous method chaining it inherits from .Java. something.length.toString.length to align text? Really? Same goes for F#, though. Thanks, Scala team, and I look forward to revisiting your language sometime in the future. Until then, I’m sticking with Erlang/OCaml/F#, depending on my needs.

Oi, work

It’s such a joy to come in this morning to find out that somebody has helpfully shut down the virtual machine I was testing a RHEL4.7 -> 5.3 migration in. Not a supported move by RedHat (as noted, their consultant will charge us lots of money to certify a process that I’ll already have done), and it involves mucking about with lots of important parts of the system. So, when I haven’t taken a snapshot in a couple of hours, and I leave a script running overnight, no big deal, eh? Sadly, the team lead did not agree, and needed the precious ESX resources for the (3!) other VMs that were running at a total of 7% CPU load. Goodbye, 800 lines of Perl to massage the system into place. This would be less of a mess if there were any sort of reasonable rollback method, but the Linux devs, in their infinite wisdom, don’t think that’s important.

In fact, if you clone a drive, the logical volume manager will arbitrarily choose which one to boot from and prevent you from managing the other, even if the contents are totally different, since the cloned logical volume will have the same UUID. More than that, you cannot boot from a LV, so RHEL sets a label in the metadata for the boot partition (typically /boot1). This means in order to have a rollback method (a -real- rollback method), you have to clone said partition, take an LVM snapshot, rename it, hack the /etc/fstab for the partition which will become recovery, munge up the bootloader config files to get it to boot from /dev/vol/orig instead of /dev/vol/upgraded, etc. It was nicer to have straight-up partitions, in a way.

The sad part is that this is a SOLVED PROBLEM. ZFS (and Solaris Live Upgrades on UFS/VxFS/SVM) handle it with aplomb. The license simply doesn’t agree with Linux developers, so they’re slowly (and poorly) implementing it with btrfs and ext4. BSD has HAMMER now, and the license permits ZFS (though only FreeBSD has integrated it so far). Hell, OSX has ZFS support (not ZFS root, but you can get around that). Even better — ext4 loses data if you get a kernel panic, since they decided to delay allocation cycles by 115 seconds (from 5 -> 120) for a marginal performance gain unless you forcefully fsync(), which sorta defeats the point of having a journaled filesystem.

Maybe the work in Solaris Live Upgrade is from Bob. It very well may be, according to him, in one of those “I don’t know whether to gape in horror, pretend it was never said, or just say ‘lolwut’” moments. Specifically, he said that Microsoft contacted him and somebody else we went to school with 10 years ago to work on Solaris. Nevermind that Solaris is 18 years old, and Solaris <2.6 goes back further than that, that's it's based on code written around the time he was born, he doesn't really know C, and it has abso-fucking-lutely nothing to do with Microsoft. For that matter, I think Xenix (Microsoft's UNIX) was sold off to SCO before I ever used a computer. The worst part, really, is that he'll be an IP lawyer, and he wants to be one of the non-litigating ones who just researches things.

At the same time, his only knowledge of RISC processors is that the GBA and DS run on ARM (which he didn't know was RISC). Arguments about whether or not the PS3, 360, Wii, DS, PS3, his cable box, microwave, phone, and virtually every other device he uses fell on deaf ears. Which is fine, normally. I don't expect people to know that their cable box uses big-endian MIPS, or that the Wii, 360, and PS3 essentially run on the same processor with minor tweaks (ok, major tweaks, but they're all PPC970 at the core). Then again, I don't expect people to litigate patents related to these things, either.

It's just that I had no way to express to him how absolutely out of his depth he was -- that this is what I do for a living (about 50% of our systems are still AIX on POWER, and some are HP-UX on PA-RISC, with a little Solaris 10), and that he's dead wrong. No successful argument could have been made, since it wasn't Microsoft on x86. At the same time, though, I couldn't explain that a $400 PS3 has no tangible benefits over a $0 laptop (given that I already have it) when it comes to running emulators hooked up to a HDTV, since the controllers work on PCs just fine, and that throwing $400 at a PS3 so I could run Linux on POWER (well, the Cell) isn't really that interesting. At some point, I stopped doing things because I could, and just went with solutions that work for what I need (hence, fileserver is Solaris, for which he insisted I needed to buy a MyBook, but he's never seen a drive fail, and the fact that it comes with them set up in RAID0 with a processor so wimpy it cannot saturate a single GigE interface isn’t a problem).

On the other hand, hanging out with Bob is entertaining most of the time, and it definitely got my mind off of the pain I’m in. Tomorrow, I think, I’ll actually sign into WoW (assuming that I don’t go watch the crappy miniseries based on crappy books [Sword of Truth] that Bob is putting on a showing of).

Dammit

Ok, so cue this as being slightly more complicated.

I’ve got XML files describing most of what I need. The problem is that it’s only “most.” Example:

  <class>
    <name>Rogue</name>
    <role>Striker. You dart in to attack, do massive damage, and then retreat to safety. You do best when teamed with a defender to flank enemies.</role>
    <powerSource>Martial. Your talents depend on extensive training and constant practice, innate skill, and natural coordination.</powerSource>
    <keyAbilities>Dexterity, Strength, Charisma</keyAbilities>
    <armorProficiencies>Cloth, Leather</armorProficiencies>
    <weaponProficiencies>Dagger, Hand Crossbow, Shuriken, Sling, Short Sword</weaponProficiencies>
    <bonusToDefense>+2 Reflex</bonusToDefense>
    <firstLevelHitPoints>12 + Constitution score</firstLevelHitPoints>
    <firstLevelHitPointScript>12 + root.con.current</firstLevelHitPointScript>
    <hitPointsPerLevel>5</hitPointsPerLevel>
    <healingSurges>6 + root.con.currentMod</healingSurges>
    <trainedSkills><![CDATA[<html>
      Stealth and Thievery. From the class skills list below, choose four more trained skills at 1st level.<br/><i>Class Skills: </i> Acrobatics (Dex), Athletics (Str), Bluff (Cha), Dungeoneering (Wis), Insight (Wis), Intimidate (Cha), Perception (Wis), Stealth (Dex), Streetwise (Cha), Thievery (Dex)</html> 
    ]]></trainedSkills>
    <buildOptions>Brawny rogue, trickster rogue</buildOptions>
    <classFeatures>First Strike, Rogue Tactics, Rogue Weapon Talent, Sneak Attack</classFeatures>
    <modifierSetName>class.rogue</modifierSetName>
  </class>

Or:

<power>
  <name>Positioning Strike</name>
  <macroName>P_E_Positioning_Strike</macroName>
  <level>1</level>
  <source>Rogue</source>
  <flavor>A false stumble and a shove place the enemy exactly where you want him.</flavor>
  <type>Encounter</type>
  <keywords>Martial, Weapon</keywords>
  <action>Standard</action>
  <attackTypeAndRange>'Melee weapon'</attackTypeAndRange>
  <requirementsMet>root.currentWeapon.melee</requirementsMet>
  <target>One creature</target>
  <attack>'Dexterity'</attack>
  <attackModifier>root.dex.currentMod</attackModifier>
  <defense>Will</defense>
  <hit>1[W] + Dexterity damage, and you slide the </hit>
  <hitWeaponDamageMultiplier>1</hitWeaponDamageMultiplier>
  <hitDamageModifier>root.dex.currentMod</hitDamageModifier>
</power>

See anything mentioning paragon paths? What archetypes change the effect (I think that one is Artful Dodger)? Neither do I. I’m doing to end up having to convert the PDFs to HTML and parse those anyway (though not for much, I don’t think). Yay.