Resizing / on ext3 remotely

This is a feature request we had at work, and ignoring the RedHat consultant’s “If all you have is a hammer…” mentality (he suggested Anaconda again, no matter how unsuitable it is for updating 4,000 remote servers), it’s not actually that difficult of a task. The prime difficult here, of course, is that you cannot shrink a live filesystem, and good luck unmounting / while the system is running. Additionally, it seems like nobody else on the internet has had to do it (or documented the process and changes needed), other than one guy whose documentation is useless (he put up a frakking binary of resize2fs, took it down because it was “a binary without source” [which the GPL abhors], and said he “didn’t have the sources anymore”. You’ll see further down exactly how large the necessary change to publicly available sources is [one line, and not a complicated one]).

In case you’ve never look at it before, the initramfs (and initrd before it, though RHEL still calls the initramfs an initrd, despite the fact that you cannot stuff an ext2 filesystem in there anymore to my knowledge) is a gzipped cpio archive, which makes mucking with it dead simple. Just make yourself a directory somewhere (I used ~/initrd), and issue:

gzip -d --suffix=".img" | cpio -id < /path/to/initrd

You'll find a directory structure resembling:

drwxr-xr-x 2 root root 4096 May  8 13:03 bin
drwxr-xr-x 2 root root 4096 Mar  6 08:54 dev
drwxr-xr-x 4 root root 4096 Mar  6 08:54 etc
-rwxr-xr-x 1 root root 2007 May 11 04:58 init
drwxr-xr-x 2 root root 4096 May  8 09:52 lib
lrwxrwxrwx 1 root root    3 May  8 12:09 lib64 -> lib
drwxr-xr-x 2 root root 4096 Mar  6 08:54 loopfs
drwxr-xr-x 2 root root 4096 Mar  6 08:54 proc
lrwxrwxrwx 1 root root    3 Mar  6 08:54 sbin -> bin
drwxr-xr-x 2 root root 4096 Mar  6 08:54 sys
drwxr-xr-x 2 root root 4096 Mar  6 08:54 sysroot

The bulk of the work is done in ./init, which calls out to nash (an ridiculously limited shell with almost zero constructs for doing useful stuff other than booting). It mounts /proc and /sys, makes device nodes for the consoles, null, zero, and shared memory, loads drivers (LVM/SCSI), then finds the root filesystem, mounts it, and pivots the root to continue normal bootup. The only real caveat here is that every binary in there is statically linked -- and that's not necessarily a bad thing, since portability is a lot easier.

To resize, we'll need resize2fs built statically, and e2fsck wouldn't be a bad idea either. There are make targets for both of them in e2fsprogs (either an SRPM or just sources, your pick). All the routines in e2fsprogs check to see whether or not the filesystem is mounted before they do anything by checking /etc/mtab or /proc/mounts (and nash's mount doesn't put an entry in /proc/mounts for reasons that I haven't dug into). That file does not exist during the init process, and while e2fsck will spit out a warning and continue to run, resize2fs isn't so lucky. From resize/main.c:

/*
* Figure out whether or not the device is mounted, and if it is
* where it is mounted.
*/
len=80;
while (1) {
  mtpt = malloc(len);
  if (!mtpt)
     return ENOMEM;
  mtpt[len-1] = 0;
  retval = ext2fs_check_mount_point(device_name, &mount_flags,
                                                  mtpt, len);
  if (retval) {
    com_err("ext2fs_check_mount_point", retval,
                _("while determining whether %s is mounted."),
                 device_name);
                exit(1);
  }
  if (!(mount_flags & EXT2_MF_MOUNTED) || (mtpt[len-1] == 0))
     break;
  free(mtpt);
  len = 2 * len;
}

That comes from lib/ext2fs/ismounted.c, which is 50% comments from Theodore T'so bitching about the HURD, and I don't feel like editing it (or reposting it verbatim, since the sources are easy enough to get). The gist is that it's checking for a non-zero return value on retval and bailing if it find one (which it will). The decision of modifying ismounted.c or just making a quick hack to resize/main.c is up to you (I opted for the quick hack option, which could be commenting out the if statement, deleting the block, or just setting retval to zero, which involves the least typing).

As an aside, I know braces aren't necessary in lots of languages if there's only one statement in the block (C being one of them), but it makes it seriously ugly without them. Let's hope he never has to add another statement and forgets to add braces.

With that modified (I opted not to modify e2fsck, since it runs anyway), we have handy-dandy make targets for the utilities we need.

make -C resize resize2fs.static && make -C e2fsck e2fsck.static

It should be obvious, but, ah... Don't do a `make install` on this. resize2fs is dangerous now.

Move the new binaries (located in $src/{resize,e2fsck}/*.static) to $initrd/bin, and `$EDITOR $initrd/init`.

At least on CentOS/RHEL, we're running an LVM root. If you're not, the correct place may be slightly different, but not too much. Find `lvm vgchange -ay...`, and tack the following lines after it:

echo Creating root device node...
#RHEL/CentOS ignore the 'defaults' parameter, but other systems may not
mkrootdev -t ext3 -o defaults,ro /dev/your/device/here
echo Resizing filesystem...
e2fsck -fy /dev/your/device/here
/bin/resize2fs -p /dev/your/device/here $newsize
#If you're running lvm, the initrd will contain a static lvm binary, too
#but not of the aliases to `lvmresize` and such, which is basically `lvm $argv[1] $@`
#Caveat: lvm will not resize volumes with snapshots yet, but if you don't have any
#you can resize now if you like.  You don't have to, since lvm will shrink online volumes
#once you're into the actual system
lvm lvresize -L /dev/your/device/here

I would probably recommend putting this into a different initrd than the one you usually use, setting it as the default in grub, and having a one-off script change grub back and delete itself (rc.local would be a good place for this, since knocking itself out of there would be trivial).

find . -print | cpio -o -c | gzip -c9 >/boot/$newinitrd

The new initrd will be ~1.5MB larger than the old one (less if you stripped the binaries), but that's a trivial size different compared to a rescue image or (gods forbid) Anaconda simply to resize. Remember: TMTOWTDI, and what OS vendors/consultants recommend is not always ideal for your situation (it was not for ours).

Bored

So bored it’s unreal, actually. Other than a strong desire find a Haskell developer and treat him like a punching bag (but that’s another rant, probably this weekend, now that I’m finally done getting a working Xmonad/xmobar/Mutt/zsh combination on OpenSolaris with proper statistics), I haven’t had anything to do. I haven’t logged into WoW in a month or so, though I may do that this weekend also. I’m sort of stuck in a rut here.

Half-Price Books had their warehouse sale last weekend, so I picked up some new books, but those aren’t going to keep me occupied for very long. Heather’s website has to be rewritten at some point, but I’m still debating technologies to use for that (I like Rails, I really do, but the core team doesn’t give a shit if they break features during minor version releases, so almost all the documentation you find is out of date without going back $n versions and applying all the errata). Django is seeming appealing. I do not, however, have any idea how well that’s supported on Dreamhost. Sure, there are entries for it on their wiki, but Rails wasn’t exactly stable when they first added support for that. For a personal site, m’kay. For a site for her business? I’d rather have something reliable. Not that Django isn’t reliable, just that I don’t know how reliable it is on Dreamhost. Maybe I’ll set up a Django site on my VPS and just point her DNS servers (which are not hosted by Dreamhost) there.

I am, in truth, spending an inordinate amount of time questing for the Amulet of Yendor. I forgot how much I loved NetHack, and Slash’em just makes it better. On the downside, valkyries don’t appear to start with a longsword in Slash, so I’m not going to get any free Excaliburs at level 5. Monks aren’t bad at all, but food is kind of a pain in the ass. I’d also guess that blessed scrolls of genocide drop less frequently, leading to me getting killed by liches a few times. Awesome.

I’m sort of considering implementing Vassal in PyGame. Vassal 40k is a really neat system. Dice rolling isn’t handled all that well, it doesn’t import AB files, and the Vassal engine doesn’t do reforms/wheels well at all. I’m not going to hack at the Java to fix Vassal (thought the developers have started work on Vassal Fantasy Battles), and I sort of want to learn PyGame anyway, which seems like a good motivation to do it. Dan and I never did finish the C# implementation (in fact, I never really did anything other than the XML parser, and I have no idea how far he got). I don’t imagine it taking me all that long to do it in Python though, with PyGame doing the heavy lifting of the GDI crap C# needs for me. Thoughs (take a look at Vassal 40k, for instance)? I don’t picture it being difficult to render sprites from images of models I find on Google Image Search, but I guess you never know.

I do have lots of time to read news, though, and I’ve come to the following conclusions:

• Slashdot: My first, and probably still my favorite. The Firehose is worthless, but the community moderation system is incredibly effective. It is the only place I visit where I can regularly expect to see somebody with a post-graduate degree in whatever subject it is posting. People like NYCountryLawyer (who is actually a lawyer) make the site. There are, of course, some libertarian “omg government never solves any problems” trolls (who don’t think they are trolls), but that goes across the spectrum (generally people who say this in response to anything have never looked at numbers comparing government spending to private spending in the same field — Medicare being a great example — and mistakenly believe that corporations wouldn’t rape everybody). Increasingly dominated by people who have never worked worked in the real world (with user ids near 1.5 million) and an astounding amount of GPL trolls (who think it’s the greatest license since sliced bread and slander any company which does not abide by it — including Sun’s CDDL, which was devised because they don’t own the IP on some of their drivers — while simultaneously whining that they can’t steal code [ZFS, dtrace, etc] from alternatively-licensed projects). These, though, are generally modded into oblivion also.
• (Total)Fark: Biting headlines, little regard for the average human being (which I sort of appreciate, actually, since most of the people in their stories deserved what they got). Effective moderation, but the comment threading system sucks (not that I really need to read the comments with TotalFark having ~1 story per minute, most of which are worth a read). JonSnow is awesome.
• Reddit: Reddit seems to be a mixed bag. I love /r/programming, and some of the other SubReddits are great. At the same time, the average Redditor believes that they are much more intelligent than they actually are. I often see people talking about how they were “too smart” to be in school which, while possibly true, isn’t really relevant given that school is primarily a social tool. People spew nonsense about how the public education system was designed to make for complacent workers who will follow commands and obey the bell (this is true, but that was 130 years ago, and I’d hardly say school is psychological prep for working in a meat processing plant these days). There’s a lot of talk (on Reddit and Slashdot) about home schooling by parents who believe their children are beautiful, brilliant, unique snowflakes. Their child is doing $randomthing (3d rendering, programming, whatever) or is in “7th grade” at age 9, so it must be better, though they have no unbiased metrics to use for comparison.

  This particular aspect of Reddit comes out in shit like this (which is, by the way, the 2nd time that subject has been on AskReddit in the last few months). I’m all for being inclusive, but seriously? Not everybody deserves it. Reddit’s comment threading system is good, even if I don’t like it as much as Slashdot’s (in particular, if a parent post is downvoted too many times in Reddit, it hides the entire tree, even if one of the replies has 1,000,000 upvotes), but I don’t know where the fuck the moderation is sometimes. To give this guy a platform where he can answer selected questions to rationalize his behavior and make it seem as if he doesn’t need help is, well… Ephebophilia isn’t that uncommon, but this guy states he’s attracted to 2 year olds, FFS, and that he “admires them” at playgrounds and whatnot (he says he’s never touched them or seen images/videos, but that’s extremely dubious, and it’s virtually assured that he will act upon his urges at some point). I’m not easily sickened but, well, read the comments. Virtually nobody calls him out on it. They just stroke his ego.

Somebody give me a suggestion. If not, I’m just going start writing a frontend for a Vassal clone (while that may sound stupid, I suspect that the biggest obstacle WILL end up in unit reformation/wheeling/etc rather than back-end logic, so I’ll write it first. Integration tests can cover everything else when I write it).

Oi, work

It’s such a joy to come in this morning to find out that somebody has helpfully shut down the virtual machine I was testing a RHEL4.7 -> 5.3 migration in. Not a supported move by RedHat (as noted, their consultant will charge us lots of money to certify a process that I’ll already have done), and it involves mucking about with lots of important parts of the system. So, when I haven’t taken a snapshot in a couple of hours, and I leave a script running overnight, no big deal, eh? Sadly, the team lead did not agree, and needed the precious ESX resources for the (3!) other VMs that were running at a total of 7% CPU load. Goodbye, 800 lines of Perl to massage the system into place. This would be less of a mess if there were any sort of reasonable rollback method, but the Linux devs, in their infinite wisdom, don’t think that’s important.

In fact, if you clone a drive, the logical volume manager will arbitrarily choose which one to boot from and prevent you from managing the other, even if the contents are totally different, since the cloned logical volume will have the same UUID. More than that, you cannot boot from a LV, so RHEL sets a label in the metadata for the boot partition (typically /boot1). This means in order to have a rollback method (a -real- rollback method), you have to clone said partition, take an LVM snapshot, rename it, hack the /etc/fstab for the partition which will become recovery, munge up the bootloader config files to get it to boot from /dev/vol/orig instead of /dev/vol/upgraded, etc. It was nicer to have straight-up partitions, in a way.

The sad part is that this is a SOLVED PROBLEM. ZFS (and Solaris Live Upgrades on UFS/VxFS/SVM) handle it with aplomb. The license simply doesn’t agree with Linux developers, so they’re slowly (and poorly) implementing it with btrfs and ext4. BSD has HAMMER now, and the license permits ZFS (though only FreeBSD has integrated it so far). Hell, OSX has ZFS support (not ZFS root, but you can get around that). Even better — ext4 loses data if you get a kernel panic, since they decided to delay allocation cycles by 115 seconds (from 5 -> 120) for a marginal performance gain unless you forcefully fsync(), which sorta defeats the point of having a journaled filesystem.

Maybe the work in Solaris Live Upgrade is from Bob. It very well may be, according to him, in one of those “I don’t know whether to gape in horror, pretend it was never said, or just say ‘lolwut’” moments. Specifically, he said that Microsoft contacted him and somebody else we went to school with 10 years ago to work on Solaris. Nevermind that Solaris is 18 years old, and Solaris <2.6 goes back further than that, that's it's based on code written around the time he was born, he doesn't really know C, and it has abso-fucking-lutely nothing to do with Microsoft. For that matter, I think Xenix (Microsoft's UNIX) was sold off to SCO before I ever used a computer. The worst part, really, is that he'll be an IP lawyer, and he wants to be one of the non-litigating ones who just researches things.

At the same time, his only knowledge of RISC processors is that the GBA and DS run on ARM (which he didn't know was RISC). Arguments about whether or not the PS3, 360, Wii, DS, PS3, his cable box, microwave, phone, and virtually every other device he uses fell on deaf ears. Which is fine, normally. I don't expect people to know that their cable box uses big-endian MIPS, or that the Wii, 360, and PS3 essentially run on the same processor with minor tweaks (ok, major tweaks, but they're all PPC970 at the core). Then again, I don't expect people to litigate patents related to these things, either.

It's just that I had no way to express to him how absolutely out of his depth he was -- that this is what I do for a living (about 50% of our systems are still AIX on POWER, and some are HP-UX on PA-RISC, with a little Solaris 10), and that he's dead wrong. No successful argument could have been made, since it wasn't Microsoft on x86. At the same time, though, I couldn't explain that a $400 PS3 has no tangible benefits over a $0 laptop (given that I already have it) when it comes to running emulators hooked up to a HDTV, since the controllers work on PCs just fine, and that throwing $400 at a PS3 so I could run Linux on POWER (well, the Cell) isn't really that interesting. At some point, I stopped doing things because I could, and just went with solutions that work for what I need (hence, fileserver is Solaris, for which he insisted I needed to buy a MyBook, but he's never seen a drive fail, and the fact that it comes with them set up in RAID0 with a processor so wimpy it cannot saturate a single GigE interface isn’t a problem).

On the other hand, hanging out with Bob is entertaining most of the time, and it definitely got my mind off of the pain I’m in. Tomorrow, I think, I’ll actually sign into WoW (assuming that I don’t go watch the crappy miniseries based on crappy books [Sword of Truth] that Bob is putting on a showing of).

Ruby vs. Perl vs. Python

Sigh. I didn’t check my fileserver before I left home. It booted into a non-xVM kernel, so the code in my CentOS virtual machine isn’t accessible. Not that it’s a big deal, it just means that I’ll only be posting the code I’ve got here at work for a routine to check file modification times via SFTP. The reason d’être seems to be making sure customers aren’t trying to grab files whilst they’re being updated, but I’m not sure on that. They wanted the code, I wrote the code, now it’s doing… something.

At this point, I’m attempting to write every script in Perl, Python, and Ruby simultaneously. It shouldn’t be a surprise that I’m most productive in Perl, but I’d still like to keep up skills in Python (and build them in Ruby). For one reason or another (the fact that HP-UX systems don’t come with Python, for one; Ruby performing about 10 times as slow as Perl/Python on the 1.8 interpreter, and the 1.9 interpreter not being production ready for another [and yes, JRuby is really fast {faster than Ruby 1.9}, but the JVM startup time is a killer for a script running out of cron, and not every system in my datacenter has a JVM installed]), the Perl version is almost always the one that goes onto a production box somewhere.

It’s also worth noting that writing Ruby/Python in a procedural manner, like it’s Perl with methods tacked onto the objects, doesn’t make any sense to me. Sure, it’s kinda fun to write it that way, and blocks in Ruby are really handy (even if they’re slightly more confusing to a non-Rubyist than pointers were the first time I used them), but it feels against the spirit of it somehow. I don’t know.

That being said, this code isn’t nearly what would go into production anyway. The Python really isn’t far off. Perl’s kinda slow when it comes to objects, though, and I’d never use Class::Struct in something which weren’t totally network or I/O bound (probably iterate through arrays or hashes instead). The Ruby? Well, ahh, it’d probably look a lot like this. I’m not exactly a Rubyist at this point, though, and I’m sure it could be cleaned up a lot more without playing Perl Golf with it (hopefully by integrating Rubyisms which don’t make it God-awful unreadable, like the code at the top of this post).

As a total aside, I loathe Coding Horror just a little bit more than I hate Joel on Software. We’re talking about two guys who probably spend as much time blogging as they do working, treat their readers like idiots, preach bad practices (Joel’s company wrote its own fucking programming language rather than a DSL, and he advocates against using Exceptions, among other things. Jeff Atwood [Coding Horror] has no idea what the phrase “use the right tool for the right job” means, and would rather advocate using .NET for everything).

FWIW, I realize now why I changed to this theme. WIDE textarea. Not all the Wordpress themes I’ve played with work nicely once I start fucking with the margins for it in CSS. I don’t give a damn about widget ready or what have you. All that stuff is easy enough to add by hand. I want a theme that’s not going to waste 1/3rd of the page on blankness.

WTB blogs from Dan.

So, here’s one script (the Craigslist parser will get posted tomorrow, I guess).

In Perl:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174

#!/usr/bin/perl
 
###############
# SFTPCheck 
#
# 1.0
#
# Description:
# Checks update times for files on Omaha's server to troubleshoot scp problems
 
#Import what we need
use Class::Struct;
use Net::SFTP;
use Net::SFTP::Util qw{fx2txt};
use Net::SFTP::Attributes;
use strict;
use utf8;
 
#Set up  a struct we can dump data into
 
struct fileinfo => { filename => '$', 
 			  modtime => '$',
			  oldtime => '$',
			  waschanged => '$'};
 
#Set up logging
open(OUTFILE, ">> /data/testscript/ryan/sftpcheck.out");
print OUTFILE "-----------------------------------------------------------------------------------------\n";
my $now = localtime(time());
print OUTFILE "Starting at $now\n";
 
#Set up the connection parameters
my ($user, $pass, $host, $dir, $stamp, $diff) = @ARGV;
 
my @listers;
 
 
#Open it
my $sftp = undef;
$sftp = OpenSFTP($sftp, $user, $pass, $host);
 
#Set the directory path, since Net::SFTP->ls doesn't return a fully-qualified one
my $prefix = $dir . "/";
 
#Run through a loop 30 times, sleeping for one second inbetween
for(my $count = 0; $count <= 30; $count++) {
 
	#Do an ls
	@listers = &lookforfiles;
 
	foreach my $file (@listers) {
		#Set the fully qualified name so we can stat is
		my $remote = $prefix . $file;
		#Grab the modification time
		my $stat = $sftp->do_lstat($remote);
		#If it changed, set waschanged and put the new value in
		if ($file->oldtime != $stat->mtime) {
			$file->modtime($stat->mtime);
			$file->waschanged(1);
		}
		else {
			$file->modtime($stat->mtime);
		}
	}
	sleep(1);
}
 
foreach my $file (@listers) {
	if ($file->waschanged) {
		print OUTFILE $file->filename . " was changed during the check!\n";
	}
 
	#Compare it to now
	my $mtime = $file->modtime;
	my $difference = time() - $mtime;
	#If it's less than five minutes, pass it off in seconds
	if ($difference <= 300) {
		print OUTFILE $file->filename . " was updated $difference seconds ago\n";
	}
	elsif ($difference <=3600) {
		#Otherwise, minutes with two decimal places should be precise enough
		my $minutes = $difference / 60;
		printf(OUTFILE $file->filename . " was updated %.2f minutes ago\n", $minutes);
	}
	else {
		#If it's really that old, just print hours
		my $hours = $difference / 3600;
		printf(OUTFILE $file->filename . " was updated %.2f hours ago\n", $hours);
	}
}
 
sub OpenSFTP
{
	my ($sftp, $username, $pass, $host) = @_;
	my %args = ( 
		user => $username,
		password => $pass,
		debug => '1',
	);
	$username = utf8::encode($username);
	$pass = utf8::encode($pass);
	print "Trying to connect to $host as $username:$pass\n";
	$sftp = Net::SFTP->new($host, %args);
	my $status = fx2txt($sftp);
	if ($sftp) {
		return($sftp);
	}
}
 
sub lookforfiles
{
	#Figure out what the timestamp should be
	my @lookfor = &sftpstamp($stamp, $diff);
	#Check for yesterday's date too, why not.
	$lookfor[1] = $lookfor[0]--;
	my @found;
 
	#Do the LS, which doesn't support globbing, and pass it off to a regexp to find the files we need
	my @list = $sftp->ls("$dir", wanted => sub { $_[0]->{filename}});
	foreach my $day (@lookfor) {
		foreach my $file (@list) {
			my $name = $file->{filename};
			if ($name =~ /$day/) {
				my $there = 0;
				while (!$there) {
					foreach my $loop (@listers) {
						if($loop->filename($name)) {
							#If the filename is already in the array, break
							$there = 1;
						}
					}
					my $foundname = fileinfo->new();
					$foundname->filename($name);
					my $stat = $sftp->do_lstat($prefix . $name);
					$foundname->oldtime($stat->mtime);
					push(@listers, $foundname);
					$there = 1
				}
			}
		}
	}
	return(@found);
}
 
sub sftpstamp
{
    (my $stamp, my $diff) = @_;
	#Set the tzinfo so we know what the stamp should be
	my $timediff = 3600 * $diff; 
 
	#Check if we're in DST
	my $dststatus = 0;
	my $dstfile = "/data/.DST_Status";
	open(DST, $dstfile);
	$dststatus = (<DST>);
	chomp($dststatus);
	close(DST);
 
	#If so, modify the time accordingly
	if ($dststatus) {
		$timediff -= 3600;
	}
 
	#Get the time and format it
	my @StampNow = localtime(time() - $timediff);
	my $day = $StampNow[3];
	$day = "0$day" if $day < 10;
	my $month = $StampNow[4] + 1;
	$month = "0$month" if $month < 10;
 
	#Pass the appropriate value back
	$stamp = "$month$day";
	return $stamp;
}

In Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184

import base64
import getpass
import os
import platform
import socket
import sys
import time
import traceback
 
import paramiko
 
class foundfile:
    def __init__(self, filename=None, modtime=None, oldtime=None, waschanged=None):
        self.filename = filename
        self.modtime = modtime
        self.oldtime = oldtime
        self.waschanged = waschanged
 
class sftpcheck:
 
    def __init__(self, host=None, username=None, password=None, dirname=None, stamp=None, diff=None):
 
 
 
        #Set up the initial variables for login
        self.host = host
        self.username = username 
        self.password = password
        self.dirname = dirname
        self.stamp = stamp
        self.diff = diff
        self.sftp = None
        self.found = []
 
        #Set up logging
        if platform.uname()[0] == "Windows":
            self.LogPath = "\\\\filer2\\data\\testscript\\ryan\\"
            self.DSTPath = "\\\\filer2\\data\\.DST_Status"
            self.DebugPath = "\\\\filer2\\data\\testscript\\ryan\\"
        else:
            self.LogPath = "/data/testscript/ryan/"
            self.DSTPath = "/data/.DST_Status"
            self.DebugPath = "/data/testscript/ryan/"
        self.DebugFilename = "sftpdebug.log"
        self.LogFilename = "sftpcheck.log"
        self.LogFileObj = None
 
    def login(self, host, username, password):
        #Connect to the server
        t = paramiko.Transport((host, 22))
        t.connect(username=username, password=password)
        self.sftp = paramiko.SFTPClient.from_transport(t)
 
        return (self.sftp != None)
 
    def listfiles(self):
        #Pick up how it's supposed to be formatted
        lookfor = self.formattime(self.diff)
 
        #Get a list of files in the directory
        ls = self.sftp.listdir(self.dirname)
 
        #Check if any match
        for fileinfo in ls:
            if re.search(lookfor, fileinfo):
                there = False
                while there != True:
                    for record in found:
                        if record.filename == fileinfo:
                            there = True
                    newone = foundfile(filename = fileinfo)
                    mtime = self.sftp.lstat(self.dirname + fileinfo)[8]
                    newone.oldtime = mtime
                    self.found.append(newone)
                    there = True
 
        return found
 
    def statfiles(self):
        #Run through the list of files we found and stat them
        files = self.listfiles()
 
        for fileinfo in files:
            #Stat the file
            fullpath = self.dirname + fileinfo.filename
            mtime = self.sftp.lstat(fullpath)[8]
            if fileinfo.oldtime != mtime:
                fileinfo.modtime = mtime
                fileinfo.waschanged = True
            else:
                fileinfo.modtime = mtime
 
 
    def finalize(self):
        for gotem in self.found:
            mtime = gotem.modtime
            nowtime = time.time()
            if gotem.waschanged:
                self.log_write("%s was changed during the check!" % gotem.filename)
            #Figure out when the last time it was updated was
            difference = nowtime - mtime
 
            #If it's less than five minutes, seconds for logging
            if difference <= 300:
                self.log_write("%s was updated %d seconds ago" % (gotem.filename, difference))
 
            #Less than an hour?  Minutes with two decimal places
            elif difference <= 3600:
                minutes = difference / 60
                self.log_write("%s was updated %.2f minutes ago" % (gotem.filename, minutes))
 
            #Otherwise, hours with two decimal places    
            else:
                hours = minutes / 3600
                self.log_write("%s was updated %.2f hours ago" % (gotem.filename, hours))
 
    def formattime(self, diff):
        #Set timezone for PST, since I don't need to import more 
        #python libs
        timediff = 3600 * diff
 
        #Check if we're in DST
        dsthandle = open(self.DSTPath, 'r')
        dststatus = dsthandle.read().rstrip("\n")
        dsthandle.close()
 
        if dststatus == '1':
            timediff = timediff - 3600
 
        #Get the time in epoch seconds, subtract the diff
        #pass it back in a struct so we can format it
        sftptime = time.gmtime(int(time.time() - timediff))
        sftpstamp = sftptime.strftime("%m%d")
 
 
        return sftpstamp
 
    def log_open(self):
        #Set up logging
        logfile = "%s%s" % (self.LogPath, self.LogFilename)
        try:
            file = open(filename, "a")
            self.LogFileObj = file
        except IOError:
            print "Error: Cannot open log file %s!" % logfile
        else:
            self.log_write("Log file opened: %s!" % logfile)
 
    def log_write(self, message):
        #Actually write to the log
        timestamp = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime())
        message_text = "%s %s\n" % (timestamp, message)
        self.LogFileObj.write(message_text)
 
if __name__ == "__main__":
    #Instantiate it
    check = sftpcheck(username=args[0], password=args[1], hosts=args[2], dirname=args[3], stamp=args[4], diff=args[5])
 
    #Set up logging to troubleshoot the connection
    debugfile = "%s%s" % (check.DebugPath, check.DebugFilename)
    paramiko.util.log_to_file(debugfile)
    args = sys.argv[1:]
 
 
    #Let us know when we're starting
    check.log_open()
    check.log_write("--------------------------------------------------------")
    check.log_write("Start sftpcheck.py")
    check.log_write("Try to login")
 
    #Try to get the info
    if check.login(check.host, check.username, check.password):
        check.log_write("Successfully logged in to %s as %s" % (check.host, check.username))
        #Loop 30 times, sleeping for a second inbetween
        for i in range(1, 30):
            check.statfiles()
            time.sleep(1)
        check.finalize()
 
    #If we can't, log it can exit
    else:
        check.log_write("Login to %s as %s failed!  Check %s for more information." % (check.host, check.username, debugfile))
 
    check.log_write("Stop sftpcheck.py")

In Ruby:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147

#!/usr/bin/ruby
 
require 'net/sftp'
require 'dir'
 
class FoundFiles
  #Set up a class to hold info
  attr_accessor :filename, :modtime, :oldtime, :waschanged
end
 
class SFTPCheck
  #Initialize variables
  attr_accessor :username, :password, :host, :dir, :stamp, :diff
  @found = Array.new
  @logpath = "/data/testscript/ryan/"
  @dstpath = "/data/.DST_Status"
  @logfilename = "sftpcheck.log"
 
  def login(host, username, password)
    #Log in
    @sftp = Net::SFTP.start(host, username, :password => password)
    return @sftp
  end
 
  def formattime(diff)
    timediff = 3600 * diff
    #Read the DST status file.  Sure, Ruby has a .isdst? method in the time 
    #class, but that doesn't necessarily mesh with work
    File.open("#{dstpath}", "r") do |line|
      #Strip the newline
      dststatus = line.rstrip
    end
 
    if dststatus
      #If it's DST, change it
      timediff -= 3600
    end
 
    timenow = Time.now
    sftptime = time.at(timenow.to_i - timediff).strftime("%m%d")
    return sftptime
  end
 
  def listfiles
    lookfor = formatfile(diff)
 
    #Loop through
    ls = sftp.Dir.each_entry do |file|  
      #Ruby supports PCRE matching.  Yay!
      if file =~ /#{lookfor}/
        there = false
        #Same BS as Perl and Python.  I don't want to use a .contains method
        while there != true
          found.each do |record|
            if record.filename == file
              there = true
            end
          end
          newone = FoundFiles.new(file)
          newone.oldtime = sftp.stat(@dir + file).mtime
          #Could also be written as found.push(newone)
          @found << newone
        end
      end
    end
 
  end
 
  def statfiles
    listfiles()
 
    @found.each do |fileinfo| 
      fullpath = @dir + fileinfo.filename
      mtime = sftp.stat(fullpath).mtime
      #If it's changed, modify the object
      if fileinfo.oldtime != mtime
        fileinfo.modtime = mtime
        fileinfo.waschanged = true
      else
        fileinfo.modtime = mtime
      end
    end
  end
 
  def log_open
    #Set up logging
    logfile = @logpath + @logfilename
    @logfile = File.open(logfile, "a")
    @logfile.write("Log file opened: #{@logfile}!\n")
  end
 
  def log_write(message)
    #Actually write to the log
    @logfile.write(time.now.strftime("%Y-%m-%d %H:%M:%S") + message)
  end
 
  def finalize
    @found.each do |gotem|
      #Figure out when the last time it was updated was
      difference = gotem.mtime - time.now.to_i
      if gotem.waschanged?
        log_write("#{gotem.filename} was changed during the check!\n")
      end
 
 
      #If it's less than five minutes, seconds for logging
      if difference <= 300
        log_write("#{gotem.filename} was updated #{difference} seconds ago\n")
      #Less than an hour?
      elsif difference <= 3600
        log_write("#{gotem.filename} was updated #{(difference/60).round * 0.01} minutes ago\n")
      #Otherwise, hours.  
      else
        log_write("#{gotem.filename} was updated #{(difference/3600).round * 0.01} hours ago\n")
      end
    end
  end
 
end
 
username = ARGV[0]
password = ARGV[1]
host = ARGV[2]
dirname = ARGV[3]
stamp = ARGV[4]
diff = ARGV[5]
 
#Instantiate it
check = sftpcheck.new(username, password, host, dirname, stamp, diff)
 
#Let us know when we're starting
check.log_open
check.log_write("-------------------------------------------------------")
check.log_write("Start sftpcheck.rb")
check.log_write("Try to login")
 
#Try to get the info
if check.login(check.host, check.username, check.password)
  #Loop 30 times, sleeping for a second inbetween
  (1..30).each do |nothing|
    check.statfiles
  end
  check.finalize
else
  check.log_write("Login to #{check.host} as #{check.username} failed!\n")
end
check.log_write("Stopping sftpcheck.rb")

DISCLAIMER:
Use this code totally at your own risk. The Python and Ruby should work, but I haven’t tested them.

EDIT:
WTF. The Ruby came in significantly shorter than the Python and the Perl, plus it’s more fun to write? That makes me wish we had it installed on any of our servers here.

Just a taste –

Since I’ve been bitching to Dan lately about the code I inherited, here are a few examples:

System.DateTime answer = today.AddDays(-1);
YesterdayTemp = answer.ToString("dd");
TodayTemp = today.ToString("dd");
string TodayMonth = today.ToString("MM");
string currlogdir = @"C:\Faxserver\Logs\";
int currHour = DateTime.Now.Hour;
int currMinute = DateTime.Now.Minute;
int currSecond = DateTime.Now.Second;
if (currHour == 00 && currMinute == 00 && currSecond == 00)
{
      if (File.Exists(currlogdir + "AutoFaxErrorCountLog_" + TodayTemp + ".txt"))
      {
           DateTime tempcreatetime6 = File.GetCreationTime(currlogdir + "AutoFaxErrorCountLog_" + TodayTemp + ".txt");
           string tempday6 = tempcreatetime6.ToString("dd");
           string tempmonth6 = tempcreatetime6.ToString("MM");
           if (TodayTemp != tempday6 || TodayMonth != tempmonth6)
           {
                File.Delete(currlogdir + "AutoFaxErrorCountLog_" + TodayTemp + ".txt");
           }
      }
      FaxCountLog(YesterdayTemp, lblFaxesTodayDisplay.Text);
      FaxErrorCountLog(YesterdayTemp, lblErrorsTodayDisplay.Text);
      lblFaxesYesterdayDisplay.Text = lblFaxesTodayDisplay.Text;
      lblErrorsYesterdayDisplay.Text = lblErrorsTodayDisplay.Text;
      TextlblDisplayTodayAdd("0");
      TextlblDisplayTodayErrorAdd("0");
      TextlblDisplayHourAdd("0");
      TextlblDisplayHourErrorAdd("0");
      FaxesHour = 0;
      ErrorsHour = 0;
      FaxesToday = 0;
      ErrorsToday = 0;
}
if (currHour != 00 && currMinute == 00 && currSecond == 00)
{
     FaxCountLog(TodayTemp, lblFaxesTodayDisplay.Text);
     FaxErrorCountLog(TodayTemp, lblErrorsTodayDisplay.Text);
     TextlblDisplayHourAdd("0");
     TextlblDisplayHourErrorAdd("0");
     FaxesHour = 0;
     ErrorsHour = 0;
}
if (ElapsedHours == 23 && ElapsedMinutes == 59)
{
     ElapsedDays += 1;
     ElapsedHours = 00;
     ElapsedMinutes = 00;
     ElapsedSeconds = 00;
}
if (ElapsedMinutes == 59)
     {
     ElapsedHours += 1;
     ElapsedMinutes = 00;
}
if (ElapsedSeconds == 59)
{
     ElapsedMinutes += 1;
     ElapsedSeconds = 00;
}
else
{
     ElapsedSeconds += 1;
}
if (ElapsedHours >= 0 && ElapsedHours <= 9)
{
     ElapsedHoursTemp = "0" + ElapsedHours;
}
else
{
     ElapsedHoursTemp = ElapsedHours.ToString();
}
if (ElapsedMinutes >= 0 && ElapsedMinutes <= 9)
{
     ElapsedMinutesTemp = "0" + ElapsedMinutes;
}
else
{
     ElapsedMinutesTemp = ElapsedMinutes.ToString();
}
if (ElapsedSeconds >= 0 && ElapsedSeconds <= 9)
{
     ElapsedSecondsTemp = "0" + ElapsedSeconds;
}
else
{
     ElapsedSecondsTemp = ElapsedSeconds.ToString();
}
blElapsedTimeDisplay.Text = ElapsedDays + " Day(s), " + ElapsedHoursTemp + ":" + ElapsedMinutesTemp + ":" + ElapsedSecondsTemp;
}

Replaced with:

int totalSeconds = DateTime.Today.TotalSeconds;
Stopwatch elapsedTime = new Stopwatch();
elapsedTime.Start();
public class Log {
     public void Rotate (FileInfo logFile)
      {
          if ((logFile.Exists) && (logFile.LastAccessTime.Date != DateTime.Today.Date))
          {
               logFile.Delete()
          }
      }
      /* snip */
}
public class Reset {
     public void Labels(bool day, bool hour)
          {
                lblFaxesYesterdayDisplay.Text = lblFaxesTodayDisplay.Txt;
           /*snip*/
           }
     public void Counts(bool day, bool hour)
     {
                /*snip*/
     }
}
Reset reset = new reset;
FileInfo fileErrorLog = new FileInfo(logdir + "AutoFaxErrorCountLog_" + DateTime.Today.Day + ".txt");
if (totalSeconds % 3600 == 0)
{
     if (totalSeconds == 0)
     {
          log.Rotate(fileErrorLog);
          reset.Labels(true, true);
          reset.Counts(true, true);
     }
     else 
     {
          reset.Labels(false, true);
          reset.Labels(false, true);
     }
reset.Dispose();
}
TimeSpan ts = elapsedTime.Elapsed;
lblElapsedTimeDisplay.Text = String.Format("{0} Day(s), {1:00}:{2:00}:{3:00}", ts.Days, ts.Hours, ts.Minutes, ts.Seconds);
}

No, I’m not using Delgates until I can decouple this:
AutoFax.Form1.btnCleanErrors_Click(object, System.EventArgs) AutoFax.Form1.btnRestartPrintService_Click(object, System.EventArgs)
AutoFax.Form1.btnStart_Click(object, System.EventArgs)
AutoFax.Form1.CheckRdy() AutoFax.Form1.CheckSaved()
AutoFax.Form1.ChooseThreads(int)
AutoFax.Form1.CreateNotAndRdy()
AutoFax.Form1.Dispose(bool)
AutoFax.Form1.ErrorOutSecondsNumberGet()
AutoFax.Form1.ErrorOutSecondsNumberSet(int)
AutoFax.Form1.FaxCountLog(string, string)
AutoFax.Form1.FaxErrorCountLog(string, string)
AutoFax.Form1.FaxLog(string, bool)
AutoFax.Form1.Form1()
AutoFax.Form1.Form1_Exit(object, System.EventArgs)
AutoFax.Form1.GetFaxCount()
AutoFax.Form1.InitializeComponent()
AutoFax.Form1.Main()
AutoFax.Form1.menuItemAbout_Click(object, System.EventArgs)
AutoFax.Form1.menuItemShutDown_Click(object, System.EventArgs)
AutoFax.Form1.OnChanged(object, System.IO.FileSystemEventArgs)
AutoFax.Form1.ProcessFile(string, bool)
AutoFax.Form1.ProgressIncrement(int)
AutoFax.Form1.Rdy()
AutoFax.Form1.read_ini_settings()
AutoFax.Form1.SendAlert(string)
AutoFax.Form1.SendEmail(string)
AutoFax.Form1.SendFax()
AutoFax.Form1.SendFaxToProgram(string, string)
AutoFax.Form1.Textbox1Add(string)
AutoFax.Form1.Textbox1Append(string)
AutoFax.Form1.TextbtnStartadd(string)
AutoFax.Form1.TextlblDisplayHourAdd(string)
AutoFax.Form1.TextlblDisplayHourErrorAdd(string)
AutoFax.Form1.TextlblDisplayTodayAdd(string)
AutoFax.Form1.TextlblDisplayTodayErrorAdd(string)
AutoFax.Form1.TextlblFaxesAdd(string)
AutoFax.Form1.TextlblQueAdd(string)
AutoFax.Form1.TextlblSendAdd(string)
AutoFax.Form1.timer1_Tick_1(object, System.EventArgs)
AutoFax.Form1.timer2_Tick(object, System.EventArgs)
AutoFax.Form1.timerHBT_Tick(object, System.EventArgs)

Who needs classes? That Form1 class can just get every method known to man.
More joy:

String rdydir = @"C:\RDY\INCOMING\";
String notdir = @"G:\NOT\";
String str1 = path.ToUpper();
String delim = rdydir;
String delim1 = "NOT.RDY";
String str2 = str1.Trim(delim.ToCharArray());
if ((File.GetAttributes(path) & FileAttributes.Normal) == FileAttributes.Hidden)
{
    File.Delete(path);
    FaxLog("deleted file: [" + path + "]", false);
}
else
{
File.SetAttributes(path, FileAttributes.Normal);
File.Delete(path);
FaxLog("deleted file: [" + path + "]", false);
}
String str3 = str2.Trim(delim1.ToCharArray());
String NotFile = str3 + ".NOT";
TextlblQueAdd("looking for: [" + NotFile + "]");

Replacement:

 string notdir = @"G:\NOT\";
string NotFile = Regex.Replace(path, @".*\\(\w+\d+).*", "$1" + ".NOT").ToUpper();

This is just…

string templine = sr.ReadLine();
sr.Close();
if (templine != null)
{
      int test = templine.IndexOf("[");
      int test2 = templine.IndexOf("]");
      string test3 = templine.Substring(test + 1, test2 - 1 - test);
      if (test3 != null)
      {
            Regex reDateTime = new Regex(@"^(\d{2})/(\d{2})/(\d{4}) (\d{2}):(\d{2}):(\d{2})$");
            if (reDateTime.IsMatch(test3))
            {
                  int test4 = test3.IndexOf(" ");
                  string test5 = test3.Substring(0, test4);
                  if (test5 == TodayMonthTempStr + "/" + TodayDayTempStr + "/" + TodayYearTemp)
                  {
                        filedatetimeisthesame = true;
                        // file exists and it has today's date, so just append to file
                        FileStream fs2 = new FileStream(templogpath + BeginLogName + MiddleLogName + TodayDayTempStr + ".txt", FileMode.Append, FileAccess.Write);
                        StreamWriter m_streamWriter2 = new StreamWriter(fs2);
                        string FaxLogDate2 = DateTime.UtcNow.ToUniversalTime().ToShortDateString();
                        string FaxLogTime2 = DateTime.UtcNow.ToUniversalTime().ToLongTimeString();
                        m_streamWriter2.WriteLine("[" + FaxLogDate2 + " " + FaxLogTime2 + "], " + FaxLog);
                        m_streamWriter2.Flush();
                        m_streamWriter2.Close();
                        fs2.Close();
                   }
              }
         }
    }
}

Replace:

if (fileAutoFaxLog.LastWriteTime.Date == DateTime.Today.Date) 
{  
      filedatetimeisthesame = true;
       // file exists and it has today's date, so just append to file
       FileStream fs2 = new FileStream(@"C:\Faxserver\Logs\AutofaxLog_" + DateTime.Now.ToString("dd") + ".txt", FileMode.Append, FileAccess.Write);
       StreamWriter m_streamWriter2 = new StreamWriter(fs2);
       m_streamWriter2.WriteLine("[" + DateTime.Now.ToString() + "], " + FaxLog);
       m_streamWriter2.Flush();
       m_streamWriter2.Close();
       m_streamWriter2.Dispose();
       fs2.Close();
       fs2.Dispose();
}

Lots of these (15 or so) in the code:

DateTime hbtTime = DateTime.UtcNow.ToUniversalTime();
string hbtMonth = hbtTime.Month.ToString();
if (hbtMonth.Length == 1)
{
     hbtMonth = "0" + hbtMonth;
}
string hbtDay = hbtTime.Day.ToString();
if (hbtDay.Length == 1)
{
     hbtDay = "0" + hbtDay;
}
string hbtYear = hbtTime.Year.ToString();
string hbtHour = hbtTime.Hour.ToString();
if (hbtHour.Length == 1)
{
     hbtHour = "0" + hbtHour;
}
string hbtMinute = hbtTime.Minute.ToString();
if (hbtMinute.Length == 1)
{
     hbtMinute = "0" + hbtMinute;
}
     string hbtSecond = hbtTime.Second.ToString();
if (hbtSecond.Length == 1)
{
      hbtSecond = "0" + hbtSecond;
}
string strHBTtime = hbtMonth + "/" + hbtDay + "/" + hbtYear + ", " + hbtHour + ":" + hbtMinute + ":" + hbtSecond;

Which is really:

string strHBTtime = DateTime.Now.ToString("MM/dd/yy HH:mm:ss");

This isn’t mentioning all the other weird shit in the code, like queues are always:

object whythefuck = queue.Peek();
string eh = whythefuck.ToString();
queue.Dequeue();

Rather than:

string morereadablelesswaste = queue.Dequeue().ToString();

Flags being set in 1,000 places to do the same thing. Every catch is handled the same way, but lots of copy+paste code rather than passing the Message to a method to handle it (better to just repeat ad-nauseum). Re-inventing booleans with lower performance by setting strings to “true” or “false” then comparing those (they never have any other value). No Dipose() or Finalize(). Ever. No comments. Pointless variable names that mean nothing, like tmpStr14. That also brings up: Hungarian notation. Gods, why use it in a strongly-typed language?

What started off as a 20 minute “add a method to email me” is going to be a “rework 4500 lines of code into (probably) 2000, by cleaning up the trash everywhere then adding classes and delegates.” Joy.

It’s Time to Explore My Feelings (or so says my horoscope)

So, yes, I’m on nights for another month or so. Nominally, I’m training somebody. Our night operators don’t seem to make it more than a few months. It’s possible that this is attributable to a pervasive incompetence in the lower levels of the IT ladder. The kind of people who apply for the job think that they’re God’s gift to computing just because they can troubleshoot a desktop or build a gaming machine. It’s a wholly different environment in a server room (and a UNIX server room at that). We don’t build our own systems. We’d rather have an enterprise level support contract, so we can have a new drive here same-day if we need to. We want remote administration (LOM, iLOM, whatever $vendor calls it). We want error reporting RAM and hot-swappable drive caddies. We want NAS clustering with intelligent rollover, and brocade switches. VLANs and BGP filtering. Yes, you can do all of these things on your own, but it’s generally not worth it. Troubleshooting comes down to the OS and application level. Enterprise-level hardware troubleshoots itself, then sends you (and ideally the vendor) an email letting you know what needs to be replaced.

Still, our night operators don’t make enough to be expected to know much about anything when they start. That’s why we train. It’s an exercise in futility. Without being here during the working week (daytime hours), you have no idea which developer handles which application. If something goes catastrophically wrong, or you have no idea what to do about some problem, you get to call somebody in the middle of the night. That person is often me. I’m ok with that, really. It’s far preferable to somebody who doesn’t even realize that there is a problem. Primary responsibilities of the night shift involve dumping files to tape. These are automated. An elementary schooler could handle it. Of our two night operators, neither one can manage. Mind, I’m training one of the two right now (and I doubt if the other will make it). This is day 6. Every night, he asks me the same questions. Repetition isn’t teaching this one. Documentation isn’t teaching this one. All that’s left to do is cross my fingers and hope that there’s a spontaneous genesis of gray matter in the next month. I suspect it’s not going to happen.

True, there is a language barrier. I don’t think that prevents one from following simple directions. If I tell you to type something, don’t blankly stare at me. Turn around and face the keyboard. I’d rather not have to spell “increment” for you each night when I watch you run one of our backups. At the age of 28, you should be able to spell “historical.” After all, you moved here when you were three. I’ve sent emails to better explain processes which are largely undocumented. I’ve walked him through the steps (ok, step: grep $thing /data/csv/spdump.csv) for finding when a particular stored procedure runs, or how to find the name of a stored procedure which needs to be run if a file hasn’t updated (it’s the same one-step process). He’s not going to make it more than a few weeks after training ends, and that’s fine with me, because you’re a cretinous lout with all the ethical integrity of a member of the house of Windsor.

It’s rare that I meet people with whom it genuinely makes me ill to speak with. This is such a person. The type who makes me realize that the modern feminist movement still has work to do. Generally, I have no problem fitting into any social situation or finding common ground with even the basest representatives of humanity. Suffice it to say that I’ve ceased speaking with him except when it’s necessary to teach him something (in vain). What kind of person complains that their wife is too tired to do the dishes, do the laundry, clean the house, and make dinner when she (by his own admittance) works 65 hours a week? What kind of person expects their wife to quit said job (which she likes) so he has more time to relax? Is it reasonable that he sees it as her duty, and hers alone, to take care of their children?

He’s a mentally stunted man, in every sense possible. After even fifteen minutes of conversation with me, it should be obvious that I do not empathize his plight. I do not agree that he doesn’t need to provide support for computers he sells. It’s great that they work at his house when he sells them. He’s selling them to people without much money, since in his mind, everybody should have a computer. Alright. I can see that, maybe. If that’s the goal, you should not bitch about people trying to negotiate a lower price as “cutting into your profit.” Selling computers which are lemons and cease to function after they are brought to the buyer’s home is not a pastime which anybody with moral compunctions engages in.

So here’s to you, misogynistic douchebag. I will relish with great zeal the moment in which you are escorted from the building, and out of my consciousness. I do not expect that to take long.

As an aside, what does it say about me when the only two people I can think of to invite to a wedding reception are my friend’s exes (note that it’s another friend’s wedding, and neither of them would really be appropriate)? Sure, I’m dating. Yes, H would likely come with me. I may meet somebody I like better by early August, but I’m skeptical at best.