Fun with OpenSolaris and OpenVPN

Since it seems that my new neighbors run an unsecured wireless network (not that I’m one to talk, as I certainly don’t bother encrypting mine), I though I’d hop on. With names (I’m assuming by the same person) like Tubgirl.org and 2girls1cup I was a bit leery about them redirecting all my traffic, but it wasn’t to be the case. I have no problems doing anything, but I’m suspecting they ran an unsecured SMTP server which was used as a spam relay at some point. Port 25 is blocked. SSH port forwarding is apparently too complicated for Heather in the few times she can actually get on the networks (chalk that up to shitty transmission power and network code from Windows, since we have virtually identical laptops and I’ve got no issues in Solaris or OSX), so she can’t send anything.

I figured I’d set up a VPN server on the connection I still have at Steph’s house until such time as we actually get cable installed. I can’t do it myself, since I still have an account open with them. I could transfer the service, but then I wouldn’t get all the new subscriber deals, so…

Solaris doesn’t come with device nodes for tunneling IP connections (other than IPSEC, which I’m not going to set up due to the complexity involved for a short-term solution). It does have device nodes and drivers for bridging, but they’re kind of limited when it comes to pinging the host that initiated the tunnel from the client. A virtual ethernet adapter would get around that, but it’d be needlessly hacky.

For whatever reason, the tun/tap drivers didn’t want to compile. Well, it’s not “whatever reason.” It’s “developer doesn’t bother to check whether or not Solaris systems are 64 bit since he only gives a fuck about Linux.” Easy fix in the Makefile:
CFLAGS = $(DEFS) -O2 -Wall -D_KERNEL -I. -m64 -mcmodel=kernel -mno-red-zone

After compiling, more problems. The device didn’t want to attach. Kernel drivers for 64 bit x86 don’t go in /platform/i86pc/kernel/drv/amd64, they go in /usr/kernel/drv/amd64. Fixed and loaded. On to OpenVPN:

Compile and install. Generate SSL certificates, sign keys for VPN clients. Set up configuration:
local 192.168.0.8
port 443 #nobody's going to block outgoing HTTPS, and I'll never run an HTTPS server at home
proto udp
dev tun #L2 routing, not ethernet bridging. Clients don't need broadcast.
ca ca.crt
cert alucard.crt
key alucard.key
dh dh1024.pem #more encryption
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt #DHCP for the VPN
push "route 192.168.0.0 255.255.255.0" #routes to get to the rest of my networks
push "route 10.0.0.0 255.255.0.0"
push "redirect-gateway" #all traffic over the tunnel
push "dhcp-option DNS 192.168.0.1" #Windows needs this
push "dhcp-option WINS 192.168.0.2" #Also Windows
client-to-client #let clients see each other
keepalive 10 120 #send pings to keep the connection alive
tls-auth ta.key 0 #helps revent DoS
cipher BF-CBC #use Blowfish encryption
comp-lzo #compression
user nobody
group nobody #take away server priviledges
persist-key
persist-tun #keep keys and tunnels even after priviledge is disabled
status openvpn-status.log
log-append openvpn.log #don't overwrite logs on restart
verb 6 #verbosity for the log to diagnose connection problems
mute 20 #don't repeat more than 20 sequential messages in the log

Routes are set to let them see my network at home, but network clients need to know how to get back to them:
root@OpenWrt:~$ route add -net 10.8.0.0 netmask 255.255.255.0 gw 192.168.0.8
root@OpenWrt:~$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.8.0.0 192.168.0.8 255.255.255.0 UG 0 0 0 br0
10.0.0.0 192.168.0.8 255.255.0.0 UG 0 0 0 br0
192.168.0.0 * 255.255.255.0 U 0 0 0 br0
24.118.228.0 * 255.255.254.0 U 0 0 0 vlan1
default c-3-0-ubr04.eag 0.0.0.0 UG 0 0 0 vlan1
default 192.168.0.1 0.0.0.0 UG 0 0 0 br0
root@OpenWrt:~$
Good so far. NAT would be a good thing, though. They can all appear to come from my server. Turning on the firewall?
Configuration:
alucard[/]# cat /etc/ipf/ipnat.conf
map bge0 10.8.0.0/24 -> 0/32 portmap tcp/udp auto
map bge0 10.8.0.0/24 -> 0/32
alucard[/]#ipnat -f /etc/ipf/ipnat.conf
alucard[/]# ipnat -l
List of active MAP/Redirect filters:
map bge0 10.8.0.0/24 -> 0.0.0.0/32 portmap tcp/udp auto #change source port to random between 10000 and 20000
map bge0 10.8.0.0/24 -> 0.0.0.0/32 #catch everything else. ICMP/etc

List of active sessions:
alucard[/]#svcadm restart ipfilter
alucard[/]#/etc/init.d/openvpn start
Starting OpenVPN…
alucard[/]#ifconfig tun0
tun0: flags=10011008d1 mtu 1500 index 4
inet 10.8.0.1 –> 10.8.0.2 netmask ffffffff
ether 3c:7:0:ff:ff:ff

Tunnel’s up with the right address. Client config:
client
dev tun
proto udp
remote my.dyndns.address 443
resolv-retry infinite #never give up
nobind #don't bind to local address and port
persist-key
persist-tun
ca ca.crt
cert heather.crt
key heather.key
comp-lzo
verb 5
tls-auth ta.key 1
pull #gets everything with a "push" directive in the server's conf file
Connecting a client and testing:
ryan@xerxes ~ $ ping 10.8.0.1
10.8.0.1 is alive
ryan@xerxes ~ $ ping 192.168.0.2
192.168.0.2 is alive
ryan@xerxes ~ $ ping google.com
google.com is alive
ryan@xerxes ~ $ traceroute google.com
traceroute: Warning: google.com has multiple addresses; using 72.14.207.99
traceroute: Warning: Multiple interfaces found; using 192.168.0.8 @ bge0
traceroute to google.com (72.14.207.99), 30 hops max, 40 byte packets
1 10.8.0.1 (10.8.0.1) 8.643 ms 8.704 ms 8.412 ms
2 192.168.0.1 (192.168.0.1) 5.602 ms 1.691 ms 1.458 ms
3 73.237.96.1 (73.237.96.1) 10.583 ms 12.434 ms 18.580 ms
4 ge-1-40-ur01.webster.mn.minn.comcast.net (68.87.179.189) 10.214 ms 11.131 ms *
5 te-8-1-ur02.minnehaha.mn.minn.comcast.net (68.87.174.149) 10.884 ms 11.120 ms *
6 te-8-3-ur01.minnehaha.mn.minn.comcast.net (68.87.174.145) 11.021 ms 12.867 ms *
7 * te-8-1-ar02.roseville.mn.minn.comcast.net (68.87.174.141) 105.276 ms *
8 te-0-0-0-0-cr01.omaha.ne.ibone.comcast.net (68.86.84.29) 27.271 ms 26.469 ms 26.695 ms
9 COMCAST-IP.car2.Denver1.Level3.net (4.79.82.58) 35.925 ms 37.085 ms 43.306 ms
10 te-3-4.car2.Denver1.Level3.net (4.79.82.57) 35.323 ms 40.357 ms 37.765 ms
11 ae-32-56.ebr2.Denver1.Level3.net (4.68.107.190) 50.018 ms 48.416 ms 37.038 ms
12 ae-3.ebr3.Chicago1.Level3.net (4.69.132.62) 44.406 ms 47.588 ms 40.422 ms
13 ae-68.ebr1.Chicago1.Level3.net (4.69.134.57) 42.095 ms 43.703 ms 36.046 ms
14 ae-2.ebr2.NewYork1.Level3.net (4.69.132.66) 74.493 ms 67.525 ms 69.368 ms
15 ae-82-82.csw3.NewYork1.Level3.net (4.69.134.90) 66.745 ms 65.820 ms 69.375 ms
16 ae-33-89.car3.NewYork1.Level3.net (4.68.16.133) 63.640 ms * *
17 GOOGLE-INC.car3.NewYork1.Level3.net (4.71.172.82) 64.239 ms 65.020 ms 65.755 ms
18 72.14.236.215 (72.14.236.215) 68.622 ms 66.218 ms 65.106 ms
19 72.14.233.113 (72.14.233.113) 79.852 ms 78.445 ms 81.719 ms
20 66.249.94.92 (66.249.94.92) 79.104 ms 83.073 ms 81.338 ms
21 72.14.236.134 (72.14.236.134) 89.334 ms 83.721 ms 66.249.94.50 (66.249.94.50) 81.548 ms
22 eh-in-f99.google.com (72.14.207.99) 80.819 ms 79.071 ms 83.579 ms
Everything looks as it should, plus it’s routed through the VPN. Good. Well, good other than Comcast’s bullshit. What the fuck, 4 connections in Minnesota. I’m getting an extra 10 seconds of latency in minnehaha in (presumably) the same datacenter? I can’t fathom why it wouldn’t just immediately go to Roseville, and that 106ms latency in Roseville is ridiculous. Level 3 I’ll forgive because they’re a backbone provider. Comcast, not so much.

I know nobody else will really be interested in reading this, so it’s more for my own benefit should I need to duplicate it than anything, but… Should you need to set up a VPN server, you know who to call.

As an aside, my fileserver’s going to get an exciting 8 port PCI-X SATA card and 8 500GB drives. Another 7.5TB should hold me for a while, plus it’s a lot more spindles for performance. I’m really considering just getting Sun Rays (thin clients) instead of using a full-fledged desktop (they’ll do multi-head, too), and just turning on my Wintendo for games.

Tags: OpenVPN, Solaris

I’ve got way too many tabs

It’s been a while since I’ve made a post, not that I (seemingly) give a damn about that.  I keep collecting links, and it’s time for a dump.  It’s doesn’t really feel like a hell of a lot has changed in the gap between the last post and this, but it probably has.  I’m set to go back to school in January.  Actually, I’m registered at two schools (which I’ll be attending concurrently), and depending on how the coursework goes, I may end up taking classes at one of the colleges around here as well to ram myself through as quickly as possible.  I simply don’t see a point in delaying the inevitable, nor in taking an extraordinary amount of time to complete it, given that I’m doubting if it’ll pose a challenge.  The only real problem (as always) is the financial aspect.  I’m not keen on taking out thousands of dollars in student loans, but I make too much to get federal aid, seemingly.  This particular aspect of life I don’t take issue with, as it’s something that I’m doing on my own terms and which amounts to a commitment of ~6 months (a semester) should I determine to do something different in the middle of it, which is doubtful.

I don’t intend to be the average college douchebag who treats it as an extended four years of drinking and vacation from real life, on somebody else’s dime (largely because I have little regard for college students), but then, I don’t think anybody would expect that from me anyway (and I already got something closely approximating that in Georgia, anyway).  In other ways, though?  Not nearly as comfortable with the way things are headed.  Moving into a 30-year mortgage which locks me in my current geographic location (and for the moment, job) is just screaming “mistake,” and I can’t explain why.  Admittedly, neither the Economist nor the Financial Times would indicate this as a good idea.  If the market isn’t predicted to bottom out for a year (give or take), why buy now? There’s no sense of urgency for me in removing myself from my current living situation, and the only reason to do so that I can see is that FHA loans are going to have higher interest rates as of Jan 15th if your credit isn’t virtually perfect (mine is not).

Is a (maybe) 2% rise in interest rates going to counteract a (maybe) $20,000 drop in price?  It’s about the same as near as I can tell, at least in the short run. Sure, in the long term, the interest rate hike would end up costing me a lot more, but refinancing is not an impossible objective, and the short run is the only goal for now.  Presumably, when I’m done with going back to school, the person I’m with will actually have an income, and mine will increase.  I don’t particularly care whether or not she has an income, but it is a factor.  Qualifying for a $170,000 dollar house by yourself just isn’t much fun, and I have little interest in being mortgage poor.  If my projected bills are $number, I’d like to have some leeway with extra money so I can take vacations, put money in an “oh-shit” fund in case of sudden job-loss, car-loss, or the like, et al.  What I like least about it is being rushed.  To find somewhere in the next two weeks or so, get approved for that home, and all the rest so it’ll clear before the 15th.  To look for a while and find something you really like is one thing, but to be railroaded into it on a needlessly short timeline is something else entirely.  Sure, I found something that would be acceptable, but who’s to say that the homes which are just out of my “I can afford this and still have a reasonable lifestyle” price range won’t drop precipitously in the next year?  To note, I’m fine with compromising on some issues with a home, but I’d rather have more discretion.

Couple that with my extreme job dissatisfaction.  It’s getting kind of boring (rollouts and deployments are done for now, and there’s not much for me to do other than the day-to-day stuff which takes about two hours out of my 12 hours here), my hours have been fucked up forever, and they’re rearranging the upper management so I’m now directly reporting to somebody who, frankly, has no idea what we do here (IT at the corporate office) and how our methodologies work.  I want to take a position with a different company, but I’m not able to do that until said mystical approval is done.  There’s a glut of interesting jobs here right now, and I have no idea if there will be in two months or whatever it takes.  Maybe if I had somebody who actually understood what I do?  I don’t exactly have forty-five minutes in a block to speak with a broker or call a credit card company for which I have no information and no authorization to get myself removed as an authorized user from the account of somebody whom I’ve not spoken with in a year.  I actually have -gasp- work to do!  Last Friday when the city cut all of our copper (fiber was thankfully intact, but we still lost 4 T1s and all of our phone lines), I had other things to do.  Even on a regular day, the nature of the work is such that somebody could walk in at any moment and have a problem which needs to be solved immediately because it affects production.

While I’ve got a lot of leeway here, and a lot of freetime, I’m just not comfortable trying to make those phonecalls while I’m here, and that shouldn’t be hard to understand.  All in all?  I’m fine having a “joint” life, but I need to have some control over my aspect of it.  If you had been talking about getting a different job for the last three months and you saw some things you were interested in pop up recently, I wouldn’t tell you that you “needed” to wait because $thing, unless that need (emphasis on that word because it has a meaning which is not “want”) was urgent.  Well, what’s urgent?  I don’t know. Pregnancy? Current roommate is selling the house and you have nowhere to live unless…? Moving your business and you need somebody who you can lean on (if necessary) financially for a little while? Medical bills? Needs to get a new car because your old one is somehow unusable, stolen, or whatever? Legal problems? I don’t know. Just, y’know, needs. On some level, I’m utterly convinced that this’ll be the end of things if I don’t acquiesce, which is galling, and a big part of the reason I’m hesitant about moving forward. Nothing says that things are stagnating merely because they’re not progressing at a breakneck pace. Still, something feels… wrong. Then again, I always feel that way around this time of year.

Enough about me, though.  Onto the tab unload.
First off, Solaris kicks Linux’s ass:

-bash-3.00$ ./bonnie++ -d /tankWriting with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
alucard 6648M 81021 74 134971 28 97563 23 87675 94 213019 21 805.2 3
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 31624 99 +++++ +++ +++++ +++ 32376 97 +++++ +++ +++++ +++
-bash-3.00$ ./bonnie++ -b -d /tank
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
alucard 6648M 109277 99 129352 27 95762 23 88443 96 214448 21 632.2 2
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 181 1 +++++ +++ 182 1 180 1 +++++ +++ 184 1
#zfs set compression=on tank
-bash-3.00$ ./bonnie++ -d /tank
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
alucard 6648M 97067 92 195806 42 144370 33 84743 91 432407 43 10006 31
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 17807 99 +++++ +++ 13575 99 31412 99 +++++ +++ +++++ +++
-bash-3.00$ ./bonnie++ -b -d /tank
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
alucard 6648M 108341 98 179270 41 141544 38 83036 90 428718 46 1756 7
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 186 1 +++++ +++ 180 1 179 1 +++++ +++ 182 1

Take that, Linux software RAID (note that he’s got a few more drives in there than I do.  I’m running 4 U320 drives and 2 SATA drives, and he’s got 7 ATA drives.  The second option forces Bonnie++ to sync ever ywrite, so the cache on the controller isn’t netting me any extra performance there. 430MB/s? I can live with that. Given protocol overhead, that’ll cap the aggregated GigE card.

Secondly?  This.  It’s hard to imagine what kind of dick classifies himself as a “seduction artist,” but after seeing this on the front page of (of course) Digg, I think I have a pretty good idea how to become one.

1. Be desperate for other people’s approval.  So desperate, in fact, that you’re willing to make up anything at all in order to get comments from people who exist (or don’t) on your blog.

2. Pretend every conversation with a member of the opposite sex is flirting, regardless of whether or not they’re involved with somebody else, a lesbian, much older/hotter than you, playing cockblocker at the bar, etc.  In fact, that bartender who talks to you and spends time near you?  I’m sure it’s not because you spend a shitload of money on drinks and she wants bigger tips.  Nope.  She wants to bed you.  As a rule (and I’ve gotten bitched at for this enough times), people in the service industry are friendly because their income depends on you being pleased with them.  Sociable does not mean interested.

3. Give myself a dumbass nickname.  It seems that “Mystery,” “Shark,” “Style,” and other similarly-awesome monikers are taken.

4. Post sycophantic comments on other people’s blogs.  Clearly the guys who claim “triple-digit lays” are worthy of emulation and rimming.  No women would find that digusting.  In fact, if I find it digusting (and I’m not exactly a pillar of morality), there’s something very wrong with it.

5. Buy a shithole that doesn’t have a bathroom mirror and in such disrepair that the mailman will not come to my house.  Proclaim it a “babe lair.”  As a general rule, by the time you bring somebody back to your house from the bar (or wherever), it’s not going to matter much what the interior looks like unless it’s a total shithole (see: his house).

6. Be a mysogynist.  This step is key.  If you’re going to treat women as objects or toys whose only purpose is your own amusement, having any respect for the opposite sex whatsoever would kill my chances of being a “seduction artist.”  Maybe if I had a long history of rejection by women, I’d grow to loathe them enough to become a “seduction artist.”  Especially if I happened to be a virgin in my late 20’s who went home to cry after a girl at a party went home with one of my friends.

So, instead, here are some rules:

1. Don’t be a dick.  This precludes anybody who would ever call themselves a ”seduction artist.”

2. People are people.  Some people are vulnerable, some people are vengeful, some people just want sex. There’s nothing complicated about it. Ugly people, attractive people, old people, young people, they’re just people.  Talk to them as you’d talk to anybody else, and it builds rapport, or you have a two minute conversation or whatever.  If you’re out for blood (or sex), it’s not that hard to tell from a glance.  You’re better off having no expectations whatsoever.

3. Maybe it’s willful ignorance, but I can’t recall ever being rejected.  Ever.  Why?  See above.  Also, I have interests and goals other than adding another notch to my bedpost, which may interest people.

4. Stop giving a shit about sex.  Why?  See above.

5. Stop being a piece of shit who desperately reads idiotic books and internet posts about how to pick up women from the comfort of your basement/bedroom/whatever in hopes of avoiding another tear-soaked pillow because your friend (who you should be happy for, I guess) got sex and you didn’t.  Talk to people and see what happens.  People are attracted to confident, not cocky.  For that matter, people are attracted to those who don’t come off as fake.

6. When people talk to you, they are being human.  It doesn’t necessarily mean they’re attracted to you.  Babies can distinguish between good and bad socialization, much less adults.

It seems that this cottage industry (i.e. teaching people how to be more selfish and regard other less) is worth a fair amount of money. I’m going to publish a book.

Next:  college students are worthless.   Not much else to say about that, really.  Maybe we should take away their right to vote.  It’s not like they bothered in the last election.

And to conclude this (I’ll just end up writing another post with rants on economics anyway), here are a few Daily WTFs, including the one I think looks like code from U-haul. A gem I’ll probably use:

if (Jack.WorkQuantity == "All" && Jack.PlayQuantity == "No")
  {
      Jack.BoyType = BoyTypes.Dull;
  }

/sigh

I’m fed up with Dreamhost’s performance. Well, to be fair, I’m not entirely sure it’s Dreamhost’s fault. Part of it could be the god-awful slow Javascript parsing of Firefox. It doesn’t help that Firefox takes 450MB of memory for 56 tabs (yes, it’s a tad ridiculous, I realize that), when Opera takes 170MB for the same. I haven’t really touched Opera in a while, since I’m too attached to Greasemonkey, Firefox’s Javascript console, and the DOM inspector. Opera seems to have have reasonable alternatives for those now (other than Greasemonkey). My one gripe at this point is Opera’s tab handling, which was a plus before. I’m finding myself preferring Firefox’s “endlessly scroll through your tabs” option (or the dropdown), just because I can see what they’re titled, and easily check whether or not I have new GMail.

I suppose now that GMail supports IMAP, I should just set up Opera to poll that, and the windows widget is pretty good, when it comes to it. At least it doesn’t slow to a crawl when Slashdot loads an animated ad (I refuse to use AdBlock for sites I actually like. Slashdot’s whitelisted, and I find myself occasionally clicking their ads). The RSS feeder wipes the floor with Firefox, it doesn’t peg the CPU when I open it up (along with however many tabs I left last time I closed it), it remembers page and window positioning between instances. I kind of wonder why I ever switched.

It doesn’t help that the clueless dipshit who wrote one of our monitoring applications has no idea how threading is supposed to work. A program with a 20MB footprint should not soak 50% of a 3Ghz Xeon every 4-5 seconds while it polls. I haven’t looked at the source, so I have no idea what’s happening there, but it can’t be right. One of my Perl scripts (which totals HTTP hits) chews through 4GB of logs every day in about 6 seconds, at 30% CPU. I find it hard to believe that a non-forked Perl script is somehow more optimized than the C# threading library. It also doesn’t help that Outlook takes an extraordinary amount of memory to do anything, nor that Windows aggressively swaps programs you haven’t used in a little while. That’s nice, except that I have, at any given time, 9-15 programs open. Putty’s fine to swap. Outlook, WINWORD (which Outlook still calls for composing messages, even plaintext), IE, Citrix, and the like are not. A 10 second delay when I click on Outlook again? Nuts to that. If I could convince our Exchange admin to turn on IMAP/POP, I’d just move to Linux. MAPI sucks, and I’ve never gotten Evolution’s Outlook Web Access plugin to work properly.

VMware is a possible solution, but it’s ridiculous to virtualize Windows just so I can run Outlook. Similarly, I’d like to get our AD admin to enable LDAP spanning so I can get our *nix systems on the domain and stop replicating the forest to an internal LDAP server just to keep accounts synced.

As it turns out, it’s not just a problem at work. Dreamhost’s response times are pitiful from home, too. Nine seconds to respond to a HTTP request? Pass. I’m seriously considering migrating to Joyent’s OpenSolaris hosting, even though it may cost more. However, they only let you run one Mongrel (the server Rails works best with) instance. That’s fine, and Rails should respond in virtually no time. However, I need to more closely research Apache reverse proxying. I could move to Typo, Mephisto, Radiant, or some other system for blogging, but one Mongrel instance isn’t going to cut it if I’m running a few Rails apps, and Mongrel doesn’t handle PHP. Maybe FastCGI performance is better at Joyent. I don’t know. Just that I can’t handle this pitiful performance anymore.

I’ll likely see what sustained performance is like on the Intellistation (which will be a web server) via a redirected subdomain monitoring a SNMP daemon (realtime CPU/network graphing). I know my home connection holds up really well via FreeNX, but it remains to be seen whether or not Comcast decides to block port 80 if they see a lot of traffic.

As a total aside, I feel like it should be “an HTTP request” and “an SNMP” daemon, though all proper rules of English say it should be “a HTTP request” or “a SMTP” daemon. IETF (SNMP) and w3 (HTTP) both have websites which agree with the usage of “an” (via a Google search for “an SNMP” vis-a-vis “a SNMP” and likewise for HTTP), but I’ve yet to find definitive rules for usage with regard to acronyms. Instinct tells me it should only be used when it’s referring to a singular adjective phrase versus a predicate or plural, but I can’t establish why. Any thoughts, grammar Nazi?

Also, I highly recommend A Fine Frenzy’s CD.

I’ve mastered the art of squandering my time

Hell, I can’t even find time to write a blog on a semi-regular basis. It’s a little sad. In the time since my last post, I’ve decided to change my home network architecture. Energy prices aren’t getting any lower, and speed just isn’t keeping up any more. The C3600, Octane2, and DS20E have got to go (or at least be powered down). I’m ganking a recently decommissioned Intellistation 6224-33U from a colleague. IBM refuses to upgrade the firmware for dual-core support (ditto for Sun on their W2100z, which is built on the same platform), probably because it would have killed the incentive to upgrade to a 6217 (same platform, does have dual-core support), though the 6217 has PCIe. So, it’s a dual Opteron 254 with 4GB of RAM. Currently one U320 hard drive and a Quadro FX 1100. I don’t really give a damn about the Quadro, but hey, it’s there.

I aim to put in a PERC 4/DC (dual channel PCI-X U320 controller) and 3 more U320 drives (36GB) I have laying around. It probably won’t even cap a channel, but there’s not really enough internal expansion to add more unless I rip out the CDROM drive and put a 3 bay hotswap cage. That’s a distinct possibility, but if so, I’d be doing it with a PCI-X SATA (or SAS, whichever is cheaper, since SAS supports SATA drives) controller. There’s only two SATA ports on the motherboard, and even with eBayed SCSI prices, it’s just not worth it. Sure, I could get 3 147GB 15k drives or 3 300GB 10k drives, but I don’t see the point. They’re still going to cost as much as 500GB SATA drives (or more). The SCSI will be used for a decommissioned external array. External SATA arrays are ludicrously expensive, given that I’d rather use eSATA if possible instead of a depreciated standard (internal connectors for external devices). SATA arrays with U320/Fibre Channel interfaces are ridiculously expensive, and they pretty much all use (crap) proprietary hardware RAID. Sun makes a few JBODs that’d work, but again, ludicrously expensive. A Powervault 220S (I already have one, but more never hurt) with dual U320 controllers is $99 on eBay with sleds (no drives), so I’ll attach one of those if I need more spindles or more space.

It is, of course, entirely possible that the price of SATA arrays with FC/SCSI connections will drop by the time I need to add more storage, but I’m not counting on that for now. In the meantime, I’ll be putting 2 400GB SATA drives in. I’m just not sure on what kind of filesystem layout I want to have. I’ve got an Intel Pro/1000 MT (quad GigE) PCI-X card that’ll be going in there. The Fibre Channel array is a nogo until I decide to actually spend some money and pick up PCI-X HBAs on eBay, since it’s a PCI64/66 card. I’m not sure how many PCI-X buses the Intellistation has, but PCI-X is a parallel bus, so adding a 66Mhz card would drop the throughput of whatever bus it’s on to 533MB/s. That’s no good. If it ends up being on the same bus as the SCSI controller and GigE card, I’d be capped.

It’s not that PCI-X (64/100) is much better (800MB/s), but I’d rather avoid bringing it all down. It -should- be a split bus. prtconf -pv lists five PCI-X bridges, but until I actually try swapping cards in, it’ll be hard to tell if it’s actually split electrically, or if they’re just hanging the onboard USB/GigE/SCSI/SATA off different bridges and all the PCI-X ports are on the same bus. If it’s actually split, then no worries.

The idea is this:

• Intellistation A Pro 6224-33U. Dual Opteron 254 with 4GB of RAM running Solaris Express Developer Edition (SXDE).
• 4×36GB U320 drives, HW RAID5 on the PERC4, stuck in a ZFS pool
• 2×400GB SATA150 drives, pooled with the SCSI drives
• 10×73GB FC drives over 2×1GB HBAs (if the bus is split), pooled with the rest. If it’s not split, grab a dual 2GB PCI-X/133 FC HBA off of eBay when I have $200 to blow and attach it that way.

Additional possibilities:

• Powervault 220Sx2 with whatever drives I can scrape up. This would mean moving the current U320 drives to the onboard SCSI controller (again, dual channel U320, just that it’s only RAID0/1/0+1, not 5, and there’s no offload engine or battery backed cache). 80 pin (SCA-2) SCSI drives are much cheaper than 68 pin, since there’s a ton of servers getting decommissioned. SCSI (well, parallel SCSI) is disappearing as the trend to SAS and SATA drives continues in the datacenter, this should only get better for me.
• Some other kind of FC array.
• FC or SCSI array with SATA drives. MTTL is much lower, but /shrug. It’s cheap!
• Bump the Intellistation to 8GB of RAM, assuming DDR1 ECC prices get better (unlikely). If it comes down to it, I’d rather have more spindles than more RAM anyway.

It can, at the very least, take over the role of OpenVPN server, DNSMasq server (I like having DNS on my home network), Postgre server, Oracle server, SSH gateway, and LDAP server if I feel like being a pain in the ass and making everyone authenticate to the server (plus RADIUS) to get on my network. I don’t have encryption set up on my wireless network, and I’m not about to change that, but I could (should) set up a trunked VLAN subnet on the wireless which can only get out to the internet (and not route to the wired network) until you authenticate, at which point you get into the main subnet. I mean, what if some random person comes to my house (or parks outside) and needs the internet, like, now! Sure, there’s a coffee shop a block away, but what if they need it at 3AM when the coffee shop is closed?

Now, concerns…

ZFS keeps an ‘intent log.’ Similar to most journaled filesystems, it’s got a record of what it does and doesn’t do. Unlike most journaled filesystems (jfs, reiserfs, ext3 -j/ext4, NTFS, HFS+, VxFS, XFS), it doesn’t check the filesystem and replay the journal if the system crashes. That’s not an issue in many cases. ZFS relies on filesystem metadata and self-heals. Due to this, ZFS requires that the writes be committed (fwrite()) every time. Without replaying the journal, writes could be lost on power loss, and there’s not a way I know of to automagically fsck the filesystem when it comes up (actually, to my knowledge, fsck.zfs doesn’t exist). That being the case, you’d end up having corruption. ZFS can fix that. If it’s in the kernel or essential processes, though? You just hosed the server.

The entire point of a cache-backed drive is that it waits for sequential writes so it’s not constantly flipping around the platters, which really helps performance. A cache backed controller immediately returns success to the OS, though the write is not committed yet.When the system comes back on after a power loss or crash, it flushes the cache to disks, and you’re good to go. With flaky SATA drives, JBODs on a plain Jane controller (no cache, which a lot of Fibre Channel HBAs are), forcing a sync is good. With the cache backed controller, it’s bad. Solaris has a syscontrol setting you can change to prevent this from happening (while still leaving the ZFS Intent Log up and running, though turning that off is another way around it, which is not at all recommended). That works great if everything in your ZFS pool is cache backed (real hardware RAID arrays, drives run by a cache-backed controller, etc). In a mixed environment (as mine will be)? I take either the risk of poor SCSI performance or data corruption. I could forgo the hardware RAID, but then why use the PERC at all? The only advantage I can see is that I’d still have a writeback cache, which would be flushed far too often. There’s a way to set this in per controller in sd.conf, but that’s for Fibre Channel LUNs, not SCSI. Turning off ZFS’s cache flushing would negatively affect performance on the SATA disks. Best solution for now? Make each SCSI disk its own logical drive on the PERC, then zpool those with the SATA disks.

The max throughput of GigE is 125MB/s. Given protocol overhead, 80-90MB/s is more realistic. The cost of a GigE switch which supports 802.3ad (link aggregation) is $50. That being the case, I’m going to put another wireless router in my house in repeater mode, upstairs (where my computer is, and where this’ll probably be), put a 802.3ad switch on it, connect the GigE on the Intellistation to the router, and the quad GigE to the switch. That’ll solve the problem with certain Broadcom wireless cards not coming up until I log into Gnome, since I’ll just wire them, plus I’ll have the advantage of being able to issue WOL packets (Wake On LAN). Link aggregation effectively makes multiple NICs appear to be one, along with the bandwidth. Intel’s got a proprietary way to do it via ‘teaming,’ but that’s only supported on their cards. Yeah, I have one, but Intel’s implementation is nonexistent on Solaris. Fortunately, Solaris doesn’t need it. I can aggregate whatever I want, regardless of the vendor (take that, Linux bonding, FreeBSD IPMP, and Windows lack of any comparable feature!).

I’ll have four aggregated GigE connections on the switch with a different subnet, so lookups for filesharing succeed with an IP in the hostfile (rather than routing through the wireless for no reason). This gives me an optimal throughput of 360MB/s or so on the network, and that can always be increased via another Pro/1000 MT (PCI-X versions are cheap!). I’d have to pick up a PCIe multiple port GigE card (another Intel, probably) for my desktop if I want more than 90MB/s, but that’s not necessary just yet. It’s faster than my hard drive is, anyway. Ideally, once throughput gets high enough (more spindles), and I have more throwaway money, I’ll pick one up. PCIe has a direct link to the CPU/RAM anyway, so it doesn’t need to touch my hard drive if I’m just streaming it over the network into RAM.

How to share it, though? NFS and CIFS (Samba) are both rather unintelligent, and they issue an assload of commands for everything. Not a big deal on copying a few large files, but ever tried to move a ton of small files (say, music) over the network? Suck. NFSv4 fixes this. I don’t know of a Windows NFSv4 client. SMB/CIFSv2 fixes this. That’s only supported on Vista and Server 2008. What do I do here, then? I could install 2008 in a virtual machine just to share files. Seems like a damn waste, and I’ll never touch 99% of what it does. The machine’s going to be headless, I don’t want to use RDP. I don’t want Active Directory on my network. SMB/CIFSv2 is the only thing Server 2008 offers me. Solaris does everything else I want to do better. iSCSI has none of this overhead, but I’d need to specify a create a volume, export it to a system, then format it on the client. I can’t get direct access to that from multiple clients, and it doesn’t grow nicely. Yeah, I could create another iSCSI device, export it, mount it, and use the support for Volumes windows has to span them, but that sucks, and I still can’t access it from multiple systems. So, I could create a VM, install Server 2008, have it share the volume, and add more virtual disks as necessary (again, spanning via Windows) to share.

Again, this is not an ideal solution. My storage isn’t unified, and it’s a big hassle for me to go add more. Creating a 22TB ZFS pool at work took 15 seconds. Any idea how long that takes on Windows? Plus I have to go through filesystem checks if it crashes. I don’t really know what I’m going to do about that. Latency on closing a 1K file via NFS is about 4 seconds. It’s similar for CIFS. Assuming the process reading/writing is multithreaded, it shouldn’t bottleneck. I have no idea how many of the applications I use are actually multithreaded, though, and I don’t really feel like digging around Process Explorer to find out.

Best case scenario?

Get a Thumper. Given that I don’t have $25,000 to blow? Get a PCI-X SATA card and an external case. The idea here is to save money on electricity, and attaching 3 arrays with redundant power supplies isn’t going to help that. A single case with a 300W with 8 SATA drives and cables funneled out of the Intellistation might work, since every company out there seems to be full of jackasses. It can’t honestly be that hard to support the SATA2 spec (no, it’s not 3GB/s throughput max) and give me a cheap port multiplier. Sequential throughput on a SATA drive is about 70MB/s. Random access is closer to 40. With 2 SATA drives plus 4 SCSI drives, I ought to be able to saturate a single GigE link pretty easily for now. With more drives, that’ll go up. Ten SATA drives plus the 4 SCSI should put me over the cap for the quad GigE. It’s not like I can’t add another card and aggregate those, too, but how fast do I really need it? PCI-X might disappear also (PCIe is gradually replacing it in servers), but the price of quad GigE PCI-X cards can only get better.

Just think of my zoning times at 360MB/s! That’s not going to happen now (or for a while), but I should at least be getting twice the speed of my hard drive.

The real problem for scalability is that I only get two cores. Hopefully, by the time it doesn’t scale to the demands of database/fileserver load/whatever, it’ll be a long time from now. 1TB drives should be less than $100 in a year. Who knows what I could get by the time this is obsoleted? I’d still like to dangle SCSI/Fibre Channel arrays off it, but I don’t think that’s going to go over well.

Looking at moving, condos, marriage, etc. Given the cost of weddings, it’s unlikely that I’ll have extra money any time soon. Given the average size of a condo, I don’t think I’d get a good reaction from whirring and clicking arrays, no matter how appealing the blinkenlights may be, plus 3U equipment is loud (ok, not as loud as 2U or 1U, but 60dB isn’t quiet). Regardless, perhaps I should appeal against condos with $250+ association fees on the basis that they’re costing me at least 1.5TB (7200RPM SATA) or 1TB (10k 300GB SCSI) a month, or more RAM, or something…