Zebras, I tell you!

Here’s the conclusion first, so you don’t have to read this whole post:

I got very little writing done, some, but not much. But the little I did was good. I have things planned out for the next chapter.

On to the zebras (things that are opposite)

I’m going back on my vow never to talk about work. I’ve decided that’s dumb. For reason’s I’ll get to time permitting (sorry, it wasn’t). Yesterday was a darn good day. I had to jet out to SSU to drop off copies of one of my forms for graduating. I need signatures from the grad advisor and the profs on my committee. I figured I’d have to get my signatures on separate pages. But no! ALL of them were there! I was able to get all my signatures on one page. Then I had to jet home to do the Mom thing etc so I didn’t get much done, though I did meet my words per day target. So, yeah. Good day, I’d say.

Today, not so. I was at work until past 8:30pm for an urgent project implementation from 5 to 6:30 if things went slow. Ahem. OK, so people were discussing and testing until 5:45. My job should have been simple. Apply a SQL server hotfix, which installed flawlessly on test, no restart required, then backup the database, run a script and wait around until QA was sure we wouldn’t roll back. I applied the hotfix and WTF? The server needed to restart. OK, that’s annoying as heck, but I sent the restart command. The server is in a colo facility in So. Cal, by the way. The server does not come back up. Uh-oh. We call the colo facility to have a tech go hard boot the box. They say, OK. 20 minutes later still no server….

Now pretend the server name is something like SERVER9 (it isn’t) and that you have previously told the tech the server name (SERVER9) and its location in the rack chassy.

We call back. The guy wants to call us back because he doesn’t know where the tech is and he’s really busy. Ooookay. I say, doesn’t he have a cell phone? Can’t you call him and ask him if he’s lost (it’s a BIG facility) or having trouble finding our racks or (urk) the server won’t come up? We convince him he should do that. The tech, when contacted is heard to day, I can only see SERVER1 through SERVER9. Well, all righty. Press the button on SERVER9 would ya? (By the way, you count to 10 while pressing the button, then wait a bit for things to spin down and press the button again) They promise to call us if the server doesn’t come up. We’re constantly pinging the server. 15 minutes go by. No reply from the server yet. at 7pm I called my boss to discuss whether we should fail over to the warm server. In the middle of that conversation, the server FINALLY replies.

I swear I’m almost done. A drive failed some weeks ago, the hot spare failed over and the array got rebuilt no problem. Which we knew. But on the restart, the failed drive came back on line and the BIOS was waiting for a reply to its message that the known number of drives didn’t match the current number of drives, which message we can’t see from the tools at our disposal. When the COLO tech finally managed to restart us, that message went away because on this reboot now the number of drives matched the number from the previous time it counted up. So, we finally get started with our project at 7:30pm. The hotfix worked, because the server did not blow up while I was profiling it, scripts were run, stuff got tested and we all went home at 8:30 or so.

Now I’m going to bed.


Tags: ,

Comments are closed.