Copyright ©1997-2011 Glenn Fleishman except as noted otherwise. All rights reserved. For permission to reprint, contact Glenn Fleishman at glenn at glennf.com. Photo © 2008 Laurence Chen; used with permission.
Turning technology from mumbo-jumbo into rich tasty gumbo
I talked about Web 2.0 on KUOW last night. Web 2.0 is a loose concept of assembling many services across the Internet into unique presentations that benefit the service offerers. So combine Google Maps with Craig’s List real estate, or RSS feeds from 1,000 radio stations into a comprehensive directory of podcasts. Web 2.0 is mashing up content into richer results through services that companies or organizations expose with interfaces. Unintentional uses through intention sharing, more or less.
KUOW is moving into Radio 2.0 by broadcasting on both analog and digital FM using HD Radio for the latter, and by simulcasting not just its main signal on the Web, but also what it calls KUOW2 24 hours a day on the Internet and on a second HD Radio channel. HD Radio allows FM stations to have several subchannels at varying levels of quality. KUOW is testing running the BBC News 24 hours a day on a third HD Radio channel, too. Radio 2.0 or maybe even 3.0.
Posted by Glennf at 1:28 PM
This is a note to myself in the future, because I apparently keep forgetting.
I have three Linux rack-mounted servers at a co-location facility run by digital forest, a company the name of which I shall embroider in gold thread on a sampler to be mounted above my office computer. I’ve had a few peculiarities with them recently, including a weird problem in getting PHP to work reliably in specific ways in conjunction with Apache, so I decided to upgrade to Fedora Core 4 from Red Hat 9. Fedora is a project that continues the open-source, freely available Linux distribution that was at the heart of Red Hat’s software. Red Hat now focuses on supported-driven commercial versions of Linux designed for enterprises. (In fact, I also bought a copy of their Enterprise Linux distribution for one of these servers, a plan I’m rethinking the deployment of.)
After performing some tests, including a full upgrade of a plain Red Hat 9 installation on an unused Linux box, I went in Sunday night expecting to be there three to four hours in the best case and six to seven in the worst case. Instead, I was there from about 7.45 pm til about 6.45 am.
What went wrong? I forgot a lesson I’d learned before. Despite many people’s experiences in having successful Linux upgrades, including using the yum software update tool to upgrade Linux while it was running and then reboot into an entirely upgraded installation, I have rarely had good luck. Linux has poor revert positions. The Red Hat and Fedora installers don’t leave an intact system, but rather write over software as they go. Unlike Mac OS X, Archive and Install option, and it’s basic behavior of not rewriting boot blocks and replacing items until the installation is complete, Red Hat/Fedora just plow ahead. I should probably look into safer Linux distributions, but Red Hat works fine for me as a platform.
My path should have been to migrate services from one of the boxes to another copying all non-system data. Then wipe that box and install. In the event of failure, I’d still have working services. With a successful install, I’d customize it with my settings and move services back. Repeat as need be.
Instead, I wound up with two servers down and hours of unhappiness in working through the difficulties of sorting out what went wrong. The best thing I had was an installer that let me run “linux rescue,” a limited version that let me mount volumes, copy files, and try again.
I had made good backups before starting of critical files on a fourth server, an Apple Xserve, and thus wasn’t worried about losing critical data. Less important data has been backed up digital forest and I need to review whether any of those files need retrieval. Probably just a few.
I left in the morning (during rush hour, no less) having gotten my third machine to run all the services save one that the other two handled. This allowed me to bring up my most important Web site. The one I had to leave down with an apology note in response to all requests was isbn.nu, which has a high database dependency.
Before leaving, I’d sent email to Penguin Computing, the source of all goodness I now know, about the problems. By the time I got home, they’d answered my emails, and a series of email and phone calls continued after I woke up from three hours sleep to head back in. With their advice, we determined that there was probably a hardware fault with my highest performance server, which used SCSI instead of IDE. It had a SCSI card not supported in Red Hat 9, so they had installed a customized version. But Fedora Core 4 and the Red Hat commercial version should both have worked fine. (I managed to do a full install of FC4 which hung on reboot at the SCSI load stage, while the commercial Red Hat booted once, refused to join a network, and then hung on reboots.)
They helped get me to a position where I was able to wipe and install one of the servers with Red Hat 9 and get it to a working position. Last night after my boy went to sleep, I turned it into a database server and re-enabled isbn.nu, which is a real moneymaker.
Today, Penguin issued me an RMA for the SCSI-equipped server, and the fine folks at digital forest unracked it, packed it, and shipped it overnight for me. Despite me installing unsupported software, the Penguin folks are going above and beyond by helping out. They may replace the SCSI hardware—I have a 3 year on-site warranty, so they could have sent a tech, but that tech doesn’t do software—but they’ll also wipe and replace the faulty OS and make sure it works.
So my lesson learned, I say to anyone reading this far and my future self: Copy, wipe, install, restore. No more of this upgrade nonsense for production systems. Life’s too short for server room all nighters. The other tip: never try to handle two servers at once. If I’d tried on one and it had failed, I could have moved databases with great ease. Instead, I tried on two at the same time.
All praise to digital forest, for being a great, great place, and having a 24 by 7 network operation center (new since their move to south of Boeing in a wonderful new facility) and Penguin Computing for their prompt and incredibly helpful efforts to get me running.
I pause to blog long enough to note that I have had three hours sleep between Saturday morning and now, and it’s not to do with the baby. Rather, servers. Rather, installing operating system upgrades for Linux. Rather, unexpected failures. Everything is back to status quo ante, one server will be unracked, sent back to the mothership for analysis and a potential SCSI card replacement, and then rejoin its siblings.
I am seriously tired. Why am I blogging? I’m that tired.
What kind of things? Two items have been bugging me for weeks and I keep forgetting to figure out how to report them: a wayward bike rack and a broken street sign right in front of my Fremont office.
I did 2 seconds of Google research and took care of it. If you live in Seattle, here’s where you report bike rack problems (for Seattle Department of Transportation racks), and here’s where you find out how to report broken and missing street signs (among dozens of other SDOT questions answered).
I’ve managed to avoid going to conferences for most of this year (just one in March), and I’ve missed CES (Consumer Electronics Show) forever. But there’s too much Wi-Fi at the show for me to skip this year. In January, I’ll be at CES on the Saturday and Sunday of the show, and then off to Macworld for the next Tuesday (keynote day) and Wednesday. And then home to collapse.
I filed my NY Times piece on this in late July with three tabletop radios from $250 to $600 about to ship. They’re still about to ship…but even later. A few weeks ago, I wrote about how all three were delayed. This report from News.com indicates that shipments in quantity are probably not happening until maybe Spring 2006.
The CEO of the firm that created the HD Radio standard, which uses a very low-powered signal around existing AM and FM analog broadcasts, is dismissing the slippage. “We are seeing a month-or-two slippage, which is not uncommon with new technologies,” he said in the News.com article.
I wish the reporter had hit harder on this. In a list of receiver shipping dates given to me by a radio station engineer that was assembled by someone in the public radio world, the receivers were supposed to ship early in 2005. The shipping dates slipped to mid-summer, then August/September. Now we’re looking at Dec. to March, with strong indications that March is the most likely date for quantity. Boston Acoustics should be soonest, by Thanksgiving; Radiosophy with some units by December; and Polk by March 2006.
Receivers for autos are widely available and becoming much cheaper. But they’re still selling in small quantities.
October 2011 | August 2011 | June 2011 | May 2011 | February 2011 | December 2010 | November 2010 | October 2010 | September 2010 | August 2010 | July 2010 | June 2010 | May 2010 | April 2010 | January 2010 | December 2009 | November 2009 | October 2009 | September 2009 | August 2009 | July 2009 | May 2009 | April 2009 | March 2009 | February 2009 | January 2009 | December 2008 | November 2008 | October 2008 | September 2008 | August 2008 | July 2008 | June 2008 | May 2008 | April 2008 | March 2008 | February 2008 | January 2008 | December 2007 | November 2007 | October 2007 | September 2007 | August 2007 | July 2007 | June 2007 | May 2007 | April 2007 | March 2007 | February 2007 | January 2007 | December 2006 | November 2006 | October 2006 | September 2006 | August 2006 | July 2006 | June 2006 | May 2006 | April 2006 | March 2006 | February 2006 | January 2006 | December 2005 | November 2005 | October 2005 | September 2005 | August 2005 | July 2005 | June 2005 | May 2005 | April 2005 | March 2005 | February 2005 | January 2005 | December 2004 | November 2004 | October 2004 | September 2004 | August 2004 | July 2004 | June 2004 | May 2004 | April 2004 | March 2004 | February 2004 | January 2004 | December 2003 | November 2003 | October 2003 | September 2003 | August 2003 | July 2003 | June 2003 | May 2003 | April 2003 | March 2003 | February 2003 | January 2003 | December 2002 | November 2002 | October 2002 | September 2002 | August 2002 | July 2002 | June 2002 | May 2002 | April 2002 | March 2002 | February 2002 | January 2002 | December 2001 | November 2001 | October 2001 |