Coding Relic

Code Snippet: SIOCGIFCONF

2010-03-10T07:01:00.000-08:00

A little while ago in this space we discussed SO_BINDTODEVICE, the socket option to control which physical interface will be used for packet ingress/egress. Recently in the comments of that post a question was posed: if you know the IP address of the interface, how do you programmatically find its name?

If there is a direct way to pass in an IP address and get back the interface name, I don't know it. The mechanism I know of is to retrieve the interface list from the kernel and walk through it until you find the IP address you're looking for. The code snippet below demonstrates the technique: the first use of SIOCGIFCONF determines the amount of memory we need, the second retrieves the interface list.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/ioctl.h>
#include <linux/if.h>
#include <netinet/in.h>
#include <arpa/inet.h>

int main()
{
   struct ifreq *ifr;
   struct ifconf ifc;
   int s, i;
   int numif;

   // find number of interfaces.
   memset(&ifc, 0, sizeof(ifc));
   ifc.ifc_ifcu.ifcu_req = NULL;
   ifc.ifc_len = 0;

   if ((s = socket(AF_INET, SOCK_DGRAM, 0)) < 0) {
     perror("socket");
     exit(1);
   }

   if (ioctl(s, SIOCGIFCONF, &ifc) < 0) {
     perror("ioctl");
     exit(2);
   }

   if ((ifr = malloc(ifc.ifc_len)) == NULL) {
     perror("malloc");
     exit(3);
   }
   ifc.ifc_ifcu.ifcu_req = ifr;

   if (ioctl(s, SIOCGIFCONF, &ifc) < 0) {
     perror("ioctl2");
     exit(4);
   }
   close(s);

   numif = ifc.ifc_len / sizeof(struct ifreq);
   for (i = 0; i < numif; i++) {
     struct ifreq *r = &ifr[i];
     struct sockaddr_in *sin = (struct sockaddr_in *)&r->ifr_addr;

     printf("%-8s : %s\n", r->ifr_name, inet_ntoa(sin->sin_addr));
   }

   free(ifr);
   exit(0);
}

Recursive Login Error

2010-03-08T07:01:00.000-08:00

If I log in, won't I end up here again?

Two Bots Enter, One Bot Leaves

2010-03-05T07:01:00.000-08:00

Let me share part of an email I recently received:

I just discovered Coding Relic, which is a really great resource.  I really enjoyed your article from last November on Foursquare.
 
<Redacted> is a client of our communications firm, and the topics discussed on your blog are well-matched with the themes of ...

Etc, etc. The rest of the email suggested that I write a post about the client. Unfortunately the truth is he did not just discover this blog, in reality he has no idea of its existence. He supplied a list of keywords to an automated tool which searches Technorati/IceRocket/BlogSearch/etc to send an email to those with contact information available. One of the keywords was "foursquare," and it found the one solitary article I wrote which matched.

This email was generated by a bot. It is given an initial direction, which it uses to autonomously pursue a goal. The goal in this case is to generate blog posts about the client. I've chosen not to write a blog post about the client as it is not related to the topics I'm interested in. Instead, I'm going to write about the bot.

Warning: Sports Analogies Approaching

This bot plays offense, in that it takes action to achieve a goal for the person who unleashed it upon the net. There are many similar bots which play on offense:

Email spam bots should top the list: harvest addresses, send spam email, repeat.
Twitter bots will follow, wait a few days for reciprocation, then unfollow in an attempt to build a large follower count. If you do follow back they generate a generic "Welcome and thanks for following my tweets!" message.
Hype generation bots, which retweet anything containing a keyword.
Google Alerts, which inform you when a site matching specified keywords is crawled by Googlebot.

We're in the middle of an arms race between the bots on offense versus the bots playing defense. Some examples of bots playing defense:

Email spam filters like spamassassin.
The ubiquitous anti-virus software.
Services like socialtoo block dangerous tweets, such as those containing a link to malware or a phishing site.
Services like Google Voice block spam callers, like the "your warranty is expiring" robo-call

We're still in the early days of networked bots vying for and against our interests. The attacking bots so far have been mostly unfocussed, broadcasted out widely in hopes of catching someone. We're just starting to see the rise of targeted malware, coded with information about the specific person(s) it is trying to reach. The offense is learning tactics and strategy. The defense needs to step up its game. Defensive bots now are specific to particular communications media, monitoring twitter or email or telephone. We're moving to a world where your online presence actively monitors communications channels you use, actively defending your interests.

Playing Offense

There is another good bit of advice from the world of sports: You can't win by playing defense. Many useful tools to actively monitor things you are interested in are free, and easily available. Anyone can use them, not just marketing and PR firms. One which I use is Google Alerts, a service by Google to monitor the web index for search terms you specify and email any new results. You can choose to be notified immediately, or to batch results together for a full day or week. You can also choose whether to monitor the entire index, or only specific venues like videos or blog search.

I use Google Alerts to keep track of my online presence, I want to know what the search results for my name will be. "You never get a second chance to make a first impression" is as true now as it ever was, but now the first impression is often formed via a web search for someone's name.

Brand Control Rage

2010-03-01T07:01:00.000-08:00

I wonder if some brand manager at Microsoft is lying awake at night, fuming:

It's Windows Phone 7 Series. The "Series" evokes future promise, the outline of an evolutionary path to reassure the possessor that their investment in Micosoft technology will accrue benefits for years to come. Calling it "Windows Phone 7" cheapens it into just another product, flaring oh-so-briefly briefly before it is trodden into the mud by Windows Phone 8. Curse you, Mary-Jo! Curse you!

Windows Phone 7 Series

Thatsa Lottabytes

2010-02-23T07:01:00.000-08:00

There is a Facebook fan page to establish hella- as the official SI prefix for 10²⁴. This is good, but I don't think it goes far enough. I propose that lotta be designated to mean "several orders of magnitude larger than currently achievable."

In 2010 lotta would be defined as 10^18.3, as having exabytes of storage seems comfortably out of reach. Each year it would be redefined, marching slowly to infinity (and beyond).

kilo	10³
mega	10⁶
giga	10⁹
tera	10¹²
peta	10¹⁵
exa	10¹⁸
zetta	10²¹
yotta	10²⁴
hella	10²⁷	(proposed)
lotta	> 10¹⁸	(my proposal)

Email Contact Virii

2010-02-18T07:01:00.000-08:00

We're all familiar with computer viruses: bits of code which attempt to transmit themselves to another system, from which they look for yet more systems to infect. The key point is that the virus must transmit code to infect another system.

Very occasionally there are more obscure examples of viral behavior, of data alone unaccompanied by code. Certainly, there is code involved, but it is the normal code of the system which performs a useful function and is not intended to spread data virally.

For example: people usually refer to me by the nickname Denny. My personal email address is denny@geekhold.com (1) but my work address is not "denny". One person had my personal email address. Shortly after I started he sent an email to a few team members. He typed "Denny," and GMail helpfully auto-completed my personal account from his contacts. When another member of the team responded to that email, the address was automatically added to his contacts. When he later sent an email to another set of coworkers... the Contact virus spread further.

The trouble with data-driven viral behavior is that there are few tools to stamp it out. With code viruses, a huge ecosystem of tools and malware signatures has been created. There are few tools to deal with an annoying bit of data spreading through the system, just manual exhortations to not respond to the email containing the external address.

Do people know of other instances of a viral spread of data, unaccompanied by code? There are certainly instances of poor auto-complete behavior, such as a mistyped URL leading to the browser forevermore suggesting the bogus site, but none I can think of which spread from one person to another. I suspect the root cause is a poor model for how auto-complete is supposed to work: the developer wants it to be completely automatic and not something the user should have to adjust.

(1): I long ago gave up keeping my email address off of spam lists. As I've used this address since 1996, I am wholly reliant on good filtering.

Resource Leak

2010-02-16T07:01:00.000-08:00

Resource leak located after extensive debugging. This portion of the system required significant refactoring to resolve the leak.

Apple == A, Plus Four More Letters

2010-02-05T07:01:00.000-08:00

What the web needs right now is another blog post about the iPad.

No, don't run away! This will be different, I promise. We'll focus on Apple's A4, a custom CPU first used in the iPad. It has been widely assumed that A4 uses a licensed ARM Cortex A8 core. I have no reason to dispute this assertion, it seems like a fine choice. It has also been asserted that because the same core is used in parts from Samsung, TI, and Qualcomm, Apple should not have bothered making its own chip. Today, Gentle Reader, we'll explore that notion a bit.

Apple ASIC Expertise

Apple has a long history of ASIC design. Apple produced custom silicon for various Macintosh models since at least the late 1980s, when they designed the audio chips used in the Quadra 700 and 900 (a chip called "Batman"). Later, Apple designed entire chipsets to interface with the PowerPC 60x bus. Apple licensed a gigabit Ethernet MAC design from Sun, and used it plus IDE controller and other peripherals in chipsets for several Powermac models. With the switch to x86, Apple's efforts became much more constrained. The x86 bus interface is difficult to license, and Intel's own chipsets are quite reasonable. So far as I know, x86 Macintoshes no longer use custom Apple ASICs.

Custom chip design isn't a radical departure for Apple.

With that as background, what might Apple have done in the A4 chip? I have absolutely no inside information about the iPad or A4 processor, I'm going to make stuff up because its fun to speculate.

Graphics & OpenCL

Apple holds a nearly 10% stake in Imagination Technologies Group, which designs the PowerVR graphics accelerator and other IP relating to massively threaded processing. Apple uses their PowerVR SGX 535 in iPhone 3GS, and used various PowerVR graphics in earlier iPhone models. The A4 chip will certainly integrate a graphics core from PowerVR. As with essentially all GPU designs today, the PowerVR makes use of multiple, specialized CPU cores. There is relatively little information about its instruction set on the web, ~~only that it is called META MTX and uses 16 bit RISC-ish instruction words~~. Update: PowerVR SGX does not use the META architecture, it has a distinct architecture of its own. Additional information can be downloaded after registration.

Apple has also invested heavily in two relevant technologies: OpenCL and LLVM.

OpenCL allows processing to be distributed across multiple CPUs in the system, even if they have different instruction sets. OpenCL algorithms are written in a language with syntax very similar to C99, and the framework handles the rest.
LLVM is a compiler toolkit, one aspect of which is a machine independent instruction set. Source code can be compiled to the LLVM virtual machine, and from there be translated into the equivalent opcodes for the target CPU. The compilation can be done statically before running it, or by a Just-In-Time compiler while interpreting the LLVM bytecodes.

iPhone applications are compiled to ARM instructions, but it is not much of a stretch to imagine support for sections of LLVM bytecodes as well. If the hardware has sufficient GPU power, the bytecode could be translated to the GPU instruction set and offloaded. Devices with less sophisticated GPUs would use the ARM instead. Apple does not allow iPhone apps to include their own virtual machine in this way, but would be free to provide the VM function as part of the OS.

I suspect this is the most compelling reason for Apple to build its own chip as opposed to buying off the shelf. The rest of the mobile industry is satisfied to offload 3D graphics and video decoding to the GPU. Apple has greater ambitions, and could make use of significantly more GPU pipelines. By controlling the complete platform from CPU to software, Apple can make tradeoffs which are not practical for the rest of the market. For example: a very large GPU plus very fast ARM would generate more heat than can be dissipated in a small form factor like a phone. Apple has the option to dynamically throttle the ARM clock speed in order to open up more thermal envelope for the GPUs, if sufficient OpenCL workload is ready to run. When the GPUs are less busy, the ARM clock speed can be brought back up.

Multi Package Modules

The CPU in the iPhone 3GS is a Samsung S5PC100. This is a multi package module with CPU, I/O chip, and SDRAM sandwiched tightly together. Multi chip modules have been around for a long time, where multiple dies wired together in one big package. The amount of testing which can be done on a raw die is rather limited, so MCM yields suffer as one bad die ruins the whole assembly.

Multi package modules are relatively new: each chip is in its own package, but use very tight pin spacings and do not have a heat spreader. They are soldered together on a small PCB, which in turn has a Ball Grid Array on the bottom with normal pin spacings. Because each chip is packaged separately a full suite of test vectors can be run before the final assembly is put together, improving the yield considerably and lowering the cost of the final product.

If we examine the main board of an iPhone 3GS, the largest component is not the Samsung processor - it is the Flash memory, an MPM containing a number of flash chips. The Samsung CPU in the iPhone 3GS comes in a close second in size, and is also a multi package module with CPU, I/O chip, and SDRAM.

With the A4, Apple will probably have one die containing both CPU and I/O. Samsung uses different I/O chips to tailor their offering to many market segments, which is not a goal for Apple. By arranging the pinout carefully, Apple might be able to make an MPM containing CPU, SDRAM, and Flash, reducing the total board area. Different Flash capacities could be offered by not stuffing portions of the MPM. The iPad itself might not need such an MPM as it is a much larger device, but future iPhones would benefit more.

To be clear: assembling an MPM is not something you can easily do when buying merchant silicon. The pins on one package have to be arranged so as to be easy to route to the pins on the other packages within the MPM.

Wrapping up

I'll say it again: I made this all up. I have no information on the specifics of the A4, just speculation. In the unlikely event that anyone reads this (instead of running away from yet another iPad blog post), don't copy it into Wikipedia as though it were verified information.

What about future iterations? Its tempting to consider a single chip containing the entire iPhone feature set, including radio and wireless networking. The A4 itself clearly doesn't do this, as GSM support is optional in the iPad. I suspect that even in future chips, Apple won't pull in the baseband radio. The front end portions of that chip are rather sensitive to noise, and generally don't work well when integrated in the corner of a gigantic ASIC. Also integrating the radio functionality would make it that much harder to keep up with advancements in wireless networks.

Another future possibility is to use this chip in Apple's other small form factor products, like the Airport Base Stations, Time Capsule, and AppleTV. This is certainly possible, but aside from obvious additional peripherals like SATA I'm not sure it adds many requirements to the chip.

Other articles about A4 you might find interesting:

The New York Times writes about the history of the A4 design team.
Louis Gray writes about Apple's heavy recruiting push to staff up their ASIC team.
Wikipedia already has an article, which will improve over time as more details emerge.

P.S.: While we're at it, the title of this post is a guess about the origins of the "A4" nomenclature: "Apple" is a capital A followed by four more letters.

Odd Calendar Behavior

2010-02-02T07:01:00.000-08:00

I wonder what that means...

Fuzzy Dice

2010-02-01T07:01:00.000-08:00

Even electric vehicles can be made cooler with a simple application of fuzzy dice.

On the Ancestry of Consumer Electronics

2010-01-27T07:01:00.000-08:00

Writing about the Newtpocalypse the other day inspired me to get my MessagePad 120 working again. In the comments of that post, Grant Hutchinson pointed out that my MP120 would not be affected and should still work. Indeed, it does! The backup lithium battery still had a little charge left, so my files are still present. The Newton pre-dates Flash memory, and stores all files in battery backed SRAM. The battery faithfully kept the SRAM powered for the last 13 years sitting in my garage, patiently waiting for me to use it again.

This morning Kara Swisher republished a Steve Jobs interview from 2004, talking about the products which Apple chooses to pursue and not to pursue. Its an interesting perspective on the eve of Apple's media event today. As she notes, in some respects Apple's new device will be "the Newton’s great-grandson."

Important Safety Announcement About Hot Air

2010-01-25T07:01:00.000-08:00

We interrupt these random musings to bring you an important safety announcement:

When a very small person spills juice in your laptop, a hairdryer is not the right tool for the job.

It still won't power on, and now nine of the key caps are mangled. I think I've simply disfigured its corpse.

Sun Microsystems, 1982-2010

2010-01-22T07:01:00.000-08:00

I was a summer intern at Sun Microsystems in 1991, working on Verilog test cases for a 155 Mbps ATM adaptor. I still have the T-Shirt, shown here. Only after we had the shirts made did the intern coordinator really look at it and notice the Sun logo was orange instead of purple, a bit of a gaffe. On the back are shown the companies who we, the interns, perceived as competitors. I've no idea why Apple is there while SGI is not, chalk it up to the exuberant misinformation of youth.

I returned to Sun as an ASIC designer (eventually transitioning into software) and worked there from 1992-2000. The first few years in that span were rough for Sun: the market was just beginning its transition from Unix-based CAD and graphical workstations to Windows NT on x86. Sun managed the transition pretty well, moving the bulk of its business into large SMP servers which the Intel boxes of the time did not scale to. Then came the dotCom boom, which were heady days for Sun.

Sun had a tradition of elaborate April Fools jokes. One year a complete hole of miniature golf was constructed in Scott McNealy's office. Another year a platform was built submerged 1" under the surface of the campus lake, with Andy Bechtolsheim's Ferrari parked upon it. I have the T-Shirt from the 1993 joke: the SHARCStation with superSHARC processor. A complete workstation was submerged at the bottom of a pool, with a scuba diver operating it. I don't remember whether it worked while underwater, but given the culture of Sun at the time I suspect they had come up with a way to waterproof it. Amusingly, I just searched for superSHARC: its now the name of a real processor, a DSP from Analog Devices.

I left Sun 10 years ago... but I've missed it. In the mid 1990s it was a heady place, a company at the top of its game with extensive resources to push the industry along. Its corporate culture valued engineers highly, which showed up in lots of little ways but one big one: enclosed offices. Sun would use cubicles if necessary, but the corporate preference was an office with a door. MTS level positions would be doubled up in an office, Staff Engineers and above were allocated a single office.

After the dotCom bust, Sun's fortunes changed considerably. The way is now clear for Sun to be purchased by Oracle. Rest in Peace, Sun. These last few years have not been kind.

Colloquialisms in Code

2010-01-21T07:01:00.000-08:00

Both of my parents are from Tennessee. Per capita, Tennessee has the highest density of colloquialisms than any other place in the western hemisphere. So ... lets code them up. You can mouse over each item to show the original phrase it purports to represent. Note: the Javascript rollovers don't work in the RSS feed, only on the web page. So far as I know, its all akin to voodoo.

--------

justification = my_ass.pull();

I pulled the justification out of my ass.

--------

Drawer = Set(["knife1", "knife2"])
if not "sharpest" in Drawer:

Not the sharpest knife in the drawer.

--------

if not (hand[left].find(his_ass) ||
        hand[right].find(his_ass) ||
        flashlight.find(his_ass)):

He couldn't find his ass with both hands and a flashlight

--------

ASSERT(dead.val() > doornail.val());

Dead'er than a doornail.

--------

if (them.fitness == fiddle.fitness):

Fit as a fiddle.

--------

CHECK(!(water[FISH_CONSTANT]));

Like a fish out of water.

--------

error: prototype for 'Silk_Purse(Cows_Ear)' does not match any in class 'Silk_Purse'

You can't make a silk purse out of a sow's ear.

--------

interface Wagon {
  public:
    void Wheels();
}
class Grandma implements Wagon {

If Grandma had wheels, she'd be a wagon.

--------

Bricks[LOAD - 2];

Two bricks shy of a load.

--------

class Shadow {
 public:
  Shadow(Shadow); // copy constructor
}
class Person {
 public:
  Shadow Stand();
}

Person Skinny;
Shadow S(2 * Skinny.Stand())

So skinny they have to stand up twice to make a shadow.

Recollecting The Calculus

2010-01-19T07:01:00.000-08:00

Allow me to summarize my recollections of The Calculus:

One must always refer to it as "The Calculus," never just "calculus."
The first derivative is the instantaneous rate of change, i.e. the velocity.
The second derivative is the instantaneous rate of change in the velocity, i.e. accelleration. Etc, etc.
Laplace transforms are a much easier way to compute derivatives.
I have no recollection of how to use the Laplace transform.

Thats pretty much it, in a nutshell. I suspect that had the modern web existed back then, my calculus classes would have been different but my retention of the material decades later would be about the same.

Intel Acquiring FPGA Vendor?

2010-01-15T15:01:00.000-08:00

EE Times reports on a JP Morgan Analyst prediction that Intel will acquire an FPGA vendor. The purported reason: to expand its competitiveness in embedded systems and system-on-chip. The two obvious market leaders in that category are Altera and Xilinx, though there are several smaller vendors like Actel and Lattice as well.

The SoC angle is interesting, in terms of the disruptive change it might allow. Freescale carries a huge variety of part numbers with various combinations of PowerPC core plus networking, USB, CAN-BUS, encryption, etc. Some of the functionality is implemented via an independant communications processor (a 68k descendant) alongside the PowerPC, to try to make each chip more flexible and able to serve different markets. Nonetheless, its still a very large collection of chips. Intel could be aiming for just a few different parts, with embedded FPGA blocks of various sizes and I/O pinouts. Need CAN-BUS? Buy a part with the right type of pins bonded out, with a license for soft logic IP to load into it. More sophisticated customers could load their own design logic into the FPGA blocks.

At one time Xilinx offered parts with hard logic PowerPC cores, but the CPU performance was modest and did not remain competitive over time. Xilinx and Altera both now emphasize soft logic CPU cores instead. These certainly work... but implementing a CPU in FPGA gates is an awfully expensive way to get your software to run. If Intel were to enter this space it would come from the opposite direction: a modern CPU core paired with a modest amount of FPGA logic.

EETimes published a subsequent rebuttal of the acquisition rumor, throwing around big numbers about the premium to be paid for Altera or Xilinx. The numbers make my head hurt, but its worth a read if you're interested in the topic.

Long Hidden Easter Eggs

2010-01-11T07:01:00.000-08:00

As we have a 4 year old daughter, computer games have become part of the background noise in our household. One of the popular games at the moment is Pajama Sam 2. It was published in 1998, and we'll see in a moment why that is relevant.

One scene in the game involves the Chairman of the Board brainstorming who should fill the empty board seat with the other board members. As it is a kids game the Chairman is an overstuffed seat, and the board members are anthropomorphic lumber. Of course. The dialog is complete filler, chosen at random from a small set of snippets:

Board Member:	"Did I suggest Bernie from Research?"
Chairman:	"Yes."
Board Member:	"Did I already mention Bill, the Security guy?"
Chairman:	"Yes."
Board Member:	"Did somebody suggest the supply clerk?"
Chairman:	"Yes."

There is one more snippet, weighted to be played with very low probability. I only happened to hear it once, when the game was left running for an hour. The developers had a little joke.

Board Member:	"Maybe we could get Gil Amelio to be on the board."
Chairman:	"I'd prefer an internal candidate."

Gil Amelio was the CEO of Apple at the time. Pajama Sam ran on both Windows and Macintosh, which was rather unusual for 1998. That was the deepest part of the "beleaguered Apple" period, where the death and dismemberment of the company was predicted daily.

This is funny for the parents, of course, because a corporate Board is supposed to consist mainly of outside directors to prevent excessive inward focus and make the company more Oh, nevermind.

(*) Amazon affiliate link

Y2.01K

2010-01-06T08:01:00.000-08:00

Y2K was a big deal. A huge amount of money was spent updating older software which stored dates using only two digits. A common solution at the time was to treat dates of 00-09 as post-Y2K, and 10-99 as pre-Y2K. Effectively this only delayed the problem for ten years - our ten years ran out a few days ago. Some examples of failures experienced since Jan 1, 2010:

SpamAssassin rejects mail with a 20XX date.
Symantec Endpoint Protection Manager rejects all updates dated after Dec 31, 2009. As a workaround before a fix is ready, they are releasing updates with a 2009 date.
30 million German debit cards stopped working on January 1.
SAP Spool queue entries are not automatically cleaned up if they expire after 12/31/2009, eventually filling up the disk.
Some Windows Mobile handsets and various Point-of-Sale terminals insist that it is actually 2016. This is an odd symptom, and may not be related to a Y2K fix.

A huge amount of money and effort was expended in the 1990s to upgrade systems for Y2K. When Jan 1 rolled around and civilization did not end, there was much criticism that the whole thing had been hyped by vendors eager to sell new gear. Quite likely we overspent... but I believe the consequences of underspending would have been more expensive to clean up afterwards.

Fortunately, it appears that magic pixie dust is available to fix all Y2.01K problems, according to a press release from BogusTech. Obviously a spoof, its very funny and well worth reading.

A Big Day for Mobile Technology

2010-01-05T06:02:00.000-08:00

Today, January 5, 2010, will be a big day in the history of mobile technology. A huge day. Any tech news site worthy of your attention should devote many resources to the events of today.

I am, of course, referring to the impending Newtpocalypse.

The Apple Newton was a groundbreaking device. The picture accompanying this post is my much-beloved Newton MP120.

NewtonScript tracks time intervals using a 30 bit signed integer, with an epoch of 1993. 2²⁹ bits from 1993 is January 5, 2010, at 6:48:31 pm. Tonight. When the NetwonScript time overflows into bit 30 it becomes a negative number, and Bad Things Happen. Fixes have been developed, but only for the later model Newtons. My beloved MP120 will stop working tonight. Sniff. Of course, it hasn't been charged since 1999 so its effective behavior will be unchanged.

PS: in addition to being the Newtpocalypse, I think Google has some kind of announcement today relating to mobile tech.

Access Hatches

2010-01-04T07:01:00.000-08:00

Yes, the rumors are true. Google has constructed a series of secret underground labs connected by high pressure pipes, to synthesize and transport Google Juice around the planet. Google Juice is devilishly difficult to generate and must be infused into a website quickly before it begins to destabilize, though once absorbed by the server it is remarkably stable.

The apparatus which powers this system requires constant supervision and maintenance, and if you watch carefully you might spot the access hatches scattered here and there.

(Kidding. Really. Nothing to see, please move on.)

Crowdsourcing Backup

2009-12-28T07:01:00.000-08:00

Jeff Atwood recently suffered a catastrophic loss of data of his long-running blog Coding Horror. The site was running on a virtual machine, and apparently VM backups at the hosting provider had been routinely failing for years without anybody noticing. Jeff maintained his own backups... within the VM itself, which were lost when the VM was lost. Jeff's story has a happy ending as one of his readers, Carmine Paolino, had a complete archive.

Obviously the happenstance of somebody on the Internet having a complete copy of data important to us does not constitute a practical backup strategy, but it got me to thinking about the idea of crowdsourcing backups. Everybody should have offsite backups, but practically nobody does it. Could a system be designed where each participant wanting to back up their most important data would in return offer a chunk of local disk space to use for storing data for other people?

With terabyte drives becoming common, it seems like many systems have an abundance of disk space which could be better taken advantage of. Perhaps the data you want to be backed up can be broken into chunks and stored in the free space of a number of other backup users, while your drive simultaneously stores their data.

Your data would have to be encrypted, as it will be stored on media controlled by random and potentially untrustworthy people.
A large amount of redundancy would have to be baked in, as people could drop out of the system at any time and take a chunk of stored information away. Many copies of each chunk would be stored in multiple places.
Forward Error Correction would also be good, to further improve survivability in the face of missing data. Recovering most of the chunks would be sufficient to reconstruct the rest.

The practicality of the details aside, with Amazon, RackSpace and others offering cloud storage options, would it even be worthwhile to construct such a crowdsourced system? In 2010, I'm not sure that it is. I suspect this is an idea whose time has come... and gone.

Satellites Should Respond to My Whims

2009-12-23T16:26:00.000-08:00

There was a pretty spectacular accident in Jamaica last night, where a 737 skidded off the runway and broke into pieces. Check the picture in the linked story, I'll wait. Amazingly only two passengers were injured.

Surely I'm not the only person who immediately checked satellite imagery, on the off chance that maybe, just maybe the periodic flyover happened to be this morning. Alas, no.

View Larger Map

North Pole Compression Algorithm

2009-12-21T07:01:00.001-08:00

Note the lack of the "HD" logo on the dust cover? Santa must have remarkable compression technology to fit the entire 1998 naughty/nice list on an 800k disk. The prevalence of popular baby names from year to year probably helps, there is a lot of duplication.

Slashdenfreude

2009-12-16T10:13:00.000-08:00

Slashdenfreude [slash-den-froi-duh] (noun) : To take joy in the slashdotting of others.

A Coroutine, Thread, and Semaphore Walk into a Bar...

2009-12-14T07:01:00.000-08:00

This article about multicore programming techniques is pure comedic gold.

In particular, threads suffer badly from 'race conditions'. The race of despised worker threads is made to do boring, low status, 'background' tasks. Meanwhile, the high privilege 'system' threads get to party with the hardware. It's the same the whole world over.

It is a great read with that peculiar British ~~humor~~ humour which The Register is so good at. It is also a good technical overview of techniques for taking advantage of multiple cores.