chxo internets

Farmville on Web Scaling

Wed, 10 Feb 2010 12:48:09 -0500

This was a pretty good read, How FarmVille Scales to Harvest 75 Million Players a Month. The lessons learned are worth generalizing:

Interactive apps are write-heavy.
Design every component as a degradable service, and provide a mechanism to degrade services server-side during peak demand.
Cache incoming data - they cache everything coming in from Facebook
Usage spikes during new releases of an app
Use sampling to make sense of large traffic flows

De-columnize Regex

Tue, 02 Feb 2010 22:59:47 -0500

\s+\|\s*\n\|

This regular expression matches the pipes and spaces in mysql output like:

| $dq$$form                  | 
| $dq$$module                | 
| $dq$$place                 | 
| $dq$$site                  | 
| $dq$$thing                 |

Sometimes it’s handy to extract just the values from a pasted query result.

Finding /dev/null from ZDE preferences.

Sat, 23 Jan 2010 17:19:00 -0500

Finding /dev/null from ZDE preferences.

Zend Studio 5.5 hangs "loading source control data"

Sat, 23 Jan 2010 17:17:44 -0500

Thanks to a post buried in the Google cache of Zend’s support boards, I give you the means to solve this annoying bug: set the svn and cvs paths to /dev/null.

I tried setting just the svn path to /dev/null, but ZDE.app was still loading source control data when opening the problem project. Then I set the cvs path to /dev/null, restarted, and no more loading.

According to the thread, this also works, changing ZDE/config_5.5/cvs_options.xml so that the value is false. Not sure if it works, I like the /dev/null solution too much.

Revert svn deleted symlink

Tue, 22 Dec 2009 20:59:00 -0500

Let’s say you have a symlink or two in your svn repository, and one day you decide to delete one of them. You use the svn rm command to do so.

$ ls -l tinymce-3.2.6
lrwxrwxrwx ... tinymce-3.2.6 -> ../../jslibs/tinymce-3.2.6
$ svn rm tinymce-3.2.6
D         tinymce-3.2.6 
$

But a minute later you realize that you actually need that symlink. You haven’t committed anything. Normally you would just svn revert and go about your business.

Alas, your working directory is in limbo now; svn revert won’t restore, and svn up ignores the missing symlink. The trick, apparently, is that you must commit the deletion before you can roll it back.

$ svn commit -m "Uh, accidentally deleted wrong version of tinymce"
Deleting       tinymce-3.2.6

Committed revision 12679.

Then you copy (I kid you not) the link back in from a previous revision, and commit that.

$ svn copy -r 12678 http://svn.example.org/path/to/tinymce-3.2.6 \ 
 ./tinymce-3.2.6
A         tinymce-3.2.6
$ svn commit -m "Respawned tinymce-3.2.6 from previous revision"
svn commit -m "'Resurrected' link from previous revision"
Adding         tinymce-3.2.6

Committed revision 12680.

Now the symlink is back and you can carry on.

The SVN Book calls this “resurrection”, I call it “respawning.” Thanks to this post for the tip.

Rock Solid HTML Emails

Sat, 19 Dec 2009 12:55:31 -0500

Great (no, really great!) summary of the many quirks to keep in mind when building an HTML email template:

http://24ways.org/2009/rock-solid-html-emails

I had to learn most of these the hard way recently, but I haven’t tested against Y! mail so I’m sure some of my work is broken.

The biggest disappointment, by the way, is Gmail. I mean, inline CSS only? Really? What kind of kludge is that? It bulks up message size considerably. I guess the alternative is to use… an iframe. Oh well.

"The goal is to look over our shoulder and see Snow White and Pinocchio and Dumbo standing there..."

Thu, 17 Dec 2009 08:15:27 -0500

““The goal is to look over our shoulder and see Snow White and Pinocchio and Dumbo standing there saying, ‘Be this good.’ We shouldn’t be intimidated by them; they’re an arrow pointing someplace.””

- Roy E. Disney [NYT]

Setting middle click to three finger click in BetterTouchTool.

Thu, 19 Nov 2009 15:00:06 -0500

Setting middle click to three finger click in BetterTouchTool.

Middle Click on Multitouch Magic Mouse

Thu, 19 Nov 2009 14:59:00 -0500

Using my shiny new multi-touch “magic” mouse now, and I love it. Scrolling is as natural now as it is on a trackpad, and right-click works flawlessly for a mouse with no buttons. But, oops! How do I send a middle-click (button 3)? You know, to open a link in a new tab?

Typical Apple: two steps forward and one step back. The traditional Mouse preference pane allows you to assign button 3 to the scroll wheel. But the new Mouse preference pane, for the magic mouse, doesn’t. It plays little videos to show you how to use the mouse, but the number of things you can do with it has been limited. Ugh.

Enter BetterTouchTool by Andreas Hegenberg, available at http://blog.boastr.net/. Launch it, map “three finger click” to MiddleClick, and you’re done. It ain’t pretty (yet?) but it works.

Host-Based Redirect in Apache

Fri, 11 Sep 2009 11:48:00 -0400

Here’s a recipe for redirecting by hostname in Apache. It’s like using Redirect but you can have more than one per VirtualHost container.

This is especially useful when you have a single SSL host with a lot of different sub-sites on it, and you want to provide the convenience of virutal host names to colleagues or clients. It’s obviously much easier for people to remember (and type) board.example.org than ssl.example.org/sites/board, and this technique makes it easy to provide that.


 ServerAdmin webmaster@example.org
 ServerName ops.example.org
 ServerAlias board.example.org it.example.org
 DocumentRoot /usr/share/apache2/htdocs

 RewriteEngine On

 RewriteCond %{HTTP_HOST} ops.example.org
 RewriteRule ^(.*) https://ssl.example.org/sites/ops$1 [R=301]

 RewriteCond %{HTTP_HOST} board.example.org
 RewriteRule ^(.*) https://ssl.example.org/sites/board$1 [R=301]

 RewriteCond %{HTTP_HOST} it.example.org
 RewriteRule ^(.*) https://ssl.example.org/files/it$1 [R=301]

These directives provide redirects for three different virtual hosts, subdomains of example.org. Each will redirect to a specific location at https://ssl.example.org/.

The $1 at the end of the RewriteRule causes the originally requested location to be appended to the redirect. In other words, a request for http://board.example.org/minutes.html will redirect to https://ssl.example.org/sites/board/minutes.html

Finally, the [R=301] flag causes Apache to issue a 301 Moved Permanently redirect, rather than the default 302. Some say this is better practice for search engines and such. That doesn’t really apply in this specific example (since these redirects are to secure sites) but it doesn’t hurt, either, and might save some browser overhead on subsequent requests for the virtual hostname.

Installing Java 1.5 on Snow Leopard

Tue, 08 Sep 2009 15:09:00 -0400

Important Note/Update: Installing “Java Update for OS X 10.6 Update 1” through Software Update will put your system back to square one. But you should update, since there are vulnerabilities that have been fixed in the new version. One workaround: move /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0 to a safe place before updating, then move it back later. Or as one commenter suggests, put it in a different place and link to it, then just restore the links after updating.

Original Howto:

A few other pages on the web give instructions for installing the Java 1.5 support that is (inexplicably) missing in OS X 10.6.0. Unfortunately they rely on an untrusted gzip file posted on a server at washington.edu that is getting hammered.

Here’s how you get it from Apple:

Download the official Java package from Apple, “Java for Mac OS X 10.5 Update 4” dated June 15, 2009.

Then use the excellent shareware utility Pacifist to open the downloaded JavaForMacOSX10.5Update4.pkg file.

How to install

1) First use Finder to go to System > Library > Frameworks > JavaVM.framework > Versions and delete the two aliases (symlinks) “1.5” and “1.5.0”. Don’t skip this step, because otherwise the extraction will follow the symlinks and overwrite the contents of the 1.6.0 folder, oops.

2) In Pacifist, drill down into Contents > System > Library > Frameworks > JavaVM.framework > Versions.

3) In Pacifist, select 1.5 and 1.5.0, right-click, and chose Install to Default Location

Now you can launch Zend Studio 5.5 or whatever else you needed Java 1.5 support for. Enjoy!

(And many thanks to Davey Shafik for pointing me in the right direction with his Fixing ZDE 5.5 in Snow Leopard post.)

Semantic Google

Mon, 27 Jul 2009 14:04:55 -0400

Marking up structured data

Google is finally starting to pay attention to semantic markup, starting with experimental support for review, product, and vcard microformats.

I’m looking forward to the day when they extend this to support rights metadata (Creative Commons) and some subset of the Dublin Core, but this is a great (late) start.

Publishers say e-books at $9 are unsustainable

Tue, 21 Jul 2009 07:50:59 -0400

“I’m thrilled that another major player is entering the fast-emerging e-book market,” [publisher] David Young said. “But I remain deeply concerned that our most valuable front-list titles are being sold at mass-market paperback prices.” — Barnes & Noble Plans an Extensive E-Bookstore, New York Times.

Here’s the problem in a nutshell, Mr. Young: readers will not pay full price for a DRM-crippled e-book. A first-edition hardback is solid and shareable, and meant to last for ages. It looks handsome on the shelf. Paying $26 for it is like an investment.

An e-book is locked away on a device, and digitally protected in a way that makes it clear that you only own the right to read it. If this is worth much less than what you want to charge, I suggest you find a way to add some serious value, fast. Note that iTunes charges more for tracks without DRM…

Kindle download policy clarified... mostly

Mon, 22 Jun 2009 20:43:00 -0400

KindleGate: Confusion Abounds Regarding Kindle Download Policy | Gear Diary

The long and short of it seems to be that, as promised, you can download a book as many times as you need to, but it can only be on a certain number of devices at a time. Usually, but not always, six.

As in, “Six readers ought to be enough for any book.”

DRM sucks, but okay, at least they’re playing fair with the unlimited downloads. We can call off the lawyers for now.

Is Internet Voting Safe?

Mon, 08 Jun 2009 08:00:00 -0400

Is Internet Voting Safe? The answer, of course, is not really. And it likely never will be in the way most folks imagine it. Here’s why:

Small-scale fraud is relatively easy with both paper ballots and electronic ballots. Look up any recent federal election for ample evidence of both.

But, as pervasive as it is, physical ballot fraud doesn’t scale well. It’s much harder to stuff ballots statewide than it is in just one county or polling place. Too many people have to be involved there are too many eyes on the system for large-scale fraud to go undetected.

Electronic fraud, because carried out in software, scales easily and automatically as soon as some exploit or security compromise can be found that enables it. Just look at how much spam is unwittingly sent by people’s PCs. Spamming and e-ballot-stuffing are two very different crimes, but the same kinds of client expoloits can be used for both.

Given sufficient motivation and time, an attacker can find one or more exploits in any internet voting system the runs on untrusted hardware, or on any upstream access point or router. He can then use that exploit to deploy programs that generate, alter, or prevent some number of ballots, systemwide.

Yes, properly implemented cryptographic potocols help, as do physical steps in the process (paper trails). The Arizona system discussed in the Wired article is a printable absentee ballot, not an online voting booth.

But no, I don’t think these methods help enough, not when real power is at stake.

At the very least, electronic voting should be limited to low-stakes offices and opinion polls, and even then results should be vetted using independent sampling. Votes should also be, for good measure, cryptographically verifiable by voters.

Amazon Recalling Some Kindle2s?

Sun, 07 Jun 2009 11:14:51 -0400

I got an email from Kindle support the other day, asking me to call them “at your earliest convenience.” Thinking that it was some kind of phishing attempt, I ignored it. But when they left a message on my home phone my curiousity got the better of me and I had them give me a call (via the Call Me feature of Amazon’s site).

The whole thing might be an experiment in how many times they can put a customer on hold before actually getting to the point of why they want to talk to you (note to Amazon: can you please just be up-front about why you’re contacting me?) but I had nothing better to do so I stuck with it.

Turns out they’re sending me a replacement Kindle2, because of something to do with the wireless hardware and planned upgrades they are making to their system. I asked if my first-gen Kindle would be affected and the rep said no. At no point was the word “recall” used, and yet that’s sure what it feels like.

Smart GMail Trick: Mute a Thread

Mon, 01 Jun 2009 11:58:41 -0400

You know those email threads which you don’t really care about, but which someone seems to post to every day for a week? You archive them, but they keep reappearing in your inbox with every new post.

Google calls this “the thread that just won’t die”, aka thread from hell.

Well, now you can silence them permanently by using GMail’s mute feature. View or select the thread(s) and type ‘m’ to mute. Bliss for compulsive inbox cleaners!

SSH - How to find remote host fingerprint

Sun, 10 May 2009 20:20:15 -0400

With OpenSSH, ssh-keygen is the go-to utility for discovering RSA public key fingerprints, both local and remote, using the -l (lowercase L) switch, and either -f for local keys or -F for keys stored in your known_hosts file.

To find out a remote server’s known-good ssh key fingerprint, use the ssh-keygen command:

ssh-keygen -l -F

Normally, ssh shows you the remote RSA key fingerprint when you first connect. But then you never see it again, because the key itself is stored in your .ssh/known_hosts file. Even when connecting in verbose mode, you don’t see it. Ssh compares the key it receives from the remote with the key in known_hosts, skipping the fingerprint step altogether.

So why would you want to see the fingerprint of a known key?

Well let’s say, for example, that you routinely ssh to cloud.example.com from your workstation. Then you get a netbook. The first time you ssh to cloud.example.com on the netbook, using wi-fi, the client shows you what looks like the wrong fingerprint. Is someone in the middle?

You can run ssh-keygen -l -F cloud.example.com on your workstation to see the key fingerprint for aws.example.com. According to the ssh-keygen manual, this looks up cloud.example.com in your known-hosts file, and then shows you the fingerprint of the matching entry:

ssh-keygen -l -F cloud.example.com
# Host cloud.example.com found: line 25 type RSA
2048 c4:63:f2:8b:ca:71:9b:e5:a7:2e:8a:de:49:ef:99:25 cloud.example.com (RSA)

Now you can compare with what the netbook sees, and either continue connecting or go raise hell with the wi-fi operator.

Friday Java Bashing

Sat, 09 May 2009 12:42:27 -0400

“Solr is popular with the enterprise crowd, who love its Java.

“Being a Java program, Solr includes no shortage of technology whose acronyms contain the letters J and X. This tickles the enterprise pink, because these sorts of developers love nothing more than hanging out around a whiteboard drawing boxes and arrows and, from time to time, writing XML to make it look like they’re doing real work.” - Ted Dziuba, in a short feature on Sphinx

Oh, snap!

Kindle is a Short Tail product

Fri, 08 May 2009 08:50:36 -0400

In re The Kindle Lets amazon Make a Lot From the Few (via Daring Fireball), I feel the need to point out two things, which should be of interest to any company or organization pushing intellectual property in our brave new digital world.

First, a $400 book reader is only going to appeal to people who read a lot of books. You don’t spend that kind of money if you only read books on vacation. So of course Kindle purchasers are going to be some of Amazon’s best customers.

Second, the Kindle provides instant gratification for book lovers. I can buy a book from anywhere with Sprint coverage and be reading it in under 5 minutes. No trip to the bookstore, no sales clerks, no waiting for UPS to redeliver. It’s right there, in my hands, whenever I want it.

I read a lot of books. 99% of the books I buy are now purchased, electronically, from Amazon, at what I can only assume is an enormous markup for them despite whatever wireless fees they pay to Sprint.

So what’s the lesson for other sellers-of-media? If you have a lot of content, find your most important customers and give them the shortest possible path to that content. Put it in their hands instantly, anywhere, and give them a capable way to play or read it.