Vysnu » 2008 » April

Hpricot/LibXML Benchmark

Tuesday, April 29, 2008

I’m off to release an internal XML conversion script. I began using Hpricot, but after hearing good things about LibXML, decided to switch to that. Here’s a benchmark:

real	0m11.692s
user	0m11.497s
sys	0m0.188s

… for Hpricot versus…

real	0m6.441s
user	0m6.258s
sys	0m0.177s

… for LibXML

So that’s almost twice as fast. Nice to know.

Comment! | Permalink

Symfony 0.6.2 Sandbox Script

Thursday, April 10, 2008

This is for people who still use the old Symfony 0.6.2.

Get the script from here and execute it so:

sh ./symfony_sandbox.sh

This is largely plagiarized from the create quickstart script from the Symfony SVN but changed to make the pake install work.

Comment! | Permalink

kup.in: URL redirection off Google’s App Engine

Wednesday, April 9, 2008

First the URL: Kup.in is a fast URL redirection service run off Google’s new App Engine.

This was mostly a test project than anything else, just to gauge how easy App Engine is to start up and get going (very easy!). I’ll post in detail later, but here are bulleted observations:

1. I’ve been sitting on this project for ~6 months. The dilemma was that if I did this in PHP, I’d be slow coding it (and also ugly) but fast to deploy. If I did this in Python/Ruby, it’d be much easier, but deployment’s a bitch. With the App Engine, deployment is a simple appcfg.py update . which is uber cool.
2. You’ve got to have py25-openssl installed if you’re on Macports.
3. I couldn’t get Django templates to work (some import _md5 error which I suspect again is my Macports). I didn’t need templates here.
4. The code is opensource on Github. Fork it and do what you will! I’ll decide on a license after I wake up!

Speaking of which, it’s 3 in the morn here and I’ve got office tomorrow!

Comment! | Permalink

file_get_contents_with_timeout

Sunday, April 6, 2008

So here’s a piece of code which I think you guys will like. It’s a URL call with timeout function in PHP.

/* This works only for simple URIs
  Returns:

  array(headers, body): if everything's fine.
  TIMED OUT if it times out.
  UNABLE TO OPEN if we can't connect to host.
  */
private function file_get_contents_with_timeout($url, $read_timeout = 5, $connection_timeout = 5) {
  $url_parts = parse_url($url);

  $host = $url_parts['host'];
  $get = $url_parts['path'] . '?' . $url_parts['query'];

  $fp = fsockopen($host, 80, $errno, $errstr, $connection_timeout);
  if (!$fp) {
    return "UNABLE TO OPEN";
  } else {
    $out = "GET $get HTTP/1.1\r\n";
    $out .= "Host: $host\r\n";
    $out .= "Connection: Close\r\n\r\n";

    fwrite($fp, $out);
    stream_set_timeout($fp, $read_timeout);

    $result = stream_get_contents($fp);
    $divider = strpos($result, "\r\n\r\n");
    $headers = substr($result, 0, $divider);

    $body = substr($result, $divider, strlen($result));

    $info = stream_get_meta_data($fp);

    fclose($fp);

    if ($info['timed_out']) {
      return 'TIMED OUT';
    } else {
      return array('headers' => $headers, 'body' => $body);
    }
  }
}

Yup, a bit crude, but hey it works for me! (Oh, and it probably needs PHP5). Feel free to take the core logic out and wrap it up in exceptions and what have you.

Why did I have to write this contraption? Ah, the joys of the Indian SMS scene. Let me take you through a tour: Fastalerts is a bulk SMS solution for end users and resellers. The web frontend is written in PHP using Symfony (an older version, 0.6). The API is written in plain PHP and it connects to a SOAP-based (using nuSOAP) SMS sending solution.

This is a comment I have on top of the new backend code which sums up all the complexity:

/*
  A note on message sending.

  These actions have to happen as transactions:
  * Calling the GATEWAY.
  * Entering data into LOGS.
  * Reducing CREDITS.

  ONLY if:
  * User has "enough" credits.
  * Input is valid (numbers and mesage)
  * User has valid credentials

  AND we have to handle:
  * Gateway timeouts
  * Gateway errors
  * CDMA senderid correction.

  ALSO:
  If the user doesn't have enough credits to send the entirety of numbers,
  messages are sent until his credits are exhausted.

  AND:
  A separate status message is returned for each of these conditions.

  BUT WE ENSURE:
  That everything is logged appropriately:
  
  * All successfully sent messages are logged and credits reduced.
  * In every other case, credits are NOT reduced (we are customer friendly).
  * When gateway times out or errors out, the messages are logged.

*/

Interested people should note that there is a lot more you can add on to this: dynamic gateway switches (automatic failover), regular gateway tests, more backend support etc. but this is the bare minimum that’s needed for the backend to work. If you spend some time thinking about the problem, you’ll come to the realization that this bit:

 AND we have to handle:
  * Gateway timeouts
  * Gateway errors

means that we have to handle gateways that time out and never return a response. Most of the SMPP providers in India use something called NowSMS to manage their backend connectivity to the operator. In short: we connect to a vendor via his common gateway, he routes it (depending on destination) to multiple operators. NowSMS exposes a simple HTTP service (a URL call in other words) to add SMS to the vendor queue and it’s this service that’s preferred by a majority of the good vendors. However, HTTP traffic being what it is, a monolithic backend in PHP will stall if the request doesn’t return a response. Hence that file_get_contents_with_timeout.

Note: this is hardly an ideal solution (that would be a separate daemon). But without adding moving parts, this simple function should increase the reliability of our backend.

Comments (2) | Permalink