I code things

Recently I’ve created an API to manage and control a SaaS PHP App that I’ve built for my job. This API manages everything, deploying new versions of the App, provisioning new clients on the App, etc.

The deploy new version call is quite a long running one. It zips up a clone of the repo, puts it in S3, copies the static CSS, JS, images, fonts, etc into S3 for static content serving behind CloudFront and SSHs into all the EC2 instances of an AutoScaling group to deploy the app to a subfolder.

That whole process, mainly due to PHPs single-thread-ness is very slow and long running. It can take upwards of 15 minutes to deploy.

We have a control panel that issues these commands, but the problem is the browser. Browsers don’t always like 15 minute requests via Ajax as well as Elastic Load Balancers having a hard timeout limit of 1-2 minutes, and its just not reliable enough.

We could set ignore_user_abort(true) but to me, that doesn’t seem clean enough. I want to return a 200 from the Version Create API call.

Solution

Send a PHP-CLI process to the background of the server to do the long running hard work for you. This’ll only work in linux with php-cli installed.

This method requires two files: one to create the process and the long running PHP file you’d like to execute in the background.

Lets say you have a 15 minute script called long/deploy.php. This is how you’d fork of the script into the background to run for as long as it needs.

exec("php ".__dir__."/long/deploy.php >/dev/null 2>&1 &");

This would simply send the long deploy script of into the background. Sending all of it’s output to /dev/null and the error stream there too (that’s the 2>&1 bit).

Going further

Because we are sending the long running php process to the abyss, we aren’t able to ask it how it’s doing, unless we make the script write how it’s doing to a text file or a database.

In the example of my API’s deploy call, I find out how my long running script is going by updating a version_status INT(4) column in my versions table. The catch is, I create this row before I start the deploy script. This is so that I’m able to return the new version_id of the version created. But means we have to pass the long running process some arguments so it knows what row to update.

Sending arguments can be done easily by adding to the exec command above:

$result_id = $mysqli->insert_id; /* EG: 1234215 */
$args = array(0 => $result_id);
exec("php ".__dir__."/long/deploy.php ".implode(" ", $args)." >/dev/null 2>&1 &");

Inside long/deploy.php

print_r($argv);

/* Would output: */

Array (
    0 => 1234215 /* The $result_id above */
)

So this means you’ll be able to pass any string or int via the command and use it in your long running php process later on.

 Just want to say, this is all my own opinion. Also, I’m not a writer at all, I just wanted to contribute to this argument.

Just recently Australia’s Collation government released their Broadband network plans. This deployment would be a fiber to the node, then existing copper to the home type network promising up to 50mbit. This is apposed the the Labor’s NBN fiber to the home plan with capabilities over 1gbit.

The “benefits” of the Collation’s plan is simple:

• It costs less and;
• We can get it done sooner

And sure, thats true and to some people this option might make sense. Spend 1/3^rd and get it a lot sooner. And unfortunately that is a semi valid point. Cheaper and sooner sounds better to people who don’t even know what NBN means.

Now, I’ve learnt that ‘cheaper and sooner’ is a shit way of doing anything at all because you are missing out on one major factor – quality.

I will try and give you an example. Let’s say you needed a car. Tomorrow you could go out and buy a $5,000 used 1995 model. Sure, you’d have a car tomorrow with minimal investment but just imagine how that used car will feel after 2 years of using it. It will slow down, it will break more often and just turn to shit. Netting your total time spent ~3 years and ~$6,000-10,000 in total costs for this car (repairs and shit – old cars are expensive). About $3,000 to $5,000 per year over the lifetime of the car.

Now, lets say you spend a few months of catching the train to work, having only one car in the house, whatever, to save for car. A few months later you’ve got $10,000. You take out a further $20,000 long term loan and you’ve got $30,000 in your pocket. You wonder on down to your local Toyota dealer and put in an order for a GT86 (or any other car that costs about $30,000 – this is irrelevant). It arrives, brand new, ready to go. And you drive this car for a good 15-20 years, you service it regally and you are confidant that . With the servicing it might end up costing you $38,000 – $45,000 after 15-20 years. So anywhere from $2,500 to ~$3,000 a year.

Oh look at that, at the start it took you a while of dealing with catching the train or whatever for a final better result netting you around the less costs per year.

Now, this isn’t a perfect example and I’m sure it’ll make some people mad (enjoy that). But the way I see this whole NBN thing is:

Yes, it is going to take time.
Yes, it is going to take money.
But when it comes to 20 years down the track. There wont be any ‘Oh shit, the car just broke down time for an upgrade. Lets buy another shitty car that will only last 2 years.’ type moments.

The existing copper networks with a fiber to the node deployment is not enough for the future. The Coalition’s idiotic statement of “We can not predict the future” “why invest in something we don’t know is going to be needed?” is utter bull shit and a fucking waste of time. Fiber is the future. The world is limited by light speed. Fiber is light. It’s physics. You can’t go faster. It is the upper limit of computing speeds. That’s a fact. Maybe Tony thinks the laws of physics is going to change. Who knows?

The initial investment is expensive not only in time and money for a FTTH network. But it is the future. The is no doubting that. Fuck copper.