In my most recent blog, I wrote about how I feel that too much of the world's logic is coming onto the shoulders of PHP, these days. Today, I'm will be showing you why and how PHP's powers could be harnessed better and more. We, as PHP web developers, should be absolutely fully aware that we're allowing insane amounts of processing power to do too much work when it's absolutely unnecessary to do so dynamically.

Say what now?

Here's part of the problem: PHP developers usually don't really know the difference between what's static and what's not. Simply put: there is only one variable. The request. Anything else should be as static as possible. If I would introduce a configuration file with loads of variables, a typical C programmer would argue: "Those are build parameters. They don't change unless your environment changes, so they are static and should be handled by a preprocessor and macros. You don't need continuous evaluation of static data". A typical Java programmer would probably put stuff like that in a static final constant property somewhere, knowing (or assuming) that the compiler will optimize and inline usage of these settings ¹.

PHP doesn't do either. We have no preprocessor, and we have no compiler optimizations on that level. Mainly, because PHP originally was a platform that was supposed to do some simple processing and spew out some dynamic content, as fast as possible. But the PHP community is growing up. We want real OO programming, we use design patterns, we think about dependencies, maintainability, scalability. Stuff that grown-ups do.

But then, what the hell is wrong with this picture?!

I was having performance trouble with one of my Symfony projects at work. So I decided to do some performance testing. Please note that I absolutely love Symfony. There is no framework out there that will actually help you design your application better, and it brought me and many of my colleagues more joy in life. Really. Please understand that this isn't really about Symfony, because I am pretty sure most of it goes for any other current-generation framework.

I set up some benchmarking comparisons. This isn't scientific evidence, to be frank. This is experimental science. I was having a hunch, and this indicates that my hunch was correct. So here goes.

The example controller in the Symfony standard edition

Whenever you start a Symfony project, you would typically bootstrap from the standard edition, remove the demo code and add your own bundle. The example controller does something pretty simple. It responds to a URL /hello/..., where the dots may contain anything but a forward slash. Here's the code:

#!php
<?php

class DefaultController extends Controller
{
    /**
     * @Route("/hello/{name}")
     * @Template()
     */
    public function indexAction($name)
    {
        return array('name' => $name);
    }
}

The idea is simple. /hello/{name} routes to this controller, the controller returns an array of named variables, and the @Template annotation makes sure the template corresponding to the controller is rendered. This is usually a twig template, and in this case, the template looks like this:

Hello {{ name }}!

Not just to be a pain in the ass...

How would you have done this 15 years ago? Probably something like this. Create a PHP file in hello/index.php containing:

#!php
Hello <?= htmlspecialchars($_GET['name']) ?>!

and a .htaccess in the web root containing:

RewriteEngine On

RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule hello/([^/]+) hello/index.php?name=$1 [L]

Right? It doesn't really do anything more than that. And if I'm absolutely honest about it, useless as the example may be, it shouldn't be done any other way.

So, just for the sake of curiosity, lets see what a difference in performance this is. I set up two identically configured hosts on my local machine in Apache, but with different webroots, just to be as close as possible to a regular development environment I normally use. The first web root is the regular web/ directory from Symfony (and I renamed it to web-1). The second is another directory containing the setup as described above.

With 'ab' (Apache Benchmark), you can get a pretty good idea of performance increases if you're optimizing your website, one way or another. You can basically pass in an URL and have multiple concurrent users hammer that URL for a specified number of times. So that's what I did. I usually start with 1000 requests, with 20 concurrent users. This usually gives a good comparable impression of speed.

First off, the dev, debugging version of the symfony app. Of course, you wouldn't normally test performance in a development or debugging environment, but it's here for the sake of comparison. Here's the most relevant² results of 'ab'.

#!shell
Concurrency Level:      20                    
Time taken for tests:   25.058 seconds        
Complete requests:      1000                  
Requests per second:    39.91 [#/sec] (mean)  

Second, the prod, non-debugging version of the symfony app.

#!shell
Concurrency Level:      20
Time taken for tests:   5.302 seconds
Complete requests:      1000
Requests per second:    188.61 [#/sec] (mean)

That seems pretty decent. Just under 200 requests per second were managed. Let's increase concurrency to see what happens:

#!shell
Concurrency Level:      250
Time taken for tests:   7.153 seconds
Complete requests:      1000
Requests per second:    139.80 [#/sec] (mean)

A slight drop in performance was to be expected, but it is at least pretty steady and stable.

Now let see how the 1998 version of the same "application" would perform.

#!shell
Concurrency Level:      20
Time taken for tests:   0.178 seconds
Complete requests:      1000
Requests per second:    5614.76 [#/sec] (mean)

Wait. What? Over 5500 requests handled each second? That means a performance increase of 30. Not 30%, a FACTOR of 30. Nearly 30 times faster. The last time I saw numbers like these were when I ran benchmarks on a Varnish cache....

This would mean that if I were to run the same test with 30 times as much requests, the 1998 version should manage equally:

#!shell
Concurrency Level:      20
Time taken for tests:   4.482 seconds
Complete requests:      30000
Requests per second:    6693.93 [#/sec] (mean)

At these numbers, the measurements apparently show even better results! And if we would increase concurrency, it would at least have the same relative performance as the other one:

#!shell
Concurrency Level:      250
Time taken for tests:   6.366 seconds
Complete requests:      30000
Requests per second:    4712.83 [#/sec] (mean)

Let's look at this for a few seconds. Just to be sure we understand what this means. This means the stripped version of the functionality runs roughly a factor 30 faster than the one implemented using the framework, both with 20 and 250 concurrent users.

But if these numbers are possible, why then, why do we think all the framework's benefits outweigh these mind-boggling disadvantages?

The counter-arguments

You have no request listeners and therefore no security, no logging, etc, etc...

I know. And you are right. I don't. But none of those are needed in this example. And that's exactly what's wrong with our mindset. We load tons of utilities and tons of logic to requests of which in 80% of the cases, only 20% actually need, and we keep on saying that the ease of use and the well-thought design are good arguments to have such a horrendous performance impact.

Be honest. If you would buy a new server, would you rather have it installed bare bone without any software on it so you can carefully pick your stuff, or would you have the entire Ubuntu Software center installed, so you can use it whenever you can?

Sure, there is a middle road, but lets find a middle road that is the best, not the one that is the most convenient just for us developers.

But you can cache it!

There is only one right answer to such a suggestion. Caching sucks³. Yes, I know, caching is cool, because it can make stuff insanely fast (try Varnish one time. Unsavory goo will drip from your lips, I promise you). But that's not the point, really. Caching complicates stuff. Complications are just like expensive toys. 1) You don't really need them, 2) they make you worry about breaking things that shouldn't really matter and 3) you already have enough of them.

So we should install php3 again and get rid of the frameworks?

Balls, no! This is something the people behind the frameworks should be aware of. But at the very least, the people using them should be aware of it. Don't tell your project manager (or worse, your client) that the performance is okay, and that you just need a few days R&D to implement caching, just tell them the framework is relatively slow but that you think the extra cost in hardware and complexity of caching mechanisms outweigh the design and maintainability benefits. And be very careful to really mean it and know what you're talking about.

Code generation is paramount to performance!

I'm not the type of guy to point out problems and not think of solutions beforehand. I have been thinking about this for quite some time, and thought of building something myself. But the fact of the matter is: I don't have the time nor the persuasive character to actually get it done.

What would be needed is a build system that "folds out" into smaller pieces of somewhat repetitive PHP. We write the development code just like we did before, but when deploying to a non-development environment, all the bits and pieces should be as static as possible. Anything that is dynamic but could, in theory, be static, must be factored out. This means, in case of Symfony, that

• It is a bad practice (or even unsupported) to have dynamics inside your Bundle classes. Use the service container for this. This way, the bundles needn't be loaded nor initialized on each request;
• The service container and accompanying files should be loaded lazily. This can be achieved using require_once calls whenever necessary. The compiled code should do the same.
• Service container and compiled DI code (we have this already, but could even be more effective) compiles into separate files in stead of a big class that gets loaded everytime.
• Configuration must be compiled statically
• Templating must be optimized such that it generates the most efficient PHP code possible. Using magic like the detection of 'getters' and such should be removed. May be even consider it a bad practice to use objects in templates at all.
• Routing must be done statically. Dynamic routing is runtime logic and can be done inside the controllers.
• Request listener code is compiled into the controller files, so templating listeners, error handling and security are as efficient as possible.
• Resolution of bundle aliases is done at compile time
• Handling sub requests is done by simple includes

And on goes the list.

And now, for some action

I have built a little "proof of concept" that shows you the idea. I wrote the resulting code by hand, but with most of the logic already in place, a compiler with some optimization passes in the resulting AST could do anything that i did by hand automatically.

Here's the performance impact difference:

The 'hello {name}' example:

An example using doctrine

In the second example, (which is configured at route db/{name}), the entire service container is loaded, by including the appProdProjectContainer from the cache. In the last example, routed through db-optim/{name}, the doctrine service and it's dependencies are loaded by including files with their service definition. This adds another ~10% increase in performance, which is still worth considering, imho.

Note that the doctrine services only use per-request caching (ArrayCache), in this example.

What's the catch?

The catch is, most of the overhead from Symfony's kernel is from the request listeners. There are a whopping 15 (fifteen) services that listen to every request in Symfony. All of these listeners can be optimized into pieces of code which are included in the controller files, where the "listening logic" (i.e., checking if the listener should do anything) can be done by a static line of code. In the examples above, you can see this happening for the templating listener, which is only checking if the return value of the controller is an array. The fact that it's in place right there, can be controlled by the @Template annotation. The same would fly for @Secure and @Route annotations. Some listeners contain global static logic (such as the locale listener) or are generic, but nonetheless static (such as exception listeners.

With the Kernel structure as it is now, there is no way to have these listeners come into play only when needed. They are initialized every request, and therefore all dependencies that these listeners may have, are initialized. There is only one way to get around this. Compile all initialization into the controller files, but guarded within the conditions these listeners should provide. Just like the is_array() example for the logic of the Templating listener, this can be done for any other listener.

The sharp reader must have noticed that I disabled all listener logic in my example. This is the reason.

To conclude

I have posted before about a paradigm shift towards declarative programming. I am also arguing that this shift should include a better sense of balance between performance and maintainability. PHP's road to success was paved by performance. If we lose that, there is not much reason left not to choose other platforms.

A final disclaimer: I am not bashing. I hope that this will lead to more good in the PHP community. At the very least, I dropped my thoughts and can leave it simmering for now. If you read the post entirely, thanks for your patience.