Juozas devBlog

During last two months I spent massive amount of time tweaking Doctrine ORM framework and making it to perform as fast as possible (as you might have noticed from my never ending tweets). This post is devoted to performance and efficiency, with practical tips & tricks how to reduce memory usage, make it work faster and save resources.

Doctrine is a very powerful framework, however you should study its behavior a little bit to get it working properly. As it turns out – it’s not that hard.

Speed

One of the first things I recommend looking at is query cache. Because there is quite a lot happening in turning DQL statement into the SQL query, caching that process can increase performance by a big margin. Best choice – APC, although robo47.net has an example code for use of Zend_Cache adapters, just don’t forget that file back-end is probably not the best pick.

Hydrators are probably the easiest thing to misuse. Hydration in Doctrine language is turning SQL query results into data graph. It can be an array, object or single value. It’s very nice to get results as PHP objects (in this case – models), however it’s slow. Slower than hydrating like array, and much slower than getting raw result from PDO. That’s why I always recommend answering one simple question: “will you be updating/deleting records?”. If the answer is no – hydrate as array, because you are not going to need an actual model (usually).

For this reason I always try to use array notation to access properties rather than like-object-vars one. Basically instead of $product->name I tend to write $product['name'], which returns the same result, but makes switching to array hydration very easy. You can find more tips and tricks at Doctrine manual here, but the key moment is to remember that the faster data structure in PHP is array (I believe so), so if application is not performing well – start playing with hydrations first.

Doctrine has a very nice support for relations, where they work like proxies and data can be retrieved when it’s needed (lazy-loading). However, this is wrong:

$user = Doctrine_Core::getTable('User')->find(1);
 
foreach ($user->Comments as $comment)
{
   print $comment->NewsItem->title . '<br />';
}

This code is bad, because for each comment you will load news item using a separate query, hence you are wasting db server resources and making code slower. Optimization is pretty straightforward:

$query = Doctrine_Query::create();
 
$query->from('Comments C')
        ->innerJoin('C.NewsItem N')
        ->where('C.user_id = ?');
 
foreach ($query->execute(array(1)) as $comment)
{
   print $comment->NewsItem->title . '<br />';
}

Here all the data is loaded in one query and everything happens much faster (and in this case making it to hydrate as array would also improve the performance).

However, make sure you know what you are joining. For example imagine that in previous example news item is also one-to-many (comment has many news items) relation and user 1 has written 1000 comments where each comment is attached to average news items. Query above will return 50′000 records (50 * 1000) which Doctrine will need to hydrate then. This will be very slow and probably going to kill your web server after some time. One day I had a query which was returning 2GB of data, server admins where probably not very happy about it…

Memory

One of my favorite parts of software development is playing with memory usage and making it efficient. Even though it sounds really simple, debugging it and finding it where the memory is leaking is not an easy task. Recently I was using Doctrine to work with quite big datasets (on average 50′000 of records) and probably have tried all the possible tricks to make Doctrine memory efficient. My code looked really simple:

$data = Doctrine_Query::create()->execute();
 
foreach ($data as $item)
{
    // do some work here
}

You might expect that memory usage would be steady, however it is not. Doctrine uses identity map and objects also have a lot of references which makes freeing up memory a tricky job. As my experience showed, even though records have a method free() which is supposed to de-reference it, sometimes it doesn’t help.

So to make memory management work, make sure to try these:

• $record->free(true) – deep free-up, calls free() on all relations too
• $collection->free() – free all collection references
• Doctrine_Manager::connection()->clean() – cleanup connection (and remove identity map entries)

With some debugging and profiling (and these methods) you hopefully can make memory usage to be low. I’m also using some custom iterators (available here) to divide query into chunks, because loading 50′000 of objects in one go is not going to work, so you might want to look at it too.

Another recommended tip: make sure to free queries too. As collections hold references to all records, query object also has some references to parsed sub-parts of query. For this I use auto-free setting enabled by (available in my first part post also here):

// enable automatic queries resource freeing
$manager->setAttribute(
	Doctrine_Core::ATTR_AUTO_FREE_QUERY_OBJECTS,
	true
);

It’s very easy to forget to free a query and it’s again creating these references which makes garbage collector’s work hard. Nevertheless, after enabling this one I don’t know any other place where the memory can start to leak.

Last step – PHP 5.3. I’ve been working with this version for a few months and it works great, so if it’s possible – I recommend using it (you will also help with testing frameworks, but both Zend Framework and Doctrine already should work fine). One important function here is improved garbage collector, so the actual script takes less memory to execute. I haven’t recorded benchmarks on this one, but what I’ve noticed during development is that 5.3 does in fact has a lower peak memory usage.

Conclusion

I think I haven’t missed anything here, or at least these are the tips which made applications work fast (if I have – let me know). To finish with – good optimizations are done by comprehensive benchmarking and profiling, so the fact that Doctrine is a big framework doesn’t necessary mean it’s slow too. At the end of the day, it’s usually only a matter of reading a manual.

All parts:

1. Zend Framework and Doctrine. Part 1
2. Zend Framework and Doctrine. Part 2
3. Zend Framework and Doctrine. Part 3