Installation and configuration of the Magento Mage_Cache_Backend_File in versions prior EE1.13 / CE 1.8

The Latest Magento releases (community edition 1.8 apha and enterprise 1.13) come with a new cache backend, Mage_Cache_Backend_File.

This backend reduces inode usage when saving the cached content.

You are on a prior version but want to use it? No problem, but there are few things to fix before

Continue reading “Installation and configuration of the Magento Mage_Cache_Backend_File in versions prior EE1.13 / CE 1.8”

The Magento Mage_Cache_Backend_File backend gives some holidays for your inodes

Magento allows to customize the way your website manages its cache. This is done by choosing the appropriate cache backend. I’ve provided an example of this update for the full page cache here

By default, your cache will save cached content into files, due to a cache backend existing in both Magento community and enterprise editions, based upon the Zend_Cache_Backend_File, that saves cached content into files

Magento EE 1.13 comes with a new cache storage: Mage_Cache_Backend_File. This backend is an extension of the native cache backend Zend_Cache_File with some improvements regarding performances: Let’s see what is good inside and if we can use it yet

Continue reading “The Magento Mage_Cache_Backend_File backend gives some holidays for your inodes”

Support for cache tags for all cache engines in the Magento enterprise version 1.13

Did you follow the Magento news, you probably heard that Magento Enterprise version 1.13 completely moved the indexer’s logic to the database server.

This for sure leads to a significant increase in your performance.

But are the indexers the only explanation for those better performances? No: there is a few other things involved in this upgrade: one of these is the support of the cache tags for all cache backends.

Continue reading “Support for cache tags for all cache engines in the Magento enterprise version 1.13”

Enhance Magento enterprise full page cache performance by updating its storage backend

Magento enterprise supports since the 1.7 version the full page cache module: instead of storing only each block content in cache, this is the full page content that’s stored in cache

Everybody agrees this is a really performance enhancement

Like the native cache, storing these datas in a file is not the fastest way and becomes a problem when you need to share it between each front. How can we customize it?

The full_page_cache configuration node in the Magento enterprise.xml file

History of the full page cache configuration in the Magento enterprise versions

Until 1.11.0.0, there was no node to configure the full page cache option in the default Magento enterprise configuration files.

Since 1.11.0.0, you have seen a new node appearing in the app/etc/enterprise.xml file:

<config>
    <global> 
        <full_page_cache>
            <backend_options>
                <cache_dir>full_page_cache&</cache_dir>
           </backend_options>
        </full_page_cache>
   </global> 
</config>

This new node allows to customize the cache directory folder where the cache content will be located

Customize your full page cache configuration

Everywhere on the net you can find ways to enhance performance of the Magento cache storage engine by customizing the following configuration node

<config>
<global>
<cache>
...
</cache>
</global>
</config>

As with the config > global > cache node, the full page cache node works exactly: all nodes you’ll specify under will configure how full page cache works. For example, The following node defines the usage of memcached as cache storage engine for the full page cache content

<config>
<global>
<full_page_cache>
            <backend>memcached</backend>
            <slow_backend>database</slow_backend>
            <memcached>
                <servers>
                    <server>
                        <host><![CDATA[127.0.0.1]]></host>
                        <port><![CDATA[11211]]></port>
                        <persistent><![CDATA[0]]></persistent>
                        <weight><![CDATA[2]]></weight>
                        <timeout><![CDATA[10]]></timeout>
                        <retry_interval><![CDATA[10]]></retry_interval>
                        <status><![CDATA[1]]></status>
                    </server>
                </servers>
            </memcached>
        </full_page_cache>
</global>
</config>

Can the full_page_cache node be used within Magento enterprises version prior 1.11?

Yes. Even if this node was initialized in 1.11, all prior versions look for full page cache configuration in this node. So you can also customize your full page cache in all Magento enterprise versions which embeds full page cache module

Don’t forget to customize the full page cache management

As you can see, the two nodes config > global > cache and config > global > full_page_cache are strictly independent: you can provide two different ways to manage cache between Zend cache api and full page cache: we often see that there is cache backend configuration, but do it also for the full page cache

Make a clean sweep of commons Magento cache_lifetime workarounds usage

As I mentioned before, Magento cache policy API is based upon the Zend Framework Zend_Cache component. In this cache policy, there are three elements :

  • Cache key that defines the unity of the data we want to cache
  • Cache tags that allow to cluster our cache data
  • Cache lifetime that defines the duration within we consider that our stored data are valid and do not need to be rebuilt

My current topic will be about this latest point: cache lifetime. Its usage is quite simple: we define a duration while data will be stored in cache; during this period, data is always considered as valid, and so, not rebuilt. After that, even if data exists in cache, it will be rebuilt because considered as expired. Simple, no?

But we can hear so many things about cache lifetime usage that it seems interesting to evoke some specific points about its usage in Magento

What happens when Magento cache lifetime is specified as null?

When Magento cache lifetime is considered as null?

Excluding case you have defined a getCacheLifetime method which returns null value, this case occurs when you do not define the cache lifetime on one object which extends Varien_Object:

  • When rendering block content, Magento will call the getCachelifetime block method
  • If you have not defined a getCacheLifetime method, Magento will use Varien_Object __call method to retrieve cache_lifetime from attribute _data
  • If this index does not exists in _data, __call will return null

How is managed cache lifetime null value by Magento for block data?

All blocks extends Mage_Core_Block_Abstract. This class provides the following cache management:

As you can see, if lifetime is null the Magento Cache API will never fetch something from cache, so rebuild each time the block content. But also nothing will be stored in the cache(_saveCache method check if also there is a non null lifetime before save)

If you do not set up a cache lifetime for a block content, it will never be cached

How is managed cache lifetime null value by Magento blocks for non block data?

The cache API can also be used for other things than block output. This is the case for configuration cache. In this case, the control made by Mage_Core_Block_Abstract does not exist, and data will be cached: we’ll see that if cachelifetime duration is null for something else than block, it will use the default lifetime value like id you set up false as a cache lifetime value: 7200sec (2 hours)

What happens when cache lifetime is set up to 0 ?

Sometimes, we can hear that setting 0 as cache_lifetime duration disable cache. No, and this is for the following reason: 0 is a valid cache_lifetime duration.

Magento will define the timestamp until you consider your block content as valid with the following rule: current timestamp of data generation + cache_lifetime; so calculated expiration time will be the generation time. For sure, next time you’ll check if data is in cache and valid, you’ll compare new timestamp with expired timestamp, and there is a great probability that your current timestamp will be later.

Setting 0 as cache lifetime duration render your cached block content always expired

This is one of the worst case, because, each time you build content, you save it in cache, and invalidate it next time you’ll check it

What happens when cache lifetime is set up to false ?

The Zend Framework lifetime management for false values

Every cached backend used in Magento inherits from the Zend_Cache_Backend class

When saving content, Zend_Cache_Backend class will calculate the expiration timestamp trough the Zend_Cache_Backend::getLifetime method. If specified lifetime === false, backend use a default duration stored in its attributes

If you check Zend_Cache_Backend class, you’ll check that this value is hard_coded as 3600 sec (one hour)

But our content is stored longer, why?

Magento forces a default lifetime in its frontend

When the Magento cache API initializes the cache frontend object in the Varien_Cache_Core model, it merges cache options with directives options

This is the case for the cache_lifetime duration: default cache lifetime duration specified on Varien_Cache_Core is merged with directive options

But Varien_Cache_Core class extends Zend_Cache_Core. So when backend will be set up to frontend, we’ll use the native Zend Framework behaviour and so, merge frontend options as a backend directives.

So for all of our backend usage, because Magento forces usage of a Varien_Cache_Core frontend, default lifetime will be set up as 7200s

If your lifetime is set up to false, by default your block will be considered as valid during 7200s

Init Magento cache block policy in _construct method… or not

We often see the Magento cache block policy initialization in the _construct method, like in this example:

class Namespace_Module_Block_Type extends Mage_Core_Block_Template
{
     protected function _construct() {
          $category = Mage::registry('current_category');

          $this->addData(array(
               'cache_lifetime' => 86400,
               'cache_tags'     => array(Mage_Catalog_Model_Category::CACHE_TAG."_".$category->getId()),
               'cache_key'      => $category->getId(),
          ));
     }
     ....

This is not a quit good example of cache block policy initialization, for the following reasons

Init cache block policy in _construct adds unnecessary treatment

The problem is due to the way of how Magento loads the page structure and how it will render blocks content loaded from cache:

When you load the layout, Magento will parse and build all defined elements in layouts files. This means instantiate all defined blocks. After, Magento will check if your block content is already in cache, and will use it if yes, or build it if not.

But instantiating one block will call the _construct method. This means even if your block content is in cache, your block will execute each time all instructions within _construct method.

Do you require to rebuild cache tags list each time?

The answer is no. The cache_tags list is required only when saving generated content into cache. So if you initialize the cache tags list in _construct, it would have an interest only if your block content doesn’t come from your cache

Init Magento cache block policy in _construct let you avoid to use layout properties

Another problem is due to accessible data in _construct method: when calling _construct, block initialization is not finished: Magento will populate data, set name in layout, reference current layout in block and so all these informations won’t yet be available.

so layout informations are not yet available, and you restrict cache block policy only to data managed in block

Possible solutions to avoid these problems?

Specifying a cache block policy is not an option for performances reasons. But defining it in _construct method can reduce performances benefits if you continue to load some data within. A possible solution to bypass these problems is to define on your block the methods getCacheLifetime, getCacheKey and getCacheTags: treatments within will be executed only when they’ll be called

Zend_Cache component usage in Magento

Magento performances is a hot topic subject. One element of the Magento performance is the cache block policy which allows to store in cache part of the page content: You probably have all read the tutorial available through the Magento wiki, written by my old colleague Laurent BOURREL, about the cache block policy available in Magento.

All concepts of this policy are based upon Zend_Cache component. Let’s see how Magento use it

Reminder about Zend_Cache component

Zend cache module structure

Zend_Cache component is divided into two parts, frontends and backends models:

  • Frontend components define what is covered by cache policy:
    • Files
    • Output rendering
    • Method call
    • Class
  • Backend components define where will be stored cached data:
    • In file
    • In memcached
    • In apc
    • ….

This model allows to easily dissociate which data will be cached, and where they will be cached

Zend_Cache usage in Magento

Embedded Zend_Cache backend in Magento

If we check Zend framework version from CE 1.3.2.4 to CE 1.7.0.0 (and related Entreprise versions), we should be able to use the following backend within Magento:

CoreApcBlackholeFileLibmemcachedMemcachedSqliteStaticTestTwolevelsXcacheZend_PlateformZend_Server diskZend_Server shared memory
CE 1.3.2.41.7.2
CE 1.3.3.01.7.2
CE 1.4.0.11.9.6
CE 1.4.1.01.9.6
CE 1.4.1.11.9.6
CE 1.4.2.01.10.8
CE 1.5.0.11.11.1
CE 1.5.1.01.11.1
CE 1.6.0.11.11.1
CE 1.6.1.01.11.1
CE 1.6.2.01.11.1
CE 1.7.0.01.11.1
EE 1.6.0.01.9.3PL1
EE 1.7.0.01.9.6
EE 1.8.0.01.9.6
EE 1.9.0.01.10.5
EE 1.9.1.11.10.8
EE 1.10.0.11.11.1
EE 1.10.0.21.11.1
EE 1.11.1.11.11.1
Loaded from Zend FrameworkAdded or updated by Magento sources, but available

How does Magento use Zend_Cache?: the factory Mage_Core_Model_Cache

Before CE 1.4, Cache factory was made by the Mage_Core_Model_App model

Since Magento CE 1.4, all cache management is done through the Mage_Core_Model_Cache class; this class is a factory model that will load the Zend_Cache frontend and backends

Magento Zend_Cache frontend

In Mage_Core_Model_Cache, Frontend class is hard coded and restricted to Varien_Cache_Core. Varien_Cache_Core inherits from the frontend class Zend_Cache_Core.

So we are unable to use natively other frontends than Zend_Cache_Core

Magento Zend_Cache backend

Why is there some Backend available in Magento which does not exist in Zend_Cache component?

But this factory method also provides the ability to use some other backends storage that does not exist in Zend_Cache; see the following examples :

  • Database
  • Eaccelerator

These backends have been written by Varien core team

In conclusion, the Zend_Cache backend defined by Magento are the following ones
CoreApcBlackholeFileLibmemcachedMemcachedSqliteStaticTestTwolevelsXcacheZend_PlateformZend_Server diskZend_Server shared memoryEacceleratorDatabase
CE 1.3.2.41.7.2
CE 1.3.3.01.7.2
CE 1.4.0.11.9.6
CE 1.4.1.01.9.6
CE 1.4.1.11.9.6
CE 1.4.2.01.10.8
CE 1.5.0.11.11.1
CE 1.5.1.01.11.1
CE 1.6.0.11.11.1
CE 1.6.1.01.11.1
CE 1.6.2.01.11.1
CE 1.7.0.01.11.1
EE 1.6.0.01.9.3PL1
EE 1.7.0.01.9.6
EE 1.8.0.01.9.6
EE 1.9.0.01.10.5
EE 1.9.1.11.10.8
EE 1.10.0.11.11.1
EE 1.10.0.21.11.1
EE 1.11.1.11.11.1
Loaded from Zend FrameworkAdded or updated by Magento sources, but available

So we will update the available zend_cache backend list according to Magento versions:

And so, what If I want to use a backend that has not been thought of by Varien?

Perhaps you can want to use blackhole, test, or zend_plateform backend for exemple. Is it really difficult? no

Since Mage_Core_Model_Cache model exists (CE 1.4.0.1 and higher, related EE) the factory method has been also thought so that perhaps you want to use another backend :

The value defined in the local.xml node <config><global><cache><backend> can be a class name that implements the Zend_Cache_Backend_Interface.

So, and for my backend configuration? To specify some parameters to this new backend one, you can define a node <config><global><cache><backend_options> which will be passed as arguments when constructing the backend model

Other possibilities to use a custom backend model are the following ones:

  • Define your own backend class that will set up the expected configurations values
  • Overload Mage_Core_Model_Cache to be able to provide other configurations loading

Here’s an example of specific configuration you can have and which works

Conclusion about Zend_Cache usage in Magento

cache tags, cache lifetime and cache key are concepts brought by the zend_cache component

Even if there is no documentation about how to use some custom backend in Magento, it’s not really difficult: just specify the full backend class name in local.xml backend node, and specify arguments through the backend_options node

The only problem I can see with this method that is able to load any backend class is the method used to check if backend exists: it uses class_exists method, which, In Magento context, throws a fatal error if the class cannot be found. It could be interesting, if we provide a backend class which does not exist, that the default one (file) would be used instead of throwing a fatal exception

Updating zend_cache frontend class is quite difficult because it depends on how PHP code was written. So for now, even in Magento 4, 5, I’m not sure that we can use the buffer frontend