High traffic websites under Magento: the Teleshopping example

Téléshopping, one of the most older Magento project in France. With 41Meuros, website represents a tier of the turnover. Let’s see which technical choices has been made to provide a high performance website

Hello Gilles, maybe everyone doesn’t know you yet, could you please introduce yourself?

I’m in charge of the production center at Teleshopping. I am in charge of the operation, administration and system architectures for ERP and websites and for its subsidiaries (Euroshopping, Place des tendances and partly Direct Optic).

Two of your websites, Teleshopping and Euroshopping, are Magento-developed projects. Could you please tell us more about them?

Sure, Teleshopping’s website is among the first french Magento-developed websites, deployed in April 2009, and regularly updated since. There are approximately 1M unique visitors per month.

It allows its users to order products they have seen during the TF1 TV show. Ordering products from the catalog is done through another website.

Euroshopping’s website was deployed a little later, and there are about 100 000 unique visitors per month. It allows people to order products they have seen during TV shows on different TNT channels or on the cable.

Could you describe the general architecture within these two websites?

For Teleshopping, we have 10 front-end servers available for visitors, and one more for the back-office access

Proxy cache varnish is ahead each one of the servers. This may not be the best solution, but it allows us to regulate the number of front-ends without changing the configuration or having to ask ourselves too many questions.

Concerning the application, the source code is installed on every front-end server, there’s no replication and deliveries are hand-made.

But we share the media and skin directories on an NFS resource.

We use a memcached server with 1 port to put data in cache, and another port to put sessions in cache; this way we can flush data without any impact on the user’s sessions.

Concerning the database, we use a MySQL server which is replicated on a secondary server. All accesses are done on the main server.

Last element, the CDN. Medias are served by a CDN, which allows us to relieve at the outside our front-ends, even during the business campaigns.

Source images are put on an NFS resource available on every front-end.

For Euroshopping it’s at the most the same, except there’s no varnish, the 4 servers available are more than enough to manage the load.

How often do you deploy new versions?

It’s an average, but we do about 1 delivery in the production environment per month. Sometimes we may have more deliveries than that.

Could you tell us about your last deployment?

Every developer has its own development environment on a virtualized server, which is strictly the same as the production one.

A testing environment is available to validate the package before it’s installed on a pre-production environment. This delivery is done by my team.

Once the project manager has tested and approuved this package, it is delivered in the production environment, my team still doing it.

You are sharing the front-ends server’s contents on a NAS. What content do you exactly share?

We share the media and skin directories on our NAS, a filer NetApp. This sharing system is almost as performing as the local disk access, and its vantage is that we do not have to duplicate media files.

What made you choose this setup?

This sharing process is almost as performing as the local disk access, and its vantage is that we do not have to duplicate media files.

NFS and PHP tuning allows to manage access to the shared files with different cache levels.

We can manage the stock volume without any outage.

Could you tell us about the main problems you encountered while you were setting up this model?

We didn’t have any particular problem, only some tuning was needed for the NFS:

  • We mount the NFS shared folder with the following options tcp, rw, bg, hard, suid, noatime, nodiratime, nointr, timeo=600, retrans=2, rsize=32768, wsize=32768, actimeo=30
  • We have increased the PHP realpath cache to have a size of 1M and a TTL of 1 day

What benefits do you get from the source sharing between the different front-ends on a NAS?

Source sharing is useful to deliver the source code only once, it is then available for all of the front-ends; we do not need a “master” server from who to replicate the files.

Presently, the websites’s sources don’t allow us to share the whole source code, partly because of the /var/reports and /var/logs directories. For the next versions if source code allows it, we intend to share the whole source code for every front-end server.

How to schedule Magento sales rule correctly?

Magento provides a sales rule engine which allows you to offer discount to your clients: free shipping, buy x products and get y products free, providing a discount from a voucher are samples usage of prices rules

Two important concepts exist in this model:

  • Can we cluster sale rules: can rules be combined or if a rule is applied, are the others automatically excluded ?
  • Sorting rules importance within the priority concept

How does Magento apply prices rules on cart ?

Each time you update your cart content or each time you display the cart page, Magento loads all the active sales rules for the current date, current website, current user group and test for-each rule if the conditions match the cart content. If yes, Magento applies specified discounts to items in cart.

If a rule has been flagged as “stop rule processing”, we stop iteration on the rule collection. If not, we continue and apply the others matching discount until we have tested all rules or we have applied a discount which stop iteration

The Magento sales rule priority

Priority is a numeric id defining the rule importance and how it’ll be tested: the less is the priority value, the more important is the rule

Priority is specified when the sales rule is defined from back-office

Priority is not a mandatory field: if you do not specify a value, the priority applied will be 0, and so the rule will be tested at first before all other sale rules

Technically, when the rules collection is loaded, it’ll be sorted by the priority value (sort_order field)

Priority is not unique: you can define several rules having the same priority

What happens if some Magento sale rules have the same priority?

There is no second filter to order rules: so if you have two rules which can be applied, this is the oldest one which will be used and applied firstly. Is it really what you were expecting? Perhaps not.

Furthermore, if you have many sale rules, you should ensure to know exactly each conditions criteria to understand exactly how are applied rules and in which order. Possibly very difficult in a long time or if you have many sale rules defined

How can you reduce the risk that some rules are not executed as you want?

Reviewing the rules to ensure that priority is unique is the only one solution to avoid risk that rules are not executed the way you are expecting them. But it could be difficult to maintain if you define later others sales rules.

If you want to reorder priority, you can define priority levels: the most important one have a priority defined with one digit, lesser priority with two digits, oldest one or more generic sale rules with a priority of three digits. In this case, you’ll ensure that you’ll know with an unique priority how they’ll be tested, and be able to add others rules easily

One other best practice is to defind strong application conditions criteria: the more generic they are, the more risk you have to apply a rule that should not be executed

Enhance Magento enterprise full page cache performance by updating its storage backend

Magento enterprise supports since the 1.7 version the full page cache module: instead of storing only each block content in cache, this is the full page content that’s stored in cache

Everybody agrees this is a really performance enhancement

Like the native cache, storing these datas in a file is not the fastest way and becomes a problem when you need to share it between each front. How can we customize it?

The full_page_cache configuration node in the Magento enterprise.xml file

History of the full page cache configuration in the Magento enterprise versions

Until 1.11.0.0, there was no node to configure the full page cache option in the default Magento enterprise configuration files.

Since 1.11.0.0, you have seen a new node appearing in the app/etc/enterprise.xml file:

<config>
    <global> 
        <full_page_cache>
            <backend_options>
                <cache_dir>full_page_cache&</cache_dir>
           </backend_options>
        </full_page_cache>
   </global> 
</config>

This new node allows to customize the cache directory folder where the cache content will be located

Customize your full page cache configuration

Everywhere on the net you can find ways to enhance performance of the Magento cache storage engine by customizing the following configuration node

<config>
<global>
<cache>
...
</cache>
</global>
</config>

As with the config > global > cache node, the full page cache node works exactly: all nodes you’ll specify under will configure how full page cache works. For example, The following node defines the usage of memcached as cache storage engine for the full page cache content

<config>
<global>
<full_page_cache>
            <backend>memcached</backend>
            <slow_backend>database</slow_backend>
            <memcached>
                <servers>
                    <server>
                        <host><![CDATA[127.0.0.1]]></host>
                        <port><![CDATA[11211]]></port>
                        <persistent><![CDATA[0]]></persistent>
                        <weight><![CDATA[2]]></weight>
                        <timeout><![CDATA[10]]></timeout>
                        <retry_interval><![CDATA[10]]></retry_interval>
                        <status><![CDATA[1]]></status>
                    </server>
                </servers>
            </memcached>
        </full_page_cache>
</global>
</config>

Can the full_page_cache node be used within Magento enterprises version prior 1.11?

Yes. Even if this node was initialized in 1.11, all prior versions look for full page cache configuration in this node. So you can also customize your full page cache in all Magento enterprise versions which embeds full page cache module

Don’t forget to customize the full page cache management

As you can see, the two nodes config > global > cache and config > global > full_page_cache are strictly independent: you can provide two different ways to manage cache between Zend cache api and full page cache: we often see that there is cache backend configuration, but do it also for the full page cache