Magento layouts: the limit of reference syntax

If you ever tried to update the Magento page content, you probably used the reference node: this instruction tells that every node under the reference node is applied to the block having the name used in the reference. Here’s an example:

<reference name="head">
<action method="addJs"><script>prototype/prototype.js</script></action>
</reference>

The example above will perform the addJs action on the block with the attribute name head.

This syntax is very useful when, in your custom layouts files, you want to refer to another page content block without updating the layout file which contains the referenced block. This instruction is very useful when you create your own modules and want to only customize Magento page

But sometimes, it does not work.

Why the Magento reference node’s content is sometimes not interpreted?

To understand when this case happens, take a look at the following two modules

Test_FirstModule and Test_SecondModule; Each of these modules embeds its own layout update. The layout of the second module defines a block in the content node called test_secondmodule_content

<?xml version="1.0" encoding="UTF-8">
<layout version="1.0">
    <default>
        <reference name="content">
            <block type="test_secondmodule/content" name="test_secondmodule_content" template="test/secondmodule/content.phtml" />
        </reference>
    </default>
</layout>

The first module refers to this block to provide a child block

<?xml version="1.0" encoding="UTF-8">
<layout version="1.0">
    <default>
        <reference name="test_secondmodule_content">
            <block type="test_firstmodule/content_child" name="test_secondmodule_content_child" template="test/firstmodule/content.phtml" />
        </reference>
    </default>
</layout>

In this case block “test_secondmodule_content_child” will never be available: when Magento builds the blocks list composing the page content, it iterates all defined nodes in the appropriate handles within the Mage_Core_Model_Layout::generateBlock method. But merged layout are parsed in the order they have been added in the configuration, so by default by their name. So in our case, firstmodule.xml will be handled before secondmodule.xml. And this is the problem: Reference expects that referenced block already exists. In our example, it won’t be the case because it will be processed after; so the child block won’t make any render

If you call a reference prior the block, all the references content won’t be processed

The only way to use reference is when they have already be defined

Possible solutions to bypass the reference layout limitation

Play with names of your modules and your database layout updates

You can try to manage order by updating the name of your modules and name of the layout files, but this solution depends on the installed modules: there’s still a risk that your layout will be handled before the expected one.

Use the Magento database layout updates

The database layout updates are always managed after the file’s ones: so if you define your references in database layout updates, they should always refer to a valid defined block. But it’s a more complicated way to develop your module.

Remove reference instructions from the layout

This is a cleaner way: before generating blocks, you’ll move all the references nodes content into the appropriate block node: no more references in your layout, no more risk that there is something wrong because they have been loaded in the wrong order.

The Kyp_LayoutFix module

I’ve created a module, Kyp_LayoutFix, that does that very thing: before generating block content, it removes all references instructions from the used layout

This module has been tested on all Community Editions from 1.4.1 to 1.7.0.2 and on the related Enterprises versions and is a sure way to avoid this references problem

This module is provided under the Open Software License

Download the Kyp_LayoutFix module

This module can be downloaded from my pear channel:

pear channel-discover channel.kyp.fr
pear install kyp/kyp_layoutfix

Or you can download it in a tar.gz file here

Let me know if you have any issue with this module.

Update Magento page structure by overloading the layout content

There are many ways to update Magento page structure content: all lead to the same thing, a custom page, but not in the same ways. Each method has its own advantages and drawbacks.

Within this topic we’ll overview overloading page structure by overloading native Magento layout files

How to overload Magento page structure using loading mechanism ?

All this process is based upon template loading mechanism which I explained before:

  • You define your design through back-office
  • You copy the Magento layout file to update in the layout folder of your custom design folder

For example, if you want to overload Magento customer account page content, you can do it by copying the original customer.xml into your custompackage layout folder

When Magento will be looking for the customer.xml file it will find yours before the default one: this is yours which will be used instead of the native one

Very easy and quick method to update Magento layout…

Advantages of updating Magento page content by overloading native layouts

You are totally independent of the native page structure

With this method, you are totally independent of the existing page structure: you can do what you want inside

Quick way to update page structure content

This is also the quickest way to update page structure: you’ve found which layout file contains the element you want to update, you just have to copy it and update it. No layout update definition, no new module to load to embed some update

Code more clear

Almost everybody knows now which files describe Magento page structure. Using this method makes the code reading easier: no many layout update to find which are involved in a page, this is our layout copy which is involved in your page. There are also less treatments to build your page because there are less files to parse to build your page

Drawback of updating Magento page content by overloading native layouts

Difficult to capitalize

Your new layout page structure depends on the design applied. Even if you can add files of some specific packages in a Magento connect package, it’ll only work if the Magento has configured the same design: force users to use a specific value of design is a difficult choice and will make all other modules installation harder

Difficult to maintain

Creating a fork makes it more difficult to take advantage of developments that may contain Magento updates, and generally the community.

Do you use this kind of method to overload Magento page structure?

Don’t forget app/design/adminhtml/base/default folder when adding customs elements to Magento back-office

From the beginning of Magento until today (eg Community edition 1.7 and Enterprise Edition 1.12), Magento admin design folder has always been located in the app/design/adminhtml/default/default folder.

This folder contains all the required files to build adminhtml pages:

  • layouts to define adminhtml pages structure
  • templates used by theses layouts

So, If you need to provide specific back-office interfaces, where will you put them? in this app/design/adminhtml/default/default folder?

Magento page structure loading file API

Adminhtml is managed like the same loading API used in the front office; this API has been reviewed in Magento CE 14 and related Enterprises editions to load content in the following order:

  • Custom package / Custom theme folder
  • Custom package / default theme folder
  • Base package / default theme folder

I have previously explained the aim of this rule and what you can do with it

But the question is: adminhtml default default folder is: base package or custom package?

app/design/admintml/base/default as a default package folder

Yes it is. The folder app/design/admintml/default/default is not a default one, but the the default theme for the adminhtml default package; folder app/design/admintml/base/default can also exists even if it’s not embedded with magento sources.

So for sure, this is not an error to enhance this default package, but it does not respect the role of this loading api.

So it could be interesting to use app/design/adminhtml/base/default as folder for your new back-office interfaces

Conclusion

app/design/adminhtml/base/default folder does not exist in Magento sources but you can create it as you wish.
In a dream world, back-office updates should be managed as this:

  • Move app/design/adminhtml/default/default folder to app/design/adminhtml/base/default folder: we ensure this way that we have the default magento design for back-office
  • Use a custom package for the back-office updates of the existing interfaces
  • Provide new back-office interfaces in the adminhtml/base/default folder

Do you have already created the app/design/adminhtml/base/default folder for your own back-office interfaces or you do not take care of this case?

Some best practices to manage templates with the magento template loading API

I presented you a few days ago an introduction on how are built the Magento pages. Now we we’ll see how the templates files are loaded since Magento 1.4 with the template loading API and how we can use this loading API to reduce our work with Magento.

Reminder: how are defined the templates path in a layout file?

Templates are defined in the layout files in a couple block / template with a relative path like this:


<block type="module_identifier/class_suffix" template="relative/path/to/template/file/in/the/template/design/subfolder.phtml"/>

For example, here’s the definition of the block / template that defines the search form field available in your catalog pages:

So, how does Magento find the full path for the relative path defined in this structure?

Magento template loading API available since Magento community edition 1.4 and above (Enterprise 1.7 and above)

Since Magento 1.4.x, when a page is requested, Magento analyzes all couples block / template that compose it. Afterthat, Magento will look for the full path of the template file in the following folders:

  • 1. app/design/%context%/%design%/%theme%/template
  • 2. app/design/%context%/%design%/default/template
  • 3. app/design/%context%/base/default/template

Magento takes the first file found and use it.

What happens if the file is not found? By default, if template file is not found in these folders:

  • No output is renderering
  • A trace in log will be written if log is enabled

This loading API explains why we can define easily some temporary themes for your site like for christmas, summer, sales…: after setting up a new theme, you must only update some elements in your template in a new theme template folder

This is this loading API which allow Magento to drive some merchandising design on your website

We will now see how we can work more efficiantly whis this loading template API

Some best practices when building a new Magento design and theme

Copy paste the default theme when building your package?

When looking for the template file in the folders mentionned before, Magento runs a file_exists. Even if copy paste is the devil, when you build your package, duplicate content will avoid many file_exists when rendering your page, so it could be interesting to duplicate the base/default theme

But one of the drawback of duplicate base/default content will appear during magento upgrade: you won’t benefit of the new templates design for templates you have not overloaded: after upgrading the magento source code: do you really want to check if all template have changed? Or keep your own one and do not use the new functionnalities?

Today, I haven’t seen any limit of the file_exists loading. There are many others points to check before this performance option to make magento faster…

New functionnality in Magento: in base/default !!!!

Another common workaround in Magento project is that we can hear that it’s strictly forbidden to write in the in base/default template folder. I never understood why: if you run in multiple stores model and have specific packages defined for each store, this is the only way you have to share templates between each store. In case we do not want to write in base/default, the only way that we have to share template is to copy paste it in each package/default theme… wonderfull 🙂

Furthermore, if you want to share your module in Magento connect, this is the only way you have to avoid requiring a post configuration by copying the embedded template files in the current package folder (modules should be installed through only one click)

How organize your new templates file in your theme?

When we define a new couple block / template, we provide a relative path to define the related template to load. As a best practice, I cluster all my templates in a subfolder like %namespace%/%module%

This best practice has the following advantages:

  • When you develop your module, this is quicker to find it in the directory tree
  • If you share your module in Magento connect, this reduce the risk that other modules will have already existing templates in directory tree

Be lazy when you work with Magento templates!

I’ve made hosting for one of the biggest french bank and I liked this work: check everything is ok or be able to solve problem quickly and easily with the tools you made. Magento loading API is able to help you to provide you quick answer and easily manageable way to customize the appareance of a Magento website

Sometimes I hear that for performance issues, some people overload the Magento template loading API. It’s a shame to avoid using this model because your client won’t be able to drive the website marketing design: model is quite fine and allows you to do what you want easily without too much work. There are really many thing to do before updating this API for performance issues

Introduction to the Magento page structure: layout, block, templates, themes and packages

If you are developer or web designer, you have been face to face with the Magento page structure. This is not difficult to update design, but you must know how the Magento pages are built

Introduction

Before describing the Magento page structure, I’ll explain what is a package and a theme, and how they are linked with Magento file system, and what is their interest

Definition: the Package and theme concepts

With a fresh install of Magento, all pages are ready to use. But I suppose that you want to customize the design to provide your own design, and packages and themes will help us

Package

A package defines the global structure of your website: how they are structured: where is located your footer, …

Even if Magento comes with a default one, you should define your own package

The base package embedded in a native Magento is called (for frontend), base

Theme

A theme is a variant of your package: in december you would define a design based on christmas, or for the february 14th your marketing service would perhaps make a special version for the valentine’s day: your website will keep its structure, but some elements will be updated to provide a specific but temporary design: these updates will be made with a new theme of your website

Magento come with a default theme called default

Configure your frontend design: define the package and the theme

Frontend design is configured through the administrative admin panel available through the menu

System > Configuration > Design


Location context: different page structures for front-office, back-office and installation process

As you can see, page structure is different between front-office and back-office: Package and theme are dependant on one of the following context:

  • frontend: for all front-office available pages
  • adminhtml: for the administrative panel pages
  • install: for the pages displayed during the installation process

According to the context, it will be able to find the required elements which define your website structure

Magento design and directory tree

Design and themes are strongly linked with the package and theme specified and configuration values will help Magento to find how to load your page structure and content:

Page structure are defined in the folder : app/design/%context%/%package%/%theme%

For exemple, the folder app/design/adminhtml/base/default define the folder where to look for the files of the default pages embedded in the Magento back-office

Is package and theme folder case senstive? Yes. So be sure in unix locations to use (by convention) the lowercase

Magento page structure

Now you know where Magento will look for the required files to define how the page is built. We will see now what comprises a Magento page

Magento page structure content: the couple block / template

Homepage content on a Magento with a sample data package: some container can be identified
Category page content on a Magento with a sample data package: the same container can also be identified

As you can see on a default page structure, some elements can be found in many pages: this is each time the same “content”, the couple block / template: Each unitary element defined in layout is a couple block / template: these elements are part of the MVC (Model View Controller) “design pattern” embedded in Magento.

Each couple block/template define the unitary elements of your layout: the breadcrumb, the code pool, the product sheet price display, have specific rules managements to work, and are relatively independant of the others ones: do you need to know how code pool works to display the breadcrumb? does the footer has specific rules related to product sheet displayed? no; they are independant and do not need to know themselves

Templates manage only differents marketing output (christmas, summer, sales, ...) and technical output (html, xhtml, mobile, etc)

The block in the couple block/template: the logic

Block are PHP classes which define all the management rules for the display embedded in the couple block/template.

A block should not build output string: it must only prepare required values used in template

To display HTML, Blocks class should be an instance of Mage_Core_Block_Template

Templates: the design part in the couple block / template

Ok, you have defined your rules management in the block, now we must display the output: this is the template role. Template is a PHP file responsible for generating the content to send (often some HTML). It’ll only fetch from the block whose data it must manage, and process them in: no loading, no calc, this is only display

Templates files are located in the template subfolder of the current theme applied

Display the couple block template usage in our pages

If you want to display the couple block / template usage, you can enable the developper option in the system > configuration > developper > debug hint location (context other than global): this will highlight you which block class and templates are used with your layout

Magento Page structure is managed though layouts

Now that we know what is the content of a Magento page, we’ll see how they are arranged

Each page of a Magento is loaded with a layout: layout defines how the page will be built: this is a reference of how the page will be built: each update will update the page structure

For each theme you can define a layout, and so, update the page content. All used layouts are located in the layout subfolder of your couple package/theme design folder

Magento design folder will contain layouts and templates subfolders

Layout files are XML files. The root node of the XML file is <layout> to have the minimal following structure


<xml version="1.0' encoding="UTF-8"?>
<layout>
</layout>

Don’t forget roles of each elements in a page structure!

If you have something to remember from this topic is the role of each element which compose a magento page: very often we can see that roles used of each composant are not the expected one and shortcuts taken to build the page can make harder it’s maintenance

This structure is available for all versions on Magento 1.x. Magento 2 seems to update how will be organized the code content. We wait and see a stable release 🙂

Configure how web crawler will index your Magento website

ROBOTS.txt is a standard which provides instructions for web crawlers that index your website. We'll see in this post how Magento allows us to manage robots.txt, and how we can extend its work.

Robots.txt, a quick overview

What is robots.txt ?

W3c explains us that robots.txt is a standard way integrated in HTML to tell search engines bots how you want them to index your website:

  • Do you want to allow them to index the page content?
  • Do you want to allow them to follow internal links to search new pages to index?

All these actions can be controlled by the following instructions:

  • index / noindex: manage your wish to index the current page content
  • follow / nofollow: manage your wish if you want that bots can search for links in current page

Hum, interesting, how we can set up these values?

Where do I set up robots.txt?

There is two ways to set up robots informations:

Robots.txt file

By default, web crawlers will search in your document root directory for a file named robots.txt. In this file, we will define to web crawler our global indexation policy.

If this file does not exist, web crawler will check in the page header

With the ROBOTS meta header

We also have the choice to manage the robots informations directly in page headers

In this case, we send in the page header a meta name "ROBOTS" which provides our indexation preferences. Here's an example of possible header sent

<meta name="ROBOTS" content="NOINDEX,NOFOLLOW" />

What's the interest of setting a good robots.txt policy?

Interest of these method is that you can control how indexers work:

  • I suppose that you wish to index your catalog pages
  • Do you really require that checkout page was indexed?
  • Do the links provided in cart page require an indexation?

Ok, I'm convinced. But Magento does not provides a mechanism to set up robots values?

How Magento manage robots ?

Magento provides a mechanism to configure robots values in header. This administrative panel is available in the System > Configuration > Design > HTML Head. Here's a screenshot:

As you can see, we can define the index / follow property but only in a global context: all pages will have the same meta ROBOTS values: so for now we are unable to provide differents robots instructions for our pages. Furthermore, if you do not set up a configuration value in the adminstrative panel, default applied is *; in this case it's the robot that decides to index your content!

Ok, but how can we set up differents values for our pages?

Setting differents robots values in Magento

There is a solution: we will use the layout structure of our pages: In these layouts, we can reference our block Mage_Page_Block_Html_Head which is responsible for displaying the robots instructions in our page. If we check the block class, it extends Varien_Object (so it's a classic Magento object), and have a method getRobots defined as this:

As you can see, robots values are stored in the _data property, under index 'robots'. So we just have to call setRobots method on your block, and it will update the related _data property, and so our robots values for our page.

Here's an example of layout robots definition:

Conclusion

With this method, we do not have to make an overload. Magento layout allows to define easily our robots preferences. Now we are able to restrict indexation in checkout, do not follow links in customer account, etc.

If you embed these instructions in some reuse module, you can now have your common robots policy set up in 5 min for each of your projects

This configuration will only works for crawlers that take care of the robots instructions. Some of them does not use them and index everything. For these ones, we have to find another way 🙂