Best practices when using Magento setup scripts

As I mentionned before, Magento setup scripts enable you to make your software setup more reliable. But working with setup require to use some best practices. If not, it could become a nightmare…

Here’s a list of points you should take care of when writing setup scripts

Do not update a setup script already under revision control

A script already under version control has perhaps already been played in some locations, and so core_resource record for this setup has already been updated

In this case, in order to play a second time this script, you must restore the initial state before it:

  • At least, downgrade the core_resource related record
  • Perhaps destroy all that has been run within the script

It’s very problematic because we lost all the automation within the setup. So you should really be careful before sharing a script to ensure that all the script’s content is well written

Be independant from database structure

This is why I wrote previously to not use the SQL files and prefer using the PHP scripts: Magento allow you to prefix your table names for security reasons. If in production you’ve prefixed your tables but not in testing, your script must work fine in each case.

So you must be able to find the table name in currnet context, and this is the role of the methode Mage_Core_Model_Setup::getTableName method

Be independant of the server type which hosts your Magento

For now, this is only in theory that Magento can be run on other database servers than MySQL. But even if it’s a dream, perhaps the module you developed will exist very longer and in 2015, when Magento will be able to use another SGBD, your setup script will also work fine

To do that, it’s really easy since the 1.6CE / 1.11EE because database connexion layer has been updated to be more abstract

Reduce SQL instructions to their minimal to have the more accurate error management

Take a look at the following SQL query we want to run:

CREATE TABLE foo AS
SELECT entity_id, sku FROM catalog_product_entity;
ALTER TABLE foo ENGINE=Memory;
ALTER TABLE foo ADD INDEX (entity_id);
</code>
<p>The historical way to run it in Magento could something like this:</p>
<code>
$this->run("CREATE TABLE {$this->getTableName('module/foor')} AS
SELECT entity_id, sku FROM {$this->getTableName('catalog/product')}
ALTER TABLE  {$this->getTableName('module/foor')} ENGINE=Memory;
ALTER TABLE  {$this->getTableName('module/foor')} ADD UNIQUE (entity_id)");

Fine, but each SQL instruction can have its own errors. So if we play back the script, it's possible to have some instructions run many times and could lead to an inconsistent context, like:

  • Same index defined multiple times
  • Insert run many times
  • ...

To reduce this risk, divide each SQL query to its minimal entity: if an SQL error is encountered, the following code won't be executed and so next time you'll launch the script you won't run the same SQL request. So in this example, use multple run instructions

Take care of existing

It's very important to take care of possible existing resources, and so, for the following reasons:

  • In 1.6 and previous versions, there is no semaphore in Magento setup scripts; in multiple servers, it's possible that you've started your setup while another server was aleady running it (but have not yet updated the core_resource table)
  • If the first time you've run it, there was an error, you'll need to execute it a second time. Testing already existing resources will avoid you to restore initial state before executing it a second time

It can be done in SQL instruction or within PHP code

Don't use constants in Magento setup scripts

Constants are used to declare some reference content. So it's often the case when you write some setups scripts.

But if you reuse the constant definition in multiples scripts, take care of ensuring that you test if the constant already exists: When Magento run some upgrade scripts, it will execute all the required scripts in one time by including all scripts. So it could lead to a case thasituation where the constant is defined multiple times, and so through an error.

Leave Magento use its own database connexion management

It means, do not access connexions objects in setup scripts: this is not a problem when you use only one database server, but if you use different read and write database servers, by using database connexion objects you 'll may change where will be written the data you want to setup

So leave Magento make it's own work

How Magento load your modules configuration? the activation files located in app/etc/module folder

This post will be the first part of two topics based on the Magento configuration and on how this configuration is cached. If you are already experimented in Magento, this topic is probably something you already know, but I hope it would be helpfull for those who start with Magento

Magento is based on a module architecture. Each module can define its own components, blocks, scripts, models, databases connections, etc. Each module can embed multiples configurations files, but the main one is the config.xml file located in the module etc subfolder. To avoid reading each configuration file each time, every module configuration is saved in cache. Here’s how it works

First, Magento requires to know which scripts must be played: Magento load each XML files in app/etc/modules folder to find which are the modules we want to declare

Module activation file: the files list in app/etc/modules

For a quick reminder, files located in app/etc/modules folder should have the following structure:

<?xml version="1.0" encoding="UTF-8"?>
<config>
    <global>
         <modules>
              <%Module_Identifier%>
                   <active>true</active><!--true or false -->
                   <codePool>local</codePool><!-- local / core or community -->
                   <depends>
                      <!-- list of %Module_Identifier% prerequise to make our module works fine -->
                  </depends>
              </%Module_Identifier%>
     </global>
 </config>

active node in Magento XML activation file

This node informs Magento that we want to use this module. Expected values are case sensitives and should be true or false. If you set false, there will be no output, no override, no functionnalities, no layout updates

Magento bootstrap will only take care of the modules which are active true. Yhe Other ones are ignored.

codePool node in XML activation file

Codepool values are the subfolders located in app/code folder. As I mentionned before in my post on the loading APIs, changing codePool can modify how your classes are loaded, and so, if they can be overloaded or not

CodePool value provided in the activation XML file will help Magento to know where to look for our module configuration file

Module identifier

Module identifier allow Magento to link the configuration file with the module folder: The expected value for the Module identifier is based on %Namespace%_%ModulePath%: with this value and the codePool, magento is able to determine where to look for our module etc/config.xml file

XML identifier

I’ve noticed that encoding and version are attributes required for the XML files header. Truly, only version is required. But take the habitude to also note your encoding, and normalize your XML version as UTF-8

After browsing the app/etc/modules and read all XML files in, Magento knows which modules must be activated and loaded in our Magento

Dependancy tree between modules: the depends node in Magento XML module activation file

This node is not also required. But it can modify how setups are played: without dependancies, modules identifiers are sorted in alphabetical order. If your namespace starts with a character before M (for Mage namespace), do not define the dependancies and deploy all your data without an existing magento database, your setup will be runned before the Magento’s one, and so, there is a big risk that it fails if you call some Magento entities. So take also the habitude to define at least Mage_Core in your module dependancies.

Sometimes we can see that dependancies are defined in your module configuration file (config.xml). because dependancy tree is built before config.xml has been read, it has no interest

Merge configurations files

After reading the activations files, Magento knows which modules it must load (he knows namespace_module path, and codePool). So it can now go reading each modules configurations defined in module configuration file, the config.xml file: all config.xml files for active modules will be merged in a single one XML structure which will serve as global configuration

Now magento can save all data in cache

How to organize your modules definition in activation files?

Because it’s a XML structure and they are merged, you have two choices for your modules

  • Define each module location in a dedicated file
  • Define all yours modules locations in an only file

Magento chooses to use only one file to define all its modules: the file app/etc/modules/Mage_All.xml

If you work without versionning system (like git or subversion), there is no risk to work with only one file

But if you are numerous to work on the same project, you should prefer to use one file per module for the following reasons:

  • You can reduce conflicts risks (the less developpers work on the same file, the less risk you have to get conflicts)
  • If you want to publish some modules on Magento connect, you should make reworks in a single file usage

Conclusion

Is there a way to check that my module is well loaded?

The quickest way to check if your activation file is well read by Magento is in the Administrative panel available through System > Configuration > Advanced: If your module identifier is listed there, all is fine. In other cases, be sure to check its syntax

Thank you for this topic. When will be the second part published?

some suggested me to publish more often 🙂 Next part should be post quicker

Magento modules versions

If you produce some modules for Magento connect, you have perhaps test your modules under differents releases.

In major and minor upgrades, you know that there is some upgrade in code source and so, if you want to have only one module version instead of using branchs, you must test in which case you are.

I’ve made here a summary of available module versions defined from 1.3 to current version (1.5 when I wrote this post), and for the related enterprise versions.

Magento core module versions summary

CE 1.3.2.4
CE 1.3.3.0
CE 1.4.0.1
CE 1.4.1.0
CE 1.4.1.1
CE 1.4.2.0
CE 1.5.0.1
CE 1.5.1.0
EE 1.6.0.0
EE 1.7.0.0
EE 1.8.0.0
EE 1.9.0.0
EE 1.9.1.1
EE 1.10.0.1
EE 1.10.0.2
EE 1.10.1.1
EE 1.11.0.0
Enterprise_AdminGws
Enterprise_Banner
Enterprise_CatalogEvent
Enterprise_CatalogPermissions
Enterprise_Cms
Enterprise_CustomerBalance
Enterprise_CustomerSegment
Enterprise_Checkout
Enterprise_Customer
Enterprise_Eav
Enterprise_Enterprise
Enterprise_GiftCard
Enterprise_GiftCardAccount
Enterprise_Giftregistry
Enterprise_GiftWrapping
Enterprise_ImportExport
Enterprise_Invitation
Enterprise_License
Enterprise_Logging
Enterprise_PageCache
Enterprise_Pci
Enterprise_Persistent
Enterprise_PricePermissions
Enterprise_PromotionPermissions
Enterprise_Reminder
Enterprise_Reward
Enterprise_Rma
Enterprise_SalesArchive
Enterprise_Search
Enterprise_Staging
Enterprise_Targetrule
Enterprise_Pbr/idge
Enterprise_WebsiteRestriction
Mage_Admin
Mage_Adminhtml
Mage_AdminNotification
Mage_AmazonPayments
Mage_Api
Mage_Authorizenet
Mage_Backup
Mage_Bundle
Mage_Catalog
Mage_CatalogIndex
Mage_CatalogInventory
Mage_CatalogRule
Mage_CatalogSearch
Mage_Centinel
Mage_Checkout
Mage_Cms
Mage_Compiler
Mage_Connect
Mage_Contacts
Mage_Core
Mage_Cron
Mage_Customer
Mage_Dataflow
Mage_Directory
Mage_Downloadable
Mage_Eav
Mage_Giftcert
Mage_GiftMessage
Mage_Giftregistry
Mage_GoogleAnalytics
Mage_GoogleBase
Mage_GoogleCheckout
Mage_GoogleOptimizer
Mage_ImportExport
Mage_Index
Mage_Install
Mage_Log
Mage_Media
Mage_Newsletter
Mage_Ogone
Mage_PageCache
Mage_Page
Mage_Paygate
Mage_Payment
Mage_Paypal
Mage_PaypalUk
Mage_Persistent
Mage_Poll
Mage_ProductAlert
Mage_Rating
Mage_Reports
Mage_Review
Mage_Rule
Mage_Rss
Mage_Sales
Mage_SalesRule
Mage_Sendfriend
Mage_Shipping
Mage_Sitemap
Mage_Tag
Mage_Tax
Mage_Usa
Mage_Widget
Mage_XmlConnect
Mage_Weee
Mage_Wishlist
        0.0.11.11.0.0
        1.6.0.0.41.11.0.0
        0.0.61.11.0.0
        0.0.80.0.91.11.0.0
        1.6.0.0.41.6.0.0.91.11.0.0
        0.0.111.11.0.0
        0.0.81.6.0.01.6.0.0.11.11.0.0.2
1.8.0.0.01.11.0.0
           0.1.10.1.21.11.0.0
1.11.0.0
        0.0.21.11.0.0
        0.0.80.0.91.11.0.0
        0.0.131.11.0.0
           1.9.0.0.41.9.0.0.51.11.0.0
1.10.0.0.81.11.0.0
1.11.0.1
        0.0.31.11.0.0
1.7.0.0.01.11.0.0
        0.2.20.2.31.11.0.0
         1.6.0.0.01.11.0.0
        0.0.30.0.41.11.0.0
1.0.0.0
1.11.0.0
1.11.0.0
1.8.0.0.01.11.0.0
         1.7.0.0.141.7.0.0.151.11.0.0
1.11.0.8
1.7.0.01.11.0.0
1.8.0.0.01.8.0.0.21.11.0.0
        0.1.151.11.0.0.1
        1.6.0.0.11.6.0.0.21.6.0.0.31.6.0.0.41.11.0.0
           1.8.0.0.0
        0.0.11.11.0.0
0.7.10.7.21.6.0.0
0.7.1
1.0.01.6.0.0
0.1.20.1.21.6.0.0
0.8.11.6.0.0
0.0.10.0.11.5.0.0
0.7.01.6.0.0
0.1.70.1.110.1.120.1.130.1.140.1.100.1.110.1.120.1.130.1.141.6.0.0
0.7.691.4.0.0.211.4.0.0.281.4.0.0.381.4.0.0.431.4.0.0.441.4.0.0.141.4.0.0.181.4.0.0.261.4.0.0.331.4.0.0.381.4.0.0.431.4.0.0.441.6.0.0.5
0.7.101.6.0.0
0.7.50.7.80.7.50.7.81.6.0.0
0.7.70.7.80.7.100.7.80.7.101.6.0.0
0.7.60.7.71.6.0.0
 1.4.0.0.0  1.4.0.0.01.6.0.0
0.9.30.9.50.9.40.9.51.6.0.0
0.7.80.7.130.7.120.7.131.6.0.0
0.1.01.6.0.0
1.4.01.4.01.6.0.0
0.8.01.6.0.0
0.8.130.8.260.8.270.8.280.8.140.8.250.8.260.8.270.8.281.6.0.1
0.7.11.6.0.0
0.8.111.4.0.0.61.4.0.0.71.4.0.0.131.4.0.0.141.4.0.0.41.4.0.0.61.4.0.0.71.4.0.0.121.4.0.0.131.4.0.0.141.6.0.0
0.7.41.6.0.0
0.8.50.8.100.8.110.8.80.8.100.8.111.6.0.0
0.1.140.1.161.4.0.11.4.0.21.4.0.30.1.161.4.0.11.4.0.21.4.0.31.6.0.0.1
0.7.130.7.150.7.160.7.150.7.161.6.0.0
0.7.0
0.7.20.7.60.7.20.7.61.6.0.0
0.1.00.1.0
0.7.01.6.0.0
0.1.10.1.20.1.10.1.21.6.0.0
0.7.30.7.40.7.30.7.41.6.0.0
0.1.21.6.0.0
0.1.00.1.01.6.0.2
  1.4.0.21.6.0.0
0.7.0
0.7.60.7.71.6.0.0
0.7.01.6.0.0
0.8.00.8.20.8.30.8.10.8.20.8.31.6.0.0
         0.0.1 0.0.1 1.6.0.0
1.5.0.0.0         
0.7.01.6.0.0
0.7.00.7.11.6.0.0
0.7.01.6.0.0
0.7.20.7.41.4.0.11.4.0.20.7.20.7.41.4.0.01.4.0.11.4.0.21.6.0.0
0.7.01.6.0.0
1.0.0.0
0.7.21.6.0.0
0.7.21.6.0.0
0.7.21.6.0.0
0.7.70.7.101.6.0.0
0.7.40.7.60.7.50.7.61.6.0.0
0.7.01.6.0.0
0.8.01.6.0.0
0.9.380.9.390.9.561.4.0.151.4.0.211.4.0.250.9.440.9.561.4.0.81.4.0.171.4.0.21.4.0.251.6.0.3
0.7.70.7.121.4.0.0.41.4.0.0.60.7.90.7.121.4.0.0.41.4.0.0.61.6.0.0
0.7.20.7.40.7.30.7.41.6.0.0
0.7.01.6.0.0
0.7.21.6.0.0
0.7.20.7.50.7.70.7.30.7.50.7.71.6.0.0
0.7.80.7.111.4.0.01.4.0.10.7.100.7.111.4.0.01.4.0.11.6.0.0
0.7.00.7.11.6.0.1
  1.4.0.0.01.6.0.0
1.4.0.131.4.0.131.6.0.0
0.131.6.0.0
0.7.40.7.70.7.80.7.90.7.50.7.70.7.80.7.91.6.0.0

Magento module versions usage

In previous Magento versions, there was some source code updates but no update in module version. So use module version carrefully

I'll upgrade this listing frequently