A long time ago,
in a (not) so far away office,
a Magento project is coming
we could hear something like this
Guys, I have an error when I display product sheet: “table does not exist”, what’s wrong?
The little guy on the openspace corner who talks like a robot answered him: It’s normal, you must execute the SQL script create_table.sql located in var/temp
This is not about the force I’ll talk about, but the power of the dark side, the lazyness: How Magento can help us to solve deployment problems? With the magento setup scripts
Introduction to Magento setup
Main interest of the Magento setup
Have you ever asked how Magento install its own database? This setup process is run against the Magento setup scripts: all database tables, initials configuration values or required records are inserted with these scripts: no need to play any SQL, all is managed with the native setup Magento API
Files involved in Magento setup
This mechanism is strongly linked with Magento modules architecture: if your module requires a dedicated table, it must be able to create the prerequisite to work properly. Another case study is when project is started: you made some modeling choices, and those ones require perhaps a specific configuration. it’ll very interesting that this configuration will be set up automaticly on each location (developpement, testing, prod, …), without any human action.
After reading this post, if you check what is executed by Magento during database setup, you’ll see that each module is responsible for its own setup:
- Mage_Catalog create the catalog attributes
- Mage_Checkout create the tables of agreements
All modules files involved in setup are located in the module sql subfolder
Let’s see how it works
How does Magento find the module list that have some setups to run?
As I mentionned before, all is based upon the configuration files located in the app/etc/modules folder. When configuration is merged, Magento knows which are active modules.
From this merged configuration, Magento will find the setup list to run.
But not every module will have some setup. To know which modules have installers, it does not browse the sql folder for each module but looks for some dedicated XML nodes in configuration. These nodes depend on resources and are defined like this:
Without this node, and even if you have defined some sql scripts in your sql subfolder, nothing will happen
Now Magento can iterate through the configuration saved in cache, look for the <config><global><resources> to find the availables resources defined. if this resource has a setup subnode defined, it will be known as “with setup”
In which state are your Magento module sources? the version notion
Your module sources can be in 3 differents states with your Magento:
- Install mode: you have just installed the module source on your Magento, but for now nothing has already been set up
- Update mode: you have just made an update of your module sources, and the new setup has not already been played.
- Downgrade mode: you’ve made a downgrade of your module sources. And now, the state of your database is too far from your module sources
All these states are managed with two version numbers: the current version number of your sources and the latest installed version number of your module in database
Magento current installed module version number
This number is defined in the core_resource table. This table contains for each resource the latest version number installed
Magento current module version number
This number should be defined in the config.xml file located in module etc subfolder with the following node:
<config> <global> <modules> <%Namespace%_%ModulePath%> <version>you_version_here<version> </%Namespace%_%ModulePath%> </modules> </global> </config>
Is there a specific format for version number?
No, Magento does not expect a specific format. It’s a versionning problem and often we define the version number with a format like X.Y.Z, where:
- X: the major version number, and increases when you review the software architecture
- Y: the minor version number, and increases when you add a new functionnality in module
- Z: the bug fix version number, and increases when you fix a defect
What’s important is that your version number means something and changes according to your source code update
How does Magento find the list of setup to run?
The Magento resource defines where are located files and if there is some setup defined for your module
How does Magento link resource name with file path to find script list involved in setup?
The resource name is used to find where are located setup scripts:
This is in this folder that Magento will find the script used in setup
Setup state will help Magento to find possible file list to run
Then Magento will check in which state are your modules sources:
- install: there is no db record for your %resource_name% in core_resource table
- upgrade mode: core_resource database record is lower than the version defined in the module source
- downgrade mode: core_resource database record is higher than the version defined in the module source
According to this state, Magento defines in which case you are:
- install (Mage_Core_Model_Resource_Setup::TYPE_DB_INSTALL)
- upgrade (Mage_Core_Model_Resource_Setup::TYPE_DB_UPGRADE)
- rollback (Mage_Core_Model_Resource_Setup::TYPE_DB_ROLLBACK)
Version comparison between already installed version and source version is based upon the version_compare PHP method
Possibles file list to run
Here’s how Magento knows if there are some scripts to run. Let’s see how this file list is built
Can we use subfolders to organise our setups scripts in app/code/%Your_Module_CodePool%/%Your_Module_Namespace%/%Your_Module_Name%/sql/%resource_name%/ folder?
First, Magento iterates on these folders with dir class and so, will iterate only in this subfolder; so you cannot put your scripts in a subfolder, they won’t be played
Which extensions can be used to run setup scripts?
Magento iterates on all files and expects to find .SQL or .PHP files to run.
As a best practice, you must use PHP file to avoid using hard coded table names and field names: For example Magento allows you during install to provide a table name prefix. The easyest way to find this prefix without using PHP setup scripts.
How can Magento take care of the database server type used with your Magento? the database connexion connector type
On paper, Magento can use multiple databases server types. Default one is MySQL.
By default, Magento setups scripts can take care of the setups differences between each database servers with the connector type: Connector type is defined by the configuration node model used by the connection defined to run your module. By default, all connections of your Magento will inherit the connection default_setup, defined in the app/etc/config.xml. As you can see in the following screenshot, this node is by default defined as mysql4
When determining which scripts can be involved in the setup subfolder, Magento will look for the files matching the pattern %connector_type%-%source_state%-.+
How does Magento avoid playing two times the same setup script? the role of version
After finding all files matching the previous pattern, Magento reduces the list according to the current database state to only required versions with the version compare usage
At the end of this step, every script in the available list must be run
Running the scripts: how does Magento increase resource version number in core_resource table?
Plus the setup list to run, Magento will include one other that follows. Version number in core_resource table is not updated at the end of all the scripts, but after a script has completed normally. So if you need to restart the setup process, you’ll restart at the last error setup script
Conclusion about this Magento installer mechanism
This mechanism is really helpfull to deploy and normalize your different projects locations: by setting these scripts, you are sure that every location has the same required configuration values, concepts specified. Deploying a new location for a new developper or for the production server will be more easily because all required configuration values, tables, etc will be already set up.
One other great advantage of these scripts is to allow you to versionate only the scources code: no SQL state versionning requirement, all is managed through sources modules files
Develop and test setups scripts may have a cost to take care, but reduce risks based on misconfiguration or a forgotten SQL script