Using periodic tasks (cronjobs)
Table of contents
- 1 Setting up cron jobs
- 2 Recommendations
- 3 Debugging cron issues
- 3.1 Test if the cron’s timing is actually what you are expecting
- 3.2 Test if the cron is currently running
- 3.3 Check if the cron daemon is running
- 3.4 Verify the cron command from your logs
- 3.5 Check if all cronjobs use their own lockfile for flock or share one
- 3.6 Test if the lock files are cleaned up
- 3.7 Test if the cron has ever run since you configured it
- 3.8 Verify whether your PATH variable and other environment variables are set correctly.
- 3.9 Verify whether your locations are setup correctly
- 3.10 Verify whether your cron is working when you run it on the commandline
- 3.11 Ensure yourself you are not using special characters like % and @ in your cron command
- 3.12 Verify the cron output
- 3.13 Catching error output in a log file
Setting up cron jobs
Enabling the Magento cron
This is recommended for every installation! Log in using SSH and run
crontab -e and add the following lines:
# Run the Magento cron every 5 minutes, skip if previous run is still # busy, and mail any output to firstname.lastname@example.org # MAILTOemail@example.com # Uncomment to receive the error output on your own email address */5 * * * * flock -n ~/.cron.lock php /data/web/public/cron.php
After adding the cronjob, press CTRL+X, Y and then ENTER to save the cronjob into your crontab.
Enabling the Magento 2 cron
To configure the cron for Magento2 add the following snippet to your crontab:
# MAILTOfirstname.lastname@example.org # Uncomment to receive the error output on your own email address * * * * * flock -n ~/.cron.lock php /data/web/magento2/bin/magento cron:run * * * * * flock -n ~/.update-cron.lock php /data/web/magento2/update/cron.php * * * * * flock -n ~/.setup-cron.lock php /data/web/magento2/bin/magento setup:cron:run
Magento 2 uses another mechanism for scheduling tasks. That’s why the Magento 2 cron needs to run every minute instead of every 5 minutes. More information can be found in the Magento 2 documentation.
Import cronjobs from a Byte account
To copy your cronjobs from your Byte Magento plan to your Hypernode account, use the following command:
ssh email@example.com "crontab -l" | crontab -
Make sure to adjust any absolute paths, and replace
flock, otherwise your cron will not work.
Adding custom cronjobs
For most cases, cronjob syntax is actually quite easy. See the full documentation or use these examples.
# Run an hourly job at 10 minutes past the hour 10 * * * * flock -n ~/.myjob.lock php /data/web/mycron.php
# Run a daily job at 3:20 in the night 20 3 * * * flock -n ~/.myjob.lock php /data/web/mycron.php
# Run a job on the first day of the month at 3:20 20 3 1 * * flock -n ~/.myjob.lock php /data/web/mycron.php
Stopping cronjobs after a timeout period
To avoid cronjobs running for hours, blocking all other crontasks to be executed by magento, you can make use of the
This tool can be given a “max execution time” and will stop the running task when this time period has exeeded.
# Run a daily job at 3:20 in the night and kill after 120 seconds 20 3 * * * timeout --kill-after=120 php /data/web/mycron.php
Cron Deadlock Protection
Our system will automatically add
flock to your cron commands. This will prevent many problems, for example when multiple concurrent imports cause a database deadlock. If you do not want our auto flock (not recommended!) you can add
# noflock to your command, like this:
# Run a command that does not care whether it is already running * * * * * php /data/web/mycron.php # noflock
Changing your crontab editor
When you use
crontab -e for the first time, you’ll be asked which editor you want to use. If you’d like to change your editor after your initial choice, please use the
select-editor command. If you want to temporary change your editor, you can do this by exporting the
export EDITOR=vim; crontab -e
More info can be found in our article about configuring your editor
- Idem-potency: When you write your own cron script, make sure the script is idempotent. When a script should only be run after a certain time or when a new import file is present, always make sure to exit the script when the conditions are not met. This avoids things breaking down when people run cron scripts manually to debug cron issues.
Time is always based on UTC (= GMT). This might differ from your local time: Dutch time is UTC+1 in winter and UTC+2 in summer. Or use this handy converter)
Do not run the Magento crons using
curl, as this is slow, might give partial results, and occupies webslots for real visitors). See above how to invoke the non-HTTP Magento cron.
Do not run the same cron simultaneously on multiple hosts, as results are unpredictable. When migrating, do not forget to disable the old crons.
Be carefull when using day of month and day of week together. Using
Dowwithout wildcards makes this an
ORcondition and not an
AND, running the job more often then you’d expect.
Cron does not deal with seconds so you can’t have cron job’s going off in any time period dealing with seconds. Like a cronjob going off every 30 seconds.
Monitor your cronjobs! You can do this by checking the exitcode and or the mail that is sent, or using a third party monitor. A free third party cron monitoring tool is healthchecks.io
Debugging cron issues
For new users, cron can be difficult to work with, as it’s hard finding out wat is causing cron not to run or to end before finishing the job. This section gives you some tips and tricks to debug cron issues.
Test if the cron’s timing is actually what you are expecting
If the cron is configured, but appears not to be running, check if the time configuration is at what you’d expect.
A great helper utility to validate the cron’s timing, is crontab.guru.
Test if the cron is currently running
If it appears your cron is not running anymore, check whether it is currently running using flock.
Sometimes a cronjob takes more time than expected or ends up running in a recursive loop.
If this happens, the lock never gets removed causing the cron daemon to delay the consecutive jobs.
To resolve this issue, find out why it’s still running and kill the current running cronjob. Then start a new one.
You can use the
ps faux command to check the running processes. If you see a cronjob still running, you can kill it by using its PID number:
You can kill the process (with PID 2) by using this command:
kill -9 2.
Check if the cron daemon is running
We monitor cron extensively, but to ensure yourself,
grep in the process list to verify whether the daemon is really running:
ps -ef | grep -i cron
Verify the cron command from your logs
When a cronjob ran, a log entry is written to
/var/log/syslog. Retrieve the cron command from the logs and check whether the command can be run on the commandline to avoid typo’s or badly copied example cronjobs.
If multiple cronjobs share the same lockfile and one job is running, the other can’t start.
To avoid this, ensure that all crons use a unique lockfile.
Test if the lock files are cleaned up
Sometimes when a cronjob crashes, the locks are not cleaned up successfully. This will obstruct the cronjob from running.
To fix, cleanup the old lockfiles by removing them:
This will remove the file, effectively removing the lock set on the file as well.
Test if the cron has ever run since you configured it
To get a history of all cron actions, like editing, updating and listing the cron files, all jobs of all users etc, you can easily grep for cron in syslog:
grep -i cron /var/log/syslog
This will generate a huge list of all jobs cron has run including editting, listing and removing the cronjob.
To verify whether it has run, grep for all cronjobs that are run by user app and check if your cronjob has ever run:
grep -i cron /var/log/syslog | grep '(app)'
Verify whether your
PATH variable and other environment variables are set correctly.
Cron does not automatically source environment and or shell settings from
You can’t use predefined shell environment settings like
$USER or homedir expansion like
If you want to use these settings you’d have to source the corresponding settings file accordingly by adding it to your cronjob:
Or alternatively if you want all the settings you use in the shell, source your
bashrc file prior to executing:
*/5 * * * * source /data/web/.bashrc && cp $HOME/somefile $HOME/public/exports
You can set things like
PATH= and other environment variables the shell uses.
IMPORTFILE="/data/web/public/some-very-long-import-file-in-cvs-format.products-from-some-website.cvs" DESTINATION="/data/web/public/some/very/long/path/in/the/magento/directory` */5 * * * * cp $IMPORTFILE $DESTINATION/
Verify whether your locations are setup correctly
Verify the correct path in cron arguments.
Cronjobs often contain full paths to scripts inside your homedirectory. Be sure to replace them for your situation at Hypernode. For example, if your site is currently hosted at Byte, you need to replace
Verify whether your cron is working when you run it on the commandline
If your cronjob appears to be started by cron, but does not finish the job it is supposed to handle, check what output is generated by running the cron on the commandline.
Ensure yourself you are not using special characters like
@ in your cron command
Cron uses some special characters in it’s syntax. Therefor you cannot use chars:
If used with command substitution (IE: the
date command) you can use shell command substitution (
To use special chars, escape them with a backslash:
\% instead of
Verify the cron output
Many users will add
2> /dev/null or
2>&1 > /dev/null to their cronjobs, redirecting all error output to
If you use this option, you will never see the error output the job creates.
To check whether a job generates error output, redirect output to standard out and let cron mail it’s output by setting a
MAILTO variable containing your email address. This will send all output of the cronjob to your email address.
If you only want to receive a mail when the jobs fails, use
chronic. This lovely tool only returns output when the exit code is not 0, therefor only sending the output to cron when the job did not succeed.
Catching error output in a log file
If you want to debug cron timing issues, use a wrapper that logs all issues to log files:
Save it as
~/bin/debug-cron and make it executable with
chmod +x ~/bin/debug-cron
Now you can debug your cron issues using the following syntax:
*/5 * * * * flock -n ~/.a6fe0d58.lock -c '/data/web/bin/debug-cron php -f /data/web/public/cron.php'
To check the output and run/stop time of the cronjob, check the logfiles in