vBulletin Cron Based Email Sending – vB mail queue back log
vBulletin cron based email sending is a new feature in vBulletin 4.2 and is enabled by default and configured for only 10 batch emails sends until that queue is cleared. There is an email delay setting which allows you to set the delay between each email sent within a batch to either none, 1, 2 or 5 second delay. Then there’s the delay arising from vB Scheduled Cron Task being set to process each batch of emails every 10 minutes by default. Update: vBulletin developer Paul M provided more info, seems the batch email defaults to 10 in vBulletin 4.1.x and 50 in fresh new installed vBulletin 4.2.x. But on vBulletin 4.1.x to 4.2.x upgrades it preserves the old batch email value of 10 instead of 50.
As described on vBulletin.com article:
Cron Based Email Sending is new feature released with vBulletin 4.2.0 that allows you to queue batches of these emails. So, instead of having a hundred emails sent out in a batch, you can have batches of 20 emails sent out until the queue is clear. In addition, you can control the delay between each email that’s sent from your server.
Note: You must have the Use Mailqueue System enabled in order for this to work. Turning it off will also disable the Cron Based Email Sending feature.
vB Cron Based Email Problem:
Problem is working with alot of the larger and very active vBulletin forum clients, I’ve noticed alot of complaints of email notification delays by members on this forums. When I look into vB MySQL database’s mailqueue table, I see a back log of 30,000 to 60,000 emails queued up with the default 10 emails per batch setting. With email batch on a 10min vB Scheduled Task cron by default, you can imagine how long it would take for emails to clear the vB mail queue!
vbmailqueue.sh
So how do you figure out an optimal value for the number of emails to send per batch ? For VPS and dedicated server customers with root/SSH telnet access, I wrote a bash script, vbmailqueue.sh which can:
- output and display the entire mail queue
- count number of emails back logged in the mail queue
- provide the last mailqueue id value
- also display the current vB AdminCP email options.
- log all output to a time stamped text file for viewing and analysis
The vbmailqueue.sh script is attached to my vBulletin.com Blog post for download. Update: Looks like vB5 upgrade nuked the attachment on my blog article so you can find the vbmailqueue.sh contents posted here. Just edit the script’s 5 variables explained within the script, upload to server and make it executable.
So if you uploaded script to /home/tools/vbmailqueue.sh (ASCII mode)
chmod +x /home/tools/vbmailqueue.sh cd /home/tools ./vbmailqueue.sh view ./vbmailqueue.sh count |
You can manually run vbmailqueue.sh or setup a cron job task for it. After a period of statistics collection, you can view the info paying attention to the mail queue size and then adjust your vB AdminCP email options appropriately.
vbmailqueue.sh output
menu options – view the full mail queue by mailqueueid, count the number of emails queued and well debug is just something I had to setup to check I was getting the right last mailqueue id in the queue.
root@ [~/tools]# ./vbmailqueue.sh ./vbmailqueue.sh view ./vbmailqueue.sh count ./vbmailqueue.sh debug |
This count output was with one vB client with batched email processing increased from 10 default to 5000 per batch. Before this change, up to 60,000 emails were queued up.
root@ [~/tools]# ./vbmailqueue.sh count ------------------------------------------------------------------ vbmailqueue.sh 0.0.3 - George Liu (eva2000) vbtechsupport.com ------------------------------------------------------------------ As at: Mon Jun 4 07:51:04 CDT 2012 ------------------------------------------------------------------ First Mail Queue ID: 14181939 Last Mail Queue ID: 14181940 ------------------------------------------------------------------ vBulletin Mail Queue row size is: 2 ------------------------------------------------------------------ vBulletin email settings are set as follows: +----------------+------------+--------------------+--------------+ | varname | grouptitle | value | defaultvalue | +----------------+------------+--------------------+--------------+ | smtp_pass | email | | | | smtp_tls | email | ssl | none | | smtp_user | email | no-reply@gmail.com | | | smtp_port | email | 465 | 25 | | smtp_host | email | smtp.gmail.com | | | use_smtp | email | 0 | | | utf8encode | email | 0 | 0 | | usebulkheader | email | 0 | 0 | | needfromemail | email | 0 | | | bounceemail | email | | | | mail_delay | email | 0 | 0 | | emailsendnum | email | 5000 | 50 | | mailqueue | email | 1 | 1 | | emailfloodtime | email | 10 | 60 | | usemailqueue | email | 1 | 1 | | secureemail | email | 1 | 1 | | enableemail | email | 1 | 1 | | displayemails | email | 1 | 1 | +----------------+------------+--------------------+--------------+ ------------------------------------------------------------------ Stats Output saved at: /root/vbmailqueuelogs/mailqueue_040612-075104.log ------------------------------------------------------------------ total 4.0K -rw-r--r-- 1 root root 2.2K Jun 4 07:51 mailqueue_040612-075104.log ------------------------------------------------------------------ |
Viewing the entire vBulletin mail queue entries
root@ [~/tools]# ./vbmailqueue.sh view ------------------------------------------------------------------ vbmailqueue.sh 0.0.3 - George Liu (eva2000) vbtechsupport.com ------------------------------------------------------------------ As at: Mon Jun 4 07:51:12 CDT 2012 ------------------------------------------------------------------ +-------------+------------+ | mailqueueid | dateline | +-------------+------------+ | 14181939 | 1338814229 | | 14181940 | 1338814244 | +-------------+------------+ ------------------------------------------------------------------ Stats Output saved at: /root/vbmailqueuelogs/mailqueue_040612-075112.log ------------------------------------------------------------------ total 8.0K -rw-r--r-- 1 root root 2.4K Jun 4 07:51 mailqueue_040612-075104.log -rw-r--r-- 1 root root 681 Jun 4 07:51 mailqueue_040612-075112.log ------------------------------------------------------------------ |
vBulletin 4.2.0 default email options:
vBulletin Scheduled Task Manager has the email queue running every 10 minutes (Process E-Mail Queue)
The alternative is if your mail server can handle the load, just disable vB Cron Based Email sending by setting in vB AdminCP -> vBulletin Options -> Email Options -> Use Cron Based Sending = NO
If you are on shared hosting without SSH / Telnet access, you can check your mail queue size, via vB AdminCP -> Maintenance -> Diagnostics -> System Information -> Table Status. Find the mailqueue table and check row size.
Summary:
Basically, there are 3 ways to control the delay and mail queue.
- change email batch size
- change vB scheduled cron task for processing email queue
- disable cron based email sending
