vBulletin 3.x to vBulletin 4.x / 4.1.x / 4.2.x upgrade process for large vBulletin forums
eva2000’s upgrade process
My vB 3.8.x to vB 4.1.x upgrade process which I personally use myself as well as for private paying clients is as follows and assumes:
- you have ssh2 telnet access
- your vB 3.x forums are only using MySQL default MyISAM/Memory tables with NO InnoDB tables
- you have enough free disk space to hold all the backups approximately 3x to 6x times the size of your forum database/files/attachments in free disk space.
- public root directory = /home/username/public_html
- live database name = livedbname
- new databasename name = newdbname
Step 1. Tune php and mysql settings specifically for backup, restore and upgrade process. Tune mysql to ensure enough memory is allocated to buffers to speed things up, increase timeout values for mysql to ensure no mysql timeouts and for php. Ensure mysql server has InnoDB support disabled otherwise you’ll have to deal with InnoDB tuning parameters before you start as vB4 automatically converts 6 tables to InnoDB if mysql server has InnoDB support enabled. Read the reasons of how and why you’d want InnoDB support disabled in my blog post at http://vbtechsupport.com/675/ and if you do decide to use InnoDB storage engine for some database tables, I’d also look at upgrading your MySQL version to a better performing MySQL version such as MySQL 5.5.12, Percona 5.5.10-20.2 or my recommendation MariaDB 5.2.x series.
Step 2. Backup live forums via ssh2 telnet
cd /home/username/ mysqldump --opt -Q -u mysqlusername -p livedbname > /home/username/vb3backup.sql cp -a public_html public_html_oldvb38 |
if you have attachments outside of public_html i.e. /home/username/attachments
cd /home/username cp -a attachments attachments_oldvb38 |
You could compress these vBulletin forum sql, attachment and public_html backups, but for large vBulletin forums it justs adds additional time to the upgrade task and if you have to revert to one backup it adds more time in uncompressing the vBulletin backups. So I usually don’t compress these backups until after the vBulletin 3.x to 4.x upgrade process is completed fully.
Step 3. Create new empty database called newdbname and import backup vb3backup.sql into newdbname
mysql -u mysqlusername -p newdbname < /home/username/vb3backup.sql |
Step 4. Get new vB version files uploaded to /home/username/public_html. Two ways of doing this – I prefer 2nd method if I am on the clock and don’t have much time.
Method #1
- Empty out /home/username/public_html and upload new vB version files from zip folder /upload
Method #2
- On local computer, extract vB new version zip file and in /upload rename config.php.new to config.php and populate all the settings pointing to step #3’s newdbnamedatabase which now has a copy of your live database imported into newdbname.
- Also add to vB new version config.php following 2 lines to disable plugins and mail for upgrade process
define("DISABLE_HOOKS", true); define('DISABLE_MAIL', 1);
- If you use vB datastore caching in config.php either APC, Xcache, File or Memcached datastore caching, disable it temporarily for the upgrade by commenting it out with double forward slashes.
- Then zip up /upload folder i.e. upload.zip and upload that to /home/username/ via FTP
- Then ssh2 telnet into server as same username as your vB forum site account (this is so extracted files retain the correct file owner/group permissions).
- Once in ssh2 go to /home/username directory and extract upload.zip you’ll have a folder at /home/username/upload now which contains all the new vB version’s php files etc
unzip upload - Still in ssh2 telnet, copy all files in /home/username/upload and overwrite all files in /home/username/public_html
cd /home/username/upload unalias cp cp -Rfp * /home/username/public_html
Now you should have overwritten all files in /home/username/public_html with new vB version files
[box type=”warning”] At this stage, if you are using vBSEO addon and have MaxCDN or any other CDN hosted vB php/js/css files served externally, please flush and purge the CDN caches from your MaxCDN or CDN provider control panels to ensure you’re upgrading using the uploaded new files and not cached CDN files.[/box]
Step 5. vB 3.6.x to vB 3.8.x upgrades.
- Run upgrade to 3.7.x and then repeat Step 2 and backup 3.7.x forums.
- Run upgrade from 3.7.x to 3.8.x and then repeat Step 2 and backup 3.8.x forums.
- Note: if you are upgrading using vB 4.0.9+ and higher vB php files, you can from the start using PHP CLI command line upgrade method outlined in Step 7. Just make sure to drop the fulltext indices mentioned in Step 6, BEFORE running PHP CLI command line upgrade.
Step 6. vB 3.8.x to vB 4.0.9 upgrades.
- Similar to large imports mentioned here, drop the post table’s fulltext index on title BEFORE running vB 3.8.x to vB 4.1.x stage as vB 4.x no longer uses it. There’s 3 other fulltext indices you can remove too which are no longer used in vB 4. Run query on newdbname database (copy of your live forums). In ssh2 telnet type following to enter mysql command line:
mysql -u mysqlusername -p newdbname
- Then run query to drop fulltext indices in post, thread, groupmessage and socialgroup tables. If you get error saying the key doesn’t exist you can ignore it and move on.
ALTER TABLE `post` DROP INDEX `title`; ALTER TABLE `thread` DROP INDEX `title`; ALTER TABLE `groupmessage` DROP INDEX `gm_ft`; ALTER TABLE `socialgroup` DROP INDEX `name`;
- Now begin upgrade process from vB 3.8.x to 4.0.9
- Note: if you are upgrading using vB 4.0.9+ and higher vB php files, you can from the start using PHP CLI command line upgrade method outlined in Step 7. Just make sure to drop the fulltext indices mentioned in Step 6, BEFORE running PHP CLI command line upgrade.
Step 7. vB 4.0.9 to vB 4.1.x latest – utilise the new command line upgrade process oulined at http://www.vbulletin.com/docs/html/upgrade_cli. Note if you have php opcode caching such as Xcache installed, please disable Xcache from the server and turn off Xcache vBulletin datastore caching from config.php BEFORE running PHP CLI command line upgrade as it seems Xcache doesn’t support PHP CLI.
- ssh2 telnet into server as same username as vB site account and run from /home/username/public_html/install directory this command
php -f upgrade.php
Step 8. vB 4.x upgrade tasks and backup
- Once upgrade complete, I like to make another 2 backups by repeat Step 2 and backup 4.1.x forums BEFORE vB 4 upgrade tasks are done and another backup AFTER vB4 upgrade tasks are done.
- vB4 upgrade tasks are those notice prompts when you first log into vB4.x admincp, you’ll be prompted to do the following: rebuild search index, adding keywords to threads and rebuilding thread information.
Rebuilding Search Index
Rebuilding search index can be a time consuming task, so vBulletin included a way of doing this rebuild via SSH2 telnet via PHP command line. In the downloaded vB zip file folder called do_not_upload there’s a file called searchindex.php. You upload searchindex.php to the root directory of your vBulletin forum installation. In this example it would be uploaded to /home/username/public_html/searchindex.php. You then log in via SSH2 telnet as the same user as the forum domain/site user and run the following two commands:
cd /home/username/public_html php searchindex.php |
You’ll be prompted to enter the path to your vBulletin installation in this case = /home/username/public_html.
Rebuilding Threads
Rebuilding threads can also be time consuming for large vB forums. vBulletin developer, Michael Anders unofficially posted a PHP command line script for such task as well called threadcli.php (download link for registered vB members only as it isn’t included in official vB4 zip downloads). Works pretty much the same way as searchindex.php but asks for starting thread id and number of threads to update per pass.
Run it from includes folder:
cd /home/username/public_html/includes php threadcli.php |
If threadcli.php doesn’t work for you and you get access denied message, edit threadcli.php and add right after the first line:
<?php define('VB_ENTRY','1'); |
or try Paul K’s cli.php script instead which has more options.
cd /home/username/public_html php cli.php |
php cli.php > Checking vB installation > Installation seems to be fine, continuing ################################################# > Please select an option 1) Quit 2) Update User Titles and Ranks 3) Rebuild Thread Information 4) Rebuild Forum Information 5) Add Add Missing Thread Keywords 6) Update User Names 7) Update Post Counts 8) Fix Broken User Profiles 9) Rebuild Post Cache 10) Empty Signature Cache 11) Delete Duplicate Threads |
Step 9. Revert your php and mysql settings back to everyday 24/7 values and then remove from you config.php the lines
define("DISABLE_HOOKS", true); define('DISABLE_MAIL', 1); |
Then re-enable vB datastore caching in config.php if you had disabled it for the upgrade.
Step 10. Update YUI version – by either updating to vB 4.1.3 PL2 or higher or you can edit includes/class_core.php and update the version of YUI used from v2.7.0 to v2.9.0. Then after vB 4.x is upgraded, in AdminCP -> Options -> Server Settings and Optimization Options -> Use Remote YUI = select Google
from
define('YUI_VERSION', '2.7.0'); // define the YUI version we bundle |
to
define('YUI_VERSION', '2.9.0'); // define the YUI version we bundle |
So basically, you do the upgrade on a copy of your live database in database named newdbname – leaving live database untouched in case you have to revert to a backup if upgrade messes up. You’ll end up with a few backups at different stages so it’s easier to revert to a particular stage rather than completely start over from scratch if upgrade messes up somewhere between.
You’ll have vB 3.6.x backup, vB 3.7.x backup, vB 3.8.x backup and vB 4.1.x BEFORE upgrade tasks done and AFTER upgrade tasks done. If your database is very large then disk space could be an issue, so you could be selective as to how many backups you do, I’d at least have 3 backups, original live database backup, vB 3.8.x backup and vB 4.1.x backup.
Largest vB forum I have tried this process on was aproximately 38GB size. If you have problems or vBulletin errors during upgrade, best to check out vBulletin 4 Installations and Upgrades support forums.
Once you have upgraded to vBulletin 4.x, you can check out Shawn/Digitalpoint.com’s vBulletin 4 Optimization Guide.
Update: June 9th, 2012
Added youtube video for steps 1 to 4 outlined above as an example of upgrading from vB 4.2.0 to 4.2.0 PL1. Creating this video was a bit awkward as for years now, I normally use multiple monitors for different apps and screens so having to condense this video to single monitor recording was a bit of a challenge.
Video best viewed in HD 720P mode full screen.
Updated Notes:
If upgrading vBulletin 3.x with vBulletin 3.x API installed to vBulletin 4.2.x, you may get an error at vBulletin 4.1.10 Alpha 1 part of the upgrade script as follows:
Upgrading to 4.1.10 Alpha 1 ---------------------------------- Step 1 - Updating setting table Step 2 - Updating settinggroup table Step 3 - Database error in vBulletin 4.1.9: Invalid SQL: DELETE FROM navigation WHERE productid = 'vbapi'; MySQL Error : Table 'databasename.navigation' doesn't exist Error Number : 1146 Request Date : Error Date : Script : http:// Referrer : IP Address : Username : Classname : vB_Database MySQL Version : 5.1.61-log |
This bug is fixed in upcoming vBulletin 4.2.1
