vBulletin Datastore Memcached server cache corruption errors

While setting up 2x load balanced Litespeed web servers for a vBulletin v4.1.2 forum, I came across an issue when I enabled vBulletin config.php for datastore caching via Memcached where vBulletin would report the datastore cache was corrupt with the below quoted message. Rebuilding vBulletin Bitfields from tools.php only fixed the problem temporarily on front end, but AdminCP had it’s own set of errors.

Frontend greeted with error

vBulletin datastore error caused by one or more of the following:

1. You may have uploaded vBulletin files without also running the vBulletin upgrade script. If you have not run the upgrade script, do so now.
2. The datastore cache may have been corrupted. Run Rebuild Bitfields from tools.php, which you can upload from the do_not_upload folder of the vBulletin package.

AdminCP after running tools.php

Warning: implode() [function.implode]: Invalid arguments passed in [path]/admincp/index.php on line 230
 
Fatal error: vBulletin datastore cache incomplete or corrupt in [path]/includes/init.php on line 247
 
Warning: array_keys() expects parameter 1 to be array, null given in [path]/includes/adminfunctions_navpanel.php on line 157
 
Warning: implode() [function.implode]: Invalid arguments passed in [path]/includes/adminfunctions_navpanel.php on line 157
 
Warning: Invalid argument supplied for foreach() in [path]/includes/adminfunctions_navpanel.php on line 160

I couldn’t for the life of me figure out what the problem was as I have done numerous Memcached datastore caching setups before with same Litespeed, PHP and Memcached/Memcache versions. Turns out the client had modified the class_core.php, class_datastore.php and init.php files with pre vBulletin v4.1.x series tweaks which had worked on single server setup, but not for 2x load balanced Litespeed web servers running vBulletin v4.1.2.

Morale of the story, always first check vBulletin AdminCP -> Maintenance -> Diagnostics -> Suspect File Versions to make sure vBulletin php files haven’t been modified. Could save yourself some headaches :)

For real life example of Memcached server statistics read here.