Archiving the miniBB-based forums

An effective solution for hundred thousands of forum messages

Imagine that you have thousands of old, outdated pictures stored on your disk, which take too much space without obvious purpose. You already checked those pictures, printed some of them, showed to your family and friends, and in a perspective of many years, they may not be required anymore. Your next step is to burn them on CD or DVD, i.e. archive, and then delete from the disk, freeing up the space for new files.

With forums, we have almost the same. As your forums grow, mySQL database, which holds topics and replies, grows, too. But as larger the database becomes, as slower it starts to work. This specially may affect shared and virtual hosting, which resources are split to other websites. From my experience, I have noticed that forums having about 150,000 - 200,000 posts or even more, start to work significally slower than those having up to 100,000 posts. It may be limitation of mySQL table size or server's resources; anyway, the only thing we taken into attention when working this problem out, is how to let the forums keep large amount of messages and work fast at the same time.

miniBB stores topics, messages and forums information in 3 mySQL tables, named by default 'minibbtable_forums', 'minibbtable_topics' and 'minibbtable_posts'. 'minibbtable_posts' is the bigger one, and when the script starts reading from it, specially reading based on difficult criterias, like in Search section, it slows down. The primary idea about "archiving" is to remove older messages from that table, copying them to a completely standalone table, also copy all associated data from 'minibbtable_forums', 'minibbtable_topics'. We consider any topic as "old", if it's latest reply has been posted in the specified archiving dates range, most likely in a certain year.

That way, having multiple standalone tables, and using built-in miniBB's functions for running multiple forums on the same core, it is possible to provide a special code and design standalone templates for each archive, keeping the older information in read-only mode, and freeing up the space for the live table.

A quick review of what happens

Visually, there are only few changes applied to your forums. Let's say before archiving, you may see something similar under forums Statistics section ^*:

And each forums member has similar statistics regarding his forum activities, under the forums Profile:

After archiving, the Statistics will be extended with the additional information:

Member's Profile, too, will list how many live and archived messages a certain user has in total. This makes possible to extend add-ons which are based on total user's postings amount:

Clicking the "Archived" links, you are forwarded straight to the certain section of the archived forum, either Statistics or Profile.

All archives are set up in the Archives listing section, which may look like this:

While archiving forums, it doesn't mean you should just move each topic manually. In general, live topics can't be move to any archive at all. This is performed by our special archiving script, which in steps copies certain topics to an archive, calculates users and forums statistics, and only then frees up the live table. Such "steps approach" allows to validate each level of archiving and to control it very deeply. With combination of the proper back-up of forums, this zeroes risks of losing the big board database. At once we have thought about the speed, performance and security of this process, and all its future's execution.

More specific features and configuration

Links from the aforementioned section lead to the certain archive, which may have URL like '.....forums/archives/2005_2007/' or '.....forums/archives/2009/'. You may set up a standalone prefix, or put archives right under the immediate URLs like '.....forums/2005_2007/' or '.....forums/2009/'.

An archive runs like a standalone read-only forum, but is tied up with the generic users table, and performs on the same miniBB core. That means, if you upgrade miniBB version, all archives are upgraded automatically. Besides of it, each archive has a specific set of templates, which could make its look completely different from default forums.

Each such archive is automatically set up in read-only mode, so the new messages can not be added. Despite of it, for the forums administrator it's possible to bring any archived topic "back to life", i.e. move it back to the live forum anytime. miniBB automatically handles all such topic URL adresses: if the topic has been moved to the Archive, or otherwise, and the outdated URL is entered or visited, it automatically forwards the visitors to the new proper URL.

Configuring archives, it's possible to specify any alphanumerical archiving prefix for each of them, and assign specific set of common options for the certain archive, or all archives at once. That means, for example, that for Archives it's possible to turn the "Recent Topics" section off, switch the "old-school" forums index layout instead of the modern one, which miniBB provides these days by default.

If there is a File Attachments add-on installed, all attached files will be archived amongst forum postings. In the Archvies, it's also possible to switch on or off any of the standard miniBB add-ons installed. For example, on live forums, you may have Avatars add-on installed; but for better performance, in Archives Avatars may be turned off. Managing archives, it's possible to use Merging add-on or Moving replies add-on. All standard miniBB functions, like closing a topic or moving it to another forum, will work by default.

Get involved!

With the archiving process, your forums will enter completely another area of competition. You as forums administrator, may keep the live, dynamic content that way, which may be outdated, but still attracting new visitors through search engines, or providing informative historical meaning. Like you may notice, Internet nowadays often looks like a short-term can: information is changing dramatically each minute; what was actual yesterday, today is forgotten and removed. With the miniBB approach, you have all chances to keep your forums even just for fun - we are sure, if you take just a few years look back, you will be surprised! :-) This is just another step to make something long-term, and respect users, which built content for your for many years, before opening the new path.

Because of the complexity level of the archiving process, and the very individual work involved in setting up the archives, we provide this solution on a paid basis only. Feel free to contact us privately for getting in touch, if you feel that your forums are riched the "archiving stage", or if you have any other questions regarding this subject.

^* The screenshots used in this article, are taken from Polish Forums, one of the greatest miniBB customers of all times, which is also the initial sponsor of this feature. Check their website to see how the Archiving works live.