The Sonic Center

Welcome Center => News and Updates => Topic started by: GerbilSoft on April 05, 2007, 06:24:15 pm

Title: TSC's Back Up!
Post by: GerbilSoft on April 05, 2007, 06:24:15 pm

I'm temporarily hosting TSC for a while until Rolken can find a new host (or host TSC himself). While attempting to run TSC on my server, I found that TSC's database structure was way too inefficient for my server to handle it, so Rolken and I spent the past few days optimizing the database.

Old Database Size: 1.6 GB
New Database Size: 660 MB
Saved: 1 GB :-)

Not only has the database size decreased substantially (without deleting old data), many of the queries have been optimized. The statistics processor has been improved to process stats much faster. (Before, it took around 125 seconds to process 200 stats. Now it takes around 10 seconds.) [Note that these stats include points and overall updates.]

That's it for the major changes. Stats updating is currently set to once every 10 minutes instead of 5 minutes as before, but I may change this if it doesn't seem to impact the server that much.

Also, if anyone encounters any weird bugs on the site, post them here or in the IRC channel. We're still fixing some conversions from SELECT * to SELECT `columns`, which improves performance but causes issues when the required columns aren't specified.

Status of various subsections:
- On-site videos: Up (partially)
- Off-site videos: Up
- Chat Statistics: Up
- CGI:IRC: Up

UPDATE: Apparently, Apache was causing massive CPU load. I switched to lighttpd, which uses less resources. Lighttpd is now configured for TSC and all directories are updated. :-)

UPDATE 2: There seems to be an intermittent problem where some submitted statistics generate an invalid queue entry. The invalid queue entry blocks statistics processing until it is removed. That statistic is then "lost" and needs to be resubmitted. We're working on figuring out the cause of the problem.

UPDATE 3: In case you're wondering what the downtime was from 12 PM to 2 PM EDT (2007/04/06), I accidentally unplugged the network cable. <_< (The cable in use is missing the pin, so it's easy to knock out. I need to replace it sometime.)

UPDATE 4: Enabled lighttpd's mod_compress module for compressing static pages (currently only HTML and plain text) for faster loading and less bandwidth usage.

UPDATE 5: The server doesn't have email capability, so anyone who registered today was unable to activate. I've turned activation off for now and activated everyone who registered.

Title: Re: TSC's Back Up!
Post by: antiwwe on April 06, 2007, 09:34:40 am

hooray to TSC!

Title: Re: TSC's Back Up!
Post by: P.P.A. on April 06, 2007, 10:38:02 am

*parties*
I hope TSC'll find a server soon.

Title: Re: TSC's Back Up!
Post by: Groudon on April 06, 2007, 10:41:43 am

Quote from: GerbilSoft on April 05, 2007, 06:24:15 pm

Status of various subsections:
- On-site videos: Up, but not all
- Off-site videos: Up
- Chat Statistics: Down
- CGI:IRC: Up

fix'd

Title: Re: TSC's Back Up!
Post by: magnum12 on April 06, 2007, 03:13:17 pm

Good job dood. Making stuff more efficient in the memory department will make things a lot easier on us in the future, especially when TSC4 eventually shows up.

Title: Re: TSC's Back Up!
Post by: Auriman1 on April 06, 2007, 03:22:33 pm

1GB of optimization is an impressive feat, especially without deleting anything.  Glad to see TSC up and running once more.

Title: Re: TSC's Back Up!
Post by: Conker T. Squirrel on April 06, 2007, 05:59:47 pm

yey!

Title: Re: TSC's Back Up!
Post by: GerbilSoft on April 13, 2007, 12:07:39 pm

Bah, Yahoo and Google were hammering the site earlier. I banned the user-agents. Unfortunately, this means stats won't be available on Google Cache for now. :-(

EDIT: ...and Become.com. Three web crawlers hammering the site at once = bad. <_< All three are b&.