Major Game Services issues Saturday 21st March 2020 14:32:00

The game is currently experiencing performance issues in game-wide services (such as the authentication server, teleportation, and friends lists). We have isolated the symptoms to a suspected root cause of database performance. We have deployed database access optimizations to the game and are looking into more potential optimizations and fixes.

We have rolled out a server-side fix for the Toon Details Panel issue. We will be monitoring the performance of our database on a feature-by-feature basis.

We will be monitoring DB performance going forward to see if any other features are causing performance issues. Thank-you for your patience and understanding while we resolved this issue.

Posted 4 years ago

We have reverted the changes made during last night's maintenance. Our new theory is that a bug in the implementation of the Toon Details Panel on the server side is causing increased database load. We are working to roll out a server-side fix.

Posted 4 years ago

We are aware of continued performance issues in game-wide services. We are still investigating problem points in our database performance. Thank you for your patience.

Posted 4 years ago

We've now deployed a fix that should address all the issues, so hop in and play! We'll be closely monitoring the situation to make sure nothing else is amiss.

Posted 4 years ago

Our previous fixes failed to address the root cause of the issues, but given the current circumstances we've decided to take the game offline for maintenance to apply a more permanent fix -- allowing for us to maintain high player population over time. We have high hopes for this fix, so stay "Tooned" for the fix to be completed!

Posted 4 years ago

We believe we have identified the root cause of the problem. We are working on a solution and will attempt to deploy a fix as soon as possible. We apologize for the continued inconvenience and appreciate your patience.

Posted 4 years ago

We have determined the issues are due to the increase of game activity causing unexpected lag between our game servers, the web site, and the database systems. We are working on a solution to reduce the lag to a point the game can play more smoothly with higher numbers of activity.

Posted 4 years ago