Big Fish Games is a global leader in the online games industry and distributes more games worldwide than any other online site. Within three years of its debut, BigFishGames.com rocketed into the Top 10 game portals on the Web and now serves millions of downloads every day.

Their Business Challenge

BigFishGames.com is a fast-growing website with over 25 million unique customer accounts and over 2.5 million visitors per month. In addition to the English site, Big Fish Games also offers international game portals in Japanese, German, French and Spanish. Their ever-growing user base is a huge boost to their business, but it also raises big challenges around IT capacity planning. To ensure the highest quality game experience, Big Fish Games has to accurately predict demand and increase bandwidth at the right time to keep a balance between over-utilizing the system, introducing delays and a bad user experience, and under-utilizing the system, resulting in a waste of capacity and money.

Their MySQL Solution

Big Fish Games started using MySQL as a small start-up. MySQL allowed Big Fish Games to quickly grow their business with lower cost and hardware requirements, and has scaled with the company as it has grown into an industry leader. Today, Big Fish Games deploys 40 MySQL servers to power its popular gaming website which offers thousands of games, with new games introduced every day. To achieve the scalability and reliability required by this high-trafficked website, Big Fish Games relies on MySQL Replication. Plus, DRBD is used to improve high availability. In addition to customer-facing material such as the dynamic website content, e-commerce store, game coupons and discussion forums, the MySQL database is also used for internal operations, tracking game downloads, account authentication, game activations and server logs.

MySQL Query Analyzer

In order to accommodate the growth in website traffic, the DBA team at Big Fish Games has been looking into opportunities to improve application performance. Tuning and optimizing the database is one of the options, but it won't help if the performance problem is caused by poorly-written SQL code.

To gain insights into the quality of the SQL code and execution statistics, Big Fish Games has been using the command line tools to identify target areas for potential performance improvement. However, for every problem resolution, extra effort was required to combine information from multiple sources because each command only provided a limited perspective.

Now, the MySQL Query Analyzer provides a consolidated view of query activities and execution details, and has enabled Big Fish Games to quickly identify poorly running queries and tackle the root causes directly in the SQL code. With the help of the MySQL Query Analyzer, the DBA team caught a "bad" query running 400,000 times overnight which never showed up in query logs. Furthermore, the MySQL Query Analyzer is very easy to use and doesn't require the user to be a world-class MySQL expert to fully leverage its benefits.

Since the Query Analyzer uses a Service Agent listening to application queries and performances metrics, the MySQL servers can always be live and operational when being analyzed. There is no need to switch the servers back and forth between on-line and off-line, which eliminates unnecessary risks to server availability and reliability.

After deploying the MySQL Query Analyzer, Big Fish Games tripled its database performance within three days, rather than weeks.

MySQL Enterprise Monitor

Big Fish Games also relies on the MySQL Enterprise Monitor and the Dashboard graphs, which show the number of queries per second, CPU load and replication status, to ensure that the website is performing well. Big Fish Games finds the MySQL Enterprise Monitor valuable because it is built for MySQL and offers more relevant and useful information than generic monitoring tools.

The MySQL Enterprise Monitor provides critical data points for Big Fish Games to analyze and determine the optimal number of slaves to serve its current website traffic and to plan for the future capacity requirements. This tool also helps the DBA team to gain insight into the system status, usage patterns and potential problems, without having to wait to be notified by the operations group.

Big Fish Games chooses to deploy MySQL Enterprise for the following reasons:

• High Performance: MySQL provides fast transaction speed to serve over 300,000 simultaneous users on BigFishGames.com
• Ease of use: MySQL is very easy to use which allows DBAs to manage MySQL servers without a steep learning curve
• Low Maintenance: Using MySQL Enterprise Monitor, Big Fish Games employs just two DBAs to monitor over 70 MySQL servers, 40 in active production and 30 in the testing environment
• Low TCO: MySQL enabled Big Fish Games to launch their business, grow fast and establish themselves as the industry leader at a fraction of the cost compared to using a proprietary database
• Unlimited Deployment: MySQL Enterprise Unlimited gives Big Fish Games the fixed-cost predictability to deploy additional servers without additional costs. This is especially beneficial for companies with rapidly growing data
• 24x7 support: MySQL offers top quality support, with long-time MySQL developers providing guaranteed 30 minutes response time for MySQL Enterprise Platinum customers. It's invaluable for Big Fish Games to receive problem solving advice from MySQL support engineers when business-critical applications go down at midnight
• Support for popular Operating Systems: MySQL is well-integrated with all major Linux , Solaris and Unix distributions, saving time for DBAs and improving administrative experience
• Support for C, C++, C#, PHP, Python, Ruby and Java: MySQL supports drivers for a wide range of programming languages, including PHP, used by Big Fish Games for the front-end presentation layer, and Java, used with the Tomcat application server in the middleware layer

Memcached

In addition to MySQL Replication, Big Fish Games further increases scalability by using Memcached, a distributed caching layer. All the web content is stored in Memcached, and most of the website queries are processed by this in-memory cache, which significantly improves response time as well as scalability.

Sun Fire x64 Servers

Big Fish Games utilizes a 3-tier server deployment strategy, and Sun's x64 servers have been chosen because of their excellent reputation for performance and reliability.

• Sun Fire X2100 server is best for applications which require lots of local disk space but less I/O or CPU speed
• Sun Fire X4100 server works well for applications which demand fast processors but don't need speedy local disk I/O
• Sun Fire X4140 server is optimal with 8 drive bays for applications where faster local disk I/O via RAID 10 and battery backed up write cache is essential

By identifying the requirements for each application and the right server for each condition, Big Fish Games has gained 20x in performance by merely replacing an X4100 server with an X4140 machine.

• Read the full case study »