Building Web Scale Architectures with Network-based Caching

IT Editorial

Thursday, 18 September 2008 14:06

Written by Gary Orenstein

Examples of balancing popularity and scale include social networking sites and the enormous amount of photos they accumulate and share, music sites that serve millions of titles to millions of users, and video sites that allow voyeurs to comb through thousands of new titles appearing every day. Each of these services requires supporting infrastructure that must continually grow and evolve to keep up.

To help scale these popular web applications, architects have used caching at various layers within the web stack, often tailoring caching solutions to specific applications and specific servers. While these solutions help alleviate short term performance concerns, the lack of a unified solution ultimately comes at the price of excessive management and maintenance costs.

New solutions are now being deployed that consolidate and centralize a globally addressable cache that can be accessed by multiple applications across more than one tier within a typical web stack. The advantage of this network-based resource is akin to the flexibility of network-based storage compared to direct-attached storage: more flexibility and more scalability.

Scalability Barriers for Web Scale Applications
Changes in workloads and user behavior directly impact scalability barriers for web applications. Over the last several years, we have rapidly shifted from a one-to-one to a many-to-many model for data sharing. For example, we used to write and read data on our own desktop computers. Now we spend far more time reading documents collaboratively, with the majority of that content hosted on central web repositories. The rise of sharing all types of digital content has flipped the workload patterns such that new approaches are needed. This evolution is showcased in Figure 1.

Figure 1
 
File Access Bottlenecks
The increase in broad collaboration and access results in scalability bottlenecks for web applications. The first element of this is the rise in concurrent users which can easily reach in to the millions with large sites that have a social component. The second element is that this large user base functions as content creators as well as content consumers.

When too many concurrent users are submitting, searching, and retrieving content at the same time, the I/O loads of the web, application, and database servers can overwhelm the underlying mechanical disk storage systems. Frequently, administrators will add excessive amounts of storage to compensate, but this over-provisioning does not address the root problem of the mechanical media, and even with such over-provisioning, user response time suffers.

Figure 2
 
Using Caching to Alleviate Bottlenecks
The first remedy to I/O related bottlenecks after storage over-provisioning is to deploy cache memory, generally within a single server, and often dedicated to a specific application. This type of deployment can alleviate the bulk of an existing I/O bottleneck, but falls short in that it focuses on single servers and applications. For example, deploying cache memory within a single storage system only applies to that storage system. It cannot be easily shared with other storage systems. The outcome of this approach is outlined in Figure 3.

Figure 3
 

This dedicated approach inhibits scaling and flexibility which are now critical to web scale applications supporting hundreds of thousands to millions of concurrent users. To increase the overall efficiency, data center managers must consider new alternatives.
Comparing Cache Locations
Cache memory can benefit infrastructure at many different layers and levels. But it is critical to keep the big picture in mind regarding efficiency of the overall infrastructure as not all placement locations are created equal. Figure 4 shows three scenarios.

In the first Low Device Count scenario, it really does not matter where the cache memory is placed. When the system operates within the confines of a single server and single storage system, placing the cache at either location will provide full coverage.

In the second Multiple Device Configuration – far more representative of web-scale architectures – placing the cache memory on a single server, single storage system, or single disk drive has limited effectiveness. There are restrictions on sharing resources, and the alternative of placing similar resources on every node can drive excessive overall costs.

The third Multiple Device Configuration demonstrates how network-based cache deployment applies to all servers and storage devices within the environment, maximizing the effectiveness and efficiency of the cache memory.

Figure 4
 

Another way to think about cache placement is to chart the two main objectives 1) coverage and utilization and 2) proximity to the server. In terms of coverage, placement between disks and servers is the most ideal for cache memory to reach the largest number of end node devices. But the other objective is to get the cache memory as close to the server as possible in order to keep server utilization high and I/O latencies low. But if the cache memory is too close to an individual server (such as within that server) than we lose the advantage of the network-based reach. Figure 5 shows how coverage/utilization and server proximity result in an optimal, network-based location slightly closer to the servers.

Figure 5
 
Shared Centralized Network-based Caching
Applying the principles from Figure 5 we can see the benefits of deploying a network-based, centralized scalable caching appliance to support multiple web stacks.

• Broad network reach
o Placement in the network provides comprehensive reach to all servers and all storage devices, and the cache can be shared across multiple web stacks
• Scale
o Network placement enables the addition of cache modules to the appliance, scaling performance and capacity without disrupting applications
• Lower total cost and smaller footprints
o Instead of replicating cache memory across all servers or storage devices, a consolidated approach helps lower costs and space requirements
• Reliable service delivery
o By employing a clustered, high-availability approach, caching appliances guarantee reliable service delivery in the event of a single cache module failure
• Data-readiness
o Caching appliances accommodate multiple applications and handle those applications with different cache profiles, enabling a degree of readiness for rapidly changing workloads

The diagram in Figure 6 further outlines these benefits:

Figure 6
 

Case Study – Indexing for Large Scale Applications
Let’s take a closer look at a case study example of centralized caching at work. In this scenario, manual workarounds and convoluted architectures were responsible for delayed updates and in turn a poor user experience.

This application is responsible for collecting, indexing and then serving requests for user contributed content. The basic sequence is as follows, and outlined in Figure 7:

1. The index server crawl the database where user-generated content resides
2. Those servers generate an index file on the NFS storage
3. Were the index servers to access the NFS storage directly, bottlenecks would ensue, so the index files are manually propagated to each individual index server, causing excessive maintenance and delays in the index “freshness”
4. Index requests are served in limited manner

This configuration cannot easily scale due to the manual propagation process. Having the index servers access the NFS storage directly is also not an option because they drive too much I/O traffic to a single disk-based storage system.

Figure 7
 

By applying the techniques and architectural models for centralized caching, the configuration is radically streamlined and the bottlenecks are removed. This simplified sequence is outlined in Figure 8.

1. The index server crawl the database where user-generated content resides
2. Those servers generate an index file on the NFS storage
3. Propagation is no longer required as the index servers can access the data through the memory-based caching appliance providing scale, performance, and instant data access

Figure 8
 

Conclusions
Today’s web applications need to scale simply and quickly to support the increasing numbers of concurrent users. The shear size and scale of these compute challenges defy conventional thinking and require a new approach to solve I/O bottlenecks and deliver seamless user experience. Caching helps alleviate many of the issues, and adopting a centralized, shared network-based approach delivers the costs savings and equipment efficiency needed to achieve best-in-class results.

About the Author: Gary Orenstein is Vice President of Marketing at Gear6

TrackBack URI for this entry

Comments (0)

Subscribe to this comment's feed