SAP Uses Second Life As Well

The other night, while I was trying to find Cory's blog, I ran across an interesting blog from a guy at SAP. Turns out we were both at the same talk in Palo Alto. (See my previous post for more info). SAP is using Second Life for online training and demos. For details, read Mario's blog. Pretty neat.

Virtual Worlds Hosted in Data Centers

A few weeks ago, I went to a talk by Linden Labs CTO, Cory Ondrejka. His background leading up to Second Life is quite interesting. He started off developing simulators for the military, then moved to writing arcade games, and then finally ended up creating Second Life. Pretty interesting fellow, and a good speaker as well. For those of you not at the talk, here's an interesting article in Information Week about Second Life (SL).

SL runs on dual-cpu x86 servers with Linux and MySQL. A 16 acre "plot" in SL runs on a single core on one of these servers. A human (or company) in the real world can purchase a 16-acre plot for an initial cost of US$1900 and a monthly payment of US$300. SL is essentially a hosting company for virtual real estate and virtual goods that are traded in the real world. As an example, the SL city of Amsterdam sold for US$50,000 on eBay. (Check it out)

Here's some metrics on SL:

• Bandwidth: 10 GB/second outbound
• Storage: 40 TB of user data
• Scalability: 100,000,000 SQL queries/day (using MySQL)

Cory also claims that SL is one of the largest (or the largest) deployment of MySQL. And SL is adding 1-2 racks of servers each week to keep up with demand. Now those are some pretty impressive stats.

Welcome, Connie Weiss!

One of my friends, Connie Weiss, joined the blogosphere. Check out her blog for a welcome change of pace. Although I don't have the variety of pets that Connie does, I do have four parakeets at home!

Data Centers are Hot Again, Thanks to Web 2.0

Yesterday's San Jose Mercury News describes how data centers are "hot again" due to traffic from the growing popularity of social networking Web sites, such as YouTube and Facebook. Demand for data centers is increasing, as companies such as Apple, Yahoo and Kaiser are purchasing or leasing new data centers to keep up with demand for their applications.

Some interesting excerpts from the article:

• While prime office space in downtown San Jose costs about $2 a square foot per month, space in a data center rents from $15 to $30 a square foot per month.
• In the past two years, one million square feet of space in Silicon Valley data centers was purchased or leased, more than all the space taken off the market from 2001 through 2004.
  

The article mentions Apple's 2006 purchase of a data center in Newark, California. I wonder if that's the location that serves my daughter's iTunes purchases-- just a short hop across the Dumbarton bridge.

The Most Energy-Efficient Server Is One That's Powered Off

"Green data centers" and energy-efficient servers are the latest buzz in IT and government, as it turns out. Information Week's March 12 features green data center technologies as the cover storr. The online version doesn't have the same catchy Gordon Gecko-esque "Green is Good" title, but it's still a good read.

Today's data centers are built like offshore platforms from the oil and gas industry. Every 100 years, there's a monster wave, usually 50 feet high, from a massive hurricane or storm. So, offshore platforms are built to withstand this 100-year wave. The platform is taller than the wave's crest, and the platform is built to withstand the force of the wave.

Data centers are typically provisioned to meet peak demand, much the same way as oil platforms are engineered for that 100-year wave. Servers are provisioned to handle peak applications loads or traffic from major events, such as the quarterly sales promotion or special news event. On a typical day, however, utilization is much lower, but you still have all of those servers humming away, using electricity and generating heat.

What if you could power off all of those unused servers? Not only would you save electricity, you would also reduce your cooling costs. Could you determine which servers are idle and then those shut down? And could you do this automatically? How fast could you respond to changes in your environment? If load increases, could you power on the additional servers you need?

These are tough demands for your data center, but these changes can reduce your energy costs. The local utility, PG&E, is providing rebates to companies who can reduce their overall energy consumption or reduce their energy consumption on-demand in order to prevent rolling black-outs. We're participating in a new initiative by the Silicon Valley Leadership Group for energy-efficient data centers. As part of this initiative, we're showing how Cassatt Collage can be used to implement these use cases to reduce your overall power costs.

Back on the home front, we have several hundred servers in our system test lab. Since they're managed by Collage, only a third of the servers are usually powered on at a given time. When we do large scale-out tests, Collage fires up most of the servers in the lab. However, we usually schedule those scale-out tests for the weekends. Our lab A/C consists of three different units, each of which turn on when there is demand. One unit is constantly running; the second one kicks in during parts of the day; the third one only kicks in during heat waves or scale-out tests. Pretty cool, huh? (Pun sort-of intended)

Getting "Out There"

A friend of a friend (funny how that works) gave me some pointers to get my blog plugged into the broader "blogosphere." I just created a profile on technorati.com and am "posting claim" to my blog there.
Technorati Profile

Would You Please Get Those VM’s Under Control!

With all the recent controversy in financial markets and politics, it’s time for us geeks to stir up a little controversy of our own. A recent article in the Register talks about the challenges in managing Virtual Machines and licensing issues associated with the ability to deploy new (virtual) servers at will. A recent blog at Server Virtualization captured some of the opinions regarding the topic of how closely you should control the VM’s in your environment and whether or not developers can/should “hide a VM under their desk.”

Instead of finding new ways to circumvent your IT department, what if your IT department could provide you access to the VM’s you need, when you need them? When you’re done with your VM’s, you could give them back so that others can use them. Since there’s no physical asset, borrowing a VM should be easy. What if your IT department could go one step further and provide your VM’s freshly installed with the O/S and applications you need. This is a win-win situation. The IT department can still keep tabs on server resources and software licenses. And developers get access to the environments they need in their dev/test cycles. Cassatt Collage can help you set up this dynamic VM environment. (Read more)

At Cassatt, our IT director, Kirk, is the one who’s on the hook for Sarbanes-Oxley compliance. I really don’t want to see Kirk go to jail for failure to comply with software licenses-- one of the possible consequences of not getting a handle on how many VM's you have in your enterprise. After all, Kirk is a nice guy, and he does bring farm-fresh eggs for Karen, our CFO. So even though I’m a developer at heart, I do my part to make sure that Kirk stays out of the slammer, Karen gets her eggs, and I get my paychecks. So don't you think it might be worthwhile to get those VM's under control?

What Can You Do with a Web Services Interface?

Last year, we were working with a large software company and using Collage to provision and manage this company’s application suite. We presented our results to the VP sponsoring our project. We showed several use cases for provisioning new server instances within minutes and automatically responding to server failure. He was impressed with the functionality, but he requested that we provide a high-level “dashboard” application that could integrate with his company’s application suite. The VP designed this new User Interface (UI) on the whiteboard at the end of the meeting, and the UI looked nothing like our current UI (of course).

After returning to our office, I tapped Mukund, who wears many hats at Cassatt. I needed him to write a new UI, drawing on his prior experiences as a Java Swing developer. He used the Collage Web Services interface to build a Java Swing app that provided a high-level view of the application domain. He also included controls to increase service levels of the managed application.

A week-and-a-half later, we went back to the VP and showed him the new app that he had designed on the whiteboard. The VP was quite pleased-- surprised, in fact-- that we were able to turn this around so quickly. Later, we had a chance to present our results (and the Swing app) to their executive management team. We’re still working with that customer, and we’re still using the Web Services interface to provide custom functionality in their environment.

What Should You Do if a Server Fails? Absolutely Nothing.

A server failure can be disruptive to your business and your personal life. Servers tend to fail at the worst times— late at night, over the weekend, or in the middle of an important demo. Disk drives fail, motherboards burn out and software crashes; these things happen. To fix the problem, someone could reboot a server or reinstall software on a new server. What if your data center could automatically recover by performing these actions for you?

Cassatt Collage goes beyond provisioning. Collage constantly monitors the applications and servers in your data center. Collage polls for a heartbeat from each server—using standard OS-level and application-level monitors available in Linux, Windows and Solaris. You can also introduce your own custom monitors, such as a customized agent or a script running inside a database, and have Collage monitor that as well. When a server has failed, Collage will replace the server with a new one from the free pool and boot the same application/service on the new server—all of this within minutes and without losing any data.

For each application, you specify the monitoring parameters. You can define which monitors to use, the polling interval, and how many retries you should allow. In a smaller configuration with less than 100 servers, I like to monitor SNMP at 30-second intervals with 3 retries. For larger configurations with ~400 servers, I would increase the monitoring interval to 60 seconds. For Apache servers, I add an HTTP monitor on a system URL embedded in my web app so that I can ensure that the Apache service is running.

As part of our standard customer demo, I like to pull a blade server that’s running an application and watch Collage respond to the failure. I’ll let the customer pull the blade out of the chassis in the lab; blinking lights, fan noise and 10 racks of servers always adds a little extra to the demo. By the time we return to the conference room (with the failed blade in hand), Collage has quarantined the failed blade in a maintenance pool, allocated a new server from the free pool and booted this server to the same application. All of this takes only 3 minutes from bare metal to a running application on a new server. This demo is always very memorable and illustrates High Availability in a very simple manner. (Seeing is believing.) I had given this demo to some visiting executives from TCS. A year later, I had met them again in San Jose, and they still remembered the demo.

Business Lessons from “Buff Bill”

Two years ago, I had put together a fun video for our Sales kick-off event. I wanted to illustrate several use cases and the derived benefits from using Cassatt Collage to manage your data center. The video featured Cassatt employees and Bill Coleman, our CEO, as “Buff Bill,” head of Buff Bill’s Boards and Bikes. The video was definitely fun to make and somewhat funny (as in, “don’t quit your day job” kind of funny). In the video, Buff Bill used Collage to:

1. Quickly integrate the IT systems of a recent acquisition.
2. Shift the distribution of computing resources between different web portals based on seasonal demand.
3. Respond quickly to increased workload by ramping up capacity as needed and in advance of new sales promotions.

The video featured Mukund as the poor sys admin who had to miss a Blink182 concert to bring new systems online-- but that was before he started managing his data center with Collage. After Mukund started managing his systems with Collage, he had way more free time—enough time to party with his friends and even take in a Linkin Park concert with Buff Bill.

Although we didn’t win any awards in the Sundance Film Festival (no surprises there), Buff Bill tackled some of the same challenges facing our customers today. One of our recent customers has achieved the top spot in their industry, and their only means to grow top-line revenue is through acquisitions. Their biggest headache has been integrating the IT systems of the acquired company. They are planning to use Cassatt Collage to manage all their IT systems so that they can quickly integrate newly acquired companies.

Another large customer faces predictable traffic spikes at different times during the week. Certain web properties are accessed almost exclusively on the weekend, whereas other business-related web properties are most active during the weekdays. We’re talking with them about using Collage to shift resources dynamically between their different web properties so that they can reduce their CapEx and OpEx.

One of these days I might post the Buff Bill video on YouTube. (I’ve been forbidden by our CFO, Karen. That almost sounds like a challenge to me, but I do like my paychecks.) In the meantime, you should check out Bill's thoughts on utility computing and what data centers may look like in the future. Recently, Bill was interviewed by Laurianne McLaughlin , the technical editor of CIO Magazine. Bill talked about a future where phone companies will run your data center. You can read about it in Laurianne’s blog. It’s a very interesting read, even though it's not as funny as my video.

Blog Tag

Okay, so I just got tagged a few days ago by Ken Oestreich. It’s taken me a while to post a list of five things about me that you probably don’t know, but here goes:

1. When I was in high school, I sold my first computer program and accompanying article to an Apple II enthusiast magazine—Nibble Magazine. I sold them 3 articles and programs in total, but they only published one of them. It was assembly code and an accompanying bitmapped font set to display text with high-resolution graphics on the Apple II+.
2. In 1989, I started my own three-person software company (Victory Software) to write role-playing games for the Apple IIgs. We managed to sell a few thousand copies of three different games. However, we shut down the company a few years later after Apple killed the IIgs line in favor of the color Macs. All of this was in the pre-Internet era, but it’s amazing what historical references you can find on the web these days.
3. I don’t write game software anymore, but I have taken up golf. A few years ago, my daughter and I were playing at Blackberry Farm in Cupertino. The pro shop told us that Julie Inkster was playing two holes ahead of us. We did manage to see Julie and her kids from across the lake, but we never did close enough for an autograph. My golf swing and short game are nothing to write about (even in a blog), but I have been a pretty good swing coach for my kids. My kids and I never saw anyone else famous on the golf course, but my son is a big Tiger fan. He wears Tiger’s Sunday red-and-black combination when we play on Sundays.
4. When I worked in Rochester, Minnesota in the 80’s, I actually placed fifth in a city road-race. I ran just under 11 minutes for a two-mile road race— nothing great, but good enough for a small town of 60,000. I still have my plaque somewhere in the garage. The race was in early May, but it was 30 degrees outside and snowing! During the last 100 yards, I got passed by a teenager, which was a little disheartening. Back in my youth, I never did manage to break the 5-minute-mile barrier, but I did get within 5 seconds. I still run these days, but it’s more like 7:30 miles. :-)
5. I have a US patent. It’s patent #6,683,553. I got it when I worked in the Java Web Services group at Sun. The patent belongs to Sun, but I still have the plaque on my desk.

Okay, so now it’s my turn to tag 5 folks. Unfortunately, I’m a relatively new blogger, and I don’t have a bunch of blogging friends. So, I’ll tag Floyd Strimling and Rob Gingell. And I’ll put out 3 requests to folks who would be good bloggers. (I’ll just nudge them privately.)