Subscribe to our podcast!

Episode 4: Scaling Large Web Sites with Joe Stump, Lead Architect at DIGG



Subscribe to the show via Subscribe via iTunes Store or

kick it on DotNetKicks.com

 

About This Episode

Have you ever wanted to learn how top 100 web sites are architected?  Deep Fried Bytes hosts Keith Elder and Chris Woodruff sat down with Joe Stump, Lead Architect at DIGG to discuss scaling large web sites, his life, development experiences and team building.


Have you ever felt like you needed to boost the performance of your .NET app? Or that you needed to understand how your code is behaving or why it isn't behaving in the way you want it to? Well, knowing how your code performs is really easy with a code profiler like ANTS Profiler from Red Gate. It can help you locate performance bottlenecks really quickly. It provides you with quantitative data such as hit count and code timings, so you don't have to waste time guessing where bottlenecks might be. It is ideal if you want to improve parts of your applications and make your algorithms run more efficiently. You can download a 14-day free trial of ANTS Profiler from the box ad below.


Thanks to our guest this episode

joestump

Joe Stump has been developing large scale LAMP web sites since before Web 1.0 was Web 1.0. He's currently the Lead Architect for DIGG where he spends his time partitioning data, creating internal services, and ensuring the code frameworks are in working order. In his spare time he plays disc golf, works on side projects, and contributes to PEAR.

Joe’s blog can be found here.

 

Show Notes

 Download Show

Subscribe To The Show

There are three ways to subscribe to our show so you can stay current. We support standard RSS, subscriptions via the iTunes Store and we are also available in the Zune Market place. Chose your poison by clicking on your choice below.

Subscribe to our podcast! Subscribe via iTunes Store Subcribe via Zune Market Place

#1 D1gg3r1 on 6.18.2008 at 11:23 PM

Welcome to the front page of digg, woot!

#2 Watch Television on 6.19.2008 at 1:48 AM

well he definitely knows his stuff, Digg has made vast imporvements in scaling since 2004.

#3 Derek on 6.19.2008 at 9:58 AM

What is the "track svn" solution mentioned? It's an integrated SNV and Bug Tracking system, but I can't seem to find out exactly what he is talking about.

#4 Josh on 6.19.2008 at 12:41 PM

I crashed and burned with < 10k visitors in a day. Listening now, hope this helps me out as I'm desperate for some of this info.

#5 keithelder on 6.19.2008 at 2:34 PM

@Josh, curious as to how many page views 10k visitors gave you? If those 10k visitors clicked on 8 page views a day that is still less than one page a second since there are 86,400 seconds in a day. Most sites should be able to handle 1 page view a second easily in my opinion without even trying to optimize anything.

#6 Josh on 6.19.2008 at 4:58 PM

I was using several API's to display pages with mysql stuff wrapped around it. Crashed several times because of max 25 sql connections. But that shouldn't be too restricting of a number if the page loaded quickly. So the bottleneck was how long each mysql connections was left open (during cURL stuff).

Page views was not the problem, elapsed time for building pages was.

#7 Joe on 6.20.2008 at 12:05 AM

@Derek:

trac is trac.edgewall.org and has integrated wiki, ticket system, source code (can use subversion or git or other backends) &tc.

#8 Martin W. on 7.09.2008 at 1:31 PM

We've got a large service and we're tracking around 10 million unique visitors with over 40 million page views daily. We just started using memcached and it's really helped. I was really excited to hear about some of these other real-world technologies people are using as it's a constant challenge to improve speed & reliability.

#9 Mehul Harry on 7.29.2008 at 1:02 AM

Good podcast with Joe. Interesting his takes about I/O vs language. Also liked the bit about your local beers at the end.

Leave a Comment

DevExpress