Sunday, September 21, 2008

Local file-sharing drastically cuts network load

P4P System for Efficient Internet Usage

Data distribution under traditional, P2P and P4P architecture. (Courtesy of Doug Pasko and Laird Popkin)
The 160-mile download diet: Local file-sharing drastically cuts network load

Ever since Bram Cohen invented BitTorrent, Web traffic has never been the same. Whether that's a good thing or a bad thing, however, is a matter of debate.
Peer-to-peer networking, or P2P, has become the method of choice for sharing music and videos. While initially used to share pirated material, the system is now used by NBC, BBC and others to deliver legal video content and by Hollywood studios to distribute movies online. Experts estimate that peer-to-peer systems generate 50 to 80 percent of all Internet traffic. Most predict that number will keep going up.

Tensions remain, however, between users of bandwidth-hungry peer-to-peer users and struggling Internet service providers.

To ease this tension, researchers at the University of Washington and Yale University propose a neighborly approach to file swapping, sharing preferentially with nearby computers. This would allow peer-to-peer traffic to continue growing without clogging up the Internet's major arteries, and could provide a basis for the future of peer-to-peer systems. A paper on the new system, known as P4P, will be presented this week at the Association for Computing Machinery's Special Interest Group on Data Communications meeting in Seattle.

"Initial tests have shown that network load could be reduced by a factor of five or more without compromising network performance," said co-author Arvind Krishnamurthy, a UW research assistant professor of computer science and engineering. "At the same time, speeds are increased by about 20 percent."

"We think we have one of the most extensible, rigorous architectures for making these applications run more efficiently," said co-author Richard Yang, an associate professor of computer science at Yale.

The project has attracted interest from companies. A working group formed last year to explore P4P and now includes more than 80 members, including representatives from all the major U.S. Internet service providers and many companies that supply content.

"The project seems to have a momentum of its own," Krishnamurthy said. The name P4P was chosen, he said, to convey the idea that this is a next-generation P2P system.

In typical Web traffic, the end points are fixed. For example, information travels from a server at Amazon.com to a computer screen in a Seattle home and the Internet service provider chooses how to route traffic between those two fixed end points. But with peer-to-peer file-sharing, many choices exist for the data source because thousands of users are simultaneously swapping pieces of a larger file. Right now the choice of P2P source is random: A college student in a dorm room would be as likely to download a piece of a file from someone in Japan as from a classmate down the hall.

"We realized that P2P networks were not taking advantage of the flexibility that exists," Yang said.

For the networks considered in the field tests, researchers calculated that the average peer-to-peer data packet currently travels 1,000 miles and takes 5.5 metro-hops, which are connections through major hubs. With the new system, data traveled 160 miles on average and, more importantly, made just 0.89 metro-hops, dramatically reducing Web traffic on arteries between cities where bottlenecks are most likely to occur.

Tests also showed that right now only 6 percent of file-sharing is done locally. With the tweaking provided by P4P algorithms, local file sharing increased almost tenfold, to 58 percent.

The P4P system requires Internet service providers to provide a number that acts as a weighting factor for network routing, so cooperation between the Internet service provider and the file-sharing host is necessary. But key to the system is that it does not force companies to disclose information about how they route Internet traffic. ###

Other authors of the paper are Haiyong Xie, a Yale graduate now working at Akamai Technologies Inc., Yanbin Liu, at IBM's Thomas J. Watson Research Center, and Avi Silberschatz, professor and chair of computer science at Yale. The UW research was supported by the National Science Foundation.

For more information, contact Krishnamurthy at (206) 616-0957 or arvind@cs.washington.edu and Yang at (203) 432-6400 or yry@cs.yale.edu.

The presentation was Thursday, Aug. 21, 1:45 p.m. at the Grand Hyatt Regency in Seattle. For more information on the presentation, see hccr.sigcomm.org/online/.

For more information on P4P, see Yale press release at opa.yale.edu/news/.

Contact: Hannah Hickey hickeyh@u.washington.edu 206-543-2580 University of Washington

7 comments:

Anonymous said...

Hi guys,

Could you please take this article down or at least get rid of Palin's mugshot - I will pay you $100 ?

arvind

sookietex said...

hey Arvind nice to hear from you, very interesting work your doing and i wish you all the best. i live in new york city and the network here, even on broadband cable is to slow.

i know Governor Palin is so hot you find it difficult to concentrate on dry theory and practice of distributed systems and computer networks :-) but you better get used to it, let's see 8 years as vp then 8 more as pres. :-)

at any rate thanks for stopping by and keep up the good work.

your friend sookietex, editor

Anonymous said...

Thank you for revealing Arvind as a double agent. I have always had suspicions.

Anonymous said...

Ed - Welcome to the dark side.

That wasn't me who put in the first comment - Someone was spoofing it. I am willing to pay $100 if good old sookietex removes the article and puts in a bigger pic of Palin (I have secret crush on her)

Arvind Krishnamurthy

Anonymous said...

Wow Arvind - You have a crush on Palin! ME TOO!

Lets get together and chat about this in the next systems seminar.

Ed Lazowska

sookietex said...

hey ed that's what we're here for, to reveal that soft underbelly :-)

hey arvind VERY hi-res version available on Governor Palin's Biography page :-)

fyi we feature science and tech every sunday last 3 were

Carnegie Mellon System Thwarts Internet Eavesdropping, Available as Free Download for Firefox 3.0 Browser, very interesting would highly recommend. NC State aims to develop 'internet for energy' at new NSF engineering research center and Galaxy Zoo -- an Internet superstar, asking computer users to apply their brain power to help sort one type of galaxy from another.

i say this by way of mentioning we have a small but motivated readership many .gov and .edu if we can help advance the ball, block, carry water, etc. we need this to happen asap.

thanks again, your friend, sookietex

Anonymous said...

Hey Sookietex,

Thanks for the link to pic - I will have it framed and put up in the UW cs building here.

In fact, it would be a great improvement over some of the art here...

Could you maybe fwd it to our head honcho - Hank. His taste in art is a bit wanting, but I think even he would fall for this!

Ed Lazowska

Post a Comment