[cluster-l] Networking Equipment

[Jay A. Kreibich]

On Wed, Mar 28, 2007 at 04:54:09PM -0500, Jim Phillips scratched on the wall:
> 
> I think jumbo frames were more of a benefit when gigabit was new, cards 
> were dumb, and processors were slow.  I can measure 900 Mbit/s with 
> netperf on our cluster so I don't think jumbo frames would help much.

  Yes and no.  While you might get close to max capacity without jumbo
  frames, that's not useful if you're using 95% of your available CPU
  to do it.  In terms of the network stack (especially TCP), a very
  large percent of the processing costs are "per packet" and not "per
  byte."  This is even more true if you have cards that off-load the
  IP checksums and such.

  From that standpoint, jumbo frames have the ability to cut the
  resources consumed by the networking stack to 1/6th their old cost.
  Since the primary goal of a cluster is to spend CPU resources solving
  problems, rather than running house-keeping tasks, that can be a
  significant win.

  On the other hand, if your cluster isn't filling 1500 byte frames to
  start with, jumbos will buy you nothing but configuration headaches.
  As with so many things, it depends a lot on the applications you're
  trying to run and how those systems utilize the interconnect.

    -j

-- 
Jay A. Kreibich < J A Y  @  K R E I B I.C H >

"'People who live in bamboo houses should not throw pandas.' Jesus said that."
   - "The Ninja", www.AskANinja.com, "Special Delivery 10: Pop!Tech 2006"