: Software Routed Network Configuration
The CESDIS Beowulf parallel workstation architecture, an emerging standard for PopC clusters, realizes high-performance distributed computing from strictly commodity hardware. The experiments detailed here were performed on the original Beowulf prototype, a 16 node cluster with each node running a copy of the Linux operating system and configured as follows:
The original Beowulf network topology consisted of a pair of Ethernet busses that spanned the entire cluster and operated in parallel as a single virtual bus. This configuration's aggregate network bandwith of 20Mbps was a serious limiting factor on the cluster's performance. The alternative configurations explored here retain the 10Mbps technology and dual NICs per node of the original prototype but increase the aggregate network bandwidth by creating eight seperate Ethernet busses. Each node attaches to two busses, each bus spans four nodes as shown in Figure 1. A variation on this interconnect scheme, depicted in Figure 2, adds two four-port Ethernet switches; the four vertical network segments are connected to each other this way as are the horizontal networks. While the theoretical maximum aggregate bandwidth has quadrupled, there are constraints on the effective aggregate bandwidth and performance.
The original Beowulf network topology (Bonded Dual Net) provided each node with a direct route to all nodes in the cluster. From Figure 1 we can see that the unswitched alternative topology (Routed Mesh) allows any node in the cluster to communicate directly with only six other nodes (local nodes), the three nodes with which it shares a vertical network segment and the three nodes with which it shares a horizontal network segment. For a node to communicate with the seven remaining nodes (remote nodes), an intermediate node which is local to both the sender and receiver must act as a router, forwarding the communication across network segments. Any two nodes may communicate using no more than one intermediate router node. The two consequences of the alternative topologies are:
: Switched Network Configuration
The experiments described in Section 4 do not include replicated packets in their measure of aggregate throughput. The bandwidth consumed by these extra packets actually reflects the degradation in performance for remote node transactions discussed in Section 5. Another factor reflected by the experiments is that the Beowulf architecture makes no distinction between I/O nodes and compute nodes. Packet routing consumes both CPU cycles and bus to memory bandwidth that causes contention with other operations such as file I/O.
In the experiments, the logical topology is static and constructed such that the routing load is distributed fairly for a uniform traffic case. The routed mesh scheme requires no hardware not already necessary for the original bonded dual net topology. The switched mesh topologies (Switched Mesh, Bonded Switched Mesh) retain the eight-segment basic physical interconnect of the routed scheme, but add high-speed Ethernet switches to perform the routing between segments. To avoid becoming a bottleneck, a switch backplane must support more bandwidth than the network segments. The experiments conducted on switched topolgies would reveal any impact of the switch backplane on aggregate bandwidth.
A notable feature of the original logical interconnect was channel bonding (Section 3); this allows two or more network segments to be joined into a single logical segment in a fashion completely transparent to applications. While the original interconnect (Bonded Dual Net) bonded two identical networks into a single logical segment, channel bonding can be applied whenever there are multiple possible routes between two nodes. This technique has been applied to the switched interconnect scheme, and its effect on performance is presented and discussed in Sections 4 and 5.
