You need to have two or more NICs on your server and multiple distribution switches on network in order to have a redundant network connection for your server. Depending on the operating system your server is running, different configurations (and names) are used. There are usualy two possible schemes to do this, one using load balancing method, and the other using fail-over option.

In the following lines you can find both redhat and solaris configuration examples to implement this.

RedHat Bonding (Linux network channel bonding)

Make sure tha you have the bonding driver:

[root@system ~]# lsmod|grep bonding
bonding        65896    0

If you don't, enable it appending the following lines to your /etc/modprobe.conf:

alias bond0 bonding
options bond0 mode=1 miimon=100

Where miimon is the number of miliseconds that kernel checks the availability of the NIC link. Make sure that your NIC supports link detection:

[root@system ~]# ethtool eth0 |grep -i "link detected"
Link detected: yes

mode option refers to bonding policy, which is the key option to switch between load-balance and fail-over schemes. Default is 0 (balance-rr), and 1 refers to active-backup policy (aka fail-over).

Reference: /usr/src/linux/Documentation/networking/bonding.txt

After module configuration, you have to configure network scripts. Create a bond0 interface for master, and "attach" both eth0 and eth1 interfaces as slaves on bond0:

[root@system ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
IPADDR=123.45.67.89
NETMASK=255.255.255.0
NETWORK=123.45.67.0
BROADCAST=123.45.67.255
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
[root@system ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
HWADDR=01:23:45:67:89:ab
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
[root@system ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
HWADDR=01:23:45:67:89:ac
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes

and finally reload your network settings:

[root@system ~]# /etc/rc.d/init.d/network restart

Monitor the configuration using proc filesystem:

[root@system ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v2.6.3-rh (June 8, 2005)

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 01:23:45:67:89:ac

make sure you have a physical console attached to server, in order to recover in case of network configuration failure.

Solaris IP Multipathing

Solaris implementation is quite different than linux. You assign an IP address to each of your NICs, and a third one to use it as a virtual interface (such as bond0) to refer to the bonded channel.
In the following example, i have two NICs (hme0 & hme1) using 123.45.67.89 and 123.45.67.90, and i use 123.45.67.91 as the bonded IP address.

Firstly make sure tha you have local-mac-address option enabled in your system:

[root@system ~]# eeprom | grep local-mac
local-mac-address?=true

If not, do it!
[root@system ~]# eeprom 'local-mac-address?=true'

Append the following to your NICs configurations (and make sure that are both plumbed using ifconfig hme{0|1} plumb):

[root@system ~]# cat hostname.hme0
123.45.67.91 netmask + broadcast + group gigabit + failover up \
addif 123.45.67.89 deprecated -failover netmask + broadcast + up
[root@system ~]# cat hostname.hme1
123.45.67.90 netmask + broadcast + deprecated -failover standby group gigabit up

Finally, assign options to the multipathing driver:

[root@system ~]# grep -v "^#" /etc/default/mpathd
FAILURE_DETECTION_TIME=100
FAILBACK=yes
TRACK_INTERFACES_ONLY_WITH_GROUPS=yes

where failure_detection_time is in miliseconds, just as miimon in redhat

In order to enable your configuration to the running system, just type the contents of /etc/hostname.hme* to your shell:

[root@system ~]# ifconfig hme0 123.45.67.91 netmask + broadcast + group gigabit + failover up \
addif 123.45.67.89 deprecated -failover netmask + broadcast + up
[root@system ~]# ifconfig hme1 123.45.67.90 netmask + broadcast + deprecated \
-failover standby group gigabit up