NetworkInfrastructure: Difference between revisions
No edit summary |
No edit summary |
||
| Line 12: | Line 12: | ||
== High speed networks == | == High speed networks == | ||
=== Intel OmniPath === | |||
Pegasus | OACISS has a 100Gbps Intel OmniPath switch and several nodes with OP hardware: | ||
* Pegasus | |||
* Godzilla | |||
* Voltar | |||
* Delphi | |||
They are accessed by $HOST-op; These are stored in /etc/hosts on these machines. | |||
= Hardware = | === EDR InfiniBand === | ||
OACISS has a 36 port SB7700 100Gbps EDR Infiniband switch, with access to the following hosts: | |||
* Cyclops | |||
* Gorgon | |||
* Eagle | |||
* Aurora (sx-aurora) | |||
All hosts are available using IPoIB as $HOST-ib; These designations are stored in /etc/hosts | |||
= Network Hardware = | |||
The NIC switching infrastructure has a total of five switches currently. | The NIC switching infrastructure has a total of five switches currently. | ||
A total of three are utilized for the private network (red): | A total of three are utilized for the private network (red): | ||
* Dell PowerConnect 8024 | * Dell PowerConnect 8024 (24 port 10Gbps) | ||
** Web monitor interface at 172.17.202.249 | ** Web monitor interface at 172.17.202.249 | ||
* | * Cisco Catalyst 2960s (24 port 1Gb + two 10Gbps SFP+ trunk ports) | ||
** Console connection available on Mecha at ttyS0 | ** Console connection available on Mecha at ttyS0 | ||
** 10Gbps optical trunk port to Powerconnect | ** 10Gbps optical trunk port to Powerconnect | ||
*** If connection fails, | *** If connection fails, Cisco side may "lose" transmit power; this can be restored by turning the sfp port off and back on | ||
** Backup 1Gbe wire connection to Cisco port 17 [deactivated by STP] | ** Backup 1Gbe wire connection to Cisco port 17 [deactivated by STP] | ||
* The R35 private network switch is present to allow IPMI ports on R35 hosts to use shorter cables | * The R35 private network switch is present to allow IPMI ports on R35 hosts to use shorter cables | ||
Two switches are used for the public-facing (128.223.202.0/25) network. | Two switches are used for the public-facing (128.223.202.0/25) network. | ||
* | * R34 TOR switch (HP A5500) has the actual WAN link | ||
* The R35 switch has one 1Gbps link to the R34 switch | * The R35 switch has one 1Gbps link to the R34 switch | ||
Currently experiencing difficulties with 10Gbps uplink on A5500. A5500 has a total of 6 10Gbps-capable ports, all SFP+. | |||
[[File:Oaciss_switches.png]] | [[File:Oaciss_switches.png]] | ||
Revision as of 23:24, 6 July 2020
OACISS naming / DNS
The majority of OACISS hardware has two network connections: One which is publicly routable, and one that provides access to the storage/NFS network.
Public connections are currently 1Gbps (this is likely to change in the relatively near future). The storage network is mixed 10Gbps/1Gbps. The majority of nodes currently have 10Gbps access.
The public interface of a machine is associated with its public DNS record under the nic.uoregon.edu domain (e.g. delphi.nic.uoregon.edu, or just delphi from within UO).
If a racked OACISS node is connected to the storage network, this interface will have a private name.stor DNS record (e.g. delphi.stor). If the node has IPMI/LOM/iDRAC/ILO/BMC available, this will be reached under name-ipmi.stor (e.g. centaur-ipmi.stor). These records resolve only from within the UO NIC subnet.
High speed networks
Intel OmniPath
OACISS has a 100Gbps Intel OmniPath switch and several nodes with OP hardware:
- Pegasus
- Godzilla
- Voltar
- Delphi
They are accessed by $HOST-op; These are stored in /etc/hosts on these machines.
EDR InfiniBand
OACISS has a 36 port SB7700 100Gbps EDR Infiniband switch, with access to the following hosts:
- Cyclops
- Gorgon
- Eagle
- Aurora (sx-aurora)
All hosts are available using IPoIB as $HOST-ib; These designations are stored in /etc/hosts
Network Hardware
The NIC switching infrastructure has a total of five switches currently.
A total of three are utilized for the private network (red):
- Dell PowerConnect 8024 (24 port 10Gbps)
- Web monitor interface at 172.17.202.249
- Cisco Catalyst 2960s (24 port 1Gb + two 10Gbps SFP+ trunk ports)
- Console connection available on Mecha at ttyS0
- 10Gbps optical trunk port to Powerconnect
- If connection fails, Cisco side may "lose" transmit power; this can be restored by turning the sfp port off and back on
- Backup 1Gbe wire connection to Cisco port 17 [deactivated by STP]
- The R35 private network switch is present to allow IPMI ports on R35 hosts to use shorter cables
Two switches are used for the public-facing (128.223.202.0/25) network.
- R34 TOR switch (HP A5500) has the actual WAN link
- The R35 switch has one 1Gbps link to the R34 switch
Currently experiencing difficulties with 10Gbps uplink on A5500. A5500 has a total of 6 10Gbps-capable ports, all SFP+.
