NetworkInfrastructure: Difference between revisions

From OACISS Systems Wiki
Jump to navigation Jump to search
No edit summary
Line 34: Line 34:
= Network Hardware =
= Network Hardware =


The NIC switching infrastructure has a total of five switches currently.
The NIC Ethernet setup has four switches, two per cabinet, one each for private/storage and public networks.


A total of three are utilized for the private network (red):
* Dell N4032 x 2: Private network
* Dell PowerConnect 8024 (24 port 10Gbps)
** Switches are stacked. R35 unit is master (#1) and R34 is failover/member (#2)
** Web monitor interface at 172.17.202.249
** 40Gbps trunk link between racks (Fo1/1/1 and Fo2/1/1)
* Cisco Catalyst 2960s (24 port 1Gb + two 10Gbps SFP+ trunk ports)
** 40GbE to Mnemosyne (Fo1/1/2)
** Console connection available on Mecha at ttyS0
* Dell PowerConnect 8024 + N4032: Public network
** 10Gbps optical trunk port to Powerconnect
** PC8024 in R34
*** If connection fails, Cisco side may "lose" transmit power; this can be restored by turning the sfp port off and back on
*** 10Gbps UOnet uplink (Te1/0/1)
** Backup 1Gbe wire connection to Cisco port 17 [deactivated by STP]
*** 2 ports LAG trunk to R35 (Te1/0/21 & 22)
* The R35 private network switch is present to allow IPMI ports on R35 hosts to use shorter cables


Two switches are used for the public-facing (128.223.202.0/25) network.
* R34 TOR switch (HP A5500) has the actual WAN link
* The R35 switch has one 1Gbps link to the R34 switch


Currently experiencing difficulties with 10Gbps uplink on A5500. A5500 has a total of 6 10Gbps-capable ports, all SFP+.


[[File:Oaciss_switches.png]]
[[File:Oaciss_switches.png]]

Revision as of 03:57, 5 August 2020

OACISS naming / DNS

The majority of OACISS hardware has two network connections: One which is publicly routable, and one that provides access to the storage/NFS network.

Public connections are currently 1Gbps (this is likely to change in the relatively near future). The storage network is mixed 10Gbps/1Gbps. The majority of nodes currently have 10Gbps access.

The public interface of a machine is associated with its public DNS record under the nic.uoregon.edu domain (e.g. delphi.nic.uoregon.edu, or just delphi from within UO).

If a racked OACISS node is connected to the storage network, this interface will have a private name.stor DNS record (e.g. delphi.stor). If the node has IPMI/LOM/iDRAC/ILO/BMC available, this will be reached under name-ipmi.stor (e.g. centaur-ipmi.stor). These records resolve only from within the UO NIC subnet.

High speed networks

Intel OmniPath

OACISS has a 100Gbps Intel OmniPath switch and several nodes with OP hardware:

  • Pegasus
  • Godzilla
  • Voltar
  • Delphi

They are accessed by $HOST-op; These are stored in /etc/hosts on these machines.

EDR InfiniBand

OACISS has a 36 port SB7700 100Gbps EDR Infiniband switch, with access to the following hosts:

  • Cyclops
  • Gorgon
  • Eagle
  • Aurora (sx-aurora)

All hosts are available using IPoIB as $HOST-ib; These designations are stored in /etc/hosts

Network Hardware

The NIC Ethernet setup has four switches, two per cabinet, one each for private/storage and public networks.

  • Dell N4032 x 2: Private network
    • Switches are stacked. R35 unit is master (#1) and R34 is failover/member (#2)
    • 40Gbps trunk link between racks (Fo1/1/1 and Fo2/1/1)
    • 40GbE to Mnemosyne (Fo1/1/2)
  • Dell PowerConnect 8024 + N4032: Public network
    • PC8024 in R34
      • 10Gbps UOnet uplink (Te1/0/1)
      • 2 ports LAG trunk to R35 (Te1/0/21 & 22)


Oaciss switches.png

Retrieved from "https://systems.nic.uoregon.edu/internal-wiki/index.php?title=NetworkInfrastructure&oldid=3074"