NetworkInfrastructure: Difference between revisions

From OACISS Systems Wiki
Jump to navigation Jump to search
No edit summary
Line 3: Line 3:
The majority of OACISS hardware has two network connections: One which is publicly routable, and one that provides access to the storage/NFS network.
The majority of OACISS hardware has two network connections: One which is publicly routable, and one that provides access to the storage/NFS network.


Public connections are currently 1Gbps (this is likely to change in the relatively near future).
All main nodes except Grover currently have 10Gbps public and private ethernet.
The storage network is mixed 10Gbps/1Gbps. The majority of nodes currently have 10Gbps access.


The public interface of a machine is associated with its public DNS record under the nic.uoregon.edu domain (e.g. ''delphi.nic.uoregon.edu'', or just ''delphi'' from within UO).
= Network Hardware =
 
== Ethernet ==
 
The NIC Ethernet setup has four switches, two per cabinet, one each for private/storage and public networks.
 
* Dell N4032 x 2: Private network
** Switches are stacked. R35 unit is master (#1) and R34 is failover/member (#2)
** 40Gbps trunk link between racks (Fo1/1/1 and Fo2/1/1)
** 40GbE to Mnemosyne (Fo1/1/2)
* Dell PowerConnect 8024 + N4032: Public network
** PC8024 in R34
*** 10Gbps UOnet uplink (Te1/0/1)
*** 2 ports LAG trunk to R35 (Te1/0/21 & 22)
 
Ethernet interfaces are associated with DNS records:
* The public interface (e.g. ''FOO.nic.uoregon.edu'')
* The private interface with local-only DNS record (e.g. ''FOO.stor'')
* If present, the IPMI/ILO/BMC/etc interface with local-only DNS record (e.g. ''FOO-ipmi.stor'')


If a racked OACISS node is connected to the storage network, this interface will have a private name.stor DNS record (e.g. ''delphi.stor''). If the node has IPMI/LOM/iDRAC/ILO/BMC available, this will be reached under name-ipmi.stor (e.g. ''centaur-ipmi.stor''). These records resolve only from within the UO NIC subnet.
[[File:Oaciss_switches.png]]


== High speed networks ==
== High speed networks ==
Line 29: Line 46:
* Eagle
* Eagle
* Aurora (sx-aurora)
* Aurora (sx-aurora)
* Mnemosyne


All hosts are available using IPoIB as $HOST-ib; These designations are stored in /etc/hosts
All hosts are available using IPoIB as $HOST-ib; These designations are stored in /etc/hosts
= Network Hardware =
The NIC Ethernet setup has four switches, two per cabinet, one each for private/storage and public networks.
* Dell N4032 x 2: Private network
** Switches are stacked. R35 unit is master (#1) and R34 is failover/member (#2)
** 40Gbps trunk link between racks (Fo1/1/1 and Fo2/1/1)
** 40GbE to Mnemosyne (Fo1/1/2)
* Dell PowerConnect 8024 + N4032: Public network
** PC8024 in R34
*** 10Gbps UOnet uplink (Te1/0/1)
*** 2 ports LAG trunk to R35 (Te1/0/21 & 22)
[[File:Oaciss_switches.png]]

Revision as of 04:47, 5 August 2020

OACISS naming / DNS

The majority of OACISS hardware has two network connections: One which is publicly routable, and one that provides access to the storage/NFS network.

All main nodes except Grover currently have 10Gbps public and private ethernet.

Network Hardware

Ethernet

The NIC Ethernet setup has four switches, two per cabinet, one each for private/storage and public networks.

  • Dell N4032 x 2: Private network
    • Switches are stacked. R35 unit is master (#1) and R34 is failover/member (#2)
    • 40Gbps trunk link between racks (Fo1/1/1 and Fo2/1/1)
    • 40GbE to Mnemosyne (Fo1/1/2)
  • Dell PowerConnect 8024 + N4032: Public network
    • PC8024 in R34
      • 10Gbps UOnet uplink (Te1/0/1)
      • 2 ports LAG trunk to R35 (Te1/0/21 & 22)

Ethernet interfaces are associated with DNS records:

  • The public interface (e.g. FOO.nic.uoregon.edu)
  • The private interface with local-only DNS record (e.g. FOO.stor)
  • If present, the IPMI/ILO/BMC/etc interface with local-only DNS record (e.g. FOO-ipmi.stor)

Oaciss switches.png

High speed networks

Intel OmniPath

OACISS has a 100Gbps Intel OmniPath switch and several nodes with OP hardware:

  • Pegasus
  • Godzilla
  • Voltar
  • Delphi

They are accessed by $HOST-op; These are stored in /etc/hosts on these machines.

EDR InfiniBand

OACISS has a 36 port SB7700 100Gbps EDR Infiniband switch, with access to the following hosts:

  • Cyclops
  • Gorgon
  • Eagle
  • Aurora (sx-aurora)
  • Mnemosyne

All hosts are available using IPoIB as $HOST-ib; These designations are stored in /etc/hosts

Retrieved from "https://systems.nic.uoregon.edu/internal-wiki/index.php?title=NetworkInfrastructure&oldid=3075"