MyriNet Low Level Performance

Paderborn Center for Parallel Computing: Jens Simon (simon(at)upb.de)

  • MyriNet 10G low-level benchmarks

Benchmark programs

mx_dmabench, mx_pingpong, mx_stream

Author

MyriCom

program parameters

mx_dmabench     # DMA performance host memory to local NIC memory

mx_pingpong -d <slave hostname>:0 -M 2 -E 10500000

mx_stream  -d <slave hostname>:0  -V -l 8000000

Machine Network Software 1 Mbyte DMA
(4 kbyte per DMA)
send / recv
[Mbyte/s]		 mx_pingpong
0byte
[µs]		 mx_pingpong
1 Mbyte [Mbyte/s]		 mx_pingpong
64 kbyte [Mbyte/s]		 mx_pingpong
4 kbyte [Mbyte/s]		 mx_stream
8 MByte
[Mbyte/s]		 remarks
Dual 3.00GHz Intel Xeon 5160 Dual-Core, Intel 5000V,
4 x 2 GByte DDR2-555
 MyriCom MyriNet 2000
NIC:
M3F-PCIXD-2
Switch:
back-to-back		 mx-1.1.6 714 929           MX_RCACHE=1
Scientific Linux, 2.6.9-42                
03.04.2007                
                 
Dual 3.00GHz Intel Xeon 5160 Dual-Core, Intel 5000V,
4 x 2 GByte DDR2-555
 MyriCom MyriNet10G
NIC:
10G PCIe-8A-C
Switch:
10G-SW16LC-8C		 mx-1.2.0h-rc1 1359 1639 2,62 1064 718 336 1204 MX_RCACHE=1
RedHat AS4 U4, 2.6.9-42                
30.01.2007                
                 
Dual 3.00GHz Intel Xeon 5160 Dual-Core, Intel 5000V,
4 x 2 GByte DDR2-555		 MyriCom MyriNet10G
NIC:
10G PCIe-8A-C
Switch:
-		 mx-1.2.0h-rc1 1359 1639 2,26 1076 731 345 1204 MX_RCACHE=1
RedHat AS4 U4, 2.6.9-42                
30.01.2007                
                 
Dual 2.66 GHz Intel Xeon 5355 Quad-Core, Intel 5000X,
4 x 2 GByte DDR2-667		 MyriCom MyriNet10G
NIC:
10G PCIe-8A-C
Switch:
10G-SW16LC-8C		 mx-1.2.0h-rc1 1506 1517 2,58 1200 842 337 1221 MX_RCACHE=1
RedHat AS4 U4, 2.6.9-42                
23.01.2007                
                 
Dual 2.66 GHz Intel Xeon 5355 Quad-Core, Intel 5000X,
4 x 2 GByte DDR2-667		 MyriCom MyriNet10G
NIC: 10G PCIe-8A-C
Switch:
-		 mx-1.2.0g 1507 1513 2,23 1068 762 348 1222 MX_RCACHE=1
RedHat AS4 U4, 2.6.9-42                
11.01.2007                
                 

 

 

author:

MyriCom

name:

gm_allsize

slave parameter:

 --slave --size=20

master parameters:

 -u -lat -h <slave host name> -g --size=20

 -b -lat -h <slave host name> -g --size=20

 -u -bw -h <slave host name> -g --size=20

 --both-ways -bw -h <slave host name> -g --size=20

Machine NIC /
Switch		 Driver pt-to-pt gap [µs]
1 byte package		 HRT time [µs]
1 byte package		 unidir. bw [Mbyte/s]
1 045 byte package		 unidir. bw [Mbyte/s]
1 Mbyte package		 bidir. bw [Mbyte/s]
1 045 byte package		 bidir. bw [Mbyte/s]
1 Mbyte package		 remarks
INTEL
Dual 3.4 GHz Xeon EM64T
, FSB800
0.5 Gbyte DDR2-RAM
RedHat AS3 U2, 2.4.21-15.EL		 M3F-PCIXD-2 64/133 /
back-to-back		 GM-2.1.1
gcc 3.2.3		 6,28 7,44 121 248 161 490  
             
             
             
INTEL
Dual 3.4 GHz Xeon EM64T
, FSB800
0.5 Gbyte DDR2-RAM
RedHat AS3 U2, 2.4.21-15.EL		 M3F-PCIXD-2 64/100 /
back-to-back		 GM-2.1.1
gcc 3.2.3		 6,42 7,51 121 248 161 490  
             
             
             
INTEL
Dual 3.4 GHz Xeon EM64T
, FSB800
0.5 Gbyte DDR2-RAM
RedHat AS3 U2, 2.4.21-15.EL		 M3F-PCIXD-2 64/133 /
back-to-back		 GM-2.0.12
gcc 3.2.3		 6,34 8,1 147 248 172 489  
             
             
             
INTEL
Dual 3.4 GHz Xeon EM64T
, FSB800
0.5 Gbyte DDR2-RAM
RedHat AS3 U2, 2.4.21-15.EL		 M3F-PCIXD-2 64/100 /
back-to-back		 GM-2.0.12
gcc 3.2.3		 6,33 8,23 144 248 171 489  
             
             
             

 

 

benchmark program:

gm_debug

author:

MyriCom

parameter:

 -L

Machine NIC Driver bus read [Mbyte/s]
4 kByte pack.		 bus write [Mbyte/s]
4 kByte pack.		 bus r/w [Mbyte/s]
4 kByte pack.		 remarks
INTEL
Dual 3.4 GHz Xeon EM64T
FSB800
0.5 Gbyte DDR2-RAM
RedHat AS3 U2, 2.4.21-15.EL		 M3F-PCIXD-2 64/133 GM-2.1.1
gcc 3.2.3		 856 1044 471  
       
       
       
INTEL
Dual 3.4 GHz Xeon EM64T
FSB800
0.5 Gbyte DDR2-RAM
RedHat AS3 U2, 2.4.21-15.EL		 M3F-PCIXD-2 64/100 GM-2.1.1
gcc 3.2.3		 700 784 370  
       
       
       
Newisys 2100
Dual 1.4 GHz Opteron
chip set AMD 8131
2 x 1 Gbyte DDR-RAM		 M3F-PCI2C 64/66 GM-2.0
gcc 3.2.3 -m64		 461 489    
       
       
       
Newisys 2100
Dual 1.4 GHz Opteron
chip set AMD 8131
2 x 1 Gbyte DDR-RAM		 M3F-PCIXD-2 64/66 GM-2.0
gcc 3.2.3 -m64		 483 517   PCI slot used, not PCI-X!
       
       
       
Supermicro X5DA8
Dual 2.8 GHz Xeon
chip set Intel E7505, FSB533
2 Gbyte DDR-RAM		 M3F-PCIXD-2 64/100 GM-2.0
gcc 2.96 		673 782    
       
       
       
HP ZX6000
Dual 1 GHz Itanium2
chip set HP ZX1
8 Gbyte DDR-RAM		 MFM-PCI64B 64/66 GM-2.0
gcc 2.96 		458* 498*   * 16 kByte packages
       
       
       
HP ZX6000
Dual 1 GHz Itanium2
chip set HP ZX1
8 Gbyte DDR-RAM		 M3F-PCIXD-2 64/133 GM-2.0
gcc 2.96 		482* 501*   * 16 kByte packages
       
       
       
My Staffweb