| 1 |
Infiniband Fabric Simulator - ibsim |
| 2 |
=================================== |
| 3 |
|
| 4 |
Basic Description and main features. |
| 5 |
----------------------------------- |
| 6 |
|
| 7 |
ibsim emulates the fabric behavior by using MAD communication with the |
| 8 |
SM/SA and the PerfMgr. This simple tool is ideally suitable for various |
| 9 |
research, development, debug and testing tasks where IB subnet |
| 10 |
management is involved. |
| 11 |
|
| 12 |
For generation and reception of this MAD traffic it replaces the |
| 13 |
/dev/umadX file descriptor interface between libibumad and ib_umad |
| 14 |
kernel module by using preloaded libumad2sim.so shared library (umad2sim |
| 15 |
wrapper - part of ibsim distribution). This conveys MADs to/from |
| 16 |
SM/SA/PerfMgr to ibsim. |
| 17 |
|
| 18 |
|
| 19 |
OpenSM Diags utils App (using libibumad) |
| 20 |
| | | |
| 21 |
V V | |
| 22 |
libosmvendor libibmad | |
| 23 |
| | | |
| 24 |
V V V |
| 25 |
libibumad |
| 26 |
| |
| 27 |
V |
| 28 |
--> LD_PRELOAD=libumad2sim.so <--> ibsim |
| 29 |
--------------------------------- |
| 30 |
ib stack (ib_umad.ko) |
| 31 |
|
| 32 |
|
| 33 |
Any libibumad based application may work with ibsim. Kernel support |
| 34 |
and userspace application recompilation are not required. |
| 35 |
|
| 36 |
ibsim works locally via unix sockets or remotely via inet sockets. |
| 37 |
|
| 38 |
For fabric topology description, ibsim uses a text file in the format |
| 39 |
compatible with ibnetdiscover command output (for examples see |
| 40 |
net-examples/) and it can be generated using a real cluster snapshot. |
| 41 |
|
| 42 |
ibsim has a simple console command interface and can simulate random |
| 43 |
packets drops and link up/down events. It is possible to run batch |
| 44 |
commands from file via pipe or named fifo. |
| 45 |
|
| 46 |
|
| 47 |
Building and using ibsim |
| 48 |
------------------------ |
| 49 |
|
| 50 |
1. cd to unpacked simulator directory. |
| 51 |
|
| 52 |
2. make ibsim and umad2sim wrapper: |
| 53 |
|
| 54 |
$ make |
| 55 |
|
| 56 |
Notes: |
| 57 |
- by default this will build ibsim against installed in /usr/local |
| 58 |
version of libib* libraries. If you like to build it against |
| 59 |
development tree use IB_DEV_DIR variable (or export it into |
| 60 |
environment): |
| 61 |
|
| 62 |
$ make IB_DEV_DIR=${HOME}/src/management |
| 63 |
|
| 64 |
- 'make dep', 'make clean' and 'make install' are available now |
| 65 |
|
| 66 |
3. run ibsim: |
| 67 |
|
| 68 |
$ ./ibsim/ibsim -s <netfile> |
| 69 |
|
| 70 |
, where <netfile> is network definition file, it is compatible with |
| 71 |
output of ibnetdiscover command (+some more options are available as |
| 72 |
well). Some examples can be found under 'net-examples' directory. |
| 73 |
|
| 74 |
Notes: |
| 75 |
- '-s' option is in order to start the IB network immediately, the |
| 76 |
ibsim console may be used instead. |
| 77 |
- '-h' option shows ibsim command line usage and 'help' and '?' |
| 78 |
ibsim console commands show brief console help. |
| 79 |
|
| 80 |
4. run libibumad based application with preloaded umad2sim wrapper: |
| 81 |
|
| 82 |
$ LD_PRELOAD=./umad2sim/libumad2sim.so ibnetdiscover |
| 83 |
|
| 84 |
, or |
| 85 |
|
| 86 |
$ LD_PRELOAD=./umad2sim/libumad2sim.so opensm -f - |
| 87 |
|
| 88 |
Notes: |
| 89 |
- in order to run OpenSM as non-privileged user you may need to |
| 90 |
export OSM_CACHE_DIR variable and to use '-f' option in order to |
| 91 |
specify writable path to OpenSM log file. |
| 92 |
- Point of attachment is indicated by SIM_HOST environment variable. |
| 93 |
If not specified, first entry in topology file is used. For OpenSM, |
| 94 |
if -g option is used, it must be the same as this. |
| 95 |
|
| 96 |
5. Enjoy and comment. |
| 97 |
|
| 98 |
|
| 99 |
Feedback. |
| 100 |
--------- |
| 101 |
|
| 102 |
Please send your feedback and patches to the maintainer: |
| 103 |
Tamir Ronen <[email protected]> |