| 1 |
The scripts in this directory are experimental and are used to create testenvs |
| 2 |
in separate linux namespaces. This avoids the need for socket-wrapper. |
| 3 |
|
| 4 |
What are Namespaces |
| 5 |
=================== |
| 6 |
Namespaces allow the kernel to segregate its system resources (files, CPU, |
| 7 |
etc), so that different processes only see the set of resources they are |
| 8 |
allowed to use. There are several different types of namespace: network, |
| 9 |
user, process, file, IPC, and so on. |
| 10 |
|
| 11 |
Key points to grasp are: |
| 12 |
* Each type of namespace gets managed separately by the kernel, i.e. process |
| 13 |
namespaces are managed separately to network namespaces, which are separate |
| 14 |
to user namespaces. These scripts give each testenv its own network namespace, |
| 15 |
but otherwise they all still share the same user/process/etc namespace. |
| 16 |
(In future, we may want to give each testenv its own process and user |
| 17 |
namespace, to better mimic a production DC). |
| 18 |
* Namespaces are created using the 'unshare' utility. The new selftest |
| 19 |
namespaces are anonymous/nameless, and so the different namespaces are |
| 20 |
identified by the PID of the processes running within the namespace |
| 21 |
(typically samba). |
| 22 |
* Linux supports nesting namespaces within namespaces. In this case, each |
| 23 |
testenv DC has its own network namespace, which is a child of the overarching |
| 24 |
selftest namespace (which itself is a child of whatever namespace you run |
| 25 |
'make test' from - usually this would be the root namespace). |
| 26 |
|
| 27 |
How does it work? |
| 28 |
================= |
| 29 |
Normally when 'make test' is run, every testenv uses a 10.53.57.x IP address |
| 30 |
and socket-wrapper passes the packets between them. |
| 31 |
|
| 32 |
With namespaces, we also use 10.53.57.x IP addresses but have the packets pass through |
| 33 |
the kernel's IP stack normally, as it forwards them between namespaces. |
| 34 |
|
| 35 |
We use veth interfaces for this. veth is a type of virtual interface supported |
| 36 |
by the kernel. veth interfaces come in pairs, and act as a tunnel - any packets |
| 37 |
sent on a veth interface simply end up as received packets on the pair veth |
| 38 |
interface. |
| 39 |
|
| 40 |
We create a new veth interface pair for each testenv, and use them to connect |
| 41 |
up the namespaces. One end of the veth pair is added to the main selftest |
| 42 |
namespace, and the other end is added to a new namespace that we'll run |
| 43 |
samba in. E.g. |
| 44 |
|
| 45 |
selftest.pl veth21-br ------------------------ veth21 samba (ad_dc_ntvfs) |
| 46 |
10.53.57.11 10.53.57.21 |
| 47 |
Namespace 1 Namespace 2 |
| 48 |
|
| 49 |
However, we need to run multiple different testenvs and have them talk to |
| 50 |
each other. So to do this, we need a bridge interface ('selftest0') to connect |
| 51 |
up the namespaces, which essentially just acts as a hub. So connecting together |
| 52 |
multiple testenvs looks more like this: |
| 53 |
|
| 54 |
selftest.pl +-- veth21-br ------------------------ veth21 samba (ad_dc_ntvfs) |
| 55 |
| 10.53.57.21 |
| 56 |
selftest0 --+ Namespace 2 |
| 57 |
10.53.57.11 | |
| 58 |
+-- veth22-br ------------------------ veth22 samba (vampire_dc) |
| 59 |
10.53.57.22 |
| 60 |
Namespace 1 Namespace 3 |
| 61 |
|
| 62 |
The veth interfaces are named vethX and vethX-br, where X is the |
| 63 |
SOCKET_WRAPPER_DEFAULT_IFACE for the testenv. The vethX-br interface is always |
| 64 |
added to the selftest0 bridge interface. |
| 65 |
|
| 66 |
How do I use it? |
| 67 |
================ |
| 68 |
To use namespaces instead of socket-wrapper, just add 'USE_NAMESPACES=1' to the |
| 69 |
make command, e.g. |
| 70 |
|
| 71 |
To run the 'quick' test cases using namespaces: |
| 72 |
USE_NAMESPACES=1 make test TESTS=quick |
| 73 |
|
| 74 |
To setup an ad_dc testenv using namespaces: |
| 75 |
USE_NAMESPACES=1 SELFTEST_TESTENV=ad_dc make testenv |
| 76 |
|
| 77 |
You can connect secondary shells to the namespace your testenv is running in. |
| 78 |
The command to do this is a little complicated, so a helper 'nsenter.sh' script |
| 79 |
gets autogenerated when the testenv is created. E.g. to connect to the testenv |
| 80 |
that the ad_dc is running in, use: |
| 81 |
./st/ad_dc/nsenter.sh |
| 82 |
|
| 83 |
This script also sets up the shell with all the same $SERVER/$USERNAME/etc |
| 84 |
variables that you normally get in xterm. |
| 85 |
|
| 86 |
To run the ad-dc-backup autobuild job using namespaces: |
| 87 |
USE_NAMESPACES=1 script/autobuild.py samba-ad-dc-backup --verbose --nocleanup \ |
| 88 |
--keeplogs --tail --testbase /tmp/samba-testbase |
| 89 |
|
| 90 |
Using the customdc testenv, you can basically now essentially your own |
| 91 |
light-weight samba VM. E.g. |
| 92 |
MY_BACKUP=/home/$USER/samba-backup-prod-domain.tar.bz2 |
| 93 |
USE_NAMESPACES=1 BACKUP_FILE=$MY_BACKUP SELFTEST_TESTENV=customdc make testenv |
| 94 |
|
| 95 |
You can then talk to that DC in any other shell by using |
| 96 |
./st/customdc/nsenter.sh which enters the DC's network namespace (with |
| 97 |
all the $SERVER/etc env variables defined). |
| 98 |
|
| 99 |
How to join VMs to the testenv |
| 100 |
---------------------------------------- |
| 101 |
I haven't tried this (beyond basic IP connectivity), but using namespaces it |
| 102 |
should now be possible to connect a Windows VM to a Samba testenv. |
| 103 |
|
| 104 |
1. Work out the main selftest.pl namespace PID manually, e.g. |
| 105 |
SELFTEST_PID= ps waux | grep selftest.pl |
| 106 |
|
| 107 |
2. Create a new veth to bridge between the selftest namespace and your PC's |
| 108 |
default namespace: |
| 109 |
sudo ip link add dev testenv-veth0 type veth peer name testenv-veth1 |
| 110 |
|
| 111 |
3. Move one end of the veth tunnel into the selftest namespace: |
| 112 |
sudo ip link set testenv-veth1 netns $SELFTEST_PID |
| 113 |
|
| 114 |
4. Configure the veth end in the default namespace to be in the same subnet |
| 115 |
as the selftest network: |
| 116 |
sudo ip link set dev testenv-veth0 up |
| 117 |
sudo ip addr add 10.53.57.63/24 dev testenv-veth0 |
| 118 |
|
| 119 |
5. Enter the selftest namespace, bring that end of the pipe up, and add it to |
| 120 |
to the main selftest0 bridge (that connects all the DCs together). We also need |
| 121 |
to add a default route from selftest back to your PC's default namespace. |
| 122 |
nsenter -t $SELFTEST_PID --net --user --preserve-credentials |
| 123 |
ip link set dev testenv-veth1 up |
| 124 |
ip link set testenv-veth1 master selftest0 |
| 125 |
ip route add default via 10.53.57.63 |
| 126 |
logout |
| 127 |
|
| 128 |
Your Windows VM and samba testenv should now be able to talk to each |
| 129 |
other over IP! |
| 130 |
|
| 131 |
6. The other step is to get DNS working. You probably need to add dns_hub |
| 132 |
(10.53.57.64) as a nameserver (at least on your Windows VM). |
| 133 |
|
| 134 |
This should work for using RSAT tools on samba, or joining Windows to Samba |
| 135 |
(depending on the schema version). Joining samba to Windows is a bit more |
| 136 |
tricky, as the namespaces are tied to the *running* samba process. |
| 137 |
|
| 138 |
What you'd probably want to do is run the join command to the windows VM |
| 139 |
outside of testenv, create an offline backup-file of the resulting DB, and |
| 140 |
then plug that backup-file into the customdc testenv. (And then follow the |
| 141 |
above veth/bridge steps to join samba to the VM). |
| 142 |
|
| 143 |
Note that the namespace disappears once you stop the testenv, so you'd |
| 144 |
need to do the above steps with creating the veth interface every time |
| 145 |
you restarted the testenv. |
| 146 |
|
| 147 |
Known limitations |
| 148 |
================= |
| 149 |
- When running a testenv, sometimes xterm can fail to startup, due to a |
| 150 |
permissions problem with /dev/pts. This seems to be a particular problem |
| 151 |
with the 'none' testenv. |
| 152 |
A short-term work-around is to use a terminal that doesn't try to access |
| 153 |
/dev/pts, e.g. just use bash as the terminal: |
| 154 |
TERMINAL=bash TERMINAL_ARGS='--norc' USE_NAMESPACES=1 \ |
| 155 |
SELFTEST_TESTENV=none make testenv |
| 156 |
- Some test cases rely on socket-wrapper, so will fail when run using |
| 157 |
namespaces. |
| 158 |
- Currently USE_NAMESPACES maps you (i.e. $USER) to root in the new namespace. |
| 159 |
This means any test cases that rely on being a non-root user will fail (i.e. |
| 160 |
anything that fails under 'sudo make test' will also fail with namespaces). |
| 161 |
- Namespaces should work within docker, but currently the 'unshare' system |
| 162 |
call is disallowed on the gitlab CI runners. |