| 1 |
.. |
| 2 |
This work is licensed under a Creative Commons Attribution 3.0 Unported |
| 3 |
License. |
| 4 |
|
| 5 |
http://creativecommons.org/licenses/by/3.0/legalcode |
| 6 |
|
| 7 |
============================ |
| 8 |
Tap-as-a-Service for Neutron |
| 9 |
============================ |
| 10 |
|
| 11 |
|
| 12 |
Launchpad blueprint: |
| 13 |
|
| 14 |
https://blueprints.launchpad.net/neutron/+spec/tap-as-a-service |
| 15 |
|
| 16 |
This spec explains an extension for the port mirroring functionality. Port |
| 17 |
mirroring involves sending a copy of packets ingressing and/or egressing one |
| 18 |
port (where ingress means entering a VM and egress means leaving a VM) to |
| 19 |
another port, (usually different from the packet's original destination). |
| 20 |
A port could be attached to a VM or a networking resource like router. |
| 21 |
|
| 22 |
While the blueprint describes the functionality of mirroring Neutron ports as |
| 23 |
an extension to the port object, the spec proposes to offer port mirroring as a |
| 24 |
service, which will enable more advanced use-cases (e.g. intrusion detection) |
| 25 |
to be deployed. |
| 26 |
|
| 27 |
The proposed port mirroring capability shall be introduced in Neutron as a |
| 28 |
service called "Tap-as-a-Service". |
| 29 |
|
| 30 |
Problem description |
| 31 |
=================== |
| 32 |
|
| 33 |
Neutron currently does not support the functionality of port mirroring for |
| 34 |
tenant networks. This feature will greatly benefit tenants and admins, who |
| 35 |
want to debug their virtual networks and gain visibility into their VMs by |
| 36 |
monitoring and analyzing the network traffic associated with them (e.g. IDS). |
| 37 |
|
| 38 |
This spec focuses on mirroring traffic from one Neutron port to another; |
| 39 |
future versions may address mirroring from a Neutron port to an arbitrary |
| 40 |
interface (not managed by Neutron) on a compute host or the network controller. |
| 41 |
|
| 42 |
Different usage scenarios for the service are listed below: |
| 43 |
|
| 44 |
1. Tapping/mirroring network traffic ingressing and/or egressing a particular |
| 45 |
Neutron port. |
| 46 |
2. Tapping/mirroring all network traffic on a tenant network. |
| 47 |
3. Tenant or admin will be able to do tap/traffic mirroring based on a |
| 48 |
policy rule and set destination as a Neutron port, which can be linked |
| 49 |
to a virtual machine as normal Nova operations or to a physical machine |
| 50 |
via l2-gateway functionality. |
| 51 |
4. Admin will be able to do packet level network debugging for the virtual |
| 52 |
network. |
| 53 |
5. Provide a way for real time analytics based on different criteria, like |
| 54 |
tenants, ports, traffic types (policy) etc. |
| 55 |
|
| 56 |
Note that some of the above use-cases are not covered by this proposal, at |
| 57 |
least for the first step. |
| 58 |
|
| 59 |
|
| 60 |
Proposed change |
| 61 |
=============== |
| 62 |
|
| 63 |
The proposal is to introduce a new Neutron service plugin, called |
| 64 |
"Tap-as-a-Service", |
| 65 |
which provides tapping (port-mirroring) capability for Neutron networks; |
| 66 |
tenant or provider networks. This service will be modeled similar to other |
| 67 |
Neutron services such as the firewall, load-balancer, L3-router etc. |
| 68 |
|
| 69 |
The proposed service will allow the tenants to create a tap service instance |
| 70 |
to which they can add Neutron ports that need to be mirrored by creating tap |
| 71 |
flows. The tap service itself will be a Neutron port, which will be the |
| 72 |
destination port for the mirrored traffic. |
| 73 |
|
| 74 |
The destination Tap-as-a-Service Neutron port should be created beforehand on |
| 75 |
a network owned by the tenant who is requesting the service. The ports to be |
| 76 |
mirrored that are added to the service must be owned by the same tenant who |
| 77 |
created the tap service instance. Even on a shared network, a tenant will only |
| 78 |
be allowed to mirror the traffic from ports that they own on the shared |
| 79 |
network and not traffic from ports that they do not own on the shared network. |
| 80 |
|
| 81 |
The ports owned by the tenant that are mirrored can be on networks other |
| 82 |
than the network on which tap service port is created. This allows the tenant |
| 83 |
to mirror traffic from any port it owns on a network on to the same |
| 84 |
Tap-as-a-Service Neutron port. |
| 85 |
|
| 86 |
The tenant can launch a VM specifying the tap destination port for the VM |
| 87 |
interface (--nic port-id=tap_port_uuid), thus receiving mirrored traffic for |
| 88 |
further processing (dependent on use case) on that VM. |
| 89 |
|
| 90 |
The following would be the work flow for using this service from a tenant's |
| 91 |
point of view |
| 92 |
|
| 93 |
0. Create a Neutron port which will be used as the destination port. |
| 94 |
This can be a part of ordinary VM launch. |
| 95 |
|
| 96 |
1. Create a tap service instance, specifying the Neutron port. |
| 97 |
|
| 98 |
2. If you haven't yet, launch a monitoring or traffic analysis VM and |
| 99 |
connect it to the destination port for the tap service instance. |
| 100 |
|
| 101 |
3. Associate Neutron ports with a tap service instance if/when they need to be |
| 102 |
monitored. |
| 103 |
|
| 104 |
4. Disassociate Neutron ports from a tap service instance if/when they no |
| 105 |
longer need to be monitored. |
| 106 |
|
| 107 |
5. Destroy a tap-service instance when it is no longer needed. |
| 108 |
|
| 109 |
6. Delete the destination port when it is no longer neeeded. |
| 110 |
|
| 111 |
Please note that the normal work flow of launching a VM is not affected while |
| 112 |
using TaaS. |
| 113 |
|
| 114 |
|
| 115 |
Alternatives |
| 116 |
------------ |
| 117 |
|
| 118 |
As an alternative to introducing port mirroring functionality under Neutron |
| 119 |
services, it could be added as an extension to the existing Neutron v2 APIs. |
| 120 |
|
| 121 |
|
| 122 |
Data model impact |
| 123 |
----------------- |
| 124 |
|
| 125 |
Tap-as-a-Service introduces the following data models into Neutron as database |
| 126 |
schemas. |
| 127 |
|
| 128 |
1. tap_service |
| 129 |
|
| 130 |
+-------------+--------+----------+-----------+---------------+-------------------------+ |
| 131 |
| Attribute | Type | Access | Default | Validation/ | Description | |
| 132 |
| Name | | (CRUD) | Value | Conversion | | |
| 133 |
+=============+========+==========+===========+===============+=========================+ |
| 134 |
| id | UUID | R, all | generated | N/A | UUID of the tap | |
| 135 |
| | | | | | service instance. | |
| 136 |
+-------------+--------+----------+-----------+---------------+-------------------------+ |
| 137 |
| project_id | String | CR, all | Requester | N/A | ID of the | |
| 138 |
| | | | | | project creating | |
| 139 |
| | | | | | the service | |
| 140 |
+-------------+--------+----------+-----------+---------------+-------------------------+ |
| 141 |
| name | String | CRU, all | Empty | N/A | Name for the service | |
| 142 |
| | | | | | instance. | |
| 143 |
+-------------+--------+----------+-----------+---------------+-------------------------+ |
| 144 |
| description | String | CRU, all | Empty | N/A | Description of the | |
| 145 |
| | | | | | service instance. | |
| 146 |
+-------------+--------+----------+-----------+---------------+-------------------------+ |
| 147 |
| port_id | UUID | CR, all | N/A | UUID of a | An existing Neutron port| |
| 148 |
| | | | | valid Neutron | to which traffic will | |
| 149 |
| | | | | port | be mirrored | |
| 150 |
+-------------+--------+----------+-----------+---------------+-------------------------+ |
| 151 |
| status | String | R, all | N/A | N/A | The operation status of | |
| 152 |
| | | | | | the resource | |
| 153 |
| | | | | | (ACTIVE, PENDING_foo, | |
| 154 |
| | | | | | ERROR, ...) | |
| 155 |
+-------------+--------+----------+-----------+---------------+-------------------------+ |
| 156 |
|
| 157 |
2. tap_flow |
| 158 |
|
| 159 |
+----------------+--------+----------+-----------+---------------+-------------------------+ |
| 160 |
| Attribute | Type | Access | Default | Validation/ | Description | |
| 161 |
| Name | | (CRUD) | Value | Conversion | | |
| 162 |
+================+========+==========+===========+===============+=========================+ |
| 163 |
| id | UUID | R, all | generated | N/A | UUID of the | |
| 164 |
| | | | | | tap flow instance. | |
| 165 |
+----------------+--------+----------+-----------+---------------+-------------------------+ |
| 166 |
| name | String | CRU, all | Empty | N/A | Name for the tap flow | |
| 167 |
| | | | | | instance. | |
| 168 |
+----------------+--------+----------+-----------+---------------+-------------------------+ |
| 169 |
| description | String | CRU, all | Empty | N/A | Description of the | |
| 170 |
| | | | | | tap flow instance. | |
| 171 |
+----------------+--------+----------+-----------+---------------+-------------------------+ |
| 172 |
| tap_service_id | UUID | CR, all | N/A | Valid tap | UUID of the tap | |
| 173 |
| | | | | service UUID | service instance. | |
| 174 |
+----------------+--------+----------+-----------+---------------+-------------------------+ |
| 175 |
| source_port | UUID | CR, all | N/A | UUID of a | UUID of the Neutron | |
| 176 |
| | | | | valid Neutron | port that needed to be | |
| 177 |
| | | | | port | mirrored | |
| 178 |
+----------------+--------+----------+-----------+---------------+-------------------------+ |
| 179 |
| direction | ENUM | CR, all | BOTH | | Whether to mirror the | |
| 180 |
| | (IN, | | | | traffic leaving or | |
| 181 |
| | OUT, | | | | arriving at the | |
| 182 |
| | BOTH) | | | | source port | |
| 183 |
| | | | | | IN: Network -> VM | |
| 184 |
| | | | | | OUT: VM -> Network | |
| 185 |
+----------------+--------+----------+-----------+---------------+-------------------------+ |
| 186 |
| status | String | R, all | N/A | N/A | The operation status of | |
| 187 |
| | | | | | the resource | |
| 188 |
| | | | | | (ACTIVE, PENDING_foo, | |
| 189 |
| | | | | | ERROR, ...) | |
| 190 |
+----------------+--------+----------+-----------+---------------+-------------------------+ |
| 191 |
|
| 192 |
|
| 193 |
REST API impact |
| 194 |
--------------- |
| 195 |
|
| 196 |
Tap-as-a-Service shall be offered over the RESTFull API interface under |
| 197 |
the following namespace: |
| 198 |
|
| 199 |
http://wiki.openstack.org/Neutron/TaaS/API_1.0 |
| 200 |
|
| 201 |
The resource attribute map for TaaS is provided below: |
| 202 |
|
| 203 |
.. code-block:: python |
| 204 |
|
| 205 |
direction_enum = ['IN', 'OUT', 'BOTH'] |
| 206 |
|
| 207 |
RESOURCE_ATTRIBUTE_MAP = { |
| 208 |
'tap_service': { |
| 209 |
'id': {'allow_post': False, 'allow_put': False, |
| 210 |
'validate': {'type:uuid': None}, 'is_visible': True, |
| 211 |
'primary_key': True}, |
| 212 |
'project_id': {'allow_post': True, 'allow_put': False, |
| 213 |
'validate': {'type:string': None}, |
| 214 |
'required_by_policy': True, 'is_visible': True}, |
| 215 |
'name': {'allow_post': True, 'allow_put': True, |
| 216 |
'validate': {'type:string': None}, |
| 217 |
'is_visible': True, 'default': ''}, |
| 218 |
'description': {'allow_post': True, 'allow_put': True, |
| 219 |
'validate': {'type:string': None}, |
| 220 |
'is_visible': True, 'default': ''}, |
| 221 |
'port_id': {'allow_post': True, 'allow_put': False, |
| 222 |
'validate': {'type:uuid': None}, |
| 223 |
'is_visible': True}, |
| 224 |
'status': {'allow_post': False, 'allow_put': False, |
| 225 |
'is_visible': True}, |
| 226 |
}, |
| 227 |
'tap_flow': { |
| 228 |
'id': {'allow_post': False, 'allow_put': False, |
| 229 |
'validate': {'type:uuid': None}, 'is_visible': True, |
| 230 |
'primary_key': True}, |
| 231 |
'name': {'allow_post': True, 'allow_put': True, |
| 232 |
'validate': {'type:string': None}, |
| 233 |
'is_visible': True, 'default': ''}, |
| 234 |
'description': {'allow_post': True, 'allow_put': True, |
| 235 |
'validate': {'type:string': None}, |
| 236 |
'is_visible': True, 'default': ''}, |
| 237 |
'tap_service_id': {'allow_post': True, 'allow_put': False, |
| 238 |
'validate': {'type:uuid': None}, |
| 239 |
'required_by_policy': True, 'is_visible': True}, |
| 240 |
'source_port': {'allow_post': True, 'allow_put': False, |
| 241 |
'validate': {'type:uuid': None}, |
| 242 |
'required_by_policy': True, 'is_visible': True}, |
| 243 |
'direction': {'allow_post': True, 'allow_put': False, |
| 244 |
'validate': {'type:string': direction_enum}, |
| 245 |
'is_visible': True}, |
| 246 |
'status': {'allow_post': False, 'allow_put': False, |
| 247 |
'is_visible': True}, |
| 248 |
} |
| 249 |
} |
| 250 |
|
| 251 |
|
| 252 |
Security impact |
| 253 |
--------------- |
| 254 |
|
| 255 |
A TaaS instance comprises a collection of source Neutron ports (whose |
| 256 |
ingress and/or egress traffic are being mirrored) and a destination Neutron |
| 257 |
port (where the mirrored traffic is received). Security Groups will be |
| 258 |
handled differently for these two classes of ports, as described below: |
| 259 |
|
| 260 |
Destination Side: |
| 261 |
|
| 262 |
Ingress Security Group filters, including the filter that prevents MAC-address |
| 263 |
spoofing, will be disabled for the destination Neutron port. This will ensure |
| 264 |
that all of the mirrored packets received at this port are able to reach the |
| 265 |
monitoring VM attached to it. |
| 266 |
|
| 267 |
Source Side: |
| 268 |
|
| 269 |
Ideally it would be nice to mirror all packets entering and/or leaving the |
| 270 |
virtual NICs associated with the VMs that are being monitored. This means |
| 271 |
capturing ingress traffic after it passes the inbound Security Group filters |
| 272 |
and capturing egress traffic before it passes the outbound Security Group |
| 273 |
filters. |
| 274 |
|
| 275 |
However, due to the manner in which Security Groups are currently implemented |
| 276 |
in OpenStack (i.e. north of the Open vSwitch ports, using Linux IP Tables) this |
| 277 |
is not possible because port mirroring support resides inside Open vSwitch. |
| 278 |
Therefore, in the first version of TaaS, Security Groups will be ignored for |
| 279 |
the source Neutron ports; this effectively translates into capturing ingress |
| 280 |
traffic before it passes the inbound Security Group filters and capturing |
| 281 |
egress traffic after it passes the outbound Security Group filters. In other |
| 282 |
words, port mirroring will be implemented for all packets entering and/or |
| 283 |
leaving the Open vSwitch ports associated with the respective virtual NICs of |
| 284 |
the VMs that are being monitored. |
| 285 |
|
| 286 |
There is a separate effort that has been initiated to implement Security Groups |
| 287 |
within OpenvSwitch. A later version of TaaS may make use of this feature, if |
| 288 |
and when it is available, so that we can realize the ideal behavior described |
| 289 |
above. It should be noted that such an enhancement should not require a change |
| 290 |
to the TaaS data model. |
| 291 |
|
| 292 |
Keeping data privacy aspects in mind and preventing the data center admin |
| 293 |
from snooping on tenant's network traffic without their knowledge, the admin |
| 294 |
shall not be allowed to mirror traffic from any ports that belong to tenants. |
| 295 |
Hence creation of 'Tap_Flow' is only permitted on ports that are owned by the |
| 296 |
creating tenant. |
| 297 |
|
| 298 |
If an admin wants to monitor tenant's traffic, the admin will have to join that |
| 299 |
tenant as a member. This will ensure that the tenant is aware that the admin |
| 300 |
might be monitoring their traffic. |
| 301 |
|
| 302 |
|
| 303 |
Notifications impact |
| 304 |
-------------------- |
| 305 |
|
| 306 |
A set of new RPC calls for communication between the TaaS server and agents |
| 307 |
are required and will be put in place as part of the reference implementation. |
| 308 |
|
| 309 |
|
| 310 |
IPv6 impact |
| 311 |
-------------------- |
| 312 |
None |
| 313 |
|
| 314 |
|
| 315 |
Other end user impact |
| 316 |
--------------------- |
| 317 |
|
| 318 |
Users will be able to invoke and access the TaaS APIs through |
| 319 |
python-neutronclient. |
| 320 |
|
| 321 |
|
| 322 |
Performance Impact |
| 323 |
------------------ |
| 324 |
|
| 325 |
The performance impact of mirroring traffic needs to be examined and |
| 326 |
quantified. The impact of a tenant potentially mirroring all traffic from |
| 327 |
all ports could be large and needs more examination. |
| 328 |
|
| 329 |
Some alternatives to reduce the amount of mirrored traffic are listed below. |
| 330 |
|
| 331 |
1. Rate limiting on the ports being mirrored. |
| 332 |
2. Filters to select certain flows ingressing/egressing a port to be |
| 333 |
mirrored. |
| 334 |
3. Having a quota on the number of TaaS Flows that can be defined by the |
| 335 |
tenant. |
| 336 |
|
| 337 |
|
| 338 |
Other deployer impact |
| 339 |
--------------------- |
| 340 |
|
| 341 |
Configurations for the service plugin will be added later. |
| 342 |
|
| 343 |
A new bridge (br-tap) mentioned in Implementation section. |
| 344 |
|
| 345 |
|
| 346 |
Developer impact |
| 347 |
---------------- |
| 348 |
This will be a new extension API, and will not affect the existing API. |
| 349 |
|
| 350 |
|
| 351 |
Community impact |
| 352 |
---------------- |
| 353 |
None |
| 354 |
|
| 355 |
|
| 356 |
Follow up work |
| 357 |
-------------- |
| 358 |
|
| 359 |
Going forward, TaaS would be incorporated with Service Insertion [2]_ similar |
| 360 |
to other existing services like FWaaS, LBaaS, and VPNaaS. |
| 361 |
|
| 362 |
While integrating Tap-as-a-Service with Service Insertion the key changes to |
| 363 |
the data model needed would be the removal of 'network_id' and 'port_id' from |
| 364 |
the 'Tap_Service' data model. |
| 365 |
|
| 366 |
Some policy based filtering rules would help alleviate the potential performance |
| 367 |
issues. |
| 368 |
|
| 369 |
We might want to ensure exclusive use of the destination port. |
| 370 |
|
| 371 |
We might want to create the destination port automatically on tap-service |
| 372 |
creation, rather than specifying an existing port. In that case, network_id |
| 373 |
should be taken as a parameter for tap-service creation, instead of port_id. |
| 374 |
|
| 375 |
We might want to allow the destination port be used for purposes other than |
| 376 |
just launching a VM on it, for example the port could be used as an |
| 377 |
'external-port' [1]_ to get the mirrored data out from the tenant virtual |
| 378 |
network on a device or network not managed by openstack. |
| 379 |
|
| 380 |
We might want to introduce a way to tap a whole traffic for the specified |
| 381 |
network. |
| 382 |
|
| 383 |
We need a mechanism to coordinate usage of various resources with other |
| 384 |
agent extensions. E.g. OVS flows, tunnel IDs, VLAN IDs. |
| 385 |
|
| 386 |
|
| 387 |
Implementation |
| 388 |
============== |
| 389 |
|
| 390 |
The reference implementation for TaaS will be based on Open vSwitch. In |
| 391 |
addition to the existing integration (br-int) and tunnel (br-tun) bridges, a |
| 392 |
separate tap bridge (br-tap) will be used. The tap bridge provides nice |
| 393 |
isolation for supporting more complex TaaS features (e.g. filtering mirrored |
| 394 |
packets) in the future. |
| 395 |
|
| 396 |
The tapping operation will be realized by adding higher priority flows in |
| 397 |
br-int, which duplicate the ingress and/or egress packets associated with |
| 398 |
specific ports (belonging to the VMs being monitored) and send the copies to |
| 399 |
br-tap. Packets sent to br-tap will also be tagged with an appropriate VLAN id |
| 400 |
corresponding to the associated TaaS instance (in the initial release these |
| 401 |
VLAN ids may be reserved from highest to lowest; in later releases it should be |
| 402 |
coordinated with the Neutron service). The original packets will continue to be |
| 403 |
processed normally, so as not to affect the traffic patterns of the VMs being |
| 404 |
monitored. |
| 405 |
|
| 406 |
Flows will be placed in br-tap to determine if the mirrored traffic should be |
| 407 |
sent to br-tun or not. If the destination port of a Tap-aaS instance happens to |
| 408 |
reside on the same host as a source port, packets from that source port will be |
| 409 |
returned to br-int; otherwise they will be forwarded to br-tun for delivery to |
| 410 |
a remote node. |
| 411 |
|
| 412 |
Packets arriving at br-tun from br-tap will get routed to the destination ports |
| 413 |
of appropriate TaaS instances using the same GRE or VXLAN tunnel network that |
| 414 |
is used to pass regular traffic between hosts. Separate tunnel IDs will be used |
| 415 |
to isolate different TaaS instances from one another and from the normal |
| 416 |
(non-mirrored) traffic passing through the bridge. This will ensure that proper |
| 417 |
action can be taken on the receiving end of a tunnel so that mirrored traffic |
| 418 |
is sent to br-tap instead of br-int. Special flows will be used in br-tun to |
| 419 |
automatically learn about the location of the destination ports of TaaS |
| 420 |
instances. |
| 421 |
|
| 422 |
Packets entering br-tap from br-tun will be forwarded to br-int only if the |
| 423 |
destination port of the corresponding TaaS instance resides on the same host. |
| 424 |
Finally, packets entering br-int from br-tap will be delivered to the |
| 425 |
appropriate destination port after the TaaS instance VLAN id is replaced with |
| 426 |
the VLAN id for the port. |
| 427 |
|
| 428 |
|
| 429 |
Assignee(s) |
| 430 |
----------- |
| 431 |
|
| 432 |
* Vinay Yadhav |
| 433 |
|
| 434 |
|
| 435 |
Work Items |
| 436 |
---------- |
| 437 |
|
| 438 |
* TaaS API and data model implementation. |
| 439 |
* TaaS OVS driver. |
| 440 |
* OVS agent changes for port mirroring. |
| 441 |
|
| 442 |
|
| 443 |
Dependencies |
| 444 |
============ |
| 445 |
|
| 446 |
None |
| 447 |
|
| 448 |
|
| 449 |
Testing |
| 450 |
======= |
| 451 |
|
| 452 |
* Unit Tests to be added. |
| 453 |
* Functional tests in tempest to be added. |
| 454 |
* API Tests in Tempest to be added. |
| 455 |
|
| 456 |
|
| 457 |
Documentation Impact |
| 458 |
==================== |
| 459 |
|
| 460 |
* User Documentation needs to be updated |
| 461 |
* Developer Documentation needs to be updated |
| 462 |
|
| 463 |
|
| 464 |
References |
| 465 |
========== |
| 466 |
|
| 467 |
.. [1] External port |
| 468 |
https://review.openstack.org/#/c/87825 |
| 469 |
|
| 470 |
.. [2] Service base and insertion |
| 471 |
https://review.openstack.org/#/c/93128 |