Debian Codegraph @

specs/mitaka/tap-as-a-service.rst

raw
1 ..
2 This work is licensed under a Creative Commons Attribution 3.0 Unported
3 License.
4
5 http://creativecommons.org/licenses/by/3.0/legalcode
6
7 ============================
8 Tap-as-a-Service for Neutron
9 ============================
10
11
12 Launchpad blueprint:
13
14 https://blueprints.launchpad.net/neutron/+spec/tap-as-a-service
15
16 This spec explains an extension for the port mirroring functionality. Port
17 mirroring involves sending a copy of packets ingressing and/or egressing one
18 port (where ingress means entering a VM and egress means leaving a VM) to
19 another port, (usually different from the packet's original destination).
20 A port could be attached to a VM or a networking resource like router.
21
22 While the blueprint describes the functionality of mirroring Neutron ports as
23 an extension to the port object, the spec proposes to offer port mirroring as a
24 service, which will enable more advanced use-cases (e.g. intrusion detection)
25 to be deployed.
26
27 The proposed port mirroring capability shall be introduced in Neutron as a
28 service called "Tap-as-a-Service".
29
30 Problem description
31 ===================
32
33 Neutron currently does not support the functionality of port mirroring for
34 tenant networks. This feature will greatly benefit tenants and admins, who
35 want to debug their virtual networks and gain visibility into their VMs by
36 monitoring and analyzing the network traffic associated with them (e.g. IDS).
37
38 This spec focuses on mirroring traffic from one Neutron port to another;
39 future versions may address mirroring from a Neutron port to an arbitrary
40 interface (not managed by Neutron) on a compute host or the network controller.
41
42 Different usage scenarios for the service are listed below:
43
44 1. Tapping/mirroring network traffic ingressing and/or egressing a particular
45 Neutron port.
46 2. Tapping/mirroring all network traffic on a tenant network.
47 3. Tenant or admin will be able to do tap/traffic mirroring based on a
48 policy rule and set destination as a Neutron port, which can be linked
49 to a virtual machine as normal Nova operations or to a physical machine
50 via l2-gateway functionality.
51 4. Admin will be able to do packet level network debugging for the virtual
52 network.
53 5. Provide a way for real time analytics based on different criteria, like
54 tenants, ports, traffic types (policy) etc.
55
56 Note that some of the above use-cases are not covered by this proposal, at
57 least for the first step.
58
59
60 Proposed change
61 ===============
62
63 The proposal is to introduce a new Neutron service plugin, called
64 "Tap-as-a-Service",
65 which provides tapping (port-mirroring) capability for Neutron networks;
66 tenant or provider networks. This service will be modeled similar to other
67 Neutron services such as the firewall, load-balancer, L3-router etc.
68
69 The proposed service will allow the tenants to create a tap service instance
70 to which they can add Neutron ports that need to be mirrored by creating tap
71 flows. The tap service itself will be a Neutron port, which will be the
72 destination port for the mirrored traffic.
73
74 The destination Tap-as-a-Service Neutron port should be created beforehand on
75 a network owned by the tenant who is requesting the service. The ports to be
76 mirrored that are added to the service must be owned by the same tenant who
77 created the tap service instance. Even on a shared network, a tenant will only
78 be allowed to mirror the traffic from ports that they own on the shared
79 network and not traffic from ports that they do not own on the shared network.
80
81 The ports owned by the tenant that are mirrored can be on networks other
82 than the network on which tap service port is created. This allows the tenant
83 to mirror traffic from any port it owns on a network on to the same
84 Tap-as-a-Service Neutron port.
85
86 The tenant can launch a VM specifying the tap destination port for the VM
87 interface (--nic port-id=tap_port_uuid), thus receiving mirrored traffic for
88 further processing (dependent on use case) on that VM.
89
90 The following would be the work flow for using this service from a tenant's
91 point of view
92
93 0. Create a Neutron port which will be used as the destination port.
94 This can be a part of ordinary VM launch.
95
96 1. Create a tap service instance, specifying the Neutron port.
97
98 2. If you haven't yet, launch a monitoring or traffic analysis VM and
99 connect it to the destination port for the tap service instance.
100
101 3. Associate Neutron ports with a tap service instance if/when they need to be
102 monitored.
103
104 4. Disassociate Neutron ports from a tap service instance if/when they no
105 longer need to be monitored.
106
107 5. Destroy a tap-service instance when it is no longer needed.
108
109 6. Delete the destination port when it is no longer neeeded.
110
111 Please note that the normal work flow of launching a VM is not affected while
112 using TaaS.
113
114
115 Alternatives
116 ------------
117
118 As an alternative to introducing port mirroring functionality under Neutron
119 services, it could be added as an extension to the existing Neutron v2 APIs.
120
121
122 Data model impact
123 -----------------
124
125 Tap-as-a-Service introduces the following data models into Neutron as database
126 schemas.
127
128 1. tap_service
129
130 +-------------+--------+----------+-----------+---------------+-------------------------+
131 | Attribute | Type | Access | Default | Validation/ | Description |
132 | Name | | (CRUD) | Value | Conversion | |
133 +=============+========+==========+===========+===============+=========================+
134 | id | UUID | R, all | generated | N/A | UUID of the tap |
135 | | | | | | service instance. |
136 +-------------+--------+----------+-----------+---------------+-------------------------+
137 | project_id | String | CR, all | Requester | N/A | ID of the |
138 | | | | | | project creating |
139 | | | | | | the service |
140 +-------------+--------+----------+-----------+---------------+-------------------------+
141 | name | String | CRU, all | Empty | N/A | Name for the service |
142 | | | | | | instance. |
143 +-------------+--------+----------+-----------+---------------+-------------------------+
144 | description | String | CRU, all | Empty | N/A | Description of the |
145 | | | | | | service instance. |
146 +-------------+--------+----------+-----------+---------------+-------------------------+
147 | port_id | UUID | CR, all | N/A | UUID of a | An existing Neutron port|
148 | | | | | valid Neutron | to which traffic will |
149 | | | | | port | be mirrored |
150 +-------------+--------+----------+-----------+---------------+-------------------------+
151 | status | String | R, all | N/A | N/A | The operation status of |
152 | | | | | | the resource |
153 | | | | | | (ACTIVE, PENDING_foo, |
154 | | | | | | ERROR, ...) |
155 +-------------+--------+----------+-----------+---------------+-------------------------+
156
157 2. tap_flow
158
159 +----------------+--------+----------+-----------+---------------+-------------------------+
160 | Attribute | Type | Access | Default | Validation/ | Description |
161 | Name | | (CRUD) | Value | Conversion | |
162 +================+========+==========+===========+===============+=========================+
163 | id | UUID | R, all | generated | N/A | UUID of the |
164 | | | | | | tap flow instance. |
165 +----------------+--------+----------+-----------+---------------+-------------------------+
166 | name | String | CRU, all | Empty | N/A | Name for the tap flow |
167 | | | | | | instance. |
168 +----------------+--------+----------+-----------+---------------+-------------------------+
169 | description | String | CRU, all | Empty | N/A | Description of the |
170 | | | | | | tap flow instance. |
171 +----------------+--------+----------+-----------+---------------+-------------------------+
172 | tap_service_id | UUID | CR, all | N/A | Valid tap | UUID of the tap |
173 | | | | | service UUID | service instance. |
174 +----------------+--------+----------+-----------+---------------+-------------------------+
175 | source_port | UUID | CR, all | N/A | UUID of a | UUID of the Neutron |
176 | | | | | valid Neutron | port that needed to be |
177 | | | | | port | mirrored |
178 +----------------+--------+----------+-----------+---------------+-------------------------+
179 | direction | ENUM | CR, all | BOTH | | Whether to mirror the |
180 | | (IN, | | | | traffic leaving or |
181 | | OUT, | | | | arriving at the |
182 | | BOTH) | | | | source port |
183 | | | | | | IN: Network -> VM |
184 | | | | | | OUT: VM -> Network |
185 +----------------+--------+----------+-----------+---------------+-------------------------+
186 | status | String | R, all | N/A | N/A | The operation status of |
187 | | | | | | the resource |
188 | | | | | | (ACTIVE, PENDING_foo, |
189 | | | | | | ERROR, ...) |
190 +----------------+--------+----------+-----------+---------------+-------------------------+
191
192
193 REST API impact
194 ---------------
195
196 Tap-as-a-Service shall be offered over the RESTFull API interface under
197 the following namespace:
198
199 http://wiki.openstack.org/Neutron/TaaS/API_1.0
200
201 The resource attribute map for TaaS is provided below:
202
203 .. code-block:: python
204
205 direction_enum = ['IN', 'OUT', 'BOTH']
206
207 RESOURCE_ATTRIBUTE_MAP = {
208 'tap_service': {
209 'id': {'allow_post': False, 'allow_put': False,
210 'validate': {'type:uuid': None}, 'is_visible': True,
211 'primary_key': True},
212 'project_id': {'allow_post': True, 'allow_put': False,
213 'validate': {'type:string': None},
214 'required_by_policy': True, 'is_visible': True},
215 'name': {'allow_post': True, 'allow_put': True,
216 'validate': {'type:string': None},
217 'is_visible': True, 'default': ''},
218 'description': {'allow_post': True, 'allow_put': True,
219 'validate': {'type:string': None},
220 'is_visible': True, 'default': ''},
221 'port_id': {'allow_post': True, 'allow_put': False,
222 'validate': {'type:uuid': None},
223 'is_visible': True},
224 'status': {'allow_post': False, 'allow_put': False,
225 'is_visible': True},
226 },
227 'tap_flow': {
228 'id': {'allow_post': False, 'allow_put': False,
229 'validate': {'type:uuid': None}, 'is_visible': True,
230 'primary_key': True},
231 'name': {'allow_post': True, 'allow_put': True,
232 'validate': {'type:string': None},
233 'is_visible': True, 'default': ''},
234 'description': {'allow_post': True, 'allow_put': True,
235 'validate': {'type:string': None},
236 'is_visible': True, 'default': ''},
237 'tap_service_id': {'allow_post': True, 'allow_put': False,
238 'validate': {'type:uuid': None},
239 'required_by_policy': True, 'is_visible': True},
240 'source_port': {'allow_post': True, 'allow_put': False,
241 'validate': {'type:uuid': None},
242 'required_by_policy': True, 'is_visible': True},
243 'direction': {'allow_post': True, 'allow_put': False,
244 'validate': {'type:string': direction_enum},
245 'is_visible': True},
246 'status': {'allow_post': False, 'allow_put': False,
247 'is_visible': True},
248 }
249 }
250
251
252 Security impact
253 ---------------
254
255 A TaaS instance comprises a collection of source Neutron ports (whose
256 ingress and/or egress traffic are being mirrored) and a destination Neutron
257 port (where the mirrored traffic is received). Security Groups will be
258 handled differently for these two classes of ports, as described below:
259
260 Destination Side:
261
262 Ingress Security Group filters, including the filter that prevents MAC-address
263 spoofing, will be disabled for the destination Neutron port. This will ensure
264 that all of the mirrored packets received at this port are able to reach the
265 monitoring VM attached to it.
266
267 Source Side:
268
269 Ideally it would be nice to mirror all packets entering and/or leaving the
270 virtual NICs associated with the VMs that are being monitored. This means
271 capturing ingress traffic after it passes the inbound Security Group filters
272 and capturing egress traffic before it passes the outbound Security Group
273 filters.
274
275 However, due to the manner in which Security Groups are currently implemented
276 in OpenStack (i.e. north of the Open vSwitch ports, using Linux IP Tables) this
277 is not possible because port mirroring support resides inside Open vSwitch.
278 Therefore, in the first version of TaaS, Security Groups will be ignored for
279 the source Neutron ports; this effectively translates into capturing ingress
280 traffic before it passes the inbound Security Group filters and capturing
281 egress traffic after it passes the outbound Security Group filters. In other
282 words, port mirroring will be implemented for all packets entering and/or
283 leaving the Open vSwitch ports associated with the respective virtual NICs of
284 the VMs that are being monitored.
285
286 There is a separate effort that has been initiated to implement Security Groups
287 within OpenvSwitch. A later version of TaaS may make use of this feature, if
288 and when it is available, so that we can realize the ideal behavior described
289 above. It should be noted that such an enhancement should not require a change
290 to the TaaS data model.
291
292 Keeping data privacy aspects in mind and preventing the data center admin
293 from snooping on tenant's network traffic without their knowledge, the admin
294 shall not be allowed to mirror traffic from any ports that belong to tenants.
295 Hence creation of 'Tap_Flow' is only permitted on ports that are owned by the
296 creating tenant.
297
298 If an admin wants to monitor tenant's traffic, the admin will have to join that
299 tenant as a member. This will ensure that the tenant is aware that the admin
300 might be monitoring their traffic.
301
302
303 Notifications impact
304 --------------------
305
306 A set of new RPC calls for communication between the TaaS server and agents
307 are required and will be put in place as part of the reference implementation.
308
309
310 IPv6 impact
311 --------------------
312 None
313
314
315 Other end user impact
316 ---------------------
317
318 Users will be able to invoke and access the TaaS APIs through
319 python-neutronclient.
320
321
322 Performance Impact
323 ------------------
324
325 The performance impact of mirroring traffic needs to be examined and
326 quantified. The impact of a tenant potentially mirroring all traffic from
327 all ports could be large and needs more examination.
328
329 Some alternatives to reduce the amount of mirrored traffic are listed below.
330
331 1. Rate limiting on the ports being mirrored.
332 2. Filters to select certain flows ingressing/egressing a port to be
333 mirrored.
334 3. Having a quota on the number of TaaS Flows that can be defined by the
335 tenant.
336
337
338 Other deployer impact
339 ---------------------
340
341 Configurations for the service plugin will be added later.
342
343 A new bridge (br-tap) mentioned in Implementation section.
344
345
346 Developer impact
347 ----------------
348 This will be a new extension API, and will not affect the existing API.
349
350
351 Community impact
352 ----------------
353 None
354
355
356 Follow up work
357 --------------
358
359 Going forward, TaaS would be incorporated with Service Insertion [2]_ similar
360 to other existing services like FWaaS, LBaaS, and VPNaaS.
361
362 While integrating Tap-as-a-Service with Service Insertion the key changes to
363 the data model needed would be the removal of 'network_id' and 'port_id' from
364 the 'Tap_Service' data model.
365
366 Some policy based filtering rules would help alleviate the potential performance
367 issues.
368
369 We might want to ensure exclusive use of the destination port.
370
371 We might want to create the destination port automatically on tap-service
372 creation, rather than specifying an existing port. In that case, network_id
373 should be taken as a parameter for tap-service creation, instead of port_id.
374
375 We might want to allow the destination port be used for purposes other than
376 just launching a VM on it, for example the port could be used as an
377 'external-port' [1]_ to get the mirrored data out from the tenant virtual
378 network on a device or network not managed by openstack.
379
380 We might want to introduce a way to tap a whole traffic for the specified
381 network.
382
383 We need a mechanism to coordinate usage of various resources with other
384 agent extensions. E.g. OVS flows, tunnel IDs, VLAN IDs.
385
386
387 Implementation
388 ==============
389
390 The reference implementation for TaaS will be based on Open vSwitch. In
391 addition to the existing integration (br-int) and tunnel (br-tun) bridges, a
392 separate tap bridge (br-tap) will be used. The tap bridge provides nice
393 isolation for supporting more complex TaaS features (e.g. filtering mirrored
394 packets) in the future.
395
396 The tapping operation will be realized by adding higher priority flows in
397 br-int, which duplicate the ingress and/or egress packets associated with
398 specific ports (belonging to the VMs being monitored) and send the copies to
399 br-tap. Packets sent to br-tap will also be tagged with an appropriate VLAN id
400 corresponding to the associated TaaS instance (in the initial release these
401 VLAN ids may be reserved from highest to lowest; in later releases it should be
402 coordinated with the Neutron service). The original packets will continue to be
403 processed normally, so as not to affect the traffic patterns of the VMs being
404 monitored.
405
406 Flows will be placed in br-tap to determine if the mirrored traffic should be
407 sent to br-tun or not. If the destination port of a Tap-aaS instance happens to
408 reside on the same host as a source port, packets from that source port will be
409 returned to br-int; otherwise they will be forwarded to br-tun for delivery to
410 a remote node.
411
412 Packets arriving at br-tun from br-tap will get routed to the destination ports
413 of appropriate TaaS instances using the same GRE or VXLAN tunnel network that
414 is used to pass regular traffic between hosts. Separate tunnel IDs will be used
415 to isolate different TaaS instances from one another and from the normal
416 (non-mirrored) traffic passing through the bridge. This will ensure that proper
417 action can be taken on the receiving end of a tunnel so that mirrored traffic
418 is sent to br-tap instead of br-int. Special flows will be used in br-tun to
419 automatically learn about the location of the destination ports of TaaS
420 instances.
421
422 Packets entering br-tap from br-tun will be forwarded to br-int only if the
423 destination port of the corresponding TaaS instance resides on the same host.
424 Finally, packets entering br-int from br-tap will be delivered to the
425 appropriate destination port after the TaaS instance VLAN id is replaced with
426 the VLAN id for the port.
427
428
429 Assignee(s)
430 -----------
431
432 * Vinay Yadhav
433
434
435 Work Items
436 ----------
437
438 * TaaS API and data model implementation.
439 * TaaS OVS driver.
440 * OVS agent changes for port mirroring.
441
442
443 Dependencies
444 ============
445
446 None
447
448
449 Testing
450 =======
451
452 * Unit Tests to be added.
453 * Functional tests in tempest to be added.
454 * API Tests in Tempest to be added.
455
456
457 Documentation Impact
458 ====================
459
460 * User Documentation needs to be updated
461 * Developer Documentation needs to be updated
462
463
464 References
465 ==========
466
467 .. [1] External port
468 https://review.openstack.org/#/c/87825
469
470 .. [2] Service base and insertion
471 https://review.openstack.org/#/c/93128