Port Mirroring by SPAN or RSPAN on an HP C7000 Blade
This is just a heads-up to hopefully save someone else a bit of time and pain... but the HP Virtual Connect doesn't support SPAN or RSPAN to mirror traffic from a physical device into the chassis to, for example, a Virtual Machine. Basically, Port Mirroring, such as through SPAN or RSPAN, uses unicast to duplicate network traffic from a source port or ports onto a destination port. This technique is useful for troubleshooting, in case you can't get a packet capture running on either end of a network flow, or for monitoring (as was our intended use case).
IANANG (I Am Not a Networking Guy), but my understanding is that the problem is due to the nature of a SPAN port and how those packets look to the Virtual Connect. When Port Mirroring is configured, all traffic is duplicated and sent out to the Virtual Connect. These packets are not changed in this process, keeping their original source and destination MAC addresses; the SPAN port is forwarding these packets to the VC despite the fact that it knows that the destination address isn't behind that port.
The VC receives these packets and checks its ARP table, to see if it knows where to send them. If it hasn't seen that MAC Address before, it broadcasts the packet to all ports and it actually gets through to the monitoring server just fine. It adds the source address to its ARP table and everything moves on. The challenge here comes from the fact that the SPAN port is going to forward all traffic, both to and from that source, and so that MAC Address is going to show up in both the Source and Destination of some packets.
Once a packet comes to the VC with that MAC Address as the Source, that data will be saved in the VC's ARP table. At that point, it will no longer broadcast traffic with that destination and will instead send it directly to the port that it knows that MAC Address is behind. Unfortunately, in this case, that port is the same port that it received the traffic from. This ends up looking a bit like a network loop, as the VC is receiving traffic on a port that is destined for a device on that same port. It knows better than to pass that traffic back to the same port, so it just drops it.
After a few packets are forwarded, the VC will eventually learn (correctly) that all of the source MAC Addresses exist outside of itself, and so will drop any traffic that comes in with one of those MAC Addresses as a destination... which happens to be all of the SPAN traffic that we're attempting to mirror into the device.
In most situations, this would be a desirable behavior, as it can help mitigate the damage from a poorly configured network. As a side effect, this behavior prohibits the use of SPAN or RSPAN from physical devices. Other network devices get around this issue by having special configurations that are built to support SPAN or RSPAN, but the VC does not offer this functionality. So, long story short, after a few days of configuration and troubleshooting, we just needed to set up a physical server that plugged directly into the SPAN port, and everything was good.
IANANG (I Am Not a Networking Guy), but my understanding is that the problem is due to the nature of a SPAN port and how those packets look to the Virtual Connect. When Port Mirroring is configured, all traffic is duplicated and sent out to the Virtual Connect. These packets are not changed in this process, keeping their original source and destination MAC addresses; the SPAN port is forwarding these packets to the VC despite the fact that it knows that the destination address isn't behind that port.
The VC receives these packets and checks its ARP table, to see if it knows where to send them. If it hasn't seen that MAC Address before, it broadcasts the packet to all ports and it actually gets through to the monitoring server just fine. It adds the source address to its ARP table and everything moves on. The challenge here comes from the fact that the SPAN port is going to forward all traffic, both to and from that source, and so that MAC Address is going to show up in both the Source and Destination of some packets.
Once a packet comes to the VC with that MAC Address as the Source, that data will be saved in the VC's ARP table. At that point, it will no longer broadcast traffic with that destination and will instead send it directly to the port that it knows that MAC Address is behind. Unfortunately, in this case, that port is the same port that it received the traffic from. This ends up looking a bit like a network loop, as the VC is receiving traffic on a port that is destined for a device on that same port. It knows better than to pass that traffic back to the same port, so it just drops it.
After a few packets are forwarded, the VC will eventually learn (correctly) that all of the source MAC Addresses exist outside of itself, and so will drop any traffic that comes in with one of those MAC Addresses as a destination... which happens to be all of the SPAN traffic that we're attempting to mirror into the device.
In most situations, this would be a desirable behavior, as it can help mitigate the damage from a poorly configured network. As a side effect, this behavior prohibits the use of SPAN or RSPAN from physical devices. Other network devices get around this issue by having special configurations that are built to support SPAN or RSPAN, but the VC does not offer this functionality. So, long story short, after a few days of configuration and troubleshooting, we just needed to set up a physical server that plugged directly into the SPAN port, and everything was good.
Comments
Post a Comment
Sorry guys, I've been getting a lot of spam recently, so I've had to turn on comment moderation. I'll do my best to moderate them swiftly after they're submitted,