Figure out why some data has duplicate attributes

[alistairking]

For example, the RIB dumps we get from route-views.eqix seem to often have duplicate community values.

E.g.: http://bgp-archive.caida.org/routeviews/route-views.eqix/ribs/2018/03/29/routeviews.route-views.eqix.ribs.1522281600.bz2:

alistair@gibi:~/Downloads$ parsebgp mrt:routeviews.route-views.eqix.ribs.1522281600 >/dev/null
INFO: Parsing routeviews.route-views.eqix.ribs.1522281600 (Type: mrt)
WARN: Duplicate Path Attribute (8) found. Skipping
WARN: Duplicate Path Attribute (8) found. Skipping

[kenkeys]

I suspect in these examples the second COMMUNITY attribute was actually a mislabelled LARGE_COMMUNITY attribute. The first 4 bytes of the value 0x000215ac appear to be a 4-byte ASN 136620, which does not fit in 2 bytes, so does not work in a COMMUNITY, but does if parsing it as a LARGE_COMMUNITY. Parsing as a COMMUNITY also yields communities with ASNs 0 and 2, which don't seem plausible. The full parse as LARGE_COMMUNITY is 136620:174:101, 136620:6939:999, which does look plausible: 136620 is one of the ASNs that appeared in the AS_PATH, and 174 (COGENT) and 6939 (HURRICANE) are upstream peers of 136620.

I don't think there's any better way for us to handle this. Skipping the duplicate attribute is consistent with RFC 7606 section 3g, and is what other bgp speakers are expected to do with it.

[alistairking]

Interesting. Do you have any idea if this is caused by the peer router or by the collector itself?
I'll try and talk with some Route Views folks today to see if they have any insight.

[kenkeys]

I have no idea, but it does feel more likely that such a bug would go unnoticed in a collector than in a router.

[dteach-rv]

Hey, which prefixes are you seeing this on, and who are the advertising peers. I'm working on narrowing this down.

[alistairking]

@dteach-rv what ended up happening with this? Were you still waiting on more info from me?

[dteach-rv]

I haven't been able to determine what's going on here based on prefixes associated with this error. My guess is @kenkeys is probably right. If I can capture one of the malformed updates in a pcap with debugging enabled on the collector, that might should be enough raise an issue with the FRR project. FWIW, we're working towards upgrading our collectors this summer. Eqix is on the list, so this might get addressed via the upgrade. I've got debugging turned on for the prefixes you had mentioned to me previously.
23.129.128.0/24
67.221.245.0/24
84.205.80.0/24
89.46.125.0/24