Skip to content

Don't scan all target groups/ELBs, limit to just ones that are associated with the cluster #274

Description

@2rs2ts

The following code is inefficient especially in large organizations with thousands of target groups and ELBs:

// get all target groups
targetGroups := []*elbv2.TargetGroup{}
if slices.Contains(ctx.DeregisterTargetTypes, TargetTypeTargetGroup.String()) {
err := elbv2Client.DescribeTargetGroupsPages(&elbv2.DescribeTargetGroupsInput{}, func(page *elbv2.DescribeTargetGroupsOutput, lastPage bool) bool {
targetGroups = append(targetGroups, page.TargetGroups...)
return page.NextMarker != nil
})
if err != nil {
return scanResult, err
}
}
// get all classic elbs
elbDescriptions := []*elb.LoadBalancerDescription{}
if slices.Contains(ctx.DeregisterTargetTypes, TargetTypeClassicELB.String()) {
err := elbClient.DescribeLoadBalancersPages(&elb.DescribeLoadBalancersInput{}, func(page *elb.DescribeLoadBalancersOutput, lastPage bool) bool {
elbDescriptions = append(elbDescriptions, page.LoadBalancerDescriptions...)
return page.NextMarker != nil
})
if err != nil {
return scanResult, err
}
}

If you were to use the GetResources API, filtering using the kubernetes.io/cluster/<cluster name> tag and the elasticloadbalancing:loadbalancer and elasticloadbalancing:targetgroup resource types.

In the aws CLI, it'd be like this:

$ aws --region us-east-2 resourcegroupstaggingapi get-resources --tag-filters "Key=kubernetes.io/cluster/${CLUSTER_NAME}" --resource-type-filters 'elasticloadbalancing:loadbalancer' 'elasticloadbalancing:targetgroup'
{
    "ResourceTagMappingList": [
        {
            "ResourceARN": "arn:aws:elasticloadbalancing:us-east-2:<account>:loadbalancer/<name>",
            "Tags": [
                {
                    "Key": "kubernetes.io/cluster/<cluster name>",
                    "Value": "owned"
                }, ...
            ]
        }, ...
    ]
}

I understand that there may be edge cases where people have independently configured target groups and ELBs that are not managed by Kubernetes, but in that case I think all you need to do is add a flag to specify additional tags to filter for (which would require multiple GetResources lookups, but ultimately should still be faster) and/or a flag to specify the additional resource ARNs. Or, maybe you keep this original implementation around and let people use a flag to fall back to that. (Or maybe since this requires a different IAM permission, you'd make this new behavior of using GetResources be opt-in for a time?)

Regardless, I hope you can implement this because as it stands, we can't use the --with-deregister flag at our organization. It takes too long to run and it causes API rate limiting.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions