Merge pull request #511 from /issues/510

fixes #510 - 10.0.0 release

.gitignore

@@ -72,3 +72,4 @@ results/
 .release_position.json
 dev/terraform.tfstate
 man/
+share/

CHANGES.rst

@@ -1,6 +1,45 @@
 Changelog
 =========
 
+.. _changelog.10_0_0:
+
+10.0.0 (2020-12-07)
+-------------------
+
+IMPORTANT - Breaking Changes
+++++++++++++++++++++++++++++
+
+* This release makes significant changes to how Trusted Advisor is used; see below.
+* This release requires the following new IAM permissions: ``eks:ListClusters``, ``eks:DescribeCluster``, ``eks:ListNodegroups``, ``eks:ListFargateProfiles``, ``eks:DescribeFargateProfile``, ``kinesis:DescribeLimits``.
+* This release introduces a number of new limits, as well as new services. Please see below for details.
+* This release **removes** the ``EC2/Security groups per VPC`` limit, which no longer exists, and adds the new ``EC2/VPC security groups per Region`` limit.
+
+All Changes
++++++++++++
+
+* `Issue #466 <https://github.com/jantman/awslimitchecker/issues/466>`__ - **Significant** changes to Trusted Advisor support.
+
+  * In June 2019, AWS `announced <https://aws.amazon.com/about-aws/whats-new/2019/06/introducing-service-quotas-view-and-manage-quotas-for-aws-services-from-one-location/>`__ the new Service Quotas service (great name) that allows us to retrieve limit/quota information from a unified API. In addition, many individual services now provide limit information via their own APIs. At this point (late 2020) all of the limit/quota information that was previously available via Trusted Advisor is now available via a combination of the individual service APIs and Service Quotas.
+  * In February 2020, the layout of Trusted Advisor checks was changed, and the "Performance / Service Limits" check that we previously used to obtain limit information was moved to its own category in Trusted Advisor. While I can't confirm this, as far as I can tell, this change was only made in the standard AWS regions/partitions (i.e. not GovCloud or China).
+  * awslimitchecker still has not been updated for this new Trusted Advisor layout.
+  * This release **disables Trusted Advisor by default outside China and GovCloud**, as it provides no additional information outside of these regions/partitions.
+  * If you are running in China or GovCloud and have issues with awslimitchecker retrieving information from Trusted Advisor, please `open an issue <https://github.com/jantman/awslimitchecker/issues>`__.
+  * My current intent is to leave Trusted Advisor support in this state until Service Quotas is available in China and GovCloud, at which point I plan on completely removing all Trusted Advisor support.
+* Migrate CI builds from travis-ci.org to travis-ci.com.
+* `Issue #503 <https://github.com/jantman/awslimitchecker/issues/503>`__ - Fix ``Units set to "None"`` error when retrieving load balancer data from Service Quotas. We now allow the (A|E)LB per Region quota with a unit of either "Count" (prior to November 2020) or "None" (November 2020 on).
+* `Issue #489 <https://github.com/jantman/awslimitchecker/issues/489>`__ / `PR #490 <https://github.com/jantman/awslimitchecker/pull/490>`__ - Add missing RDS limits: ``Manual Cluster Snapshots``, ``Custom Endpoints Per DB Cluster``, ``DB Instance Roles``, and ``DB Cluster Roles``. Thanks to `sebasrp <https://github.com/sebasrp>`__ for this contribution!
+* `Issue #472 <https://github.com/jantman/awslimitchecker/issues/472>`__ / `PR #494 <https://github.com/jantman/awslimitchecker/pull/494>`__ - Add support for the ``EKS`` service, and 8 new limits for it. Thanks to `sebasrp <https://github.com/sebasrp>`__ for this contribution!
+* `Issue #495 <https://github.com/jantman/awslimitchecker/issues/495>`__ / `PR #496 <https://github.com/jantman/awslimitchecker/pull/496>`__ - Add support for the ``Kinesis`` service, and one new limit for it. Thanks to `sebasrp <https://github.com/sebasrp>`__ for this contribution!
+* `PR #499 <https://github.com/jantman/awslimitchecker/pull/499>`__ - Set quota_name for VPC "Entries per route table" limit, so that the current limit will be automatically retrieved from Service Quotas. Thanks to `patuck <https://github.com/patuck>`__ for this contribution!
+* `Issue #498 <https://github.com/jantman/awslimitchecker/issues/498>`__ - Fix multiple issues relating to VPC limits:
+
+  * Update the EC2 / ``Rules per VPC security group`` limit to support retrieving the current limit value from Service Quotas.
+  * Remove the ``EC2/Security groups per VPC`` limit, which no longer exists.
+  * Add the new ``EC2/VPC security groups per Region`` limit.
+
+* `Issue #501 <https://github.com/jantman/awslimitchecker/issues/501>`__ - Update ``VPC/Network interfaces per Region`` limit for new calculation method.
+* `Issue #488 <https://github.com/jantman/awslimitchecker/issues/488>`__ / `PR #491 <https://github.com/jantman/awslimitchecker/pull/491>`__ - Update new ElastiCache default limits. Thanks to `sebasrp <https://github.com/sebasrp>`__ for this contribution!
+
 .. _changelog.9_0_0:
 
 9.0.0 (2020-09-22)

README.rst

@@ -27,8 +27,8 @@ awslimitchecker
 
 Master:
 
-.. image:: https://secure.travis-ci.org/jantman/awslimitchecker.png?branch=master
-   :target: http://travis-ci.org/jantman/awslimitchecker
+.. image:: https://api.travis-ci.com/jantman/awslimitchecker.svg?branch=master
+   :target: http://travis-ci.com/jantman/awslimitchecker
    :alt: travis-ci for master branch
 
 .. image:: https://codecov.io/github/jantman/awslimitchecker/coverage.svg?branch=master
@@ -41,8 +41,8 @@ Master:
 
 Develop:
 
-.. image:: https://secure.travis-ci.org/jantman/awslimitchecker.png?branch=develop
-   :target: http://travis-ci.org/jantman/awslimitchecker
+.. image:: https://api.travis-ci.com/jantman/awslimitchecker.svg?branch=develop
+   :target: http://travis-ci.com/jantman/awslimitchecker
    :alt: travis-ci for develop branch
 
 .. image:: https://codecov.io/github/jantman/awslimitchecker/coverage.svg?branch=develop

awslimitchecker/quotas.py

@@ -145,10 +145,11 @@ def get_quota_value(
                 return converter(val, svc[quota_name.lower()]['Unit'], units)
             logger.error(
                 'ERROR: Service Quota service_code=%s QuotaName="%s" has '
-                'Units set to "%s"; awslimitchecker does not know how to '
+                'Units set to "%s", but expected units to be "%s"; '
+                'awslimitchecker does not know how to '
                 'handle this. This quota will be ignored. Please open a bug '
                 'report.', service_code, quota_name,
-                svc[quota_name.lower()]['Unit']
+                svc[quota_name.lower()]['Unit'], units
             )
             return None
         return val

awslimitchecker/services/__init__.py

@@ -51,8 +51,10 @@
 from awslimitchecker.services.elasticache import _ElastiCacheService
 from awslimitchecker.services.elasticbeanstalk import _ElasticBeanstalkService
 from awslimitchecker.services.elb import _ElbService
+from awslimitchecker.services.eks import _EksService
 from awslimitchecker.services.firehose import _FirehoseService
 from awslimitchecker.services.iam import _IamService
+from awslimitchecker.services.kinesis import _KinesisService
 from awslimitchecker.services.lambdafunc import _LambdaService
 from awslimitchecker.services.rds import _RDSService
 from awslimitchecker.services.redshift import _RedshiftService

awslimitchecker/services/ec2.py

@@ -640,12 +640,12 @@ def _get_limits_spot(self):
     def _find_usage_networking_sgs(self):
         """calculate usage for VPC-related things"""
         logger.debug("Getting usage for EC2 VPC resources")
-        sgs_per_vpc = defaultdict(int)
+        sg_count = 0
         rules_per_sg = defaultdict(int)
         for sg in self.resource_conn.security_groups.all():
             if sg.vpc_id is None:
                 continue
-            sgs_per_vpc[sg.vpc_id] += 1
+            sg_count += 1
             """
             see: https://github.com/jantman/awslimitchecker/issues/431
 
@@ -676,12 +676,10 @@ def _find_usage_networking_sgs(self):
                 )
             rules_per_sg[sg.id] = max(counts)
         # set usage
-        for vpc_id, count in sgs_per_vpc.items():
-            self.limits['Security groups per VPC']._add_current_usage(
-                count,
-                aws_type='AWS::EC2::VPC',
-                resource_id=vpc_id,
-            )
+        self.limits['VPC security groups per Region']._add_current_usage(
+            sg_count,
+            aws_type='AWS::EC2::SecurityGroup',
+        )
         for sg_id, count in rules_per_sg.items():
             self.limits['Rules per VPC security group']._add_current_usage(
                 count,
@@ -727,14 +725,16 @@ def _get_limits_networking(self):
         :rtype: dict
         """
         limits = {}
-        limits['Security groups per VPC'] = AwsLimit(
-            'Security groups per VPC',
+        limits['VPC security groups per Region'] = AwsLimit(
+            'VPC security groups per Region',
             self,
-            500,
+            2500,
             self.warning_threshold,
             self.critical_threshold,
             limit_type='AWS::EC2::SecurityGroup',
             limit_subtype='AWS::EC2::VPC',
+            quotas_name='VPC security groups per Region',
+            quotas_service_code='vpc'
         )
         limits['Rules per VPC security group'] = AwsLimit(
             'Rules per VPC security group',
@@ -744,6 +744,8 @@ def _get_limits_networking(self):
             self.critical_threshold,
             limit_type='AWS::EC2::SecurityGroup',
             limit_subtype='AWS::EC2::VPC',
+            quotas_name='Inbound or outbound rules per security group',
+            quotas_service_code='vpc'
         )
         limits['VPC Elastic IP addresses (EIPs)'] = AwsLimit(
             'VPC Elastic IP addresses (EIPs)',