A suite of utilities for AWS Lambda Functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier

Overview

AWS Lambda Powertools (Python)

Build codecov.io PythonSupport PyPI version PyPi monthly downloads

A suite of Python utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more. (AWS Lambda Powertools Java is also available).

πŸ“œ Documentation | 🐍 PyPi | Roadmap | Quick hello world example | Detailed blog post

Join us on the AWS Developers Slack at #lambda-powertools - Invite, if you don't have an account

Features

  • Tracing - Decorators and utilities to trace Lambda function handlers, and both synchronous and asynchronous functions
  • Logging - Structured logging made easier, and decorator to enrich structured logging with key Lambda context details
  • Metrics - Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF)
  • Event handler: AppSync - AWS AppSync event handler for Lambda Direct Resolver and Amplify GraphQL Transformer function
  • Event handler: API Gateway and ALB - Amazon API Gateway REST/HTTP API and ALB event handler for Lambda functions invoked using Proxy integration
  • Bring your own middleware - Decorator factory to create your own middleware to run logic before, and after each Lambda invocation
  • Parameters utility - Retrieve and cache parameter values from Parameter Store, Secrets Manager, or DynamoDB
  • Batch processing - Handle partial failures for AWS SQS batch processing
  • Typing - Static typing classes to speedup development in your IDE
  • Validation - JSON Schema validator for inbound events and responses
  • Event source data classes - Data classes describing the schema of common Lambda event triggers
  • Parser - Data parsing and deep validation using Pydantic
  • Idempotency - Convert your Lambda functions into idempotent operations which are safe to retry
  • Feature Flags - A simple rule engine to evaluate when one or multiple features should be enabled depending on the input

Installation

With pip installed, run: pip install aws-lambda-powertools

Examples

Credits

License

This library is licensed under the MIT-0 License. See the LICENSE file.

Issues
  • feat(feat-flags): new simple feature toggles rule engine

    feat(feat-flags): new simple feature toggles rule engine

    https://github.com/awslabs/aws-lambda-powertools-python/issues/470

    First poc branch. Has only pure python, no pydantic. Missing: md doc , schema validation implementstion (currently has a todo commengt there), more tests for exception cases

    usage example can be found at this gist: https://gist.github.com/risenberg-cyberark/903bc41079218bb719c98321f8e6152b

    area/utilities feature size/XXL tests 
    opened by ran-isenberg 49
  • feat(event-handler): prefixes to strip for custom mappings

    feat(event-handler): prefixes to strip for custom mappings

    Issue #, if available: https://github.com/awslabs/aws-lambda-powertools-roadmap/issues/34

    • closes #576

    Description of changes:

    Changes:

    • Add prefix to be stripped from the event path, default is None which does not modify the path
    • Add docstring for serializer

    Example usage:

    • Passing in multiple strip_prefixes "/unique/v1" and "/foo/v1"
    • Route /status will then match on /unique/v1/status (and /status)
    • By default strip_prefixes is None, and therefore does not touch the event path
    • Note the prefix should not end with a /
    import os
    from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
    
    app = ApiGatewayResolver(strip_prefixes=["/unique/v1", "/foo/v1"])
    
    @app.get("/status")
    def foo():
        return {"status": "OK"}
    
    # All calls should return a 200
    response = app({"httpMethod": "GET", "path": "/unique/v1/status"}, None)
    assert response["statusCode"] == 200
    response = app({"httpMethod": "GET", "path": "/foo/v1/status"}, None)
    assert response["statusCode"] == 200
    response = app({"httpMethod": "GET", "path": "/status"}, None)
    assert response["statusCode"] == 200
    

    Checklist

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    feature size/M area/event_handlers tests missing/docs 
    opened by michaelbrewer 25
  • feat: Advanced parser utility (pydantic)

    feat: Advanced parser utility (pydantic)

    Issue, if available: #147, #95

    Description of changes:

    Added a new validation module. It has the validator decorator code with 3 envelopes (eventbridge, dynamoDB and custom user) and tests. Also has the eventbridge & dynamoDB schemas.

    Checklist

    Breaking change checklist

    **RFC issue #95 :

    • [ ] Migration process documented
    • [ ] Implement warnings (if it can live side by side)

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    area/utilities feature 
    opened by ran-isenberg 25
  • `child=True` is not creating a logger?

    `child=True` is not creating a logger?

    What were you trying to accomplish?

    Re-use a logger in a sub-module, using the child=True feature, per the documentation.

    Expected Behavior

    With the following setup, there should be two log entries, one with "service": "foo" and one with "service": "foo.bar".

    cat foo.py:

    $ cat foo.py
    import aws_lambda_powertools
    import bar
    
    log = aws_lambda_powertools.Logger(service="foo")
    
    log.info('hallelujah')
    bar.bar()
    

    cat bar.py:

    import aws_lambda_powertools
    
    log = aws_lambda_powertools.Logger(child=True)
    
    def bar():
        log.info('doublestuff')
    

    Current Behavior

    $ python foo.py
    {"timestamp": "2020-08-27 11:20:16,245", "level": "INFO", "location": "<module>:6", "service": "foo", "sampling_rate": 0.0, "message": "hallelujah"}
    

    Even setting the service explicitly in the submodule to foo.bar, the service remains foo in the output, though at least now their are two log entries.

    cat bar.py:

    $ cat bar.py
    import aws_lambda_powertools
    
    log = aws_lambda_powertools.Logger(service="foo.bar", child=True)
    
    def bar():
        log.info('doublestuff')
    
    $ξ‚° python foo.py
    {"timestamp": "2020-08-27 11:26:05,330", "level": "INFO", "location": "<module>:6", "service": "foo", "sampling_rate": 0.0, "message": "hallelujah"}
    {"timestamp": "2020-08-27 11:26:05,330", "level": "INFO", "location": "bar:6", "service": "foo", "sampling_rate": 0.0, "message": "doublestuff"}
    

    Steps to Reproduce (for bugs)

    See above.

    Environment

    • Powertools version used: 1.4.0
    • Packaging format (Layers, PyPi): PyPi
    • AWS Lambda function runtime: running locally, not in lambda yet
    documentation area/logger 
    opened by lorengordon 24
  • feat(apigateway): add Router to allow large routing composition

    feat(apigateway): add Router to allow large routing composition

    Add Router class to ApiGatewayResolver

    Issue #, if available: #644

    Description of changes:

    Add a Router class to be used as a proxy for ApiGatewayResolver. The class is able to at as a placeholder for the app instance and collect routes to later be applied; it includes all the functionality of current routes.

    To access query strings or headers the original event and context are made available as router.current_event and router.lambda_context respectively. Also, to improve performance of a function which uses an endpoint with multiple methods, it offers a slight variation of the route() method's signature allowing a list/tuple.

    Usage example:

    app/main.py:

    from aws_lambda_powertools import Logger, Tracer
    from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
    from aws_lambda_powertools.logging import correlation_paths
    
    from .routers import users, items
    
    tracer = Tracer()
    logger = Logger()
    app = ApiGatewayResolver()
    app.include_router(users.router)
    app.include_router(items.router, prefix="/items")
    
    @logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
    @tracer.capture_lambda_handler
    def lambda_handler(event, context):
        return app.resolve(event, context)
    

    app/routers/items.py:

    from aws_lambda_powertools import Logger, Tracer
    from aws_lambda_powertools.event_handler.api_gateway import Router
    
    tracer = Tracer()
    logger = Logger(child=True)
    router = Router()
    
    @router.get('/hello')
    @tracer.capture_method
    def get_hello():
        return {
            "message": ['/hello', 'GET'],
            "query_string": router.current_event.query_string_parameters,
        }
    
    @router.route('/world', ('GET','POST'))
    @tracer.capture_method
    def multi_world():
        return {
            "message": ['/world', 'GET/POST'],
            "query_string": router.current_event.query_string_parameters,
        }
    

    Checklist

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    feature size/L area/event_handlers tests 
    opened by BVMiko 23
  • Ability to swap out the logger with default python logger during local testing

    Ability to swap out the logger with default python logger during local testing

    Is your feature request related to a problem? Please describe.

    When working locally, we often wish to visualise the "message" of logs without structured logging as it is more readable and we have no intention of searching the logs. We also wish to log to file often when testing locally.

    Is there any way we can hot-swap the default Python logger or replace the logger universally across a codebase?

    Describe the solution you'd like

    Ability to disable structured logging across a code base and default to a standard python logger such that we have a way to make the code base log to local file and display without structured logs during local testing.

    Describe alternatives you've considered

    I thought initially we could just mock out the logger with the default logger, but the introduction of child=True (which is not in the default logger) would mean that does not work I think.

    I was thinking maybe we could also try and add the default logger as an additional logger by patching the init of the powertools logger maybe?

    status/pending-information area/logger feature-request 
    opened by bml1g12 23
  • Add AWS CDK examples for creating and using a layer from the Serverless Application Repository

    Add AWS CDK examples for creating and using a layer from the Serverless Application Repository

    What were you initially searching for in the docs? Using the AWS CDK, I wanted to deploy a Lambda layer from the Powertools Serverless Application Repository ARN for re-use within my CDK application.

    The Lambda Layer documentation does not include how to do this.

    Is this related to an existing part of the documentation? Please share a link https://awslabs.github.io/aws-lambda-powertools-python/#lambda-layer

    Describe how we could make it clearer A code snippet and details could be provided to make it easier for a customer to get up and running faster.

    If you have a proposed update, please share it here

    Using similar language to the existing documentation, but some rewording could also used to provide the overall detail with SAM and CDK examples below it.

    If using the AWS CDK, you can create include this SAR App and lock to a specific semantic version. The Layer can be used in the same CDK application. Once deployed, it'll be available across the account this is deployed to.

    ** insert a TypeScript CDK example **

    # Create the AWS Lambda Powertools for Python layer
    powertools_arn = 'arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer'
    powertools_application = aws_sam.CfnApplication(self, 'AWSLambdaPowertoolsApplication',
                                                    location={'applicationId': powertools_arn,
                                                              'semanticVersion': '1.12.0'})
    powertools_resource = cdk.CfnResource(powertools_application, 'AWSLambdaPowertoolsResource',
                                          type='AWS::Serverless::Application',
                                          properties={'Location': {
                                              'ApplicationId': powertools_arn,
                                              'SemanticVersion': '1.12.0'
                                          }})
    powertools_layer = aws_lambda.LayerVersion.from_layer_version_arn(
        self, 'AWSLambdaPowertoolsLayer', powertools_resource.get_att("Outputs.LayerVersionArn").to_string())
    
    # Reference the Layer in a Lambda definition
    my_function = aws_lambda.Function(
        self, "MyFunction",
        layers=[powertools_layer],
        runtime=aws_lambda.Runtime.PYTHON_3_8,
    )
    
    documentation 
    opened by austoonz 22
  • RFC: Collection of tiny utilities

    RFC: Collection of tiny utilities

    Background

    At GA, Powertools will offer Tracer, Metrics, Logger, and Middleware factory as the core utilities.

    Optionally, I'm pondering on the idea of providing a set of tiny handy utilities that can be used either as standalone functions, or as part of a custom middleware - For example, JSON serialization, detect retries, fetch secrets, etc.

    The benefit of providing utilities as functions is two-fold a) ease of maintenance, b) pick and choose and create a single custom middleware instead of nesting a myriad of decorators.

    Propose solution

    Use case: Custom middleware using a handful of utilities

    from aws_lambda_powertools.middleware_factory import lambda_handler_decorator
    from aws_lambda_powertools.utilities import validate_event, cors, decode_json, detect_retry
    
    @lambda_handler_decorator
    def custom_middleware_name(handler, event, context):
        # Before
        detect_retry(event)
        event = decode_json(event)
        validate_event(event)
    
        response = handler(event, context)
    
        # After
        response = cors(response)
        return response
    
    @custom_middleware_name
    def lambda_handler(event, context):
        ...
    

    Use case: Using utilities standalone

    from aws_lambda_powertools.utilities import validate_event, cors, decode_json, detect_retry
    
    @custom_middleware_name
    def lambda_handler(event, context):
        detect_retry(event)
        event = decode_json(event)
        validate_event(event)
    
        return cors(result)
    

    Request for comments

    As part of this RFC, I'd like to know what utilities are the most useful to have upfront - Leave a comment, and vote using πŸ‘ on each comment instead of a new comment.

    For ideas, here are some utilities as decorators created by other awesome authors: Lambda Decorators by Grid smarter cities, and Lambda Decorators by Daniel Schep.

    Tenets

    • AWS Lambda only – We optimise for AWS Lambda function environments only. Utilities might work with web frameworks and non-Lambda environments, though they are not officially supported.
    • Eases the adoption of best practices – The main priority of the utilities is to facilitate best practices adoption, as defined in the AWS Well-Architected Serverless Lens; all other functionality is optional.
    • Keep it lean – Additional dependencies are carefully considered for security and ease of maintenance, and prevent negatively impacting startup time.
    • We strive for backwards compatibility – New features and changes should keep backwards compatibility. If a breaking change cannot be avoided, the deprecation and migration process should be clearly defined.
    • We work backwards from the community – We aim to strike a balance of what would work best for 80% of customers. Emerging practices are considered and discussed via Requests for Comment (RFCs)
    • Idiomatic – Utilities follow programming language idioms and language-specific best practices.

    * Core utilities are Tracer, Logger and Metrics. Optional utilities may vary across languages.

    help wanted RFC 
    opened by heitorlessa 22
  • copy_config_to_registered_loggers results in duplicate logs when using with existing loggers

    copy_config_to_registered_loggers results in duplicate logs when using with existing loggers

    When using utils.copy_config_to_registered_loggers() to copy the PowerLogger config to existing loggers created by a library, this results in duplicate log entries in AWS Lambda

    Expected Behavior

    When copying config from the AWS Powertools logger to existing loggers instantiated by a library, I expect the log statements of that library to appear once in the log stream, formatted as JSON

    Current Behavior

    When copying config from the AWS Powertools logger to existing loggers, I see log statements from existing loggers appearing twice in the log stream: once properly formatted as JSON and once as plain text

    image

    Possible Solution

    I don't know why is this happening, so I can't really comment on a solution πŸ˜…

    Steps to Reproduce (for bugs)

    1. Have a library that instantiates loggers the "regular" Python way, like this:
    logger = logging.getLogger(__name__)
    
    class MyClass:
        def my_func():
            logger.info("Hello")
    
    1. Import this library into your Lambda function
    2. Instantiate a logger from AWS Lambda Powertools and copy the config to existing loggers:
    from aws_lambda_powertools import Logger
    from mylibrary import MyClass
    
    logger = Logger() # POWERTOOLS_SERVICE_NAME=myservice
    utils.copy_config_to_registered_loggers(source_logger=logger)
    
    1. Observe duplicated logs in the Lambda log stream

    Environment

    • Powertools version used: 1.25.1
    • Packaging format (Layers, PyPi): Docker image
    • AWS Lambda function runtime: Python 3.9
    bug 
    opened by DonDebonair 21
  • feat(feature_flags): add in feature_flags RE_* actions and update tests/docs

    feat(feature_flags): add in feature_flags RE_* actions and update tests/docs

    Issue #, if available:

    #690

    Description of changes:

    Extend feature_toggles to enable regular expression actions... next step - lets just use embedded lua!

    Checklist

    Breaking change checklist

    Nope

    RFC issue #:

    Nope

    • [ ] Migration process documented
    • [ ] Implement warnings (if it can live side by side)

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.


    View rendered docs/utilities/feature_flags.md


    View rendered docs/utilities/feature_flags.md

    documentation status/pending-information area/utilities size/L tests p3 
    opened by whardier 21
  • refactor(feature-flags): optimize UX and maintenance

    refactor(feature-flags): optimize UX and maintenance

    Issue #, if available: #494

    Description of changes:

    Checklist

    New UX

    from typing import List
    
    from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore
    
    app_config = AppConfigStore(
        environment="test",
        application="powertools",
        name="test_conf_name",
        cache_seconds=600,
    )
    
    feature_flags: FeatureFlags = FeatureFlags(store=app_config)
    
    is_my_feature_active: bool = feature_flags.evaluate(name="my_feature", context={}, default=False)
    
    # TODO: This needs reviewing, possibly meant as List[Dict[str, bool]]???
    all_features: List[str] = feature_flags.get_enabled_features(context={})
    

    Using envelope argument to pluck feature flags in an arbitrary key from fetched config

    app_config = AppConfigStore(
        environment="test",
        application="powertools",
        name="test_conf_name",
        cache_seconds=600,
        envelope="features"    # feature toggles are under `features` key in fetched config
    )
    
    feature_flags: FeatureFlags = FeatureFlags(store=app_config)
    

    Changes

    Major changes made during initial doc writing - Due to the size, we're splitting this PR to separate actual docs.

    • [x] General
      • [x] ~~Rename feature_toggles to feature_flags~~
      • [x] ~~Refactor error loggers to contextual exceptions~~
    • [x] Improve exceptions
      • [x] ~~Consider an additional fields to improve exception handling such as error type, feature, etc~~
        • [x] ~~String concatenation ended up slowing things down, dropping it~~
      • [x] ~~Consider fine grained ConfigurationError instead of a single ConfigurationError~~
      • [x] ~~Rename it to InvalidSchemaError~~
    • [x] Refactor Schema to ease maintenance
      • [x] ~~Simplify schema by removing features key~~
      • [x] ~~Make each rule in rules unique by using Dict over List, and remove rule_name~~
      • [x] ~~Review long and redundant names such as "feature": { "feature_default_value"...}, "rules": [{"rule_name"...}]~~
      • [x] ~~Rename feature_default_Value to default~~
      • [x] ~~Rename value_when_applies to when_match~~
      • [x] ~~Refactor schema into multiple validators to ease maintenance~~
      • [x] ~~Use module logger over class logger~~
      • [x] ~~Create classes for each validation over multiple methods~~
      • [x] ~~Add typing extensions as feature flags and other features become difficult to maintain without things like TypedDict, improved generics for mypy, etc.~~ - Note: MyPy doesn't support TypedDic when you use a variable name as a dict key; ignoring it
      • [x] ~~Rename Action to RuleAction enum~~
      • [x] ~~Suggest what RuleActions are valid when invalid is provided~~
    • [x] Refactor ConfigurationStore to FeatureFlags
      • [x] ConfigurationStore to FeatureFlags
      • [x] ~~rules_context to context~~
      • [x] ~~Rename get_feature method to evaluate~~
      • [x] ~~Rename get_all_enabled_feature_toggles to get_enabled_features~~
    • [x] Refactor SchemaFetcher to StoreProvider
      • [x] SchemaFetcher to StoreProvider
      • [x] ~~Use base.py for interfaces for consistency (e.g. Metrics, Tracer, etc.)~~
      • [x] ~~AppConfigFetcher to AppConfigStore~~
      • [x] ~~AppConfig construct parameter names for consistency (e.g. configuration_name -> name, service -> application)~~
      • [x] ~~Rename value_if_missing param to default to match Python consistency (e.g. os.getenv("VAR", default=False))~~
      • [x] ~~Rename get_configuration method to get_json_configuration~~
    • [x] Complete docstrings and logging
      • [x] ~~Store section~~
      • [x] ~~FeatureFlags section~~
      • [x] ~~Schema section~~
      • [x] ~~Add schema specification~~
    • [x] Test conditional value key being a dictionary
    • [x] Load test

    Time allowing

    • [ ] Alternative to testing feature flag private method e.g. test_is_rule_matched_no_matches

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    documentation area/utilities feature internal size/XXL tests 
    opened by heitorlessa 21
  • Docs: Idempotency - `Exception not affecting idempotency record sample` example has python syntax errors

    Docs: Idempotency - `Exception not affecting idempotency record sample` example has python syntax errors

    Expected Behaviour

    Handling exceptions - Exception not affecting idempotency record sample example should not have python syntax errors or missing imports.

    Current Behaviour

    This example have python syntax errors and missing imports

    Screen Shot 2022-05-17 at 9 27 47 PM

    Code snippet

    def lambda_handler(event, context):
        # If an exception is raised here, no idempotent record will ever get created as the
        # idempotent function does not get called
        do_some_stuff()
    
        result = call_external_service(data={"user": "user1", "id": 5})
    
        # This exception will not cause the idempotent record to be deleted, since it
        # happens after the decorated function has been successfully called
        raise Exception
    
    
    @idempotent_function(data_keyword_argument="data", config=config, persistence_store=dynamodb)
    def call_external_service(data: dict, **kwargs):
        result = requests.post('http://example.com', json={"user": data['user'], "transaction_id": data['id']}
        return result.json()
    

    Possible Solution

    Fix python syntax errors like the PR #1119 , and deployed over here https://gyft.github.io/aws-lambda-powertools-python/latest/utilities/idempotency/#handling-exceptions

    import requests
    
    from aws_lambda_powertools.utilities.idempotency import DynamoDBPersistenceLayer, IdempotencyConfig, idempotent_function
    
    dynamodb = DynamoDBPersistenceLayer(table_name="idem")
    config = IdempotencyConfig(event_key_jmespath="order_id")
    
    
    def lambda_handler(event, context):
        # If an exception is raised here, no idempotent record will ever get created as the
        # idempotent function does not get called
        do_some_stuff()
    
        result = call_external_service(data={"user": "user1", "id": 5})
    
        # This exception will not cause the idempotent record to be deleted, since it
        # happens after the decorated function has been successfully called
        raise Exception
    
    
    @idempotent_function(data_keyword_argument="data", config=config, persistence_store=dynamodb)
    def call_external_service(data: dict, **kwargs):
        result = requests.post("http://example.com", json={"user": data["user"], "transaction_id": data["id"]})
        return result.json()
    

    Steps to Reproduce

    1. Go to https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/idempotency/#handling-exceptions
    2. Copy Exception not affecting idempotency record sample
    3. Code example should not be able to compile

    AWS Lambda Powertools for Python version

    latest

    AWS Lambda function runtime

    3.9

    Packaging format used

    PyPi

    Debugging logs

    N/A
    
    bug triage 
    opened by michaelbrewer 0
  • Docs: Logger - `Cloning Logger config to all other registered standard loggers` example has errors

    Docs: Logger - `Cloning Logger config to all other registered standard loggers` example has errors

    Expected Behaviour

    Cloning Logger config to all other registered standard loggers - example in FAQ should not have any errors

    Current Behaviour

    Running the provided snippet results in the following error

    Screen Shot 2022-05-17 at 8 38 16 PM

    Code snippet

    import logging
    
    from aws_lambda_powertools import Logger
    from aws_lambda_powertools.logging import utils
    
    logger = Logger()
    
    external_logger = logging.logger()
    
    utils.copy_config_to_registered_loggers(source_logger=logger)
    external_logger.info("test message")
    

    Possible Solution

    Use the correct method getLogger and the code will run, see PR #1121 for a fix and visit https://gyft.github.io/aws-lambda-powertools-python/latest/core/logger/#faq for a deployed version of the docs

    import logging
    
    from aws_lambda_powertools import Logger
    from aws_lambda_powertools.logging import utils
    
    logger = Logger()
    
    external_logger = logging.getLogger(name="example")
    
    utils.copy_config_to_registered_loggers(source_logger=logger)
    external_logger.info("test message")
    

    Now the output is:

    {"level":"INFO","location":"<module>:11","message":"test message","timestamp":"2022-05-17 20:41:19,810-0700","service":"service_undefined"}
    

    Steps to Reproduce

    1. Go to https://awslabs.github.io/aws-lambda-powertools-python/latest/core/logger/#faq
    2. Copy code example for Cloning Logger config to all other registered standard loggers
    3. Run the example code

    AWS Lambda Powertools for Python version

    latest

    AWS Lambda function runtime

    3.9

    Packaging format used

    PyPi

    Debugging logs

    N/A
    
    bug triage 
    opened by michaelbrewer 0
  • Docs: JMESPath - python syntax error in `custom_jmespath_function.py`

    Docs: JMESPath - python syntax error in `custom_jmespath_function.py`

    Expected Behaviour

    Code example in custom_jmespath_function.py should be valid python

    Current Behaviour

    Python example has a syntax error Expression expected

    Screen Shot 2022-05-17 at 7 29 32 PM

    Code snippet

    from aws_lambda_powertools.utilities.jmespath_utils import (
        PowertoolsFunctions, extract_data_from_envelope)
    from jmespath.functions import signature
    
    
    class CustomFunctions(PowertoolsFunctions):
        @signature({'types': ['string']})  # Only decode if value is a string
        def _func_special_decoder(self, s):
            return my_custom_decoder_logic(s)
    
    custom_jmespath_options = {"custom_functions": CustomFunctions()}
    
    def handler(event, context):
        # use the custom name after `_func_`
        extract_data_from_envelope(data=event,
                                  envelope="special_decoder(body)",
                                  jmespath_options=**custom_jmespath_options)
        ...
    

    Possible Solution

    Fix code example like in PR #1120 , a deployed example https://gyft.github.io/aws-lambda-powertools-python/latest/utilities/jmespath_functions/#bring-your-own-jmespath-function

    from jmespath.functions import signature
    
    from aws_lambda_powertools.utilities.jmespath_utils import PowertoolsFunctions, extract_data_from_envelope
    
    
    class CustomFunctions(PowertoolsFunctions):
        @signature({"types": ["string"]})  # Only decode if value is a string
        def _func_special_decoder(self, s):
            return my_custom_decoder_logic(s)
    
    
    custom_jmespath_options = {"custom_functions": CustomFunctions()}
    
    
    def handler(event, context):
        # use the custom name after `_func_`
        extract_data_from_envelope(
            data=event,
            envelope="special_decoder(body)",
            jmespath_options=custom_jmespath_options,
        )
        ...
    

    Steps to Reproduce

    1. Go to https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/jmespath_functions/#bring-your-own-jmespath-function
    2. Copy code example for custom_jmespath_function.py
    3. Try to compile

    AWS Lambda Powertools for Python version

    latest

    AWS Lambda function runtime

    3.9

    Packaging format used

    PyPi

    Debugging logs

    N/A
    
    bug triage 
    opened by michaelbrewer 0
  • Docs: Metrics - Invalid AWS Serverless Application Model (SAM) example

    Docs: Metrics - Invalid AWS Serverless Application Model (SAM) example

    Expected Behaviour

    AWS Serverless Application Model (SAM) template.yaml should be valid without errors

    Current Behaviour

    Current example has missing attributes and generates the following errors.

    Screen Shot 2022-05-17 at 6 23 19 PM

    Code snippet

    Resources:
      HelloWorldFunction:
        Type: AWS::Serverless::Function
        Properties:
          Runtime: python3.8
          Environment:
          Variables:
            POWERTOOLS_SERVICE_NAME: payment
            POWERTOOLS_METRICS_NAMESPACE: ServerlessAirline
    

    Possible Solution

    Template yaml should be, like in PR #1122 , deployed example is here https://gyft.github.io/aws-lambda-powertools-python/latest/core/metrics/#getting-started

    AWSTemplateFormatVersion: "2010-09-09"
    Transform: AWS::Serverless-2016-10-31
    Resources:
      HelloWorldFunction:
        Type: AWS::Serverless::Function
        Properties:
          CodeUri: src/
          Handler: app.lambda_handler
          Runtime: python3.9
          Environment:
            Variables:
              POWERTOOLS_SERVICE_NAME: payment
              POWERTOOLS_METRICS_NAMESPACE: ServerlessAirline
    

    Steps to Reproduce

    1. Go to https://awslabs.github.io/aws-lambda-powertools-python/latest/core/metrics/#getting-started
    2. Copy template.yml example
    3. Run sam validate and cfn-lint

    AWS Lambda Powertools for Python version

    latest

    AWS Lambda function runtime

    3.9

    Packaging format used

    PyPi

    Debugging logs

    N/A
    
    bug triage 
    opened by michaelbrewer 0
  • Docs: Metrics - Assert multiple EMF blobs with pytest has invalid python syntax

    Docs: Metrics - Assert multiple EMF blobs with pytest has invalid python syntax

    Expected Behaviour

    Metrics - Assert multiple EMF blobs with pytest example should be valid python syntax

    Current Behaviour

    Current example has a python syntax error of ',' or ')' expected

    Screen Shot 2022-05-17 at 6 02 25 PM

    Code snippet

    from aws_lambda_powertools import Metrics
    from aws_lambda_powertools.metrics import MetricUnit
    
    from collections import namedtuple
    
    import json
    
    def capture_metrics_output_multiple_emf_objects(capsys):
        return [json.loads(line.strip()) for line in capsys.readouterr().out.split("\n") if line]
    
    def test_log_metrics(capsys):
        # GIVEN Metrics is initialized
        metrics = Metrics(namespace="ServerlessAirline")
    
        # WHEN log_metrics is used with capture_cold_start_metric
        @metrics.log_metrics(capture_cold_start_metric=True)
        def lambda_handler(evt, ctx):
            metrics.add_metric(name="SuccessfulBooking", unit=MetricUnit.Count, value=1)
            metrics.add_dimension(name="environment", value="prod")
    
        # log_metrics uses function_name property from context to add as a dimension for cold start metric
        LambdaContext = namedtuple("LambdaContext", "function_name")
        lambda_handler({}, LambdaContext("example_fn")
    
        cold_start_blob, custom_metrics_blob = capture_metrics_output_multiple_emf_objects(capsys)
    
        # THEN ColdStart metric and function_name dimension should be logged
        # in a separate EMF blob than the application metrics
        assert cold_start_blob["ColdStart"] == [1.0]
        assert cold_start_blob["function_name"] == "example_fn"
    
        assert "SuccessfulBooking" in custom_metrics_blob  # as per previous example
    

    Possible Solution

    Fix the python syntax, like in PR #1122, deployed at https://gyft.github.io/aws-lambda-powertools-python/latest/core/metrics/#functional-testing

    import json
    from collections import namedtuple
    
    from aws_lambda_powertools import Metrics
    from aws_lambda_powertools.metrics import MetricUnit
    
    
    def capture_metrics_output_multiple_emf_objects(capsys):
        return [json.loads(line.strip()) for line in capsys.readouterr().out.split("\n") if line]
    
    
    def test_log_metrics(capsys):
        # GIVEN Metrics is initialized
        metrics = Metrics(namespace="ServerlessAirline")
    
        # WHEN log_metrics is used with capture_cold_start_metric
        @metrics.log_metrics(capture_cold_start_metric=True)
        def lambda_handler(evt, ctx):
            metrics.add_metric(name="SuccessfulBooking", unit=MetricUnit.Count, value=1)
            metrics.add_dimension(name="environment", value="prod")
    
        # log_metrics uses function_name property from context to add as a dimension for cold start metric
        LambdaContext = namedtuple("LambdaContext", "function_name")
        lambda_handler({}, LambdaContext("example_fn"))
    
        cold_start_blob, custom_metrics_blob = capture_metrics_output_multiple_emf_objects(capsys)
    
        # THEN ColdStart metric and function_name dimension should be logged
        # in a separate EMF blob than the application metrics
        assert cold_start_blob["ColdStart"] == [1.0]
        assert cold_start_blob["function_name"] == "example_fn"
    
        assert "SuccessfulBooking" in custom_metrics_blob  # as per previous example
    

    Steps to Reproduce

    1. Go to https://awslabs.github.io/aws-lambda-powertools-python/latest/core/metrics/#functional-testing
    2. Copy code for Assert multiple EMF blobs with pytest
    3. Try to compile the python file

    AWS Lambda Powertools for Python version

    latest

    AWS Lambda function runtime

    3.9

    Packaging format used

    PyPi

    Debugging logs

    N/A
    
    bug triage 
    opened by michaelbrewer 0
  • Docs:  Batch - Testing your code examples has syntax errors

    Docs: Batch - Testing your code examples has syntax errors

    Expected Behaviour

    Batch -Testing your code examples should not contain python syntax errors

    Current Behaviour

    Missing closing quote ["] present in the code examples

    Screen Shot 2022-05-17 at 2 48 49 PM

    Code snippet

    import json
    
    from pathlib import Path
    from dataclasses import dataclass
    
    import pytest
    from src.app import lambda_handler, processor
    
    
    def load_event(path: Path):
        with path.open() as f:
            return json.load(f)
    
    
    @pytest.fixture
    def lambda_context():
        @dataclass
        class LambdaContext:
            function_name: str = "test"
            memory_limit_in_mb: int = 128
            invoked_function_arn: str = "arn:aws:lambda:eu-west-1:809313241:function:test"
            aws_request_id: str = "52fdfc07-2182-154f-163f-5f0f9a621d72"
    
        return LambdaContext()
    
    @pytest.fixture()
    def sqs_event():
        """Generates API GW Event"""
        return load_event(path=Path("events/sqs_event.json"))
    
    
    def test_app_batch_partial_response(sqs_event, lambda_context):
        # GIVEN
        processor = app.processor  # access processor for additional assertions
        successful_record = sqs_event["Records"][0]
        failed_record = sqs_event["Records"][1]
        expected_response = {
            "batchItemFailures: [
                {
                    "itemIdentifier": failed_record["messageId"]
                }
            ]
        }
    
        # WHEN
        ret = app.lambda_handler(sqs_event, lambda_context)
    
        # THEN
        assert ret == expected_response
        assert len(processor.fail_messages) == 1
        assert processor.success_messages[0] == successful_record
    

    Possible Solution

    Fix the code testing examples as in #1114 and deployed as an example here: https://gyft.github.io/aws-lambda-powertools-python/latest/utilities/batch/#testing-your-code

    import json
    from dataclasses import dataclass
    from pathlib import Path
    
    import pytest
    from src.app import lambda_handler, processor
    
    
    def load_event(path: Path):
        with path.open() as f:
            return json.load(f)
    
    
    @pytest.fixture
    def lambda_context():
        @dataclass
        class LambdaContext:
            function_name: str = "test"
            memory_limit_in_mb: int = 128
            invoked_function_arn: str = "arn:aws:lambda:eu-west-1:809313241:function:test"
            aws_request_id: str = "52fdfc07-2182-154f-163f-5f0f9a621d72"
    
        return LambdaContext()
    
    
    @pytest.fixture()
    def sqs_event():
        """Generates API GW Event"""
        return load_event(path=Path("events/sqs_event.json"))
    
    
    def test_app_batch_partial_response(sqs_event, lambda_context):
        # GIVEN
        processor = app.processor  # access processor for additional assertions
        successful_record = sqs_event["Records"][0]
        failed_record = sqs_event["Records"][1]
        expected_response = {
            "batchItemFailures": [
                {
                    "itemIdentifier": failed_record["messageId"],
                },
            ],
        }
    
        # WHEN
        ret = app.lambda_handler(sqs_event, lambda_context)
    
        # THEN
        assert ret == expected_response
        assert len(processor.fail_messages) == 1
        assert processor.success_messages[0] == successful_record
    
    

    Steps to Reproduce

    1. Copy code example from https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/batch/#testing-your-code
    2. Try to compile this and it should result in a Missing closing quote ["] error

    AWS Lambda Powertools for Python version

    latest

    AWS Lambda function runtime

    3.9

    Packaging format used

    PyPi

    Debugging logs

    N/A
    
    bug triage 
    opened by michaelbrewer 0
Releases(v1.25.10)
  • v1.25.10(Apr 29, 2022)

    Summary

    This patch release addresses the following main issues:

    • Event Handler: Previously, Lambda handlers decorated with event source data class that also use Event Handler were incompatible
    • Event Handler: Raising a ServiceError like NotFound as part of an exception handler ended up propagating a ServiceError instead of converting into a non-successful response (e.g., 4xx)
    • Event Source Data Classes: Missing date field type annotation in SESEvent to string instead of list, and missing fields like cc, bcc, sender, and reply_to, and dkim related fields
    • Parameters: Previously, customers couldn't easily clear parameters (SSM, Secrets Manager, AppConfig, DynamoDB) cache as part of their automated tests

    image

    Changes

    🌟 New features and non-breaking changes

    • feat(parameters): add clear_cache method for providers (#1194) by @heitorlessa

    πŸ› Bug and hot fixes

    • fix(data-classes): fix SES date field typing and add missing fields (#1045) by @michaelbrewer
    • fix(deps-dev): remove jmespath due to dev deps conflict (#1148) by @sthulb
    • fix(deps): ignore boto3 updates until needed (#1151) by @sthulb
    • fix(event_handler): allow event source data class events (#1159) by @michaelbrewer
    • fix(event_handler): exception_handler to handle ServiceError exceptions (#1160) by @michaelbrewer

    πŸ”§ Maintenance

    • chore(deps-dev): bump flake8-bugbear from 22.1.11 to 22.4.25 (#1156) by @dependabot
    • chore(deps-dev): bump mypy from 0.942 to 0.950 (#1162) by @dependabot
    • chore(deps-dev): bump flake8-eradicate from 1.2.0 to 1.2.1 (#1158) by @dependabot
    • chore(deps): bump github/codeql-action from 1 to 2 (#1154) by @dependabot
    • chore(ci): disable mergify configuration after breaking changes (#1188) by @heitorlessa
    • fix(deps): ignore boto3 updates until needed (#1151) by @sthulb
    • chore(deps-dev): bump mkdocs-git-revision-date-plugin from 0.3.1 to 0.3.2 (#1146) by @dependabot
    • chore(deps): bump codecov/codecov-action from 3.0.0 to 3.1.0 (#1143) by @dependabot
    • chore(deps-dev): bump xenon from 0.8.0 to 0.9.0 (#1145) by @dependabot
    • chore(deps-dev): bump mypy from 0.931 to 0.942 (#1133) by @dependabot
    • fix(deps-dev): remove jmespath due to dev deps conflict (#1148) by @sthulb

    This release was made possible by the following contributors:

    @dependabot, @dependabot[bot], @heitorlessa, @huonw, @michaelbrewer, and @sthulb

    Source code(tar.gz)
    Source code(zip)
  • v1.25.9(Apr 21, 2022)

    Summary

    Quick patch release to fix jmespath pip marker to exact Python 3.6 version to address conflict in later versions.

    Changes

    πŸ› Bug and hot fixes

    • fix(deps): correct py36 marker for jmespath

    This release was made possible by the following contributors:

    @dependabot[bot], @heitorlessa and @mergify[bot]

    Source code(tar.gz)
    Source code(zip)
  • v1.25.8(Apr 21, 2022)

    Summary

    This patch release updates jmespath marker to address a dependency conflict for customers bundling the latest version of boto3, as part of their Lambda deployment package. Boto's latest version upgraded jmespath to 1.0 which drops support for Python 3.6.

    Changes

    πŸ“œ Documentation updates

    • docs(layer): remove link from clipboard button (#1135) by @heitorlessa

    πŸ› Bug and hot fixes

    • fix(deps): update jmespath marker to support 1.0 and py3.6 (#1139) by @michaelbrewer

    πŸ”§ Maintenance

    • chore(deps-dev): bump mkdocs-material from 8.2.4 to 8.2.7 (#1131) by @dependabot
    • chore(deps-dev): bump pytest from 6.2.5 to 7.0.1 (#1063) by @dependabot

    This release was made possible by the following contributors:

    @Dunedan, @dependabot, @dependabot[bot], @heitorlessa, @corey-cole, @sthulb, and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.25.7(Apr 8, 2022)

    Summary

    This patch release addresses bugs on Logger, Event Handler REST API, Idempotency, and a static type annotation on middleware factory utility.

    We also updated our issues to use the new GitHub Issue Form, including a new Ask a Question, and Static typing mismatch report. The latter addresses an increase of bug reports that are better categorized as static typing mismatch - this will help us understand whether we might be able to support both Mypy and Pyright in the future.

    image

    Logger

    Thanks to @eliasbrange, customers using clear_state=True will no longer be affected by custom keys being accidentally removed in subsequent invocations - a regression introduced two releases ago. We improved tests on clear_state functionality.

    Idempotency

    Thanks to @ojongerius, customers changing the incoming event within the Lambda handler will no longer impact idempotency hash calculation. This was caused by Python's default pass by reference behaviour.

    Event Handler REST API

    Thanks to @sthuber90, customers receiving API calls that contain whitespace in path parameters - /customer/power tools - will no longer return 404s. Despite API Gateway correctly decoding whitespace, our regex for safe and unsafe URI chars missed an explicit whitespace.

    Middleware Factory

    Thanks to @huonw, Mypy customers using --strict mode will no longer complain about the return type of custom decorators using lambda_handler_decorator. It's a hard problem to solve considering our support for Python 3.6+. Later in the year, we'll look into the value-effort ratio of supporting strict mode, and how to gradually bring typing_extensions package across the board.

    Huge thanks to new contributors: @sthuber90, @eliasbrange, and @ojongerius

    Changes

    πŸ› Bug and hot fixes

    • fix(api_gateway): allow whitespace in routes' path parameter (#1099) by @sthuber90
    • fix(middleware_factory): ret type annotation for handler dec (#1066) by @huonw
    • fix(logger): clear_state should keep custom key formats (#1095) by @eliasbrange
    • fix(idempotency): pass by value on idem key to guard inadvertent mutations (#1090) by @ojongerius

    This release was made possible by the following contributors:

    @eliasbrange, @heitorlessa, @huonw, @mergify[bot], @ojongerius and @sthuber90

    Source code(tar.gz)
    Source code(zip)
  • v1.25.6(Apr 1, 2022)

    Summary

    This patch release fixes a regression in Logger introduced in 1.25.3 when using clear_state=True. A fix for customer formatters introduced in 1.25.3 ended up introducing a regression, where standard keys like level, location and timestamp no longer being present when using clear_state=True for the default Logger formatter.

    Thanks to @alexanderluiscampino for a high quality bug report, and @HAK-CODE for confirming the issue two hours ago, we managed to reproduce it quickly, improved our tests, and thus making this emergency release.

    Changes

    πŸ› Bug and hot fixes

    • fix(logger): clear_state regression on absent standard keys (#1088) by @heitorlessa

    This release was made possible by the following contributors:

    @heitorlessa and @mergify[bot]

    Source code(tar.gz)
    Source code(zip)
  • v1.25.5(Mar 18, 2022)

    Summary

    Thanks to @DandyDev, this emergency release fixes a regression on logger utils to clone formatting configuration from Powertools Logger to external Loggers.

    Changes

    πŸ› Bug and hot fixes

    • fix(logger-utils): regression on exclude set leading to no formatter (#1080) by @mploski

    This release was made possible by the following contributors:

    @heitorlessa, @mergify[bot] and @mploski

    Source code(tar.gz)
    Source code(zip)
  • v1.25.4(Mar 17, 2022)

    Summary

    This is a patch release for those using Logger utils to clone Lambda Powertools Logger configuration to all existing Python Loggers.

    Thanks to @DandyDev for spotting a log duplication for external loggers, and for @mploski for helping fix it.

    Changes

    πŸ› Bug and hot fixes

    • fix(logger-utils): ensure external loggers doesn't propagate logs on config copy (#1075) by @mploski

    This release was made possible by the following contributors:

    @heitorlessa, @mploski and Michal Ploski

    Source code(tar.gz)
    Source code(zip)
  • v1.25.3(Mar 9, 2022)

    Summary

    This patch release is strictly about customers bringing their own Logging Formatter.

    This release ensures that when using Clear State with a custom formatter, all previously added keys will be cleared. It also guarantees that Lambda Context information will no longer be cleared abruptly before logs are printed (also for custom formatters).

    There is no code change needed on customers end. We also improved the documentation to clarify the most common use cases on bringing your own formatter, and exceptional cases where you want to replace the entire Powertools Logger Formatter logic.

    Huge thanks to @jacobdarrossi for spotting, and @ilias-at-adarma for confirming this bug.

    Changes

    πŸ“œ Documentation updates

    • fix(logger): ensure state is cleared for custom formatters (#1072) by @heitorlessa
    • docs(plugin): add mermaid to create diagram as code (#1070) by @dreamorosi

    πŸ› Bug and hot fixes

    • fix(logger): ensure state is cleared for custom formatters (#1072) by @heitorlessa

    This release was made possible by the following contributors:

    @dreamorosi and @heitorlessa

    Source code(tar.gz)
    Source code(zip)
  • v1.25.2(Mar 7, 2022)

    Summary

    This patch release addresses two main bugs: 1/ API Gateway Lambda Authorizer data class regex to validate for proxy resources (proxy+), and 2/ metrics to be automatically flushed when a single metric has 100 values.

    Next release will focus on two newly identified bugs on Logger (custom formatter, child logger). Subsequently, our bandwidth remains focused on addressing the subpar experiences and operational excellence.

    Changes

    πŸ“œ Documentation updates

    • docs(parser): APIGatewayProxyEvent to APIGatewayProxyEventModel (#1061) by @darnley
    • fix(event_handler): docs snippets, high-level import CorsConfig (#1019) by @michaelbrewer

    πŸ› Bug and hot fixes

    • fix(metrics): flush upon a single metric 100th data point (#1046) by @knightjoel
    • fix(lambda-authorizer): allow proxy resources path in arn (#1051) by @michaelbrewer
    • fix(event_handler): docs snippets, high-level import CorsConfig (#1019) by @michaelbrewer

    πŸ”§ Maintenance

    • chore(deps-dev): bump mkdocs-material from 8.1.9 to 8.2.4 (#1054) by @dependabot
    • chore(deps): bump actions/setup-python from 2.3.1 to 3 (#1048) by @dependabot
    • chore(deps): bump actions/checkout from 2 to 3 (#1052) by @dependabot
    • chore(deps): bump actions/github-script from 5 to 6 (#1023) by @dependabot
    • chore(deps): bump fastjsonschema from 2.15.2 to 2.15.3 (#949) by @dependabot

    This release was made possible by the following contributors:

    @darnley, @dependabot, @dependabot[bot], @heitorlessa, @knightjoel and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.25.1(Feb 14, 2022)

    Summary

    Emergency release to fix a critical bug in the new BatchProcessor feature spotted by Huon Wilson@huonw, and thanks to @michaelbrewer for the quick fix. The issue was exceptions not being consistently cleared between invocations (cold vs warm state).

    Changes

    πŸ› Bug and hot fixes

    • fix(batch): bugfix to clear exceptions between executions (#1022) by @michaelbrewer

    This release was made possible by the following contributors:

    @heitorlessa, @mergify[bot] and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.25.0(Feb 9, 2022)

    Summary

    This release continues to focus on addressing suboptimal experiences (a.k.a papercuts).

    But first... big thanks to our first time contributors @am1ru1, @houbie - thank you for helping us improve everyone's experience!

    Major improvements you should know

    1. Fully mypy compliant. After a multi-month task addressing over 600 issues, Mypy users can accurately use all 14 utilities typing annotations - thanks @mploski for getting us to the finishing line!
    2. New specialized Event Handler Resolvers. When using ApiGatewayResolver to access event properties, customers didn't have full IntelliSense support from their IDEs. This release introduces APIGatewayRestResolver, APIGatewayHttpResolver, and ALBResolver to accurately provide self-documented access to all properties available - big thanks to @michaelbrewer.
    3. Testing your code docs. We've updated Parameters & Event Handler GraphQL docs to be more realistic on how you can unit test your code - thanks to @cakepietoast

    Roadmap changes

    We're going to fully turn our attention to our list of papercuts. Due to new features and other bugs, we let it slip a couple of times, one being as old as ~7 months. We also learned that a central roadmap didn't work well for us. It made planning harder due to the split view (2x repos to look at) situation despite its best intentions.

    With GitHub's new Beta projects, we're confident we can migrate the roadmap back to each repository and give the visibility customers have been asking us. That is work we're currently doing, themes that need further research, areas we'd love contributions, and ideas we'd love to execute but lack bandwidth.

    Once this is complete, we will start a draft RFC of what a modularized Powertools v2 could look like. We will take everyone's input as much as possible to help shape what breaking changes are necessary, how we might facilitate new utilities like Testing, and what an early beta as well as migration guide should look like.

    Changes

    🌟New features and non-breaking changes

    • feat(event-handler): new resolvers to fix current_event typing (#978) by @michaelbrewer
    • feat(logger): log_event support event data classes (e.g. S3Event) (#984) by @michaelbrewer
    • feat(mypy): complete mypy support for the entire codebase (#943) by @mploski

    πŸ“œ Documentation updates

    • docs: fix syntax errors and line highlights (#1004) by @michaelbrewer
    • docs(parameters): add testing your code section (#1017) by @cakepietoast
    • docs(theme): upgrade mkdocs-material to 8.x (#1002) by @heitorlessa
    • docs(event-handler): improve testing section for graphql (#996) by @cakepietoast
    • docs(tutorial): fix broken internal links (#1000) by @heitorlessa
    • feat(event-handler): new resolvers to fix current_event typing (#978) by @michaelbrewer
    • fix(docs): indentation in tutorial snippets (#988) by @am1ru1
    • fix(apigateway): remove indentation in debug_mode (#987) by @heitorlessa

    πŸ› Bug and hot fixes

    • fix(batch): delete >10 messages in legacy sqs processor (#818) by @whardier
    • fix(logger): exclude source_logger in copy_config_to_registered_loggers (#1001) by @houbie
    • feat(event-handler): new resolvers to fix current_event typing (#978) by @michaelbrewer
    • fix(logger): test generates logfile (#971) by @michaelbrewer
    • fix(docs): indentation in tutorial snippets (#988) by @am1ru1
    • fix(apigateway): remove indentation in debug_mode (#987) by @heitorlessa

    πŸ”§ Maintenance

    • chore(metrics): fix tests when warnings are disabled (#994) by @michaelbrewer
    • docs(theme): upgrade mkdocs-material to 8.x (#1002) by @heitorlessa
    • feat(mypy): complete mypy support for the entire codebase (#943) by @mploski
    • chore(deps-dev): bump flake8-bugbear from 21.11.29 to 22.1.11 (#955) by @dependabot

    This release was made possible by the following contributors:

    @am1ru1, @cakepietoast, @dependabot, @dependabot[bot], @heitorlessa, @houbie, @mergify[bot], @michaelbrewer, @mploski and @whardier

    Source code(tar.gz)
    Source code(zip)
  • v1.24.2(Jan 21, 2022)

    Summary

    Quick patch release to fix an issue with API Gateway Authorizer in Event Source Data Classes not supporting resource names using _. It's an old bug in the original blueprint this code was based on. However, it prevents anyone from having a _ in the resource name get_something which is significant enough to warrant a patch release for that alone.

    Thanks to Chris Elkin for raising and @michaelbrewer for the quick fix (as always).

    Changes

    πŸ› Bug and hot fixes

    • fix(data-classes): underscore support in api gateway authorizer resource name (#969) by @michaelbrewer

    This release was made possible by the following contributors:

    @heitorlessa, @mergify[bot] and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.24.1(Jan 20, 2022)

    Summary

    This is an emergency release to fix a critical Batch bug spotted by @kimberlyamandalu, where multiple failed records weren't reported due to dictionary key uniqueness - If you are using BatchProcessor, please update to 1.24.1 as soon as possible.

    New official tutorial

    Thanks to the gigantic effort by @mploski and @am29d on reviewing it, we now finally have an official tutorial covering core Powertools features. The tutorial demonstrates how new customers can add one feature at a time. It also opens the door to create other tutorials covering other complex features in more depth.

    Please do let us know your thoughts and what other tutorials we should focus on next ;-)

    image

    New tiny function to copy Logger config to external loggers

    A common question we receive is: How can I enable powertools logging for imported libraries?

    Thanks to @mploski, we now have a tiny standalone function you can use to copy your current Logger config to any or a select list of loggers you want.

    Why would people want that? It's a fair question. Reason is sometimes you want external libraries to have the exact same structured logging that your application has. This function allows you to specify which explicit loggers you wanna copy config to (or all), and whether you want to set a different log level.

    import logging
    
    from aws_lambda_powertools import Logger
    from aws_lambda_powertools.logging import utils
    
    logger = Logger()
    
    external_logger = logging.logger()
    
    utils.copy_config_to_registered_loggers(source_logger=logger)
    external_logger.info("test message")
    

    Big thanks to new contributors @thehananbhat @j2clerck on helping us make documentation better (critical to us!).

    Changes

    🌟New features and non-breaking changes

    • feat(logger): clone powertools logger config to any Python logger (#927) by @mploski

    🌟 Minor Changes

    • fix(data-classes): docstring typos and clean up (#937) by @michaelbrewer

    πŸ“œ Documentation updates

    • docs(tutorial): fix path to images (#963) by @mploski
    • docs(batch): snippet typo in custom batch processor (#961) by @thehananbhat
    • docs(batch): snippet typo on batch processed messages iteration (#951) by @j2clerck
    • docs(nav): make REST and GraphQL event handlers more explicit (#959) by @heitorlessa
    • docs(logger): fix code block syntax in FAQ (#952) by @mozz100
    • docs(tutorial): add new tutorial covering core features (#769) by @mploski
    • docs(homepage): link to typescript version (#950) by @michaelbrewer
    • fix(parameters): appconfig internal _get docstrings (#934) by @ran-isenberg
    • docs(batch): fix typo in context manager keyword (#938) by @heitorlessa
    • feat(logger): clone powertools logger config to any Python logger (#927) by @mploski

    πŸ› Bug and hot fixes

    • fix(batch): report multiple failures (#967) by @heitorlessa
    • fix(parameters): appconfig internal _get docstrings (#934) by @ran-isenberg

    πŸ”§ Maintenance

    • docs(tutorial): fix path to images (#963) by @mploski
    • chore(deps): bump pydantic from 1.8.2 to 1.9.0 (#933) by @dependabot
    • chore(deps-dev): bump mypy from 0.930 to 0.931 (#941) by @dependabot

    This release was made possible by the following contributors:

    @dependabot, @dependabot[bot], @heitorlessa, @j2clerck, @michaelbrewer, @mozz100, @mploski, @ran-isenberg, @thehananbhat and Michal Ploski

    Source code(tar.gz)
    Source code(zip)
  • v1.24.0(Dec 31, 2021)

    Summary

    For the last release of the year (happy 2022!), we bring a major enhancements to Idempotency and Feature Flags.

    We also addresses important papercuts like caching parsed JSON data in Event Sources, support for datetime format codes in Logger and the ability to ignore certain endpoints from being traced.

    Did I say that 90% of this release was contributed by the community? thank you everyone!!!

    HUGE shoutout to @DanyC97 on helping us make all of our documentation banners (warning, tip) consistent.

    Big thanks to new contributors too (you rock!)

    • @trey-rosius for adding a medium-size GraphQL API example using a myriad of Lambda Powertools features
    • @nayaverdier for a future proof change on how we internally convert string to bool (distutils being deprecated in Python 3.12)

    idempotent_function now supports dataclasses & Pydantic models

    Docs with samples

    When using idempotent_function to make any Python synchronous function idempotent, you might have data available as Dataclasses or Pydantic models, not just dictionaries.

    This release gives you more flexibility on what data can be used as your idempotency token - it could be an entire Dataclass, Pydantic model, or fields within these models.

    image

    Going beyond boolean feature flags

    Docs with sample

    You can now use the new boolean_feature: false parameter in your schema to signal Feature Flags that you will return any JSON valid value.

    Example scenario: you might have a list of features to unlock for premium customers, or a set of beta features for select customers

    {
        "premium_features": {
            "boolean_type": false,
            "default": [],
            "rules": {
                "customer tier equals premium": {
                    "when_match": ["no_ads", "no_limits", "chat"],
                    "conditions": [
                        {
                            "action": "EQUALS",
                            "key": "tier",
                            "value": "premium"
                        }
                    ]
                }
            }
        }
    }
    

    Translating to the following API:

    from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore
    
    app_config = AppConfigStore(
        environment="dev",
        application="product-catalogue",
        name="features"
    )
    
    feature_flags = FeatureFlags(store=app_config)
    
    def lambda_handler(event, context):
        # Get customer's tier from incoming request
        ctx = { "tier": event.get("tier", "standard") }
    
        # Evaluate `has_premium_features` base don customer's tier
        premium_features: list[str] = feature_flags.evaluate(name="premium_features",
                                                            context=ctx, default=False)
        for feature in premium_features:
            # enable premium features
            ...
    
    

    Ignoring HTTP endpoints from tracer

    AWS X-Ray has a limit of 64K tracing data. This could be a problem if you're making hundreds of HTTP requests to the same endpoint.

    Alternatively, there are sensitive endpoints you might want them to not be included in your tracing data.

    You can now use ignore_endpoint for this purpose - globs (*) are allowed!

    from aws_lambda_powertools import Tracer
    
    tracer = Tracer()
    # ignore all calls to `ec2.amazon.com`
    tracer.ignore_endpoint(hostname="ec2.amazon.com")
    # ignore calls to `*.sensitive.com/password` and  `*.sensitive.com/credit-card`
    tracer.ignore_endpoint(hostname="*.sensitive.com", urls=["/password", "/credit-card"])
    
    
    def ec2_api_calls():
        return "suppress_api_responses"
    
    @tracer.capture_lambda_handler
    def handler(event, context):
        for x in long_list:
            ec2_api_calls()
    

    Changes

    🌟New features and non-breaking changes

    • feat(logger): support use_datetime_directive for timestamps (#920) by @huonw
    • feat(feature_flags): support beyond boolean values (JSON values) (#804) by @ran-isenberg
    • feat(idempotency): support dataclasses & pydantic models payloads (#908) by @michaelbrewer
    • feat(tracer): ignore tracing for certain hostname(s) or url(s) (#910) by @michaelbrewer
    • feat(event-sources): cache parsed json in data class (#909) by @michaelbrewer

    πŸ“œ Documentation updates

    • docs(tracer): new ignore_endpoint feature (#931) by @heitorlessa
    • feat(logger): support use_datetime_directive for timestamps (#920) by @huonw
    • feat(feature_flags): support beyond boolean values (JSON values) (#804) by @ran-isenberg
    • docs(general): consistency around admonitions and snippets (#919) by @DanyC97
    • docs(homepage): new GraphQL sample API in examples section (#930) by @trey-rosius
    • feat(idempotency): support dataclasses & pydantic models payloads (#908) by @michaelbrewer

    πŸ› Bug and hot fixes

    • fix(event-sources): handle dynamodb null type as none, not bool (#929) by @michaelbrewer
    • fix(apigateway): support @app.not_found() syntax & housekeeping (#926) by @michaelbrewer

    πŸ”§ Maintenance

    • chore(deps-dev): bump mypy from 0.920 to 0.930 (#925) by @dependabot
    • fix(apigateway): support @app.not_found() syntax & housekeeping (#926) by @michaelbrewer
    • fix(internals): future distutils deprecation (#921) by @nayaverdier

    This release was made possible by the following contributors:

    @DanyC97, @dependabot, @dependabot[bot], @heitorlessa, @huonw, @michaelbrewer, @nayaverdier, @ran-isenberg and @trey-rosius

    New Contributors

    • @nayaverdier made their first contribution in https://github.com/awslabs/aws-lambda-powertools-python/pull/921
    • @trey-rosius made their first contribution in https://github.com/awslabs/aws-lambda-powertools-python/pull/930

    Full Changelog: https://github.com/awslabs/aws-lambda-powertools-python/compare/v1.23.0...v1.23.1

    Source code(tar.gz)
    Source code(zip)
  • v1.23.0(Dec 20, 2021)

    Summary

    This Christmas release was heavily focused on papercuts and two must needed improvements - Batch now supports SQS, DynamoDB, and Kinesis using the new native partial response (decreased cost), and Event Handler API Gateway now supports exception handling and overriding HTTP 404 errors.

    For the next release, we're going to further invest in Mypy compliance, support native serialization of Python Dataclasses and Pydantic Models in Idempotency utility, and more!

    Join us on Slack #lambda-powertools to help us make the next release even better!

    New Batch Processor

    image

    Docs: Infra required, SQS, Kinesis, and DynamoDB

    A few weeks ago Lambda added native support for partial response. Since GA, Batch utility provided that capability to SQS by handling partial failures and deleting successful messages on your behalf.

    With the new capability, we've added a new BatchProcessor that can process both SQS, Kinesis Data Streams, and DynamoDB Streams.

    The new capability requires an infrastructure change in the Lambda Event Source Mapping hence why a new Batch Processor - We've made sure to add complete SAM examples for SQS, Kinesis, and DynamoDB Streams on everything you need to get going including minimal IAM permissions.

    We also took this opportunity to a) provide tight integration with Event Source Data Classes for self-documented batch records, b) Parser (Pydantic) integration so you can bring your own models for data schema and validation, and c) mypy is fully compliant regardles of the event type or Pydantic model you bring.

    Did I also say we refreshed the entire documentation with sample events, responses, how to unit test, and a Migration Guide?

    Migration guide

    If you were using SQSBatchProcessor before as a decorator or as a context manager, we've written a guide to help you transition to the new more performant and cost efficient way: https://awslabs.github.io/aws-lambda-powertools-python/develop/utilities/batch/#migration-guide

    A warm start with tracing, logging, data transformation, and exception handling can run as fast as 8ms end-to-end now.

    Event Handler API Gateway improvements

    Overriding 404 errors

    Event Handler API Gateway snippet demonstrating 404 override

    You can now override how we handle unmatched routes (404) by using a decorator not_found. The experience is similar to how you'd define routes, except your function will receive an exception - NotFoundError in this case.

    You can customize the response by simply returning Response like you would in a more advanced route -- This should give you more flexibility to generate metrics, log errors, or anything you might want for unmatched routes.

    Exception handling

    Event Handler API Gateway snippet demonstrating exception handling

    Besides 404 errors, you might want to handle custom validation errors, or any exception you might raise as part of your route handling. You can now use exception_handler decorator and pass any valid Python Exception.

    Tracer

    X-Ray Trace Annotations screenshot for the handler subsegment

    Tracer now automatically adds a Service annotation if either service parameter or POWERTOOLS_SERVICE_NAME is set - This makes it easier to slice and dice all of your traces by the exact service you have deployed.

    Moreover, we now include ColdStart=false for warm start invocations so you can more easily sample and analyze performance differences between cold and warm starts.

    CloudFormation Custom Resources

    As our Java Powertools colleagues added a new utility for easily writing CloudFormation Custom Resources, we added a link in our docs pointing to the official Python library for crafting custom resources.... just in case it's not widely known: https://github.com/aws-cloudformation/custom-resource-helper

    Changes

    🌟New features and non-breaking changes

    • feat(logger): allow handler with custom kwargs signature (#913) by @heitorlessa
    • feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis (#886) by @heitorlessa
    • feat(apigateway): add exception_handler support (#898) by @michaelbrewer
    • feat(tracer): add service annotation when service is set (#861) by @heitorlessa
    • feat(apigateway): access parent api resolver from router (#842) by @cakepietoast

    🌟 Minor Changes

    • fix(metrics): explicit type to single_metric ctx manager (#865) by @whardier

    πŸ“œ Documentation updates

    • docs(apigateway): add new not_found feature (#915) by @heitorlessa
    • docs(nav): reference cloudformation custom resource helper (#914) by @heitorlessa
    • feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis (#886) by @heitorlessa
    • fix(parser): mypy support for payload type override as models (#883) by @heitorlessa
    • docs(apigateway): fix sample layout provided (#864) by @michaelbrewer
    • feat(tracer): add service annotation when service is set (#861) by @heitorlessa
    • fix(apigateway): allow list of HTTP methods in route method (#838) by @cakepietoast

    πŸ› Bug and hot fixes

    • fix(parser): overload parse when using envelope (#885) by @heitorlessa
    • fix(parser): kinesis sequence number is str, not int (#907) by @heitorlessa
    • fix(event-sources): Pass authorizer data to APIGatewayEventAuthorizer (#897) by @michaelbrewer
    • fix(parser): mypy support for payload type override as models (#883) by @heitorlessa
    • fix(parameters): appconfig transform and return types (#877) by @ran-isenberg
    • fix(event-sources): handle Cognito claimsOverrideDetails set to null (#878) by @michaelbrewer
    • fix(idempotency): include decorated fn name in hash (#869) by @michaelbrewer
    • fix(tracer): add warm start annotation (ColdStart=False) (#851) by @heitorlessa
    • fix(apigateway): allow list of HTTP methods in route method (#838) by @cakepietoast

    πŸ”§ Maintenance

    • chore: minor housekeeping before release (#912) by @michaelbrewer
    • chore(deps-dev): bump mypy from 0.910 to 0.920 (#903) by @dependabot
    • chore(deps): bump fastjsonschema from 2.15.1 to 2.15.2 (#891) by @dependabot
    • chore(deps-dev): bump black from 21.11b1 to 21.12b0 (#872) by @dependabot
    • chore(deps): bump aws-xray-sdk from 2.8.0 to 2.9.0 (#876) by @dependabot
    • chore(deps): support arm64 when developing locally (#862) by @michaelbrewer
    • chore(deps-dev): bump flake8 from 3.9.2 to 4.0.1 (#789) by @dependabot
    • chore(deps): bump actions/setup-python from 2.3.0 to 2.3.1 (#852) by @dependabot
    • chore(deps-dev): bump black from 21.10b0 to 21.11b1 (#839) by @dependabot
    • chore(deps): bump actions/setup-python from 2.2.2 to 2.3.0 (#831) by @dependabot

    This release was made possible by the following contributors:

    @cakepietoast, @dependabot, @dependabot[bot], @heitorlessa, @michaelbrewer, @ran-isenberg and @whardier

    Full Changelog: https://github.com/awslabs/aws-lambda-powertools-python/compare/v1.22.0...v1.22.1

    Source code(tar.gz)
    Source code(zip)
  • v1.22.0(Nov 17, 2021)

    Summary

    This release adds two major changes: 1/ New Router feature in Event Handler utility including GraphQL Resolvers composition in AppSync, and 2/ Idiomatic tenet has been updated to Progressive.

    Additionally, we now support ActiveMQ and RabbitMQ in the Event Source Data Classes, and primary composite key for Idempotency when using DynamoDB Storage. There's been lots of improvements to documentation around Lambda Layers install, and a bug fix for Parser (Pydantic) to address API Gateway v1/v2 supporting a null body.

    This release note will primarily cover the new Router feature in Event Handler given how significant this is. Also, we created a new section named Considerations in the docs to share an opinionated set of trade-offs when going with a monolithic vs micro function approach, when using API Gateway, ALB, or AppSync.

    Router feature in Event Handler

    You can now use separate files to compose routes and GraphQL resolvers. Before this feature, you'd need all your routes or GraphQL resolvers in the same file where your Lambda handler is.

    API Gateway and ALB

    This is how it would look like before this feature in either API Gateway, ALB, and AppSync:

    app.py

    import itertools
    from typing import Dict
    
    from aws_lambda_powertools import Logger
    from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
    
    logger = Logger(child=True)
    app = ApiGatewayResolver()
    
    USERS = {"user1": "details_here", "user2": "details_here", "user3": "details_here"}
    
    
    @app.get("/users")
    def get_users() -> Dict:
        # /users?limit=1
        pagination_limit = app.current_event.get_query_string_value(name="limit", default_value=10)
    
        logger.info(f"Fetching the first {pagination_limit} users...")
        ret = dict(itertools.islice(USERS.items(), int(pagination_limit)))
        return {"items": [ret]}
    
    @app.get("/users/<username>")
    def get_user(username: str) -> Dict:
        logger.info(f"Fetching username {username}")
        return {"details": USERS.get(username, {})}
    

    With Router, you can now split the /users routes in a separate file and change ApiGatewayResolver with Router, for example:

    users.py

    import itertools
    from typing import Dict
    
    from aws_lambda_powertools import Logger
    from aws_lambda_powertools.event_handler.api_gateway import Router
    
    logger = Logger(child=True)
    router = Router()
    USERS = {"user1": "details_here", "user2": "details_here", "user3": "details_here"}
    
    
    @router.get("/users")
    def get_users() -> Dict:
        # /users?limit=1
        pagination_limit = router.current_event.get_query_string_value(name="limit", default_value=10)
    
        logger.info(f"Fetching the first {pagination_limit} users...")
        ret = dict(itertools.islice(USERS.items(), int(pagination_limit)))
        return {"items": [ret]}
    
    @router.get("/users/<username>")
    def get_user(username: str) -> Dict:
        logger.info(f"Fetching username {username}")
        return {"details": USERS.get(username, {})}
    

    Note that the user experience is exactly the same on accessing request details and defining routes, except we use Router instead of ApiGatewayResolver.

    Next, within your Lambda entry point, you have to use the new include_router method to inject routes from /users at runtime:

    app.py

    from typing import Dict
    
    from aws_lambda_powertools import Logger
    from aws_lambda_powertools.event_handler import ApiGatewayResolver
    from aws_lambda_powertools.utilities.typing import LambdaContext
    
    import users
    
    logger = Logger()
    app = ApiGatewayResolver()
    app.include_router(users.router)
    
    @logger.inject_lambda_context
    def lambda_handler(event: Dict, context: LambdaContext):
        return app.resolve(event, context)
    

    GraphQL Resolvers

    Similarly to API Gateway and ALB, you can now use Router to split GraphQL resolvers allowing for further composition:

    resolvers/location.py

    from typing import Any, Dict, List
    
    from aws_lambda_powertools import Logger
    from aws_lambda_powertools.event_handler.appsync import Router
    
    logger = Logger(child=True)
    router = Router()
    
    
    @router.resolver(type_name="Query", field_name="listLocations")
    def list_locations(merchant_id: str) -> List[Dict[str, Any]]:
        return [{"name": "Location name", "merchant_id": merchant_id}]
    
    
    @router.resolver(type_name="Location", field_name="status")
    def resolve_status(merchant_id: str) -> str:
        logger.debug(f"Resolve status for merchant_id: {merchant_id}")
        return "FOO"
    

    app.py

    from typing import Dict
    
    from aws_lambda_powertools import Logger, Tracer
    from aws_lambda_powertools.event_handler import AppSyncResolver
    from aws_lambda_powertools.logging.correlation_paths import APPSYNC_RESOLVER
    from aws_lambda_powertools.utilities.typing import LambdaContext
    
    from resolvers import location
    
    tracer = Tracer()
    logger = Logger()
    app = AppSyncResolver()
    app.include_router(location.router)
    
    
    @tracer.capture_lambda_handler
    @logger.inject_lambda_context(correlation_id_path=APPSYNC_RESOLVER)
    def lambda_handler(event: Dict, context: LambdaContext):
        app.resolve(event, context)
    

    Tenet update

    We've updated Idiomatic tenet to Progressive to reflect the new Router feature in Event Handler, and more importantly the new wave of customers coming from SRE, Data Analysis, and Data Science background.

    • BEFORE: Idiomatic. Utilities follow programming language idioms and language-specific best practices.
    • AFTER: Progressive. Utilities are designed to be incrementally adoptable for customers at any stage of their Serverless journey. They follow language idioms and their community’s common practices.

    Changes

    🌟New features and non-breaking changes

    • feat(idempotency): support composite primary key in DynamoDBPersistenceLayer (#740) by @Tankanow
    • feat(data-classes): ActiveMQ and RabbitMQ support (#770) by @michaelbrewer
    • feat(appsync): add Router to allow large resolver composition (#776) by @michaelbrewer
    • feat(apigateway): add Router to allow large routing composition (#645) by @BVMiko

    🌟 Minor Changes

    • refactor(apigateway): Add BaseRouter and duplicate route check (#757) by @michaelbrewer

    πŸ“œ Documentation updates

    • docs(apigateway): re-add sample layout, add considerations (#826) by @heitorlessa
    • docs(appsync): add new router feature (#821) by @heitorlessa
    • docs(tenets): update Idiomatic tenet to Progressive (#823) by @heitorlessa
    • docs: use higher contrast font (#822) by @heitorlessa
    • docs(api-gateway): add support for new router feature (#767) by @michaelbrewer
    • docs(idempotency): add support for DynamoDB composite keys (#808) by @cakepietoast
    • docs: updated Lambda Layers definition & limitations. (#775) by @eldritchideen
    • feat(data-classes): ActiveMQ and RabbitMQ support (#770) by @michaelbrewer
    • docs: fix indentation of SAM snippets in install section (#778) by @jonemo
    • docs(middleware): fix sample code (#772) by @arthurf1969
    • docs(parser): Removed unused import, added typing imports, fixed typo in example. (#774) by @eldritchideen
    • docs(install): improve public lambda layer wording, clipboard buttons (#762) by @heitorlessa
    • docs(install): add amplify-cli instructions for public layer (#754) by @AlessandroVol23

    πŸ› Bug and hot fixes

    • fix(parser): body/QS can be null or omitted in apigw v1/v2 (#820) by @heitorlessa

    πŸ”§ Maintenance

    • chore(deps): bump boto3 from 1.20.3 to 1.20.5 (#817) by @dependabot
    • chore(deps): bump boto3 from 1.19.6 to 1.20.3 (#809) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.3.5 to 7.3.6 (#791) by @dependabot
    • fix: change supported python version from 3.6.1 to 3.6.2, bump black (#807) by @cakepietoast
    • chore(deps-dev): bump mkdocs-material from 7.3.3 to 7.3.5 (#781) by @dependabot
    • chore(deps-dev): bump flake8-isort from 4.0.0 to 4.1.1 (#785) by @dependabot
    • chore(deps): bump urllib3 from 1.26.4 to 1.26.5 (#787) by @dependabot
    • chore(deps-dev): bump flake8-eradicate from 1.1.0 to 1.2.0 (#784) by @dependabot
    • chore(deps): bump boto3 from 1.18.61 to 1.19.6 (#783) by @dependabot
    • chore(deps-dev): bump pytest-asyncio from 0.15.1 to 0.16.0 (#782) by @dependabot
    • chore(deps-dev): bump coverage from 6.0.1 to 6.0.2 (#764) by @dependabot
    • chore(deps): bump boto3 from 1.18.59 to 1.18.61 (#766) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.3.2 to 7.3.3 (#758) by @dependabot
    • chore(deps-dev): bump flake8-comprehensions from 3.6.1 to 3.7.0 (#759) by @dependabot
    • chore(deps): bump boto3 from 1.18.58 to 1.18.59 (#760) by @dependabot
    • chore(deps-dev): bump coverage from 6.0 to 6.0.1 (#751) by @dependabot
    • chore(deps): bump boto3 from 1.18.56 to 1.18.58 (#755) by @dependabot

    This release was made possible by the following contributors:

    @AlessandroVol23, @BVMiko, @Tankanow, @arthurf1969, @cakepietoast, @dependabot, @dependabot[bot], @eldritchideen, @heitorlessa, @jonemo and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.21.1(Oct 7, 2021)

    Summary

    Patch release to address regression in Metrics with mypy not recognizing a Callable when using log_metrics().

    New Public Lambda Layers ARNs

    Oh! It's finally here!!!

    This release adds our first batch of public Lambda Layers for every AWS region supported by AWS Lambda - huge thanks to @am29d.

    This means you no longer need to deploy a SAR App in order to use Lambda Powertools as a Lambda Layer.

    image

    That being said, we will keep SAR App in order to give you the flexibility to choose which semantic version you want to use as a Lambda Layer, until it is officially supported by Lambda Layers.

    Changes

    πŸ“œ Documentation updates

    • docs: add new public layer ARNs (#746) by @am29d

    πŸ› Bug and hot fixes

    • revert(metrics): typing regression on log_metrics callable (#744) by @heitorlessa

    πŸ”§ Maintenance

    • chore(deps): bump boto3 from 1.18.54 to 1.18.56 (#742) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.3.1 to 7.3.2 (#741) by @dependabot
    • chore: ignore constants in test cov (#745) by @heitorlessa

    This release was made possible by the following contributors:

    @am29d, @dependabot, @dependabot[bot] and @heitorlessa

    Source code(tar.gz)
    Source code(zip)
  • v1.21.0(Oct 5, 2021)

    Summary

    After some vacation period, we're back with a new minor release with major features:

    • Bring your own boto3 sessions for cross-account operations & snapshot testing
    • New features on Feature Flags
    • Idempotency unit testing made easier
    • JSON Schema Validation utility contains new data elements to more easily construct a validation error
    • New utility: we're now exposing our internal custom JMESPath Functions so you can easily decode and deserialize JSON objects found in various formats within Lambda Event Sources.

    New Contributors

    I'd like to personally thank our new contributors to the project :)

    • @Tankanow made their first contribution in #697
    • @DanyC97 made their first contribution in #716
    • @gwlester made their first contribution in #710

    Detailed changes

    Boto3 sessions

    You can now pass in your own boto3 session when using Parameters, Batch, and Idempotency.

    This is helpful in two typical scenarios: 1/ You want to run an operation in another account like fetching secrets/parameters somewhere else, 2/ Use snapshot testing tools like Placebo that will replay session data that happened earlier when doing unit testing.

    from aws_lambda_powertools.utilities import parameters
    import boto3
    
    boto3_session = boto3.session.Session()
    ssm_provider = parameters.SSMProvider(boto3_session=boto3_session)
    
    def handler(event, context):
        # Retrieve a single parameter
        value = ssm_provider.get("/my/parameter")
        ...
    

    Feature flags

    There's been three main improvements in Feature flags utility as part of this release: New rule conditions, Bring your own Logger for debugging, and Getting a copy of fetched configuration from the store

    New rule conditions

    You can now use the following new rule conditions to evaluate your feature flags for inequality, comparison, and more explicit contains logic, where a is the key and b is the value passed as a context input for evaluation:

    Action | Equivalent expression ------------------------------------------------- | --------------------------------------------------------------------------------- KEY_GREATER_THAN_VALUE | lambda a, b: a > b KEY_GREATER_THAN_OR_EQUAL_VALUE | lambda a, b: a >= b KEY_LESS_THAN_VALUE | lambda a, b: a < b KEY_LESS_THAN_OR_EQUAL_VALUE | lambda a, b: a <= b KEY_IN_VALUE | lambda a, b: a in b KEY_NOT_IN_VALUE | lambda a, b: a not in b VALUE_IN_KEY | lambda a, b: b in a VALUE_NOT_IN_KEY | lambda a, b: b not in a

    Example

    Feature flag schema

    {
        "premium_features": {
            "default": false,
            "rules": {
                "customer tier equals premium": {
                    "when_match": true,
                    "conditions": [
                        {
                            "action": "VALUE_IN_KEY",
                            "key": "groups",
                            "value": "PAID_PREMIUM",
                        }
                    ]
                }
            }
        },
        "ten_percent_off_campaign": {
            "default": false
        }
    }
    

    App

    from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore
    
    app_config = AppConfigStore(
        environment="dev",
        application="product-catalogue",
        name="features"
    )
    
    feature_flags = FeatureFlags(store=app_config)
    
    def lambda_handler(event, context):
        # groups: ["FREE_TIER", "PAID_BASIC", "PAID_PREMIUM"]
        ctx={"tenant_id": "6", "username": "a", "groups": event.get("groups", [])}
    
        # Evaluate whether customer's tier has access to premium features
        # based on `has_premium_features` rules
        has_premium_features: bool = feature_flags.evaluate(name="premium_features",
                                                            context=ctx, default=False)
        if has_premium_features:
            # enable premium features
            ...
    

    Accessing raw configuration fetched

    Previously, if you were using a single application configuration and a feature schema in a single AppConfig key, we would only use the feature flags schema and discard the rest.

    You can now access the raw configuration with a new property get_raw_configuration within AppConfig Store:

    from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore
    
    app_config = AppConfigStore(
        environment="dev",
        application="product-catalogue",
        name="configuration",
        envelope = "feature_flags"
    )
    
    feature_flags = FeatureFlags(store=app_config)
    
    config = app_config.get_raw_configuration
    

    Unit testing idempotency

    We have improved how you can unit test your code when using @idempotent and @idempotent_function decorators.

    You can now disable all interactions with the idempotence store using POWERTOOLS_IDEMPOTENCY_DISABLED environment variable, and monkeypatch the DynamoDB resource client Idempotency utility uses if you wish to either use DynamoDB Local or mock all I/O operations.

    import boto3
    
    import app
    
    def test_idempotent_lambda():
        # Create our own Table resource using the endpoint for our DynamoDB Local instance
        resource = boto3.resource("dynamodb", endpoint_url='http://localhost:8000')
        table = resource.Table(app.persistence_layer.table_name)
        app.persistence_layer.table = table
    
        result = app.handler({'testkey': 'testvalue'}, {})
        assert result['payment_id'] == 12345
    

    New data elements for JSON Schema validation errors

    When validating input/output with the Validator, you can now access new properties in SchemaValidationError to more easily construct your custom errors based on what went wrong.

    Property | Type | Description ------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- message | str | Powertools formatted error message validation_message | str, optional | Containing human-readable information what is wrong, e.g. data.property[index] must be smaller than or equal to 42 name | str, optional | name of a path in the data structure, e.g. data.property[index] path | List, optional | path as an array in the data structure, e.g. ['data', 'property', 'index'] value | Any, optional | The invalid value, e.g. {"message": "hello world"} definition | Any, optional | JSON Schema definition rule | str, optional | rule which the data is breaking (e.g. maximum, required) rule_definition | Any, optional | The specific rule definition (e.g. 42, ['message', 'username'])

    Sample

    image

    from aws_lambda_powertools.utilities.validation import validate
    from aws_lambda_powertools.utilities.validation.exceptions import SchemaValidationError
    
    import schemas
    
    def handler(event, context):
        try:
            validate(event=event, schema=schemas.INPUT)
        except SchemaValidationError as e:
            message = "data must contain ['message', 'username'] properties"
    
            assert str(e.value) == e.value.message
            assert e.value.validation_message == message
            assert e.value.name == "data"
            assert e.value.path is not None
            assert e.value.value == data
            assert e.value.definition == schema
            assert e.value.rule == "required"
            assert e.value.rule_definition == schema.get("required")
    
            raise
    
        return event
    

    New JMESPath Powertools functions

    Last but not least, as part of a documentation revamp in Idempotency by @walmsles, we're now exposing an internal feature used by many Lambda Powertools utilities which is the ability to extract and decode JSON objects.

    You can now use JMESPath (JSON Query language) Lambda Powertools functions to easily decode and deserialize JSON often found as compressed (Kinesis, CloudWatch Logs, etc), as strings (SNS, SQS, EventBridge, API Gateway, etc), or as base64 (Kinesis).

    We're exposing three custom JMESPath functions you can use such as powertools_json, powertools_base64, powertools_base64_gzip, and a new standalone function that will use JMESPath to search and extract the data you want called extract_data_from_envelope.

    Sample

    from aws_lambda_powertools.utilities.jmespath_utils import extract_data_from_envelope
    from aws_lambda_powertools.utilities.typing import LambdaContext
    
    
    def handler(event: dict, context: LambdaContext):
        payload = extract_data_from_envelope(data=event, envelope="powertools_json(body)")
        customer = payload.get("customerId")  # now deserialized
        ...
    

    SNS sample using built-in envelope

    from aws_lambda_powertools.utilities.jmespath_utils import extract_data_from_envelope, envelopes
    from aws_lambda_powertools.utilities.typing import LambdaContext
    
    
    def handler(event: dict, context: LambdaContext):
        payload = extract_data_from_envelope(data=event, envelope=envelopes.SNS)
        customer = payload.get("customerId")  # now deserialized
        ...
    

    Sample event

    {
        "Records": [
            {
                "messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
                "receiptHandle": "MessageReceiptHandle",
                "body": "{\"customerId\":\"dd4649e6-2484-4993-acb8-0f9123103394\",\"booking\":{\"id\":\"5b2c4803-330b-42b7-811a-c68689425de1\",\"reference\":\"ySz7oA\",\"outboundFlightId\":\"20c0d2f2-56a3-4068-bf20-ff7703db552d\"},\"payment\":{\"receipt\":\"https:\/\/pay.stripe.com\/receipts\/acct_1Dvn7pF4aIiftV70\/ch_3JTC14F4aIiftV700iFq2CHB\/rcpt_K7QsrFln9FgFnzUuBIiNdkkRYGxUL0X\",\"amount\":100}}",
                "attributes": {
                    "ApproximateReceiveCount": "1",
                    "SentTimestamp": "1523232000000",
                    "SenderId": "123456789012",
                    "ApproximateFirstReceiveTimestamp": "1523232000001"
                },
                "messageAttributes": {},
                "md5OfBody": "7b270e59b47ff90a553787216d55d91d",
                "eventSource": "aws:sqs",
                "eventSourceARN": "arn:aws:sqs:us-east-1:123456789012:MyQueue",
                "awsRegion": "us-east-1"
            }
        ]
    }
    

    Changes

    🌟New features and non-breaking changes

    • feat: expose jmespath powertools functions (#736) by @heitorlessa
    • feat(feature-flags): Added inequality conditions (#721) by @gwlester
    • feat(feature-flags): Bring your own logger for debug (#709) by @gwlester
    • feat(feature-flags): improve "IN/NOT_IN"; new rule actions (#710) by @gwlester
    • feat(idempotency): makes customers unit testing easier (#719) by @cakepietoast
    • feat(feature-flags): get_raw_configuration property in Store (#720) by @heitorlessa
    • feat: boto3 sessions in batch, parameters & idempotency (#717) by @cakepietoast
    • feat(validator): include missing data elements from a validation error (#686) by @michaelbrewer

    🌟 Minor Changes

    • fix(idempotency): use ExpressionAttributeNames in DynamoDBPersistenceLayer _put_record (#697) by @Tankanow

    πŸ“œ Documentation updates

    • chore(deps-dev): bump pytest-cov from 2.12.1 to 3.0.0 (#730) by @dependabot
    • chore(deps-dev): bump coverage from 5.5 to 6.0 (#732) by @dependabot
    • docs(parser): fix incorrect import in root_validator example (#735) by @heitorlessa
    • docs: Terraform reference for SAR Lambda Layer (#716) by @DanyC97
    • docs(idempotency): fix misleading idempotent examples (#661) by @walmsles
    • docs(event-handler): document catch-all routes (#705) by @heitorlessa
    • refactor(data-classes): clean up internal logic for APIGatewayAuthorizerResponse (#643) by @michaelbrewer
    • fix(data-classes): use correct asdict funciton (#666) by @michaelbrewer

    πŸ› Bug and hot fixes

    • fix(feature-flags): rules should evaluate with an AND op (#724) by @risenberg-cyberark
    • fix(logger): push extra keys to the end (#722) by @heitorlessa
    • fix(mypy): a few return types, type signatures, and untyped areas (#718) by @heitorlessa

    πŸ”§ Maintenance

    • chore(deps-dev): bump pytest-cov from 2.12.1 to 3.0.0 (#730) by @dependabot
    • chore(deps-dev): bump coverage from 5.5 to 6.0 (#732) by @dependabot
    • chore(deps): bump boto3 from 1.18.51 to 1.18.54 (#733) by @dependabot
    • chore(deps-dev): bump flake8-bugbear from 21.9.1 to 21.9.2 (#712) by @dependabot
    • chore(deps): bump boto3 from 1.18.49 to 1.18.51 (#713) by @dependabot
    • chore(deps): bump boto3 from 1.18.41 to 1.18.49 (#703) by @dependabot
    • chore(deps): bump codecov/codecov-action from 2.0.2 to 2.1.0 (#675) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.2.8 to 7.3.0 (#695) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.2.6 to 7.2.8 (#682) by @dependabot
    • chore(deps-dev): bump flake8-bugbear from 21.4.3 to 21.9.1 (#676) by @dependabot
    • chore(deps): bump boto3 from 1.18.38 to 1.18.41 (#677) by @dependabot
    • chore(deps-dev): bump radon from 4.5.2 to 5.1.0 (#673) by @dependabot
    • chore(deps): bump boto3 from 1.18.32 to 1.18.38 (#671) by @dependabot
    • refactor(data-classes): clean up internal logic for APIGatewayAuthorizerResponse (#643) by @michaelbrewer
    • chore(deps-dev): bump xenon from 0.7.3 to 0.8.0 (#669) by @dependabot

    This release was made possible by the following contributors:

    @DanyC97, @Tankanow, @cakepietoast, @dependabot, @dependabot[bot], @gwlester, @heitorlessa, @michaelbrewer, @risenberg-cyberark and @walmsles

    Full Changelog: https://github.com/awslabs/aws-lambda-powertools-python/compare/v1.20.2...v1.21.0

    Source code(tar.gz)
    Source code(zip)
  • v1.20.2(Sep 2, 2021)

    Summary

    Bugfix release to address a bug with event handler which caused issues for api gateway events when using strip_prefix together with a root level handler. Thanks @BVMiko for the fix!

    Changes

    πŸ› Bug and hot fixes

    • fix: Fix issue with strip_prefixes (#647) by @BVMiko

    πŸ”§ Maintenance

    • chore(deps-dev): bump mkdocs-material from 7.2.4 to 7.2.6 (#665) by @dependabot
    • chore(deps): bump boto3 from 1.18.26 to 1.18.32 (#663) by @dependabot
    • chore(deps-dev): bump pytest from 6.2.4 to 6.2.5 (#662) by @dependabot
    • chore(license): Add THIRD-PARTY-LICENSES to pyproject.toml (#641) by @BastianZim

    This release was made possible by the following contributors:

    @BVMiko, @BastianZim, @cakepietoast

    Source code(tar.gz)
    Source code(zip)
  • v1.20.1(Aug 22, 2021)

    Summary

    Hotfix release to address a bug in idempotency hashing logic to ensure data is sorted as part of the process - Huge thanks to @walmsles!

    We also now support CodeSpaces to ease contribution, also thanks to @michaelbrewer

    Changes

    πŸ“œ Documentation updates

    • chore: markdown linter fixes (#636) by @michaelbrewer

    πŸ› Bug and hot fixes

    • fix(idempotency): sorting keys before hashing (#639) by @heitorlessa

    πŸ”§ Maintenance

    • chore: setup codespaces (#637) by @michaelbrewer
    • chore(license): add third party license (#635) by @BastianZim

    This release was made possible by the following contributors:

    @BastianZim, @heitorlessa and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.20.0(Aug 21, 2021)

    Summary

    This release highlights 1/ support for Python 3.9, 2/ support for API Gateway and AppSync Lambda Authorizers, 3/ support for API Gateway Custom Domain Mappings, 4/ support to make any Python synchronous function idempotent, and a number of documentation improvements & bugfixes.

    Lambda Authorizer support

    AppSync

    This release adds Data Class support for AppSyncAuthorizerEvent, AppSyncAuthorizerResponse, and correlation ID in Logger.

    You can use AppSyncAuthorizerEvent to easily access all self-documented properties, and AppSyncAuthorizerResponse to serialize the response in the expected format.

    You can read more in the announcement blog post for more details

    from typing import Dict
    
    from aws_lambda_powertools.logging import correlation_paths
    from aws_lambda_powertools.logging.logger import Logger
    from aws_lambda_powertools.utilities.data_classes.appsync_authorizer_event import (
        AppSyncAuthorizerEvent,
        AppSyncAuthorizerResponse,
    )
    from aws_lambda_powertools.utilities.data_classes.event_source import event_source
    
    logger = Logger()
    
    
    def get_user_by_token(token: str):
        """Look a user by token"""
    
    
    @logger.inject_lambda_context(correlation_id_path=correlation_paths.APPSYNC_AUTHORIZER)
    @event_source(data_class=AppSyncAuthorizerEvent)
    def lambda_handler(event: AppSyncAuthorizerEvent, context) -> Dict:
        user = get_user_by_token(event.authorization_token)
    
        if not user:
            # No user found, return not authorized
            return AppSyncAuthorizerResponse().to_dict()
    
        return AppSyncAuthorizerResponse(
            authorize=True,
            resolver_context={"id": user.id},
            # Only allow admins to delete events
            deny_fields=None if user.is_admin else ["Mutation.deleteEvent"],
        ).asdict()
    
    

    API Gateway

    This release adds support for both Lambda Authorizer for payload v1 - APIGatewayAuthorizerRequestEvent, APIGatewayAuthorizerResponse - and v2 formats APIGatewayAuthorizerEventV2, APIGatewayAuthorizerResponseV2.

    Similar to AppSync, you can use APIGatewayAuthorizerRequestEvent and APIGatewayAuthorizerEventV2 to easily access all self-documented properties available, and its corresponding APIGatewayAuthorizerResponse and APIGatewayAuthorizerResponseV2 to serialize the response in the expected format.

    You can read more in the announcement blog post for more details

    v2 format

    from aws_lambda_powertools.utilities.data_classes import event_source
    from aws_lambda_powertools.utilities.data_classes.api_gateway_authorizer_event import (
        APIGatewayAuthorizerEventV2,
        APIGatewayAuthorizerResponseV2,
    )
    from secrets import compare_digest
    
    
    def get_user_by_token(token):
        if compare_digest(token, "Foo"):
            return {"name": "Foo"}
        return None
    
    
    @event_source(data_class=APIGatewayAuthorizerEventV2)
    def handler(event: APIGatewayAuthorizerEventV2, context):
        user = get_user_by_token(event.get_header_value("x-token"))
    
        if user is None:
            # No user was found, so we return not authorized
            return APIGatewayAuthorizerResponseV2().asdict()
    
        # Found the user and setting the details in the context
        return APIGatewayAuthorizerResponseV2(authorize=True, context=user).asdict()
    

    v1 format

    from aws_lambda_powertools.utilities.data_classes import event_source
    from aws_lambda_powertools.utilities.data_classes.api_gateway_authorizer_event import (
        APIGatewayAuthorizerRequestEvent,
        APIGatewayAuthorizerResponse,
        HttpVerb,
    )
    from secrets import compare_digest
    
    
    def get_user_by_token(token):
        if compare_digest(token, "admin-foo"):
            return {"isAdmin": True, "name": "Admin"}
        elif compare_digest(token, "regular-foo"):
            return {"name": "Joe"}
        else:
            return None
    
    
    @event_source(data_class=APIGatewayAuthorizerRequestEvent)
    def handler(event: APIGatewayAuthorizerRequestEvent, context):
        user = get_user_by_token(event.get_header_value("Authorization"))
    
        # parse the `methodArn` as an `APIGatewayRouteArn`
        arn = event.parsed_arn
        # Create the response builder from parts of the `methodArn`
        policy = APIGatewayAuthorizerResponse(
            principal_id="user",
            region=arn.region,
            aws_account_id=arn.aws_account_id,
            api_id=arn.api_id,
            stage=arn.stage
        )
    
        if user is None:
            # No user was found, so we return not authorized
            policy.deny_all_routes()
            return policy.asdict()
    
        # Found the user and setting the details in the context
        policy.context = user
    
        # Conditional IAM Policy
        if user.get("isAdmin", False):
            policy.allow_all_routes()
        else:
            policy.allow_route(HttpVerb.GET, "/user-profile")
    
        return policy.asdict()
    

    Custom Domain API Mappings

    When using Custom Domain API Mappings feature, you must use the new strip_prefixes param in the ApiGatewayResolver constructor.

    Scenario: You have a custom domain api.mydomain.dev and set an API Mapping payment to forward requests to your Payments API, the path argument will be /payment/<your_actual_path>.

    This will lead to a HTTP 404 despite having your Lambda configured correctly. See the example below on how to account for this change.

    from aws_lambda_powertools import Logger, Tracer
    from aws_lambda_powertools.logging import correlation_paths
    from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
    
    tracer = Tracer()
    logger = Logger()
    app = ApiGatewayResolver(strip_prefixes=["/payment"])
    
    @app.get("/subscriptions/<subscription>")
    @tracer.capture_method
    def get_subscription(subscription):
        return {"subscription_id": subscription}
    
    @logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
    @tracer.capture_lambda_handler
    def lambda_handler(event, context):
        return app.resolve(event, context)
    

    Make any Python function idempotent

    Previously, you could only make the entire Lambda function handler idempotent. You can now make any Python function idempotent with the new idempotent_function.

    This also enables easy integration with any other utility in Powertools. Take example the Batch utility, where you wouldn't want to make the entire Lambda handler idempotent as the batch will vary, instead you'd want to make sure you can process a given message only once.

    As a trade-off to allow any Python function with an arbitrary number of parameters, you must call your function with a keyword argument, and you tell us upfront which one that might be using data_keyword_argument, so we can apply all operations like hashing the idempotency token, payload extraction, parameter validation, etc.

    import uuid
    
    from aws_lambda_powertools.utilities.batch import sqs_batch_processor
    from aws_lambda_powertools.utilities.idempotency import idempotent_function, DynamoDBPersistenceLayer, IdempotencyConfig
    
    
    dynamodb = DynamoDBPersistenceLayer(table_name="idem")
    config =  IdempotencyConfig(
        event_key_jmespath="messageId",  # see "Choosing a payload subset for idempotency" docs section
        use_local_cache=True,
    )
    
    @idempotent_function(data_keyword_argument="data", config=config, persistence_store=dynamodb)
    def dummy(arg_one, arg_two, data: dict, **kwargs):
        return {"data": data}
    
    
    @idempotent_function(data_keyword_argument="record", config=config, persistence_store=dynamodb)
    def record_handler(record):
        return {"message": record["body"]}
    
    
    @sqs_batch_processor(record_handler=record_handler)
    def lambda_handler(event, context):
        # `data` parameter must be called as a keyword argument to work
        dummy("hello", "universe", data="test")
        return {"statusCode": 200}
    

    Changes

    🌟New features and non-breaking changes

    • feat(data-classes): authorizer for http api and rest api (#620) by @michaelbrewer
    • feat(data-classes): data_as_bytes prop KinesisStreamRecordPayload (#628) by @hjurong
    • feat(general): support for Python 3.9 (#626) by @heitorlessa
    • feat(event-handler): prefixes to strip for custom mappings (#579) by @michaelbrewer
    • feat(data-classes): AppSync Lambda authorizer event (#610) by @michaelbrewer
    • feat(idempotency): support for any synchronous function (#625) by @heitorlessa

    πŸ“œ Documentation updates

    • docs(data-classes): make authorizer concise; use enum (#630) by @heitorlessa
    • feat(data-classes): authorizer for http api and rest api (#620) by @michaelbrewer
    • feat(data-classes): AppSync Lambda authorizer event (#610) by @michaelbrewer
    • chore(docs): correct markdown based on markdown lint (#603) by @michaelbrewer
    • docs(feature-flags): correct link and json examples (#605) by @michaelbrewer

    πŸ› Bug and hot fixes

    • fix(api-gateway): strip stage name from request path (#622) by @michaelbrewer

    πŸ”§ Maintenance

    • feat(idempotency): support for any synchronous function (#625) by @heitorlessa
    • chore(deps): bump boto3 from 1.18.25 to 1.18.26 (#627) by @dependabot
    • feat(general): support for Python 3.9 (#626) by @heitorlessa
    • chore(deps): bump boto3 from 1.18.24 to 1.18.25 (#623) by @dependabot
    • chore(api-docs): enable allow_reuse to fix the docs (#612) by @michaelbrewer
    • chore(deps): bump boto3 from 1.18.22 to 1.18.24 (#619) by @dependabot
    • refactor(event_handler): match to match_results; 3.10 new keyword (#616) by @michaelbrewer
    • chore(deps-dev): bump flake8-comprehensions from 3.6.0 to 3.6.1 (#615) by @dependabot
    • chore(deps): bump boto3 from 1.18.21 to 1.18.22 (#614) by @dependabot
    • chore(shared): fix cyclic import & refactor data extraction fn (#613) by @heitorlessa
    • chore(deps-dev): bump flake8-comprehensions from 3.5.0 to 3.6.0 (#609) by @dependabot
    • chore(deps): bump boto3 from 1.18.17 to 1.18.21 (#608) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.2.3 to 7.2.4 (#607) by @dependabot

    This release was made possible by the following contributors:

    @dependabot, @dependabot[bot], @heitorlessa, @hjurong and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.19.0(Aug 11, 2021)

    Summary

    This release highlights 1/ a brand new Feature Flags utility, 2/ auto-disable Tracer in non-Lambda environments to ease unit testing, 3/ API Gateway event handler now supports a custom JSON serializer, and a number of documentation improvements & bugfixes.

    We hope you enjoy this new utility as much as we did working on it!!

    New Feature Flags utility in Beta

    Special thanks to: @risenberg-cyberark, @michaelbrewer, @pcolazurdo and @am29d

    You might have heard of feature flags when:

    • Looking to conditionally enable a feature in your application for your customers
    • A/B testing a new feature for a subset of your customers
    • Working with trunk-based development where a feature might not be available right now
    • Working on short-lived features that will only be enabled for select customers

    This new utility makes this entire process so much easier by fetching feature flags configuration from AWS AppConfig, and evaluating contextual values against rules you defined to decide whether a feature should be enabled.

    Let's dive into the code to better understand what this all means.

    Evaluating whether a customer should have access to premium features

    from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore
    
    app_config = AppConfigStore(
        environment="dev",
        application="product-catalogue",
        name="features"
    )
    
    feature_flags = FeatureFlags(store=app_config)
    
    def lambda_handler(event, context):
        # Get customer's tier from incoming request
        ctx = { "tier": event.get("tier", "standard") }
    
        has_premium_features: bool = feature_flags.evaluate(name="premium_features",
                                                            context=ctx, default=False)
        if has_premium_features:
            # enable premium features
            ...
    

    Sample feature flag configuration in AWS AppConfig

    {
      "premium_features": {
        "default": false,
        "rules": {
          "customer tier equals premium": {
            "when_match": true,
            "conditions": [
              {
                "action": "EQUALS",
                "key": "tier",
                "value": "premium"
              }
            ]
          }
        }
      },
      "ten_percent_off_campaign": {
        "default": false
      }
    }
    

    Notice we have premium_features flag that will conditionally be available for premium customers, and a static feature flag named ten_percent_off_campaign that is disabled by default.

    Sample invocation event for this function

    {
        "username": "lessa",
        "tier": "premium",
        "basked_id": "random_id"
    }
    

    There's a LOT going on here. Allow me to break it down:

    1. We're defining a feature flag configuration that is stored in AWS AppConfig
    2. We initialize an AppConfigStore using AWS AppConfig values created via Infrastructure as code (available on docs)
    3. We initialize FeatureFlags and use our previously instantiated AppConfigStore
    4. We call evaluate method and pass the name of the premium feature, along with our contextual information our rules should run against, and a sentinel value to be used in case service errors happen
    5. Feature flags go through the rules defined in premium_features and evaluate whether tier key has the value premium
    6. FeatureFlags returns True which is then stored as has_premium_features variable

    But what if you have multiple feature flags and only want all enabled features?

    We've got you covered! You can use get_enabled_features to make a single call to AWS AppConfig and return a list of all enabled features at once.

    from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
    from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore
    
    app = ApiGatewayResolver()
    
    app_config = AppConfigStore(
        environment="dev",
        application="product-catalogue",
        name="features"
    )
    
    feature_flags = FeatureFlags(store=app_config)
    
    
    @app.get("/products")
    def list_products():
        ctx = {
            **app.current_event.headers,
            **app.current_event.json_body
        }
    
        # all_features is evaluated to ["geo_customer_campaign", "ten_percent_off_campaign"]
        all_features: list[str] = feature_flags.get_enabled_features(context=ctx)
    
        if "geo_customer_campaign" in all_features:
            # apply discounts based on geo
            ...
    
        if "ten_percent_off_campaign" in all_features:
            # apply additional 10% for all customers
            ...
    
    def lambda_handler(event, context):
        return app.resolve(event, context)
    
    

    But hang on, why Beta?

    We want to hear from you on the UX and evaluate how we can make it easier for you to bring your own feature flag store such as Redis, HashiCorp Consul, etc.

    When would you use feature flags vs env variables vs Parameters utility?

    Environment variables. For when you need simple configuration that will rarely if ever change, because changing it requires a Lambda function deployment.

    Parameters utility. For when you need access to secrets, or fetch parameters in different formats from AWS System Manager Parameter Store or Amazon DynamoDB.

    Feature flags utility. For when you need static or feature flags that will be enable conditionally based on the input and on a set of rules per feature whether that applies for all customers or on a per customer basis.

    In both Parameters and Feature Flags utility you can change their config without having to change your application code.

    Changes

    Changes

    🌟New features and non-breaking changes

    • feat(tracer): auto-disable tracer when for non-Lambda envs (#598) by @michaelbrewer
    • feat(feature-flags): Add not_in action and rename contains to in (#589) by @risenberg-cyberark
    • refactor(feature-flags): add debug for all features evaluation" (#590) by @heitorlessa
    • refactor(feature-flags): optimize UX and maintenance (#563) by @heitorlessa
    • feat(api-gateway): add support for custom serializer (#568) by @michaelbrewer
    • feat(params): expose high level max_age, raise_on_transform_error (#567) by @michaelbrewer
    • feat(data-classes): decode json_body if based64 encoded (#560) by @michaelbrewer

    πŸ“œ Documentation updates

    • fix(feature-toggles): correct cdk example (#601) by @michaelbrewer
    • docs(parameters): auto-transforming values based on suffix (#573) by @dreamorosi
    • docs(feature-flags): create concrete documentation (#594) by @am29d
    • refactor(feature-flags): optimize UX and maintenance (#563) by @heitorlessa
    • docs(readme): add code coverage badge (#577) by @michaelbrewer

    πŸ› Bug and hot fixes

    • fix(feature-flags): bug handling multiple conditions (#599) by @risenberg-cyberark
    • fix(parser): apigw wss validation check_message_id; housekeeping (#553) by @michaelbrewer

    πŸ”§ Maintenance

    • chore(deps): bump boto3 from 1.18.15 to 1.18.17 (#597) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.2.2 to 7.2.3 (#596) by @dependabot
    • chore(deps): bump boto3 from 1.18.1 to 1.18.15 (#591) by @dependabot
    • chore(deps): bump codecov/codecov-action from 2.0.1 to 2.0.2 (#558) by @dependabot
    • fix(deps): bump poetry to latest (#592) by @michaelbrewer
    • chore(deps-dev): bump mkdocs-material from 7.2.1 to 7.2.2 (#582) by @dependabot
    • refactor(feature-flags): add debug for all features evaluation" (#590) by @heitorlessa
    • chore(deps-dev): bump pdoc3 from 0.9.2 to 0.10.0 (#584) by @dependabot
    • docs(feature-flags): correct docs and typing (#588) by @michaelbrewer
    • refactor(feature-flags): optimize UX and maintenance (#563) by @heitorlessa
    • chore(deps-dev): bump isort from 5.9.2 to 5.9.3 (#574) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.2.0 to 7.2.1 (#566) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.1.11 to 7.2.0 (#551) by @dependabot
    • chore(deps-dev): bump flake8-black from 0.2.1 to 0.2.3 (#541) by @dependabot

    This release was made possible by the following contributors:

    @am29d, @dependabot, @dependabot[bot], @dreamorosi, @heitorlessa, @michaelbrewer, @risenberg-cyberark and @pcolazurdo

    Source code(tar.gz)
    Source code(zip)
  • v1.18.1(Jul 23, 2021)

    Summary

    Fix a pesky regression introduced when fixing API Gateway/ALB Event Handler routing regex in 1.18.0. This bug made numeric and safe URI chars to return 404 -- Big thanks to @moretension for raising it

    Changes

    πŸ› Bug and hot fixes

    • fix(api-gateway): route regression for non-word and unsafe URI chars (#556) by @heitorlessa

    This release was made possible by the following contributors:

    @heitorlessa

    Source code(tar.gz)
    Source code(zip)
  • v1.18.0(Jul 20, 2021)

    Summary

    This release mainly focused on bug fixes and a few minor features, so we can spend time documenting a new utility (Feature Toggles) that will be fully available in 1.19.0.

    Bug fixes were largely on MyPy errors (~600 to 38) across the entire code base. We also fixed a a) faulty greedy regex in the API Gateway event handler when dealing with long and dynamic routes (more in Changes section), b) Parser authorization and version fields for API Gateway that should've been optional, and c) Event Source Data Classes to conform with AppSync Scalar by including milliseconds in the time resolution.

    We also had two new first-time contributors: :clap: @walmsles and @whardier, we appreciate your help with this release!

    Changes

    New get_correlation_id method in Logger

    You can now retrieve the latest correlation ID previously set in the Logger instance at any part of the code base. This is useful when propagating correlation ID for external calls whether these are AWS service integrations or 3rd party endpoints.

    from aws_lambda_powertools import Logger
    
    logger = Logger(service="payment")
    
    @logger.inject_lambda_context(correlation_id_path="headers.my_request_id_header")
    def handler(event, context):
        logger.debug(f"NEW Correlation ID => {logger.get_correlation_id()}")
    

    Debug mode and HTTP service errors in Event Handlers

    You can now easily enable debug mode for API Gateway or ALB event handlers. Additionally, we've made it easier to raise quick HTTP service errors in response to malformed requests, resources not found, etc.

    from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
    from aws_lambda_powertools.event_handler.exceptions import (
        BadRequestError,
        InternalServerError,
        NotFoundError,
        ServiceError,
        UnauthorizedError,
    )
    
    app = ApiGatewayResolver(debug=True)
    
    
    @app.get(rule="/bad-request-error")
    def bad_request_error():
        # HTTP  400
        raise BadRequestError("Missing required parameter")
    
    @app.get(rule="/unauthorized-error")
    def unauthorized_error():
        # HTTP 401
        raise UnauthorizedError("Unauthorized")
    
    @app.get(rule="/not-found-error")
    def not_found_error():
        # HTTP 404
        raise NotFoundError
    
    @app.get(rule="/internal-server-error")
    def internal_server_error():
        # HTTP 500
        raise InternalServerError("Internal server error")
    
    @app.get(rule="/service-error", cors=True)
    def service_error():
        raise ServiceError(502, "Something went wrong!")
        # alternatively
        # from http import HTTPStatus
        # raise ServiceError(HTTPStatus.BAD_GATEWAY.value, "Something went wrong)
    
    def handler(event, context):
        return app.resolve(event, context)
    

    Data model sub-classing in AppSync event handler

    When building data-driven APIs using GraphQL and AppSync, you might have a set of reusable methods you want closer to the data model. Event Handler for AppSync supports a new parameter data_model to facilitate that.

    You can now subclass AppSyncResolverEvent from Event Source Data Classes while handling incoming requests with Event Handler for AppSync.

      from aws_lambda_powertools import Logger, Tracer
    
      from aws_lambda_powertools.logging import correlation_paths
      from aws_lambda_powertools.event_handler import AppSyncResolver
    
      tracer = Tracer(service="sample_resolver")
      logger = Logger(service="sample_resolver")
      app = AppSyncResolver()
    
    
      class MyCustomModel(AppSyncResolverEvent):
          @property
          def country_viewer(self) -> str:
              return self.request_headers.get("cloudfront-viewer-country")
    
      @app.resolver(field_name="listLocations")
      @app.resolver(field_name="locations")
      def get_locations(name: str, description: str = ""):
          if app.current_event.country_viewer == "US":
            ...
          return name + description
    
      @logger.inject_lambda_context(correlation_id_path=correlation_paths.APPSYNC_RESOLVER)
      @tracer.capture_lambda_handler
      def lambda_handler(event, context):
          return app.resolve(event, context, data_model=MyCustomModel)
    

    Bugfix API Gateway routing params

    This fixes a regex bug that used a greedy pattern ending with incorrect path resolution, as any path after a pattern would be included in the argument.

    Excerpt:

    @app.get("/accounts/<account_id>")
    def get_account(account_id: str):
        print(f"Account ID ({account_id}) would be 123")
    
    # Greedy regex would inject the incorrect function parameter
    @app.get("/accounts/<account_id>/source_networks")
    def get_account_networks(account_id: str):
        print(f"Account ID ({account_id}) would be 123/source_networks")
    

    In this example, say we have a GET request as /accounts/123 and another as /accounts/123/source_networks, we'd have the following effect prior to this fix:

    Function | Regex | Effective account_id value ------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- get_account | r'^/accounts/(?P<account_id>.+)$' | 123 get_account_networks | r'^/accounts/(?P<account_id>.+)/source_networks$' | 123/source_networks

    With this fix, account_id parameter would be 123 in both occasions due to word boundary not being non-greedy. This also allows an arbitrary number of dynamic route paths and static route paths.

    Function | Regex | Effective account_id value ------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- get_account | r'^/accounts/(?P<account_id>\\w+\\b)$' | 123 get_account_networks | r'^/accounts/(?P<account_id>\\w+\\b)/source_networks$' | 123

    🌟New features and non-breaking changes

    • feat(appsync): Support AppSyncResolverEvent subclassing (#526) by @whardier
    • feat(logger): add get_correlation_id method (#516) by @michaelbrewer
    • feat(feat-toggle): new simple feature toggles rule engine (WIP) (#494) by @risenberg-cyberark
    • feat(api-gateway): add debug mode (#507) by @michaelbrewer
    • feat(api-gateway): add common HTTP service errors (#506) by @michaelbrewer

    πŸ“œ Documentation updates

    • docs(api-gateway): new HTTP service error exceptions (#546) by @heitorlessa
    • docs(logger): new get_correlation_id method (#545) by @heitorlessa
    • feat(appsync): Support AppSyncResolverEvent subclassing (#526) by @whardier

    πŸ› Bug and hot fixes

    • fix(api-gateway): non-greedy route pattern regex (#533) by @heitorlessa
    • fix(tracer): mypy generic to preserve decorated method signature (#529) by @heitorlessa
    • fix(data-classes): include milliseconds in scalar types (#504) by @michaelbrewer
    • fix(parser): Make ApiGateway version, authorizer fields optional (#532) by @walmsles
    • fix(mypy): addresses lack of optional types (#521) by @michaelbrewer

    πŸ”§ Maintenance

    • chore: bump 1.18.0 (#547) by @heitorlessa
    • chore(deps-dev): bump mkdocs-material from 7.1.10 to 7.1.11 (#542) by @dependabot
    • chore(deps): bump codecov/codecov-action from 1 to 2.0.1 (#539) by @dependabot
    • refactor(feature-toggles): code coverage and housekeeping (#530) by @michaelbrewer
    • chore(deps): bump boto3 from 1.18.0 to 1.18.1 (#528) by @dependabot
    • chore(deps): bump boto3 from 1.17.110 to 1.18.0 (#527) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.1.9 to 7.1.10 (#522) by @dependabot
    • chore(deps): bump boto3 from 1.17.102 to 1.17.110 (#523) by @dependabot
    • chore(deps-dev): bump isort from 5.9.1 to 5.9.2 (#514) by @dependabot
    • chore(mypy): add mypy support to makefile (#508) by @michaelbrewer

    This release was made possible by the following contributors:

    @dependabot, @dependabot[bot], @heitorlessa, @michaelbrewer, @risenberg-cyberark, @walmsles and @whardier

    Source code(tar.gz)
    Source code(zip)
  • v1.17.1(Jul 2, 2021)

    Summary

    This patch release fixes JSON Schema Validation utility when a built-in format date-time is used but previously unrecognized.

    Additionally, this includes Dark Mode (πŸ•ΆοΈ) to the Documentation, Serverless Framework and CDK examples for installing Lambda Layers from SAR, and clarifications on logger and auto-capture tracer's feature.

    Changes

    πŸ“œ Documentation updates

    • docs(logger): add FAQ for cross-account searches (#501) by @heitorlessa
    • docs: add Layers example for Serverless framework & CDK (#500) by @heitorlessa
    • docs(tracer): additional scenario when to disable auto-capture (#499) by @heitorlessa
    • docs: enable dark mode switch (#471) by @michaelbrewer

    πŸ› Bug and hot fixes

    • fix(validator): handle built-in custom formats correctly (#498) by @heitorlessa

    πŸ”§ Maintenance

    • chore: bump 1.17.1 (#502) by @heitorlessa
    • chore(deps): bump boto3 from 1.17.101 to 1.17.102 (#493) by @dependabot
    • chore(deps-dev): bump flake8-eradicate from 1.0.0 to 1.1.0 (#492) by @dependabot
    • chore(deps-dev): bump isort from 5.8.0 to 5.9.1 (#487) by @dependabot
    • chore(deps): bump boto3 from 1.17.91 to 1.17.101 (#490) by @dependabot
    • chore(deps): bump email-validator from 1.1.2 to 1.1.3 (#478) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.1.7 to 7.1.9 (#491) by @dependabot
    • chore(deps): bump boto3 from 1.17.89 to 1.17.91 (#473) by @dependabot

    This release was made possible by the following contributors:

    @dependabot, @dependabot[bot], @heitorlessa and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.17.0(Jun 8, 2021)

    Summary

    This release brings a 1/ new handy decorator event_source for data classes, 2/ API Gateway HTTP API support for Parser, 3/ a new parameter to reset logging state upon every Lambda invocation in Logger, and 4/ high level methods to fetch attribute values and their types when using DynamoDBStreamEvent data class.

    Event source decorator

    Similarly to Parser's parse decorator, you can now use event_source decorator to instantiate a data class as part of the invocation more easily.

    from typing import Dict, Any
    
    from aws_lambda_powertools.utilities.data_classes import APIGatewayProxyEventV2, event_source
    from aws_lambda_powertools.utilities.typing import LambdaContext
    
    
    @event_source(data_class=APIGatewayProxyEventV2)
    def lambda_handler(event: APIGatewayProxyEventV2, context: LambdaContext) -> Dict[str, Any]:
        assert event.get_header_value("x-foo") == "Foo"
    

    This also gives you the ability to combine other utilities like Idempotency which expects a dictionary to serialize/deserialize without losing the benefits of self-documented schema and high level data methods.

    from aws_lambda_powertools.utilities.data_classes import APIGatewayProxyEventV2, event_source
    from aws_lambda_powertools.utilities.typing import LambdaContext
    from aws_lambda_powertools.utilities.idempotency import (
        DynamoDBPersistenceLayer, idempotent
    )
    
    persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
    
    @event_source(data_class=APIGatewayProxyEventV2)  
    @idempotent(persistence_store=persistence_layer)
    def lambda_handler(event: APIGatewayProxyEventV2, context):
        assert isinstance(event, APIGatewayProxyEventV2)
    

    API Gateway HTTP API Parser support

    Parser now supports API Gateway HTTP API as a native model in addition to REST API.

    from typing import Dict, Any
    
    from aws_lambda_powertools.utilities.parser import envelopes, event_parser
    from aws_lambda_powertools.utilities.parser.models import APIGatewayProxyEventV2Model
    
    
    @event_parser(model=APIGatewayProxyEventV2Model)
    def lambda_handler(event: APIGatewayProxyEventV2Model, context: LambdaContext) -> Dict[str, Any]:
        ...
    

    Like REST API, Parser also supports an envelope for HTTP API to help you extract your model from the payload, providing deep data validation, type enforcement at runtime, and more.

    from typing import Dict, Any
    
    from aws_lambda_powertools.utilities.parser import envelopes, event_parser
    from aws_lambda_powertools.utilities.parser.models import APIGatewayProxyEventV2Model
    from aws_lambda_powertools.utilities.parser.pydantic import EmailStr, UUID4, Json
    
    
    class UserModel(BaseModel):
        id: UUID4
        username: str
        preferences: Json
        email: EmailStr
    
    
    @event_parser(model=APIGatewayProxyEventV2Model, envelope=envelopes.ApiGatewayV2Envelope)
    def lambda_handler(event: UserModel, context: LambdaContext) -> Dict[str, Any]:
        assert event.username == "ran_isenberg"
    

    Resetting Logger state on every invocation

    As Logger is initialized in the global scope, logging state can persist across invocations. It is a good practice to always append new keys and use default values (None if unavailable), so Logger can update and remove keys on a per invocation basis.

    However, there are times where you might add keys conditionally depending on the incoming event. For these type of use cases, you can now use clear_state parameter when using inject_lambda_context decorator.

    from aws_lambda_powertools import Logger
    
    logger = Logger(service="payment")
    
    @logger.inject_lambda_context(clear_state=True)
    def handler(event, context):
        if event.get("special_key"):
            # Should only be available in the first request log
            # as the second request doesn't contain `special_key`
            logger.append_keys(debugging_key="value")
    
        logger.info("Collecting payment")
    

    DynamoDB Stream Event AttributeValueType and AttributeValue

    You can now introspect what DynamoDB Record type and value is more easily with AttributeValueType and AttributeValue. This is useful when you might want to deserialize or deal with certain records and their attributes differently based on their type, e.g. a map vs string.

    Each DynamoDB key available in the stream record now has high level methods such as get_value and get_type.

    from aws_lambda_powertools.utilities.data_classes import event_source, DynamoDBStreamEvent
    from aws_lambda_powertools.utilities.data_classes.dynamo_db_stream_event import AttributeValueType, AttributeValue
    from aws_lambda_powertools.utilities.typing import LambdaContext
    
    
    @event_source(data_class=DynamoDBStreamEvent)
    def lambda_handler(event: DynamoDBStreamEvent, context: LambdaContext):
        for record in event.records:
            key: AttributeValue = record.dynamodb.keys["id"]
            if key == AttributeValueType.Number:
                # {"N": "123.45"} => "123.45"
                assert key.get_value == key.n_value
                print(key.get_value)
            elif key == AttributeValueType.Map:
                assert key.get_value == key.map_value
                print(key.get_value)
    

    You can find their respective implementations and all contributors below.

    Changes

    🌟New features and non-breaking changes

    • feat(logger): add option to clear state per invocation (#467) by @heitorlessa
    • feat(data-classes): add AttributeValueType to DynamoDBStreamEvent (#462) by @michaelbrewer
    • feat(data-classes): decorator to instantiate data_classes and docs updates (#442) by @michaelbrewer
    • feat(parser): add support for API Gateway HTTP API #434 (#441) by @risenberg-cyberark

    πŸ“œ Documentation updates

    • feat(logger): add option to clear state per invocation (#467) by @heitorlessa
    • feat(data-classes): decorator to instantiate data_classes and docs updates (#442) by @michaelbrewer
    • docs: include new public roadmap (#452) by @heitorlessa
    • feat(parser): add support for API Gateway HTTP API #434 (#441) by @risenberg-cyberark

    πŸ”§ Maintenance

    • chore(deps): bump boto3 from 1.17.88 to 1.17.89 (#466) by @dependabot
    • chore(deps): bump boto3 from 1.17.87 to 1.17.88 (#463) by @dependabot
    • chore(deps-dev): bump mkdocs-material from 7.1.6 to 7.1.7 (#464) by @dependabot
    • chore(deps): bump boto3 from 1.17.86 to 1.17.87 (#459) by @dependabot
    • chore(deps): bump boto3 from 1.17.85 to 1.17.86 (#458) by @dependabot
    • chore(deps): bump boto3 from 1.17.84 to 1.17.85 (#455) by @dependabot
    • chore(deps-dev): bump pytest-cov from 2.12.0 to 2.12.1 (#454) by @dependabot
    • chore: update mergify to require approval on dependabot (#456) by @heitorlessa
    • chore: bump actions/setup-python from 1 to 2.2.2 (#445) by @dependabot
    • chore: bump xenon from 0.7.1 to 0.7.3 (#446) by @dependabot
    • chore: bump boto3 from 1.17.78 to 1.17.84 (#449) by @dependabot
    • chore: bump mkdocs-material from 7.1.5 to 7.1.6 (#451) by @dependabot
    • build(deps-dev): bump pytest-asyncio from 0.14.0 to 0.15.1 (#448) by @dependabot
    • chore: enable mergify (#450) by @heitorlessa
    • chore: enable dependabot for dep upgrades (#444) by @heitorlessa
    • chore: assited changelog pre-generation, auto-label PR (#443) by @heitorlessa

    This release was made possible by the following contributors:

    @dependabot, @dependabot[bot], @heitorlessa, @michaelbrewer and @risenberg-cyberark

    Source code(tar.gz)
    Source code(zip)
  • v1.16.1(May 23, 2021)

    Summary

    Emergency release to patch Pydantic against CVE-2021-29510used by Parser utility - This causes resource starvation when parsing infinity/-inf values.

    Thanks to @risenberg-cyberark for the quick turnaround.

    Changes

    • feat(parser): security issue in Pydantic #436 (#437) by @risenberg-cyberark

    This release was made possible by the following contributors:

    @risenberg-cyberark

    Source code(tar.gz)
    Source code(zip)
  • v1.16.0(May 17, 2021)

    Summary

    This release adds support for CodePipeline Job event, and a new property to save you some keystrokes when decoding base64 encoded data in API Gateway and ALB when using Data Classes utility.

    CodePipeline Job support

    If you're new to the CodePipeline Job event, customers typically use it for out of band integrations, such as orchestrate heavy deployments with a State Machine or validate an external system/KPI before continuing.

    This release adds support for CodePipeline Job including methods to easily retrieve user parameters, download artifacts from S3 using temporary credentials sent as part of event, etc.

    image

    Base64 decode support

    Based on this thread on Twitter with Luc from SentiaCloud, decoding base64 data in the first try without looking at a snippet is not for everyone :D

    When using Data classes for API Gateway or ALB, you can now easily decode data with the new decoded_body property:

    from aws_lambda_powertools.utilities.data_classes import APIGatewayProxyEvent
    
    
    def lambda_handler(event: dict, context):
        event = APIGatewayProxyEvent(event)
        # Dynamically base64 decode body as a str
        body: str = event.decoded_body
        
        return body
    

    Changes

    🌟 New features and non-breaking changes

    • feat(data-classes): support for code pipeline job event (#416) by @michaelbrewer

    🌟 Minor Changes

    • feat(data-classes): decode base64 encoded body (#425) by @michaelbrewer

    This release was made possible by the following contributors:

    @heitorlessa and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.15.1(May 13, 2021)

    Summary

    Quick release to fix a regression in Logger for the %s operator.

    Changes

    🌟 Minor Changes

    • fix(logger): Support .info("foo %s", "bar") formatting (#426) by @huonw

    πŸ“œ Documentation updates

    • fix(docs): Use updated names for ProxyEventType (#424) by @michaelbrewer

    This release was made possible by the following contributors:

    @heitorlessa, @huonw and @michaelbrewer

    Source code(tar.gz)
    Source code(zip)
  • v1.15.0(May 6, 2021)

    Summary

    This release has three major goodies: 1/ Idempotency utility is now GA, 2/ New API Gateway and ALB event handler, and 3/ MANY enhancements to Logger.

    Idempotency

    Idempotency utility was introduced in 1.11.0 as beta. If you haven't used it yet, it converts Lambda functions into idempotent operations that are safe to retry - This is mostly needed when your code is not idempotent.

    Since launch, we've made improvements to error handling and documentation updates. This gives us confidence that the UX won't radically change and therefore can be used safely moving forward.

    Idempotency showcase

    API Gateway and ALB event handler

    huge thanks to @michaelbrewer

    This new utility provides a lightweight routing to reduce boilerplate for API Gateway REST/HTTP API and ALB. It also natively integrates with Data classes utility giving you handy methods and self-documented properties of API Gateway and ALB events.

    image

    It's important to note that this is not a replacement for fully fledged web frameworks like Flask and Djano, or microframeworks like Chalice. Instead, this gives you a nearly zero overhead (~1-2ms) lightweight solution to build applications on top of either API Gateway or ALB.

    As a trade-off for being so lightweight, you have to setup infrastructure to use Lambda Proxy Integration using your preferred framework be that CloudFormation, SAM, CDK, Terraform, etc. Rest assured we provide a sample template to demonstrate how you can configure it if you haven't done this yet.

    Did I say CORS is simplified too? :)

    image

    Logger

    This release makes Logger more flexible by allowing you to bring your own Logging Formatter and Handler. It also includes a ton of improvements such as:

    • Easily remap keys or bring own defaults by inheriting the now exposed LambdaPowertoolsFormatter
    • UTC support with as simple as utc=True flag
    • Easier methods to add or remove logging keys at runtime with append_keys() and remove_keys(). The previous method, structure_logs(append=True, ...), will continue to work for backwards compatibility and will be effectively when we decide to cut a major version (2.0) - warnings will be place in time too
    • All keys can be reordered including the ones you're going to introduce yet at runtime
    • sampling_rate key only gets added when feature is added
    • Bring your own JSON serializer or deserializer function like the ultra fast Orjson, for when you need to crunch every possible ms
    • timestamp key now includes timezone at the end: from 2021-05-03 11:47:12,494 to 2021-05-03 11:47:12,494+0200

    image

    API Gateway support for Parser

    huge thanks to @risenberg-cyberark

    You can now easily parse and run deep data validation with APIGatewayProxyEvent in Parser, including an envelope ApiGatewayEnvelope.

    Metrics

    Last but not least, you can now set default metric dimensions to ensure these will always be added across all metrics: metrics.set_default_dimensions(environment="prod", another="one")

    Changes

    🌟New features and non-breaking changes

    • refactor(event-handler): Add ResponseBuilder and more docs (#412) by @michaelbrewer
    • feat(event-handler): add http ProxyEvent handler (#369) by @michaelbrewer
    • feat(metrics): add support to persist default dimensions (#410) by @heitorlessa
    • refactor(logger): BYOFormatter and Handler, UTC support, and more (#404) by @heitorlessa
    • feat(parser): Support for API GW v1 proxy schema & envelope (#403) by @risenberg-cyberark

    🌟 Minor Changes

    • feat(event-handler): allow for cors=None setting (#421) by @michaelbrewer
    • refactor(event-handler): api gateway handler review changes (#420) by @michaelbrewer
    • refactor: simplify custom formatter for minor changes (#417) by @heitorlessa

    πŸ“œ Documentation updates

    • docs(api_gateway): new event handler for API Gateway and ALB (#418) by @heitorlessa
    • docs(logger): improvements extensibility & new features (#415) by @heitorlessa
    • fix(docs): workflow to include api ref in latest alias (#408) by @heitorlessa
    • docs(tracer): Fix line highlighting (#395) by @michaelbrewer

    πŸ› Bug and hot fixes

    • fix(parser): Improve types for parser.py (#419) by @carlos-alberto
    • fix(validator): event type annotation as any in validate fn (#405) by @heitorlessa

    Internal

    • chore: bump to 1.15.0 (#422) by @heitorlessa
    • fix(deps): Bump aws-xray-sdk from 2.6.0 to 2.8.0 (#413) by @michaelbrewer

    This release was made possible by the following contributors:

    @carlos-alberto, @heitorlessa, @michaelbrewer and @risenberg-cyberark

    Source code(tar.gz)
    Source code(zip)
aws-lambda-scheduler lets you call any existing AWS Lambda Function you have in a future time.

aws-lambda-scheduler aws-lambda-scheduler lets you call any existing AWS Lambda Function you have in the future. This functionality is achieved by dyn

Oğuzhan Yılmaz 58 May 2, 2022
Python + AWS Lambda Hands OnPython + AWS Lambda Hands On

Python + AWS Lambda Hands On Python Criada em 1990, por Guido Van Rossum. "Bala de prata" (quase). Muito utilizado em: Automatizaçáes - Selenium, Beau

Marcelo Ortiz de Santana 8 Dec 1, 2021
Aws-lambda-requests-wrapper - Request/Response wrapper for AWS Lambda with API Gateway

AWS Lambda Requests Wrapper Request/Response wrapper for AWS Lambda with API Gat

null 1 Jan 26, 2022
A python library for creating Slack slash commands using AWS Lambda Functions

slashbot Slashbot makes it easy to create slash commands using AWS Lambda functions. These can be handy for creating a secure way to execute automated

Eric Brassell 14 Apr 2, 2022
Lambda-function - Python codes that allow notification of changes made to some services using the AWS Lambda Function

AWS Lambda Function This repository contains python codes that allow notificatio

Elif ApaydΔ±n 3 Feb 11, 2022
Criando Lambda Functions para Ingerir Dados de APIs com AWS CDK

LIVE001 - AWS Lambda para Ingerir Dados de APIs Fazer o deploy de uma função lambda com infraestrutura como código Lambda vai numa API externa e extra

Andre Sionek 11 Mar 24, 2022
SSH-Restricted deploys an SSH compliance rule (AWS Config) with auto-remediation via AWS Lambda if SSH access is public.

SSH-Restricted SSH-Restricted deploys an SSH compliance rule with auto-remediation via AWS Lambda if SSH access is public. SSH-Auto-Restricted checks

Adrian Hornsby 28 Mar 30, 2022
POC de uma AWS lambda que executa a consulta de preços de criptomoedas, e é implantada na AWS usando Github actions.

Cryptocurrency Prices Overview Instalação Repositório Configuração CI/CD Roadmap Testes Overview A ideia deste projeto é aplicar o conteúdo estudado s

Gustavo Santos 2 Oct 19, 2021
AWS-serverless-starter - AWS Lambda serverless stack via Serverless framework

Serverless app via AWS Lambda, ApiGateway and Serverless framework Configuration

 BΙ™xtiyar 3 Feb 2, 2022
Automated AWS account hardening with AWS Control Tower and AWS Step Functions

Automate activities in Control Tower provisioned AWS accounts Table of contents Introduction Architecture Prerequisites Tools and services Usage Clean

AWS Samples 11 May 14, 2022
FAIR Enough Metrics is an API for various FAIR Metrics Tests, written in python

β˜‘οΈ FAIR Enough metrics for research FAIR Enough Metrics is an API for various FAIR Metrics Tests, written in python, conforming to the specifications

Maastricht University IDS 2 Apr 14, 2022
Building and deploying AWS Lambda Shared Layers

AWS Lambda Shared Layers This repository is hosting the code from the following blog post: AWS Lambda & Shared layers for Python. The goal of this rep

Flaneer 7 Dec 2, 2021
AWS Lambda - Parsing Cloudwatch Data and sending the response via email.

AWS Lambda - Parsing Cloudwatch Data and sending the response via email. Author: Evan Erickson Language: Python Backend: AWS / Serverless / AWS Lambda

Evan Scott Erickson 1 Nov 14, 2021
Troposphere and shellscript based AWS infrastructure automation creates an awsapigateway lambda with a go backend

Automated-cloudformation-infra Troposphere and shellscript based AWS infrastructure automation. Feel free to clone and edit for personal usage. The en

null 1 Jan 3, 2022
AWS SQS event redrive Lambda With Python

AWS SQS event redrive Lambda This repository contains one simple AWS Lambda function in Python to redrive AWS SQS events from source queue to destinat

null 1 Oct 19, 2021
AWS SQS event redrive Lambda

This repository contains the Lambda function to redrive sqs events from source to destination queue while controlling maxRetry per event.

null 1 Oct 19, 2021
AWS Lambda Fast API starter application

AWS Lambda Fast API Fast API starter application compatible with API Gateway and Lambda Function. How to deploy it? Terraform AWS Lambda API is a reus

OBytes 6 Apr 20, 2022
A Python AWS Lambda Webhook listener that generates a permanent URL when an asset is created in Contentstack.

Webhook Listener A Python Lambda Webhook Listener - Generates a permanent URL on created assets. See doc on Generating a Permanent URL: https://www.co

Contentstack Solutions 1 Nov 4, 2021
JAWS Pankration 2021 - DDD on AWS Lambda sample

JAWS Pankration 2021 - DDD on AWS Lambda sample What is this project? This project contains sample code for AWS Lambda with domain models. I presented

Atsushi Fukui 21 Mar 30, 2022