From 2f7ae2cf6866eede4644a653ecdb017d38e1afde Mon Sep 17 00:00:00 2001 From: thomasabishop Date: Thu, 31 Aug 2023 20:40:30 +0100 Subject: [PATCH] chore: add python todos --- .../Accessing_secrets_from_a_Lambda.md | 38 ++++++ ..._walkthrough_Lambda_creation_within_AWS.md | 70 +++++++++++ .../Python/Concepts/Testing_python_code.md | 2 +- ...rough_complex_data_structures_in_Python.md | 114 ++++++++++++++++++ .../Python/Syntax/List_comprehension.md | 23 ++++ .../Python/Syntax/Loops_in_Python.md | 2 + 6 files changed, 248 insertions(+), 1 deletion(-) create mode 100644 DevOps/AWS/AWS_Lambda/Accessing_secrets_from_a_Lambda.md create mode 100644 DevOps/AWS/AWS_Lambda/Practical_walkthrough_Lambda_creation_within_AWS.md create mode 100644 Programming_Languages/Python/Syntax/Iterating_through_complex_data_structures_in_Python.md create mode 100644 Programming_Languages/Python/Syntax/List_comprehension.md diff --git a/DevOps/AWS/AWS_Lambda/Accessing_secrets_from_a_Lambda.md b/DevOps/AWS/AWS_Lambda/Accessing_secrets_from_a_Lambda.md new file mode 100644 index 0000000..8231c09 --- /dev/null +++ b/DevOps/AWS/AWS_Lambda/Accessing_secrets_from_a_Lambda.md @@ -0,0 +1,38 @@ +--- +categories: + - DevOps + - Backend +tags: [AWS, aws-lambda] +--- + +# Accessing secrets from a Lambda + +If a Lambda connects to a database or calls an API, it is going need access to authentication tokens/passwords. + +You obviously should not store these in your code. Instead you should store them in AWS Secrets Manager and access them via the `aws-sdk` in your Lambda function. + +In order for your function to be able to access the secrets however, it will need to be given permission. While the Lambda is not itself a "user" it does have an identity in the form of its IAM role. This is disclosed by its ARN. + +To allow the Lambda to access the secret you should add a resource permission on the secret that designates the Lambda. + +> When you create a Lambda function, you specify an IAM role that AWS Lambda can assume to execute the function on your behalf. This role is referred to as the execution role. The execution role grants the function the necessary permissions to call other AWS services, access resources, and perform other operations. + +Here is an example of a resource permission giving access to a Lambda: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": "arn:aws:iam::885135949562:role/pocket-api-lambda-QueryPocketFunctionRole-GY5ZN3RW31KE" + }, + "Action": "secretsmanager:GetSecretValue", + "Resource": "arn:aws:secretsmanager:eu-west-2:885135949562:secret:pocket-api-credentials-wEvQMI" + } + ] +} +``` + +// TODO: Add link to code example diff --git a/DevOps/AWS/AWS_Lambda/Practical_walkthrough_Lambda_creation_within_AWS.md b/DevOps/AWS/AWS_Lambda/Practical_walkthrough_Lambda_creation_within_AWS.md new file mode 100644 index 0000000..484c197 --- /dev/null +++ b/DevOps/AWS/AWS_Lambda/Practical_walkthrough_Lambda_creation_within_AWS.md @@ -0,0 +1,70 @@ +--- +categories: + - DevOps + - Backend +tags: [AWS, aws-lambda, node-js] +--- + +# Practical walkthrough of creating a Lambda function via the AWS Management Console + +## Basic set-up + +First we name the function and accept the defaults: + +![](/_img/lambda_func_one.png) + +This presents us with the function dashboard - a graphical representation of the Lambda showing [triggers]() as an input and destination as an output: + +![](/_img/lambda-func-two.png) + +Beneath this we have a code editor with the handler function with a basic boilerplate: + +![](/_img/lambda-func-three.png) + +## Adding a trigger + +Next we need to add a [trigger](/DevOps/AWS/AWS_Lambda/Lambda_triggers.md) that execute the handler. + +We will do this using [AWS API Gateway](/DevOps/AWS/AWS_API_Gateway.md). We select "Add trigger" from the dashboard view and input basic settings: + +![](/_img/api-gateway-trigger.png) + +Now we see this step displayed in the dashboard: + +![](/_img/lambda-overview.png) + +With the endpoint and other settings displayed: + +![](/_img/trigger-info.png) + +If we go to the endpoint URL (`https://4kjqwbvd7g.execute-api.us-east-1.amazonaws.com/default/myFirstFunction`), we will see the output: `Hello from Lambda`. + +## Handling parameters + +We can make the example more realistic by expanding the handler to accept query parameters. We do this by accessing the value `queryStringParameters` on the `event` object: + +```js +exports.handler = async (event) => { + const name = event.queryStringParameters && event.queryStringParameters.name; + + let message = "Hello Lambda"; + + if (name !== null) { + message = `Hello ${name}`; + } + + const response { + statusCode: 200, + body: JSON.stringify(message) + + } +}; +``` + +If we now access `https://4kjqwbvd7g.execute-api.us-east-1.amazonaws.com/default/myFirstFunction?name=Thomas` + +We get `Hello Thomas` as output. + +For a more advanced API with multiple endpoints and parameters, it's easiest to use Postman: + +![](/_img/postman-lambda.png) diff --git a/Programming_Languages/Python/Concepts/Testing_python_code.md b/Programming_Languages/Python/Concepts/Testing_python_code.md index db42896..abc572f 100644 --- a/Programming_Languages/Python/Concepts/Testing_python_code.md +++ b/Programming_Languages/Python/Concepts/Testing_python_code.md @@ -14,7 +14,7 @@ It is not included with the Python standard library so it must be installed with ### Using `pytest` -- Pytest will automatically detect test files so long as they are named appropriately. E.g. for a module called `lorem`, it will detect the unit test files `lorem_test.py` and `test_lorem.py`. +- Pytest will automatically detect test files so long as they are named appropriately. E.g. for a module called `lorem`, it will detzect the unit test files `lorem_test.py` and `test_lorem.py`. - In order to detect tests it should be run from a directory level above them ### Examples diff --git a/Programming_Languages/Python/Syntax/Iterating_through_complex_data_structures_in_Python.md b/Programming_Languages/Python/Syntax/Iterating_through_complex_data_structures_in_Python.md new file mode 100644 index 0000000..639fa50 --- /dev/null +++ b/Programming_Languages/Python/Syntax/Iterating_through_complex_data_structures_in_Python.md @@ -0,0 +1,114 @@ +--- +categories: + - Programming Languages +tags: [python] +--- + +# Iterating through complex data structures in Python + +// TODO: sections : + +- Looping, mapping, filtering lists +- Looping, mapping, filtering dictionaries +- As above with multi-dimensional lists +- As above with list of dictionaries + +In JavaScript when we return data from an API we tend to use an array of objects as the canonical form of a repeating iterable, e.g: + +```js +const data = [ + { + name: "Thomas", + age: 35, + }, + { + name: "Joe", + age: 75, + }, +]; +``` + +In Python there are two common ways to handle similar data structures: + +- A list of lists: + ```py + data = [ + ["Thomas", 35], + ["Joe", 75] + ] + ``` +- A list of dictionaries: + ```py + data = [ + {"name": "Thomas", "age": 35}, + {"name": "Joe", "age": 75}, + ] + ``` + +## List of lists + +### Sorting by common property + +Assuming the sub-lists have an identical structure, you can [sort](/Programming_Languages/Python/Syntax/Sorting_lists_in_Python.md) them by a common property by passing a [lambda function](/Programming_Languages/Python/Syntax/Lambdas_in_Python.md) to the `key` value of `sorted()` and `.sort()`. + +For example, to sort the following list of lists by the second `age` property: + +```python +people = [["Alice", 30], ["Bob", 25], ["Clare", 35], ["Dave", 28]] +``` + +Using `sorted()`: + +```py +sorted_people = sorted(people, key=lambda x: x[1]) +print(sorted_people) +# Output: [['Bob', 25], ['Dave', 28], ['Alice', 30], ['Clare', 35]] +``` + +Using `.sort`: + +```py +people.sort(key=lambda x: x[1]) +print(people) +# Output: [['Bob', 25], ['Dave', 28], ['Alice', 30], ['Clare', 35]] +``` + +If you want to sort by name instead, you could change the lambda function to `lambda x: x[0]`: + +```python +# Using sorted() +sorted_people = sorted(people, key=lambda x: x[0]) +print(sorted_people) +# Output: [['Alice', 30], ['Bob', 25], ['Clare', 35], ['Dave', 28]] + +# Using .sort() +people.sort(key=lambda x: x[0]) +print(people) +# Output: [['Alice', 30], ['Bob', 25], ['Clare', 35], ['Dave', 28]] +``` + +### Updating a value within a list of lists + +We can use `map` to mutate a given value within each list. + +In the following example we have a list of the following structure: + +```py +data = [ + ["1688582410", "Article One"], + ["1688647447", "Article Two"], + ["1689023491", "Article Three"], +] +``` + +Below, we apply a function to each of the first elements which is a Unix timestamp, converting it to a readable format: + +```py + readable_date = list(map(lambda i: [convert_timestamp(i[0])] + i[1:], date)) +``` + +Key points: + +- We apply the `convert_timestamp` function to the first element of each sublist +- We wrap this first element in `[]` so that it can be merged with the other elements of the list. This is necessary otherwise we will just return a list of the first elements and not include the other properties. +- The map and lambda is the core structure. We wrap it in `list` because `map` returns an object not a list. diff --git a/Programming_Languages/Python/Syntax/List_comprehension.md b/Programming_Languages/Python/Syntax/List_comprehension.md new file mode 100644 index 0000000..368bf6d --- /dev/null +++ b/Programming_Languages/Python/Syntax/List_comprehension.md @@ -0,0 +1,23 @@ +--- +categories: + - Programming Languages +tags: [python] +--- + +# List comprehension + +> List comprehension is an older feature of Python. Now the same functionality can be achieved with greater concision using functional methods like `map` and `filter`. But you may see it used in older code. + +```python +values = [1, 2, 4, 6, 8, 9] + +new_values = [i + 1 for i in values] +print('new_values', new_values) +# new_values [2, 3, 5, 7, 9, 10] +new_list = [item + 1 for item in values if item % 2 == 0] + +print('new_list:', new_list) +# new_list: [3, 5, 7, 9] +``` + +// TODO: Compare with how the same would be achieved with lambdas diff --git a/Programming_Languages/Python/Syntax/Loops_in_Python.md b/Programming_Languages/Python/Syntax/Loops_in_Python.md index 55a5ce1..d63d9c8 100644 --- a/Programming_Languages/Python/Syntax/Loops_in_Python.md +++ b/Programming_Languages/Python/Syntax/Loops_in_Python.md @@ -64,6 +64,8 @@ There are three main types of `for` loop in Python and they all use the `for...i ### Further examples of `for in range()` loops +// TODO: segment the examples into dedicated sections + ```python # Loop over a set of values in a range print('Print out values in a range')