# notes

path: Key_characteristics_of_GraphQL.md
sortable_path: Key_characteristics_of_GraphQL.md
title: Key characteristics of GraphQL
checksum: 67f1011150d2c40caae68f5b96c1c03339c9a9b90925c06b8b05534ca54fe2e9
created: 2025-10-01 13:30:59.348841996+00:00
modified: 2025-08-13 15:57:13.744428153+00:00
metadata: {"tags":["graphql","APIs"]}
lead:
  > In a nutshell, GraphQL is all about optimizing data communication between a
  > client and a server. This includes the client asking for needed data and
  > communicating that need to the server, and the server preparing a fulfillment
  > for that need and communicating the fulfillment back to the client. GraphQL
  > allows clients to ask for the exact data they need and makes it easier for
  > servers to aggregate data from multiple data storage resources.

body:
  > In a nutshell, GraphQL is all about optimizing data communication between a
  > client and a server. This includes the client asking for needed data and
  > communicating that need to the server, and the server preparing a fulfillment
  > for that need and communicating the fulfillment back to the client. GraphQL
  > allows clients to ask for the exact data they need and makes it easier for
  > servers to aggregate data from multiple data storage resources.

  - GraphQL is a **query language** from which you can build APIs. It exists as an
    abstraction layer between a client that issues requests and a backend
    comprising any number of databases that responds to queries.
  - Crucially, with applications that utilise GraphQL, both the backend and the
    frontend use the same GraphQL syntax. They speak the same language. This means
    that neither needs to take account of the way in which the other is
    implemented. For example, a client could request data from both a relational
    database and an object database without knowing there is any difference from
    its point of view. Conversely, client requests could be made through
    JavaScript or Python and the structure of the request received by the backend
    would be the same.
  - The main benefit is its interoperability and its network economy when compared
    to REST APIs: you ask only for what you want and you get only that; there is
    no surplus data, increasing efficiency, reducing the number of network
    requests and reducing the size of the payload.

  > The best way to think of GraphQL architecturally is to think of it as a
  > different technique for instantiating the _uniform interface_, _client/server
  > decoupling_, and _layered-system architecture_ contraints of traditional
  > [REST APIs](RESTful_APIs.md#rest). However, instead of making
  > GET or POST requests to a REST endpoint URL and passing parameters, you send a
  > structured query object.

  ## Architecture

  From the point of view of the frontend, GraphQL is a query language that may be
  used via wrappers for a given framework or programming language (such as Apollo)
  or directly simply by sending requests over HTTPS to a single URL.

  From the point of view of the backend, GraphQL is a **runtime** that provides a
  structure for servers to describe the data to be exposed in their APIs. We call
  this structure the **schema**. For example, the GraphQL server might translate
  the query into SQL statements for a relational database then take what the
  storage engine responds with, translate it into JSON and send it back to the
  client application.

  > The backend implementation of GraphQL is known as the **GraphQL server**. This
  > server is distinct from any of the physical servers that the backend may rely
  > on. The client queries the GraphQL server.

  Client requests are sent over HTTPS and the data is typically returned in the
  form of JSON:

  ![](/img/graphql-architecture.png)

  ## Implementation overview

  ### Queries, mutations, subscriptions

  Queries can be used for **both** reading and modifying data.

  - When you need to read data with GraphQL you use **queries**
  - When you need to modify data, you use **mutations**

  So, for comparison, a query would be akin to a `READ` instruction in SQL or a
  `GET` request send to a REST API. A mutation would be akin to `WRITE` then
  `READ` instruction in SQL or a `POST` request to a REST API.

  There is a third request type called a **subscription**. This is used for
  real-time data monitoring requests, like a data stream, similar to a continuous
  `READ` process. Mutations typically trigger events that can be subscribed to.

  ### Frontend

  Below is an example of a request that would be made to a GraphQL server from the
  frontend:

  ```graphql
  query {
    employee(id: 42) {
      name
      email
    }
  }
  ```

  ### Backend: structure and behaviour

  We define the structure of a GraphQL API on the backend through the schema. A
  schema is strongly typed and is basically a graph of fields that have types, e.g

  ```graphql
  type Employee(id: Int!) {
    name: String!
    email: String!
  }
  ```

  `!` stands for required. In addition to the standard primitive data-types you
  can also have custom types.

  We implement the behaviour of the API through functions called **resolver
  functions**. Each field in a GraphQL schema is backed by a resolver function. A
  resolver function defines what data to fetch for its field.

  > A resolver function represents the instructions on how and where to access raw
  > data. For example, a resolver function might issue a SQL statement to a
  > relational database, read a file’s data directly from the operating system, or
  > update some cached data in a document database. A resolver function is
  > directly related to a field in a GraphQL request, and it can represent a
  > single primitive value, an object, or a list of values or objects.

  ### GraphQL endpoint

  The GraphQL server exposes a single endpoint as a URL. This is in contrast to a
  REST API where there are multiple endpoints, each corresponding to a different
  resource. With GraphQL, the request is not specified by the URL and the payload
  but rather in the GraphQL string that you send to the endpoint, like the
  `employee` query above.

  ## Benefits

  ### Single requests

  With a REST API if you require multiple resources you have to make multiple
  requests to different endpoints and then once they resolve, synthesise them on
  the client side. With GraphQL you can send a single request that encompasses
  each individual resource:

  The REST scenario:

  ![](/img/REST_request-load.png)

  The GraphQL scenario:

  ![](/img/graphQL_request_load.png)

  ### Abstraction of multiple services

  ![](/img/graphql_multiple_resources.png)

  ### Stops overfetching

  In a strict REST API you must request all resources from a given resource and
  then filter on the client. With GraphQL you request only the resource you need
  at the client-level.

  ### Single endpoint

  In GraphQL you have a single endpoint and it always remains the same. This makes
  updates easier to manage, since you don't need to broadcast new endpoints as the
  API grows. It also simplifies frontend parsing, since you don't need to account
  for, or interpolate different endpoints in your request URL.

raw_content:
  ---
  tags: [graphql, APIs]
  ---

  # Key characteristics of GraphQL

  > In a nutshell, GraphQL is all about optimizing data communication between a
  > client and a server. This includes the client asking for needed data and
  > communicating that need to the server, and the server preparing a fulfillment
  > for that need and communicating the fulfillment back to the client. GraphQL
  > allows clients to ask for the exact data they need and makes it easier for
  > servers to aggregate data from multiple data storage resources.

  - GraphQL is a **query language** from which you can build APIs. It exists as an
    abstraction layer between a client that issues requests and a backend
    comprising any number of databases that responds to queries.
  - Crucially, with applications that utilise GraphQL, both the backend and the
    frontend use the same GraphQL syntax. They speak the same language. This means
    that neither needs to take account of the way in which the other is
    implemented. For example, a client could request data from both a relational
    database and an object database without knowing there is any difference from
    its point of view. Conversely, client requests could be made through
    JavaScript or Python and the structure of the request received by the backend
    would be the same.
  - The main benefit is its interoperability and its network economy when compared
    to REST APIs: you ask only for what you want and you get only that; there is
    no surplus data, increasing efficiency, reducing the number of network
    requests and reducing the size of the payload.

  > The best way to think of GraphQL architecturally is to think of it as a
  > different technique for instantiating the _uniform interface_, _client/server
  > decoupling_, and _layered-system architecture_ contraints of traditional
  > [REST APIs](RESTful_APIs.md#rest). However, instead of making
  > GET or POST requests to a REST endpoint URL and passing parameters, you send a
  > structured query object.

  ## Architecture

  From the point of view of the frontend, GraphQL is a query language that may be
  used via wrappers for a given framework or programming language (such as Apollo)
  or directly simply by sending requests over HTTPS to a single URL.

  From the point of view of the backend, GraphQL is a **runtime** that provides a
  structure for servers to describe the data to be exposed in their APIs. We call
  this structure the **schema**. For example, the GraphQL server might translate
  the query into SQL statements for a relational database then take what the
  storage engine responds with, translate it into JSON and send it back to the
  client application.

  > The backend implementation of GraphQL is known as the **GraphQL server**. This
  > server is distinct from any of the physical servers that the backend may rely
  > on. The client queries the GraphQL server.

  Client requests are sent over HTTPS and the data is typically returned in the
  form of JSON:

  ![](/img/graphql-architecture.png)

  ## Implementation overview

  ### Queries, mutations, subscriptions

  Queries can be used for **both** reading and modifying data.

  - When you need to read data with GraphQL you use **queries**
  - When you need to modify data, you use **mutations**

  So, for comparison, a query would be akin to a `READ` instruction in SQL or a
  `GET` request send to a REST API. A mutation would be akin to `WRITE` then
  `READ` instruction in SQL or a `POST` request to a REST API.

  There is a third request type called a **subscription**. This is used for
  real-time data monitoring requests, like a data stream, similar to a continuous
  `READ` process. Mutations typically trigger events that can be subscribed to.

  ### Frontend

  Below is an example of a request that would be made to a GraphQL server from the
  frontend:

  ```graphql
  query {
    employee(id: 42) {
      name
      email
    }
  }
  ```

  ### Backend: structure and behaviour

  We define the structure of a GraphQL API on the backend through the schema. A
  schema is strongly typed and is basically a graph of fields that have types, e.g

  ```graphql
  type Employee(id: Int!) {
    name: String!
    email: String!
  }
  ```

  `!` stands for required. In addition to the standard primitive data-types you
  can also have custom types.

  We implement the behaviour of the API through functions called **resolver
  functions**. Each field in a GraphQL schema is backed by a resolver function. A
  resolver function defines what data to fetch for its field.

  > A resolver function represents the instructions on how and where to access raw
  > data. For example, a resolver function might issue a SQL statement to a
  > relational database, read a file’s data directly from the operating system, or
  > update some cached data in a document database. A resolver function is
  > directly related to a field in a GraphQL request, and it can represent a
  > single primitive value, an object, or a list of values or objects.

  ### GraphQL endpoint

  The GraphQL server exposes a single endpoint as a URL. This is in contrast to a
  REST API where there are multiple endpoints, each corresponding to a different
  resource. With GraphQL, the request is not specified by the URL and the payload
  but rather in the GraphQL string that you send to the endpoint, like the
  `employee` query above.

  ## Benefits

  ### Single requests

  With a REST API if you require multiple resources you have to make multiple
  requests to different endpoints and then once they resolve, synthesise them on
  the client side. With GraphQL you can send a single request that encompasses
  each individual resource:

  The REST scenario:

  ![](/img/REST_request-load.png)

  The GraphQL scenario:

  ![](/img/graphQL_request_load.png)

  ### Abstraction of multiple services

  ![](/img/graphql_multiple_resources.png)

  ### Stops overfetching

  In a strict REST API you must request all resources from a given resource and
  then filter on the client. With GraphQL you request only the resource you need
  at the client-level.

  ### Single endpoint

  In GraphQL you have a single endpoint and it always remains the same. This makes
  updates easier to manage, since you don't need to broadcast new endpoints as the
  API grows. It also simplifies frontend parsing, since you don't need to account
  for, or interpolate different endpoints in your request URL.


path: Key_figures_ARPANET.md
sortable_path: Key_figures_ARPANET.md
title: 4a3dc316_key_figures_ARPANET
checksum: 0b781afb66f4731f64577a49ea652afc95660de8a0054f8c9728ef2baaf05c61
created: 2025-10-01 13:30:59.357518342+00:00
modified: 2025-08-13 15:57:13.845974061+00:00
metadata: {"created":"Monday, October 28, 2024","tags":["internet","ARPANET","ARPA"]}
lead:
  - [Paul Baran](385af4b4_Baran_distributed_networks.md) for his work at RAND on
    distributed networks as an alternative to the extant
    centralised/de-centralised phone networks.

body:
  - [Paul Baran](385af4b4_Baran_distributed_networks.md) for his work at RAND on
    distributed networks as an alternative to the extant
    centralised/de-centralised phone networks.

  - Donald Davies at the UK Physical Laboratory for developing ideas on
    #packet-switching, similar to but derived independently of Baran's work.

  - Bob Kahn as the principle designer of the ARPANET, who with Vint Cerf, wrote
    the TCP/IP protocols. Became director of the Information Processing Techniques
    Office within ARPA/DARPA.

  - Vint Cerf was head of the International Networking Group and worked on the
    Network Control Protocol (the first protocol for packet switching using the
    IMPs). Later joined up with Kahn on TCP/IP and the network of packet-switching
    networks (early internet).

raw_content:
  ---
  tags: [internet, ARPANET, ARPA]
  created: Monday, October 28, 2024
  ---

  # 4a3dc316_key_figures_ARPANET

  - [Paul Baran](385af4b4_Baran_distributed_networks.md) for his work at RAND on
    distributed networks as an alternative to the extant
    centralised/de-centralised phone networks.

  - Donald Davies at the UK Physical Laboratory for developing ideas on
    #packet-switching, similar to but derived independently of Baran's work.

  - Bob Kahn as the principle designer of the ARPANET, who with Vint Cerf, wrote
    the TCP/IP protocols. Became director of the Information Processing Techniques
    Office within ARPA/DARPA.

  - Vint Cerf was head of the International Networking Group and worked on the
    Network Control Protocol (the first protocol for packet switching using the
    IMPs). Later joined up with Kahn on TCP/IP and the network of packet-switching
    networks (early internet).


path: Killing_processes.md
sortable_path: Killing_processes.md
title: Killing processes
lead: ## Kill a process running on a port
checksum: 07218a7413a82d17fabe750d5aae2b74aebb0b518682695a1842695a96dc5f3c
created: 2025-10-01 13:30:59.368505196+00:00
modified: 2025-08-13 15:57:13.850430552+00:00
metadata: {"tags":["shell","procedural"]}
body:
  ## Kill a process running on a port

  For example a local server.

  ```bash
  sudo lsof -t -i:8000
  # List files and proces ID (-t) and internet connections (-i) on port number

  $ 7890
  # Gives you process id

  sudo kill -9 7890
  # Kill the process that is running there
  ```

  ## Alterntive

  ```sh
  ss -ltnp | grep ':8000'
  ```

  Returns:

  ```
  LISTEN 0      511            [::1]:8000          [::]:*    users:(("node",pid=10493,fd=34))
  ```

  Kill with:

  ```
  kill 10493
  ```

raw_content:
  ---
  tags:
    - shell
    - procedural
  ---

  # Killing processes

  ## Kill a process running on a port

  For example a local server.

  ```bash
  sudo lsof -t -i:8000
  # List files and proces ID (-t) and internet connections (-i) on port number

  $ 7890
  # Gives you process id

  sudo kill -9 7890
  # Kill the process that is running there
  ```

  ## Alterntive

  ```sh
  ss -ltnp | grep ':8000'
  ```

  Returns:

  ```
  LISTEN 0      511            [::1]:8000          [::]:*    users:(("node",pid=10493,fd=34))
  ```

  Kill with:

  ```
  kill 10493
  ```


path: LEDs.md
sortable_path: LEDs.md
title: LEDs
checksum: 5c8ac42013743c18c7549d23da080ac4901800a4fa11ec252f85f79d3992b0b8
created: 2025-10-01 13:30:59.372310853+00:00
modified: 2025-08-13 15:57:13.944879174+00:00
metadata: {"tags":["electronics"]}
lead:
  LED' stands for **Light Emitting Diode**, a [circuit]() component that emits
  light. The symbol for an LED is displayed below:

body:
  LED' stands for **Light Emitting Diode**, a [circuit]() component that emits
  light. The symbol for an LED is displayed below:

  ![](/img/diode-led.png)

  A **diode** is a special kind of component that only permits current to flow
  through it in one direction. To achieve this it has very low resistance in one
  direction to allow current flow and high resistance in the other direction to
  impede current flow. This feature of diodes is clearly represented in the
  generic diode circuit symbol:

  ![](/img/diode.png)

  An LED diode lights up when the right amount of current flows through it. A
  standard LED has a maximum current of 20mA. An appropriate
  [resistor](Resistance.md#resistors)
  must therefore be added to the circuit to ensure the current doesn't exceed this
  amount.

raw_content:
  ---
  tags:
    - electronics
  ---

  # LEDs

  LED' stands for **Light Emitting Diode**, a [circuit]() component that emits
  light. The symbol for an LED is displayed below:

  ![](/img/diode-led.png)

  A **diode** is a special kind of component that only permits current to flow
  through it in one direction. To achieve this it has very low resistance in one
  direction to allow current flow and high resistance in the other direction to
  impede current flow. This feature of diodes is clearly represented in the
  generic diode circuit symbol:

  ![](/img/diode.png)

  An LED diode lights up when the right amount of current flows through it. A
  standard LED has a maximum current of 20mA. An appropriate
  [resistor](Resistance.md#resistors)
  must therefore be added to the circuit to ensure the current doesn't exceed this
  amount.


path: Lambda_directory_structure.md
sortable_path: Lambda_directory_structure.md
title: Lambda directory structure
lead: I am using the following structure for my lambdas:
checksum: 3a91446521948d44f2605951ee5bb5eef2a24d6133d7cd013309cce89c42bdd3
created: 2025-10-01 13:30:59.381685068+00:00
modified: 2025-08-13 15:57:14.059550768+00:00
metadata: {"tags":["AWS","aws-lambda"]}
body:
  I am using the following structure for my lambdas:

  ```
  lambda-name/
      events/
      function-name/
          src/
              index.js
          tests/
          local-env.json
          production-env.json
          package.json
      template.yaml
      README.md
  ```

raw_content:
  ---
  tags:
    - AWS
    - aws-lambda
  ---

  # Lambda directory structure

  I am using the following structure for my lambdas:

  ```
  lambda-name/
      events/
      function-name/
          src/
              index.js
          tests/
          local-env.json
          production-env.json
          package.json
      template.yaml
      README.md
  ```


path: Lambda_handler_function.md
sortable_path: Lambda_handler_function.md
title: AWS Lambda handler function
lead: Every Lambda function begins the same way with a handler function. In NodeJS:
checksum: db7083124ec79bc49cc7e3f28756a97c48e7d950d679c34bcb546b92a6982771
created: 2025-10-01 13:30:59.385921175+00:00
modified: 2025-08-13 15:57:13.971093233+00:00
metadata: {"tags":["AWS","aws-lambda"]}
body:
  Every Lambda function begins the same way with a handler function. In NodeJS:

  ```js
  module.exports.handler = function(event, context, callback){...}
  ```

  The handler is the entry point for the Lambda. It is a function that Lambda can
  call to start the execution of your code when an event triggers your Lambda
  function.

  You may put your whole code in the body of the handler, or you may call other
  modules and keep the handler clean and focused on bootstrapping.

  The `module.exports` part is just the standard way of exporting a module in
  Node. `handler` is a property that is being exported.

  You always have access to the parameters `event`, `context` and `callback`.

  - `event`
    - The triggering event of the Lambda function stored as an object. The
      object's contents will be different depending on the trigger. It could be
      any AWS service that serves a trigger (S3, DynamoDB, API Gateway etc).
    - For example, if the triggering function is a request to API Gateway, the
      `event` object will contain information about the HTTP request such as
      headers, path, and the request body.
  - `context`
    - contains information about the runtime environment and the current execution
      of the Lambda function. The context object provides properties and methods
      that you can use to interact with the AWS Lambda service and the current
      invocation.
  - `callback`
    - an optional parameter that represents a callback function provided by AWS
      Lambda. The callback function allows you to signal the completion of your
      Lambda function's execution and return a response or an error. The callback
      function accepts two arguments: error and result. If the first argument is
      not null, AWS Lambda treats it as an error, and the Lambda function
      invocation fails. If the first argument is null, AWS Lambda treats the
      second argument as the result and considers the invocation successful. Using
      the callback function is more common in non-async handler functions. When
      you use async functions, you can return a response or throw an error
      directly, without using the callback function.

  ### Full example:

  In JavaScript:

  ```js
  module.exports.handler = async (event, context) => {
    // Your logic to handle the incoming event
    // For example, you can call other AWS services, process the event data, etc.

    // Return a response
    return {
      statusCode: 200,
      body: JSON.stringify({
        message: "Your Lambda function has successfully executed!",
      }),
    };
  };
  ```

  In Python:

  ```py
  import json

  def handler(event, context):
      # Your logic to handle the incoming event
      # For example, you can call other AWS services, process the event data, etc.

      # Return a response
      response = {
          'statusCode': 200,
          'body': json.dumps({
              'message': 'Your Lambda function has successfully executed!'
          })
      }
      return response
  ```

raw_content:
  ---
  tags:
    - AWS
    - aws-lambda
  ---

  # AWS Lambda handler function

  Every Lambda function begins the same way with a handler function. In NodeJS:

  ```js
  module.exports.handler = function(event, context, callback){...}
  ```

  The handler is the entry point for the Lambda. It is a function that Lambda can
  call to start the execution of your code when an event triggers your Lambda
  function.

  You may put your whole code in the body of the handler, or you may call other
  modules and keep the handler clean and focused on bootstrapping.

  The `module.exports` part is just the standard way of exporting a module in
  Node. `handler` is a property that is being exported.

  You always have access to the parameters `event`, `context` and `callback`.

  - `event`
    - The triggering event of the Lambda function stored as an object. The
      object's contents will be different depending on the trigger. It could be
      any AWS service that serves a trigger (S3, DynamoDB, API Gateway etc).
    - For example, if the triggering function is a request to API Gateway, the
      `event` object will contain information about the HTTP request such as
      headers, path, and the request body.
  - `context`
    - contains information about the runtime environment and the current execution
      of the Lambda function. The context object provides properties and methods
      that you can use to interact with the AWS Lambda service and the current
      invocation.
  - `callback`
    - an optional parameter that represents a callback function provided by AWS
      Lambda. The callback function allows you to signal the completion of your
      Lambda function's execution and return a response or an error. The callback
      function accepts two arguments: error and result. If the first argument is
      not null, AWS Lambda treats it as an error, and the Lambda function
      invocation fails. If the first argument is null, AWS Lambda treats the
      second argument as the result and considers the invocation successful. Using
      the callback function is more common in non-async handler functions. When
      you use async functions, you can return a response or throw an error
      directly, without using the callback function.

  ### Full example:

  In JavaScript:

  ```js
  module.exports.handler = async (event, context) => {
    // Your logic to handle the incoming event
    // For example, you can call other AWS services, process the event data, etc.

    // Return a response
    return {
      statusCode: 200,
      body: JSON.stringify({
        message: "Your Lambda function has successfully executed!",
      }),
    };
  };
  ```

  In Python:

  ```py
  import json

  def handler(event, context):
      # Your logic to handle the incoming event
      # For example, you can call other AWS services, process the event data, etc.

      # Return a response
      response = {
          'statusCode': 200,
          'body': json.dumps({
              'message': 'Your Lambda function has successfully executed!'
          })
      }
      return response
  ```


path: Lambda_programming_model.md
sortable_path: Lambda_programming_model.md
title: AWS Lambda programming model
lead: The overall architecure consists in the following three processes:
checksum: fc67720166bcee6eceb573ee829de652e833340b4ad16163bb96ba51ee4ca7ad
created: 2025-10-01 13:30:59.39016553+00:00
modified: 2025-08-13 15:57:13.955911918+00:00
metadata: {"tags":["AWS","aws-lambda"]}
body:
  The overall architecure consists in the following three processes:

  1. Triggers
  2. Handler function
  3. Code

  ## Triggers

  See [AWS Lambda triggers](Lambda_triggers.md)

  ## Handler function

  See
  [AWS Lambda handler function](Lambda_handler_function.md)

  ## Code

  This is just the stuff that runs in the body of the handler or that is called
  from the handler body.

raw_content:
  ---
  tags:
    - AWS
    - aws-lambda
  ---

  # AWS Lambda programming model

  The overall architecure consists in the following three processes:

  1. Triggers
  2. Handler function
  3. Code

  ## Triggers

  See [AWS Lambda triggers](Lambda_triggers.md)

  ## Handler function

  See
  [AWS Lambda handler function](Lambda_handler_function.md)

  ## Code

  This is just the stuff that runs in the body of the handler or that is called
  from the handler body.


path: Lambda_triggers.md
sortable_path: Lambda_triggers.md
title: AWS Lambda triggers
lead: Here are some common trigger patterns
checksum: 3586771db03bd5e8ea9e70f94d4be5e2b0015c1b5c44765942f08868468cd483
created: 2025-10-01 13:30:59.402610993+00:00
modified: 2025-08-13 15:57:13.980974947+00:00
metadata: {"tags":["AWS","aws-lambda"]}
body:
  Here are some common trigger patterns

  ### API Gateway trigger

  - an HTTP request is sent to AWS API Gateway
  - API Gateway triggers a Lambda function to execute

  ### DynamoDB table change trigger

  - a record in a DynamoDB table is modified
  - DynamoDB triggers a lambda function to execute

  ### S3 trigger

  - a new file is created in S3
  - S3 triggers a Lambda function to execute

  ### SQS trigger

  - a new message is added to an [SQS](AWS_Messaging_services.md#sqs)
    queue
  - SQS triggers a Lambda function to execute

raw_content:
  ---
  tags: [AWS, aws-lambda]
  ---

  # AWS Lambda triggers

  Here are some common trigger patterns

  ### API Gateway trigger

  - an HTTP request is sent to AWS API Gateway
  - API Gateway triggers a Lambda function to execute

  ### DynamoDB table change trigger

  - a record in a DynamoDB table is modified
  - DynamoDB triggers a lambda function to execute

  ### S3 trigger

  - a new file is created in S3
  - S3 triggers a Lambda function to execute

  ### SQS trigger

  - a new message is added to an [SQS](AWS_Messaging_services.md#sqs)
    queue
  - SQS triggers a Lambda function to execute


path: Lambdas_in_Python.md
sortable_path: Lambdas_in_Python.md
title: Lambdas in Python
checksum: 9fe2f11066968420a8ac1ad471b719570bc546746041668abd6893421ce7c305
created: 2025-10-01 13:30:59.413087855+00:00
modified: 2025-08-13 15:57:13.864010057+00:00
metadata: {"tags":["python","aws-lambda","AWS"]}
lead:
  In Python, anonymous functions like arrow-functions in JavaScript (`() => {}`)
  are immediately invoked and unnamed. They are called lambdas.

body:
  In Python, anonymous functions like arrow-functions in JavaScript (`() => {}`)
  are immediately invoked and unnamed. They are called lambdas.

  Whilst they are unnamed, just like JS, the value they return can be stored in a
  variable. They do not require the `return` keyword.

  They are most often used unnamed with the functional methods
  [map, filter](Map_and_filter_in_Python.md)
  and reduce.

  Here is the two syntaxes side by side:

  ```js
  // JavaScript

  const double = (x) => x * x;
  ```

  ```py
  # Python

  double = lambda x: x * x
  ```

  Here is a lambda with multiple parameters:

  ```py
  func = lambda x, y, z: x + y + z
  print(func(2, 3, 4))
  # 9
  ```

  > Lambdas obviously enshrine functional programming paradigms. Therefore they
  > should be pure functions, not mutating values or issueing side effects. For
  > example, it would be improper (though syntactically well-formed) to use a
  > lambda to `print` something

raw_content:
  ---
  tags:
    - python
    - aws-lambda
    - AWS
  ---

  # Lambdas in Python

  In Python, anonymous functions like arrow-functions in JavaScript (`() => {}`)
  are immediately invoked and unnamed. They are called lambdas.

  Whilst they are unnamed, just like JS, the value they return can be stored in a
  variable. They do not require the `return` keyword.

  They are most often used unnamed with the functional methods
  [map, filter](Map_and_filter_in_Python.md)
  and reduce.

  Here is the two syntaxes side by side:

  ```js
  // JavaScript

  const double = (x) => x * x;
  ```

  ```py
  # Python

  double = lambda x: x * x
  ```

  Here is a lambda with multiple parameters:

  ```py
  func = lambda x, y, z: x + y + z
  print(func(2, 3, 4))
  # 9
  ```

  > Lambdas obviously enshrine functional programming paradigms. Therefore they
  > should be pure functions, not mutating values or issueing side effects. For
  > example, it would be improper (though syntactically well-formed) to use a
  > lambda to `print` something


path: Latches.md
sortable_path: Latches.md
title: Latches
checksum: 456a5efe4f003860290d55f3eacc357ef5b2f103f0fb03419c1bb61e014f3cc0
created: 2025-10-01 13:30:59.424872804+00:00
modified: 2025-08-13 15:57:13.815699049+00:00
metadata: {"tags":["logic-gates","binary","memory"]}
lead:
  The **combinatorial digital circuits** we have looked at so far have been
  non-sequential. The outcome is a function of its immediate set of inputs and
  everything happens at once: there is no means of storing state for future use.
  In other words there is no _[memory](Memory.md)_.

body:
  The **combinatorial digital circuits** we have looked at so far have been
  non-sequential. The outcome is a function of its immediate set of inputs and
  everything happens at once: there is no means of storing state for future use.
  In other words there is no _[memory](Memory.md)_.

  In contrast, the output of a **sequential digital circuit** depends not only on
  its present set of inputs but also on past inputs to the circuit. It has some
  knowledge of its own previous state through the existence of memory. This can be
  implemented via components that allow for the **storage and retrieval of binary
  data**.

  ## What is a latch?

  A latch is a circuit component that wor

  The **SR Latch** (for "set/reset") has two inputs: _S_ (for "set") and _R_ (for
  "reset") and one output, _Q_. _Q_ stands for the bit that is remembered. (There
  is also _not-Q_ which is the opposite of whatever _Q_ is currently set to.)

  The SR Latch goes through the following state changes:

  - When _S_ is set to 1, output _Q_ becomes 1 also
  - When _S_ goes to 0, _Q_ remains 1
  - When _R_ is set to 1, the memory bit is cleared and _Q_ becomes 0.
  - _Q_ remains at 0 even if _R_ goes back to 0

  This is represented more clearly in the table below:

  | S   | R   | Q   | Operation     |
  | --- | --- | --- | ------------- |
  | 1   | 1   | X   | Invalid, null |

  The most succinct account of a latch:

  > A latch is a 1-bit memory device that has a state _Q_ of either 1 or 0. The
  > _S_ input sets _Q_ to 1 and the _R_ input resets _Q_ to 0.

  _The representation of an SR Latch in a digital circuit diagram_:

  ![](/img/sr_latch_diagram.png)

  ## Creating a latch circuit

  The circuit diagram latch symbol obviously encapsulates more complex
  functionality that occurs at the sub-circuit level. We will demonstrate how this
  functionality can be achieved with two
  [NOR](Logic_gates.md#nor-gate) gates.

  The two gates are in a **cross-coupled configuration**. This basically means
  that the wires are crossed back on themselves such that the output of one is
  also an input of the other at a single stage in the sequence.

  The circuit is created as follows:

  ![](/img/sr_latch_logic_circuit.png)

  Interactive version:

  <iframe src="https://circuitverse.org/simulator/embed/nor-latch-0869192c-7d7b-4161-b13f-3f72c1bce8e9" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="400" width="600" allowFullScreen></iframe>

  <br />
  Let's talk through the logic at each state change:

  - When _S_ is set to 1, output _Q_ becomes 1 also
    - N1 is receiving 1 from S and 0 from R by way of N2. This is the inversion of
      OR so TF equals F. Thus N1 is outputting 0.
    - Thus N2 is receiving 0 from N1 as its top input and 0 from R as its bottom
      input. It is therefore outputting 1 because with NOR, FF equals T. For this
      reason, _Q_ is 1 because is directly connected to _R_.
  - When _S_ goes to 0, _Q_ remains 1
    - N2 is receiving 0 from N1 as the top input and 0 from R as the bottom input
      hence the overall input is FF which means N2 is outputting T and Q remains 1
    - N1 is outputting 0 because it is receiving 0 as its top input and 1 from its
      bottom input
  - When _R_ is set to 1, the memory bit is cleared and _Q_ becomes 0.
    - N2 is receiving 1 from R as its bottom input and 1 from the output of N1 as
      its top input. Therefore it is outputting TT which in NOR evaluates to F
      hence Q is 0
    - N1 is outputting 1 because it is receiving 0 from S as its top input and 0
      from its bottom input coming from N2. FF equals T in NOR therefore 1 is
      outputting 1
  - _Q_ remains at 0 even if _R_ goes back to 0
    - N2 is receiving 0 as its bottom input from R and 1 as its top input from N1.
      TF equals F in NOR hence the output of N2 is 0 and Q remains 0
    - N1 is outputting 1 because it is receiving 0 as its top input from S and 0
      as its bottom input from N2. FF equals T in NOR hence N1 is 1

raw_content:
  ---
  tags: [logic-gates, binary, memory]
  ---

  # Latches

  The **combinatorial digital circuits** we have looked at so far have been
  non-sequential. The outcome is a function of its immediate set of inputs and
  everything happens at once: there is no means of storing state for future use.
  In other words there is no _[memory](Memory.md)_.

  In contrast, the output of a **sequential digital circuit** depends not only on
  its present set of inputs but also on past inputs to the circuit. It has some
  knowledge of its own previous state through the existence of memory. This can be
  implemented via components that allow for the **storage and retrieval of binary
  data**.

  ## What is a latch?

  A latch is a circuit component that wor

  The **SR Latch** (for "set/reset") has two inputs: _S_ (for "set") and _R_ (for
  "reset") and one output, _Q_. _Q_ stands for the bit that is remembered. (There
  is also _not-Q_ which is the opposite of whatever _Q_ is currently set to.)

  The SR Latch goes through the following state changes:

  - When _S_ is set to 1, output _Q_ becomes 1 also
  - When _S_ goes to 0, _Q_ remains 1
  - When _R_ is set to 1, the memory bit is cleared and _Q_ becomes 0.
  - _Q_ remains at 0 even if _R_ goes back to 0

  This is represented more clearly in the table below:

  | S   | R   | Q   | Operation     |
  | --- | --- | --- | ------------- |
  | 1   | 1   | X   | Invalid, null |

  The most succinct account of a latch:

  > A latch is a 1-bit memory device that has a state _Q_ of either 1 or 0. The
  > _S_ input sets _Q_ to 1 and the _R_ input resets _Q_ to 0.

  _The representation of an SR Latch in a digital circuit diagram_:

  ![](/img/sr_latch_diagram.png)

  ## Creating a latch circuit

  The circuit diagram latch symbol obviously encapsulates more complex
  functionality that occurs at the sub-circuit level. We will demonstrate how this
  functionality can be achieved with two
  [NOR](Logic_gates.md#nor-gate) gates.

  The two gates are in a **cross-coupled configuration**. This basically means
  that the wires are crossed back on themselves such that the output of one is
  also an input of the other at a single stage in the sequence.

  The circuit is created as follows:

  ![](/img/sr_latch_logic_circuit.png)

  Interactive version:

  <iframe src="https://circuitverse.org/simulator/embed/nor-latch-0869192c-7d7b-4161-b13f-3f72c1bce8e9" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="400" width="600" allowFullScreen></iframe>

  <br />
  Let's talk through the logic at each state change:

  - When _S_ is set to 1, output _Q_ becomes 1 also
    - N1 is receiving 1 from S and 0 from R by way of N2. This is the inversion of
      OR so TF equals F. Thus N1 is outputting 0.
    - Thus N2 is receiving 0 from N1 as its top input and 0 from R as its bottom
      input. It is therefore outputting 1 because with NOR, FF equals T. For this
      reason, _Q_ is 1 because is directly connected to _R_.
  - When _S_ goes to 0, _Q_ remains 1
    - N2 is receiving 0 from N1 as the top input and 0 from R as the bottom input
      hence the overall input is FF which means N2 is outputting T and Q remains 1
    - N1 is outputting 0 because it is receiving 0 as its top input and 1 from its
      bottom input
  - When _R_ is set to 1, the memory bit is cleared and _Q_ becomes 0.
    - N2 is receiving 1 from R as its bottom input and 1 from the output of N1 as
      its top input. Therefore it is outputting TT which in NOR evaluates to F
      hence Q is 0
    - N1 is outputting 1 because it is receiving 0 from S as its top input and 0
      from its bottom input coming from N2. FF equals T in NOR therefore 1 is
      outputting 1
  - _Q_ remains at 0 even if _R_ goes back to 0
    - N2 is receiving 0 as its bottom input from R and 1 as its top input from N1.
      TF equals F in NOR hence the output of N2 is 0 and Q remains 0
    - N1 is outputting 1 because it is receiving 0 as its top input from S and 0
      as its bottom input from N2. FF equals T in NOR hence N1 is 1


path: Law_of_Non_Contradiction.md
sortable_path: Law_of_Non_Contradiction.md
title: Law of Non-Contradiction
lead: > A proposition cannot be true and false at the same time.
body:
  > A proposition cannot be true and false at the same time.

  $$
  \lnot (P \land \lnot P)
  $$

checksum: ddf0f94542234a0692e65d40c5bc7349d5e4783a9033ede76e874c531e7e2c0a
created: 2025-10-01 13:30:59.438815177+00:00
modified: 2025-08-13 15:57:13.935510996+00:00
metadata: {"tags":["propositional-logic","logic"]}
raw_content:
  ---
  tags:
    - propositional-logic
    - logic
  ---

  # Law of Non-Contradiction

  > A proposition cannot be true and false at the same time.

  $$
  \lnot (P \land  \lnot P)
  $$


path: Law_of_the_Excluded_Middle.md
sortable_path: Law_of_the_Excluded_Middle.md
title: Law of the Excluded Middle
lead:
  > Every proposition has to be either true or false. There can be no middle
  > ground.

checksum: a53ec515846745f28a9a89390a50710c2ca90f7f1a1180acfa550231761cc6b5
created: 2025-10-01 13:30:59.442474299+00:00
modified: 2025-08-13 15:57:14.000469326+00:00
metadata: {"tags":["propositional-logic","logic"]}
body:
  > Every proposition has to be either true or false. There can be no middle
  > ground.

  $$
  P \lor \sim P
  $$

raw_content:
  ---
  tags:
    - propositional-logic
    - logic
  ---

  # Law of the Excluded Middle

  > Every proposition has to be either true or false. There can be no middle
  > ground.

  $$
  P \lor \sim P
  $$


path: Let's_Encrypt.md
sortable_path: Let's_Encrypt.md
title: Let's Encrypt
checksum: 6dcd26eac7dd2f775254559524dd0f7d528405b8ebc1f87a8db080c3f34905ed
created: 2025-10-01 13:30:59.445568656+00:00
modified: 2025-08-13 15:57:13.905451702+00:00
metadata: {"created":"Sunday, February 09, 2025","tags":["networks","servers","encryption"]}
lead:
  ACME (Automatic Certificate Management Environment) is an open source protocol
  that automates the process of obtaining and renewing SSL/TLS certificates.

body:
  ACME (Automatic Certificate Management Environment) is an open source protocol
  that automates the process of obtaining and renewing SSL/TLS certificates.

  Let's Encrypt (LE) created this protocol and uses it to issue certificates.

  Certbot is a popular ACME client that communicates with the LE servers to

  - validate domain ownership
  - request certificates
  - install the certificates on the web server
  - handle automatic renewals

  Let's Encrypt was founded by the
  [Internet Security Research Group](https://www.abetterinternet.org/). Certbot
  was created independently by the EFF and is maintained by them.

raw_content:
  ---
  tags: [networks, servers, encryption]
  created: Sunday, February 09, 2025
  ---

  # Let's Encrypt

  ACME (Automatic Certificate Management Environment) is an open source protocol
  that automates the process of obtaining and renewing SSL/TLS certificates.

  Let's Encrypt (LE) created this protocol and uses it to issue certificates.

  Certbot is a popular ACME client that communicates with the LE servers to

  - validate domain ownership
  - request certificates
  - install the certificates on the web server
  - handle automatic renewals

  Let's Encrypt was founded by the
  [Internet Security Research Group](https://www.abetterinternet.org/). Certbot
  was created independently by the EFF and is maintained by them.


path: Lifecycle_methods.md
sortable_path: Lifecycle_methods.md
title: Lifecycle methods
checksum: f28d1ec8c52599af3685974a5c17a22f2d0042b296ee9ba49ed379592fa024b7
created: 2025-10-01 13:30:59.450404115+00:00
modified: 2025-08-13 15:57:13.936104028+00:00
metadata: {"tags":["javascript","react"]}
lead:
  Lifecycles are familiar from nature: something is born, it grows and reaches
  maturity and then dies and the process continues as what dies is broken down
  into constituent parts which form the basis of new life. And so on and so on.

body:
  Lifecycles are familiar from nature: something is born, it grows and reaches
  maturity and then dies and the process continues as what dies is broken down
  into constituent parts which form the basis of new life. And so on and so on.

  The same metaphor can be applied to React components relative to the document
  object model:

  - Components are 'born' (created and mounted on the DOM)
  - They 'grow' by changing and updating their state
  - They 'die' by being unmounted from the DOM

  ## Specific lifecycle phases

  The outline above corresponds to specific methods that execute when React code
  runs:

  1. `componentWillMount()` : this method is called just before the `render()`
     method is called
  2. `componentDidMount()` : this method is called after the component is rendered
     in the DOM

  The two methods above only occur once in a lifecycle.

  The 'updating' phase is achieved with the following methods. This is when the
  data of the component, in the form of state and props updates in response to
  user interaction. We call this process 're-rendering':

  3. `shouldComponentUpdate()` : determines whether component should update

  4. `componentWillUpdate()` : called before re-rerendering occurs

  5. `componentDidUpdate()` : called after re-rendering occurs

  The final phase is unmounting: when the component is removed from the DOM:

  6. `componentWillUnmount()`

  ![React component lifecycle](react-lifecycle.png)

  ## Side-effects: why lifecycle phases matter

  > React is built based on the principles of functional programming. In this
  > paradigm, a side-effect (something that is generally discouraged) is any
  > process that is not the direct result of a function executing or anything that
  > is not covered by the `return` keyword.

  Many of the methods detailed in the previous section will rarely be used in
  React development since they pertain to the inner engineering of the library,
  rather than its application in custom contexts. However sometimes you do need
  access to these methods, depending on the background context of what you are
  trying to achieving. This context could include external API and database
  requests or **anything that does not directly pertain to the rendering of the
  component on the screen**.

  React groups these processes under the heading of **side-effects.**

  ### Examples of side-effects and where in the lifecycle they should run

  - `componentWillMount` : It's a good idea to run set-up scripts at this point,
    such as connecting to and authenticating access to a database. This process
    runs before the `render` method is called so it is an optimal point to
    establish these connections
    - You should not however make data requests at this point since data is
      typically returned asynchronously. You cannot therefore be sure that the
      data will be returned before mounting occurs.
  - `componentDidMount` : This is the time to do things that can't occur without
    the DOM:
    - retrieve data
    - add event listeners
    - set timers using `setTimeout` or `setInterval`
  - Updating methods (`shouldComponentUpdate`, `componentWillUpdate` etc.)
    - As these precede or take place concurrently with the rerendering process,
      this phase is a good point to retrieve new data or run necessary external
      processes that are required by the re-rendering.

raw_content:
  ---
  tags:
    - javascript
    - react
  ---

  # Lifecycle methods

  Lifecycles are familiar from nature: something is born, it grows and reaches
  maturity and then dies and the process continues as what dies is broken down
  into constituent parts which form the basis of new life. And so on and so on.

  The same metaphor can be applied to React components relative to the document
  object model:

  - Components are 'born' (created and mounted on the DOM)
  - They 'grow' by changing and updating their state
  - They 'die' by being unmounted from the DOM

  ## Specific lifecycle phases

  The outline above corresponds to specific methods that execute when React code
  runs:

  1. `componentWillMount()` : this method is called just before the `render()`
     method is called
  2. `componentDidMount()` : this method is called after the component is rendered
     in the DOM

  The two methods above only occur once in a lifecycle.

  The 'updating' phase is achieved with the following methods. This is when the
  data of the component, in the form of state and props updates in response to
  user interaction. We call this process 're-rendering':

  3. `shouldComponentUpdate()` : determines whether component should update

  4. `componentWillUpdate()` : called before re-rerendering occurs

  5. `componentDidUpdate()` : called after re-rendering occurs

  The final phase is unmounting: when the component is removed from the DOM:

  6. `componentWillUnmount()`

  ![React component lifecycle](react-lifecycle.png)

  ## Side-effects: why lifecycle phases matter

  > React is built based on the principles of functional programming. In this
  > paradigm, a side-effect (something that is generally discouraged) is any
  > process that is not the direct result of a function executing or anything that
  > is not covered by the `return` keyword.

  Many of the methods detailed in the previous section will rarely be used in
  React development since they pertain to the inner engineering of the library,
  rather than its application in custom contexts. However sometimes you do need
  access to these methods, depending on the background context of what you are
  trying to achieving. This context could include external API and database
  requests or **anything that does not directly pertain to the rendering of the
  component on the screen**.

  React groups these processes under the heading of **side-effects.**

  ### Examples of side-effects and where in the lifecycle they should run

  - `componentWillMount` : It's a good idea to run set-up scripts at this point,
    such as connecting to and authenticating access to a database. This process
    runs before the `render` method is called so it is an optimal point to
    establish these connections
    - You should not however make data requests at this point since data is
      typically returned asynchronously. You cannot therefore be sure that the
      data will be returned before mounting occurs.
  - `componentDidMount` : This is the time to do things that can't occur without
    the DOM:
    - retrieve data
    - add event listeners
    - set timers using `setTimeout` or `setInterval`
  - Updating methods (`shouldComponentUpdate`, `componentWillUpdate` etc.)
    - As these precede or take place concurrently with the rerendering process,
      this phase is a good point to retrieve new data or run necessary external
      processes that are required by the re-rendering.


path: LineageOS_backup.md
sortable_path: LineageOS_backup.md
title: LineageOS major update procedure
lead: ## Backup
checksum: 8c8e95de8036787cb60edf83fb490b1b163818e6da4266b26ba8e7300305dcea
created: 2025-10-01 13:30:59.454433679+00:00
modified: 2025-08-13 15:57:13.880134916+00:00
metadata: {"created":"Friday, January 24, 2025","tags":["procedural","linux","android"]}
body:
  ## Backup

  - Enable developer mode by going to Settings > About Phone and tapping "Build
    Number" seven times

  - Enable USB debugging in Developer Options

  - Test by running `adb devices` in PC terminal

  - Then run `adb backup -apk -shared -all -system -f lineage_backup.ab`

  ## Download latest version of TWRP

  Make sure you get the right one for Google Pixel 2 (taimen), e.g:

  ```
  twrp-3.7.0_9-0-taimen.img
  ```

  ## Download ROM image and associated files

  Go to
  [https://wiki.lineageos.org/devices/taimen/](https://wiki.lineageos.org/devices/taimen/),
  download zip, viz.

  ```
  lineage-22.1-20250123-nightly-taimen-signed.zip
  ```

  ## Boot into fastboot

  Press Power + Vol Down

  Check connection with:

  ```
  fastboot devices
  ```

  ## Boot into TWRP

  ```
  fastboot boot  twrp-3.7.0_9-0-taimen.img
  ```

  ## Copy OS rom to Phone

  ```
  adb push lineage-22*.zip /sdcard/
  ```

  ## Install OS in TWRP

  In TWRP select Install > Select ROM zip

  Select flash

  Then reboot

  ```

  ```

raw_content:
  ---
  tags: [procedural, linux, android]
  created: Friday, January 24, 2025
  ---

  # LineageOS major update procedure

  ## Backup

  - Enable developer mode by going to Settings > About Phone and tapping "Build
    Number" seven times

  - Enable USB debugging in Developer Options

  - Test by running `adb devices` in PC terminal

  - Then run `adb backup -apk -shared -all -system -f lineage_backup.ab`

  ## Download latest version of TWRP

  Make sure you get the right one for Google Pixel 2 (taimen), e.g:

  ```
  twrp-3.7.0_9-0-taimen.img
  ```

  ## Download ROM image and associated files

  Go to
  [https://wiki.lineageos.org/devices/taimen/](https://wiki.lineageos.org/devices/taimen/),
  download zip, viz.

  ```
  lineage-22.1-20250123-nightly-taimen-signed.zip
  ```

  ## Boot into fastboot

  Press Power + Vol Down

  Check connection with:

  ```
  fastboot devices
  ```

  ## Boot into TWRP

  ```
  fastboot boot  twrp-3.7.0_9-0-taimen.img
  ```

  ## Copy OS rom to Phone

  ```
  adb push lineage-22*.zip /sdcard/
  ```

  ## Install OS in TWRP

  In TWRP select Install > Select ROM zip

  Select flash

  Then reboot

  ```

  ```


path: Link_Layer_of_Internet_Protocol.md
sortable_path: Link_Layer_of_Internet_Protocol.md
title: Link Layer of the Internet Protocol Suite
checksum: aa296b50c848330270028a57ee3f3d5ebe1b7a401db10a81319b40e651035091
created: 2025-10-01 13:30:59.458655441+00:00
modified: 2025-08-13 15:57:14.047826587+00:00
metadata: {"created":"Tuesday, August 06, 2024","tags":["networks","internet"]}
lead:
  > The Link Layer is the lowest level of the network stack that makes up the
  > Internet Protocol Suite. It is concerned with the physical and logical
  > connections between [hosts](./Network_hosts.md) on the same local network.

body:
  > The Link Layer is the lowest level of the network stack that makes up the
  > Internet Protocol Suite. It is concerned with the physical and logical
  > connections between [hosts](./Network_hosts.md) on the same local network.

  The physical and logical connections are known as **Links**.

  Link Layer protocols goven communication between devices. Examples include:

  - **WiFi**
    - For wireless communication between devices using radio waves
  - **Ethernet**
    - Wired communication

  All devices connected to the internet participate in the Link Layer since it is
  the base level and the gateway out of the local network to other networks. The
  same is not true for other layers. For example a device like a laptop
  participates in all four layers up to the Application Layer but a network router
  does not:

  ![Link Layer diagram](../img/link-layer-frame-revised.png)

  ## MAC addresses

  Each device on a link has a network address that uniquely identifies it. For
  most Link Layer Protocols this is a
  [Media Access Control (MAC) address](MAC_addresses.md).

  A device receives data addressed to its MAC address.

  MAC addresses are limited to local networks. Remote networks cannot send data to
  a specific MAC address. This is off-limits from the outside.

  ## Frames

  Link Layer data is divided into small units called "frames". The anatomy of a
  frame is as follows:

  ![Link Layer frame](../img/link-layer-frame-revised.png)

  The header contains the source and destination MAC address plus a descriptor of
  the type of data it contains. The data is the payload. The footer is used to
  detect errors.

raw_content:
  ---
  tags: [networks, internet]
  created: Tuesday, August 06, 2024
  ---

  # Link Layer of the Internet Protocol Suite

  > The Link Layer is the lowest level of the network stack that makes up the
  > Internet Protocol Suite. It is concerned with the physical and logical
  > connections between [hosts](./Network_hosts.md) on the same local network.

  The physical and logical connections are known as **Links**.

  Link Layer protocols goven communication between devices. Examples include:

  - **WiFi**
    - For wireless communication between devices using radio waves
  - **Ethernet**
    - Wired communication

  All devices connected to the internet participate in the Link Layer since it is
  the base level and the gateway out of the local network to other networks. The
  same is not true for other layers. For example a device like a laptop
  participates in all four layers up to the Application Layer but a network router
  does not:

  ![Link Layer diagram](../img/link-layer-frame-revised.png)

  ## MAC addresses

  Each device on a link has a network address that uniquely identifies it. For
  most Link Layer Protocols this is a
  [Media Access Control (MAC) address](MAC_addresses.md).

  A device receives data addressed to its MAC address.

  MAC addresses are limited to local networks. Remote networks cannot send data to
  a specific MAC address. This is off-limits from the outside.

  ## Frames

  Link Layer data is divided into small units called "frames". The anatomy of a
  frame is as follows:

  ![Link Layer frame](../img/link-layer-frame-revised.png)

  The header contains the source and destination MAC address plus a descriptor of
  the type of data it contains. The data is the payload. The footer is used to
  detect errors.


path: Linux_disk_partitions.md
sortable_path: Linux_disk_partitions.md
title: Linux disk partitions
checksum: 73864c1fb6e564152cb3aaf302a8247e12a520c1a1e1d755db9d25d5611d14fa
created: 2025-10-01 13:30:59.472673954+00:00
modified: 2025-08-13 15:57:13.760030186+00:00
metadata: {"tags":["disks","Linux","procedural"]}
lead:
  A disk is divided up into partitions which are subsections of the overall disk.
  The kernel presents each partition as a [block device](Devices.md#Devices) as it
  would with an entire disk.

body:
  A disk is divided up into partitions which are subsections of the overall disk.
  The kernel presents each partition as a [block device](Devices.md#Devices) as it
  would with an entire disk.

  The disk dedicates a small part of its contents to a **partition table**: this
  defines the different partitions that comprise the total disk space.

  ## Viewing current partitions

  Whenever you install a Linux distribution on a real or virtual machine, you must
  partition the drive. There are three main tools to choose from: `parted`,
  `g(raphical)parted`, `fdisk`.

  For a top-level overview of your disks and their main partitions you can run
  `lsblk` (_list block devices_):

  ```bash
  $ lsblk

  NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
  sda           8:0    0 465.7G  0 disk
  ├─sda1        8:1    0   200M  0 part
  └─sda2        8:2    0 465.4G  0 part /run/media/thomas/ganesh
  nvme0n1     259:0    0 476.9G  0 disk
  ├─nvme0n1p1 259:1    0   488M  0 part
  ├─nvme0n1p2 259:2    0  27.9G  0 part /
  └─nvme0n1p3 259:3    0 448.5G  0 part /home
  ```

  We can use `parted -l` to view the partition table for the current machine:

  ```bash
  Model: SKHynix_HFS512GDE9X081N (nvme)
  Disk /dev/nvme0n1: 512GB
  Sector size (logical/physical): 512B/512B
  Partition Table: gpt
  Disk Flags:

  Number  Start   End     Size    File system  Name  Flags
   1      1049kB  513MB   512MB   fat32              boot, esp
   2      513MB   30.5GB  30.0GB  ext4
   3      30.5GB  512GB   482GB   ext4
  ```

  We can use `fdisk -l` to get slightly more info:

  ```bash
  disk /dev/nvme0n1: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
  Disk model: SKHynix_HFS512GDE9X081N
  Units: sectors of 1 * 512 = 512 bytes
  Sector size (logical/physical): 512 bytes / 512 bytes
  I/O size (minimum/optimal): 512 bytes / 512 bytes
  Disklabel type: gpt
  Disk identifier: 08175E77-CB9F-C34A-9032-DF29A3F8F0FE

  Device            Start        End   Sectors   Size Type
  /dev/nvme0n1p1     2048    1001471    999424   488M EFI System
  /dev/nvme0n1p2  1001472   59594751  58593280  27.9G Linux filesystem
  /dev/nvme0n1p3 59594752 1000214527 940619776 448.5G Linux filesystem
  ```

  ## Standard Linux partitions

  The two tools disclose that the main harddrive is `/dev/nvme0n1` (equivalent to
  `sda` on older machines running Linux) and it has the standard three partitions:

  - Boot partition (`/dev/nvme0n1p1`)
  - Root dir (`/dev/nvme0n1p2`)
  - Home dir (`/dev/nvme0n1p3`)

  ### Boot partition

  - Takes up the smallest amount of space
  - Contains the Linux kernel and the bootloader (GRUB, usually) and any other
    files required for booting.
  - Once the BIOS has initialized the hardware components it hands control to the
    bootloader stored in this partition.
  - The bootloader loads the kernel contained in its partition, into memory

  ### Root partition (`/`)

  - This houses everything to do with the operation of the operating system apart
    from the initial booting process. Once the kernel is in memory it accesses the
    resources in the root partition to get the OS up and running. These resources
    include system libraries, utilities, system-wide configs and application
    software.

  - The root partition is vital for the system's functionality. Without it, the
    system cannot boot or operate. It needs to be  
     large enough to accommodate the operating system, installed programs, and system
    libraries, but it doesn't need to store user data or  
     additional applications, which can be located on other partitions.

  - This is the domain of the [superuser](User_Space.md#root-user-superuser). The
    part of the filesystem that you need sudo priveleges to access and where you
    manage users

  ### Home partition

  - This is presented as `/home/`, i.e. that it is a descendent of `/`. But this
    is because the Linux file system is hierarchical. Typically `/home/` is
    usually its own partition.
  - Separating user data and system data in this way protects user data during
    upgrades or reinstallation. Reinstalling or upgrading the OS can be done
    without affecting these files.
  - Each user has their own directory off of `/home`, e.g `/home/thomas`.
  - While the root partition (`/`) is essential for the operation of the OS,
    placing `/home` on a separate partition allows for better management of user
    data, enhancing the system's flexibility and the user's control over their own
    files, without compromising the integrity or performance of the OS. This
    approach exemplifies the Linux philosophy of giving the user control while
    maintaining a clear distinction between system  
    resources and user data.

  ## Types of partition table

  In the Linux world there are two main types: MBR and GPT. The type of table used
  determines how the OS boots. So although partition tables are also responsible
  for the partitioning of non-bootable sectors of a disk, **they are distinguished
  by the boot system they implement**. If we look at the output from `parted` and
  `fdisk` above we see that the harddrive uses the GPT partition type.

  #### Primary, extended and logical partitions

  Most standard partition tables allow for primary, extended and logical
  partitions. The primary partition is the part of the harddisk that contains the
  operating system and is thus described as 'bootable' and may be called the 'boot
  partition'. During the bootstrapping process this is injected into memory as the
  [kernel](The_kernel.md).

  The extended partition is basically everything other than the primary partition.
  This is typically subdivided into other partitions that are called _logical_
  partitions. This is because they physically reside in the same sector of the
  disk (the extended partition) but are treated as virtual and independent disks.

  In our example above:

  - `/dev/nvme0n1p1` is the primary/boot partition
  - `/dev/nvme0n1p2` and `/dev/nvme0n1p3` comprise the extended partition and by
    themselves are each logical partitions.

  <dl>
    <dt>MBR</dt>
    <dd> 
      <ul>
        <li>Stands for Master Boot Record</li> 
        <li>Uses BIOS in the boot process</li> 
        <li>Can only works with disks up to 2TB in size</li> 
        <li>Only supports 4 primary partitions. This means the number of operating systems you install is limitied to this number.</li> 
        <li>This is the first 512 bytes of a storage device, preceding the first partition.</li>
      </ul> 
    </dd>
    <dt>GPT</dt>
    <dd>
    <ul>
        <li>Stands for GUID Partition Table </li>
        <li>Gradually replacing MBR</li>
        <li>Uses UEFI instead of BIOS</li>
        <li>As name indicates, every partition on disk has its own globally-unique identifier</li>
        <li>Vastly more partitions available than with MBR (dependent on operating system)</li>
        <li>Offers greater recovery options and anti-corruption safeguards</li> 
      </ul>
  </dd>
  </dl>

  ## Related notes

  ![Creating a Linux partition table](./Creating_a_Linux_partition_table.md)

raw_content:
  ---
  tags:
    - disks
    - Linux
    - procedural
  ---

  # Linux disk partitions

  A disk is divided up into partitions which are subsections of the overall disk.
  The kernel presents each partition as a [block device](Devices.md#Devices) as it
  would with an entire disk.

  The disk dedicates a small part of its contents to a **partition table**: this
  defines the different partitions that comprise the total disk space.

  ## Viewing current partitions

  Whenever you install a Linux distribution on a real or virtual machine, you must
  partition the drive. There are three main tools to choose from: `parted`,
  `g(raphical)parted`, `fdisk`.

  For a top-level overview of your disks and their main partitions you can run
  `lsblk` (_list block devices_):

  ```bash
  $ lsblk

  NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
  sda           8:0    0 465.7G  0 disk
  ├─sda1        8:1    0   200M  0 part
  └─sda2        8:2    0 465.4G  0 part /run/media/thomas/ganesh
  nvme0n1     259:0    0 476.9G  0 disk
  ├─nvme0n1p1 259:1    0   488M  0 part
  ├─nvme0n1p2 259:2    0  27.9G  0 part /
  └─nvme0n1p3 259:3    0 448.5G  0 part /home
  ```

  We can use `parted -l` to view the partition table for the current machine:

  ```bash
  Model: SKHynix_HFS512GDE9X081N (nvme)
  Disk /dev/nvme0n1: 512GB
  Sector size (logical/physical): 512B/512B
  Partition Table: gpt
  Disk Flags:

  Number  Start   End     Size    File system  Name  Flags
   1      1049kB  513MB   512MB   fat32              boot, esp
   2      513MB   30.5GB  30.0GB  ext4
   3      30.5GB  512GB   482GB   ext4
  ```

  We can use `fdisk -l` to get slightly more info:

  ```bash
  disk /dev/nvme0n1: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
  Disk model: SKHynix_HFS512GDE9X081N
  Units: sectors of 1 * 512 = 512 bytes
  Sector size (logical/physical): 512 bytes / 512 bytes
  I/O size (minimum/optimal): 512 bytes / 512 bytes
  Disklabel type: gpt
  Disk identifier: 08175E77-CB9F-C34A-9032-DF29A3F8F0FE

  Device            Start        End   Sectors   Size Type
  /dev/nvme0n1p1     2048    1001471    999424   488M EFI System
  /dev/nvme0n1p2  1001472   59594751  58593280  27.9G Linux filesystem
  /dev/nvme0n1p3 59594752 1000214527 940619776 448.5G Linux filesystem
  ```

  ## Standard Linux partitions

  The two tools disclose that the main harddrive is `/dev/nvme0n1` (equivalent to
  `sda` on older machines running Linux) and it has the standard three partitions:

  - Boot partition (`/dev/nvme0n1p1`)
  - Root dir (`/dev/nvme0n1p2`)
  - Home dir (`/dev/nvme0n1p3`)

  ### Boot partition

  - Takes up the smallest amount of space
  - Contains the Linux kernel and the bootloader (GRUB, usually) and any other
    files required for booting.
  - Once the BIOS has initialized the hardware components it hands control to the
    bootloader stored in this partition.
  - The bootloader loads the kernel contained in its partition, into memory

  ### Root partition (`/`)

  - This houses everything to do with the operation of the operating system apart
    from the initial booting process. Once the kernel is in memory it accesses the
    resources in the root partition to get the OS up and running. These resources
    include system libraries, utilities, system-wide configs and application
    software.

  - The root partition is vital for the system's functionality. Without it, the
    system cannot boot or operate. It needs to be  
     large enough to accommodate the operating system, installed programs, and system
    libraries, but it doesn't need to store user data or  
     additional applications, which can be located on other partitions.

  - This is the domain of the [superuser](User_Space.md#root-user-superuser). The
    part of the filesystem that you need sudo priveleges to access and where you
    manage users

  ### Home partition

  - This is presented as `/home/`, i.e. that it is a descendent of `/`. But this
    is because the Linux file system is hierarchical. Typically `/home/` is
    usually its own partition.
  - Separating user data and system data in this way protects user data during
    upgrades or reinstallation. Reinstalling or upgrading the OS can be done
    without affecting these files.
  - Each user has their own directory off of `/home`, e.g `/home/thomas`.
  - While the root partition (`/`) is essential for the operation of the OS,
    placing `/home` on a separate partition allows for better management of user
    data, enhancing the system's flexibility and the user's control over their own
    files, without compromising the integrity or performance of the OS. This
    approach exemplifies the Linux philosophy of giving the user control while
    maintaining a clear distinction between system  
    resources and user data.

  ## Types of partition table

  In the Linux world there are two main types: MBR and GPT. The type of table used
  determines how the OS boots. So although partition tables are also responsible
  for the partitioning of non-bootable sectors of a disk, **they are distinguished
  by the boot system they implement**. If we look at the output from `parted` and
  `fdisk` above we see that the harddrive uses the GPT partition type.

  #### Primary, extended and logical partitions

  Most standard partition tables allow for primary, extended and logical
  partitions. The primary partition is the part of the harddisk that contains the
  operating system and is thus described as 'bootable' and may be called the 'boot
  partition'. During the bootstrapping process this is injected into memory as the
  [kernel](The_kernel.md).

  The extended partition is basically everything other than the primary partition.
  This is typically subdivided into other partitions that are called _logical_
  partitions. This is because they physically reside in the same sector of the
  disk (the extended partition) but are treated as virtual and independent disks.

  In our example above:

  - `/dev/nvme0n1p1` is the primary/boot partition
  - `/dev/nvme0n1p2` and `/dev/nvme0n1p3` comprise the extended partition and by
    themselves are each logical partitions.

  <dl>
    <dt>MBR</dt>
    <dd> 
      <ul>
        <li>Stands for Master Boot Record</li> 
        <li>Uses BIOS in the boot process</li> 
        <li>Can only works with disks up to 2TB in size</li> 
        <li>Only supports 4 primary partitions. This means the number of operating systems you install is limitied to this number.</li> 
        <li>This is the first 512 bytes of a storage device, preceding the first partition.</li>
      </ul> 
    </dd>
    <dt>GPT</dt>
    <dd>
    <ul>
        <li>Stands for GUID Partition Table </li>
        <li>Gradually replacing MBR</li>
        <li>Uses UEFI instead of BIOS</li>
        <li>As name indicates, every partition on disk has its own globally-unique identifier</li>
        <li>Vastly more partitions available than with MBR (dependent on operating system)</li>
        <li>Offers greater recovery options and anti-corruption safeguards</li> 
      </ul>
  </dd>
  </dl>

  ## Related notes

  ![Creating a Linux partition table](./Creating_a_Linux_partition_table.md)


path: List_comprehension_in_Python.md
sortable_path: List_comprehension_in_Python.md
title: List comprehension
checksum: 47f1ec37b472ca033ebbdfae22ba2e8c7ac0b0943d70dcb158fa782b62e3f28f
created: 2025-10-01 13:30:59.488897656+00:00
modified: 2025-08-13 15:57:13.979098591+00:00
metadata: {"tags":["python"]}
lead:
  List comprehension is a syntactic shorthand for applying a function to each
  element in a list without explicitly using loop syntax.

body:
  List comprehension is a syntactic shorthand for applying a function to each
  element in a list without explicitly using loop syntax.

  Since its introduction to the language, the same functionality has become
  achievable by using functional methods like
  [`map` and `filter`](Map_and_filter_in_Python.md),
  utilising [lambdas](Lambdas_in_Python.md)
  however list comprehension is often more straightforward and easier to read.

  ## Syntax

  Here is a basic example which applies `+ 1` to each integer in a list:

  ```python
  values = [1, 2, 4, 6, 8, 9]

  new_values = [i + 1 for i in values]
  print('new_values', new_values)
  # new_values [2, 3, 5, 7, 9, 10]
  ```

  The basic syntax is as follows:

  ```
  new_list  = [expression for each member in an iterable]
  ```

  - The _expression_ can be the member itself, a call to a method, or any other
    valid expression that returns a value. In the example above, the expression
    `i + i` adds one to each member value.

  - The _member_ is the object or value in the list or iterable. In the example
    above, the member value is _i_.

  - The _iterable_ is a list, set, dictionary or any other object that can return
    its elements one at a time. In the example above, the iterable is each value
    in `values`.

  This is a much more condensed way of achieving the same outcome with a
  traditional loop:

  ```py
  values = [1, 2, 4, 6, 8, 9]
  new_list = []
  for i in values:
    values.append(i+1)
  ```

  ### Another example

  In the following example, we apply list comprehension with a `in range` loop
  structure:

  ```py
  new_list = [i * i for i in range(10) ]
  print(new_list)
  # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

  ```

  ### Adding a condition

  We can apply a conditional to a comprehension:

  ```py
  new_list = [item + 1 for item in values if item % 2 == 0]

  print('new_list:', new_list)
  # new_list: [3, 5, 7, 9]
  ```

  ### Filters

  By applying a condition (and no execution to each element) we effectively create
  a filter:

  ```py
  numbers = [1, 2, 3, 4, 4, 4]
  numbers_filtered = [i for i in numbers if i > 2]
  print(numbers_filtered)
  # [3, 4, 5]
  ```

  For comparison, the same outcome could be achieved with a filter and lambda:

  ```py
  with_filter = list(filter(lambda x: x > 2, numbers))
  print(with_filter)
  # [3, 4, 5]
  ```

  ## Set comprehension

  We can also apply comprehension to sets. The syntax is practically identical but
  the difference is the resultant data structure will not contain duplicates.

  ```py
  numbers = [1, 2, 3, 4, 4, 4]
  unique =  {i for i in numbers}
  print(unique)
  # {1,2,3,4}
  ```

  ## Dictionary comprehension

  ```py
  squares = {i: i * i for i in range(5)}
  print(squares)
  {0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25}
  ```

raw_content:
  ---
  tags: [python]
  ---

  # List comprehension

  List comprehension is a syntactic shorthand for applying a function to each
  element in a list without explicitly using loop syntax.

  Since its introduction to the language, the same functionality has become
  achievable by using functional methods like
  [`map` and `filter`](Map_and_filter_in_Python.md),
  utilising [lambdas](Lambdas_in_Python.md)
  however list comprehension is often more straightforward and easier to read.

  ## Syntax

  Here is a basic example which applies `+ 1` to each integer in a list:

  ```python
  values = [1, 2, 4, 6, 8, 9]

  new_values = [i + 1 for i in values]
  print('new_values', new_values)
  # new_values [2, 3, 5, 7, 9, 10]
  ```

  The basic syntax is as follows:

  ```
  new_list  = [expression for each member in an iterable]
  ```

  - The _expression_ can be the member itself, a call to a method, or any other
    valid expression that returns a value. In the example above, the expression
    `i + i` adds one to each member value.

  - The _member_ is the object or value in the list or iterable. In the example
    above, the member value is _i_.

  - The _iterable_ is a list, set, dictionary or any other object that can return
    its elements one at a time. In the example above, the iterable is each value
    in `values`.

  This is a much more condensed way of achieving the same outcome with a
  traditional loop:

  ```py
  values = [1, 2, 4, 6, 8, 9]
  new_list = []
  for i in values:
    values.append(i+1)
  ```

  ### Another example

  In the following example, we apply list comprehension with a `in range` loop
  structure:

  ```py
  new_list = [i * i for i in range(10) ]
  print(new_list)
  # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

  ```

  ### Adding a condition

  We can apply a conditional to a comprehension:

  ```py
  new_list = [item + 1 for item in values if item % 2 == 0]

  print('new_list:', new_list)
  # new_list: [3, 5, 7, 9]
  ```

  ### Filters

  By applying a condition (and no execution to each element) we effectively create
  a filter:

  ```py
  numbers = [1, 2, 3, 4, 4, 4]
  numbers_filtered = [i for i in numbers if i > 2]
  print(numbers_filtered)
  # [3, 4, 5]
  ```

  For comparison, the same outcome could be achieved with a filter and lambda:

  ```py
  with_filter = list(filter(lambda x: x > 2, numbers))
  print(with_filter)
  # [3, 4, 5]
  ```

  ## Set comprehension

  We can also apply comprehension to sets. The syntax is practically identical but
  the difference is the resultant data structure will not contain duplicates.

  ```py
  numbers = [1, 2, 3, 4, 4, 4]
  unique =  {i for i in numbers}
  print(unique)
  # {1,2,3,4}
  ```

  ## Dictionary comprehension

  ```py
  squares = {i: i * i for i in range(5)}
  print(squares)
  {0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25}
  ```


path: List_largest_files_bash.md
sortable_path: List_largest_files_bash.md
title: List_largest_files_bash
lead:
  ```sh
  ls -lSh
  ```

body:
  ```sh
  ls -lSh
  ```

  ```sh
  # Limited to n largest

  ls -lSh | tail -10

  ```

checksum: d21241f6fbf82c7446431c475c64f3e49d764c482e140e878fd4e620dc507548
created: 2025-10-01 13:30:59.502085214+00:00
modified: 2025-08-13 15:57:13.965368276+00:00
metadata: {"created":"Monday, July 07, 2025","tags":["shell"]}
raw_content:
  ---
  tags: [shell]
  created: Monday, July 07, 2025
  ---

  # List_largest_files_bash

  ```sh
  ls -lSh
  ```

  ```sh
  # Limited to n largest

  ls -lSh | tail -10

  ```


path: Listing_and_navigating_directories_in_the_Shell.md
sortable_path: Listing_and_navigating_directories_in_the_Shell.md
title: Listing and navigating directories in the shell
lead: ## Listing options
checksum: 66c71341ea13a02e6448279e89744a8559b1000947bbab0dc5bf8de95dcd77e4
created: 2025-10-01 13:30:59.50588724+00:00
modified: 2025-08-13 15:57:13.952764593+00:00
metadata: {"tags":["shell"]}
body:
  ## Listing options

  Obviously we know that in order to list the files and sub-directories in our
  current directory we use `ls` but here are some of the most useful of the
  different modifiers:

  - `**ls -a**`
    - list and include hidden dot files
  - `**ls -l**`
    - list with user permissions, file-size and date-modified (most detailed)
  - `**ls ***`
    - list organised by folders one level deep

  ## Navigation shorthands

  - `cd -`
    - Return to the directory you were last in
  - `!!`
    - Repeat the last command

raw_content:
  ---
  tags:
    - shell
  ---

  # Listing and navigating directories in the shell

  ## Listing options

  Obviously we know that in order to list the files and sub-directories in our
  current directory we use `ls` but here are some of the most useful of the
  different modifiers:

  - `**ls -a**`
    - list and include hidden dot files
  - `**ls -l**`
    - list with user permissions, file-size and date-modified (most detailed)
  - `**ls ***`
    - list organised by folders one level deep

  ## Navigation shorthands

  - `cd -`
    - Return to the directory you were last in
  - `!!`
    - Repeat the last command


path: Lists_and_arrays_in_Bash.md
sortable_path: Lists_and_arrays_in_Bash.md
title: Arrays in Bash
lead: We define an array as follows:
checksum: 69b5c6bbf1a810f3021b6854ef962a87e0212935962e8063311cdb7e11703947
created: 2025-10-01 13:30:59.509290577+00:00
modified: 2025-08-13 15:57:13.994599173+00:00
metadata: {"tags":["shell","data-structures"]}
body:
  We define an array as follows:

  ```bash
  words=(here are some words)
  ```

  We can also explicitly define an array using `declare` :

  ```bash
  declare -a words=("element1" "element2" "element3")
  ```

  ## Index notation

  We access specific array elements by their index using the same braces style we
  use with variables:

  ```bash
  echo "${words[2]}"

  # element3
  ```

  ## Iterating through arrays

  ```bash
  for i in "${words[@]}"
  do
     echo "$i"
     # or do whatever with individual element of the array
  done
  # element1 element2 element3
  ```

  Note that `@` here is a special symbol standing for all the members of the
  `words` array.

  ## Sorting arrays

  ### Sorting an integer array highests to lowest

  ```bash
  sorted_array=($(echo "${array[@]}" | tr " " "\n" | sort -nr))
  ```

  Where `array` is the name of the original array. The sorted array will be stored
  in the `sorted_array` array.

  The `sort` command sorts the input in reverse numerical order (`-n` for
  numerical sort and `-r` for reverse sort). The `tr` command is used to convert
  the spaces in the array to newline characters so that each element is sorted on
  a separate line.

  ## Pushing, appending to an array

  We use the `+=` shorthand to add elements to a preexisting array:

  ```bash
  preExistingArray=(1 2 3)
  preExistingArray+=(4)
  echo preExistingArray
  # 1 2 3 4

  ```

  > Note that we have to put the item we want to addend within array brackets

  ## Check if array empty

  ```sh
  an_array=()

  if [ -z "${another_array[@]}" ]; then
      echo "Array is empty"
  else
      echo "Array is not empty"
  fi
  ```

  Here we pass all the elements of the array to a
  [test](Test_values_in_Bash.md) condition which
  tests for an empty string.

  > NB: This will not immediately work in the context of a function. See below.

  ## Weirdness with functions

  When you pass an array as an argument to a
  [function](Functions_in_Bash.md) it will not
  immediately be understood to be an array.

  When we use `$1` to individuate the first function argument this is read as
  string. So if you parsed an array argument as `$1`, any logic you have in the
  function will work on the assumption that the argument is a string, not an
  array.

  To get round this we have to effectively _redeclare_ the argument as an array
  before running any array logic. We do this through a **nameref** (a reference to
  a variable). The nameref resolves to the value of the variable it references.
  This allows you to indirectly manipulate the value of the original variable
  through the nameref.

  A nameref is created with the `-n` flag. The following function uses this method
  to check if an array is empty:

  ```sh
  function array_empty() {
      declare -n arr=$1
      if [ ${#arr[@]} -gt 0 ]; then
          echo "array is not empty"
      else
          echo "array is empty"
      fi
  }

  my_array=()
  array_empty "my_array"
  ```

  You'll notice when we invoke the function we pass the array as a string, this
  facilitates the nameref operation in the function.

  ## Associational arrays / maps

  With Bash 4 we gained an additional array-like data structure that is key-value
  based and similar to maps in other languages.

  ```bash
  declare -A rock=(["win"]="scissors" ["lose"]="paper")
  ```

  We would then individuate a value with `"${rock[win]}"`

  ## Lists as implicit string arrays

  When we use the term **list** in bash, we are not actually referring to a
  specific type of data structure. Instead a **list variable** is really just a
  normal variable wrapped in quote marks that has strings separated by spaces.
  Despite the fact that this is not an actual iterative data structure, we are
  still able to loop through variables of this type.

  ```bash
  A_STR_LIST="cat dog hamster"
  AN_INT_LIST="1 2 3"
  ```

  To iterate through a list variable, we can use a for loop:

  ```bash
  for ele in $A_STR_LIST; do
      echo $ele
  done
  ```

  We are leveraging this aspect of Bash when we
  [loop through each character in a string](Strings_in_Bash.md#loop-through-each-character-in-a-string).

raw_content:
  ---
  tags:
    - shell
    - data-structures
  ---

  # Arrays in Bash

  We define an array as follows:

  ```bash
  words=(here are some words)
  ```

  We can also explicitly define an array using `declare` :

  ```bash
  declare -a words=("element1" "element2" "element3")
  ```

  ## Index notation

  We access specific array elements by their index using the same braces style we
  use with variables:

  ```bash
  echo "${words[2]}"

  # element3
  ```

  ## Iterating through arrays

  ```bash
  for i in "${words[@]}"
  do
     echo "$i"
     # or do whatever with individual element of the array
  done
  # element1 element2 element3
  ```

  Note that `@` here is a special symbol standing for all the members of the
  `words` array.

  ## Sorting arrays

  ### Sorting an integer array highests to lowest

  ```bash
  sorted_array=($(echo "${array[@]}" | tr " " "\n" | sort -nr))
  ```

  Where `array` is the name of the original array. The sorted array will be stored
  in the `sorted_array` array.

  The `sort` command sorts the input in reverse numerical order (`-n` for
  numerical sort and `-r` for reverse sort). The `tr` command is used to convert
  the spaces in the array to newline characters so that each element is sorted on
  a separate line.

  ## Pushing, appending to an array

  We use the `+=` shorthand to add elements to a preexisting array:

  ```bash
  preExistingArray=(1 2 3)
  preExistingArray+=(4)
  echo preExistingArray
  # 1 2 3 4

  ```

  > Note that we have to put the item we want to addend within array brackets

  ## Check if array empty

  ```sh
  an_array=()

  if [ -z "${another_array[@]}" ]; then
      echo "Array is empty"
  else
      echo "Array is not empty"
  fi
  ```

  Here we pass all the elements of the array to a
  [test](Test_values_in_Bash.md) condition which
  tests for an empty string.

  > NB: This will not immediately work in the context of a function. See below.

  ## Weirdness with functions

  When you pass an array as an argument to a
  [function](Functions_in_Bash.md) it will not
  immediately be understood to be an array.

  When we use `$1` to individuate the first function argument this is read as
  string. So if you parsed an array argument as `$1`, any logic you have in the
  function will work on the assumption that the argument is a string, not an
  array.

  To get round this we have to effectively _redeclare_ the argument as an array
  before running any array logic. We do this through a **nameref** (a reference to
  a variable). The nameref resolves to the value of the variable it references.
  This allows you to indirectly manipulate the value of the original variable
  through the nameref.

  A nameref is created with the `-n` flag. The following function uses this method
  to check if an array is empty:

  ```sh
  function array_empty() {
      declare -n arr=$1
      if [ ${#arr[@]} -gt 0 ]; then
          echo "array is not empty"
      else
          echo "array is empty"
      fi
  }

  my_array=()
  array_empty "my_array"
  ```

  You'll notice when we invoke the function we pass the array as a string, this
  facilitates the nameref operation in the function.

  ## Associational arrays / maps

  With Bash 4 we gained an additional array-like data structure that is key-value
  based and similar to maps in other languages.

  ```bash
  declare -A rock=(["win"]="scissors" ["lose"]="paper")
  ```

  We would then individuate a value with `"${rock[win]}"`

  ## Lists as implicit string arrays

  When we use the term **list** in bash, we are not actually referring to a
  specific type of data structure. Instead a **list variable** is really just a
  normal variable wrapped in quote marks that has strings separated by spaces.
  Despite the fact that this is not an actual iterative data structure, we are
  still able to loop through variables of this type.

  ```bash
  A_STR_LIST="cat dog hamster"
  AN_INT_LIST="1 2 3"
  ```

  To iterate through a list variable, we can use a for loop:

  ```bash
  for ele in $A_STR_LIST; do
      echo $ele
  done
  ```

  We are leveraging this aspect of Bash when we
  [loop through each character in a string](Strings_in_Bash.md#loop-through-each-character-in-a-string).


path: Lists_in_Python.md
sortable_path: Lists_in_Python.md
title: Lists in Python
lead: Lists are the equivalent of a simple array in JavaScript.
checksum: 09b79170cd6ec9e52331243730ea99642a3e1dc2d08a2bfbdf47e5064f5f078f
created: 2025-10-01 13:30:59.528075599+00:00
modified: 2025-08-13 15:57:13.773474975+00:00
metadata: {"tags":["python","data-structures"]}
body:
  Lists are the equivalent of a simple array in JavaScript.

  Lists have the following properties:

  - They are **ordered**
  - They are **mutable** and can be modified
  - They **allow duplicate** members
  - They are **indexed**
  - You can increase/decrease their length by adding/removing new members

  > Lists are denoted with `[...]`

  ## Duplicating lists

  When we want to duplicate a list, we can't just reassign the list to a new
  variable and expect this to be a copy.

  If we edit this "copy" it will update the original list since it copies the
  pointer and will therefore point to the same address in memory. Instead we have
  to use the List `copy()` function which returns a new list and doesn't modify
  the original list.

  Relatedly, we should distinguish those List methods and functions which create a
  new list (non-destructive) and those that mutate the existing list.

  ## Basic usage

  ```python
  # Defining a list
  list1 = ['John', 'Paul', 'George', 'Ringo']
  list2 = [4]

  # Empty list
  list3 = []  # empty list
  list3 = list()  # Also empty list

  # Nested list
  list5 = [[2, 3], [6, 8]]
  ```

  ## Slicing

  ```python
  list1 = ['John', 'Paul', 'George', 'Ringo']

  print(list1[1])
  print(list1[-1])
  print(list1[1:3])
  print(list1[:3])
  print(list1[1:])

  """
  'John'
  'Ringo'
  ['Paul', 'George']
  ['John', 'Paul', 'George']
  ['Paul', 'George', 'Ringo']
  """
  ```

  ## Adding additional values to existing list

  ### Single value

  ```py
  list1 = ['John', 'Paul', 'George', 'Ringo']

  # Add single element to the end of a list
  list1.append('Pete')
  # ['John', 'Paul', 'George', 'Ringo', 'Pete']
  ```

  ### Multiple values

  ```py
  list1.extend(['Albert', 'Bob'])
  list1 += ['Ginger', 'Sporty']
  # ['John', 'Paul', 'George', 'Ringo', 'Pete', 'Albert', 'Bob', 'Ginger', 'Sporty']
  ```

  ### Specific index

  ```python

  ## Insert at specific index

  a_list = ['Adele', 'Madonna', 'Cher']
  a_list.insert(1, 'Paloma')
  print(a_list)

  # ['Adele', 'Paloma', 'Madonna', 'Cher']
  ```

  ## Removing elements

  We distinguish `del` from `remove` when removing elements from lists:

  - `del` requires an index value
  - `remove` requires a value reference (i.e. the name of the element rather than

  `del` is simple deletion whereas `remove` searches the list. Therefore `del` is
  more efficient.

  ### `del`

  ```py
  my_list = [10, 20, 30, 40, 50]
  del my_list[1]
  print(my_list)  # Output will be [10, 30, 40, 50]
  ```

  We can remove multiple items at once via a slice:

  ```py
  my_list = [10, 20, 30, 40, 50]

  # Delete the elements from index 1 to 3 (inclusive of start index and exclusive of end index)
  del my_list[1:4]

  # Print the updated list
  print(my_list)  # Output will be [10, 50]
  ```

  ### `remove()`

  ```py
  my_list = [10, 20, 30, 40, 50]

  # Remove the element with value 30
  my_list.remove(30)

  # Print the updated list
  print(my_list)  # Output will be [10, 20, 40, 50]
  ```

  > If the value is not found in the list,
  > `remove()`` will raise a ValueError. To avoid this, you can check whether the value exists in the list before calling `remove()`

  ### pop()

  Remove and return the element removed

  ```python
  list6 = ['Once', 'Upon', 'a', 'Time']
  print(list6.pop(2))
  # a

  ```

  ## Return index of a list element

  ```python
  list7 = [2, 3, 6, 8]
  print(list7.index(8))
  # 3

  list6 = ['Once', 'Upon', 'a', 'Time']
  print(list6.index('a'))
  # 2
  ```

  ## Nesting lists

  ```python
  l1 = [1, 43.5, 'Phoebe', True]
  l2 = ['apple', 'orange', 31]
  root_list = ['John', l1, l2, 'Denise']
  print(root_list)
  # ['John', [1, 43.5, 'Phoebe', True], ['apple', 'orange', 31], 'Denise']
  ```

  ## Merging lists

  ```py
  list1 = [1, 2, 3]
  list2 = [4, 5, 6]

  merged_list = list1 + list2
  print(merged_list)  # Output: [1, 2, 3, 4, 5, 6]
  ```

  ## Flattening a two-dimensional list (matrix)

  Use [list comprehension](List_comprehension_in_Python.md) for each row in the
  matrix:

  ```python
   matrix = [
      [9, 3, 8, 3],
      [4, 5, 2, 8],
      [6, 4, 3, 1],
      [1, 0, 4, 5],
  ]

  flat = [item for row in matrix for item in row]
  # [9, 3, 8, 3, 4, 5, 2, 8, 6, 4, 3, 1, 1, 0, 4, 5]
  ```

  ## See also

  [Sorting lists in Python](Sorting_lists_in_Python.md)

raw_content:
  ---
  tags: [python, data-structures]
  ---

  # Lists in Python

  Lists are the equivalent of a simple array in JavaScript.

  Lists have the following properties:

  - They are **ordered**
  - They are **mutable** and can be modified
  - They **allow duplicate** members
  - They are **indexed**
  - You can increase/decrease their length by adding/removing new members

  > Lists are denoted with `[...]`

  ## Duplicating lists

  When we want to duplicate a list, we can't just reassign the list to a new
  variable and expect this to be a copy.

  If we edit this "copy" it will update the original list since it copies the
  pointer and will therefore point to the same address in memory. Instead we have
  to use the List `copy()` function which returns a new list and doesn't modify
  the original list.

  Relatedly, we should distinguish those List methods and functions which create a
  new list (non-destructive) and those that mutate the existing list.

  ## Basic usage

  ```python
  # Defining a list
  list1 = ['John', 'Paul', 'George', 'Ringo']
  list2 = [4]

  # Empty list
  list3 = []  # empty list
  list3 = list()  # Also empty list

  # Nested list
  list5 = [[2, 3], [6, 8]]
  ```

  ## Slicing

  ```python
  list1 = ['John', 'Paul', 'George', 'Ringo']

  print(list1[1])
  print(list1[-1])
  print(list1[1:3])
  print(list1[:3])
  print(list1[1:])

  """
  'John'
  'Ringo'
  ['Paul', 'George']
  ['John', 'Paul', 'George']
  ['Paul', 'George', 'Ringo']
  """
  ```

  ## Adding additional values to existing list

  ### Single value

  ```py
  list1 = ['John', 'Paul', 'George', 'Ringo']

  # Add single element to the end of a list
  list1.append('Pete')
  # ['John', 'Paul', 'George', 'Ringo', 'Pete']
  ```

  ### Multiple values

  ```py
  list1.extend(['Albert', 'Bob'])
  list1 += ['Ginger', 'Sporty']
  # ['John', 'Paul', 'George', 'Ringo', 'Pete', 'Albert', 'Bob', 'Ginger', 'Sporty']
  ```

  ### Specific index

  ```python

  ## Insert at specific index

  a_list = ['Adele', 'Madonna', 'Cher']
  a_list.insert(1, 'Paloma')
  print(a_list)

  # ['Adele', 'Paloma', 'Madonna', 'Cher']
  ```

  ## Removing elements

  We distinguish `del` from `remove` when removing elements from lists:

  - `del` requires an index value
  - `remove` requires a value reference (i.e. the name of the element rather than

  `del` is simple deletion whereas `remove` searches the list. Therefore `del` is
  more efficient.

  ### `del`

  ```py
  my_list = [10, 20, 30, 40, 50]
  del my_list[1]
  print(my_list)  # Output will be [10, 30, 40, 50]
  ```

  We can remove multiple items at once via a slice:

  ```py
  my_list = [10, 20, 30, 40, 50]

  # Delete the elements from index 1 to 3 (inclusive of start index and exclusive of end index)
  del my_list[1:4]

  # Print the updated list
  print(my_list)  # Output will be [10, 50]
  ```

  ### `remove()`

  ```py
  my_list = [10, 20, 30, 40, 50]

  # Remove the element with value 30
  my_list.remove(30)

  # Print the updated list
  print(my_list)  # Output will be [10, 20, 40, 50]
  ```

  > If the value is not found in the list,
  > `remove()`` will raise a ValueError. To avoid this, you can check whether the value exists in the list before calling `remove()`

  ### pop()

  Remove and return the element removed

  ```python
  list6 = ['Once', 'Upon', 'a', 'Time']
  print(list6.pop(2))
  # a

  ```

  ## Return index of a list element

  ```python
  list7 = [2, 3, 6, 8]
  print(list7.index(8))
  # 3

  list6 = ['Once', 'Upon', 'a', 'Time']
  print(list6.index('a'))
  # 2
  ```

  ## Nesting lists

  ```python
  l1 = [1, 43.5, 'Phoebe', True]
  l2 = ['apple', 'orange', 31]
  root_list = ['John', l1, l2, 'Denise']
  print(root_list)
  # ['John', [1, 43.5, 'Phoebe', True], ['apple', 'orange', 31], 'Denise']
  ```

  ## Merging lists

  ```py
  list1 = [1, 2, 3]
  list2 = [4, 5, 6]

  merged_list = list1 + list2
  print(merged_list)  # Output: [1, 2, 3, 4, 5, 6]
  ```

  ## Flattening a two-dimensional list (matrix)

  Use [list comprehension](List_comprehension_in_Python.md) for each row in the
  matrix:

  ```python
   matrix = [
      [9, 3, 8, 3],
      [4, 5, 2, 8],
      [6, 4, 3, 1],
      [1, 0, 4, 5],
  ]

  flat = [item for row in matrix for item in row]
  # [9, 3, 8, 3, 4, 5, 2, 8, 6, 4, 3, 1, 1, 0, 4, 5]
  ```

  ## See also

  [Sorting lists in Python](Sorting_lists_in_Python.md)


path: Local_AWS_development_with_SAM.md
sortable_path: Local_AWS_development_with_SAM.md
title: Local AWS development with SAM
checksum: 125550b5c17fa1ce41fcfd41f42c41f618882dfec80122c235a838aadee59d73
created: 2025-10-01 13:30:59.543010274+00:00
modified: 2025-08-13 15:57:13.771492824+00:00
metadata: {"tags":["AWS","docker"]}
lead:
  We can run a local instance of our SAM stack for a given application without
  sending requests to the cloud. This is implemented through Docker.

body:
  We can run a local instance of our SAM stack for a given application without
  sending requests to the cloud. This is implemented through Docker.

  The SAM CLI handles all the Docker-related tasks, such as pulling the required
  Lambda runtime images, creating containers, mounting your code and dependencies,
  and running your Lambda functions inside those containers.

  ## Basic set up

  Enter your project directory.

  First build your SAM application:

  ```sh
  sam build
  ```

  We then run:

  ```sh
  sam local start-api
  ```

  If you have an API Gateway endpoint that you want to call over the local server.
  You will be able to call it after executing the above.

  This will be indicated by:

  ![](/img/local-sam-docker.png)

  If we want to invoke the function directly we use:

  ```sh
   sam local invoke [FunctionName]
  ```

  ## Using environment variables

  If you have an API key or database credentials, you are going to typically want
  to use different values dependent on environment.

  Even if the values are the same accross environments, it's a good idea to not
  call a secret when working locally since this request is billable.

  In the example below I show how to set up environment variables for an API key
  locally and in production.

  ## Create secret

  Go to AWS SecretsManager and add the API key as a secret. This will be sourced
  in production.

  ## Create local ENV file

  > These must be in JSON to work with SAM:

  ### Local env

  ```json
  // local-env.json
  {
    "FunctionName": {
      "API_KEY": "xxx-yyy-xxx",
      "NODE_ENV": "development"
    }
  }
  ```

  We save these to the root of the given function's directory not at the global
  repo level.

  > Be sure to add this to `.gitignore` so that it does not become public

  ### Update `template.yaml`

  Every environment variable you intend to use, must exist in the `template.yaml`,
  otherwise it will not be sourced at runtime:

  ```yaml
  ...
  Resources:
    Properties:
      Environment:
        Variables:
          SECRET_ARN: "arn:aws:secretsmanager:eu-west-2:885135949562:secret:wakatime-api-key-X9oF3v",
          NODE_ENV: production
          API_KEY:
  ...
  ```

  > We go ahead and populate the values for production. But we leave the variable
  > we use in development blank, since we don't want it committed and we will
  > source it at the SAM invocation. It still needs to exist though.

  ### Pass in the environment variable at invocation:

  ```sh
  sam local start-api --env-vars /home/thomas/repos/lambdas/wakatime-api/get-coding-stats/local-env.json
  ```

  In production, the variables required will be automatically sourced from the
  `template.yaml`

  ### Create handler within the Lambda itself

  You are obviously going to need to distinguish between the different deployments
  when the Lambda executes. Here is an example helper function:

  ```ts
  import * as AWS from "aws-sdk";

  const secretsManager = new AWS.SecretsManager();

  async function getApiKey(): Promise<string> {
    if (process.env.NODE_ENV === "production") {
      const response = await secretsManager
        .getSecretValue({ SecretId: process.env.SECRET_ARN as string })
        .promise();
      const secretValues = JSON.parse(response.SecretString as string);
      return secretValues.API_KEY;
    } else {
      return process.env.API_KEY as string;
    }
  }
  ```

raw_content:
  ---
  tags: [AWS, docker]
  ---

  # Local AWS development with SAM

  We can run a local instance of our SAM stack for a given application without
  sending requests to the cloud. This is implemented through Docker.

  The SAM CLI handles all the Docker-related tasks, such as pulling the required
  Lambda runtime images, creating containers, mounting your code and dependencies,
  and running your Lambda functions inside those containers.

  ## Basic set up

  Enter your project directory.

  First build your SAM application:

  ```sh
  sam build
  ```

  We then run:

  ```sh
  sam local start-api
  ```

  If you have an API Gateway endpoint that you want to call over the local server.
  You will be able to call it after executing the above.

  This will be indicated by:

  ![](/img/local-sam-docker.png)

  If we want to invoke the function directly we use:

  ```sh
   sam local invoke [FunctionName]
  ```

  ## Using environment variables

  If you have an API key or database credentials, you are going to typically want
  to use different values dependent on environment.

  Even if the values are the same accross environments, it's a good idea to not
  call a secret when working locally since this request is billable.

  In the example below I show how to set up environment variables for an API key
  locally and in production.

  ## Create secret

  Go to AWS SecretsManager and add the API key as a secret. This will be sourced
  in production.

  ## Create local ENV file

  > These must be in JSON to work with SAM:

  ### Local env

  ```json
  // local-env.json
  {
    "FunctionName": {
      "API_KEY": "xxx-yyy-xxx",
      "NODE_ENV": "development"
    }
  }
  ```

  We save these to the root of the given function's directory not at the global
  repo level.

  > Be sure to add this to `.gitignore` so that it does not become public

  ### Update `template.yaml`

  Every environment variable you intend to use, must exist in the `template.yaml`,
  otherwise it will not be sourced at runtime:

  ```yaml
  ...
  Resources:
    Properties:
      Environment:
        Variables:
          SECRET_ARN: "arn:aws:secretsmanager:eu-west-2:885135949562:secret:wakatime-api-key-X9oF3v",
          NODE_ENV: production
          API_KEY:
  ...
  ```

  > We go ahead and populate the values for production. But we leave the variable
  > we use in development blank, since we don't want it committed and we will
  > source it at the SAM invocation. It still needs to exist though.

  ### Pass in the environment variable at invocation:

  ```sh
  sam local start-api --env-vars /home/thomas/repos/lambdas/wakatime-api/get-coding-stats/local-env.json
  ```

  In production, the variables required will be automatically sourced from the
  `template.yaml`

  ### Create handler within the Lambda itself

  You are obviously going to need to distinguish between the different deployments
  when the Lambda executes. Here is an example helper function:

  ```ts
  import * as AWS from "aws-sdk";

  const secretsManager = new AWS.SecretsManager();

  async function getApiKey(): Promise<string> {
    if (process.env.NODE_ENV === "production") {
      const response = await secretsManager
        .getSecretValue({ SecretId: process.env.SECRET_ARN as string })
        .promise();
      const secretValues = JSON.parse(response.SecretString as string);
      return secretValues.API_KEY;
    } else {
      return process.env.API_KEY as string;
    }
  }
  ```


path: Logarithms.md
sortable_path: Logarithms.md
title: Using exponents to calculate logarithms
checksum: 741db4c876f0fe9332002a963e75702af62c7cd91a1d7e03790b8b372d0b720c
created: 2025-10-01 13:30:59.546845619+00:00
modified: 2025-08-13 15:57:13.871926616+00:00
metadata: {"tags":["algebra"]}
lead:
  This approach becomes rapidly difficult when working with larger numbers. It's
  not as obvious what $\log \_{5} 625$ would be using this method. For this
  reason, we use exponents which are intimately related to logarithms.

body:
  This approach becomes rapidly difficult when working with larger numbers. It's
  not as obvious what $\log \_{5} 625$ would be using this method. For this
  reason, we use exponents which are intimately related to logarithms.

  A logarithm can be expressed identically using exponents for example:

  $$
  \log \_{3} 9 = 2 \leftrightarrow 3^2 = 9


  $$

  By carrying out the conversion in stages, we can work out the answer to the
  question a logarithm poses.

  Let's work out $\log \_{2} 8$ using this method.

  1. First we add a variable (x) to the expression on the right hand:

     $$
     \log \_{2} 8 \leftrightarrow x


     $$

  1. Next we take the base of the logarithm and combine it with x as an exponent.
     Now our formula looks like this:

     $$
     \log \_{2} 8 \leftrightarrow 2^x


     $$

  1. Next we add an equals and the number that is left from the logarithm (8):

  $$
  \log \_{2} 8 \leftrightarrow 2^x = 8


  $$

  Then the problem is reduced to: how many times do you need to multiply two by
  itself to get 8? The answer is 3 : 2 x 2 x 2 or 2 p3. Hence we have the balanced
  equation:

  $$
  \log \_{2} 8 \leftrightarrow 2^3 = 8


  $$

  ## Common base values

  Often times a base won't be specified in a log expression. For example:

  $$
  \log1000


  $$

  This is just a shorthand and it means that the base value is ten, i.e that the
  logarithm is written in denary (base 10). So the above actually means:

  $$
  \log \_{10} 1000 = 3


  $$

  This is referred to as the **common logarithm**

  Another frequent base is Euler's number (approx. 2.71828) known as the **natural
  logarithm**

  An example

  $$
  \log \_{e} 7.389 = 2


  $$

raw_content:
  ---
  tags: [algebra]
  ---

  Most simply a logarithm is a way of answering the question:

  > How many of one number do we need to get another number. How many of x do we
  > need to get y

  More formally:

  > x raised to what power gives me y

  Below is an example of a logarithm:

  $$
  \log \_{3} 9


  $$

  We read it:

  > log base 3 of 9

  And it means:

  > 3 raised to what power gives me 9?

  In this case the answer is easy: $3^2$ gives me nine, which is to say: three
  multiplied by itself.

  ## Using exponents to calculate logarithms

  This approach becomes rapidly difficult when working with larger numbers. It's
  not as obvious what $\log \_{5} 625$ would be using this method. For this
  reason, we use exponents which are intimately related to logarithms.

  A logarithm can be expressed identically using exponents for example:

  $$
  \log \_{3} 9 = 2 \leftrightarrow 3^2 = 9


  $$

  By carrying out the conversion in stages, we can work out the answer to the
  question a logarithm poses.

  Let's work out $\log \_{2} 8$ using this method.

  1. First we add a variable (x) to the expression on the right hand:

     $$
     \log \_{2} 8 \leftrightarrow x


     $$

  1. Next we take the base of the logarithm and combine it with x as an exponent.
     Now our formula looks like this:

     $$
     \log \_{2} 8 \leftrightarrow 2^x


     $$

  1. Next we add an equals and the number that is left from the logarithm (8):

  $$
  \log \_{2} 8 \leftrightarrow 2^x = 8


  $$

  Then the problem is reduced to: how many times do you need to multiply two by
  itself to get 8? The answer is 3 : 2 x 2 x 2 or 2 p3. Hence we have the balanced
  equation:

  $$
  \log \_{2} 8 \leftrightarrow 2^3 = 8


  $$

  ## Common base values

  Often times a base won't be specified in a log expression. For example:

  $$
  \log1000


  $$

  This is just a shorthand and it means that the base value is ten, i.e that the
  logarithm is written in denary (base 10). So the above actually means:

  $$
  \log \_{10} 1000 = 3


  $$

  This is referred to as the **common logarithm**

  Another frequent base is Euler's number (approx. 2.71828) known as the **natural
  logarithm**

  An example

  $$
  \log \_{e} 7.389 = 2


  $$


path: Logic_gates.md
sortable_path: Logic_gates.md
title: Logic gates
checksum: 2690421e13abf67dfa546b560e0b6867deaed7409216467fccebb8a9e82ff62e
created: 2025-10-01 13:30:59.551716327+00:00
modified: 2025-08-13 15:57:14.061263213+00:00
metadata: {"tags":["logic-gates","binary"]}
lead:
  > [A logic gate consists in] three connections where there may or may not be
  > some electricity. Two of those connections are places where electricity may be
  > put into the device, and the third connection is a place where electricity may
  > come out of the device.

body:
  > [A logic gate consists in] three connections where there may or may not be
  > some electricity. Two of those connections are places where electricity may be
  > put into the device, and the third connection is a place where electricity may
  > come out of the device.

  [J.C. Scott. 2009. **But How Do It Know? The Basics of Computers for Everyone**,
  21]

  Logic gates are the basic building blocks of digital computing. **A logic gate
  is an electrical circuit that has one or more than one input and only one
  output.** The input and output points of the gate are
  [pins](Integrated_circuits.md) The
  input controls the output and the logic determining which types of input
  (on/off) lead to specific outputs (on/off) is isomorphic with the
  truth-conditions of the
  [Boolean connectives](Truth-functional_connectives.md) specifiable in
  terms of [truth tables](Truth-tables.md).

  Physically, what 'travels through' the gates is electrical current and what
  constitutes the 'gate' is a
  [transistor](Transistors.md)
  responding to the current. Going up a level of abstraction, the current/ charge
  is identified with a
  [bit](Binary_units_of_measurement.md#binary-units-of-measurement).
  It is bits that go into the gate and bits which come out: binary information
  that may be either 1 or 0.

  ## Elementary and composite gates

  We distinguish elementary from composite logic gates. An elementary gate is a
  single gate embodying a single logical connective. It cannot be reduced any
  lower as a logical abstraction. A composite gate is a gate made up of more than
  one elemen>tary gate and/or other composite gates.

  An example of a composite gate would be a three-way AND. An AND with three
  inputs rather than the standard two that furnish the elementary AND gate. This
  gate would output 1 when all three gates have the value 1 and 0 otherwise.
  [Adders](Half_adder_and_full_adder.md)
  and [latche>s](Latches.md) whilst
  being
  [integrated circuits](Integrated_circuits.md)
  are also, technically speaking, composite gates.

  ## Gate interface / gate implementation

  The gate _interface_ is an abstraction that the enables the user to think of the
  gate simply in terms of inputs and outputs, without being conc>erned with the
  technical details of how this is achieved. How it is achieved is the gate
  _implementation_.

  We can demonstrate this with the earlier example of a three-way AND. The diagram
  below represents the gate as an interface:

  // Add: Interface diagram

  Whereas this diagram presents the implementation of the gate: it shows the
  specific combination of gates which creates the enables the behaviour
  represented in the interface diagram.

  // Add: Implementation diagram

  > Importantly, a single interface may be implemented in a variety of ways. There
  > is a one-to-many relationship at work here. From the point of view of the user
  > interface these differences should not be detectable. This is another example
  > of
  > [hardware abstraction](Hardware_abstraction_and_modularity.md)

  ## NOT gate

  > The NOT gate inverts the value of whatever input it receives

  ### Truth conditions

  | $P$ | $\lnot P$ |
  | --- | --------- |
  | 1   | 0         |
  | 0   | 1         |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/not-gate-aeb5f9e5-9f58-4883-b8e5-d70f6d023185?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginhe

  ### Truth conditions

  tion) truth functional connective

  ### Symbol

  ![](/img/and-gate-new-2.png)

  ### Truth conditions

  ### Truth conditions

  | 1 | 0 | 0 | | 0 | 0 | 0 | | 0 | 0 | 0 |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/and-gate-b6937338-e83f-474b-af79-854b39c151a3?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>

  ## NAND gate

  > The NAND gate inverts the truth conditions of AND.

  ### Symbol

  ![](/img/nand-gate-new.png)

  ### Truth conditions

  | $P$ | $Q$ | $\lnot(P \land Q)$ |
  | --- | --- | ------------------ |
  | 1   | 1   | 0                  |
  | 1   | 0   | 0                  |
  | 0   | 1   | 0                  |
  | 0   | 0   | 1                  |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/nand-gate-60613ab9-9562-445e-9883-c4ea1920e206?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>

  NAND is a **universal logic gate**: equipped with just a NAND we can represent
  every other possible logical condition. In practice with circuits, it is more
  efficient to use specific dedicated gates (i.e OR, AND, NOT etc) for the other
  Boolean connectives but in principle the same output can be achieved through
  NANDs alone.

  ## OR gate

  > The OR gate represents the truth conditions of the
  > [disjunction](Truth-functional_connectives### Truth
  > conditions.md#disjunction) truth functional connective

  ### Symbol

  ![](/img/or-gate-new.png)

  ### Truth condition

  ### Truth conditions

  s

  | $P$ | $Q$ | $P \lor Q$ |
  | --- | --- | ---------- |
  | 1   | 1   | 1          |
  | 1   | 0   | 1          |
  | 0   | 1   | 1          |
  | 0   | 0   | 0          |

  ### Interactive circui

  ### Truth conditions

  t

  <iframe src="https://circuitverse.org/simulator/embed/or-gate-087e4933-7963-482d-b4bf-9e130ef05706?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>

  ## XOR gate

  > The OR gate represents the truth conditions of the exclusive OR

  ### Symbol

  ![](/img/xor-gate-new.png)

  ### Truth conditions

  | $P$ | $Q$ | $\lnot(P \Leftrightarrow Q)$ |
  | --- | --- | ---------------------------- |
  | 1   | 1   | 0                            |
  | 1   | 0   | 1                            |
  | 0   | 1   | 1                            |
  | 0   | 0   | 0                            |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/xor-gate-a240131e-a9c3-4240-a52b-e14412fdd654?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>

  ## NOR gate

  > The NOR gate inverts the function of an OR gate

  ### Symbol

  ![](/img/nor-gate-new.png)

  ### Truth conditions

  | $P$ | $Q$ | $P \lor Q$ |
  | --- | --- | ---------- |
  | 1   | 1   | 0          |
  | 1   | 0   | 0          |
  | 0   | 1   | 0          |
  | 0   | 0   | 1          |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/nor-gate-ac7946b4-f5d0-4c87-afd1-a2e92326d006?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>

raw_content:
  ---
  tags: [logic-gates, binary]
  ---

  # Logic gates

  > [A logic gate consists in] three connections where there may or may not be
  > some electricity. Two of those connections are places where electricity may be
  > put into the device, and the third connection is a place where electricity may
  > come out of the device.

  [J.C. Scott. 2009. **But How Do It Know? The Basics of Computers for Everyone**,
  21]

  Logic gates are the basic building blocks of digital computing. **A logic gate
  is an electrical circuit that has one or more than one input and only one
  output.** The input and output points of the gate are
  [pins](Integrated_circuits.md) The
  input controls the output and the logic determining which types of input
  (on/off) lead to specific outputs (on/off) is isomorphic with the
  truth-conditions of the
  [Boolean connectives](Truth-functional_connectives.md) specifiable in
  terms of [truth tables](Truth-tables.md).

  Physically, what 'travels through' the gates is electrical current and what
  constitutes the 'gate' is a
  [transistor](Transistors.md)
  responding to the current. Going up a level of abstraction, the current/ charge
  is identified with a
  [bit](Binary_units_of_measurement.md#binary-units-of-measurement).
  It is bits that go into the gate and bits which come out: binary information
  that may be either 1 or 0.

  ## Elementary and composite gates

  We distinguish elementary from composite logic gates. An elementary gate is a
  single gate embodying a single logical connective. It cannot be reduced any
  lower as a logical abstraction. A composite gate is a gate made up of more than
  one elemen>tary gate and/or other composite gates.

  An example of a composite gate would be a three-way AND. An AND with three
  inputs rather than the standard two that furnish the elementary AND gate. This
  gate would output 1 when all three gates have the value 1 and 0 otherwise.
  [Adders](Half_adder_and_full_adder.md)
  and [latche>s](Latches.md) whilst
  being
  [integrated circuits](Integrated_circuits.md)
  are also, technically speaking, composite gates.

  ## Gate interface / gate implementation

  The gate _interface_ is an abstraction that the enables the user to think of the
  gate simply in terms of inputs and outputs, without being conc>erned with the
  technical details of how this is achieved. How it is achieved is the gate
  _implementation_.

  We can demonstrate this with the earlier example of a three-way AND. The diagram
  below represents the gate as an interface:

  // Add: Interface diagram

  Whereas this diagram presents the implementation of the gate: it shows the
  specific combination of gates which creates the enables the behaviour
  represented in the interface diagram.

  // Add: Implementation diagram

  > Importantly, a single interface may be implemented in a variety of ways. There
  > is a one-to-many relationship at work here. From the point of view of the user
  > interface these differences should not be detectable. This is another example
  > of
  > [hardware abstraction](Hardware_abstraction_and_modularity.md)

  ## NOT gate

  > The NOT gate inverts the value of whatever input it receives

  ### Truth conditions

  | $P$ | $\lnot P$ |
  | --- | --------- |
  | 1   | 0         |
  | 0   | 1         |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/not-gate-aeb5f9e5-9f58-4883-b8e5-d70f6d023185?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginhe

  ### Truth conditions

  tion) truth functional connective

  ### Symbol

  ![](/img/and-gate-new-2.png)

  ### Truth conditions

  ### Truth conditions

  | 1 | 0 | 0 | | 0 | 0 | 0 | | 0 | 0 | 0 |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/and-gate-b6937338-e83f-474b-af79-854b39c151a3?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>

  ## NAND gate

  > The NAND gate inverts the truth conditions of AND.

  ### Symbol

  ![](/img/nand-gate-new.png)

  ### Truth conditions

  | $P$ | $Q$ | $\lnot(P \land Q)$ |
  | --- | --- | ------------------ |
  | 1   | 1   | 0                  |
  | 1   | 0   | 0                  |
  | 0   | 1   | 0                  |
  | 0   | 0   | 1                  |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/nand-gate-60613ab9-9562-445e-9883-c4ea1920e206?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>

  NAND is a **universal logic gate**: equipped with just a NAND we can represent
  every other possible logical condition. In practice with circuits, it is more
  efficient to use specific dedicated gates (i.e OR, AND, NOT etc) for the other
  Boolean connectives but in principle the same output can be achieved through
  NANDs alone.

  ## OR gate

  > The OR gate represents the truth conditions of the
  > [disjunction](Truth-functional_connectives### Truth
  > conditions.md#disjunction) truth functional connective

  ### Symbol

  ![](/img/or-gate-new.png)

  ### Truth condition

  ### Truth conditions

  s

  | $P$ | $Q$ | $P \lor Q$ |
  | --- | --- | ---------- |
  | 1   | 1   | 1          |
  | 1   | 0   | 1          |
  | 0   | 1   | 1          |
  | 0   | 0   | 0          |

  ### Interactive circui

  ### Truth conditions

  t

  <iframe src="https://circuitverse.org/simulator/embed/or-gate-087e4933-7963-482d-b4bf-9e130ef05706?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>

  ## XOR gate

  > The OR gate represents the truth conditions of the exclusive OR

  ### Symbol

  ![](/img/xor-gate-new.png)

  ### Truth conditions

  | $P$ | $Q$ | $\lnot(P \Leftrightarrow Q)$ |
  | --- | --- | ---------------------------- |
  | 1   | 1   | 0                            |
  | 1   | 0   | 1                            |
  | 0   | 1   | 1                            |
  | 0   | 0   | 0                            |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/xor-gate-a240131e-a9c3-4240-a52b-e14412fdd654?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>

  ## NOR gate

  > The NOR gate inverts the function of an OR gate

  ### Symbol

  ![](/img/nor-gate-new.png)

  ### Truth conditions

  | $P$ | $Q$ | $P \lor Q$ |
  | --- | --- | ---------- |
  | 1   | 1   | 0          |
  | 1   | 0   | 0          |
  | 0   | 1   | 0          |
  | 0   | 0   | 1          |

  ### Interactive circuit

  <iframe src="https://circuitverse.org/simulator/embed/nor-gate-ac7946b4-f5d0-4c87-afd1-a2e92326d006?theme=default&display_title=false&clock_time=true&fullscreen=true&zoom_in_out=true" style="border-width:; border-style: solid; border-color:;" name="myiframe" id="projectPreview" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="250" width="500" allowFullScreen></iframe>


path: Logical_consistency.md
sortable_path: Logical_consistency.md
title: Logical consistency
lead: ## Informal definition
checksum: 25b31fea3e617c20721052b3fb39a4b363a687e9183bbcf5cc6abab51e4219b6
created: 2025-10-01 13:30:59.592810436+00:00
modified: 2025-08-13 15:57:13.834492754+00:00
metadata: {"tags":["propositional-logic","logic"]}
body:
  ## Informal definition

  A set of propositions is consistent if and only if **it is possible for all the
  members of the set to be true at the same time**. A set of propositions is
  inconsistent if and only if it is not consistent.

  ### Demonstration

  The following set of propositions form an inconsistent set:

  1. Anyone who takes astrology seriously is a lunatic.
  2. Alice is my sister and no sister of mine has a lunatic for a husband.
  3. David is Alice's husband and he read's the horoscope column every morning.
  4. Anyone who reads the horoscope column every morning takes astrology
     seriously.

  The set is inconsistent because not all of them can be true. If (1), (3), (4)
  are true, (2) cannot be. If (2), (3),(4) are true, (1) cannot be.

  ## Formal definition

  > A finite set of propositions $\Gamma$ is truth-functionally consistent if and
  > only if there is at least one truth-assignment in which all propositions of
  > $\Gamma$ are true.

  ### Informal expression

  ```
  The book is blue or the book is brown
  The book is brown
  ```

  ### Formal expression

  $$
  \{P \lor Q, Q\}
  $$

  ### Truth table

  $ \{P, Q\} $ form a consistent set because there is at least one assignment when
  both propositions are true. In fact there are two (corresponding to each
  disjunct) but one is sufficient.

  | $P$ | $Q$ | $ P \lor Q $ | $Q$ |
  | --- | --- | ------------ | --- |
  | T   | T   | T            | T   |
  | T   | F   | T            | F   |
  | F   | T   | T            | T   |
  | F   | F   | F            | F   |

  ## Derivation

  > In terms of logical derivation, a finite $\Gamma$ of propositions is
  > **inconsistent** in a system of derivation for propositional logic if and only
  > if a proposition of the form $P \& \lnot P$ is derivable from $\Gamma$. It is
  > **consistent** just if this is not the case.

  In other terms, if you can derive a contradiction from the set, the set is
  logically inconsistent.

  A
  [contradiction](Logical_truth_and_falsity.md#logical-falsity)
  has very important consequences for reasoning because if a set of propositions
  is inconsistent, any other proposition is derivable from it.

  ![](/img/derivation_from_contradiction.png)

  _A demonstration of the the consequences of deriving a contradiction in a
  sequence of reasoning._

  Here we want to derive some proposition $Q$. If we can derive a contradiction
  from its negation as an assumption then, by the
  [negation elimination](Negation_Elimination.md)) rule, we can
  assert $Q$. This is why contradictions should be avoided in arguments, they
  'prove' everything which, by association, undermines any particular premise you
  are trying to assert.

raw_content:
  ---
  tags:
    - propositional-logic
    - logic
  ---

  # Logical consistency

  ## Informal definition

  A set of propositions is consistent if and only if **it is possible for all the
  members of the set to be true at the same time**. A set of propositions is
  inconsistent if and only if it is not consistent.

  ### Demonstration

  The following set of propositions form an inconsistent set:

  1. Anyone who takes astrology seriously is a lunatic.
  2. Alice is my sister and no sister of mine has a lunatic for a husband.
  3. David is Alice's husband and he read's the horoscope column every morning.
  4. Anyone who reads the horoscope column every morning takes astrology
     seriously.

  The set is inconsistent because not all of them can be true. If (1), (3), (4)
  are true, (2) cannot be. If (2), (3),(4) are true, (1) cannot be.

  ## Formal definition

  > A finite set of propositions $\Gamma$ is truth-functionally consistent if and
  > only if there is at least one truth-assignment in which all propositions of
  > $\Gamma$ are true.

  ### Informal expression

  ```
  The book is blue or the book is brown
  The book is brown
  ```

  ### Formal expression

  $$
  \{P \lor Q, Q\}
  $$

  ### Truth table

  $ \{P, Q\} $ form a consistent set because there is at least one assignment when
  both propositions are true. In fact there are two (corresponding to each
  disjunct) but one is sufficient.

  | $P$ | $Q$ | $ P \lor Q $ | $Q$ |
  | --- | --- | ------------ | --- |
  | T   | T   | T            | T   |
  | T   | F   | T            | F   |
  | F   | T   | T            | T   |
  | F   | F   | F            | F   |

  ## Derivation

  > In terms of logical derivation, a finite $\Gamma$ of propositions is
  > **inconsistent** in a system of derivation for propositional logic if and only
  > if a proposition of the form $P \& \lnot P$ is derivable from $\Gamma$. It is
  > **consistent** just if this is not the case.

  In other terms, if you can derive a contradiction from the set, the set is
  logically inconsistent.

  A
  [contradiction](Logical_truth_and_falsity.md#logical-falsity)
  has very important consequences for reasoning because if a set of propositions
  is inconsistent, any other proposition is derivable from it.

  ![](/img/derivation_from_contradiction.png)

  _A demonstration of the the consequences of deriving a contradiction in a
  sequence of reasoning._

  Here we want to derive some proposition $Q$. If we can derive a contradiction
  from its negation as an assumption then, by the
  [negation elimination](Negation_Elimination.md)) rule, we can
  assert $Q$. This is why contradictions should be avoided in arguments, they
  'prove' everything which, by association, undermines any particular premise you
  are trying to assert.


path: Logical_equivalence.md
sortable_path: Logical_equivalence.md
title: Logical equivalence
checksum: b08ec98b4bd43e93cacd533e9b9d0847dc5ae2454cd8d160eed3a4b7a635fb90
created: 2025-10-01 13:30:59.610586472+00:00
modified: 2025-08-13 15:57:13.830807061+00:00
metadata: {"tags":["propositional-logic","logic"]}
lead:
  > Two sentences, P and Q, are truth-functionally equivalent if and only if there
  > is no truth assignment in which P is true and Q is false

body:
  > Two sentences, P and Q, are truth-functionally equivalent if and only if there
  > is no truth assignment in which P is true and Q is false

  ### Informal expression

  P: If it is raining then the pavement will be wet.

  Q: The pavement is not wet unless it is raining.

  ### Formal expression

  $$
  (P \rightarrow Q) \longleftrightarrow (\lnot P \lor Q)
  $$

  ### Truth-tables

  | $P$ | $Q$ | $ P \rightarrow Q $ | $ \lnot P \lor Q$ |
  | --- | --- | ------------------- | ----------------- |
  | T   | T   | T                   | T                 |
  | T   | F   | T                   | F                 |
  | F   | T   | T                   | T                 |
  | F   | F   | F                   | T                 |

  ### Derivation

  > Propositions $P$ and $Q$ are equivalent in a system of
  > [derivation](Formal%20proofs%20in%20propositional%20logic.md) for
  > propositional logic if $Q$ is derivable from $P$ and $P$ is derivable from
  > $Q$.

  Note that the property of equivalence stated in terms of derivablity above is
  identical to the derivation rule for the
  [material biconditional](Biconditional_Introduction.md):

  ![bi-intro.png](bi-intro.png)

raw_content:
  ---
  tags:
    - propositional-logic
    - logic
  ---

  # Logical equivalence

  > Two sentences, P and Q, are truth-functionally equivalent if and only if there
  > is no truth assignment in which P is true and Q is false

  ### Informal expression

  P: If it is raining then the pavement will be wet.

  Q: The pavement is not wet unless it is raining.

  ### Formal expression

  $$
  (P \rightarrow Q) \longleftrightarrow (\lnot P \lor Q)
  $$

  ### Truth-tables

  | $P$ | $Q$ | $ P \rightarrow Q $ | $ \lnot P \lor Q$ |
  | --- | --- | ------------------- | ----------------- |
  | T   | T   | T                   | T                 |
  | T   | F   | T                   | F                 |
  | F   | T   | T                   | T                 |
  | F   | F   | F                   | T                 |

  ### Derivation

  > Propositions $P$ and $Q$ are equivalent in a system of
  > [derivation](Formal%20proofs%20in%20propositional%20logic.md) for
  > propositional logic if $Q$ is derivable from $P$ and $P$ is derivable from
  > $Q$.

  Note that the property of equivalence stated in terms of derivablity above is
  identical to the derivation rule for the
  [material biconditional](Biconditional_Introduction.md):

  ![bi-intro.png](bi-intro.png)


path: Logical_indeterminacy.md
sortable_path: Logical_indeterminacy.md
title: Logical indeterminacy
checksum: 32e3fa137e65eff2c6f39ea7a5fca9d67f8a160dfbe8ad1551a0c8995a001af3
created: 2025-10-01 13:30:59.626497015+00:00
modified: 2025-08-13 15:57:13.985371657+00:00
metadata: {"tags":["propositional-logic","logic"]}
lead:
  The vast majority of propositions in natural and formal logical languages are
  **neither
  [logically true](Logical_truth_and_falsity.md#logical-truth)
  or
  [logically false](Logical_truth_and_falsity.md#logical-falsity)**.
  This makes sense because propositions of this form are all either tautologies or
  contradictions and as such do not express information about the state of events
  in the world. We call propositions that are neither logically true or logically
  false, **logically indeterminate** propositions.

body:
  The vast majority of propositions in natural and formal logical languages are
  **neither
  [logically true](Logical_truth_and_falsity.md#logical-truth)
  or
  [logically false](Logical_truth_and_falsity.md#logical-falsity)**.
  This makes sense because propositions of this form are all either tautologies or
  contradictions and as such do not express information about the state of events
  in the world. We call propositions that are neither logically true or logically
  false, **logically indeterminate** propositions.

  ## Informal definition

  A proposition is logically indeterminate if it is neither logically true or
  logically false. This is to say: it can be both [consistently](Consistency.md)
  asserted and consistently denied.

  For example the proposition:

  ```
  It is raining.
  ```

  May be true or false thus it can it both be asserted and denied quite
  consistently. It is true if it actually is raining and false if it actually is
  not raining. There is no logical contradiction implied by saying it is raining
  when it isn't raining, this assertion is simply false. There is a contradiction
  in saying that both states obtain. Thus the proposition:

  ```
  It is raining and it is not raining.
  ```

  Cannot be consistently asserted as there is no possibility of the proposition
  being true. It is either raining or it isn't raining. Given the law for
  conjunction, both conjuncts must be true for the proposition as a whole to be
  true. But in the case of this proposition if one conjunct is true, the other
  must be false and vice versa, hence it is not possible for the proposition to be
  true at all. It can _only_ be false.

  Contrariwise the proposition:

  ```
  It is raining or it is not raining.
  ```

  Cannot be consistently denied as there is no possibility of it being false. It
  is either raining or not raining. Given the law for disjunction, either disjunct
  can be true to make the proposition as a whole true. Given that it is either
  raining or not raining in either scenario, the proposition as a whole will be
  true. Therefore there is no possibility of it being false, it can _only_ be
  true.

  ## Formal definition

  > A proposition P is truth-functionally indeterminate if and only if it is
  > neither truth-functionally true or truth-functionally false. should be avoided
  > in arguments, they 'prove' everything whi

  ```
  P
  ```

  ### Truth-table

  | $P$ | $P$ |
  | --- | --- |
  | T   | T   |
  | F   | F   |

raw_content:
  ---
  tags:
    - propositional-logic
    - logic
  ---

  # Logical indeterminacy

  The vast majority of propositions in natural and formal logical languages are
  **neither
  [logically true](Logical_truth_and_falsity.md#logical-truth)
  or
  [logically false](Logical_truth_and_falsity.md#logical-falsity)**.
  This makes sense because propositions of this form are all either tautologies or
  contradictions and as such do not express information about the state of events
  in the world. We call propositions that are neither logically true or logically
  false, **logically indeterminate** propositions.

  ## Informal definition

  A proposition is logically indeterminate if it is neither logically true or
  logically false. This is to say: it can be both [consistently](Consistency.md)
  asserted and consistently denied.

  For example the proposition:

  ```
  It is raining.
  ```

  May be true or false thus it can it both be asserted and denied quite
  consistently. It is true if it actually is raining and false if it actually is
  not raining. There is no logical contradiction implied by saying it is raining
  when it isn't raining, this assertion is simply false. There is a contradiction
  in saying that both states obtain. Thus the proposition:

  ```
  It is raining and it is not raining.
  ```

  Cannot be consistently asserted as there is no possibility of the proposition
  being true. It is either raining or it isn't raining. Given the law for
  conjunction, both conjuncts must be true for the proposition as a whole to be
  true. But in the case of this proposition if one conjunct is true, the other
  must be false and vice versa, hence it is not possible for the proposition to be
  true at all. It can _only_ be false.

  Contrariwise the proposition:

  ```
  It is raining or it is not raining.
  ```

  Cannot be consistently denied as there is no possibility of it being false. It
  is either raining or not raining. Given the law for disjunction, either disjunct
  can be true to make the proposition as a whole true. Given that it is either
  raining or not raining in either scenario, the proposition as a whole will be
  true. Therefore there is no possibility of it being false, it can _only_ be
  true.

  ## Formal definition

  > A proposition P is truth-functionally indeterminate if and only if it is
  > neither truth-functionally true or truth-functionally false. should be avoided
  > in arguments, they 'prove' everything whi

  ```
  P
  ```

  ### Truth-table

  | $P$ | $P$ |
  | --- | --- |
  | T   | T   |
  | F   | F   |


path: Logical_possibility_and_necessity.md
sortable_path: Logical_possibility_and_necessity.md
title: Logical possibility and necessity
lead: ## Logical possibility
checksum: f2b8d371f555e5b90f5e19beba80d8f0b6a488c50cc44d3ee4aca2bab05158c9
created: 2025-10-01 13:30:59.650219255+00:00
modified: 2025-08-13 15:57:14.095785117+00:00
metadata: {"tags":["propositional-logic","logic"]}
body:
  ## Logical possibility

  In distinguishing the properties of
  [logical consistency](Logical_consistency.md) and
  [validity](Validity_and_entailment.md) we make tacit use
  of the notion of **possibility**. This is because when we consider the validity
  of an argument we are assessing truth-conditions and this consists in asking
  ourselves what could or could not be the case: were it such that _P_, then it
  would be the case that _Q_. It is important to understand what possibility means
  in the context of logic and how it differs from what we might mean ordinarily
  when we use the term.

  It is evident from the case of arguments that are valid but not sound that logic
  operates with a specialised notion of possibility. For example it has to be the
  case that the proposition _Every woman can levitate_ is logically possible since
  the following argument is valid:

  ```
  1.  P: Janice is a woman.
  2.  P: Every woman can levitate.
  3.  C: Janice can levitate.
  ```

  But we know of course that women cannot levitate. When we assert that this is
  impossible we are relying on a stronger notion of possibility than logical
  possibility. It follows that the concept of possibility can have different
  degrees. The scope of the concept of possibility has been the concern of
  logicians and philosophers since at least the time of Plato and numerous
  different formulations exist. The notion that we mostly work with unreflectively
  in everyday life is nomological possibility. This means ‘governed by the
  application of laws’ where these laws pertain to our current understanding of
  the natural world as determined by physics. Levitation is therefore
  nomologically impossible but logically possible.

  If logical possibility is not constrained by the laws of physics does it place
  any restrictions on what is possible? Logic applies a single restriction, the
  law of non-contradiction: a proposition cannot both be true and false at once.
  The following propositions are examples of a contradictory propositions.

  Some examples of contradictions:

  - There is a dog that is not a dog
  - Today is Tuesday and today is not Tuesday
  - The cat that is dead is alive

  From this we can derive the following property of logical possibility:

  > A proposition is logically possible just if it does not imply a contradiction.

  ## Logical necessity

  A proposition is _logically necessary_ if it is true in every logically possible
  circumstance which is to say: true on every possible truth functional
  assignment. Necessity and
  [logical truth](Logical_truth_and_falsity.md#logical-truth)
  are therefore synonyms: anything that is logically true (a tautology) is true by
  necessity (could not be otherwise.)

  Further, every logical truth is logically possible but not everything that is
  logically possible is logically true. It is possible that it is raining but this
  is not logically necessary - it could be otherwise, i.e not raining. However it
  is not possible that it could be both raining and not raining.

raw_content:
  ---
  tags:
    - propositional-logic
    - logic
  ---

  # Logical possibility and necessity

  ## Logical possibility

  In distinguishing the properties of
  [logical consistency](Logical_consistency.md) and
  [validity](Validity_and_entailment.md) we make tacit use
  of the notion of **possibility**. This is because when we consider the validity
  of an argument we are assessing truth-conditions and this consists in asking
  ourselves what could or could not be the case: were it such that _P_, then it
  would be the case that _Q_. It is important to understand what possibility means
  in the context of logic and how it differs from what we might mean ordinarily
  when we use the term.

  It is evident from the case of arguments that are valid but not sound that logic
  operates with a specialised notion of possibility. For example it has to be the
  case that the proposition _Every woman can levitate_ is logically possible since
  the following argument is valid:

  ```
  1.  P: Janice is a woman.
  2.  P: Every woman can levitate.
  3.  C: Janice can levitate.
  ```

  But we know of course that women cannot levitate. When we assert that this is
  impossible we are relying on a stronger notion of possibility than logical
  possibility. It follows that the concept of possibility can have different
  degrees. The scope of the concept of possibility has been the concern of
  logicians and philosophers since at least the time of Plato and numerous
  different formulations exist. The notion that we mostly work with unreflectively
  in everyday life is nomological possibility. This means ‘governed by the
  application of laws’ where these laws pertain to our current understanding of
  the natural world as determined by physics. Levitation is therefore
  nomologically impossible but logically possible.

  If logical possibility is not constrained by the laws of physics does it place
  any restrictions on what is possible? Logic applies a single restriction, the
  law of non-contradiction: a proposition cannot both be true and false at once.
  The following propositions are examples of a contradictory propositions.

  Some examples of contradictions:

  - There is a dog that is not a dog
  - Today is Tuesday and today is not Tuesday
  - The cat that is dead is alive

  From this we can derive the following property of logical possibility:

  > A proposition is logically possible just if it does not imply a contradiction.

  ## Logical necessity

  A proposition is _logically necessary_ if it is true in every logically possible
  circumstance which is to say: true on every possible truth functional
  assignment. Necessity and
  [logical truth](Logical_truth_and_falsity.md#logical-truth)
  are therefore synonyms: anything that is logically true (a tautology) is true by
  necessity (could not be otherwise.)

  Further, every logical truth is logically possible but not everything that is
  logically possible is logically true. It is possible that it is raining but this
  is not logically necessary - it could be otherwise, i.e not raining. However it
  is not possible that it could be both raining and not raining.


path: Logical_truth_and_falsity.md
sortable_path: Logical_truth_and_falsity.md
title: Logical truth and falsity
lead: We say of certain propositions that they are logically true or logically false.
checksum: 4a99a064e5afe470ad3eb6ae5bb245501ff8c76df7292f13e67109dd80525769
created: 2025-10-01 13:30:59.670670857+00:00
modified: 2025-08-13 15:57:13.873271282+00:00
metadata: {"tags":["propositional-logic","logic"]}
body:
  We say of certain propositions that they are logically true or logically false.

  ## Logical falsity

  ### Informal definition

  A proposition is logically false if and only if **it is not possible for the
  proposition to be true**. The proposition itself cannot be consistently
  asserted.

  **Demonstration**

  ```
  There is a country that is not a country.
  Apples are fruits and apples are not fruits
  ```

  Neither proposition can be true because the truth of the first clause is
  contradicted by the second. By the principle of
  [consistency](Logical_consistency.md), it is not
  possible for both clauses to be true at once therefore the proposition, overall
  has the truth value of false.

  ```
  It is raining and it is not raining.
  ```

  ### Formal definition

  > A proposition P is truth-functionally false if and only if P is false on every
  > truth-value assignment

  ### Formal expression

  $$ P \land \lnot P $$

  ### Truth-table

  | $P$ | $P \land \lnot P$ |
  | --- | ----------------- |
  | T   | F                 |
  | T   | F                 |

  ## Logical truth

  ### Informal definition

  A proposition is logically true if and only if it is not possible for the
  proposition to be false. The proposition itself cannot be consistently denied.

  **Demonstration**

  ```
  A rose is a rose.
  Today is Tuesday unless today is not Tuesday.
  ```

  Regardless of any facts obtaining in the world, these propositions cannot be
  false.should be avoided in arguments, they 'prove' everything whi

  As with logically false propositions, logical truth can also apply to compound
  propositions:

  ```
  A rose is a rose and a shoe is a shoe
  ```

  ### Formal definition

  > A proposition P is truth-functionally true if and only if P is true on every
  > truth-value assignment

  $$ P \lor \lnot P$$

  ### Truth-table

  | $P$ | $P \lor \lnot P$ |
  | --- | ---------------- |
  | T   | T                |
  | F   | T                |

  ### Consequences

  The existence of logically false and logically true propositions affects the
  validity and soundness of arguments in which they are used. These are
  technicalities that have philosophically interesting consequences.

  - If an argument contains premises which are logically false than this argument
    will perforce be valid. This is because one cannot consistently assert the
    premises and deny the conclusion which is the definition of
    [validity](Validity_and_entailment.md). However the
    _reason_ why one cannot consistently assert the premises and deny the
    conclusions is because one cannot consistently assert the premises - they
    conflict with each other. Furthermore as the argument contains false premises,
    it cannot be sound.

    ```
    (P1) Russia is a country.
    (P2) Russia is not a country
    (P3) All countries have languages.
    ____________________________________________
    (C) Russian is a language.
    ```

  - Any argument with a logically true conclusion is valid. Because the conclusion
    cannot be consistently denied it follows that we cannot consistently assert
    the premises _and_ deny the conclusion. Whether or not the argument is sound
    remains an open question however. If the premises happen to be true then the
    argument will be sound on the strength of the conclusion being logically true
    but if the premises are false it will be unsound regardless of the truth of
    the conclusion.

    ```
    (P1) Horses have legs.
    (P2) Animals with legs can move.
    ____________________________________________
    (C) A horse is a horse
    ```

raw_content:
  ---
  tags:
    - propositional-logic
    - logic
  ---

  # Logical truth and falsity

  We say of certain propositions that they are logically true or logically false.

  ## Logical falsity

  ### Informal definition

  A proposition is logically false if and only if **it is not possible for the
  proposition to be true**. The proposition itself cannot be consistently
  asserted.

  **Demonstration**

  ```
  There is a country that is not a country.
  Apples are fruits and apples are not fruits
  ```

  Neither proposition can be true because the truth of the first clause is
  contradicted by the second. By the principle of
  [consistency](Logical_consistency.md), it is not
  possible for both clauses to be true at once therefore the proposition, overall
  has the truth value of false.

  ```
  It is raining and it is not raining.
  ```

  ### Formal definition

  > A proposition P is truth-functionally false if and only if P is false on every
  > truth-value assignment

  ### Formal expression

  $$ P \land \lnot P $$

  ### Truth-table

  | $P$ | $P \land \lnot P$ |
  | --- | ----------------- |
  | T   | F                 |
  | T   | F                 |

  ## Logical truth

  ### Informal definition

  A proposition is logically true if and only if it is not possible for the
  proposition to be false. The proposition itself cannot be consistently denied.

  **Demonstration**

  ```
  A rose is a rose.
  Today is Tuesday unless today is not Tuesday.
  ```

  Regardless of any facts obtaining in the world, these propositions cannot be
  false.should be avoided in arguments, they 'prove' everything whi

  As with logically false propositions, logical truth can also apply to compound
  propositions:

  ```
  A rose is a rose and a shoe is a shoe
  ```

  ### Formal definition

  > A proposition P is truth-functionally true if and only if P is true on every
  > truth-value assignment

  $$ P \lor \lnot P$$

  ### Truth-table

  | $P$ | $P \lor \lnot P$ |
  | --- | ---------------- |
  | T   | T                |
  | F   | T                |

  ### Consequences

  The existence of logically false and logically true propositions affects the
  validity and soundness of arguments in which they are used. These are
  technicalities that have philosophically interesting consequences.

  - If an argument contains premises which are logically false than this argument
    will perforce be valid. This is because one cannot consistently assert the
    premises and deny the conclusion which is the definition of
    [validity](Validity_and_entailment.md). However the
    _reason_ why one cannot consistently assert the premises and deny the
    conclusions is because one cannot consistently assert the premises - they
    conflict with each other. Furthermore as the argument contains false premises,
    it cannot be sound.

    ```
    (P1) Russia is a country.
    (P2) Russia is not a country
    (P3) All countries have languages.
    ____________________________________________
    (C) Russian is a language.
    ```

  - Any argument with a logically true conclusion is valid. Because the conclusion
    cannot be consistently denied it follows that we cannot consistently assert
    the premises _and_ deny the conclusion. Whether or not the argument is sound
    remains an open question however. If the premises happen to be true then the
    argument will be sound on the strength of the conclusion being logically true
    but if the premises are false it will be unsound regardless of the truth of
    the conclusion.

    ```
    (P1) Horses have legs.
    (P2) Animals with legs can move.
    ____________________________________________
    (C) A horse is a horse
    ```


path: Loops_in_Python.md
sortable_path: Loops_in_Python.md
title: Loops in Python
lead: ## For
checksum: ca0f453f1cd1fc7e5ac9385243923ae593595fe1ded7c3961b3a7f4396369dc1
created: 2025-10-01 13:30:59.684967048+00:00
modified: 2025-08-13 15:57:13.794824299+00:00
metadata: {"tags":["python"]}
body:
  ## For

  There are three main types of `for` loop in Python and they all use the
  `for...in` clause:

  1. Traditional `for` loop with range:

     ```py
     for i in range(0, 5):
         print(i)
     ```

     This is equivalent to the following in JavaScript:

     ```js
     for (let i = 0; i <= 5: i++) {
         console.log(i)
     }
     ```

  2. `for` loop with iterable:

     ```py
     fruits = ["apple", "banana", "cherry"]
     for fruit in fruits:
         print(fruit)
     ```

  3. `for` loop with `enumerate()` function: Iterate over an iterable whilst also
     keeping track of the index of the current element:
     ```py
     for index, fruit in enumerate(fruits):
         print(f"Index {index}: {fruit}")
     ```

  ### Loop in increments greater than 1

  The following loop increments by 2 on each iteration

  ```py
  for i in range(0, 10, 2):
      print(i, ' ', end='')
  print()

  # 0  2  4  6  8
  ```

  ### Break and continue

  #### Break

  ```py
  num = int(input('Enter a number from 1-6 to check for: '))
  for i in range(0, 6):
      if i == num:
          break
      print(i, ' ')
  ```

  #### Continue

  ```py
  for i in range(0, 10):
      print(i, ' ', end='')
      if i % 2 == 1:
          continue
      print('hey its an even number')
      print('we love even numbers')
  print('Done')

  """
  0  hey its an even number
  we love even numbers
  1  2  hey its an even number
  we love even numbers
  3  4  hey its an even number
  we love even numbers
  5  6  hey its an even number
  we love even numbers
  7  8  hey its an even number
  we love even numbers
  9  Done
  """
  ```

  ## While

  ```python
  count = 0
  print('Starting')
  while count < 10:
      print(count, '', end='')
      count += 1

  print()  # not part of the while loop
  print('Done')

  """
  Starting
  0 1 2 3 4 5 6 7 8 9
  Done
  """
  ```

  > There are no `do while` loops in Python

raw_content:
  ---
  tags: [python]
  ---

  # Loops in Python

  ## For

  There are three main types of `for` loop in Python and they all use the
  `for...in` clause:

  1. Traditional `for` loop with range:

     ```py
     for i in range(0, 5):
         print(i)
     ```

     This is equivalent to the following in JavaScript:

     ```js
     for (let i = 0; i <= 5: i++) {
         console.log(i)
     }
     ```

  2. `for` loop with iterable:

     ```py
     fruits = ["apple", "banana", "cherry"]
     for fruit in fruits:
         print(fruit)
     ```

  3. `for` loop with `enumerate()` function: Iterate over an iterable whilst also
     keeping track of the index of the current element:
     ```py
     for index, fruit in enumerate(fruits):
         print(f"Index {index}: {fruit}")
     ```

  ### Loop in increments greater than 1

  The following loop increments by 2 on each iteration

  ```py
  for i in range(0, 10, 2):
      print(i, ' ', end='')
  print()

  # 0  2  4  6  8
  ```

  ### Break and continue

  #### Break

  ```py
  num = int(input('Enter a number from 1-6 to check for: '))
  for i in range(0, 6):
      if i == num:
          break
      print(i, ' ')
  ```

  #### Continue

  ```py
  for i in range(0, 10):
      print(i, ' ', end='')
      if i % 2 == 1:
          continue
      print('hey its an even number')
      print('we love even numbers')
  print('Done')

  """
  0  hey its an even number
  we love even numbers
  1  2  hey its an even number
  we love even numbers
  3  4  hey its an even number
  we love even numbers
  5  6  hey its an even number
  we love even numbers
  7  8  hey its an even number
  we love even numbers
  9  Done
  """
  ```

  ## While

  ```python
  count = 0
  print('Starting')
  while count < 10:
      print(count, '', end='')
      count += 1

  print()  # not part of the while loop
  print('Done')

  """
  Starting
  0 1 2 3 4 5 6 7 8 9
  Done
  """
  ```

  > There are no `do while` loops in Python


path: Loops_in_bash.md
sortable_path: Loops_in_bash.md
title: Loops in Bash
lead: ## Traditional for loop
checksum: 021d8bf50cddf71c62034ac838af2c11526a8a1ce9f3debec29c67a6f8150ef6
created: 2025-10-01 13:30:59.689405939+00:00
modified: 2025-08-13 15:57:13.85167007+00:00
metadata: {"tags":["shell"]}
body:
  ## Traditional for loop

  ```bash
  for (( i=0; i<=5;i++  )); do
    echo $i
  done

  # 1 2 3 4 5
  ```

  ## `for..in`: loop through an array

  ```bash
  for element in "${arr[@]}"
  do
      echo "$element"
  done
  ```

  ## While loop

  > `while` loops execute while a condition is true (0)

  We can use a `while` loop as a condition in two senses:

  - execute while a given condition obtains
  - expand and execute a given command as long as the final command in the `while'
    for the command has an exit status of zero (i.e. truthy)

  Here is an exampe of using `while` in the former case:

  ```sh
  declare -i n=0
  while (( n<10 ))
  do
    echo "n:$n"
    (( n++ ))
  done
  ```

  Here is an example of using `while` in the latter case:

  ```sh
  while read line;
  do
      # something
  done < "$file_to_read"
  ```

  ## Until loop

  > `until` loops execute until a condition is false (1)

  ```sh
  declare -i m=0
  until (( m==10 )): do
    echo "m:$m"
    (( m++ ))
  done
  ```

  This gives us the same output as `n` with the while loop but here it runs so
  long as `m==10` is false. As soon as `m` is equal to 100, the condition becomes
  true and hence the loop stops.

raw_content:
  ---
  tags:
    - shell
  ---

  # Loops in Bash

  ## Traditional for loop

  ```bash
  for (( i=0; i<=5;i++  )); do
    echo $i
  done

  # 1 2 3 4 5
  ```

  ## `for..in`: loop through an array

  ```bash
  for element in "${arr[@]}"
  do
      echo "$element"
  done
  ```

  ## While loop

  > `while` loops execute while a condition is true (0)

  We can use a `while` loop as a condition in two senses:

  - execute while a given condition obtains
  - expand and execute a given command as long as the final command in the `while'
    for the command has an exit status of zero (i.e. truthy)

  Here is an exampe of using `while` in the former case:

  ```sh
  declare -i n=0
  while (( n<10 ))
  do
    echo "n:$n"
    (( n++ ))
  done
  ```

  Here is an example of using `while` in the latter case:

  ```sh
  while read line;
  do
      # something
  done < "$file_to_read"
  ```

  ## Until loop

  > `until` loops execute until a condition is false (1)

  ```sh
  declare -i m=0
  until (( m==10 )): do
    echo "m:$m"
    (( m++ ))
  done
  ```

  This gives us the same output as `n` with the while loop but here it runs so
  long as `m==10` is false. As soon as `m` is equal to 100, the condition becomes
  true and hence the loop stops.


path: MAC_addresses.md
sortable_path: MAC_addresses.md
title: MAC addresses
checksum: d385cd4964d14e1ef14c8f39104492844bd52abd4e3d9b6e0532bf89f721339d
created: 2025-10-01 13:30:59.694145277+00:00
modified: 2025-08-13 15:57:14.056717632+00:00
metadata: {"created":"Tuesday, August 06, 2024","tags":["networks"]}
lead:
  A Media Access Control (MAC) address is a physical address in contrast to, say,
  an IP address which is logical.

body:
  A Media Access Control (MAC) address is a physical address in contrast to, say,
  an IP address which is logical.

  Each device on a [local network](./Link_Layer_of_Internet_Protocol.md) has a MAC
  as a unique identifier. It is a hardware-based, physical attribute of the
  device, typically permanently encoded onto a non-volatile memory chip attached
  to the ![network_card](Network_card.md).

  MAC addresses consist of 6 bytes (48-bits) represented as 12
  [hexadecimal_digits](Hexadecimal_number_system.md).

raw_content:
  ---
  tags: [networks]
  created: Tuesday, August 06, 2024
  ---

  # MAC addresses

  A Media Access Control (MAC) address is a physical address in contrast to, say,
  an IP address which is logical.

  Each device on a [local network](./Link_Layer_of_Internet_Protocol.md) has a MAC
  as a unique identifier. It is a hardware-based, physical attribute of the
  device, typically permanently encoded onto a non-volatile memory chip attached
  to the ![network_card](Network_card.md).

  MAC addresses consist of 6 bytes (48-bits) represented as 12
  [hexadecimal_digits](Hexadecimal_number_system.md).


path: Machine_code.md
sortable_path: Machine_code.md
title: Machine code
checksum: 7263fea79e96bbbcfefd575e9e1dee8b4d44be390d74f55f036f5ab5ae8ada7c
created: 2025-10-01 13:30:59.709666234+00:00
modified: 2025-08-13 15:57:13.839120833+00:00
lead:
  No matter how a program was originally written, it eventually needs to execute
  on a [[CPU_architecture|CPU]] as a series of machine language instructions.

body:
  No matter how a program was originally written, it eventually needs to execute
  on a [[CPU_architecture|CPU]] as a series of machine language instructions.

  ## Example machine instruction

  The following machine instruction is an instruction that would be understood by
  an [[Instruction_set_architectures| ARM ISA]]. It moves the number `4` into the
  `r7` register:

  ```
  11100011101000000111000000000100
  ```

  This 32-bit line of binary has a series of instructions embedded within it. We
  partition each part of the sequence below, from left to right, mapping it to the
  instruction:

  | Binary sequence | Instruction          | Action                                   |
  | --------------- | -------------------- | ---------------------------------------- |
  | 1110            | condition            | always execute (unconditional)           |
  | 00              | --                   | --                                       |
  | 1               | immediate            | value is in the last 8 bits              |
  | 1101            | opcode               | `mov` : move the value                   |
  | 0               | --                   | --                                       |
  | 0000            | --                   | --                                       |
  | 0111            | destination register | destination is 0111 which means `r7`     |
  | 0000            | --                   | --                                       |
  | 00000100        | immediate value      | the binary representation of decimal '4' |

  - The blank zeros are also instructions, they are just not used in this
    instruction

  - The condition sequence tells us the instruction should run in all conditions
    not only under certain circumstances

  - The immediate bit (`1`) tells us whether the the value we are accessing is
    contained within the instruction or whether it is stored in a register. In
    this scenario `1` means the value is in the instruction. If it were `0`, the
    register where the value is located would be specified elsewhere in the
    instruction (in one of the currently blank sequences).

  - The opcode correspoons to `mov` the value.

  - THe destination register details where the value should be moved to (`r7`)

  - Finally, the immediate value is equivalent to decimal `4`

  ## Difficulty

  Binary sequences are hard to understand, even if converted to the
  [[Hexadecimal_number_system]]. We have a better way of managing operations on
  the machine level: [assembly](Assembly.md)

raw_content:
  ---
  id: iqh8
  tags:
    - binary
    - CPU
    - hardware
    - computer-architecture
  created: Monday, March 11, 2024
  ---

  # Machine code

  No matter how a program was originally written, it eventually needs to execute
  on a [[CPU_architecture|CPU]] as a series of machine language instructions.

  ## Example machine instruction

  The following machine instruction is an instruction that would be understood by
  an [[Instruction_set_architectures| ARM ISA]]. It moves the number `4` into the
  `r7` register:

  ```
  11100011101000000111000000000100
  ```

  This 32-bit line of binary has a series of instructions embedded within it. We
  partition each part of the sequence below, from left to right, mapping it to the
  instruction:

  | Binary sequence | Instruction          | Action                                   |
  | --------------- | -------------------- | ---------------------------------------- |
  | 1110            | condition            | always execute (unconditional)           |
  | 00              | --                   | --                                       |
  | 1               | immediate            | value is in the last 8 bits              |
  | 1101            | opcode               | `mov` : move the value                   |
  | 0               | --                   | --                                       |
  | 0000            | --                   | --                                       |
  | 0111            | destination register | destination is 0111 which means `r7`     |
  | 0000            | --                   | --                                       |
  | 00000100        | immediate value      | the binary representation of decimal '4' |

  - The blank zeros are also instructions, they are just not used in this
    instruction

  - The condition sequence tells us the instruction should run in all conditions
    not only under certain circumstances

  - The immediate bit (`1`) tells us whether the the value we are accessing is
    contained within the instruction or whether it is stored in a register. In
    this scenario `1` means the value is in the instruction. If it were `0`, the
    register where the value is located would be specified elsewhere in the
    instruction (in one of the currently blank sequences).

  - The opcode correspoons to `mov` the value.

  - THe destination register details where the value should be moved to (`r7`)

  - Finally, the immediate value is equivalent to decimal `4`

  ## Difficulty

  Binary sequences are hard to understand, even if converted to the
  [[Hexadecimal_number_system]]. We have a better way of managing operations on
  the machine level: [assembly](Assembly.md)

metadata:
  {"created":"Monday, March 11, 2024","id":"iqh8","tags":["binary","CPU","hardware","computer-architecture"]}


path: Magnetic_core_memory.md
sortable_path: Magnetic_core_memory.md
title: Magnetic core memory
lead: ![](../img/magnetic-core-memory.jpg)
checksum: 946a0b8cf1656b420be858bad9e76ca1c4974a70b88e00088d16b74e3055d3c4
created: 2025-10-01 13:30:59.726430645+00:00
modified: 2025-08-13 15:57:14.096339251+00:00
metadata: {"created":"Friday, September 27, 2024","tags":["computer-history","memory"]}
body:
  ![](../img/magnetic-core-memory.jpg)

  Like [Magnetic_drum_memory](Magnetic_drum_memory.md), magnetic core memory was
  faster and more reliable than [delay_line_memory](Delay_line_memory.md) and
  vacuum-tubes, and was also persistent. Thus it was used for volatile memory as
  well as storage.

  It worked as follows. Magnetic beads made of ferrite and threaded with copper
  were arranged in a crisscross grid. A bead would be placed at the overlap of two
  copper threads. This constitutes a "core".A pulse of electric current would
  magnetise/de-magnetise the cores.

  ![](../img/magnetic-core-closeup.jpg)

raw_content:
  ---
  tags: [computer-history, memory]
  created: Friday, September 27, 2024
  ---

  # Magnetic core memory

  ![](../img/magnetic-core-memory.jpg)

  Like [Magnetic_drum_memory](Magnetic_drum_memory.md), magnetic core memory was
  faster and more reliable than [delay_line_memory](Delay_line_memory.md) and
  vacuum-tubes, and was also persistent. Thus it was used for volatile memory as
  well as storage.

  It worked as follows. Magnetic beads made of ferrite and threaded with copper
  were arranged in a crisscross grid. A bead would be placed at the overlap of two
  copper threads. This constitutes a "core".A pulse of electric current would
  magnetise/de-magnetise the cores.

  ![](../img/magnetic-core-closeup.jpg)


path: Magnetic_drum_memory.md
sortable_path: Magnetic_drum_memory.md
title: Magnetic drum memory
checksum: 8ca7a01d80bbcbdcb6dca744f5a122187aa786f3c6b523a3c86f86d5ae870f01
created: 2025-10-01 13:30:59.739946204+00:00
modified: 2025-08-13 15:57:14.018623019+00:00
metadata: {"created":"Tuesday, September 24, 2024","tags":["computer-history","memory"]}
lead:
  Along with the [Williams_Tube](Williams_Tube_memory.md), another early approach
  to RAM used in 1950s-1960s era of computing.

body:
  Along with the [Williams_Tube](Williams_Tube_memory.md), another early approach
  to RAM used in 1950s-1960s era of computing.

  ![Magnetic drum](../img/magnetic-drum-memory.jpg)

  A magnetic drum was a metal cylinder coated with a magnetic material. Data was
  stored by magnetising small regions on the drum's surface. The drum would rotate
  at high speeds and read/write heads were positioned along the length of the drum
  to access data.

  Seen as a improvement on Williams Tubes and
  [delay line memory](Delay_line_memory.md) but superseded by magnetic core memory
  later. It's concept lived on in harddisk drives which became the dominant form
  of secondary storage.

  It had a larger capacity than the technologies that preceded it and it was also
  non-volatile - the data would remain intact when the power was turned off.

  It was used in the [IBM 650](The_History_of_Computing_Swade.md) (1953) and
  Ferranti Mark I (1951)

raw_content:
  ---
  tags: [computer-history, memory]
  created: Tuesday, September 24, 2024
  ---

  # Magnetic drum memory

  Along with the [Williams_Tube](Williams_Tube_memory.md), another early approach
  to RAM used in 1950s-1960s era of computing.

  ![Magnetic drum](../img/magnetic-drum-memory.jpg)

  A magnetic drum was a metal cylinder coated with a magnetic material. Data was
  stored by magnetising small regions on the drum's surface. The drum would rotate
  at high speeds and read/write heads were positioned along the length of the drum
  to access data.

  Seen as a improvement on Williams Tubes and
  [delay line memory](Delay_line_memory.md) but superseded by magnetic core memory
  later. It's concept lived on in harddisk drives which became the dominant form
  of secondary storage.

  It had a larger capacity than the technologies that preceded it and it was also
  non-volatile - the data would remain intact when the power was turned off.

  It was used in the [IBM 650](The_History_of_Computing_Swade.md) (1953) and
  Ferranti Mark I (1951)


path: Magnetic_tape.md
sortable_path: Magnetic_tape.md
title: Magnetic tape
checksum: 0b07c8e2462ce875eda2c00c8e49c806621d49ad7318bbd0ef1056ad3d26a6f1
created: 2025-10-01 13:30:59.76198576+00:00
modified: 2025-08-13 15:57:13.985948993+00:00
metadata: {"created":"Monday, September 23, 2024","tags":["computer-history","storage"]}
lead:
  Used for data storage in early digital computers (broadly from the
  [UNIVAC](./The_History_of_Computing_Swade.md)) era until the mini-computer era.

body:
  Used for data storage in early digital computers (broadly from the
  [UNIVAC](./The_History_of_Computing_Swade.md)) era until the mini-computer era.

  While the UNIVAC was not the first to use magnetic tape, it popularised the
  method due to its success and high profile.

  ![Magnetic tape reader in IBM computer](../img/magnetic-tape-ibm.jpeg)

  It worked as follows. The tape was a long, narrow strip of plastic coated with a
  magnetic material. Data is recorded on the the tape by magnetising tiny
  particles on its surface. It was first used in the 1920s for audio recording.

  Data is accessed in a linear, sequential manner. To reach a specific piece of
  data, you often need to wind through all the preceding data

  It was used for long term storage rather than RAM memory due to its sequential
  nature. It stored data and program instructions that would be read into the
  machine's memory. Data was also written out to magenetic tape, in order to store
  and further process results.

  It could store more data than punched cards and its sequential nature
  accommodated batch processing well. However its sequential nature meant it was
  slow at retreiving specific pieces of data as it would have to cycle through all
  the values stored on the tape.

  ![Magnetic tape stored as casette](../img/magnetic-tape-disk.jpeg)

  It developed from open-reel tapes to cartridges and cassettes. It was superseded
  by hard disk drives and solid state drives.

raw_content:
  ---
  tags: [computer-history, storage]
  created: Monday, September 23, 2024
  ---

  # Magnetic tape

  Used for data storage in early digital computers (broadly from the
  [UNIVAC](./The_History_of_Computing_Swade.md)) era until the mini-computer era.

  While the UNIVAC was not the first to use magnetic tape, it popularised the
  method due to its success and high profile.

  ![Magnetic tape reader in IBM computer](../img/magnetic-tape-ibm.jpeg)

  It worked as follows. The tape was a long, narrow strip of plastic coated with a
  magnetic material. Data is recorded on the the tape by magnetising tiny
  particles on its surface. It was first used in the 1920s for audio recording.

  Data is accessed in a linear, sequential manner. To reach a specific piece of
  data, you often need to wind through all the preceding data

  It was used for long term storage rather than RAM memory due to its sequential
  nature. It stored data and program instructions that would be read into the
  machine's memory. Data was also written out to magenetic tape, in order to store
  and further process results.

  It could store more data than punched cards and its sequential nature
  accommodated batch processing well. However its sequential nature meant it was
  slow at retreiving specific pieces of data as it would have to cycle through all
  the values stored on the tape.

  ![Magnetic tape stored as casette](../img/magnetic-tape-disk.jpeg)

  It developed from open-reel tapes to cartridges and cassettes. It was superseded
  by hard disk drives and solid state drives.


path: Mainframe_computers.md
sortable_path: Mainframe_computers.md
title: Mainframe computers
checksum: 75b912ea66c5f6b06475187d504b2c873b6d4f161c78d8aa54a9935bf2a75545
created: 2025-10-01 13:30:59.77474068+00:00
modified: 2025-08-13 15:57:13.960455001+00:00
metadata: {"created":"Friday, September 27, 2024","tags":["computer-history"]}
lead:
  A mainframe computer, also known as a "big iron" is a high performance computer
  used for resource-intensive tasks that require greater security and availability
  than smaller scale machines. They are typically used for centralised rather than
  distributed computing.

body:
  A mainframe computer, also known as a "big iron" is a high performance computer
  used for resource-intensive tasks that require greater security and availability
  than smaller scale machines. They are typically used for centralised rather than
  distributed computing.

  The heyday of mainframes was before the mini- and micro-computer revolution but
  they are still used.

  IBM remains the main manufacturer and service provider for mainframes. Example
  use cases include banks who need to process large volumes of transatction and
  high frequency tranding; government departments, the military and internal
  revenue who require huge databases.

raw_content:
  ---
  tags: [computer-history]
  created: Friday, September 27, 2024
  ---

  # Mainframe computers

  A mainframe computer, also known as a "big iron" is a high performance computer
  used for resource-intensive tasks that require greater security and availability
  than smaller scale machines. They are typically used for centralised rather than
  distributed computing.

  The heyday of mainframes was before the mini- and micro-computer revolution but
  they are still used.

  IBM remains the main manufacturer and service provider for mainframes. Example
  use cases include banks who need to process large volumes of transatction and
  high frequency tranding; government departments, the military and internal
  revenue who require huge databases.


path: Managing_environments_in_NodeJS.md
sortable_path: Managing_environments_in_NodeJS.md
title: Managing environments
lead: With a full-scale Node application you will typically run three environments:
checksum: 036a86541e04215403232e121dc96a105197f73db8d435c9bf4d6f02d68aa280
created: 2025-10-01 13:30:59.779051675+00:00
modified: 2025-08-13 15:57:13.889207649+00:00
metadata: {"tags":["node-js"]}
body:
  With a full-scale Node application you will typically run three environments:

  - Development
  - Testing
  - Production

  ## Accessing the current environment

  To determine the current environment we can use the variable
  **`process.env.NODE_ENV`** from the
  [global process object](Process_object.md).
  This works universally regardless of the kind of Node app we are building.

  If you have not manually set up your environments, **`process.env.NODE_ENV`**
  will return `undefined`.

  ### Setting the Node environment

  #### For the session

  `NODE_ENV` is a bash
  [environment variable](Environmental_and_shell_variables.md)
  like any other. So we can set it in the normal way:

  ```bash
  export NODE_ENV=production
  ```

  ### In Express

  In Express, there is a built in method for retrieving the current envrionment:
  `app.get('env')`. Express will default to the development environment.

  <p style="color:red">! How to keep Express and Node environment in sync?</p>

  ## Configuring environments

  We use the third-party [Config](node-config)
  package to manage different configurations based on the environment.

  Once installed we set up a dedicated config directory with a structure as
  follows:

  ```
  config/
      default.json
      development.json
      production.json
  ```

  For example:

  ```json
  // default.json
  {
    "name": "My Express app"
  }
  ```

  Then to utilise config variables:

  ```js
  const config = require("config");
  console.log("Application name:" + config.get("name"));
  ```

  If we toggled the different environments, we would see different outputs from
  the above code (assuming we had different config files in `/config` with
  different names).

  ### Sensitive config data

  We will need to store passwords, API keys and other kinds of authentication data
  for our application. We obviously shouldn't store this data openly in our config
  files since it would be made public.

  We can do so securely by utilising
  [environmental variables](Environmental_and_shell_variables.md)
  alongside the config pacakage.

  We create a file called `custom-environment-variables` (must be called this) and
  map a property to an environmental environment we have already set.

  Let's create an environmental variable for a password:

  ```bash
  export APP_PASSWORD='mypassword123'
  ```

  Then in our custom variable file:

  ```json
  {
    "password": "APP_PASSWORD"
  }
  ```

  We can then safely reference this value in the course of our normal code:

  ```js
  console.log(config.get("password"));
  ```

  <p style="color:red">! But how would this be achieved in a production server?</p>

  <p style="color:red">! And how could we do this programmatically at the start of a local development session without manually setting each environment variable in the terminal?</p>

raw_content:
  ---
  tags:
    - node-js
  ---

  # Managing environments

  With a full-scale Node application you will typically run three environments:

  - Development
  - Testing
  - Production

  ## Accessing the current environment

  To determine the current environment we can use the variable
  **`process.env.NODE_ENV`** from the
  [global process object](Process_object.md).
  This works universally regardless of the kind of Node app we are building.

  If you have not manually set up your environments, **`process.env.NODE_ENV`**
  will return `undefined`.

  ### Setting the Node environment

  #### For the session

  `NODE_ENV` is a bash
  [environment variable](Environmental_and_shell_variables.md)
  like any other. So we can set it in the normal way:

  ```bash
  export NODE_ENV=production
  ```

  ### In Express

  In Express, there is a built in method for retrieving the current envrionment:
  `app.get('env')`. Express will default to the development environment.

  <p style="color:red">! How to keep Express and Node environment in sync?</p>

  ## Configuring environments

  We use the third-party [Config](node-config)
  package to manage different configurations based on the environment.

  Once installed we set up a dedicated config directory with a structure as
  follows:

  ```
  config/
      default.json
      development.json
      production.json
  ```

  For example:

  ```json
  // default.json
  {
    "name": "My Express app"
  }
  ```

  Then to utilise config variables:

  ```js
  const config = require("config");
  console.log("Application name:" + config.get("name"));
  ```

  If we toggled the different environments, we would see different outputs from
  the above code (assuming we had different config files in `/config` with
  different names).

  ### Sensitive config data

  We will need to store passwords, API keys and other kinds of authentication data
  for our application. We obviously shouldn't store this data openly in our config
  files since it would be made public.

  We can do so securely by utilising
  [environmental variables](Environmental_and_shell_variables.md)
  alongside the config pacakage.

  We create a file called `custom-environment-variables` (must be called this) and
  map a property to an environmental environment we have already set.

  Let's create an environmental variable for a password:

  ```bash
  export APP_PASSWORD='mypassword123'
  ```

  Then in our custom variable file:

  ```json
  {
    "password": "APP_PASSWORD"
  }
  ```

  We can then safely reference this value in the course of our normal code:

  ```js
  console.log(config.get("password"));
  ```

  <p style="color:red">! But how would this be achieved in a production server?</p>

  <p style="color:red">! And how could we do this programmatically at the start of a local development session without manually setting each environment variable in the terminal?</p>


path: Map_and_filter_in_Python.md
sortable_path: Map_and_filter_in_Python.md
title: Map and filter in Python
lead: ## Map
checksum: 657b2624c0782eaa5f2b92a064f41d79185fc42eef1be5b4c8ab202f87745c76
created: 2025-10-01 13:30:59.802229552+00:00
modified: 2025-08-13 15:57:13.977975299+00:00
metadata: {"tags":["python"]}
body:
  ## Map

  ```py
  data = [1, 3, 5, 2, 7, 4, 10]

  new_data = map(lambda i: i + 10, data)
  print(new_data)
  ```

  We can also pass-in a function rather than use a lambda:

  ```py
  def add_one(i):
      return i + 1

  x = list(map(addOne, data))
  # [2, 4, 6, 3, 8, 5, 11]
  # necessary to add `list` to get some output
  ```

  ### Map returns an object

  When you run `map` against a list it returns a map object not a list. This can
  present problems if you are not aware of it since it is not an entity that you
  can loop through.

  The following map removes whitespaces from elements in a list:

  ```py
    lines = map(lambda x: x.strip(), lines)
  ```

  In order to be able to run list-like operations against the resulting `lines`
  object we would need to convert it to a list:

  ```py
    lines = list(map(lambda x: x.strip(), lines))
  ```

  ## Filter

  ```py
  data = [1, 3, 5, 2, 7, 4, 10]

  d1 = list(filter(lambda i: i % 2 == 0, data))
  print(d1)
  #  [2, 4, 10]

  def is_even(i):
      return i % 2 == 0


  # Filter for even numbers using a named function
  d2 = list(filter(is_even, data))

  # [2, 4, 10]
  ```

  ## Chaining

  ```py
  data = [1, 3, 5, 2, 7, 4, 10]
  new_data = list(map(lambda i: i + 10, filter(is_even, data)))
  print(new_data)
  # new_data: [12, 14, 20]
  ```

raw_content:
  ---
  tags: [python]
  ---

  # Map and filter in Python

  ## Map

  ```py
  data = [1, 3, 5, 2, 7, 4, 10]

  new_data = map(lambda i: i + 10, data)
  print(new_data)
  ```

  We can also pass-in a function rather than use a lambda:

  ```py
  def add_one(i):
      return i + 1

  x = list(map(addOne, data))
  # [2, 4, 6, 3, 8, 5, 11]
  # necessary to add `list` to get some output
  ```

  ### Map returns an object

  When you run `map` against a list it returns a map object not a list. This can
  present problems if you are not aware of it since it is not an entity that you
  can loop through.

  The following map removes whitespaces from elements in a list:

  ```py
    lines = map(lambda x: x.strip(), lines)
  ```

  In order to be able to run list-like operations against the resulting `lines`
  object we would need to convert it to a list:

  ```py
    lines = list(map(lambda x: x.strip(), lines))
  ```

  ## Filter

  ```py
  data = [1, 3, 5, 2, 7, 4, 10]

  d1 = list(filter(lambda i: i % 2 == 0, data))
  print(d1)
  #  [2, 4, 10]

  def is_even(i):
      return i % 2 == 0


  # Filter for even numbers using a named function
  d2 = list(filter(is_even, data))

  # [2, 4, 10]
  ```

  ## Chaining

  ```py
  data = [1, 3, 5, 2, 7, 4, 10]
  new_data = list(map(lambda i: i + 10, filter(is_even, data)))
  print(new_data)
  # new_data: [12, 14, 20]
  ```


path: Map_and_filter_to_reduce_in_JS.md
sortable_path: Map_and_filter_to_reduce_in_JS.md
title: Expressing combined map and filter operations to a single reduce
lead: ## `map` + `filter`
checksum: c55ae1785e85ae7113094c8c7f48a7190f3e0eb2cec9022f07c79701ce3fea0a
created: 2025-10-01 13:30:59.806826412+00:00
modified: 2025-08-13 15:57:13.809806098+00:00
metadata: {"tags":["javascript"]}
body:
  ## `map` + `filter`

  The following combines a `map` and a `filter` to square a list of numbers and
  then filter the ones that are greater than or equal to 10:

  ```js
  const arr = [1, 2, 3, 4];

  const filteredSquaredNumbers = arr.map((x) => x * x).filter((y) => y >= 10);
  ```

  Using `reduce` we can combine these operations more concisely:

  ```js
  const filteredSquaredNumbers = arr.reduce((acc, x) => {
    const squared = x * x;
    if (squared >= 10) {
      acc.push(squared);
    }
    return acc;
  }, []);
  ```

  ## `filter` + `map`

  Here we use a filter and then a map to filter the even numbers and then double
  them:

  ```js
  const arr = [1, 2, 3, 4];

  const evenDoubled = arr.filter((x) => x % 2 === 0).map((y) => y * 2);
  ```

  Using `reduce` :

  ```js
  const evenDoubled = arr.reduce((acc, x) => {
    if (x % 2 === 0) {
      acc.push(x * 2);
    }
    return acc;
  }, []);
  ```

  > The empty array [] is the initial value of the accumulator. As the callback
  > function processes each element of the array, it can add or modify the
  > accumulator based on the desired logic. By providing an empty array as the
  > initial value for the accumulator, we ensure that the results will be
  > collected in a new array without modifying the original array.

raw_content:
  ---
  tags: [javascript]
  ---

  # Expressing combined map and filter operations to a single reduce

  ## `map` + `filter`

  The following combines a `map` and a `filter` to square a list of numbers and
  then filter the ones that are greater than or equal to 10:

  ```js
  const arr = [1, 2, 3, 4];

  const filteredSquaredNumbers = arr.map((x) => x * x).filter((y) => y >= 10);
  ```

  Using `reduce` we can combine these operations more concisely:

  ```js
  const filteredSquaredNumbers = arr.reduce((acc, x) => {
    const squared = x * x;
    if (squared >= 10) {
      acc.push(squared);
    }
    return acc;
  }, []);
  ```

  ## `filter` + `map`

  Here we use a filter and then a map to filter the even numbers and then double
  them:

  ```js
  const arr = [1, 2, 3, 4];

  const evenDoubled = arr.filter((x) => x % 2 === 0).map((y) => y * 2);
  ```

  Using `reduce` :

  ```js
  const evenDoubled = arr.reduce((acc, x) => {
    if (x % 2 === 0) {
      acc.push(x * 2);
    }
    return acc;
  }, []);
  ```

  > The empty array [] is the initial value of the accumulator. As the callback
  > function processes each element of the array, it can add or modify the
  > accumulator based on the desired logic. By providing an empty array as the
  > initial value for the accumulator, we ensure that the results will be
  > collected in a new array without modifying the original array.


path: Mapped_types_in_TS.md
sortable_path: Mapped_types_in_TS.md
title: Mapped types in TypeScript
lead:
  A mapped type is a way to create new types based on existing ones by
  transforming their properties

checksum: ccabedb3387b7583fb0265e8e625b02b3cfe9ddc04875240fe8f29b61a4fab8e
created: 2025-10-01 13:30:59.81080983+00:00
modified: 2025-08-13 15:57:13.961044297+00:00
metadata: {"tags":["typescript"]}
body:
  A mapped type is a way to create new types based on existing ones by
  transforming their properties

  As it works on a single type and typically narrows or limits the properties it
  works in a way that is opposite to a [union]() or [intersection]() type.

  > Mapped types are an example of when `type` is preferable to `interface`, since
  > you cannot generate mapped types as interfaces, although you may generate a
  > mapped type from an interface.

  For the demonstrations we will use the following type as the example that we
  will map from:

  ```ts
  type Person = {
    name: string;
    age: number;
    city: string;
    country: string;
  };
  ```

  ## Main varieties of mapped types

  ### Read only

  Creates a type with all properties of the given type set to `readonly`:

  ```ts
  type Readonly<T> = {
    readonly [P in keyof T]: T[P];
  };

  type ReadonlyPerson = Readonly<Person>;
  ```

  This is equivalent to:

  ```ts
  type ReadonlyPerson = {
    readonly name: string;
    readonly age: number;
    readonly city: string;
    readonly country: string;
  };
  ```

  ### Partial

  Creates a type with all properties of the given type set to optional.

  ```ts
  type Partial<T> = {
    [P in keyof T]?: T[P];
  };

  type PartialPerson = Partial<Person>;
  ```

  This is equivalent to:

  ```ts
  type PartialPerson = {
    name?: string;
    age?: number;
    city?: string;
    country?: string;
  };
  ```

  ### Pick

  This is useful when you want to create a new type based on a subset of
  properties from an existing type.

  ```ts
  type Pick<T, K extends keyof T> = {
    [P in K]: T[P];
  };

  type PersonNameAndAge = Pick<Person, "name" | "age">;
  ```

  This is equivalent to:

  ```ts
  type PersonNameAndAge = {
    name: string;
    age: number;
  };
  ```

  ### Record

  Creates a type with keys of the given type and values of the specified type. It
  is a way of shoehorning keys from an existing type with new values.

  Basic syntax:

  ```ts
  Record<Keys, ValueType>;
  ```

  Example:

  ```ts
  type UserRole = "admin" | "user" | "guest";

  // Create a Record type to store user role descriptions
  type UserRoleDescriptions = Record<UserRole, string>;

  // Create an object that implements the UserRoleDescriptions type
  const roleDescriptions: UserRoleDescriptions = {
    admin: "Has full access to the system",
    user: "Can access limited resources",
    guest: "Can view public resources only",
  };
  ```

  Here is a simpler example

  ```ts
  type Person = {
    name: string;
    nationality: string;
  };

  type PersonInstance = Record<string, Person>;
  const thomasInstance: PersonInstance = {
    thomas: {
      name: "Thomas",
      nationality: "British",
    },
  };
  ```

  ### Exclude

  Creates a type by excluding specific properties from the given type.

  ```ts
  type Exclude<T, U> = T extends U ? never : T;

  type KeysWithoutAge = Exclude<keyof Person, "age">;
  ```

  This is equivalent to:

  ```ts
  type KeysWithoutAge = "name" | "city" | "country";
  ```

  ### Extract

  Creates a type by extracting specific properties from the given type. Basically
  the opposite operation to Exclude.

  ```ts
  type Extract<T, U> = T extends U ? T : never;

  type NameKey = Extract<keyof Person, "name">;
  ```

  This is equivalent to:

  ```ts
  type NameKey = "name";
  ```

  ### Non-nullable

  Creates a type by removing null and undefined from the given type.

  ```ts
  type NonNullable<T> = T extends null | undefined ? never : T;

  type NullablePerson = {
    name: string | null;
    age: number | null;
    city: string | null;
    country: string | null;
  };

  type NonNullablePerson = NonNullable<NullablePerson>;
  ```

  Equivalent to:

  ```ts
  type NonNullablePerson = {
    name: string;
    age: number;
    city: string;
    country: string;
  };
  ```

raw_content:
  ---
  tags:
    - typescript
  ---

  # Mapped types in TypeScript

  A mapped type is a way to create new types based on existing ones by
  transforming their properties

  As it works on a single type and typically narrows or limits the properties it
  works in a way that is opposite to a [union]() or [intersection]() type.

  > Mapped types are an example of when `type` is preferable to `interface`, since
  > you cannot generate mapped types as interfaces, although you may generate a
  > mapped type from an interface.

  For the demonstrations we will use the following type as the example that we
  will map from:

  ```ts
  type Person = {
    name: string;
    age: number;
    city: string;
    country: string;
  };
  ```

  ## Main varieties of mapped types

  ### Read only

  Creates a type with all properties of the given type set to `readonly`:

  ```ts
  type Readonly<T> = {
    readonly [P in keyof T]: T[P];
  };

  type ReadonlyPerson = Readonly<Person>;
  ```

  This is equivalent to:

  ```ts
  type ReadonlyPerson = {
    readonly name: string;
    readonly age: number;
    readonly city: string;
    readonly country: string;
  };
  ```

  ### Partial

  Creates a type with all properties of the given type set to optional.

  ```ts
  type Partial<T> = {
    [P in keyof T]?: T[P];
  };

  type PartialPerson = Partial<Person>;
  ```

  This is equivalent to:

  ```ts
  type PartialPerson = {
    name?: string;
    age?: number;
    city?: string;
    country?: string;
  };
  ```

  ### Pick

  This is useful when you want to create a new type based on a subset of
  properties from an existing type.

  ```ts
  type Pick<T, K extends keyof T> = {
    [P in K]: T[P];
  };

  type PersonNameAndAge = Pick<Person, "name" | "age">;
  ```

  This is equivalent to:

  ```ts
  type PersonNameAndAge = {
    name: string;
    age: number;
  };
  ```

  ### Record

  Creates a type with keys of the given type and values of the specified type. It
  is a way of shoehorning keys from an existing type with new values.

  Basic syntax:

  ```ts
  Record<Keys, ValueType>;
  ```

  Example:

  ```ts
  type UserRole = "admin" | "user" | "guest";

  // Create a Record type to store user role descriptions
  type UserRoleDescriptions = Record<UserRole, string>;

  // Create an object that implements the UserRoleDescriptions type
  const roleDescriptions: UserRoleDescriptions = {
    admin: "Has full access to the system",
    user: "Can access limited resources",
    guest: "Can view public resources only",
  };
  ```

  Here is a simpler example

  ```ts
  type Person = {
    name: string;
    nationality: string;
  };

  type PersonInstance = Record<string, Person>;
  const thomasInstance: PersonInstance = {
    thomas: {
      name: "Thomas",
      nationality: "British",
    },
  };
  ```

  ### Exclude

  Creates a type by excluding specific properties from the given type.

  ```ts
  type Exclude<T, U> = T extends U ? never : T;

  type KeysWithoutAge = Exclude<keyof Person, "age">;
  ```

  This is equivalent to:

  ```ts
  type KeysWithoutAge = "name" | "city" | "country";
  ```

  ### Extract

  Creates a type by extracting specific properties from the given type. Basically
  the opposite operation to Exclude.

  ```ts
  type Extract<T, U> = T extends U ? T : never;

  type NameKey = Extract<keyof Person, "name">;
  ```

  This is equivalent to:

  ```ts
  type NameKey = "name";
  ```

  ### Non-nullable

  Creates a type by removing null and undefined from the given type.

  ```ts
  type NonNullable<T> = T extends null | undefined ? never : T;

  type NullablePerson = {
    name: string | null;
    age: number | null;
    city: string | null;
    country: string | null;
  };

  type NonNullablePerson = NonNullable<NullablePerson>;
  ```

  Equivalent to:

  ```ts
  type NonNullablePerson = {
    name: string;
    age: number;
    city: string;
    country: string;
  };
  ```


path: Maps_and_sets_in_JS.md
sortable_path: Maps_and_sets_in_JS.md
title: Maps and Sets
checksum: 391de8d7c7b97236ff25a0ce78fc54bd629b9d1010dd5c1ed91b6de808339bac
created: 2025-10-01 13:30:59.816595877+00:00
modified: 2025-08-13 15:57:14.055004435+00:00
metadata: {"tags":["javascript"]}
lead:
  > Both `Map` and `Set` are new data structures introduced with ES6. They combine
  > the properties of the pre-existing data structures: arrays and objects and
  > offer advantages in certain non-typical use cases.

body:
  > Both `Map` and `Set` are new data structures introduced with ES6. They combine
  > the properties of the pre-existing data structures: arrays and objects and
  > offer advantages in certain non-typical use cases.

  ## Maps

  ### Concepts

  A map (not to be confused with the array map method) introduces something like
  the associative array model to JavaScript. Prior to ES6 the closest thing JS had
  to this was an object.

  Maps are like objects in that their structure consists in key/map pairs but like
  arrays in that the order in which the entries are placed is retained and can be
  referred to throughout the lifecycle of the Map. Also, the length of the Map is
  a known and easily returnable quantity, with objects we have to do some
  wrangling with `Object.keys` and `Object.entries` to retrieve meta data about
  the object itself. Moreover, with an object the order in which the keys are
  stored is not recorded.

  At the same time Maps are different from arrays because the index of an array
  element is not unique, it will always be a number of the form `[0], [1], [2]...`
  .

  Finally, the key can be any data type not just a string. This is an important
  characteristic of maps because although you can use numbers or booleans as the
  keys in an object, they are not actually numbers, they are converted to strings
  at execution. With Maps, whatever data type you assign as a key, retains its
  type.

  Maps

  - Have key/value pair structure
  - Have length property
  - Order of entries and insertions is maintained
  - Can use any data type as a key
  - Disallow duplicate keys

  ### Creating a Map and adding entries

  ```jsx
  const map = new Map();

  // We then pass in key/value pairs using `set`:
  map.set("firstName", "Luke");
  map.set("lastName", "Skywalker");
  map.set("occupation", "Jedi Knight");
  ```

  There is an alternative method for defining a map:

  ```jsx
  const map = new Map([
    ["firstName", "Luke"],
    ["lastName", "Skywalker"],
    ["occupation", "Jedi Knight"],
  ]);
  ```

  Internally, this data will be represented as follows:

  ```
  Map(3)
  0: {"firstName" => "Luke"}
  1: {"lastName" => "Skywalker"}
  2: {"occupation" => "Jedi Knight"}
  ```

  ### Working with map elements

  ```jsx
  // check for the existence of an item in a map by searching for existence of key
  map.has("firstName"); // return boolean

  // return all keys in a Map as iterator
  map.keys(); // MapIterator {'firstName', 'lastName', 'occupation'}

  // return all values as iterator
  map.values();

  // return array of key/value pairs
  map.entries(); // {"firstName" => "Luke", "lastName" => "Skywalker", "occupation" => "Jedi Knight"}
  // retrieve an item from a Map
  map.get("lastName"); // Skywalker

  // delete an item from a map by key, returns boolean as confirmation
  map.delete("occupation"); // true

  //return the length of a map (just like array.length)
  map.size; // 3
  // clear all values
  map.clear();
  ```

  ### Looping

  ```jsx
  // Map has its own .forEach method, like with arrays. Hewre they are side by side

  // Map
  Map.prototype.forEach((value, key, map) = () => {})

  // Array
  Array.prototype.forEach((item, index, array) = () => {})

  // You can also use a for of loop
  // Destructure the key and value out of the Map item
  for (const [key, value] of map) {
    // Log the keys and values of the Map with for...of
    console.log(`${key}: ${value}`)
  }
  ```

  ### Interfacing between Maps and other reference types

  Given that we can create a map with both object and array-style syntax, its easy
  to convert maps to and from the more established data structures.

  It's easy to convert an object to a map:

  ```jsx
  const luke = {
    firstName: "Luke",
    lastName: "Skywalker",
    occupation: "Jedi Knight",
  };

  const map = new Map(Object.entries(luke));
  ```

  And a map to an object:

  ```jsx
  const obj = Object.fromEntries(map);
  ```

  A map to an array:

  ```jsx
  const arr = Array.from(map);
  ```

  We saw earlier that a Map is a multidimensional array, so any multidimensional
  array can be used in the Map constructor invocation to turn it into a Map.

  ## Sets

  A set is special kind of array, an atypical supplement to arrays. It is
  basically an array of unique values. If you try to add a duplicate entry to a
  set it will be ignored.

  ```jsx
  const set = new Set();
  set.add("Beethoven");
  set.add("Bach");

  // Or:
  const set = new Set(["Beethoven", "Bach"]);

  // Convert array to set (note, this will remove duplicates on initialisation)
  const arr = [...set];
  ```

  We can use `keys()`, `values()`, and `entries()` with sets but obviously, `keys`
  and `values` will be identical.

raw_content:
  ---
  tags: [javascript]
  ---

  # Maps and Sets

  > Both `Map` and `Set` are new data structures introduced with ES6. They combine
  > the properties of the pre-existing data structures: arrays and objects and
  > offer advantages in certain non-typical use cases.

  ## Maps

  ### Concepts

  A map (not to be confused with the array map method) introduces something like
  the associative array model to JavaScript. Prior to ES6 the closest thing JS had
  to this was an object.

  Maps are like objects in that their structure consists in key/map pairs but like
  arrays in that the order in which the entries are placed is retained and can be
  referred to throughout the lifecycle of the Map. Also, the length of the Map is
  a known and easily returnable quantity, with objects we have to do some
  wrangling with `Object.keys` and `Object.entries` to retrieve meta data about
  the object itself. Moreover, with an object the order in which the keys are
  stored is not recorded.

  At the same time Maps are different from arrays because the index of an array
  element is not unique, it will always be a number of the form `[0], [1], [2]...`
  .

  Finally, the key can be any data type not just a string. This is an important
  characteristic of maps because although you can use numbers or booleans as the
  keys in an object, they are not actually numbers, they are converted to strings
  at execution. With Maps, whatever data type you assign as a key, retains its
  type.

  Maps

  - Have key/value pair structure
  - Have length property
  - Order of entries and insertions is maintained
  - Can use any data type as a key
  - Disallow duplicate keys

  ### Creating a Map and adding entries

  ```jsx
  const map = new Map();

  // We then pass in key/value pairs using `set`:
  map.set("firstName", "Luke");
  map.set("lastName", "Skywalker");
  map.set("occupation", "Jedi Knight");
  ```

  There is an alternative method for defining a map:

  ```jsx
  const map = new Map([
    ["firstName", "Luke"],
    ["lastName", "Skywalker"],
    ["occupation", "Jedi Knight"],
  ]);
  ```

  Internally, this data will be represented as follows:

  ```
  Map(3)
  0: {"firstName" => "Luke"}
  1: {"lastName" => "Skywalker"}
  2: {"occupation" => "Jedi Knight"}
  ```

  ### Working with map elements

  ```jsx
  // check for the existence of an item in a map by searching for existence of key
  map.has("firstName"); // return boolean

  // return all keys in a Map as iterator
  map.keys(); // MapIterator {'firstName', 'lastName', 'occupation'}

  // return all values as iterator
  map.values();

  // return array of key/value pairs
  map.entries(); // {"firstName" => "Luke", "lastName" => "Skywalker", "occupation" => "Jedi Knight"}
  // retrieve an item from a Map
  map.get("lastName"); // Skywalker

  // delete an item from a map by key, returns boolean as confirmation
  map.delete("occupation"); // true

  //return the length of a map (just like array.length)
  map.size; // 3
  // clear all values
  map.clear();
  ```

  ### Looping

  ```jsx
  // Map has its own .forEach method, like with arrays. Hewre they are side by side

  // Map
  Map.prototype.forEach((value, key, map) = () => {})

  // Array
  Array.prototype.forEach((item, index, array) = () => {})

  // You can also use a for of loop
  // Destructure the key and value out of the Map item
  for (const [key, value] of map) {
    // Log the keys and values of the Map with for...of
    console.log(`${key}: ${value}`)
  }
  ```

  ### Interfacing between Maps and other reference types

  Given that we can create a map with both object and array-style syntax, its easy
  to convert maps to and from the more established data structures.

  It's easy to convert an object to a map:

  ```jsx
  const luke = {
    firstName: "Luke",
    lastName: "Skywalker",
    occupation: "Jedi Knight",
  };

  const map = new Map(Object.entries(luke));
  ```

  And a map to an object:

  ```jsx
  const obj = Object.fromEntries(map);
  ```

  A map to an array:

  ```jsx
  const arr = Array.from(map);
  ```

  We saw earlier that a Map is a multidimensional array, so any multidimensional
  array can be used in the Map constructor invocation to turn it into a Map.

  ## Sets

  A set is special kind of array, an atypical supplement to arrays. It is
  basically an array of unique values. If you try to add a duplicate entry to a
  set it will be ignored.

  ```jsx
  const set = new Set();
  set.add("Beethoven");
  set.add("Bach");

  // Or:
  const set = new Set(["Beethoven", "Bach"]);

  // Convert array to set (note, this will remove duplicates on initialisation)
  const arr = [...set];
  ```

  We can use `keys()`, `values()`, and `entries()` with sets but obviously, `keys`
  and `values` will be identical.


path: Match_statements_in_Python.md
sortable_path: Match_statements_in_Python.md
title: Match statements in Python
lead: > A `match` statement is the equivalent of a switch or case statement in Python
checksum: 4eca8abdc2e26149bf6e163394aa1e01aaef771d47d653884dac8ec213313c40
created: 2025-10-01 13:30:59.823509199+00:00
modified: 2025-08-13 15:57:14.082596021+00:00
metadata: {"tags":["python"]}
body:
  > A `match` statement is the equivalent of a switch or case statement in Python

  ```python
  command = input("What are you doing next? ")

  match command:
      case "quit":
          print("Goodbye!")
      case "look":
          print("Looking out")
      case "up" | "down":
          print("up or down")
      case _:
          print("The default")

  """
  What are you doing next? up
  up or down
  """

  match command.split():
      case ["go", "left"]:
          print("go left")
      case ["go", ("fast" | "slow")]:
          print("go fast or slow")

  point = (3, 3)
  match point:
      case (x, y) if x == y:
          print(f"The point is located on the diagonal Y=X at {x}.")
      case (x, y):
          print(f"Point is not on the diagonal.")
  """
  The point is located on the diagonal Y=X at 3.
  """
  ```

raw_content:
  ---
  tags: [python]
  ---

  # Match statements in Python

  > A `match` statement is the equivalent of a switch or case statement in Python

  ```python
  command = input("What are you doing next? ")

  match command:
      case "quit":
          print("Goodbye!")
      case "look":
          print("Looking out")
      case "up" | "down":
          print("up or down")
      case _:
          print("The default")

  """
  What are you doing next? up
  up or down
  """

  match command.split():
      case ["go", "left"]:
          print("go left")
      case ["go", ("fast" | "slow")]:
          print("go fast or slow")

  point = (3, 3)
  match point:
      case (x, y) if x == y:
          print(f"The point is located on the diagonal Y=X at {x}.")
      case (x, y):
          print(f"Point is not on the diagonal.")
  """
  The point is located on the diagonal Y=X at 3.
  """
  ```


path: Matter_and_atoms.md
sortable_path: Matter_and_atoms.md
title: Matter and atoms
lead: ## Matter, elements, compounds and mixtures
checksum: 3f5628d93b77e4acadfd3a283c09d933482db0d3e12f6dda0d3487836d3b4174
created: 2025-10-01 13:30:59.828894356+00:00
modified: 2025-08-13 15:57:13.867104592+00:00
metadata: {"tags":["physics"]}
body:
  ## Matter, elements, compounds and mixtures

  - Matter is anything that occupies space and has weight. It may be found in any
    one of three states: liquid, solid, or gas.
  - Matter can be classed in terms of **elements**, **compounds** and
    **mixtures**.
    - An element is a substance that cannot be reduced to a simpler substance by
      chemical means.
    - The chemical combination of two or more elements is a compound. Compounds
      _can_ be separated or reduced by chemical means but they cannot be
      physically separated.
    - A _physical_ (note not _chemical_) combination of elements and compounds is
      a mixture.
    - For example "air" is a mixture which is made up of oxygen, nitrogen, carbon
      dioxide, and other gases.

  ## Atoms and molecules

  - Elements are made up of **atoms**.
  - A **molecule** is a type of compound where the atoms of two types of elements
    have joined together. For example sodium and chlorine are two elements that
    can join to form sodium chloride. We call this **chemical bonding**.

  ## Atomic particles
  ![](/img/atom-diagram.svg)



  - There are three particles comprising atoms: **protons**, **neutrons** and
    **electrons**.

  - Protons and neutrons comprise the nucleus of the atom. A proton is positively
    charged. A neutron has no charge.

  - Electrons orbit around the nucleus in concentric rings. Electrons are
    negatively charged. Because of this protons and electrons attract each other.

  ### Atomic number

  - The number of protons in the nucleus is called the element's **atomic**
    number.
  - We use the atomic number of elements to distinguish one element from another.

  ### Atomic weight

  - The atomic weight of an atom is its mass.
  - The mass is determined by the number of protons and neutrons in the nucleus.
  - An electron $\frac{1}{1845}$ the mass of a proton. Its mass is so neglible
    that it makes no significant difference to the atomic weight.

raw_content:
  ---
  tags: [physics]
  ---

  # Matter and atoms

  ## Matter, elements, compounds and mixtures

  - Matter is anything that occupies space and has weight. It may be found in any
    one of three states: liquid, solid, or gas.
  - Matter can be classed in terms of **elements**, **compounds** and
    **mixtures**.
    - An element is a substance that cannot be reduced to a simpler substance by
      chemical means.
    - The chemical combination of two or more elements is a compound. Compounds
      _can_ be separated or reduced by chemical means but they cannot be
      physically separated.
    - A _physical_ (note not _chemical_) combination of elements and compounds is
      a mixture.
    - For example "air" is a mixture which is made up of oxygen, nitrogen, carbon
      dioxide, and other gases.

  ## Atoms and molecules

  - Elements are made up of **atoms**.
  - A **molecule** is a type of compound where the atoms of two types of elements
    have joined together. For example sodium and chlorine are two elements that
    can join to form sodium chloride. We call this **chemical bonding**.

  ## Atomic particles
  ![](/img/atom-diagram.svg)



  - There are three particles comprising atoms: **protons**, **neutrons** and
    **electrons**.

  - Protons and neutrons comprise the nucleus of the atom. A proton is positively
    charged. A neutron has no charge.

  - Electrons orbit around the nucleus in concentric rings. Electrons are
    negatively charged. Because of this protons and electrons attract each other.

  ### Atomic number

  - The number of protons in the nucleus is called the element's **atomic**
    number.
  - We use the atomic number of elements to distinguish one element from another.

  ### Atomic weight

  - The atomic weight of an atom is its mass.
  - The mass is determined by the number of protons and neutrons in the nucleus.
  - An electron $\frac{1}{1845}$ the mass of a proton. Its mass is so neglible
    that it makes no significant difference to the atomic weight.


path: Memoization_in_React.md
sortable_path: Memoization_in_React.md
title: Memoization with useCallback and useMemo
lead: ## Rationale
checksum: aa5a1396de6830c65cebe440419eb402597ac57d198d06fa5c5fe6cea81ab4f9
created: 2025-10-01 13:30:59.833883869+00:00
modified: 2025-08-13 15:57:13.824323627+00:00
metadata: {"tags":["javascript","react"]}
body:
  ## Rationale

  In the lifecycle of a component, React re-renders the component when an update
  is made. When React checks for any changes in a component, it may detect an
  unintended or unexpected change due to how JavaScript handles equality and
  shallow comparisons. This change in the React application will cause it to
  re-render unnecessarily.

  Additionally, if that re-rendering is an expensive operation, like a long for
  loop, it can hurt performance. Expensive operations can be costly in either
  time, memory, or processing. In addition to potential technical issues, this may
  lead to poor user experience.

  If one part re-renders, it re-renders the entire component tree. `useCallback`
  and `useMemo` can be used reduce this performance impact.

  ## useCallback

  The `useCallback` hook is used to wrap functions. It tells React to not
  re-create a wrapped function when a component re-renders, unless any of the
  useCallback's dependencies change.

  `useCallback` returns a memoized version of the callback function it is passed.
  This means that the function object returned from useCallback will be the same
  between re-renders.

  Remember that in JavaScript, functions are objects and components are functions.
  As a result, every time a component containing a function re-renders, it create
  a new instance of the function in memory.

  > Given the same dependency value, the `useCallback` hook returns the same
  > function instance between renderings (aka memoization).

  This said, for small functions that are not intensive, it doesn't really matter
  if they are not memoized.

  ### Syntax

  A standard case of this would be a function that runs on a button click, for
  instance when sending data from a form to a server. In the example below there
  is quite a lot going on, and most of it is independent of the actual UI-cycle of
  the component.

  ```jsx
  const handleSubmit = useCallback(
    async (formValues) => {
      setPendSaveConfig(true);
      const payload = new GenerateConfig({
        workflowId: project_id,
        blockId: blockId,
        config: formValues,
      });
      axios
        .post(`${process.env.REACT_APP_ENDPOINT}/save-block-config`, payload)
        .then((res) => console.log(res))
        .finally(() => setPendSaveConfig(false))
        .catch((err) => console.error(err));
    },
    [blockId, project_id]
  );
  ```

  Note that the syntax is similar to [React_useEffect](React_useEffect.md): there is a
  dependency array. The effect is the same: the function contained within
  `useCallback` will only re-rendered if one of these dependencies changes.
  However (see next section) the function will run in its memoized form on every
  click.

  ### Reference versus result

  `useCallback` only memoizes the function _object_ (the reference) not the value
  that is _returned_ by the function (the result).

  In the example below, the `calculatePi()` function reference will not change
  between renders but each time the click event fires, the value returned by
  `calculatePi()` will change. In other words, it will be assigned fresh memory.

  ```jsx
  function ParentComponent() {
    const onHandleClick = useCallback(() => {
      const special = calculatePi();
    });

    return <SubComponent handleClick={onHandleClick} />;
  }
  ```

  ### Use cases

  You should not apply `useCallback` in a blanket fashion, this can reduce
  performance rather than improve it. The best scenarios are:

  1. A functional component wrapped inside React.memo() accepts a function object
     prop

  2. When the function object is a dependency to other hooks, e.g.
     `useEffect(..., [callback])`

  3. When the function has some internal state, e.g. when the function is
     debounced or throttled.

  ## useMemo

  We can think of `useMemo` as a complement to `useCallback`. The main difference
  is that whilst `useCallback` memoizes the function reference, `useMemo` memoizes
  the function result; the value that is returned.

  > In memoization, the result is “remembered” when the same parameters are
  > passed-in subsequently. If we have a function compute 1 + 1, it will return 2.
  > But if it uses memoization, the next time we run 1’s through the function, it
  > won’t add them up; it will just remember the answer is 2 without executing the
  > adding function.

  Like `useCallback`, `useMemo` takes a function and an array of dependencies for
  the same reason: if one of the dependencies changes then the function will
  recalculate (and re-memoize) the result but they don't the same memoized value
  will be returned.

  As with `useCallback`, `useMemo` is best applied to complex functions that are
  computationally expensive, it shouldn't be used indiscriminately.

  It should not be used with asynchronous functions, in these instances,
  `useEffect` would be a better choice.

  ### Syntax

  ```jsx
  const List = React.useMemo(
    () =>
      listOfItems.map((item) => ({
        ...item,
        itemProp1: expensiveFunction(props.first),
        itemProp2: anotherPriceyFunction(props.second),
      })),
    [listOfItems]
  );
  ```

raw_content:
  ---
  tags:
    - javascript
    - react
  ---

  # Memoization with useCallback and useMemo

  ## Rationale

  In the lifecycle of a component, React re-renders the component when an update
  is made. When React checks for any changes in a component, it may detect an
  unintended or unexpected change due to how JavaScript handles equality and
  shallow comparisons. This change in the React application will cause it to
  re-render unnecessarily.

  Additionally, if that re-rendering is an expensive operation, like a long for
  loop, it can hurt performance. Expensive operations can be costly in either
  time, memory, or processing. In addition to potential technical issues, this may
  lead to poor user experience.

  If one part re-renders, it re-renders the entire component tree. `useCallback`
  and `useMemo` can be used reduce this performance impact.

  ## useCallback

  The `useCallback` hook is used to wrap functions. It tells React to not
  re-create a wrapped function when a component re-renders, unless any of the
  useCallback's dependencies change.

  `useCallback` returns a memoized version of the callback function it is passed.
  This means that the function object returned from useCallback will be the same
  between re-renders.

  Remember that in JavaScript, functions are objects and components are functions.
  As a result, every time a component containing a function re-renders, it create
  a new instance of the function in memory.

  > Given the same dependency value, the `useCallback` hook returns the same
  > function instance between renderings (aka memoization).

  This said, for small functions that are not intensive, it doesn't really matter
  if they are not memoized.

  ### Syntax

  A standard case of this would be a function that runs on a button click, for
  instance when sending data from a form to a server. In the example below there
  is quite a lot going on, and most of it is independent of the actual UI-cycle of
  the component.

  ```jsx
  const handleSubmit = useCallback(
    async (formValues) => {
      setPendSaveConfig(true);
      const payload = new GenerateConfig({
        workflowId: project_id,
        blockId: blockId,
        config: formValues,
      });
      axios
        .post(`${process.env.REACT_APP_ENDPOINT}/save-block-config`, payload)
        .then((res) => console.log(res))
        .finally(() => setPendSaveConfig(false))
        .catch((err) => console.error(err));
    },
    [blockId, project_id]
  );
  ```

  Note that the syntax is similar to [React_useEffect](React_useEffect.md): there is a
  dependency array. The effect is the same: the function contained within
  `useCallback` will only re-rendered if one of these dependencies changes.
  However (see next section) the function will run in its memoized form on every
  click.

  ### Reference versus result

  `useCallback` only memoizes the function _object_ (the reference) not the value
  that is _returned_ by the function (the result).

  In the example below, the `calculatePi()` function reference will not change
  between renders but each time the click event fires, the value returned by
  `calculatePi()` will change. In other words, it will be assigned fresh memory.

  ```jsx
  function ParentComponent() {
    const onHandleClick = useCallback(() => {
      const special = calculatePi();
    });

    return <SubComponent handleClick={onHandleClick} />;
  }
  ```

  ### Use cases

  You should not apply `useCallback` in a blanket fashion, this can reduce
  performance rather than improve it. The best scenarios are:

  1. A functional component wrapped inside React.memo() accepts a function object
     prop

  2. When the function object is a dependency to other hooks, e.g.
     `useEffect(..., [callback])`

  3. When the function has some internal state, e.g. when the function is
     debounced or throttled.

  ## useMemo

  We can think of `useMemo` as a complement to `useCallback`. The main difference
  is that whilst `useCallback` memoizes the function reference, `useMemo` memoizes
  the function result; the value that is returned.

  > In memoization, the result is “remembered” when the same parameters are
  > passed-in subsequently. If we have a function compute 1 + 1, it will return 2.
  > But if it uses memoization, the next time we run 1’s through the function, it
  > won’t add them up; it will just remember the answer is 2 without executing the
  > adding function.

  Like `useCallback`, `useMemo` takes a function and an array of dependencies for
  the same reason: if one of the dependencies changes then the function will
  recalculate (and re-memoize) the result but they don't the same memoized value
  will be returned.

  As with `useCallback`, `useMemo` is best applied to complex functions that are
  computationally expensive, it shouldn't be used indiscriminately.

  It should not be used with asynchronous functions, in these instances,
  `useEffect` would be a better choice.

  ### Syntax

  ```jsx
  const List = React.useMemo(
    () =>
      listOfItems.map((item) => ({
        ...item,
        itemProp1: expensiveFunction(props.first),
        itemProp2: anotherPriceyFunction(props.second),
      })),
    [listOfItems]
  );
  ```


path: Memory_Management_Unit.md
sortable_path: Memory_Management_Unit.md
title: Memory Management Unit (MMU)
checksum: 872a7691c78622056fe66b79197c0f6f9df56d1dba50321e9f4a17e5c3b52da0
created: 2025-10-01 13:30:59.844770916+00:00
modified: 2025-08-13 15:57:13.933598008+00:00
metadata: {"created":"Monday, July 08, 2024","tags":["memory","Linux"]}
lead:
  The MMU is the bridge between the physical memory devices of the machine and the
  virtual memory space. It is a chip that sits between the CPU and the RAM and
  determines the physical location of the memory requested by the kernel as
  virtual memory.

body:
  The MMU is the bridge between the physical memory devices of the machine and the
  virtual memory space. It is a chip that sits between the CPU and the RAM and
  determines the physical location of the memory requested by the kernel as
  virtual memory.

  ![Virtual memory diagram](/img/virtual-memory-diagram.jpg)

  ## Pages

  We use the term **pages** to denote blocks of virtual memory and to distinguish
  them from **addresses** as physical blocks. The MMU possesses a **page table**
  which is registry logging which pages correspond to which physical blocks.

  ## Shared pages

  Virtual memory allows the sharing of files and memory by multiple processes.
  Crucially the shared data doesn't have to be within the address of each process,
  instead there is a reference in the page table that each process has access to
  the shared data.

  ## Page faults

  There are two kinds of error that can occur with relation to paged memory:

  - minor page faults
    - The desired page is in main memory but the MMU doesn't currently know where
      it is
  - major page faults
    - The desired page is not in main memory at all. Therefore the kernel must
      fetch it from disk

  Minor page faults are very common and are to be expected; they resolve quickly.
  On the other hand too many major page faults can slow the system down both
  because of the time-costly process of fetching data from disk and because it
  demands more kernel resources to locate the missing page, which puts other
  processes on hold.

raw_content:
  ---
  tags: [memory, Linux]
  created: Monday, July 08, 2024
  ---

  # Memory Management Unit (MMU)

  The MMU is the bridge between the physical memory devices of the machine and the
  virtual memory space. It is a chip that sits between the CPU and the RAM and
  determines the physical location of the memory requested by the kernel as
  virtual memory.

  ![Virtual memory diagram](/img/virtual-memory-diagram.jpg)

  ## Pages

  We use the term **pages** to denote blocks of virtual memory and to distinguish
  them from **addresses** as physical blocks. The MMU possesses a **page table**
  which is registry logging which pages correspond to which physical blocks.

  ## Shared pages

  Virtual memory allows the sharing of files and memory by multiple processes.
  Crucially the shared data doesn't have to be within the address of each process,
  instead there is a reference in the page table that each process has access to
  the shared data.

  ## Page faults

  There are two kinds of error that can occur with relation to paged memory:

  - minor page faults
    - The desired page is in main memory but the MMU doesn't currently know where
      it is
  - major page faults
    - The desired page is not in main memory at all. Therefore the kernel must
      fetch it from disk

  Minor page faults are very common and are to be expected; they resolve quickly.
  On the other hand too many major page faults can slow the system down both
  because of the time-costly process of fetching data from disk and because it
  demands more kernel resources to locate the missing page, which puts other
  processes on hold.


path: Memory_addresses.md
sortable_path: Memory_addresses.md
title: Memory addresses
checksum: cc19bb80c91935b16385412e0ca63bfa50e40344fd948e1b2782feeee141d9b2
created: 2025-10-01 13:30:59.849195067+00:00
modified: 2025-08-13 15:57:13.746580731+00:00
metadata: {"created":"Friday, July 12, 2024","tags":["memory"]}
lead:
  > Computers assign numeric addresses to bytes of memory and the CPU can read or
  > write to those addresses

body:
  > Computers assign numeric addresses to bytes of memory and the CPU can read or
  > write to those addresses

  We can think of the internals of RAM as grids of memory cells.

  Each single-bit cell in the grid can be identified using two dimensional
  coordinates, like in a graph. The coordinates are the location of that cell in
  the grid.

  Handling one bit at a time isn't very efficient so RAM accesses **multiple
  grids** of 1-bit memory cells in parallel. This allows for reads or writes of
  multiple bits at once, such as a whole byte.

  The location of a set of bits in memory is known as a **memory address**.

  ### Demonstration

  Let's imagine we have a computer system that can address up to 64KB of memory
  and our system is byte addressable. This means there are
  $64 \cdot 1024 = 65,536$ bytes of memory because 1KB = 1024 bytes.

  We therefore have 65,536 addresses and each address can store one byte. So our
  addresses go from 0 to 65, 535.

  We now need to consider how many bits we need to uniquely represent an address
  on this system.

  What does this mean? Although there are approximately 64 thousand bytes of
  memory, to refer to each byte we can't just use 1, 2, 3... because computers use
  binary numbers. We need a binary number to refer to a given byte in the the 64KB
  of memory. The question we are asking is: how long does this binary number need
  to be to be able to represent each of the 64 thousand bytes?

  1 bit can represent two addresses: 0 and 1. 2 bits can represent four addresses:
  00, 01, 10, 11. The formula is as follows: number of addresses = $2^n$ where $n$
  is the number of bits.

  We need to reverse this formula to find out how many bits we need to represent a
  given number of addresses. We can do this with a [logarithm](Logarithms.md).

  We can reverse the formula as follows: number of bits = $\log_2$ (number of
  addresses).

  In our case we have 65,536 addresses so we need $\log_2(65,536)$ bits to
  represent each address. This is approximately 16 bits. Thus a 16 bit memory
  iaddress is needed to address 65, 546 bytes.

  Using memory addresses we end up with tables like the following:

  | Memory address   | Data             |
  | ---------------- | ---------------- |
  | 0000000000000000 | 1010101010101010 |
  | 0000000000000001 | 0010001001001011 |
  | 0000000000000010 | 0010001001001010 |

  This is hard to parse so we can instead use
  [hexadecimal numbers](Hexadecimal_number_system.md) to represent the addresses:

  | Memory address (as hex) | Data (as binary) |
  | ----------------------- | ---------------- |
  | 0x0000                  | 1010101010101010 |
  | 0x0001                  | 0010001001001011 |
  | 0x0002                  | 0010001001001010 |

  By itself, the the data is meaningless but we know from
  [binary encoding](Binary_encoding.md) that the binary data will correspond to
  some meaningful data, such as a character or a colour, depending on the encoding
  scheme used. The above table could correspond to the characters for 'A', 'B' and
  'C' in the ASCII encoding scheme:

  | Memory address (as hex) | Data (as binary) | Data (as ASCII) |
  | ----------------------- | ---------------- | --------------- |
  | 0x0000                  | 1010101010101010 | A               |
  | 0x0001                  | 0010001001001011 | B               |
  | 0x0002                  | 0010001001001010 | C               |

raw_content:
  ---
  tags: [memory]
  created: Friday, July 12, 2024
  ---

  # Memory addresses

  > Computers assign numeric addresses to bytes of memory and the CPU can read or
  > write to those addresses

  We can think of the internals of RAM as grids of memory cells.

  Each single-bit cell in the grid can be identified using two dimensional
  coordinates, like in a graph. The coordinates are the location of that cell in
  the grid.

  Handling one bit at a time isn't very efficient so RAM accesses **multiple
  grids** of 1-bit memory cells in parallel. This allows for reads or writes of
  multiple bits at once, such as a whole byte.

  The location of a set of bits in memory is known as a **memory address**.

  ### Demonstration

  Let's imagine we have a computer system that can address up to 64KB of memory
  and our system is byte addressable. This means there are
  $64 \cdot 1024 = 65,536$ bytes of memory because 1KB = 1024 bytes.

  We therefore have 65,536 addresses and each address can store one byte. So our
  addresses go from 0 to 65, 535.

  We now need to consider how many bits we need to uniquely represent an address
  on this system.

  What does this mean? Although there are approximately 64 thousand bytes of
  memory, to refer to each byte we can't just use 1, 2, 3... because computers use
  binary numbers. We need a binary number to refer to a given byte in the the 64KB
  of memory. The question we are asking is: how long does this binary number need
  to be to be able to represent each of the 64 thousand bytes?

  1 bit can represent two addresses: 0 and 1. 2 bits can represent four addresses:
  00, 01, 10, 11. The formula is as follows: number of addresses = $2^n$ where $n$
  is the number of bits.

  We need to reverse this formula to find out how many bits we need to represent a
  given number of addresses. We can do this with a [logarithm](Logarithms.md).

  We can reverse the formula as follows: number of bits = $\log_2$ (number of
  addresses).

  In our case we have 65,536 addresses so we need $\log_2(65,536)$ bits to
  represent each address. This is approximately 16 bits. Thus a 16 bit memory
  iaddress is needed to address 65, 546 bytes.

  Using memory addresses we end up with tables like the following:

  | Memory address   | Data             |
  | ---------------- | ---------------- |
  | 0000000000000000 | 1010101010101010 |
  | 0000000000000001 | 0010001001001011 |
  | 0000000000000010 | 0010001001001010 |

  This is hard to parse so we can instead use
  [hexadecimal numbers](Hexadecimal_number_system.md) to represent the addresses:

  | Memory address (as hex) | Data (as binary) |
  | ----------------------- | ---------------- |
  | 0x0000                  | 1010101010101010 |
  | 0x0001                  | 0010001001001011 |
  | 0x0002                  | 0010001001001010 |

  By itself, the the data is meaningless but we know from
  [binary encoding](Binary_encoding.md) that the binary data will correspond to
  some meaningful data, such as a character or a colour, depending on the encoding
  scheme used. The above table could correspond to the characters for 'A', 'B' and
  'C' in the ASCII encoding scheme:

  | Memory address (as hex) | Data (as binary) | Data (as ASCII) |
  | ----------------------- | ---------------- | --------------- |
  | 0x0000                  | 1010101010101010 | A               |
  | 0x0001                  | 0010001001001011 | B               |
  | 0x0002                  | 0010001001001010 | C               |


path: Memory_leaks.md
sortable_path: Memory_leaks.md
title: Memory leaks
checksum: a4d5c6a8df018b82e31f5ba1746250ef20d36dca5a43b5a7d7e425cf5ab4db22
created: 2025-10-01 13:30:59.863718032+00:00
modified: 2025-08-13 15:57:13.929261681+00:00
metadata: {"tags":["memory"]}
lead:
  A memory leak occurs when you program memory and forget to delete it once you
  are finished. The consequences of memory leak is that it reduces the performance
  of the computer by reducing the amount of available memory. Eventually, in the
  worst case, too much of the available memory may become allocated and all or
  part of the system or device stops working correctly, the application fails, or
  the system slows down vastly.

body:
  A memory leak occurs when you program memory and forget to delete it once you
  are finished. The consequences of memory leak is that it reduces the performance
  of the computer by reducing the amount of available memory. Eventually, in the
  worst case, too much of the available memory may become allocated and all or
  part of the system or device stops working correctly, the application fails, or
  the system slows down vastly.

raw_content:
  ---
  tags: [memory]
  ---

  # Memory leaks

  A memory leak occurs when you program memory and forget to delete it once you
  are finished. The consequences of memory leak is that it reduces the performance
  of the computer by reducing the amount of available memory. Eventually, in the
  worst case, too much of the available memory may become allocated and all or
  part of the system or device stops working correctly, the application fails, or
  the system slows down vastly.


path: Memory_versus_processor.md
sortable_path: Memory_versus_processor.md
title: Memory versus processor
checksum: 6934369b9b4e8234521db8a1b40ca54359cf20097fb1930b87ed62bdf8ea3206
created: 2025-10-01 13:30:59.869928886+00:00
modified: 2025-08-13 15:57:13.739803542+00:00
metadata: {"created":"Thursday, August 01, 2024","tags":["question"]}
lead:
  > Would a more powerful processor with average or reduced memory capacity
  > perform better than, equal to, or less than a less powerful processor with
  > more memory, in principle?

body:
  > Would a more powerful processor with average or reduced memory capacity
  > perform better than, equal to, or less than a less powerful processor with
  > more memory, in principle?

raw_content:
  ---
  tags: ["question"]
  created: Thursday, August 01, 2024
  ---

  # Memory versus processor

  > Would a more powerful processor with average or reduced memory capacity
  > perform better than, equal to, or less than a less powerful processor with
  > more memory, in principle?


path: Middleware_in_NodeJS.md
sortable_path: Middleware_in_NodeJS.md
title: Middleware
lead: ## What is middleware?
checksum: cc530669082b3716b21388f1c2ef4e95e3e1c2d678e96e586dec2cc8a3f66f4b
created: 2025-10-01 13:30:59.87459523+00:00
modified: 2025-08-13 15:57:13.833144834+00:00
metadata: {"tags":["node-js","middleware"]}
body:
  ## What is middleware?

  - Anything that terminates the `req, res` cycle counts as middleware. It is
    basically anything that acts as an intermediary once the request is received
    but before the resource is sent. A good example would be the
    `app.use(express.json()` or `app.use(bodyParser.json)` functions we call in
    order to be able to parse JSON that is sent from the client.
  - You will most likely have multiple middleware functions running at once. We
    call this intermediary part of the cycle the **request processing pipeline**.
  - Generally all middleware will be added as a property on the Express `app`
    instance with the `app.use(...)` syntax.

  ## Creating custom middleware functions

  ### Basic schema

  ```js
  app.use((req, res, next) => {
    // do some middleware
    next();
  });
  ```

  ### `next`

  The `next` parameter is key, it allows Express to move onto the next middleware
  function once the custom middleware executes. Without it, the request processing
  pipeline will get blocked.

  Middleware functions are basically asynchronous requests and as such they use a
  similar syntax as Promises (e.g `then`) for sequencing processes.

  ### Example of sequence

  ```js
  app.use((req, res, next) => {
    console.log("Do process A...");
    next();
  });

  app.use((req, res, next) => {
    console.log("Do process B...");
    next();
  });
  ```

  > It makes more sense of course to define our middleware within a function and
  > then pass it as an argument to `app.use()`

  ## Including middleware based on environment

  With a full-scale Node application you will typically run three environments:

  - Development
  - Testing
  - Production

  We will not want to run certain types of middleware in all environments. For
  example, it would be costly to run logging in the app's production environment.
  It would make more sense to run this only in development.

  ### Accessing current Node environment

  We can control which middleware we run via the Node envrionment variables:
  `process.env` (see for instance [ports](Ports_in_NodeJS.md)).

  We could set
  [Morgan](Morgan.md) to run
  only in development with:

  ```js
  if (app.get("env") === "development") {
    app.use(morgan("common"));
    console.log("Morgan enabled");
  }
  ```

raw_content:
  ---
  tags:
    - node-js
    - middleware
  ---

  # Middleware

  ## What is middleware?

  - Anything that terminates the `req, res` cycle counts as middleware. It is
    basically anything that acts as an intermediary once the request is received
    but before the resource is sent. A good example would be the
    `app.use(express.json()` or `app.use(bodyParser.json)` functions we call in
    order to be able to parse JSON that is sent from the client.
  - You will most likely have multiple middleware functions running at once. We
    call this intermediary part of the cycle the **request processing pipeline**.
  - Generally all middleware will be added as a property on the Express `app`
    instance with the `app.use(...)` syntax.

  ## Creating custom middleware functions

  ### Basic schema

  ```js
  app.use((req, res, next) => {
    // do some middleware
    next();
  });
  ```

  ### `next`

  The `next` parameter is key, it allows Express to move onto the next middleware
  function once the custom middleware executes. Without it, the request processing
  pipeline will get blocked.

  Middleware functions are basically asynchronous requests and as such they use a
  similar syntax as Promises (e.g `then`) for sequencing processes.

  ### Example of sequence

  ```js
  app.use((req, res, next) => {
    console.log("Do process A...");
    next();
  });

  app.use((req, res, next) => {
    console.log("Do process B...");
    next();
  });
  ```

  > It makes more sense of course to define our middleware within a function and
  > then pass it as an argument to `app.use()`

  ## Including middleware based on environment

  With a full-scale Node application you will typically run three environments:

  - Development
  - Testing
  - Production

  We will not want to run certain types of middleware in all environments. For
  example, it would be costly to run logging in the app's production environment.
  It would make more sense to run this only in development.

  ### Accessing current Node environment

  We can control which middleware we run via the Node envrionment variables:
  `process.env` (see for instance [ports](Ports_in_NodeJS.md)).

  We could set
  [Morgan](Morgan.md) to run
  only in development with:

  ```js
  if (app.get("env") === "development") {
    app.use(morgan("common"));
    console.log("Morgan enabled");
  }
  ```


path: Mixed_and_improper_fractions.md
sortable_path: Mixed_and_improper_fractions.md
title: Mixed and improper fractions
lead: ## Improper fractions
checksum: 591035861210f541355bffec2836d38c935d09f90152f5328479bf5fca7cd409
created: 2025-10-01 13:30:59.891481543+00:00
modified: 2025-08-13 15:57:13.965970869+00:00
metadata: {"tags":["prealgebra","fractions"]}
body:
  ## Improper fractions

  - Proper fraction:
    - The numerator is smaller than the denominator
    - E.g. $\frac{2}{3}$, $-\frac{5}{10}$
  - Improper fraction
    - The numerator is greater than the denominator
    - E.g. $\frac{3}{2}$, $-\frac{5}{4}$

  ## Mixed fractions

  A mixed fraction is part whole number, part fraction. For example:
  $5 \frac{3}{4}$.

  It means, in effect: $5 + \frac{3}{4}$

  ## Converting mixed fractions into improper fractions

  Mixed fractions are hard to calculate with. We need some way to convert them to
  fractions. We can do this by converting them to improper fractions.

  With the example $4 \frac{7}{8}$, we know this means $4 + \frac{7}{8}$. We need
  to express the amount 4 in terms of eighths. It would be 4 lots of $\frac{8}{8}$
  given that 4 is a whole number not a fractional amount. Thus the process would
  be:

  $$
      \frac{8}{8} + \frac{8}{8} + \frac{8}{8} + \frac{8}{8} + \frac{7}{8}
  $$

  But we know that when we
  [add fractions with a common denominator](Add_Subtract_Fractions.md#adding-subracting-fractions-with-common-denominators),
  we only add the numerators, not the denominators. Thus the calculation would
  actually be:

  $$
  \frac{8 + 8 + 8 + 8 + 7}{8} = \frac{39}{8}
  $$

  Addition helps to explain the concepts underlying the procedure but it is more
  efficient to use multiplication.

  The procedure is as follows:

  1. Multiply the whole number part by the denominator
  2. Add the numerator
  3. Place the result over the denominator

  Thus:

  $$
      4 \frac{7}{8} = \frac{(4 \cdot 8) + 7 }{8}
  $$

  ## Converting improper fractions into mixed fractions

  It is quite obvious how to reverse the process and turn an improper fraction
  into a mixed fraction.

  Take $\frac{27}{5}$. We work out how many times the numerator is divisible by
  the denominator and make that the whole number. The remainder is then left as
  the fractional part.

  $$
  \begin{split}
  \frac{27}{5} = 27 \div 5  \\
  = 5 r 2 \\
  = 5 \frac{2}{5}
  \end{split}
  $$

  ## Multiplying and dividing by mixed fractions

  Now that we know how to convert mixed fractions into improper fractions, it is
  straight forward to multiply and divide with them. We convert the mixed fraction
  into an improper fraction and then divide and multiply as we would with a proper
  fraction.

  ### Demonstration of multiplication

  Calculate $-2\frac{1}{12} \cdot 2 \frac{4}{5}$:

  1. First convert each mixed fraction into an improper fraction:
     $$
       \begin{split}
       -2\frac{1}{12} = -2 \cdot -12  \\
       = 24 + 1 \\
       = - \frac{25}{12}
       \end{split}
     $$

  $$
  \begin{split}
    2 \frac{4}{5} =2 \cdot 5 \\
    = 10 + 4 \\
    = \frac{14}{5}
  \end{split}
  $$

  2. Then carry out the multiplication
     [using factorization](Multiplying_fractions.md#prime-factorisation-in-place):

     $$
       \begin{split}
       - \frac{25}{12} \cdot \frac{14}{5} = \\
      - \frac{(5 \cdot 5) \cdot (7 \cdot 2)}{(3 \cdot 2 \cdot 2) \cdot (5)} = - \frac{5 \cdot 7 }{2 \cdot 3} \\
       \end{split}
     $$

  3. Then simplify:

     $$
       - \frac{35}{6}
     $$

  4. Finally, convert back into a mixed fraction:

  $$
    \begin{split}
    - \frac{35}{6} = -35 \div 6 \\
    -  5 r 5 = \\
    - 5 \frac{5}{6}
    \end{split}
  $$

  ## Demonstration of division

  Again we convert the mixed fraction into an improper fraction and then follow
  the requisite rule, in the case of division this is to
  [invert and multiply](Dividing_fractions.md#formal-specification-of-how-to-divide-fractions').

  Calculate $-4 \frac{4}{5} \div 5 \frac{3}{5}$.

  1. Convert to improper fraction:
     $$
        \begin{split}
          -4 \cdot 5 = \\
      -20
     \end{split}
     $$

  ## Adding and subtracting mixed fractions

raw_content:
  ---
  tags:
    - prealgebra
    - fractions
  ---

  # Mixed and improper fractions

  ## Improper fractions

  - Proper fraction:
    - The numerator is smaller than the denominator
    - E.g. $\frac{2}{3}$, $-\frac{5}{10}$
  - Improper fraction
    - The numerator is greater than the denominator
    - E.g. $\frac{3}{2}$, $-\frac{5}{4}$

  ## Mixed fractions

  A mixed fraction is part whole number, part fraction. For example:
  $5 \frac{3}{4}$.

  It means, in effect: $5 + \frac{3}{4}$

  ## Converting mixed fractions into improper fractions

  Mixed fractions are hard to calculate with. We need some way to convert them to
  fractions. We can do this by converting them to improper fractions.

  With the example $4 \frac{7}{8}$, we know this means $4 + \frac{7}{8}$. We need
  to express the amount 4 in terms of eighths. It would be 4 lots of $\frac{8}{8}$
  given that 4 is a whole number not a fractional amount. Thus the process would
  be:

  $$
      \frac{8}{8} + \frac{8}{8} + \frac{8}{8} + \frac{8}{8} + \frac{7}{8}
  $$

  But we know that when we
  [add fractions with a common denominator](Add_Subtract_Fractions.md#adding-subracting-fractions-with-common-denominators),
  we only add the numerators, not the denominators. Thus the calculation would
  actually be:

  $$
  \frac{8 + 8 + 8 + 8 + 7}{8} = \frac{39}{8}
  $$

  Addition helps to explain the concepts underlying the procedure but it is more
  efficient to use multiplication.

  The procedure is as follows:

  1. Multiply the whole number part by the denominator
  2. Add the numerator
  3. Place the result over the denominator

  Thus:

  $$
      4 \frac{7}{8} = \frac{(4 \cdot 8) + 7 }{8}
  $$

  ## Converting improper fractions into mixed fractions

  It is quite obvious how to reverse the process and turn an improper fraction
  into a mixed fraction.

  Take $\frac{27}{5}$. We work out how many times the numerator is divisible by
  the denominator and make that the whole number. The remainder is then left as
  the fractional part.

  $$
  \begin{split}
  \frac{27}{5} = 27 \div 5  \\
  = 5 r 2 \\
  = 5 \frac{2}{5}
  \end{split}
  $$

  ## Multiplying and dividing by mixed fractions

  Now that we know how to convert mixed fractions into improper fractions, it is
  straight forward to multiply and divide with them. We convert the mixed fraction
  into an improper fraction and then divide and multiply as we would with a proper
  fraction.

  ### Demonstration of multiplication

  Calculate $-2\frac{1}{12} \cdot 2 \frac{4}{5}$:

  1. First convert each mixed fraction into an improper fraction:
     $$
       \begin{split}
       -2\frac{1}{12} = -2 \cdot -12  \\
       = 24 + 1 \\
       = - \frac{25}{12}
       \end{split}
     $$

  $$
  \begin{split}
    2 \frac{4}{5} =2 \cdot 5 \\
    = 10 + 4 \\
    = \frac{14}{5}
  \end{split}
  $$

  2. Then carry out the multiplication
     [using factorization](Multiplying_fractions.md#prime-factorisation-in-place):

     $$
       \begin{split}
       - \frac{25}{12} \cdot \frac{14}{5} = \\
      - \frac{(5 \cdot 5) \cdot (7 \cdot 2)}{(3 \cdot 2 \cdot 2) \cdot (5)} = - \frac{5 \cdot 7 }{2 \cdot 3} \\
       \end{split}
     $$

  3. Then simplify:

     $$
       - \frac{35}{6}
     $$

  4. Finally, convert back into a mixed fraction:

  $$
    \begin{split}
    - \frac{35}{6} = -35 \div 6 \\
    -  5 r 5 = \\
    - 5 \frac{5}{6}
    \end{split}
  $$

  ## Demonstration of division

  Again we convert the mixed fraction into an improper fraction and then follow
  the requisite rule, in the case of division this is to
  [invert and multiply](Dividing_fractions.md#formal-specification-of-how-to-divide-fractions').

  Calculate $-4 \frac{4}{5} \div 5 \frac{3}{5}$.

  1. Convert to improper fraction:
     $$
        \begin{split}
          -4 \cdot 5 = \\
      -20
     \end{split}
     $$

  ## Adding and subtracting mixed fractions


path: Modelling_relationships_in_MongoDB.md
sortable_path: Modelling_relationships_in_MongoDB.md
title: Modelling relationships between data
checksum: c7525be2f33aacca299e720d1ac1ad731bffae9e7b52c6557609fb4a43532a0e
created: 2025-10-01 13:30:59.909062728+00:00
modified: 2025-08-13 15:57:13.844140531+00:00
metadata: {"tags":["mongo-db","node-js","mongoose","databases"]}
lead:
  So far we have taken the values of a document to be simple: the `author` value
  is just a string but what if `author` was itself another collection for instance
  a collection with the proprties `age` and `publications` and the author name,
  say `Tosh Gnomay` was a document in this collection? This means we would have an
  interaction between two collections. In this entry we will look at how to work
  with interrelated collections. This is equivalent to establishing
  [joins](10_Joins.md) in a relational database.

body:
  So far we have taken the values of a document to be simple: the `author` value
  is just a string but what if `author` was itself another collection for instance
  a collection with the proprties `age` and `publications` and the author name,
  say `Tosh Gnomay` was a document in this collection? This means we would have an
  interaction between two collections. In this entry we will look at how to work
  with interrelated collections. This is equivalent to establishing
  [joins](10_Joins.md) in a relational database.

  There are two main approaches to modelling relationships between data:
  **normalisation** and **denormalisation**.

  ## Modelling relationships with normalisation: using references

  In the case of normalisation we use a UUID from one document to connect it to
  another document.

  Let's say that we have a course document as follows which is an instance of the
  `courses` collection:

  ```js
  {
      _id: "ceds89e"
      name: "Java course",
      author: "Tosh Gnomay"
  }
  ```

  And we have a document which is an instance of another type of collection called
  `authors`. One instance of it could be:

  ```js
  {
      _id: "de89w9",
      name: "Tosh Gnomay",
      courses: ["Java course", "C# course"]
  }
  ```

  Using normalisation, we would establish the relationship by using the value of
  the `_id` propety in the `courses` document as the value of the `name` property
  in the `authors` document. We would rewrite the `courses` document like so:

  ```js
  {
      _id: "ceds89e"
      name: "Java course",
      author: "de89w9"
  }
  ```

  ## Modelling relationships with denormalisation: embedding

  With denormalisation we achieve the same outcome but instead of using a
  reference we embed one value in another:

  ```js
  {
      _id: "ceds89e"
      name: "Java course",
      author: {
          name: "Tosh Gnomay",
          courses: ["Java course", "C# course"]
      }
  }
  ```

  ## Tradeoffs: which approach to use?

  There are drawbacks and benefits with each approach.

  - One benefit of normalisation is consistency. Because we are using the UUID and
    referencing it in another document, if we make a change to the values in the
    source document, it will automatically apply to all documents that reference
    it. We make a change in one place and it is reflected everywhere.

  - Conversely, a drawback of denormalisation is that it invites inconsistency
    between documents. Embedded values can easily go out of sync with their usage
    elsewhere as you don't have a direct reference correspondence.

  - On the other hand a drawback of normalisation is performance when querying.
    Any time you have a document that references another document than this
    document also has to be retrieved. Therefore for every query you are returning
    two documents. As denormalised references embed the second document, only one
    document is being returned on each query.

raw_content:
  ---
  tags:
    - mongo-db
    - node-js
    - mongoose
    - databases
  ---

  # Modelling relationships between data

  So far we have taken the values of a document to be simple: the `author` value
  is just a string but what if `author` was itself another collection for instance
  a collection with the proprties `age` and `publications` and the author name,
  say `Tosh Gnomay` was a document in this collection? This means we would have an
  interaction between two collections. In this entry we will look at how to work
  with interrelated collections. This is equivalent to establishing
  [joins](10_Joins.md) in a relational database.

  There are two main approaches to modelling relationships between data:
  **normalisation** and **denormalisation**.

  ## Modelling relationships with normalisation: using references

  In the case of normalisation we use a UUID from one document to connect it to
  another document.

  Let's say that we have a course document as follows which is an instance of the
  `courses` collection:

  ```js
  {
      _id: "ceds89e"
      name: "Java course",
      author: "Tosh Gnomay"
  }
  ```

  And we have a document which is an instance of another type of collection called
  `authors`. One instance of it could be:

  ```js
  {
      _id: "de89w9",
      name: "Tosh Gnomay",
      courses: ["Java course", "C# course"]
  }
  ```

  Using normalisation, we would establish the relationship by using the value of
  the `_id` propety in the `courses` document as the value of the `name` property
  in the `authors` document. We would rewrite the `courses` document like so:

  ```js
  {
      _id: "ceds89e"
      name: "Java course",
      author: "de89w9"
  }
  ```

  ## Modelling relationships with denormalisation: embedding

  With denormalisation we achieve the same outcome but instead of using a
  reference we embed one value in another:

  ```js
  {
      _id: "ceds89e"
      name: "Java course",
      author: {
          name: "Tosh Gnomay",
          courses: ["Java course", "C# course"]
      }
  }
  ```

  ## Tradeoffs: which approach to use?

  There are drawbacks and benefits with each approach.

  - One benefit of normalisation is consistency. Because we are using the UUID and
    referencing it in another document, if we make a change to the values in the
    source document, it will automatically apply to all documents that reference
    it. We make a change in one place and it is reflected everywhere.

  - Conversely, a drawback of denormalisation is that it invites inconsistency
    between documents. Embedded values can easily go out of sync with their usage
    elsewhere as you don't have a direct reference correspondence.

  - On the other hand a drawback of normalisation is performance when querying.
    Any time you have a document that references another document than this
    document also has to be retrieved. Therefore for every query you are returning
    two documents. As denormalised references embed the second document, only one
    document is being returned on each query.


path: Module_pattern.md
sortable_path: Module_pattern.md
title: Module pattern
lead: > Come back and compare this with learning from Node.js
checksum: d667727226143e5e85a1f24373d72e77864e183a5b286cf980447b711f112e10
created: 2025-10-01 13:30:59.918456286+00:00
modified: 2025-08-13 15:57:13.758846596+00:00
metadata: {"tags":["OOP","design-patterns"]}
body:
  > Come back and compare this with learning from Node.js

  With the module design pattern we create encapsulation: the variables and
  functions (as methods) are kept private inside the module and cannot be
  overwritten. This design pattern is familiar from Node.js development: every
  package you import into your project and download from NPM is a module.

  Generally you will create the module as a class, import it and then instantiate
  a new instance. However for private development, you could just as well use an
  object and duplicate it with `Object.create`.

  ## Example

  Here is an example of a simple module that returns the age of a person.

  ```js
  export default class Age {
    constructor(name, birthYear) {
      this.name = name;
      this.birthYear = birthYear;
    }
    currentYear() {
      return new Date().getFullYear();
    }
    get age() {
      return this.currentYear() - this.birthYear;
    }
  }
  ```

  ```js
  const martha = new Age("Martha", 1997);
  console.log(martha.age); // 24
  ```

  ## Controlling access

  In the example above, `aValue` could be edited in instantiations of the class.
  Given that modules should not be overwrittable, you could make it a private
  property on the class. The benefit of getters and setters is that they dictate
  what can be modified and retrieved from the module. So if you use `get` and
  `set` you can prevent overwrites.

  ## Object modules

  If you want to use an object instead of a class, you have to take greater care
  to ensure that the objects are not overwritable. **Also you cannot use the `#`
  modifier to make properties private.**

  - Use getters and setters for updating and retrieving values
  - Use `Object.seal` to prevent changes to the parent object
  - Instead of using getters and setters, for individual properties you can set
    `writable` to be `false` for properties that you don't want changed

  There are examples of each in the following:

  ```jsx
  export const age = {
    name: "",
    birthYear: new Number(),
    currentYear() {
      return new Date().getFullYear();
    },
    get age() {
      return this.currentYear() - this.birthYear;
    },
  };

  Object.seal(age);
  Object.defineProperty(age, "aValue", {
    value: 6,
    writable: false,
  });
  ```

raw_content:
  ---
  tags:
    - OOP
    - design-patterns
  ---

  # Module pattern

  > Come back and compare this with learning from Node.js

  With the module design pattern we create encapsulation: the variables and
  functions (as methods) are kept private inside the module and cannot be
  overwritten. This design pattern is familiar from Node.js development: every
  package you import into your project and download from NPM is a module.

  Generally you will create the module as a class, import it and then instantiate
  a new instance. However for private development, you could just as well use an
  object and duplicate it with `Object.create`.

  ## Example

  Here is an example of a simple module that returns the age of a person.

  ```js
  export default class Age {
    constructor(name, birthYear) {
      this.name = name;
      this.birthYear = birthYear;
    }
    currentYear() {
      return new Date().getFullYear();
    }
    get age() {
      return this.currentYear() - this.birthYear;
    }
  }
  ```

  ```js
  const martha = new Age("Martha", 1997);
  console.log(martha.age); // 24
  ```

  ## Controlling access

  In the example above, `aValue` could be edited in instantiations of the class.
  Given that modules should not be overwrittable, you could make it a private
  property on the class. The benefit of getters and setters is that they dictate
  what can be modified and retrieved from the module. So if you use `get` and
  `set` you can prevent overwrites.

  ## Object modules

  If you want to use an object instead of a class, you have to take greater care
  to ensure that the objects are not overwritable. **Also you cannot use the `#`
  modifier to make properties private.**

  - Use getters and setters for updating and retrieving values
  - Use `Object.seal` to prevent changes to the parent object
  - Instead of using getters and setters, for individual properties you can set
    `writable` to be `false` for properties that you don't want changed

  There are examples of each in the following:

  ```jsx
  export const age = {
    name: "",
    birthYear: new Number(),
    currentYear() {
      return new Date().getFullYear();
    },
    get age() {
      return this.currentYear() - this.birthYear;
    },
  };

  Object.seal(age);
  Object.defineProperty(age, "aValue", {
    value: 6,
    writable: false,
  });
  ```


path: Module_wrapping_at_runtime.md
sortable_path: Module_wrapping_at_runtime.md
title: Module wrapping at runtime in NodeJS
lead: ## The Module Wrapper Function
checksum: 7431983c6dab5cff64cb61f6e8b54daea923583fdfd33d8bf9d63152cb2c576c
created: 2025-10-01 13:30:59.924293418+00:00
modified: 2025-08-13 15:57:14.032331763+00:00
metadata: {"tags":["node-js"]}
body:
  ## The Module Wrapper Function

  When Node runs each of our module files are wrapped within an
  immediately-invoked function expression that has the following parameters:

  ```js
  (function (exports, require, module, __filename, __dirname))

  ```

  This is called the **module wrapper function**

  Note that one of these parameters is the
  [module object](Modules_in_NodeJS.md#structure-of-a-module).

  Within any module we can access these parameters: you can think of them as
  metadata about the module itself. `__filename` and `__dirname` are particularly
  useful when writing to files and modifying directories.

raw_content:
  ---
  tags:
    - node-js
  ---

  # Module wrapping at runtime in NodeJS

  ## The Module Wrapper Function

  When Node runs each of our module files are wrapped within an
  immediately-invoked function expression that has the following parameters:

  ```js
  (function (exports, require, module, __filename, __dirname))

  ```

  This is called the **module wrapper function**

  Note that one of these parameters is the
  [module object](Modules_in_NodeJS.md#structure-of-a-module).

  Within any module we can access these parameters: you can think of them as
  metadata about the module itself. `__filename` and `__dirname` are particularly
  useful when writing to files and modifying directories.


path: Modules_in_NodeJS.md
sortable_path: Modules_in_NodeJS.md
title: Modules
checksum: 9d03b5ccc104e11c3ef76f35571497cf375083499eb5f3b5f78548785b8d21f0
created: 2025-10-01 13:30:59.935661765+00:00
modified: 2025-08-13 15:57:13.875941996+00:00
metadata: {"tags":["node-js"]}
lead:
  > Modules are partitioned files where we define our variables and functions.
  > Values defined in modules are scoped to that specific module, constituting a
  > unique name space. This avoids name clashes in large programs.

body:
  > Modules are partitioned files where we define our variables and functions.
  > Values defined in modules are scoped to that specific module, constituting a
  > unique name space. This avoids name clashes in large programs.

  - Every file in a Node application is considered a module.

  - The variables and methods in a module are equivalent to `private` properties
    and methods in object-oriented programming.

  - If you wish to use a function or variable defined in a module outside of its
    modular container you need to explicitly export it and make it public.

  ## Structure of a module

  Node keeps an internal record of the properties of a module. To see this we can
  log the property `module` to the console.

  ```js
  // index.js
  console.log(module);
  ```

  This gives us:

  ```plaintext
  Module {
   id: '.',
   path: '/home/thomas/repos/node-learning',
   exports: {},
   filename: '/home/thomas/repos/node-learning/index.js',
   loaded: false,
   children: [],
   paths: [
     '/home/thomas/repos/node-learning/node_modules',
     '/home/thomas/repos/node_modules',
     '/home/thomas/node_modules',
     '/home/node_modules',
     '/node_modules'
   ]
  }
  ```

  ## Exports

  - Whenever we export a property or method from a module we are directly
    targeting the `exports` property of the module object.
  - Once we add exports to a file they will be displayed under that property of
    the module object.
  - We can export the entire module itself as the export (typically used when the
    module is a single function or class) or individual properties.

  ### Exporting a whole module

  _The example below is a module file that consists in a single function_

  ```js
  module.exports = function (...params) {
    // function body
  };
  ```

  Note the module is unnamed. We would name it when we import:

  ```js
  const myFunction = require("./filenme");
  ```

  ### Exporting sub-components from a module

  In the example below we export a variable and function from the same module.
  Note only those values prefixed with `exports` are exported.

  ```js
  exports.myFunc = (...params) => {
    // function bod[]()y
  };

  exports.aVar = 321.3;

  var nonExportedVar = true;
  ```

  This time the exports are already name so we would import with the following:

  ```js
  const { myFunc, aVar } = require("./filename");
  ```

  We can also do the exporting at the bottom when the individual components are
  named:

  ```js
  const myNamedFunc = (val) => {
    return val + 1;
  };

  function anotherNamedFunc(val) {
    return val * 2;
  }

  // This time we export at the bottom
  exports.myNamedFunc = myNamedFunc;
  exports.differentName = anotherNamedFunc; // We can use different names

  // Or we could export them together
  module.exports = { myNamedFunc, anotherNamedFunc };
  ```

  The import is the same:

  ```js
  const { myNamedFunc, anotherNamedFunc } = require("./modules/multiExports");
  ```

  ## Structuring modules

  The techniques above are useful to know but generally you would want to enforce
  a stricter structure than a mix of exported and private values in the one file.
  The best way to do this is with a single default export.

  Here the thing exported could be a composite function or an object that
  basically acts like a class with methods and properties.

  _Export a composite single function_

  ```js

  module.exports = () => {
  	foo() {...}
  	bar() {...}
  }
  ```

  _Export an object_

  ```js

  module.exports = {
  	foo : () => {...},
      bar: () => {...}
  }
  ```

  **Both of these structures would be referred to in the same way when importing
  and using them.**

  Or you could export an actual class as the default. This is practically the same
  as the two above other than that you would have to use `new` to initiate an
  instance of the class.

  ```js
  export default class {
    foo() {}
    bar() {}
  }
  ```

  Every method and property within the export will be public by default, whether
  it is an object, class or function. If you wanted to keep certain
  methods/properties private, the best approach is to define them as variables and
  functions within the module file but outside of the `export` block.

raw_content:
  ---
  tags:
    - node-js
  ---

  # Modules

  > Modules are partitioned files where we define our variables and functions.
  > Values defined in modules are scoped to that specific module, constituting a
  > unique name space. This avoids name clashes in large programs.

  - Every file in a Node application is considered a module.

  - The variables and methods in a module are equivalent to `private` properties
    and methods in object-oriented programming.

  - If you wish to use a function or variable defined in a module outside of its
    modular container you need to explicitly export it and make it public.

  ## Structure of a module

  Node keeps an internal record of the properties of a module. To see this we can
  log the property `module` to the console.

  ```js
  // index.js
  console.log(module);
  ```

  This gives us:

  ```plaintext
  Module {
   id: '.',
   path: '/home/thomas/repos/node-learning',
   exports: {},
   filename: '/home/thomas/repos/node-learning/index.js',
   loaded: false,
   children: [],
   paths: [
     '/home/thomas/repos/node-learning/node_modules',
     '/home/thomas/repos/node_modules',
     '/home/thomas/node_modules',
     '/home/node_modules',
     '/node_modules'
   ]
  }
  ```

  ## Exports

  - Whenever we export a property or method from a module we are directly
    targeting the `exports` property of the module object.
  - Once we add exports to a file they will be displayed under that property of
    the module object.
  - We can export the entire module itself as the export (typically used when the
    module is a single function or class) or individual properties.

  ### Exporting a whole module

  _The example below is a module file that consists in a single function_

  ```js
  module.exports = function (...params) {
    // function body
  };
  ```

  Note the module is unnamed. We would name it when we import:

  ```js
  const myFunction = require("./filenme");
  ```

  ### Exporting sub-components from a module

  In the example below we export a variable and function from the same module.
  Note only those values prefixed with `exports` are exported.

  ```js
  exports.myFunc = (...params) => {
    // function bod[]()y
  };

  exports.aVar = 321.3;

  var nonExportedVar = true;
  ```

  This time the exports are already name so we would import with the following:

  ```js
  const { myFunc, aVar } = require("./filename");
  ```

  We can also do the exporting at the bottom when the individual components are
  named:

  ```js
  const myNamedFunc = (val) => {
    return val + 1;
  };

  function anotherNamedFunc(val) {
    return val * 2;
  }

  // This time we export at the bottom
  exports.myNamedFunc = myNamedFunc;
  exports.differentName = anotherNamedFunc; // We can use different names

  // Or we could export them together
  module.exports = { myNamedFunc, anotherNamedFunc };
  ```

  The import is the same:

  ```js
  const { myNamedFunc, anotherNamedFunc } = require("./modules/multiExports");
  ```

  ## Structuring modules

  The techniques above are useful to know but generally you would want to enforce
  a stricter structure than a mix of exported and private values in the one file.
  The best way to do this is with a single default export.

  Here the thing exported could be a composite function or an object that
  basically acts like a class with methods and properties.

  _Export a composite single function_

  ```js

  module.exports = () => {
  	foo() {...}
  	bar() {...}
  }
  ```

  _Export an object_

  ```js

  module.exports = {
  	foo : () => {...},
      bar: () => {...}
  }
  ```

  **Both of these structures would be referred to in the same way when importing
  and using them.**

  Or you could export an actual class as the default. This is practically the same
  as the two above other than that you would have to use `new` to initiate an
  instance of the class.

  ```js
  export default class {
    foo() {}
    bar() {}
  }
  ```

  Every method and property within the export will be public by default, whether
  it is an object, class or function. If you wanted to keep certain
  methods/properties private, the best approach is to define them as variables and
  functions within the module file but outside of the `export` block.


path: MongoDB_Introduction.md
sortable_path: MongoDB_Introduction.md
title: MongoDB: Introduction
checksum: 13cf7e8e089a48db143051dddb2c02e86449cf2858d7aa9e44ec492618a668f5
created: 2025-10-01 13:30:59.941285172+00:00
modified: 2025-08-13 15:57:13.752995895+00:00
metadata: {"tags":["mongo-db","databases"]}
lead:
  MongoDB is not a relational database system like SQL, instead it is
  document-based which means you do not neeed to design database tables and
  schema, you are simply storing data as JSON and there is no transformation of
  the data.

body:
  MongoDB is not a relational database system like SQL, instead it is
  document-based which means you do not neeed to design database tables and
  schema, you are simply storing data as JSON and there is no transformation of
  the data.

  Most of the notes on Mongo will introduce it within the context of Node.js
  backend. We will be sending Javascript objects and arrays to Mongo and returning
  them as JSON.

  ## Databases, collections, documents

  Although Mongo is not a relational database it has a structure that we can
  understand in relation to that paradigm. A **database** is obviously the overall
  structure. It comprises **collections** which are organised sets of data that
  are analagous to [tables](Relational_database_architecture.md#table)
  in RDBs. Within each collection are a series of **documents** which we can think
  of as being equivalent to [rows](Relational_database_architecture.md)
  in RDB table: units that comprise the collection.

  A document is a container comprising key-value pairs in the manner of an object.

  ![](/img/mongo-db-structure.svg)

raw_content:
  ---
  tags:
    - mongo-db
    - databases
  ---

  # MongoDB: Introduction

  MongoDB is not a relational database system like SQL, instead it is
  document-based which means you do not neeed to design database tables and
  schema, you are simply storing data as JSON and there is no transformation of
  the data.

  Most of the notes on Mongo will introduce it within the context of Node.js
  backend. We will be sending Javascript objects and arrays to Mongo and returning
  them as JSON.

  ## Databases, collections, documents

  Although Mongo is not a relational database it has a structure that we can
  understand in relation to that paradigm. A **database** is obviously the overall
  structure. It comprises **collections** which are organised sets of data that
  are analagous to [tables](Relational_database_architecture.md#table)
  in RDBs. Within each collection are a series of **documents** which we can think
  of as being equivalent to [rows](Relational_database_architecture.md)
  in RDB table: units that comprise the collection.

  A document is a container comprising key-value pairs in the manner of an object.

  ![](/img/mongo-db-structure.svg)


path: MongoDB_connection.md
sortable_path: MongoDB_connection.md
title: MongoDB connection
lead:
  ```js
  const mongoose = require("mongoose");

checksum: f4de15004137d56e214f1e0ef4b780799b7af6c7019bafcd8910bfd4064d5ec7
created: 2025-10-01 13:30:59.96389469+00:00
modified: 2025-08-13 15:57:13.891259999+00:00
metadata: {"tags":["mongo-db","node-js","mongoose","databases"]}
body:
  ```js
  const mongoose = require("mongoose");

  mongoose..connect("mongodb://127.0.0.1/[databse_name]");

  const courseSchema = new mongoose.Schema({
      name: String,
      author: String,
      tags: [String],
      data: Date,
      isPublished: Boolean,
      price: Number
  })

  const Course = mongoose.model("Course", courseSchema);

  async function getCourses(){
      return await Course
          .find({isPublished: true, tags: "backend"})
          .sort({name: 1})
          .select({name: 1, author: 1});
  }
  ```

raw_content:
  ---
  tags:
    - mongo-db
    - node-js
    - mongoose
    - databases
  ---

  # MongoDB connection

  ```js
  const mongoose = require("mongoose");

  mongoose..connect("mongodb://127.0.0.1/[databse_name]");

  const courseSchema = new mongoose.Schema({
      name: String,
      author: String,
      tags: [String],
      data: Date,
      isPublished: Boolean,
      price: Number
  })

  const Course = mongoose.model("Course", courseSchema);

  async function getCourses(){
      return await Course
          .find({isPublished: true, tags: "backend"})
          .sort({name: 1})
          .select({name: 1, author: 1});
  }
  ```


path: Monitor_DNS_resolution_and_internet_connectivity_script.md
sortable_path: Monitor_DNS_resolution_and_internet_connectivity_script.md
title: Monitor DNS resolution and internet connectivity script
checksum: 2783031db8f9eb0513eac607269b61e28142fb41b6b64f915b67878f8097e49b
created: 2025-10-01 13:30:59.968505562+00:00
modified: 2025-08-13 15:57:13.780922465+00:00
metadata: {"created":"Monday, January 13, 2025","tags":["networks","procedural"]}
lead:
  ```bash
   while true; do
      date
      echo "DNS check:"
      host google.com
      echo "Ping check:"
      ping -c 1 google.com | grep "time="
      echo "---"
      sleep 5
  done
  ```

body:
  ```bash
   while true; do
      date
      echo "DNS check:"
      host google.com
      echo "Ping check:"
      ping -c 1 google.com | grep "time="
      echo "---"
      sleep 5
  done
  ```

  See [ping](./Ping.md) for more details on what this is doing.

  The `host` command contacts a DNS server and does a reverse lookup for the
  IP-address of the human-readable domain name.

raw_content:
  ---
  tags: [networks, procedural]
  created: Monday, January 13, 2025
  ---

  # Monitor DNS resolution and internet connectivity script

  ```bash
   while true; do
      date
      echo "DNS check:"
      host google.com
      echo "Ping check:"
      ping -c 1 google.com | grep "time="
      echo "---"
      sleep 5
  done
  ```

  See [ping](./Ping.md) for more details on what this is doing.

  The `host` command contacts a DNS server and does a reverse lookup for the
  IP-address of the human-readable domain name.


path: Monitoring_processes_and_resources.md
sortable_path: Monitoring_processes_and_resources.md
title: Monitoring processes and resources
lead: ## General purpose diagnostic programs (memory, CPU, I/O)
checksum: aba8ca7ab10e1624e6a830b9a98e2479627e63943d7a9232d53c77d167fbfbe4
created: 2025-10-01 13:30:59.976983382+00:00
modified: 2025-08-13 15:57:14.012959814+00:00
metadata: {"tags":["systems-programming","Linux","procedural"]}
body:
  ## General purpose diagnostic programs (memory, CPU, I/O)

  ### `top`/`htop`

  We can use [ps](Processes.md) to list the
  currently running processes but it does not provide much information about the
  resource metrics or how the process changes over time. We can use `top` to get
  more information.

  `top` provides an interactive interface for the information that `ps` displays.
  It updates in real time and shows the most active processes based on the CPU
  time that they are utilising. You can also order by memory usage.

  _Here I have pressed `u` to show only the processes associated with my user:_

  ![](/img/htop.png)

  ### Main commands

  | Command | Action                          |
  | ------- | ------------------------------- |
  | -u      | Show processes by selected user |
  | M       | Sort by memory usage            |
  | P       | Sort by cumulative CPU usage    |
  | ?       | View key and explanation        |

  ### Understanding the categories

  - `Main/IO`
    - The first covers all processes. The second focuses on input/output processes
      (i.e. reading and writing to disks and other devices)
  - `PRI`
    - This stands for _priority_. This metric reflects the kernel's current
      schedule priority for the process. The higher the value, it is less likely
      that the kernel will schedule the process if there are competing processes
      that require CPU time. The lower the value, the greater priority this
      process has over others.
  - `NI`
    - This stands for _nice value_. This metric exists in order to allow
      administrators to nudge or influence the priority of a given process. You
      cannot directly tell the kernel to _do x now instead of y_ but you can make
      what are effectively suggestions by manipulating the nice value.
    - The kernel adds the nice value to the current priority value for the given
      process to determine its next time slot. When you increase the nice value of
      process _P_ you are being "nicer" to the other processes by influencing the
      priority of _P_ downwards so that the other processes receive greater
      precedence from the kernel.
    - By default, the nice value will be 0. To reduce priority of PID 1234, you
      would use:
      ```bash
      $ renice 20 1234
      ```
  - `VIRT`
    - The total amount of
      [virtual memory](Virtual_memory_and_the_MMU_in_Linux.md) used by
      the process including: program code, data, shared libraries, pages that have
      been swapped, pages that have been mapped but not used.
  - `RES`
    - Stands for _resident size_
    - The non swapped _physical_ memory the process has used
  - `SHR`
    - The size of the process's
      [shared pages](Virtual_memory_and_the_MMU_in_Linux.md#shared-pages)
  - `S`
    - Status:
      - S for sleeping (idle)
      - R for running
      - D for disk sleep

  ### `vmstat`

  `vmstat` provides similar metrics to `htop` but tells you more about the memory
  state and the activities of the kernel in a single row.

  The default output is a single line with the averages since boot. You can add a
  delay parameter (in secs) which will then output at that interval, allowing you
  to see memory usage in realtime, e.g:

  ```
  $ vmstat 5
  procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
   r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
   2  0      0 4326768 334228 5050952    0    0     8    19   80   10  4  1 94  0  0
   0  0      0 4365520 334260 5054468    0    0     0   125 2140 3434  4  1 94  0  0
   1  0      0 4382400 334276 5068940    0    0     0    77 2102 3988  3  1 95  0  0
   1  0      0 4434000 334288 5052908    0    0     0    25 2859 4278  6  1 92  0  0
   0  0      0 4391576 334304 5086484    0    0     0   110 2899 6480  8  3 90  0  0

  ```

  - `procs`
    - The number of runnable processes (`r`) and the number of blocked (`b`)
      processes
  - `memory`
    - The core memory output distinguishing:
      - Total kbs swapped to disk
      - Total kbs free
      - Total kbs currently in
        [buffer](Role_of_memory_in_computation.md#relation-between-cache-and-buffers)
        and not written
      - Total amount of virtual memory in the
        [cache](Role_of_memory_in_computation.md#relation-between-cache-and-buffers)
  - `swap`
    - Distinguishes amount of memory
      [swapped](Swap_space.md) in (`si`) to memory and
      swapped out (`so`) to disk
  - `io`
    - Disk actions
    - Amount of data read from harddisk (`bi`)
    - Amount of data written to harddisk (`bo`)
  - `system`
    - The number of times the kernel switches to kernel code
  - `cpu`
    - Percentage of the different CPU behaviours:
      - Responding to user tasks (`us`)
      - Time that it is idle (`id`)

  ## Files being used by active processes: `lsof`

  `lsof` stands for _list open files_. It lists opened files and the processes
  using them. Without modifiers it outputs a huge amount of data. The best way to
  use it is to execute it against a specific PID. For example the below output
  gives me some useful info about which files VS Code is using:

  ![](/img/lsof.png)

  ## System calls: `strace`

  A system call is when a process requests a service from the
  [kernel](The_kernel.md), for instance an I/O operation to
  memory. We can trace these system calls with `strace`.

  ## CPU performance

  We can use the `uptime` program to assess overall CPU performance in the form of
  a load average.

  > Load average is the number of active processes currently ready to run. It is
  > an estimate of the number of processes that are capable of using the CPU at
  > any given time.

  `Uptime` gives you three load averages:

  ```bash
  $ uptime
  11:19:16 up 14 days,  3:53,  1 user,  load average: 0.84, 0.57, 0.50
  ```

  - The three numbers are load averages for the past 1 minute, 5 minutes and 15
    minutes respectively.

  - A load average close to 0 is usually a good sign because it means that your
    processor isn't being challenged and you are conserving power. Anything equal
    to or above 1 means that a single process is using the CPU nearly all the
    time. You can identify that process with `htop` and it will obviously be near
    to the top. (This is often caused by Chrome and Electron-based software.)

  ## Memory status

  We know that processes primarily interact with virtual memory in the form of
  pages which are then translated to physical blocks by the kernel via the
  [MMU](Virtual_memory_and_the_MMU_in_Linux.md). There are several tools
  which provide windows onto this process.

  ### System page size

  We can view the overall system page size which is a representation of the amount
  of virtual memory available:

  ```bash
  $ getconf PAGE_SIZE
  4096
  ```

  This will typically be the same for all Linux systems.

  ### `free` : available physical memory

  `free` displays the total amount of free and¬used physical and swap memory in
  the system, as well as the
  [buffers and caches](Role_of_memory_in_computation.md#relation-between-cache-and-buffers)
  used by the kernel.

  ```bash
  $ free
                total        used        free      shared  buff/cache   available
  Mem:        16099420     5931512     5039344     2046460     5128564     7781904
  Swap:        3145724           0     3145724
  ```

raw_content:
  ---
  tags:
    - systems-programming
    - Linux
    - procedural
  ---

  # Monitoring processes and resources

  ## General purpose diagnostic programs (memory, CPU, I/O)

  ### `top`/`htop`

  We can use [ps](Processes.md) to list the
  currently running processes but it does not provide much information about the
  resource metrics or how the process changes over time. We can use `top` to get
  more information.

  `top` provides an interactive interface for the information that `ps` displays.
  It updates in real time and shows the most active processes based on the CPU
  time that they are utilising. You can also order by memory usage.

  _Here I have pressed `u` to show only the processes associated with my user:_

  ![](/img/htop.png)

  ### Main commands

  | Command | Action                          |
  | ------- | ------------------------------- |
  | -u      | Show processes by selected user |
  | M       | Sort by memory usage            |
  | P       | Sort by cumulative CPU usage    |
  | ?       | View key and explanation        |

  ### Understanding the categories

  - `Main/IO`
    - The first covers all processes. The second focuses on input/output processes
      (i.e. reading and writing to disks and other devices)
  - `PRI`
    - This stands for _priority_. This metric reflects the kernel's current
      schedule priority for the process. The higher the value, it is less likely
      that the kernel will schedule the process if there are competing processes
      that require CPU time. The lower the value, the greater priority this
      process has over others.
  - `NI`
    - This stands for _nice value_. This metric exists in order to allow
      administrators to nudge or influence the priority of a given process. You
      cannot directly tell the kernel to _do x now instead of y_ but you can make
      what are effectively suggestions by manipulating the nice value.
    - The kernel adds the nice value to the current priority value for the given
      process to determine its next time slot. When you increase the nice value of
      process _P_ you are being "nicer" to the other processes by influencing the
      priority of _P_ downwards so that the other processes receive greater
      precedence from the kernel.
    - By default, the nice value will be 0. To reduce priority of PID 1234, you
      would use:
      ```bash
      $ renice 20 1234
      ```
  - `VIRT`
    - The total amount of
      [virtual memory](Virtual_memory_and_the_MMU_in_Linux.md) used by
      the process including: program code, data, shared libraries, pages that have
      been swapped, pages that have been mapped but not used.
  - `RES`
    - Stands for _resident size_
    - The non swapped _physical_ memory the process has used
  - `SHR`
    - The size of the process's
      [shared pages](Virtual_memory_and_the_MMU_in_Linux.md#shared-pages)
  - `S`
    - Status:
      - S for sleeping (idle)
      - R for running
      - D for disk sleep

  ### `vmstat`

  `vmstat` provides similar metrics to `htop` but tells you more about the memory
  state and the activities of the kernel in a single row.

  The default output is a single line with the averages since boot. You can add a
  delay parameter (in secs) which will then output at that interval, allowing you
  to see memory usage in realtime, e.g:

  ```
  $ vmstat 5
  procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
   r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
   2  0      0 4326768 334228 5050952    0    0     8    19   80   10  4  1 94  0  0
   0  0      0 4365520 334260 5054468    0    0     0   125 2140 3434  4  1 94  0  0
   1  0      0 4382400 334276 5068940    0    0     0    77 2102 3988  3  1 95  0  0
   1  0      0 4434000 334288 5052908    0    0     0    25 2859 4278  6  1 92  0  0
   0  0      0 4391576 334304 5086484    0    0     0   110 2899 6480  8  3 90  0  0

  ```

  - `procs`
    - The number of runnable processes (`r`) and the number of blocked (`b`)
      processes
  - `memory`
    - The core memory output distinguishing:
      - Total kbs swapped to disk
      - Total kbs free
      - Total kbs currently in
        [buffer](Role_of_memory_in_computation.md#relation-between-cache-and-buffers)
        and not written
      - Total amount of virtual memory in the
        [cache](Role_of_memory_in_computation.md#relation-between-cache-and-buffers)
  - `swap`
    - Distinguishes amount of memory
      [swapped](Swap_space.md) in (`si`) to memory and
      swapped out (`so`) to disk
  - `io`
    - Disk actions
    - Amount of data read from harddisk (`bi`)
    - Amount of data written to harddisk (`bo`)
  - `system`
    - The number of times the kernel switches to kernel code
  - `cpu`
    - Percentage of the different CPU behaviours:
      - Responding to user tasks (`us`)
      - Time that it is idle (`id`)

  ## Files being used by active processes: `lsof`

  `lsof` stands for _list open files_. It lists opened files and the processes
  using them. Without modifiers it outputs a huge amount of data. The best way to
  use it is to execute it against a specific PID. For example the below output
  gives me some useful info about which files VS Code is using:

  ![](/img/lsof.png)

  ## System calls: `strace`

  A system call is when a process requests a service from the
  [kernel](The_kernel.md), for instance an I/O operation to
  memory. We can trace these system calls with `strace`.

  ## CPU performance

  We can use the `uptime` program to assess overall CPU performance in the form of
  a load average.

  > Load average is the number of active processes currently ready to run. It is
  > an estimate of the number of processes that are capable of using the CPU at
  > any given time.

  `Uptime` gives you three load averages:

  ```bash
  $ uptime
  11:19:16 up 14 days,  3:53,  1 user,  load average: 0.84, 0.57, 0.50
  ```

  - The three numbers are load averages for the past 1 minute, 5 minutes and 15
    minutes respectively.

  - A load average close to 0 is usually a good sign because it means that your
    processor isn't being challenged and you are conserving power. Anything equal
    to or above 1 means that a single process is using the CPU nearly all the
    time. You can identify that process with `htop` and it will obviously be near
    to the top. (This is often caused by Chrome and Electron-based software.)

  ## Memory status

  We know that processes primarily interact with virtual memory in the form of
  pages which are then translated to physical blocks by the kernel via the
  [MMU](Virtual_memory_and_the_MMU_in_Linux.md). There are several tools
  which provide windows onto this process.

  ### System page size

  We can view the overall system page size which is a representation of the amount
  of virtual memory available:

  ```bash
  $ getconf PAGE_SIZE
  4096
  ```

  This will typically be the same for all Linux systems.

  ### `free` : available physical memory

  `free` displays the total amount of free and¬used physical and swap memory in
  the system, as well as the
  [buffers and caches](Role_of_memory_in_computation.md#relation-between-cache-and-buffers)
  used by the kernel.

  ```bash
  $ free
                total        used        free      shared  buff/cache   available
  Mem:        16099420     5931512     5039344     2046460     5128564     7781904
  Swap:        3145724           0     3145724
  ```


path: Morgan.md
sortable_path: Morgan.md
title: Morgan
lead: Morgan is middleware that is used to log HTTP requests to the Express instance.
checksum: 26ede6ed4dc04da810194afc819dca34945f17243a0803689e91c1a03c4c2568
created: 2025-10-01 13:31:00.048017629+00:00
modified: 2025-08-13 15:57:13.861957922+00:00
metadata: {"tags":["node-js","middleware"]}
body:
  Morgan is middleware that is used to log HTTP requests to the Express instance.

  ```js
  app.use(morgan("dev"));
  ```

  With Morgan in place, every time we run a request it will be logged on the
  console that is running our Node application, e.g:

  ```plain
  GET /api/courses 200 95 - 1.774 ms
  ```

  This uses the `tiny` default which logs the bare minimum giving us: request
  type; endpoint; response code; and time to execute. But there are more verbose
  settings.

  It defaults to logging on the console but can also be configured to write to a
  log file.

raw_content:
  ---
  tags:
    - node-js
    - middleware
  ---

  # Morgan

  Morgan is middleware that is used to log HTTP requests to the Express instance.

  ```js
  app.use(morgan("dev"));
  ```

  With Morgan in place, every time we run a request it will be logged on the
  console that is running our Node application, e.g:

  ```plain
  GET /api/courses 200 95 - 1.774 ms
  ```

  This uses the `tiny` default which logs the bare minimum giving us: request
  type; endpoint; response code; and time to execute. But there are more verbose
  settings.

  It defaults to logging on the console but can also be configured to write to a
  log file.


path: Mosaic_Netscape_and_Browser_Wars.md
sortable_path: Mosaic_Netscape_and_Browser_Wars.md
title: c301a0b3_Mosaic_Netscape_and_Browser_Wars
checksum: 2932435c9c734455a7996da2c686501f21609773035248164eb5f8b8cc59abf8
created: 2025-10-01 13:31:00.053870045+00:00
modified: 2025-08-13 15:57:14.053852256+00:00
metadata: {"created":"Sunday, November 03, 2024","tags":["world-wide-web","computer-history","Microsoft"]}
lead:
  The Mosaic browser was conceived and produced at the University of Illinois
  during the very early growth of the Web, when TBL was still at CERN.

body:
  The Mosaic browser was conceived and produced at the University of Illinois
  during the very early growth of the Web, when TBL was still at CERN.

  _Mosaic_:

  ![Mosaic browser](../img/mosaic-browser.jpg)

  It became the best of the early browsers (a group that included ViolaWWW and
  Erwise) with the notable advantage of being able to embed graphics alongside
  text (instead of inside a separate window). It also supported multiple
  [application layer](Application_Layer_of_Internet_Protocol.md) protocols - FTP,
  the Network News Transfer Protocol, Gopher - not just HTTP.

  At this point there were nascent browsers for different operating systems and
  which browser you used was mostly determined by your OS. Being the product of a
  US university, Mosaic was originally written for Unix's X-Window system but was
  ported to multiple OSs later, a key factor in its success and uptake.

  Mosaic's creators formed their own company outside of the university: Netscape.
  The browser they created, Netscape Navigator became the heir to Mosaic, released
  in 1994.

  _Netscape Navigator_:

  ![Netscape Navigator](../img/Navigator_Netscape.png)

  It was made free of charge for non-commercial use which assured rapid uptake.
  This was done to outrun Microsoft's Internet Explorer (released two months after
  Netscape went public) which they intended to bundle with all new versions of
  Windows 95. Netscape became the de facto standard browser during the early Web,
  including on Windows. It took a few years for IE to overtake it.

  _Internet Explorer 1 with licensing info_:

  ![Internet Explorer](../img/internet_explorer.png)

  Netscape also birthed JavaScript, which was written rapidly specifically for the
  browser. Prior to this, dynamic client-side functionality was provided by Java
  'applets' which never really took off.

  IE in fact shares a genealogy with Mosaic since Microsoft licensed a version of
  the Mosaic code and used it to construct IE.

  By the end of the 1990s, Netscape's usage had declined precipitously as IE
  captured the market share.

raw_content:
  ---
  tags: [world-wide-web, computer-history, Microsoft]
  created: Sunday, November 03, 2024
  ---

  # c301a0b3_Mosaic_Netscape_and_Browser_Wars

  The Mosaic browser was conceived and produced at the University of Illinois
  during the very early growth of the Web, when TBL was still at CERN.

  _Mosaic_:

  ![Mosaic browser](../img/mosaic-browser.jpg)

  It became the best of the early browsers (a group that included ViolaWWW and
  Erwise) with the notable advantage of being able to embed graphics alongside
  text (instead of inside a separate window). It also supported multiple
  [application layer](Application_Layer_of_Internet_Protocol.md) protocols - FTP,
  the Network News Transfer Protocol, Gopher - not just HTTP.

  At this point there were nascent browsers for different operating systems and
  which browser you used was mostly determined by your OS. Being the product of a
  US university, Mosaic was originally written for Unix's X-Window system but was
  ported to multiple OSs later, a key factor in its success and uptake.

  Mosaic's creators formed their own company outside of the university: Netscape.
  The browser they created, Netscape Navigator became the heir to Mosaic, released
  in 1994.

  _Netscape Navigator_:

  ![Netscape Navigator](../img/Navigator_Netscape.png)

  It was made free of charge for non-commercial use which assured rapid uptake.
  This was done to outrun Microsoft's Internet Explorer (released two months after
  Netscape went public) which they intended to bundle with all new versions of
  Windows 95. Netscape became the de facto standard browser during the early Web,
  including on Windows. It took a few years for IE to overtake it.

  _Internet Explorer 1 with licensing info_:

  ![Internet Explorer](../img/internet_explorer.png)

  Netscape also birthed JavaScript, which was written rapidly specifically for the
  browser. Prior to this, dynamic client-side functionality was provided by Java
  'applets' which never really took off.

  IE in fact shares a genealogy with Mosaic since Microsoft licensed a version of
  the Mosaic code and used it to construct IE.

  By the end of the 1990s, Netscape's usage had declined precipitously as IE
  captured the market share.


path: Motherboard.md
sortable_path: Motherboard.md
title: Motherboard
lead: <img src="../img/motherboard-pi.jpg" width="400px"/>
checksum: 0b164f3be04afb2fabe087cf47088189f48663b1e63986bb463fa53d9e6680cf
created: 2025-10-01 13:31:00.062400034+00:00
modified: 2025-08-13 15:57:13.887265897+00:00
metadata: {"tags":["hardware","computer-architecture"]}
body:
  <img src="../img/motherboard-pi.jpg" width="400px"/>

  The motherboard is the foundation of a computer. It allocates power and allows
  communication to and between the
  [CPU](CPU_architecture.md),
  [RAM](Memory.md),
  [harddisk](What_are_disks.md) and all other hardware
  components.

  It is a printed circuit board and is always the largest board within the
  computer chassis.

raw_content:
  ---
  tags:
    - hardware
    - computer-architecture
  ---

  # Motherboard

  <img src="../img/motherboard-pi.jpg" width="400px"/>

  The motherboard is the foundation of a computer. It allocates power and allows
  communication to and between the
  [CPU](CPU_architecture.md),
  [RAM](Memory.md),
  [harddisk](What_are_disks.md) and all other hardware
  components.

  It is a printed circuit board and is always the largest board within the
  computer chassis.


path: Multiple_pointers.md
sortable_path: Multiple_pointers.md
title: Multiple pointers
checksum: c247415db8592dc1c4b8fca26b76001b0c53636ef5d2ef739c91ce609c3f8618
created: 2025-10-01 13:31:00.082861731+00:00
modified: 2025-08-13 15:57:13.975536204+00:00
metadata: {"tags":["design-patterns"]}
lead:
  The multiple pointers pattern can be useful for moving through an array and
  comparing elements against each other in a time-efficient manner and without
  costly multiple loops.

body:
  The multiple pointers pattern can be useful for moving through an array and
  comparing elements against each other in a time-efficient manner and without
  costly multiple loops.

  ## Example: moving pairs

  In this example we shuffle through an array creating a pair from each element
  encountered. Thus for an array such as `[4, 3, 2, 1, 3, 6, 7]` we would return
  `[ [ 4, 3 ], [ 3, 2 ], [ 2, 1 ], [ 1, 3 ], [ 3, 6 ], [ 6, 7 ] ]`.

  ```js
  function movingPairs(arr) {
    const store = [];
    let left = 0;
    let right = 1;
    for (left; left < arr.length - right; left++) {
      store.push([arr[left], arr[right + left]]);
    }
    return store;
  }
  ```

  ## Example: identify unique values

  An example application of the above pattern would be identifying the number of
  unique values in an array that contains duplicates. One way to do this in
  JavaScript would be to use a Set however this is a more generic approach that
  only requires a single loop:

  ```js
  function uniqueValues(arr) {
    if (!arr.length) {
      return 0;
    } else {
      let count = 1;
      let left = 0;
      let right = 1;
      for (right; right < arr.length; right++) {
        if (arr[left] !== arr[right]) {
          count++;
        }
        left++;
      }

      return count;
    }
  }
  ```

  This works by having two pointers at either end of the array; one moving from
  the right and one moving from the left. Whilst the right pointer is less than
  the array length, the left pointer is incremented as the right moves closer to
  it. At each incrementation, the values of pointers (used as indexes) are
  compared for uniqueness, and a count is kept.

  ## Example: identify duplicates

  This is very similar to the above but this time we are looking for duplicates
  rather than unique values.

  ```js
  function areThereDuplicatesPointer(...input) {
    let start = 0;
    let next = 1;
    while (next < input.length) {
      let same = input[start] === input[next];
      if (same) {
        return true;
      }
      start++;
      next++;
    }

    return false;
  }
  console.log(areThereDuplicatesPointer(1, 2, 3)); // false
  console.log(areThereDuplicatesPointer(1, 2, 2)); // true
  ```

raw_content:
  ---
  tags:
    - design-patterns
  ---

  # Multiple pointers

  The multiple pointers pattern can be useful for moving through an array and
  comparing elements against each other in a time-efficient manner and without
  costly multiple loops.

  ## Example: moving pairs

  In this example we shuffle through an array creating a pair from each element
  encountered. Thus for an array such as `[4, 3, 2, 1, 3, 6, 7]` we would return
  `[ [ 4, 3 ], [ 3, 2 ], [ 2, 1 ], [ 1, 3 ], [ 3, 6 ], [ 6, 7 ] ]`.

  ```js
  function movingPairs(arr) {
    const store = [];
    let left = 0;
    let right = 1;
    for (left; left < arr.length - right; left++) {
      store.push([arr[left], arr[right + left]]);
    }
    return store;
  }
  ```

  ## Example: identify unique values

  An example application of the above pattern would be identifying the number of
  unique values in an array that contains duplicates. One way to do this in
  JavaScript would be to use a Set however this is a more generic approach that
  only requires a single loop:

  ```js
  function uniqueValues(arr) {
    if (!arr.length) {
      return 0;
    } else {
      let count = 1;
      let left = 0;
      let right = 1;
      for (right; right < arr.length; right++) {
        if (arr[left] !== arr[right]) {
          count++;
        }
        left++;
      }

      return count;
    }
  }
  ```

  This works by having two pointers at either end of the array; one moving from
  the right and one moving from the left. Whilst the right pointer is less than
  the array length, the left pointer is incremented as the right moves closer to
  it. At each incrementation, the values of pointers (used as indexes) are
  compared for uniqueness, and a count is kept.

  ## Example: identify duplicates

  This is very similar to the above but this time we are looking for duplicates
  rather than unique values.

  ```js
  function areThereDuplicatesPointer(...input) {
    let start = 0;
    let next = 1;
    while (next < input.length) {
      let same = input[start] === input[next];
      if (same) {
        return true;
      }
      start++;
      next++;
    }

    return false;
  }
  console.log(areThereDuplicatesPointer(1, 2, 3)); // false
  console.log(areThereDuplicatesPointer(1, 2, 2)); // true
  ```


path: Multiplexers_and_demultiplexers.md
sortable_path: Multiplexers_and_demultiplexers.md
title: Multiplexers (MUX) and demultiplexers (DMUX)
lead: ## Multiplexer
checksum: 0323d69672331653828e0e1eb7e963a330e2e4b67995bc169955b8bc65148eb7
created: 2025-10-01 13:31:00.08864631+00:00
modified: 2025-08-13 15:57:13.837997608+00:00
metadata: {"tags":["logic-gates","binary","nand-to-tetris"]}
body:
  ## Multiplexer

  > Multiplexing is the generic term used to describe the operation of sending one
  > or more analogue or digital signals over a common transmission line at
  > different times or speeds.

  A multiplexer selects one of several input signals and forwards ths selected
  input to a single output line.

  We have two inputs (A,B) plus a third input SEL (for "select"). Applying a value
  to SEL toggles the output between A and B.

  ![](/img/MUX.png)

  Multiplexers can be used to build larger circuits by connecting the output of
  one multiplexer to the input of another. They are often used to implement data
  selection and switching in digital systems

  ### Programable gates

  One of the main use cases of multiplexers is to implement programmable gates.
  These are gates where the logic can be switched. For example an ANDMUXOR gate
  uses the SEL value to toggle the operation of a gate between AND and OR for its
  two inputs A and B

  ## Demultiplexer

  As the name suggests, a demultiplexer reverses the functionality of a
  multiplexer. It receives a single input and based on the selection of the SEL
  input it channels it to either an A or a B output.

  ![](/img/DMUX.png)

  We can think of it as a distributor of a value into one of several possible
  channels.

raw_content:
  ---
  tags: [logic-gates, binary, nand-to-tetris]
  ---

  # Multiplexers (MUX) and demultiplexers (DMUX)

  ## Multiplexer

  > Multiplexing is the generic term used to describe the operation of sending one
  > or more analogue or digital signals over a common transmission line at
  > different times or speeds.

  A multiplexer selects one of several input signals and forwards ths selected
  input to a single output line.

  We have two inputs (A,B) plus a third input SEL (for "select"). Applying a value
  to SEL toggles the output between A and B.

  ![](/img/MUX.png)

  Multiplexers can be used to build larger circuits by connecting the output of
  one multiplexer to the input of another. They are often used to implement data
  selection and switching in digital systems

  ### Programable gates

  One of the main use cases of multiplexers is to implement programmable gates.
  These are gates where the logic can be switched. For example an ANDMUXOR gate
  uses the SEL value to toggle the operation of a gate between AND and OR for its
  two inputs A and B

  ## Demultiplexer

  As the name suggests, a demultiplexer reverses the functionality of a
  multiplexer. It receives a single input and based on the selection of the SEL
  input it channels it to either an A or a B output.

  ![](/img/DMUX.png)

  We can think of it as a distributor of a value into one of several possible
  channels.


path: Multiplicative_identity.md
sortable_path: Multiplicative_identity.md
title: The Property of Multiplicative Identity
lead: **Let $a$ represent any member of $\mathbb{W}$ or $\mathbb{Z}$ then:**
body:
  **Let $a$ represent any member of $\mathbb{W}$ or $\mathbb{Z}$ then:**

  $$ a \cdot 1 = a $$

checksum: 52f24c40c400082277ab21ab5b035d11337ff52976576ef802044d363d17c6d0
created: 2025-10-01 13:31:00.093534288+00:00
modified: 2025-08-13 15:57:13.855923879+00:00
metadata: {"tags":["prealgebra","theorems"]}
raw_content:
  ---
  tags:
    - prealgebra
    - theorems
  ---

  # The Property of Multiplicative Identity

  **Let $a$ represent any member of $\mathbb{W}$ or $\mathbb{Z}$ then:**

  $$ a \cdot 1 = a $$


path: Multiplicative_property_of_negative_one.md
sortable_path: Multiplicative_property_of_negative_one.md
title: The Multiplicative Property of Negative One
lead: > **Let $a$ represent any member of $\mathbb{Z}$, then:**
body:
  > **Let $a$ represent any member of $\mathbb{Z}$, then:**

  $$ (-1) \cdot a = -a $$

checksum: 3ea5a89ea5f9b12ba31084f176279eb700d16beb03d37584492c50100ba5a991
created: 2025-10-01 13:31:00.098061697+00:00
modified: 2025-08-13 15:57:13.996969081+00:00
metadata: {"tags":["prealgebra","theorems"]}
raw_content:
  ---
  tags:
    - prealgebra
    - theorems
  ---

  # The Multiplicative Property of Negative One

  > **Let $a$ represent any member of $\mathbb{Z}$, then:**

  $$ (-1) \cdot a = -a $$


path: Multiplying_fractions.md
sortable_path: Multiplying_fractions.md
title: Multiplying fractions
checksum: e693fb7d5cd07d76626f4e31e791a7cbb8bbd2effeb8cfe3169c7f2999b58136
created: 2025-10-01 13:31:00.102091452+00:00
modified: 2025-08-13 15:57:14.007432694+00:00
metadata: {"tags":["prealgebra","fractions","arithmetic"]}
lead:
  To find the product of two fractions $\frac{a}{b}$ and $\frac{c}{d}$ multiply
  their numerators and denominators and then reduce:

body:
  To find the product of two fractions $\frac{a}{b}$ and $\frac{c}{d}$ multiply
  their numerators and denominators and then reduce:

  $$\frac{a}{b} \cdot \frac{c}{d} = \frac{a \cdot c}{b \cdot d}$$

  ### Example

  $$
  \frac{1}{3} \cdot \frac{2}{5} = \frac{1 \cdot 2}{3 \cdot 5} = \frac{2}{15}
  $$

  ## Prime factorisation in place

  The example above did not require a reduction, so here is a more complex
  example:

  $$
  \frac{14}{15} \cdot \frac{30}{140} = \frac{420}{2100}
  $$

  It would be laborious to reduce such a large product using factor trees or the
  repeated application of divisors, as defined in
  [reducing fractions](Reducing_fractions.md). We can use a more efficient
  method. This method can be applied at the point at which we conduct the
  multiplication rather than afterwards once we have the product. We express the
  the initial multiplicands as prime factors:

  $$
  \frac{14}{15} \cdot \frac{30}{140} = \frac{(2 \cdot 7) \cdot (2 \cdot 3 \cdot 5) }{(3 \cdot 5) \cdot (2 \cdot  2 \cdot 7 \cdot 5)}
  $$

  We now have the product in factorised form before we have applied the
  multiplication so we can go ahead and cancel:

  $$
  \frac{\cancel{2}, \cancel{7}, \cancel{2}, \cancel{3}, \cancel{5}}{\cancel{3}, \cancel{5}, \cancel{2}, \cancel{2}, \cancel{7}, 5} = \frac{1}{5}
  $$

  **Note that in the above case, there was only a single 5 left as a denominator
  and no value left as a numerator. This is equivalent to there just being "one
  five" so we write $\frac{1}{5}$**

  ## Example of multiplying fractions with negative fractions containing variables

  Calculate: $$- \frac{6x}{55y} \cdot - \frac{110y^2}{105x^2}$$

  First multiply in place:

  $$
  \frac{(3 \cdot 2 \cdot x) \cdot (5  \cdot 2 \cdot 11 \cdot y \cdot y)}{(5 \cdot 11 \cdot y) \cdot (7 \cdot 5 \cdot 3 \cdot x \cdot x)}
  $$

  Then cancel:

  $$
  \frac{(\cancel{3} \cdot 2 \cdot \cancel{x}) \cdot (\cancel{5}  \cdot 2 \cdot \cancel{11} \cdot \cancel{y} \cdot y)}{(\cancel{5} \cdot \cancel{11} \cdot \cancel{y}) \cdot (7 \cdot 5 \cdot \cancel{3} \cdot \cancel{x} \cdot x)} =
  \frac{2  \cdot 2 \cdot y}{7 \cdot 5 \cdot x}
  $$

  Then reduce:

  $$
  \frac{2  \cdot 2 \cdot y}{7 \cdot 5 \cdot x} = \frac{4y}{35x}
  $$

raw_content:
  ---
  tags:
    - prealgebra
    - fractions
    - arithmetic
  ---

  # Multiplying fractions

  To find the product of two fractions $\frac{a}{b}$ and $\frac{c}{d}$ multiply
  their numerators and denominators and then reduce:

  $$\frac{a}{b} \cdot \frac{c}{d} = \frac{a \cdot c}{b \cdot d}$$

  ### Example

  $$
  \frac{1}{3} \cdot \frac{2}{5} = \frac{1 \cdot 2}{3 \cdot 5} = \frac{2}{15}
  $$

  ## Prime factorisation in place

  The example above did not require a reduction, so here is a more complex
  example:

  $$
  \frac{14}{15} \cdot \frac{30}{140} = \frac{420}{2100}
  $$

  It would be laborious to reduce such a large product using factor trees or the
  repeated application of divisors, as defined in
  [reducing fractions](Reducing_fractions.md). We can use a more efficient
  method. This method can be applied at the point at which we conduct the
  multiplication rather than afterwards once we have the product. We express the
  the initial multiplicands as prime factors:

  $$
  \frac{14}{15} \cdot \frac{30}{140} = \frac{(2 \cdot 7) \cdot (2 \cdot 3 \cdot 5) }{(3 \cdot 5) \cdot (2 \cdot  2 \cdot 7 \cdot 5)}
  $$

  We now have the product in factorised form before we have applied the
  multiplication so we can go ahead and cancel:

  $$
  \frac{\cancel{2}, \cancel{7}, \cancel{2}, \cancel{3}, \cancel{5}}{\cancel{3}, \cancel{5}, \cancel{2}, \cancel{2}, \cancel{7}, 5} = \frac{1}{5}
  $$

  **Note that in the above case, there was only a single 5 left as a denominator
  and no value left as a numerator. This is equivalent to there just being "one
  five" so we write $\frac{1}{5}$**

  ## Example of multiplying fractions with negative fractions containing variables

  Calculate: $$- \frac{6x}{55y} \cdot - \frac{110y^2}{105x^2}$$

  First multiply in place:

  $$
  \frac{(3 \cdot 2 \cdot x) \cdot (5  \cdot 2 \cdot 11 \cdot y \cdot y)}{(5 \cdot 11 \cdot y) \cdot (7 \cdot 5 \cdot 3 \cdot x \cdot x)}
  $$

  Then cancel:

  $$
  \frac{(\cancel{3} \cdot 2 \cdot \cancel{x}) \cdot (\cancel{5}  \cdot 2 \cdot \cancel{11} \cdot \cancel{y} \cdot y)}{(\cancel{5} \cdot \cancel{11} \cdot \cancel{y}) \cdot (7 \cdot 5 \cdot \cancel{3} \cdot \cancel{x} \cdot x)} =
  \frac{2  \cdot 2 \cdot y}{7 \cdot 5 \cdot x}
  $$

  Then reduce:

  $$
  \frac{2  \cdot 2 \cdot y}{7 \cdot 5 \cdot x} = \frac{4y}{35x}
  $$


path: Mutations_with_Apollo_Client.md
sortable_path: Mutations_with_Apollo_Client.md
title: Mutations with Apollo Client
lead: Queries are read-only operations. Mutations are write-only operations.
checksum: 1de70df0fe706eb5b212250f1aa7a3596e9b54cfc0587fcf729631b096a69d60
created: 2025-10-01 13:31:00.114519713+00:00
modified: 2025-08-13 15:57:13.741691468+00:00
metadata: {"tags":["graphql"]}
body:
  Queries are read-only operations. Mutations are write-only operations.

  Just like the `Query` type, the `Mutation` type serves as an entrypoint to the
  schema.

  ## Naming convention

  - Use verb: `add`, `create`, `delete`
  - Refer to the datatype

  For example `addSpaceCat(){}`

  ## Demonstration mutation

  We are going to create a mutation that increments the `numberOfViews` field on
  the `Track` type:

  ## Updating the schema

  ```js
  type Mutation {
      incrementTrackViews(id: ID!): IncrementTrackViewsResponse!
  }


  // Define a specific response type that specifically matches our needs
  type IncrementTrackViewsResponse {
      code: Int! // status code
      success: Boolean! // whether mutation was successful
      message: String! // what to say if mutation successful
      track: Track   // not nullable because might error
  }
  ```

  Based on this schema, the mutation will recieve a `Track` id and increment the
  specified `Track`. It will return an object comprising the newly updated `Track`
  and a bundle of properties that provide feedback on the status of the
  operations: a status code, whether it succeeded, and a message.

  ## Updating the data source

  Remember that our sole data source in the demonstration project is a REST API.
  We handle it within GraphQL using Apollos `RESTDataSource` class. We need to add
  a method to this class that will increment the track views. We wil use the
  `PATCH` REST method:

  ```js
  class TrackAPI extends RESTDataSource {
    constructor() {...}
    getTracksForHome() {...}
    getAuthor(authorId) {...}
    getTrack(trackId){...};

    incrementTrackViews(trackId) {
      return this.patch(`track/${trackId}/numberOfViews`);
    }
  }
  ```

  The `patch()` method is procided by the `RESTDataSouce` class that `TrackAPI`
  inherits from

  ## Adding resolver

  Next we need a resolver that corresponds to the mutation we have defined in the
  schema. We will need to handle successful responses as well as errors.

  ### Success case

  As always we match the shape of the schema:

  ```js
  const resolvers = {
    Query: {
      // ...query resolvers
    },
    Mutation: {
      // increments a track's numberOfViews property
      incrementTrackViews: async (_, { id }, { dataSources }) => {
        const track = await dataSources.trackAPI.incrementTrackViews(id);
        return {
          code: 200,
          success: true,
          message: `Successfully incremented number of views for track ${id}`,
          track,
        };
      },
    },
  };
  ```

  There's more going on with this resolver than the previous one. As is standard,
  we call the API using the `TrackAPI` class. However we don't just immediately
  return this when it executes. This is because the schema specifies that the
  return type `IncrementTrackViewsResponse` requires more than just the updated
  `Track`. So we wait this and return it with the cluster of metadata about the
  mutation response (`code`, `success`, and `message`).

  ### Error case

  We can extend the Mutation resolver to allow for errors. We'll do this by
  refactoring the resolver into a `try...catch` block and adding the error
  handling in the `catch`.

  We'll harness the details that are provided by Apollos' own `err` object which
  is returned by the `RESTDataSource` class that our resolver ultimately traces
  back to:

  ```js
  const resolvers = {

      Query: {
          // ...query resolvers
      }

      Mutation: {
          incrementTrackViews: async (_, {id}, {dataSources}) => {
              try {
                  const track = await dataSources.trackAPI.incrementTrackViews(id);
                  return {
                      code: 200,
                      success: true,
                      message: `Successfully incremented number of views for track ${id}`,
                      track
                  };
              } catch (err) {
                  return {
                      code: err.extensions.response.status,
                      success: false,
                      message: err.extensions.response.body,
                      track: null
                  };
              }
          },
      }
  }
  ```

  ## The `useMutation` hook

  We invoke the `useMutation` hook to issue mutations from the client-side.

  As with queries and
  [query constants](Apollo_Client.md#query-constants) we
  wrap our mutation in a `gql` template string:

  ```js
  const INCREMENT_TRACK_VIEWS = gql`
    mutation IncrementTrackViews($incrementTrackViewsId: ID!) {
      incrementTrackViews(id: $incrementTrackViewsId) {
        code
        success
        message
        track {
          id
          numberOfViews
        }
      }
    }
  `;
  ```

  We then pass it to the `useMutation` hook including an options object with our
  variables. (This time the specific variable is named):

  ```js
  import { gql, useMutation } from "@apollo/client";

  useMutation(INCREMENT_TRACK_VIEWS, {
    variables: { incrementTrackViewsId: id },
  });
  ```

  `useMutation` returns an array of two elements:

  1. The mutation function that actually executes
  2. An object comprising (`loading`, `error`, `data`) - this is the same as is
     the return value of `useQuery`.

  So we can destructure like so (we don't always need the second element);

  ```js
  const [incrementTrackViews, dataObject] = useMutation(INCREMENT_TRACK_VIEWS...)
  ```

  Given that we can isolate the mutation function as the first destructured
  element of the array, we could then attach `incrementTrackViews` to a button or
  other frontend interaction.

raw_content:
  ---
  tags: [graphql]
  ---
  	
  # Mutations with Apollo Client

  Queries are read-only operations. Mutations are write-only operations.

  Just like the `Query` type, the `Mutation` type serves as an entrypoint to the
  schema.

  ## Naming convention

  - Use verb: `add`, `create`, `delete`
  - Refer to the datatype

  For example `addSpaceCat(){}`

  ## Demonstration mutation

  We are going to create a mutation that increments the `numberOfViews` field on
  the `Track` type:

  ## Updating the schema

  ```js
  type Mutation {
      incrementTrackViews(id: ID!): IncrementTrackViewsResponse!
  }


  // Define a specific response type that specifically matches our needs
  type IncrementTrackViewsResponse {
      code: Int! // status code
      success: Boolean! // whether mutation was successful
      message: String! // what to say if mutation successful
      track: Track   // not nullable because might error
  }
  ```

  Based on this schema, the mutation will recieve a `Track` id and increment the
  specified `Track`. It will return an object comprising the newly updated `Track`
  and a bundle of properties that provide feedback on the status of the
  operations: a status code, whether it succeeded, and a message.

  ## Updating the data source

  Remember that our sole data source in the demonstration project is a REST API.
  We handle it within GraphQL using Apollos `RESTDataSource` class. We need to add
  a method to this class that will increment the track views. We wil use the
  `PATCH` REST method:

  ```js
  class TrackAPI extends RESTDataSource {
    constructor() {...}
    getTracksForHome() {...}
    getAuthor(authorId) {...}
    getTrack(trackId){...};

    incrementTrackViews(trackId) {
      return this.patch(`track/${trackId}/numberOfViews`);
    }
  }
  ```

  The `patch()` method is procided by the `RESTDataSouce` class that `TrackAPI`
  inherits from

  ## Adding resolver

  Next we need a resolver that corresponds to the mutation we have defined in the
  schema. We will need to handle successful responses as well as errors.

  ### Success case

  As always we match the shape of the schema:

  ```js
  const resolvers = {
    Query: {
      // ...query resolvers
    },
    Mutation: {
      // increments a track's numberOfViews property
      incrementTrackViews: async (_, { id }, { dataSources }) => {
        const track = await dataSources.trackAPI.incrementTrackViews(id);
        return {
          code: 200,
          success: true,
          message: `Successfully incremented number of views for track ${id}`,
          track,
        };
      },
    },
  };
  ```

  There's more going on with this resolver than the previous one. As is standard,
  we call the API using the `TrackAPI` class. However we don't just immediately
  return this when it executes. This is because the schema specifies that the
  return type `IncrementTrackViewsResponse` requires more than just the updated
  `Track`. So we wait this and return it with the cluster of metadata about the
  mutation response (`code`, `success`, and `message`).

  ### Error case

  We can extend the Mutation resolver to allow for errors. We'll do this by
  refactoring the resolver into a `try...catch` block and adding the error
  handling in the `catch`.

  We'll harness the details that are provided by Apollos' own `err` object which
  is returned by the `RESTDataSource` class that our resolver ultimately traces
  back to:

  ```js
  const resolvers = {

      Query: {
          // ...query resolvers
      }

      Mutation: {
          incrementTrackViews: async (_, {id}, {dataSources}) => {
              try {
                  const track = await dataSources.trackAPI.incrementTrackViews(id);
                  return {
                      code: 200,
                      success: true,
                      message: `Successfully incremented number of views for track ${id}`,
                      track
                  };
              } catch (err) {
                  return {
                      code: err.extensions.response.status,
                      success: false,
                      message: err.extensions.response.body,
                      track: null
                  };
              }
          },
      }
  }
  ```

  ## The `useMutation` hook

  We invoke the `useMutation` hook to issue mutations from the client-side.

  As with queries and
  [query constants](Apollo_Client.md#query-constants) we
  wrap our mutation in a `gql` template string:

  ```js
  const INCREMENT_TRACK_VIEWS = gql`
    mutation IncrementTrackViews($incrementTrackViewsId: ID!) {
      incrementTrackViews(id: $incrementTrackViewsId) {
        code
        success
        message
        track {
          id
          numberOfViews
        }
      }
    }
  `;
  ```

  We then pass it to the `useMutation` hook including an options object with our
  variables. (This time the specific variable is named):

  ```js
  import { gql, useMutation } from "@apollo/client";

  useMutation(INCREMENT_TRACK_VIEWS, {
    variables: { incrementTrackViewsId: id },
  });
  ```

  `useMutation` returns an array of two elements:

  1. The mutation function that actually executes
  2. An object comprising (`loading`, `error`, `data`) - this is the same as is
     the return value of `useQuery`.

  So we can destructure like so (we don't always need the second element);

  ```js
  const [incrementTrackViews, dataObject] = useMutation(INCREMENT_TRACK_VIEWS...)
  ```

  Given that we can isolate the mutation function as the first destructured
  element of the array, we could then attach `incrementTrackViews` to a button or
  other frontend interaction.


path: Naming_conventions.md
sortable_path: Naming_conventions.md
title: Python naming conventions
lead: ## Underscores
checksum: f491ff1235bf5bb1a424568b00d219bbb0760f8c2b82962ba863401322fbac87
created: 2025-10-01 13:31:00.122019776+00:00
modified: 2025-08-13 15:57:14.073413316+00:00
metadata: {"tags":["python"]}
body:
  ## Underscores

  To name a variable or method with multiple words we use underscores to separate
  each word.

  ```python
  an_int = 32
  print(an_integer.is_integer) # true
  ```

  ## Constants

  There are no constants in Python. However there a pseudo-constants established
  by convention. To denote that a variable should not change you use `UPPER_CASE`.

raw_content:
  ---
  tags: [python]
  ---

  # Python naming conventions

  ## Underscores

  To name a variable or method with multiple words we use underscores to separate
  each word.

  ```python
  an_int = 32
  print(an_integer.is_integer) # true
  ```

  ## Constants

  There are no constants in Python. However there a pseudo-constants established
  by convention. To denote that a variable should not change you use `UPPER_CASE`.


path: Natural_numbers.md
sortable_path: Natural_numbers.md
title: Natural numbers
lead: ## The set of natural numbers
checksum: 7efc4b198d7fcaf01e4bda93b0f3898b0820ee491fe04ffed59f192c44177e63
created: 2025-10-01 13:31:00.126945056+00:00
modified: 2025-08-13 15:57:13.765922182+00:00
metadata: {"tags":["prealgebra"]}
body:
  ## The set of natural numbers

  $$ \mathbb{N} = {1, 2, 3, ...} $$

  Natural numbers are most simply expressed as **the set of numbers we use for
  counting**. The set of natural numbers start at one and continuity to infinity.

raw_content:
  ---
  tags:
    - prealgebra
  ---

  # Natural numbers

  ## The set of natural numbers

  $$ \mathbb{N} = {1, 2, 3, ...} $$

  Natural numbers are most simply expressed as **the set of numbers we use for
  counting**. The set of natural numbers start at one and continuity to infinity.


path: Negation_Elimination.md
sortable_path: Negation_Elimination.md
title: Negation Elimination
checksum: 7d0d8a02311c15474ed12c7515ca6737cc8a23152c106636d6ba9197ad5c8596
created: 2025-10-01 13:31:00.13187251+00:00
modified: 2025-08-13 15:57:13.838584481+00:00
metadata: {"tags":["logic","propositional-logic"]}
lead:
  Like the [introduction](Negation_Introduction.md) rule for
  negation, the elimination rule also works by deriving a contradiction. It is
  basically _Negation Introduction_ in reverse. Instead of starting the sub-proof
  with a true proposition from which you derive a contradiction, you start with
  the negation of a proposition, derive a contradiction and then assert the
  positive of the negated proposition you started out with.

body:
  Like the [introduction](Negation_Introduction.md) rule for
  negation, the elimination rule also works by deriving a contradiction. It is
  basically _Negation Introduction_ in reverse. Instead of starting the sub-proof
  with a true proposition from which you derive a contradiction, you start with
  the negation of a proposition, derive a contradiction and then assert the
  positive of the negated proposition you started out with.

  ![](/img/negate-elim.png)

raw_content:
  ---
  tags:
    - logic
    - propositional-logic
  ---

  # Negation Elimination

  Like the [introduction](Negation_Introduction.md) rule for
  negation, the elimination rule also works by deriving a contradiction. It is
  basically _Negation Introduction_ in reverse. Instead of starting the sub-proof
  with a true proposition from which you derive a contradiction, you start with
  the negation of a proposition, derive a contradiction and then assert the
  positive of the negated proposition you started out with.

  ![](/img/negate-elim.png)


path: Negation_Introduction.md
sortable_path: Negation_Introduction.md
title: Negation Introduction
checksum: d60e1513c026e4684b3285bf1af37e05e071a19a13e398ba5277532d38ee90cd
created: 2025-10-01 13:31:00.144101309+00:00
modified: 2025-08-13 15:57:13.86255214+00:00
metadata: {"tags":["logic","propositional-logic"]}
lead:
  This is also known as _proof by contradiction_. You start with an assumption
  declared in a sub-proof. If you can derive a contradiction from this assumption
  (typically from the introduction of another proposition and its negation), then
  you are permitted to derive the negation of the auxiliary assumption in the main
  proof.

body:
  This is also known as _proof by contradiction_. You start with an assumption
  declared in a sub-proof. If you can derive a contradiction from this assumption
  (typically from the introduction of another proposition and its negation), then
  you are permitted to derive the negation of the auxiliary assumption in the main
  proof.

  ![](/img/negate-intro.png)

raw_content:
  ---
  tags:
    - logic
    - propositional-logic
  ---

  # Negation Introduction

  This is also known as _proof by contradiction_. You start with an assumption
  declared in a sub-proof. If you can derive a contradiction from this assumption
  (typically from the introduction of another proposition and its negation), then
  you are permitted to derive the negation of the auxiliary assumption in the main
  proof.

  ![](/img/negate-intro.png)


path: Negative_exponents.md
sortable_path: Negative_exponents.md
title: Negative exponents
lead:
  When calculating the exponents of a negative number the answer will always will
  be positive:

checksum: 09c48c27a5b3a332373c87afbd54f52a46d92b0637e5b788c2135c0adfb13834
created: 2025-10-01 13:31:00.149009609+00:00
modified: 2025-08-13 15:57:13.909225648+00:00
metadata: {"tags":["algebra","exponents"]}
body:
  When calculating the exponents of a negative number the answer will always will
  be positive:

  $$
  -5^2 = 25
  $$

  This confused me but it was because I was thinking of it in terms of
  $-5 \cdot 5$ when in fact it is $-5 \cdot -5$ and when two negative numbers are
  multiplied the product is always negative.

raw_content:
  ---
  tags: [algebra, exponents]
  ---

  # Negative exponents

  When calculating the exponents of a negative number the answer will always will
  be positive:

  $$
  -5^2 = 25
  $$

  This confused me but it was because I was thinking of it in terms of
  $-5 \cdot 5$ when in fact it is $-5 \cdot -5$ and when two negative numbers are
  multiplied the product is always negative.


path: Network_Layer_clarification.md
sortable_path: Network_Layer_clarification.md
title: a0ab0bfb_network_layer_clarification
checksum: 6fe734cba1763497e75b107efeb4f7ae8e97f179505adc48e73803a77e7c5ed3
created: 2025-10-01 13:31:00.15390302+00:00
modified: 2025-08-13 15:57:13.944271976+00:00
metadata: {"created":"Sunday, March 02, 2025","tags":["networks","internet"]}
lead:
  I have difficulty understanding the places at which the different layers of the
  TCP/IP model of the Internet reside and their spatiality in terms of the overall
  system. These are some clarifying remarks.

body:
  I have difficulty understanding the places at which the different layers of the
  TCP/IP model of the Internet reside and their spatiality in terms of the overall
  system. These are some clarifying remarks.

  All layers exist on every device involved in the communication process. But not
  all layers are accessed by every device in the overall transaction.

  > While the entire encapsulated packet travels across the internet, different
  > devices interact with different layers of that packet based on their role in
  > the network.

  The server and client access data at all layers:

  - [Application](./Application_Layer_of_Internet_Protocol.md)
  - [Transport](./Transport_Layer_of_Internet_Protocol.md)
  - [Internet](./Internet_Layer_of_Internet_Protocol.md)
  - [Link](./Link_Layer_of_Internet_Protocol.md)

  This is because the data must be composed on the device, a route planned and
  then sent accross the network where it is processed at the local network layer.

  Routers however only care about routing the data, forwarding packets based on
  their IP addresses, they only process data up to the Internet Layer.

raw_content:
  ---
  tags: [networks, internet]
  created: Sunday, March 02, 2025
  ---

  # a0ab0bfb_network_layer_clarification

  I have difficulty understanding the places at which the different layers of the
  TCP/IP model of the Internet reside and their spatiality in terms of the overall
  system. These are some clarifying remarks.

  All layers exist on every device involved in the communication process. But not
  all layers are accessed by every device in the overall transaction.

  > While the entire encapsulated packet travels across the internet, different
  > devices interact with different layers of that packet based on their role in
  > the network.

  The server and client access data at all layers:

  - [Application](./Application_Layer_of_Internet_Protocol.md)
  - [Transport](./Transport_Layer_of_Internet_Protocol.md)
  - [Internet](./Internet_Layer_of_Internet_Protocol.md)
  - [Link](./Link_Layer_of_Internet_Protocol.md)

  This is because the data must be composed on the device, a route planned and
  then sent accross the network where it is processed at the local network layer.

  Routers however only care about routing the data, forwarding packets based on
  their IP addresses, they only process data up to the Internet Layer.


path: Network_card.md
sortable_path: Network_card.md
title: Network card
checksum: 76c1d9ecb72c7e71e23119bd1159d51e40a3a9da798a073ba9888eddeedad437
created: 2025-10-01 13:31:00.182316178+00:00
modified: 2025-08-13 15:57:13.763801246+00:00
metadata: {"created":"Tuesday, August 06, 2024","tags":["networks","hardware"]}
lead:
  The network card (technically know as the Network Interface Card (NIC)) or
  _network adapter_ is the hardware device that allows a computer to connect to a
  network.

body:
  The network card (technically know as the Network Interface Card (NIC)) or
  _network adapter_ is the hardware device that allows a computer to connect to a
  network.

  They are the physical interface between the computer and the network
  [link](./Link_Layer_of_Internet_Protocol.md), e.g. WiFi or ethernet.

  They can be internal and built into the [motherboard](./Motherboard.md) or
  external as in the case of USB adaptors.

  Each network card has a unique [MAC address](./MAC_addresses.md) to identify the
  device on the network.

  Network cards on consumer-grade devices typically utilise the devices resources
  and the kernel manages its actions via a driver. More high-end devices
  (typically used in servers) have their own processors and memory.

raw_content:
  ---
  tags: [networks, hardware]
  created: Tuesday, August 06, 2024
  ---

  # Network card

  The network card (technically know as the Network Interface Card (NIC)) or
  _network adapter_ is the hardware device that allows a computer to connect to a
  network.

  They are the physical interface between the computer and the network
  [link](./Link_Layer_of_Internet_Protocol.md), e.g. WiFi or ethernet.

  They can be internal and built into the [motherboard](./Motherboard.md) or
  external as in the case of USB adaptors.

  Each network card has a unique [MAC address](./MAC_addresses.md) to identify the
  device on the network.

  Network cards on consumer-grade devices typically utilise the devices resources
  and the kernel manages its actions via a driver. More high-end devices
  (typically used in servers) have their own processors and memory.


path: Network_fundamentals.md
sortable_path: Network_fundamentals.md
title: Network fundamentals
checksum: cc82f0dcd8355ab2f7af5eda572679f060a725888b69ea268ca70bdaacb8a794
created: 2025-10-01 13:31:00.198696701+00:00
modified: 2025-08-13 15:57:13.836898296+00:00
metadata: {"created":"Saturday, August 03, 2024","tags":["networks","network-protocols"]}
lead:
  > A network is a system that allows computing devices to communicate and
  > exchange information with each other.

body:
  > A network is a system that allows computing devices to communicate and
  > exchange information with each other.

  In order for devices to be able to communicate they must share a common
  **communication protocol**.

  A protocol specifies **rules determining how information is to be exchanged**.
  (Simply connecting two devices is not sufficient for them to be able to
  communicate, they must have a shared language.)

  The nodes of a network are called [hosts](Network_hosts.md).

raw_content:
  ---
  tags: [networks, network-protocols]
  created: Saturday, August 03, 2024
  ---

  # Network fundamentals

  > A network is a system that allows computing devices to communicate and
  > exchange information with each other.

  In order for devices to be able to communicate they must share a common
  **communication protocol**.

  A protocol specifies **rules determining how information is to be exchanged**.
  (Simply connecting two devices is not sufficient for them to be able to
  communicate, they must have a shared language.)

  The nodes of a network are called [hosts](Network_hosts.md).


path: Network_hosts.md
sortable_path: Network_hosts.md
title: Network hosts
checksum: b4ac6a2c1a8d9cbdeb0ebd886484e545c5d398dfb3ef09c0c4aa8380b3d430f1
created: 2025-10-01 13:31:00.209628395+00:00
modified: 2025-08-13 15:57:13.921206283+00:00
metadata: {"created":"Saturday, August 03, 2024","tags":["networks"]}
lead:
  A network **host** or **node** is a single computing device attached to a
  ![network](Network_fundamentals.md)

body:
  A network **host** or **node** is a single computing device attached to a
  ![network](Network_fundamentals.md)

  Hosts can act as servers or clients, or both.

  ## Server

  A network server is a host that listens for inbound network connections and
  provides services to other hosts. Examples include web and email servers.

  ## Client

  A network client is a host that makes outbound connections and requests servcies
  from network servers.

raw_content:
  ---
  tags: [networks]
  created: Saturday, August 03, 2024
  ---

  # Network hosts

  A network **host** or **node** is a single computing device attached to a
  ![network](Network_fundamentals.md)

  Hosts can act as servers or clients, or both.

  ## Server

  A network server is a host that listens for inbound network connections and
  provides services to other hosts. Examples include web and email servers.

  ## Client

  A network client is a host that makes outbound connections and requests servcies
  from network servers.


path: Network_packet_analysis_tools.md
sortable_path: Network_packet_analysis_tools.md
title: Network packet analysis tools
checksum: 6bffbdc731d985354637c3e58cefa82c0fce3db0c5514e06c091274a960afb4c
created: 2025-10-01 13:31:00.214754783+00:00
modified: 2025-08-13 15:57:13.809156172+00:00
metadata: {"created":"Friday, December 13, 2024","tags":["networks","procedural"]}
lead:
  - `tcpdump`: View raw packet data (`tcpdump -i wlan0 -XX` for full packet
    contents)
  - tshark: Wireshark's CLI for detailed packet analysis (`tshark -i wlan0 -V`)
  - `nethogs`: Monitor per-application network usage
  - `iftop`: Show bandwidth usage per connection

body:
  - `tcpdump`: View raw packet data (`tcpdump -i wlan0 -XX` for full packet
    contents)
  - tshark: Wireshark's CLI for detailed packet analysis (`tshark -i wlan0 -V`)
  - `nethogs`: Monitor per-application network usage
  - `iftop`: Show bandwidth usage per connection

raw_content:
  ---
  tags: [networks, procedural]
  created: Friday, December 13, 2024
  ---

  # Network packet analysis tools

  - `tcpdump`: View raw packet data (`tcpdump -i wlan0 -XX` for full packet
    contents)
  - tshark: Wireshark's CLI for detailed packet analysis (`tshark -i wlan0 -V`)
  - `nethogs`: Monitor per-application network usage
  - `iftop`: Show bandwidth usage per connection


path: Network_protocols.md
sortable_path: Network_protocols.md
title: Network protocols
checksum: 25b22a05f48950de12dbb593e2bde94b3da329d5b20dec546c4b3624f40f3767
created: 2025-10-01 13:31:00.21888824+00:00
modified: 2025-08-13 15:57:13.893589581+00:00
metadata: {"created":"Saturday, August 03, 2024","tags":["networks"]}
lead:
  Protocols are agreed standards specifying and governing communication between
  nodes on a [network](Network_fundamentals.md).

body:
  Protocols are agreed standards specifying and governing communication between
  nodes on a [network](Network_fundamentals.md).

  Network protocols are typically _layered_. The implmentation of a network layer
  is a _stack_.

  Stacks enshrine encapsulation: not every layer has to be directly concerned with
  every other layer. Only adjacent layers that interface with each other need to
  be in contact.

  A prime example of a layered network stack is the
  [internet_protocol_suite](Internet_fundamentals.md).

raw_content:
  ---
  tags: [networks]
  created: Saturday, August 03, 2024
  ---

  # Network protocols

  Protocols are agreed standards specifying and governing communication between
  nodes on a [network](Network_fundamentals.md).

  Network protocols are typically _layered_. The implmentation of a network layer
  is a _stack_.

  Stacks enshrine encapsulation: not every layer has to be directly concerned with
  every other layer. Only adjacent layers that interface with each other need to
  be in contact.

  A prime example of a layered network stack is the
  [internet_protocol_suite](Internet_fundamentals.md).


path: Network_requests_in_Python.md
sortable_path: Network_requests_in_Python.md
title: Making network requests in Python
checksum: f3c9fa4c90fc5f9c3658fb0d8b0127b1faab85a129c65a5242078c2edc0dfb3a
created: 2025-10-01 13:31:00.231701868+00:00
modified: 2025-08-13 15:57:13.783013041+00:00
metadata: {"tags":["python","APIs","networks"]}
lead:
  We can use the `requests` package to make API requests to
  [RESTful](RESTful_APIs.md) resources and handle the data as JSON.

body:
  We can use the `requests` package to make API requests to
  [RESTful](RESTful_APIs.md) resources and handle the data as JSON.

  ```sh
  pip install requests
  ```

  Here is a basic architecture for making a `GET` request in Python.

  ```python
  import requests

  def get_data(url):
      response = requests.get(url)
      if response.status_code == 200:
          return response.json()
      else:
          raise Exception(
              f"Failed to fetch data from API. Status code: {response.status_code}")

  def main():
      url = "https://dummyjson.com/products/1"

      try:
          data = get_data(url)
          print(data)

      except Exception as e:
          prin(e)

  if __name__ == "__main__":
      main()
  ```

  ## Parsing returned data

  Running `main` returns:

  ```python
  {
     "id":1,
     "title":"iPhone 9",
     "description":"An apple mobile which is nothing like apple",
     "price":549,
     "discountPercentage":12.96,
     "rating":4.69,
     "stock":94,
     "brand":"Apple",
     "category":"smartphones",
     "thumbnail":"https://i.dummyjson.com/data/products/1/thumbnail.jpg",
     "images":[
        "https://i.dummyjson.com/data/products/1/1.jpg",
        "https://i.dummyjson.com/data/products/1/2.jpg",
        "https://i.dummyjson.com/data/products/1/3.jpg",
        "https://i.dummyjson.com/data/products/1/4.jpg",
        "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
     ]
  }
  ```

  This is JSON but in Python is a [dictionary](Dictionaries_in_Python.md)

  We can use standard dictionary methods to handle the data. For example, we'll
  add to the existing `try` block:

  ```python
  example_key = "brand"  # Replace with the key you want to access from the JSON data
      if example_key in data:
          print(f"Value of '{example_key}':", data[example_key])
      else:
          print(f"'{example_key}' not found in the JSON data")
  ```

  Which, if successful, outputs:

  ```
  Value of 'brand': Apple
  ```

raw_content:
  ---
  tags:
    - python
    - APIs
    - networks
  ---

  # Making network requests in Python

  We can use the `requests` package to make API requests to
  [RESTful](RESTful_APIs.md) resources and handle the data as JSON.

  ```sh
  pip install requests
  ```

  Here is a basic architecture for making a `GET` request in Python.

  ```python
  import requests

  def get_data(url):
      response = requests.get(url)
      if response.status_code == 200:
          return response.json()
      else:
          raise Exception(
              f"Failed to fetch data from API. Status code: {response.status_code}")

  def main():
      url = "https://dummyjson.com/products/1"

      try:
          data = get_data(url)
          print(data)

      except Exception as e:
          prin(e)

  if __name__ == "__main__":
      main()
  ```

  ## Parsing returned data

  Running `main` returns:

  ```python
  {
     "id":1,
     "title":"iPhone 9",
     "description":"An apple mobile which is nothing like apple",
     "price":549,
     "discountPercentage":12.96,
     "rating":4.69,
     "stock":94,
     "brand":"Apple",
     "category":"smartphones",
     "thumbnail":"https://i.dummyjson.com/data/products/1/thumbnail.jpg",
     "images":[
        "https://i.dummyjson.com/data/products/1/1.jpg",
        "https://i.dummyjson.com/data/products/1/2.jpg",
        "https://i.dummyjson.com/data/products/1/3.jpg",
        "https://i.dummyjson.com/data/products/1/4.jpg",
        "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
     ]
  }
  ```

  This is JSON but in Python is a [dictionary](Dictionaries_in_Python.md)

  We can use standard dictionary methods to handle the data. For example, we'll
  add to the existing `try` block:

  ```python
  example_key = "brand"  # Replace with the key you want to access from the JSON data
      if example_key in data:
          print(f"Value of '{example_key}':", data[example_key])
      else:
          print(f"'{example_key}' not found in the JSON data")
  ```

  Which, if successful, outputs:

  ```
  Value of 'brand': Apple
  ```


path: Network_scanning.md
sortable_path: Network_scanning.md
title: Network scanning
lead: ## List all connected devices on local WiFi network
checksum: a1117fe7c76323a1b2af2c2f9b47e0dc0000d75f045daa4530791b61c6e3fee4
created: 2025-10-01 13:31:00.247730529+00:00
modified: 2025-08-13 15:57:13.818646086+00:00
metadata: {"created":"Monday, June 10, 2024","id":"ippn","tags":["networks","procedural","Linux"]}
body:
  ## List all connected devices on local WiFi network

  ```sh
  arp-scan --interface=wlan0 --localnet
  ```

  ## Scan subnet for all connected devices (in case hiding)

  ```sh
  nmap -sn 192.168.0.0/24
  ```

  ## Check DNS resolution

  ```sh
  dig google.com
  ```

  Key output:

  ```
  ;; Query time: 6 msec
  ;; SERVER: 192.168.0.4#53(192.168.0.4) (UDP)
  ;; WHEN: Wed Jun 12 18:27:27 BST 2024
  ;; MSG SIZE  rcvd: 55
  ```

raw_content:
  ---
  id: ippn
  tags: [networks, procedural, Linux]
  created: Monday, June 10, 2024
  ---

  # Network scanning

  ## List all connected devices on local WiFi network

  ```sh
  arp-scan --interface=wlan0 --localnet
  ```

  ## Scan subnet for all connected devices (in case hiding)

  ```sh
  nmap -sn 192.168.0.0/24
  ```

  ## Check DNS resolution

  ```sh
  dig google.com
  ```

  Key output:

  ```
  ;; Query time: 6 msec
  ;; SERVER: 192.168.0.4#53(192.168.0.4) (UDP)
  ;; WHEN: Wed Jun 12 18:27:27 BST 2024
  ;; MSG SIZE  rcvd: 55
  ```


path: Network_tunnels.md
sortable_path: Network_tunnels.md
title: Network tunnels
checksum: 945df3a9470ef7167623f087f72fcd6c9a5ff8b17af60a89eafb3ff6211018bd
created: 2025-10-01 13:31:00.253466076+00:00
modified: 2025-08-13 15:57:13.973518436+00:00
metadata: {"created":"Monday, February 24, 2025","tags":["networks","VPN"]}
lead:
  A network tunnel is a
  [Transport Layer](./Transport_Layer_of_Internet_Protocol.md) mechanism whereby
  data packets can travel across a network they would otherwise not be able to
  access. Similarly how an underground physical tunnel can be use to travel across
  an inaccessible geographic region.

body:
  A network tunnel is a
  [Transport Layer](./Transport_Layer_of_Internet_Protocol.md) mechanism whereby
  data packets can travel across a network they would otherwise not be able to
  access. Similarly how an underground physical tunnel can be use to travel across
  an inaccessible geographic region.

  It is a broad concept of which there are various specific instances. However
  many involve creating a tunnel by wrapping a packet that cannot traverse a given
  network within the payload of a packet that can.

  For instance, an organisation may use [IPv6](./IP_addresses.md) addresses but
  want to send them over an IPv4 network. To achieve this they would enclose the
  packets within IPv4 packets.

  Similarly the scenario of a VPN. A user with a UK IP address wants to access a
  server in the US that has a firewall blocking non-US IPs. The user can bypass
  this restriction by connecting to a VPN running US servers. The UK packets are
  transferred under the aegis of the VPN's US IP.

  For more detail see
  [How tunnelling works with VPNs](./How_tunneling_works_with_VPNs.md).

raw_content:
  ---
  tags: [networks, VPN]
  created: Monday, February 24, 2025
  ---

  # Network tunnels

  A network tunnel is a
  [Transport Layer](./Transport_Layer_of_Internet_Protocol.md) mechanism whereby
  data packets can travel across a network they would otherwise not be able to
  access. Similarly how an underground physical tunnel can be use to travel across
  an inaccessible geographic region.

  It is a broad concept of which there are various specific instances. However
  many involve creating a tunnel by wrapping a packet that cannot traverse a given
  network within the payload of a packet that can.

  For instance, an organisation may use [IPv6](./IP_addresses.md) addresses but
  want to send them over an IPv4 network. To achieve this they would enclose the
  packets within IPv4 packets.

  Similarly the scenario of a VPN. A user with a UK IP address wants to access a
  server in the US that has a firewall blocking non-US IPs. The user can bypass
  this restriction by connecting to a VPN running US servers. The UK packets are
  transferred under the aegis of the VPN's US IP.

  For more detail see
  [How tunnelling works with VPNs](./How_tunneling_works_with_VPNs.md).


path: Nginx_vs_traefik.md
sortable_path: Nginx_vs_traefik.md
title: e383b8b3_nginx_vs_traefik
checksum: 92f468f8a952b0f9c395c853d6659984d6cb450b94215e09386f20eb510313b9
created: 2025-10-01 13:31:00.277010159+00:00
modified: 2025-08-13 15:57:13.904780395+00:00
metadata: {"created":"Monday, February 03, 2025","tags":["networks","proxies","nginx"]}
lead:
  Both nginx and traefik are server technologies (along with Apache), nginx has a
  longer pedigree but traefik is calibrated for more modern microservice-based
  architectures (i.e Docker containers). nginx is better suited to traditional
  static web-based hosting.

body:
  Both nginx and traefik are server technologies (along with Apache), nginx has a
  longer pedigree but traefik is calibrated for more modern microservice-based
  architectures (i.e Docker containers). nginx is better suited to traditional
  static web-based hosting.

  traefik has support for [HTTPS](./HTTPS.md) natively whereas nginx requires a
  plugin.

raw_content:
  ---
  tags: [networks, proxies, nginx]
  created: Monday, February 03, 2025
  ---

  # e383b8b3_nginx_vs_traefik

  Both nginx and traefik are server technologies (along with Apache), nginx has a
  longer pedigree but traefik is calibrated for more modern microservice-based
  architectures (i.e Docker containers). nginx is better suited to traditional
  static web-based hosting.

  traefik has support for [HTTPS](./HTTPS.md) natively whereas nginx requires a
  plugin.


path: NodeJS_scripts.md
sortable_path: NodeJS_scripts.md
title: NodeJS scripts
checksum: f65f2dc24fc46bb1d9ca1d4266aa146052074c8d179694d74c3832d5da363e0d
created: 2025-10-01 13:31:00.284841499+00:00
modified: 2025-08-13 15:57:14.038839619+00:00
metadata: {"created":"Friday, April 26, 2024","tags":["node-js"]}
lead:
  If we want to use NodeJS as a scripting language without going to the trouble of
  setting up an NPM-based development environment, we can use the following
  architecture.

body:
  If we want to use NodeJS as a scripting language without going to the trouble of
  setting up an NPM-based development environment, we can use the following
  architecture.

  ```js
  #!/usr/bin/env node

  const process = require("process");

  const actualScript = (firstParam, secondParam) => {
    // Do something
  };

  if (require.main === module) {
    const [, param1, param2] = process.argv;
    actualScript(param1, param2).catch(console.error);
  }
  ```

  Basically we have a function that contains the main actions of the script. This
  is then invoked within `main` in the manner of a Bash or
  ![Python script](./Single_file_Python_scripts.md)

  This obviously requires the Node binary to be in your path and the script must
  be run with executable privileges.

  If you use any module not in the Node core, you will need to have this installed
  globally for it to work. (There are
  [ways around this](https://thom4.net/2014/self-contained-node-scripts/) for Node
  scripts you publish but for personal scripts this doesn't matter.)

  ## `require.main`

  This construction checks that the script is being directy invoked via the 'node'
  command in the terminal rather than called by another module.

  This is equivalent to the [following construct]() in Python:

  ```py
  if __name__ == "__main__":
  ```

raw_content:
  ---
  tags: [node-js]
  created: Friday, April 26, 2024
  ---

  # NodeJS scripts

  If we want to use NodeJS as a scripting language without going to the trouble of
  setting up an NPM-based development environment, we can use the following
  architecture.

  ```js
  #!/usr/bin/env node

  const process = require("process");

  const actualScript = (firstParam, secondParam) => {
    // Do something
  };

  if (require.main === module) {
    const [, param1, param2] = process.argv;
    actualScript(param1, param2).catch(console.error);
  }
  ```

  Basically we have a function that contains the main actions of the script. This
  is then invoked within `main` in the manner of a Bash or
  ![Python script](./Single_file_Python_scripts.md)

  This obviously requires the Node binary to be in your path and the script must
  be run with executable privileges.

  If you use any module not in the Node core, you will need to have this installed
  globally for it to work. (There are
  [ways around this](https://thom4.net/2014/self-contained-node-scripts/) for Node
  scripts you publish but for personal scripts this doesn't matter.)

  ## `require.main`

  This construction checks that the script is being directy invoked via the 'node'
  command in the terminal rather than called by another module.

  This is equivalent to the [following construct]() in Python:

  ```py
  if __name__ == "__main__":
  ```


path: Node_and_MySQL_db.md
sortable_path: Node_and_MySQL_db.md
title: Docker example: NodeJS backend with MySQL database
lead:
  We will utilise [Docker Compose](Docker_Compose.md) to combine
  two containers:

checksum: da18db716656813a6f25424eb45c5c29a3af3b45d1c3d300c401e507e2154925
created: 2025-10-01 13:31:00.290807101+00:00
modified: 2025-08-13 15:57:14.042287524+00:00
metadata: {"tags":["docker","SQL","node-js","databases"]}
body:
  We will utilise [Docker Compose](Docker_Compose.md) to combine
  two containers:

  - A container for the NodeJS backend
  - A container for the MySQL database

  We will only create a Dockerfile for the NodeJS part since the existing `mysql`
  image is sufficient for most needs and does not require a specific
  configuration.

  Each of the files listed below would be saved to the same source directory which
  would then form the basis of the
  [build context](Creating_a_Docker_image.md#creating-a-docker-image).

  ## Docker Compose file

  ```yml
  # docker-compose.yml
  version: "3.8"
  services:
    db:
      image: mysql:8.0
      container_name: mysql_container
      environment:
        MYSQL_ROOT_PASSWORD: your_root_password
        MYSQL_DATABASE: your_database_name
        MYSQL_USER: your_database_user
        MYSQL_PASSWORD: your_database_password
      volumes:
        - mysql-data:/var/lib/mysql
      ports:
        - "3306:3306"
    app:
      build: .
      container_name: node_app
      volumes:
        - .:/usr/src/app
      environment:
        MYSQL_HOST: db
        MYSQL_USER: your_database_user
        MYSQL_PASSWORD: your_database_password
        MYSQL_DB: your_database_name
      depends_on:
        - db
      ports:
        - "3000:3000"

  volumes:
    mysql-data:
  ```

  ## Dockerfile for the NodeJS backend

  ```Dockerfile
  # Dockerfile
  FROM node:14

  WORKDIR /usr/src/app

  COPY package*.json ./

  RUN npm install

  COPY . .

  EXPOSE 3000

  CMD [ "node", "app.js" ]
  ```

  ## NodeJS project setup

  ```json
  // package.json
  {
    "name": "node-mysql-docker",
    "version": "1.0.0",
    "description": "Node.js and MySQL with Docker",
    "main": "app.js",
    "scripts": {
      "start": "node app.js"
    },
    "dependencies": {
      "express": "^4.17.1",
      "mysql2": "^2.3.2"
    }
  }
  ```

  ```js
  // app.js
  const express = require("express");
  const mysql = require("mysql2/promise");

  const app = express();

  const { MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB } = process.env;

  const createConnection = async () => {
    return await mysql.createConnection({
      host: MYSQL_HOST,
      user: MYSQL_USER,
      password: MYSQL_PASSWORD,
      database: MYSQL_DB,
    });
  };

  app.get("/", async (req, res) => {
    const connection = await createConnection();
    const [rows] = await connection.query("SELECT 1 + 1 AS solution");
    res.send(`Hello World! The solution is ${rows[0].solution}`);
  });

  const PORT = 3000;
  app.listen(PORT, () => {
    console.log(`Server is running on port ${PORT}`);
  });
  ```

  To start up the environment you would then run:

  ```
  docker-compose -up
  ```

  ## Environments

  In the example, the database connection information in the Node source is coming
  from the
  [`process.env`](Managing_environments_in_NodeJS.md)
  object, which itself is sourcing the values `MYSQL_HOST`, `MYSQL_PASSWORD` etc
  from the Docker compose file. Therefore these values are hardcoded there.

  This is not good practice as it exposes sensitive information and make managing
  different deployment environments (development, stage, test etc.) difficult.

  To get around this we would create an `.env` file in the project directory that
  is Git ignored:

  ```sh
  # .env
  MYSQL_ROOT_PASSWORD=your_root_password
  MYSQL_DATABASE=your_database_name
  MYSQL_USER=your_database_user
  MYSQL_PASSWORD=your_database_password
  ```

  Then the `docker-compose.yml` file can be updated to use these variables:

  ```yml
  version: "3.8"
  services:
    db:
      image: mysql:8.0
      container_name: mysql_container
      environment:
        MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
        MYSQL_DATABASE: ${MYSQL_DATABASE}
        MYSQL_USER: ${MYSQL_USER}
        MYSQL_PASSWORD: ${MYSQL_PASSWORD}
      volumes:
        - mysql-data:/var/lib/mysql
      ports:
        - "3306:3306"
    app:
      build: .
      container_name: node_app
      volumes:
        - .:/usr/src/app
      environment:
        MYSQL_HOST: db
        MYSQL_USER: ${MYSQL_USER}
        MYSQL_PASSWORD: ${MYSQL_PASSWORD}
        MYSQL_DB: ${MYSQL_DATABASE}
      depends_on:
        - db
      ports:
        - "3000:3000"

  volumes:
    mysql-data:
  ```

  `${VARIABLE_NAME}` syntax is used to reference environment variables from the
  .env file in the `docker-compose.yml` file. Docker Compose will automatically
  load the variables from the .env file when starting the services.

  ### Development, staging, production environments

  To specify different connection details for different environments you would
  create different `.env` files for each:

  - `.env.development`
  - `.env.staging`
  - `.env.production`

  Each file will contain **environment-specific variables**, such as database
  credentials, API keys, and other configuration details.

  For example, development and production:

  ```yml
  # docker-compose.development.yml
  version: '3.8'
  services:
    db:
      ...
    app:
      ...
      environment:
        MYSQL_HOST: db
        MYSQL_USER: ${MYSQL_USER}
        MYSQL_PASSWORD: ${MYSQL_PASSWORD}
        MYSQL_DB: ${MYSQL_DATABASE}
        NODE_ENV: development

  ```

  ```yml
  # docker-compose.production.yml
  version: '3.8'
  services:
    db:
      ...
    app:
      ...
      environment:
        MYSQL_HOST: db
        MYSQL_USER: ${MYSQL_USER}
        MYSQL_PASSWORD: ${MYSQL_PASSWORD}
        MYSQL_DB: ${MYSQL_DATABASE}
        NODE_ENV: production
  `
  ```

  Then you would select the specific environment with your run command:

  ```
  docker-compose -f docker-compose.development.yml up -d
  ```

  Docker won't know by default which `.env` file to use from that command however.
  Assuming all the files are in the same directory you can use Bash substitution
  to specify the source of the environment specific variables:

  ```sh
  export $(cat .env.development | xargs) && docker-compose -f docker-compose.development.yml up -d
  ```

raw_content:
  ---
  tags:
    - docker
    - SQL
    - node-js
    - databases
  ---

  # Docker example: NodeJS backend with MySQL database

  We will utilise [Docker Compose](Docker_Compose.md) to combine
  two containers:

  - A container for the NodeJS backend
  - A container for the MySQL database

  We will only create a Dockerfile for the NodeJS part since the existing `mysql`
  image is sufficient for most needs and does not require a specific
  configuration.

  Each of the files listed below would be saved to the same source directory which
  would then form the basis of the
  [build context](Creating_a_Docker_image.md#creating-a-docker-image).

  ## Docker Compose file

  ```yml
  # docker-compose.yml
  version: "3.8"
  services:
    db:
      image: mysql:8.0
      container_name: mysql_container
      environment:
        MYSQL_ROOT_PASSWORD: your_root_password
        MYSQL_DATABASE: your_database_name
        MYSQL_USER: your_database_user
        MYSQL_PASSWORD: your_database_password
      volumes:
        - mysql-data:/var/lib/mysql
      ports:
        - "3306:3306"
    app:
      build: .
      container_name: node_app
      volumes:
        - .:/usr/src/app
      environment:
        MYSQL_HOST: db
        MYSQL_USER: your_database_user
        MYSQL_PASSWORD: your_database_password
        MYSQL_DB: your_database_name
      depends_on:
        - db
      ports:
        - "3000:3000"

  volumes:
    mysql-data:
  ```

  ## Dockerfile for the NodeJS backend

  ```Dockerfile
  # Dockerfile
  FROM node:14

  WORKDIR /usr/src/app

  COPY package*.json ./

  RUN npm install

  COPY . .

  EXPOSE 3000

  CMD [ "node", "app.js" ]
  ```

  ## NodeJS project setup

  ```json
  // package.json
  {
    "name": "node-mysql-docker",
    "version": "1.0.0",
    "description": "Node.js and MySQL with Docker",
    "main": "app.js",
    "scripts": {
      "start": "node app.js"
    },
    "dependencies": {
      "express": "^4.17.1",
      "mysql2": "^2.3.2"
    }
  }
  ```

  ```js
  // app.js
  const express = require("express");
  const mysql = require("mysql2/promise");

  const app = express();

  const { MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB } = process.env;

  const createConnection = async () => {
    return await mysql.createConnection({
      host: MYSQL_HOST,
      user: MYSQL_USER,
      password: MYSQL_PASSWORD,
      database: MYSQL_DB,
    });
  };

  app.get("/", async (req, res) => {
    const connection = await createConnection();
    const [rows] = await connection.query("SELECT 1 + 1 AS solution");
    res.send(`Hello World! The solution is ${rows[0].solution}`);
  });

  const PORT = 3000;
  app.listen(PORT, () => {
    console.log(`Server is running on port ${PORT}`);
  });
  ```

  To start up the environment you would then run:

  ```
  docker-compose -up
  ```

  ## Environments

  In the example, the database connection information in the Node source is coming
  from the
  [`process.env`](Managing_environments_in_NodeJS.md)
  object, which itself is sourcing the values `MYSQL_HOST`, `MYSQL_PASSWORD` etc
  from the Docker compose file. Therefore these values are hardcoded there.

  This is not good practice as it exposes sensitive information and make managing
  different deployment environments (development, stage, test etc.) difficult.

  To get around this we would create an `.env` file in the project directory that
  is Git ignored:

  ```sh
  # .env
  MYSQL_ROOT_PASSWORD=your_root_password
  MYSQL_DATABASE=your_database_name
  MYSQL_USER=your_database_user
  MYSQL_PASSWORD=your_database_password
  ```

  Then the `docker-compose.yml` file can be updated to use these variables:

  ```yml
  version: "3.8"
  services:
    db:
      image: mysql:8.0
      container_name: mysql_container
      environment:
        MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
        MYSQL_DATABASE: ${MYSQL_DATABASE}
        MYSQL_USER: ${MYSQL_USER}
        MYSQL_PASSWORD: ${MYSQL_PASSWORD}
      volumes:
        - mysql-data:/var/lib/mysql
      ports:
        - "3306:3306"
    app:
      build: .
      container_name: node_app
      volumes:
        - .:/usr/src/app
      environment:
        MYSQL_HOST: db
        MYSQL_USER: ${MYSQL_USER}
        MYSQL_PASSWORD: ${MYSQL_PASSWORD}
        MYSQL_DB: ${MYSQL_DATABASE}
      depends_on:
        - db
      ports:
        - "3000:3000"

  volumes:
    mysql-data:
  ```

  `${VARIABLE_NAME}` syntax is used to reference environment variables from the
  .env file in the `docker-compose.yml` file. Docker Compose will automatically
  load the variables from the .env file when starting the services.

  ### Development, staging, production environments

  To specify different connection details for different environments you would
  create different `.env` files for each:

  - `.env.development`
  - `.env.staging`
  - `.env.production`

  Each file will contain **environment-specific variables**, such as database
  credentials, API keys, and other configuration details.

  For example, development and production:

  ```yml
  # docker-compose.development.yml
  version: '3.8'
  services:
    db:
      ...
    app:
      ...
      environment:
        MYSQL_HOST: db
        MYSQL_USER: ${MYSQL_USER}
        MYSQL_PASSWORD: ${MYSQL_PASSWORD}
        MYSQL_DB: ${MYSQL_DATABASE}
        NODE_ENV: development

  ```

  ```yml
  # docker-compose.production.yml
  version: '3.8'
  services:
    db:
      ...
    app:
      ...
      environment:
        MYSQL_HOST: db
        MYSQL_USER: ${MYSQL_USER}
        MYSQL_PASSWORD: ${MYSQL_PASSWORD}
        MYSQL_DB: ${MYSQL_DATABASE}
        NODE_ENV: production
  `
  ```

  Then you would select the specific environment with your run command:

  ```
  docker-compose -f docker-compose.development.yml up -d
  ```

  Docker won't know by default which `.env` file to use from that command however.
  Assuming all the files are in the same directory you can use Bash substitution
  to specify the source of the environment specific variables:

  ```sh
  export $(cat .env.development | xargs) && docker-compose -f docker-compose.development.yml up -d
  ```


path: Nodemon.md
sortable_path: Nodemon.md
title: Nodemon
checksum: 223c81e0636c8f7c17244cad0b4e2dc871021f2954688aa7f70ee67f8a2d6bfb
created: 2025-10-01 13:31:00.309899923+00:00
modified: 2025-08-13 15:57:14.028016364+00:00
metadata: {"tags":["node-js"]}
lead:
  We don't want to have to restart the local server every time we make a change to
  our files. We can use `nodemon` instead of `node` when running our `index.js`
  file so that file-changes are immediately registered without the need for a
  restart.

body:
  We don't want to have to restart the local server every time we make a change to
  our files. We can use `nodemon` instead of `node` when running our `index.js`
  file so that file-changes are immediately registered without the need for a
  restart.

  This is a wrapper around the `node` command so it doesn't require any
  configuration. Once installed, update your start script from `node index.js` to
  `nodemon index.js`.

raw_content:
  ---
  tags:
    - node-js
  ---

  # Nodemon

  We don't want to have to restart the local server every time we make a change to
  our files. We can use `nodemon` instead of `node` when running our `index.js`
  file so that file-changes are immediately registered without the need for a
  restart.

  This is a wrapper around the `node` command so it doesn't require any
  configuration. Once installed, update your start script from `node index.js` to
  `nodemon index.js`.


path: Non_null_assertion_in_TypeScript.md
sortable_path: Non_null_assertion_in_TypeScript.md
title: Non-null assertion
lead: Oftentimes you will encounter errors of the form:
checksum: 3afb6d654d3d9d9c24f9155a366976af6b70dc7f501848f73c2cb17e7c8e6455
created: 2025-10-01 13:31:00.315953804+00:00
modified: 2025-08-13 15:57:13.990867264+00:00
metadata: {"tags":["typescript"]}
body:
  Oftentimes you will encounter errors of the form:

  `Property [property_name] does not exist on type [type_name].ts(2339)`

  This means the interpreter is not able to determine whether the property which
  you are seeking to change exists. It can't be sure that you are not trying to
  update a property that is either `null` or `undefined`. This is good type
  checking but it means that in cases when _you know_ the property either exists
  or _will_ exist at runtime, you are going to face a code-breaking error.

  To get around this you can use non-null assertion. Like the name suggests, here
  you are saying to TypeScript: thanks but I am sure this property exists. When
  you do this you get the value you are trying to change with `null` and
  `undefined` excluded from it.

  This typically occurs when you are assigning a property to a value and TS
  doesn't know whether the value will exist or be accessible at this point in the
  runtime. A good scenario would be dynamic data passed into a function:

  ```ts
  function doSomethingWith(typeAlias: ITypeAlias) {
    let x = typeAlias!.somePropertyEvidentAtRuntime;
  }
  ```

  However it should be used carefully and sparingly because you are obviously
  turning off core type-checking and overuse nullifies the purpose of TypeScript.

  One way to get around it is to use better
  [type-guarding](Type_guarding_and_narrowing.md) and conditionality and to
  cover cases where the value may be undefined.

raw_content:
  ---
  tags:
    - typescript
  ---

  # Non-null assertion

  Oftentimes you will encounter errors of the form:

  `Property [property_name] does not exist on type [type_name].ts(2339)`

  This means the interpreter is not able to determine whether the property which
  you are seeking to change exists. It can't be sure that you are not trying to
  update a property that is either `null` or `undefined`. This is good type
  checking but it means that in cases when _you know_ the property either exists
  or _will_ exist at runtime, you are going to face a code-breaking error.

  To get around this you can use non-null assertion. Like the name suggests, here
  you are saying to TypeScript: thanks but I am sure this property exists. When
  you do this you get the value you are trying to change with `null` and
  `undefined` excluded from it.

  This typically occurs when you are assigning a property to a value and TS
  doesn't know whether the value will exist or be accessible at this point in the
  runtime. A good scenario would be dynamic data passed into a function:

  ```ts
  function doSomethingWith(typeAlias: ITypeAlias) {
    let x = typeAlias!.somePropertyEvidentAtRuntime;
  }
  ```

  However it should be used carefully and sparingly because you are obviously
  turning off core type-checking and overuse nullifies the purpose of TypeScript.

  One way to get around it is to use better
  [type-guarding](Type_guarding_and_narrowing.md) and conditionality and to
  cover cases where the value may be undefined.


path: None_in_Python.md
sortable_path: None_in_Python.md
title: None in Python
checksum: 185fdbfe796eb95a3ec84eab881a101129a444faade8c2bc76b831840d9c8a1b
created: 2025-10-01 13:31:00.329037163+00:00
modified: 2025-08-13 15:57:14.087836987+00:00
metadata: {"tags":["python","data-types"]}
lead:
  `None` is not `null`, it is closer to `undefined` in JS. If you define a
  variable as `None`, the variable exists, it is just not yet defined.

body:
  `None` is not `null`, it is closer to `undefined` in JS. If you define a
  variable as `None`, the variable exists, it is just not yet defined.

  Using `None` is a pattern similar to using `let` in JS to name a variable and
  definine it later on.

  ```python
  temperature = None
  ```

  If we logged `temperature` it would give us `None` rather than a null pointer
  error.

  With None we can use `is None` and `is not None`, special predicates for working
  with `None` only. This is a akin to using `if (x !== undefined)` in TypeScript

  ```python
  winner = None
  print('winner:', winner)
  # winner: None
  print('winner is None:', winner is None)
  # winner is None: True
  print('winner is not None:', winner is not None)
  # winner is not None: False
  print(type(winner))
  # <class 'NoneType'>
  # Now set winner to be True
  print('Set winner to True')
  # Set winner to True
  winner = True
  print('winner:', winner)
  # winner: True
  print('winner is None:', winner is None)
  # winner is None: False
  print('winner is not None:', winner is not None)
  # winner is not None: True
  print(type(winner))
  # <class 'bool'>
  ```

raw_content:
  ---
  tags: [python, data-types]
  ---

  # None in Python

  `None` is not `null`, it is closer to `undefined` in JS. If you define a
  variable as `None`, the variable exists, it is just not yet defined.

  Using `None` is a pattern similar to using `let` in JS to name a variable and
  definine it later on.

  ```python
  temperature = None
  ```

  If we logged `temperature` it would give us `None` rather than a null pointer
  error.

  With None we can use `is None` and `is not None`, special predicates for working
  with `None` only. This is a akin to using `if (x !== undefined)` in TypeScript

  ```python
  winner = None
  print('winner:', winner)
  # winner: None
  print('winner is None:', winner is None)
  # winner is None: True
  print('winner is not None:', winner is not None)
  # winner is not None: False
  print(type(winner))
  # <class 'NoneType'>
  # Now set winner to be True
  print('Set winner to True')
  # Set winner to True
  winner = True
  print('winner:', winner)
  # winner: True
  print('winner is None:', winner is None)
  # winner is None: False
  print('winner is not None:', winner is not None)
  # winner is not None: True
  print(type(winner))
  # <class 'bool'>
  ```


path: Numbers_in_Python.md
sortable_path: Numbers_in_Python.md
title: Numbers in Python
lead: ## Distinguishing `int` and `float`
checksum: 7686d82f635d53fb6d4a45f128d97422a6636463c04c96174ff31bbcbd339c69
created: 2025-10-01 13:31:00.334847862+00:00
modified: 2025-08-13 15:57:13.877766336+00:00
metadata: {"tags":["python","data-types"]}
body:
  ## Distinguishing `int` and `float`

  - In Python we have floats and integers and we can coerce one into the other
  - A `//` as an operator means float division. This obviously provides greater
    precision than int division `/`.
  - There is no increment (`++`) or decrement (`--`) operator in Python

  ```python
  # Integers and floats
  count = 1
  print(count)
  # 1
  print(type(count))
  # <class 'int'>

  exchange_rate = 1.83
  print(exchange_rate)
  # 1.83
  print(type(exchange_rate))
  # <class 'float'>

  print(float(count))
  # 1.0

  print(int(exchange_rate))
  # 1
  ```

raw_content:
  ---
  tags: [python, data-types]
  ---

  # Numbers in Python

  ## Distinguishing `int` and `float`

  - In Python we have floats and integers and we can coerce one into the other
  - A `//` as an operator means float division. This obviously provides greater
    precision than int division `/`.
  - There is no increment (`++`) or decrement (`--`) operator in Python

  ```python
  # Integers and floats
  count = 1
  print(count)
  # 1
  print(type(count))
  # <class 'int'>

  exchange_rate = 1.83
  print(exchange_rate)
  # 1.83
  print(type(exchange_rate))
  # <class 'float'>

  print(float(count))
  # 1.0

  print(int(exchange_rate))
  # 1
  ```


path: Object_language_and_meta_language.md
sortable_path: Object_language_and_meta_language.md
title: Object and metalanguages
checksum: 1be63bc78982eb0666e64209842912123a849dc70adfa066275c571ce3350c22
created: 2025-10-01 13:31:00.340208443+00:00
modified: 2025-08-13 15:57:13.870710944+00:00
metadata: {"tags":["propositional-logic","logic"]}
lead:
  When we talk about a language we call that language the **object language**. A
  **metalanguage** is a language used to describe some object language.

body:
  When we talk about a language we call that language the **object language**. A
  **metalanguage** is a language used to describe some object language.

  When we are developing a formal logical language (which we may call SL or PL for
  'sentential' and 'propositional' logic respectively), the formal language is the
  object language and natural language (e.g. English) is the metalanguage.

  **Demonstration**

  If we talk about German in English, German in the object language and English is
  the metalanguage.

  ## Use and mention

  There is an associated distinction: that of use and mention.

  When we create an expression in a language we are said to _use_ that language.
  When we remark upon said expression we are said to be _mentioning_ the language.
  This distinction may correspond to the object and metalanguage difference above
  but doesn't have to; use and mention can happen in the same language. For
  example:

  ```
  'London' is the word that denotes the capital of the UK.
  ```

  ## Metavariables

  A metalinguistic variable (metavariable for short) is an expression in the
  metalanguage that is used to talk generally about expressions of the object
  language. The convention in these notes will be to embolden single letters when
  these letters are used as metavariables.

  For example, instead of saying _'P & Q' is an expression comprising two atomic
  sentences and a conjunction_ we might say **\*P** is an expression comprising
  two atomic sentences and a conjunction. In this instance **P** is a metavariable
  in the metalanguage mentioning the expression P & Q in the object language

raw_content:
  ---
  tags:
    - propositional-logic
    - logic
  ---

  # Object and metalanguages

  When we talk about a language we call that language the **object language**. A
  **metalanguage** is a language used to describe some object language.

  When we are developing a formal logical language (which we may call SL or PL for
  'sentential' and 'propositional' logic respectively), the formal language is the
  object language and natural language (e.g. English) is the metalanguage.

  **Demonstration**

  If we talk about German in English, German in the object language and English is
  the metalanguage.

  ## Use and mention

  There is an associated distinction: that of use and mention.

  When we create an expression in a language we are said to _use_ that language.
  When we remark upon said expression we are said to be _mentioning_ the language.
  This distinction may correspond to the object and metalanguage difference above
  but doesn't have to; use and mention can happen in the same language. For
  example:

  ```
  'London' is the word that denotes the capital of the UK.
  ```

  ## Metavariables

  A metalinguistic variable (metavariable for short) is an expression in the
  metalanguage that is used to talk generally about expressions of the object
  language. The convention in these notes will be to embolden single letters when
  these letters are used as metavariables.

  For example, instead of saying _'P & Q' is an expression comprising two atomic
  sentences and a conjunction_ we might say **\*P** is an expression comprising
  two atomic sentences and a conjunction. In this instance **P** is a metavariable
  in the metalanguage mentioning the expression P & Q in the object language


path: Observer_pattern.md
sortable_path: Observer_pattern.md
title: Observer pattern
lead: The Observer design pattern is also known as the 'publish/subscribe' pattern.
checksum: 30dcbcaeaa06659911bb025762903a012710c53a980c163c8a78719369222dba
created: 2025-10-01 13:31:00.344499816+00:00
modified: 2025-08-13 15:57:13.742348799+00:00
metadata: {"tags":["OOP","design-patterns"]}
body:
  The Observer design pattern is also known as the 'publish/subscribe' pattern.

  > The observer pattern is a software design pattern in which an object, called
  > the subject, maintains a list of its dependents, called observers, and
  > notifies them automatically of any state changes, usually by calling one of
  > their methods.

  The observer pattern instantiates a **one-to-many** relationship between the
  subject and its observers: when one object updates, it notifies many other
  objects that it has been updated.

  This is another design pattern that is useful when implementing a global state.
  When the subject updates its subjects must follow suit and adapt to its changes.
  Here the subject is the instrument for maintaining the global state.

  ## Subject class

  This class maintains a list of observers that will be notified when it is
  updated. It should have the following attributes:

  - array of existing observers
  - an `addObserver` method which pushes an observer onto the observers array
  - a `removeObserver` method that does the opposite
  - a `notify` method that broadcasts to the observers that a change has happened

  ## Observer class

  Observers must implement an `update` method that will be called by the subject's
  `notify` method.

  ## Example

  ```js
  class Subject {
    observers = [];

    getCurrentTime() {
      var d = new Date();
      return `${d.getHours()}:${d.getMinutes()}:${d.getSeconds()}`;
    }

    addObserver(...observer) {
      for (const o of observer) {
        this.observers.push(o);
      }
    }
    removeObserver(observer) {
      const match = this.observers.findIndex((element) => element === observer);
      this.observers.splice(match, 1);
    }
    notify() {
      for (const o of this.observers) {
        o.update(`I was updated at ${this.getCurrentTime()}`);
      }
    }
  }

  class ObserverA {
    state = "I am observer A";
    update(newState) {
      this.state = newState;
    }
  }

  class ObserverB {
    state = "I am observer B";

    update(newState) {
      this.state = newState;
    }
  }

  let subjectInstance = new Subject();
  let obsAInstance = new ObserverA();
  let obsBInstance = new ObserverB();

  subjectInstance.addObserver(obsAInstance, obsBInstance);

  subjectInstance.notify();
  console.log(obsBInstance.state);
  ```

raw_content:
  ---
  tags:
    - OOP
    - design-patterns
  ---

  # Observer pattern

  The Observer design pattern is also known as the 'publish/subscribe' pattern.

  > The observer pattern is a software design pattern in which an object, called
  > the subject, maintains a list of its dependents, called observers, and
  > notifies them automatically of any state changes, usually by calling one of
  > their methods.

  The observer pattern instantiates a **one-to-many** relationship between the
  subject and its observers: when one object updates, it notifies many other
  objects that it has been updated.

  This is another design pattern that is useful when implementing a global state.
  When the subject updates its subjects must follow suit and adapt to its changes.
  Here the subject is the instrument for maintaining the global state.

  ## Subject class

  This class maintains a list of observers that will be notified when it is
  updated. It should have the following attributes:

  - array of existing observers
  - an `addObserver` method which pushes an observer onto the observers array
  - a `removeObserver` method that does the opposite
  - a `notify` method that broadcasts to the observers that a change has happened

  ## Observer class

  Observers must implement an `update` method that will be called by the subject's
  `notify` method.

  ## Example

  ```js
  class Subject {
    observers = [];

    getCurrentTime() {
      var d = new Date();
      return `${d.getHours()}:${d.getMinutes()}:${d.getSeconds()}`;
    }

    addObserver(...observer) {
      for (const o of observer) {
        this.observers.push(o);
      }
    }
    removeObserver(observer) {
      const match = this.observers.findIndex((element) => element === observer);
      this.observers.splice(match, 1);
    }
    notify() {
      for (const o of this.observers) {
        o.update(`I was updated at ${this.getCurrentTime()}`);
      }
    }
  }

  class ObserverA {
    state = "I am observer A";
    update(newState) {
      this.state = newState;
    }
  }

  class ObserverB {
    state = "I am observer B";

    update(newState) {
      this.state = newState;
    }
  }

  let subjectInstance = new Subject();
  let obsAInstance = new ObserverA();
  let obsBInstance = new ObserverB();

  subjectInstance.addObserver(obsAInstance, obsBInstance);

  subjectInstance.notify();
  console.log(obsBInstance.state);
  ```


path: Ohms_Law.md
sortable_path: Ohms_Law.md
title: Ohm's Law
checksum: d8bb6ce94a3a050e6f2494369e3a8fb2ae635a558fcf249321dd62a480ad4c31
created: 2025-10-01 13:31:00.350086591+00:00
modified: 2025-08-13 15:57:13.950408943+00:00
metadata: {"tags":["physics","electricity"]}
lead:
  The relationship between
  [current](Current.md),
  [voltage](Voltage.md), and
  [resistance](Resistance.md) is
  defined by Ohm's Law:

body:
  The relationship between
  [current](Current.md),
  [voltage](Voltage.md), and
  [resistance](Resistance.md) is
  defined by Ohm's Law:

  > The current flowing from one point to another is equal to the voltage accross
  > these points divided by the resistance between them.

  $$
  I = \frac{V}{R}
  $$

  As an application, consider a circuit with a 9V battery with a 10,000 $\Omega$
  resistor attached across its terminals.

  The current flowing through the resistor can be calculated as follows:

  $$
  \frac{9V} {10,000\Omega} = 0.0009 \textsf{A} (0.9 \textsf{mA})
  $$

raw_content:
  ---
  tags: [physics, electricity]
  ---

  # Ohm's Law

  The relationship between
  [current](Current.md),
  [voltage](Voltage.md), and
  [resistance](Resistance.md) is
  defined by Ohm's Law:

  > The current flowing from one point to another is equal to the voltage accross
  > these points divided by the resistance between them.

  $$
  I = \frac{V}{R}
  $$

  As an application, consider a circuit with a 9V battery with a 10,000 $\Omega$
  resistor attached across its terminals.

  The current flowing through the resistor can be calculated as follows:

  $$
  \frac{9V} {10,000\Omega} = 0.0009 \textsf{A} (0.9 \textsf{mA})
  $$


path: OpenSearch.md
sortable_path: OpenSearch.md
title: OpenSearch
lead: ## Background
checksum: d9fbc072965e3e5b5484e8077a808c28dc8740b7014912eda50b714a9ec80193
created: 2025-10-01 13:31:00.371581938+00:00
modified: 2025-08-13 15:57:13.752164617+00:00
metadata: {"tags":["AWS"]}
body:
  ## Background

  OpenSearch is the AWS implementation of the opean source Elasticsearch search
  engine. It was formally known as "AWS Elasticsearch".

  It has many features but the core usage is to create searchable indicies of a
  given content domain such as, for example, a website or content management
  system. This enables the quick search and retrieval of documents without using
  expensive database queries.

  ## Key concepts

  We will introduce the main concepts with the example of an internal intranet for
  which we want to create a searchable index. The intranet comprises hundreds of
  pages. Each page has the following metadata, conforming to the following
  example.

  ```json
  {
    "title": "Internal News",
    "author": "Jane Doe",
    "published_date": "2023-11-01T00:00:00Z",
    "tags": ["news", "internal"],
    "categories": ["communication"],
    "content": "Today's internal news and updates are..."
  }
  ```

  ### Create domain

  The OpenSearch domain is a managed environment which hosts OpenSearch
  **clusters**. It can contain one or more clusters.

  The domain provides network **endpoints** you use to communicate and send
  requests. Typical requests:

  - ingest data
  - index data
  - run search query and return matches

  #### Clusters and nodes

  A cluster is the highest level of organisation within an OpensSearch domain that
  contains your indexed data. It processes all the search queries and handles
  tasks like indexing, searching, and managing documents.

  ![](/img/opensearch-architecture.drawio.svg)

  A cluster comprises **nodes**. Nodes are individual servers that hold part of
  the cluster's data. Each node participates in the indexing and searching of the
  cluster's data.

  - This distributed architecture helps in balancing the load and ensuring high
    availability.
  - Data can be replicated accross nodes, making the system resiliant against data
    loss
  - You can add more nodes to the cluster to handle increased data and traffic,
    making the system adaptable to changing needs.

  ### Define index and mappings

  Assuming the domain has been created. The next step is to create an index for
  the data, say `intranet_pages`. An index is basically a collection where the
  data is stored, similar to a database. Each entry is a **document** in this
  collection.

  Our index will store each webpage as a document.

  We specify the data that we want to store using an index mapping. For instance
  we may not want to store all the metadata for each page, preferring only to
  index a subset of the properties. In this example we will store all the data. We
  would achieve this with the following:

  ```json
  {
    "mappings": {
      "properties": {
        "title": { "type": "text" },
        "author": { "type": "keyword" },
        "published_date": { "type": "date" },
        "tags": { "type": "keyword" },
        "content": { "type": "text" }
      }
    }
  }
  ```

  The mapping will be utilised in the following scenarios:

  - storing data:
    - when a new document is added to the `intranet_pages` index, it will adhere
      to the defined mappings
    - it will have to have the properties specified in the mapping in order to be
      added
  - searching data:
    - when executing searches, OpenSearch utilizes the mappings to understand the
      type of data in each field and optimizes search queries accordingly
  - assessing relevance:
    - proper mappings allow OpenSearch to accurately score and rank search results
      based on relevance.

  ### Ingest data (bulk import and/or scraper)

  In order to create the index that we previously defined with a mapping it is
  necessary to implement some sort of mechanism for collating the metadata that
  matches the mapping. This would be a crawler or scraper that might be
  implemented with a lambda. This is a key part of the ingestion process.

  Alternatively the data could be bulk imported in a format that maps to the
  index.

  ### Querying and searching

  Having established the crawler (and some kind of search interface), we can now
  run queries against the OpenSearch domain.

  A basic example of the structure of a query would be as follows:

  ```sh
  GET /intranet_pages/_search
  ```

  ```json
  {
    "query": {
      "match": {
        "content": "project"
      }
    }
  }
  ```

  Here we search against the `content` mapping to find pages that contain the word
  "project".

  Example of the data returned:

  ```json
  {
    "hits": {
      "total": { "value": 100 },
      "hits": [
        {
          "_index": "intranet_pages",
          "_id": "1",
          "_score": 1.2,
          "_source": {
            "title": "Project ABC Launch",
            "author": "John Doe",
            "published_date": "2023-01-01T00:00:00Z",
            "tags": ["project", "launch"],
            "content": "Details about the launch of Project ABC..."
          }
        }
        // Additional results here
      ]
    }
  }
  ```

  ## Search patterns

  Below are further examples of commonly used search patterns.

  ### Multiple conditions

  Find documents that are authored by Jane Doe that contain the word "meeting".
  `must` stands for boolean AND:

  ```json
  {
    "query": {
      "bool": {
        "must": [
          { "match": { "content": "meeting" } },
          { "match": { "author": "Jane Doe" } }
        ]
      }
    }
  }
  ```

  Find documents that contain either the word "meeting" or the word "project" in
  their content. `should` stands for boolean OR:

  ```json
  {
    "query": {
      "bool": {
        "should": [
          { "match": { "content": "meeting" } },
          { "match": { "content": "project" } }
        ],
        "minimum_should_match": 1
      }
    }
  }
  ```

  `minimum_should_match` specifies the number of conditions that should match.

  ### Query by date ranges

  Find pages published after a certain date:

  ```json
  {
    "query": {
      "range": {
        "published_date": {
          "gte": "2023-01-01T00:00:00Z"
        }
      }
    }
  }
  ```

  ```json
  {
    "query": {
      "bool": {
        "should": [
          { "match": { "fileId": "val" } },
          { "match": { "programmeId": "val" } },
          { "match": { "guid": "val" } }
        ],
        "minimum_should_match": 1
      }
    }
  }
  ```

raw_content:
  ---
  tags: [AWS]
  ---

  # OpenSearch

  ## Background

  OpenSearch is the AWS implementation of the opean source Elasticsearch search
  engine. It was formally known as "AWS Elasticsearch".

  It has many features but the core usage is to create searchable indicies of a
  given content domain such as, for example, a website or content management
  system. This enables the quick search and retrieval of documents without using
  expensive database queries.

  ## Key concepts

  We will introduce the main concepts with the example of an internal intranet for
  which we want to create a searchable index. The intranet comprises hundreds of
  pages. Each page has the following metadata, conforming to the following
  example.

  ```json
  {
    "title": "Internal News",
    "author": "Jane Doe",
    "published_date": "2023-11-01T00:00:00Z",
    "tags": ["news", "internal"],
    "categories": ["communication"],
    "content": "Today's internal news and updates are..."
  }
  ```

  ### Create domain

  The OpenSearch domain is a managed environment which hosts OpenSearch
  **clusters**. It can contain one or more clusters.

  The domain provides network **endpoints** you use to communicate and send
  requests. Typical requests:

  - ingest data
  - index data
  - run search query and return matches

  #### Clusters and nodes

  A cluster is the highest level of organisation within an OpensSearch domain that
  contains your indexed data. It processes all the search queries and handles
  tasks like indexing, searching, and managing documents.

  ![](/img/opensearch-architecture.drawio.svg)

  A cluster comprises **nodes**. Nodes are individual servers that hold part of
  the cluster's data. Each node participates in the indexing and searching of the
  cluster's data.

  - This distributed architecture helps in balancing the load and ensuring high
    availability.
  - Data can be replicated accross nodes, making the system resiliant against data
    loss
  - You can add more nodes to the cluster to handle increased data and traffic,
    making the system adaptable to changing needs.

  ### Define index and mappings

  Assuming the domain has been created. The next step is to create an index for
  the data, say `intranet_pages`. An index is basically a collection where the
  data is stored, similar to a database. Each entry is a **document** in this
  collection.

  Our index will store each webpage as a document.

  We specify the data that we want to store using an index mapping. For instance
  we may not want to store all the metadata for each page, preferring only to
  index a subset of the properties. In this example we will store all the data. We
  would achieve this with the following:

  ```json
  {
    "mappings": {
      "properties": {
        "title": { "type": "text" },
        "author": { "type": "keyword" },
        "published_date": { "type": "date" },
        "tags": { "type": "keyword" },
        "content": { "type": "text" }
      }
    }
  }
  ```

  The mapping will be utilised in the following scenarios:

  - storing data:
    - when a new document is added to the `intranet_pages` index, it will adhere
      to the defined mappings
    - it will have to have the properties specified in the mapping in order to be
      added
  - searching data:
    - when executing searches, OpenSearch utilizes the mappings to understand the
      type of data in each field and optimizes search queries accordingly
  - assessing relevance:
    - proper mappings allow OpenSearch to accurately score and rank search results
      based on relevance.

  ### Ingest data (bulk import and/or scraper)

  In order to create the index that we previously defined with a mapping it is
  necessary to implement some sort of mechanism for collating the metadata that
  matches the mapping. This would be a crawler or scraper that might be
  implemented with a lambda. This is a key part of the ingestion process.

  Alternatively the data could be bulk imported in a format that maps to the
  index.

  ### Querying and searching

  Having established the crawler (and some kind of search interface), we can now
  run queries against the OpenSearch domain.

  A basic example of the structure of a query would be as follows:

  ```sh
  GET /intranet_pages/_search
  ```

  ```json
  {
    "query": {
      "match": {
        "content": "project"
      }
    }
  }
  ```

  Here we search against the `content` mapping to find pages that contain the word
  "project".

  Example of the data returned:

  ```json
  {
    "hits": {
      "total": { "value": 100 },
      "hits": [
        {
          "_index": "intranet_pages",
          "_id": "1",
          "_score": 1.2,
          "_source": {
            "title": "Project ABC Launch",
            "author": "John Doe",
            "published_date": "2023-01-01T00:00:00Z",
            "tags": ["project", "launch"],
            "content": "Details about the launch of Project ABC..."
          }
        }
        // Additional results here
      ]
    }
  }
  ```

  ## Search patterns

  Below are further examples of commonly used search patterns.

  ### Multiple conditions

  Find documents that are authored by Jane Doe that contain the word "meeting".
  `must` stands for boolean AND:

  ```json
  {
    "query": {
      "bool": {
        "must": [
          { "match": { "content": "meeting" } },
          { "match": { "author": "Jane Doe" } }
        ]
      }
    }
  }
  ```

  Find documents that contain either the word "meeting" or the word "project" in
  their content. `should` stands for boolean OR:

  ```json
  {
    "query": {
      "bool": {
        "should": [
          { "match": { "content": "meeting" } },
          { "match": { "content": "project" } }
        ],
        "minimum_should_match": 1
      }
    }
  }
  ```

  `minimum_should_match` specifies the number of conditions that should match.

  ### Query by date ranges

  Find pages published after a certain date:

  ```json
  {
    "query": {
      "range": {
        "published_date": {
          "gte": "2023-01-01T00:00:00Z"
        }
      }
    }
  }
  ```

  ```json
  {
    "query": {
      "bool": {
        "should": [
          { "match": { "fileId": "val" } },
          { "match": { "programmeId": "val" } },
          { "match": { "guid": "val" } }
        ],
        "minimum_should_match": 1
      }
    }
  }
  ```


