# notes

path: Tuples_in_Python.md
sortable_path: Tuples_in_Python.md
title: Tuples in Python
lead: TODO: Exapand tuple notes - give more use cases
checksum: 030111bc7de9da55f9dfd32fbbf6212bc9095e33ae0e9729bc59f02b2410c4b6
created: 2025-10-01 13:31:02.488628669+00:00
modified: 2025-08-13 15:57:13.926016741+00:00
metadata: {"tags":["python","data-structures"]}
body:
  TODO: Exapand tuple notes - give more use cases

  Tuples are one of the main data-structures or containers in Python. Tuples are
  useful in cases where you want to group related data and ensure that it will not
  change.

  Tuples have the following properties:

  - They are **ordered**
  - They have a **fixed size**
  - They are **immutable** and cannot be modified
  - **Allow duplicate** members
  - They are **indexed**

  > In essence a tuple is a list that is immutable.

  As with all containers in Python they permit any data type.

  > Tuples are denoted with `(...)`

  ## Basic usage

  ```python
  tup1 = (1, 3, 5, 7)
  print(tup1[2])

  # 5
  """
  ```

  ## Slicing

  ```python
  tup1 = (1, 3, 5, 7)

  print(tup1[1:3])
  print(tup1[:3])
  print(tup1[1:])
  print(tup1[::-1])

  """
  (3, 5)
  (1, 3, 5)
  (3, 5, 7)
  (7, 5, 3, 1)
  """
  ```

  ## Looping

  ```python
  tup3 = ('apple', 'pear', 'orange', 'plum', 'apple')
  for x in tup3:
      print(x)

  """
  apple
  pear
  orange
  plum
  apple
  """
  ```

  ## Useful methods and predicates

  ```python
  tup3 = ('apple', 'pear', 'orange', 'plum', 'apple')

  # Count instances of a member
  print(tup3.count('apple'))
  # 2

  # Get index of a member
  print(tup3.index('pear'))
  # 1

  # Check for membership
  if 'orange' in tup3:
      print('orange is in the Tuple')

  # orange is in the Tuple
  ```

  ## Nest tuples

  ```python

  tuple2 = ('John', 'Denise', 'Phoebe', 'Adam')
  tuple3 = (42, tuple1, tuple2, 5.5)
  print(tuple3)

  # (42, (1, 3, 5, 7), ('John', 'Denise', 'Phoebe', 'Adam'), 5.5)

  ```

  // TODO: How to flatten a tuple?

raw_content:
  ---
  tags: [python, data-structures]
  ---

  # Tuples in Python

  TODO: Exapand tuple notes - give more use cases

  Tuples are one of the main data-structures or containers in Python. Tuples are
  useful in cases where you want to group related data and ensure that it will not
  change.

  Tuples have the following properties:

  - They are **ordered**
  - They have a **fixed size**
  - They are **immutable** and cannot be modified
  - **Allow duplicate** members
  - They are **indexed**

  > In essence a tuple is a list that is immutable.

  As with all containers in Python they permit any data type.

  > Tuples are denoted with `(...)`

  ## Basic usage

  ```python
  tup1 = (1, 3, 5, 7)
  print(tup1[2])

  # 5
  """
  ```

  ## Slicing

  ```python
  tup1 = (1, 3, 5, 7)

  print(tup1[1:3])
  print(tup1[:3])
  print(tup1[1:])
  print(tup1[::-1])

  """
  (3, 5)
  (1, 3, 5)
  (3, 5, 7)
  (7, 5, 3, 1)
  """
  ```

  ## Looping

  ```python
  tup3 = ('apple', 'pear', 'orange', 'plum', 'apple')
  for x in tup3:
      print(x)

  """
  apple
  pear
  orange
  plum
  apple
  """
  ```

  ## Useful methods and predicates

  ```python
  tup3 = ('apple', 'pear', 'orange', 'plum', 'apple')

  # Count instances of a member
  print(tup3.count('apple'))
  # 2

  # Get index of a member
  print(tup3.index('pear'))
  # 1

  # Check for membership
  if 'orange' in tup3:
      print('orange is in the Tuple')

  # orange is in the Tuple
  ```

  ## Nest tuples

  ```python

  tuple2 = ('John', 'Denise', 'Phoebe', 'Adam')
  tuple3 = (42, tuple1, tuple2, 5.5)
  print(tuple3)

  # (42, (1, 3, 5, 7), ('John', 'Denise', 'Phoebe', 'Adam'), 5.5)

  ```

  // TODO: How to flatten a tuple?


path: Turing_completeness.md
sortable_path: Turing_completeness.md
title: Turing Completeness
lead:
  We know that a [Turing machine](Turing_machines.md) is a theoretical construct
  of a computer that:

checksum: 6955170d1e0c4ae74beaea68d24b8d72bbe977f01608376028bdbd8b888f430e
created: 2025-10-01 13:31:02.4967536+00:00
modified: 2025-08-13 15:57:14.006772424+00:00
metadata: {"created":"Friday, September 13, 2024","tags":["theory-of-computation","Turing"]}
body:
  We know that a [Turing machine](Turing_machines.md) is a theoretical construct
  of a computer that:

  > contains mutable state, executes sequences of simple instructions that read
  > and write that state, and can pick different execution paths depending on the
  > state (via conditional branch instructions.)

  A Turing Complete (TC) system is a system that abides by, or can be reduced to,
  the above description.

  TC also serves as a _definition of computability_ and provides a formal basis
  for conceiving of computation at a theoretical level.

  All Turing Complete systems are functionally equivalent. This means they can
  simulate each other given enough time and memory. Similarly a TC system can in
  principle perform any computation that any other programmable computer can
  perform. This is true for _other_ TC systems and also those that are not TC
  however the inverse doesn't hold: a non-TC system cannot emulate a TS system.
  For instance a calculator cannot do what a TC smart phone can do. But a smart
  phone can act as a calculator.

  This concept of completeness is also expressed in terms of a Universal Turing
  Machine - a TM capable of simulating any other Turing machine. Given that the
  ability of a TM to simulate another TM is the condition for completeness, a TC
  system is also a UTM.

  Completeness applies to the hardware of computers as well as their software.

  Turing Completeness is the theoretical basis of the practical concept of a
  "general-purpose computer": a general-purpose computer is such because it is
  TC - it can in theory compute anything that is computable.

  Most modern programming languages are Turing Complete in that they can, in
  theory, be used to compute anything that is computable.

raw_content:
  ---
  tags: [theory-of-computation, Turing]
  created: Friday, September 13, 2024
  ---

  # Turing Completeness

  We know that a [Turing machine](Turing_machines.md) is a theoretical construct
  of a computer that:

  > contains mutable state, executes sequences of simple instructions that read
  > and write that state, and can pick different execution paths depending on the
  > state (via conditional branch instructions.)

  A Turing Complete (TC) system is a system that abides by, or can be reduced to,
  the above description.

  TC also serves as a _definition of computability_ and provides a formal basis
  for conceiving of computation at a theoretical level.

  All Turing Complete systems are functionally equivalent. This means they can
  simulate each other given enough time and memory. Similarly a TC system can in
  principle perform any computation that any other programmable computer can
  perform. This is true for _other_ TC systems and also those that are not TC
  however the inverse doesn't hold: a non-TC system cannot emulate a TS system.
  For instance a calculator cannot do what a TC smart phone can do. But a smart
  phone can act as a calculator.

  This concept of completeness is also expressed in terms of a Universal Turing
  Machine - a TM capable of simulating any other Turing machine. Given that the
  ability of a TM to simulate another TM is the condition for completeness, a TC
  system is also a UTM.

  Completeness applies to the hardware of computers as well as their software.

  Turing Completeness is the theoretical basis of the practical concept of a
  "general-purpose computer": a general-purpose computer is such because it is
  TC - it can in theory compute anything that is computable.

  Most modern programming languages are Turing Complete in that they can, in
  theory, be used to compute anything that is computable.


path: Turing_machines.md
sortable_path: Turing_machines.md
title: Turing Machines
lead: ## What is a Turing Machine?
checksum: 77c4c42bdf565593e4df5a982bb47a1527efcfc69dd5cf0904e13b214dea73d8
created: 2025-10-01 13:31:02.514386213+00:00
modified: 2025-08-13 15:57:13.96831508+00:00
metadata: {"tags":["theory-of-computation","Turing"]}
body:
  ## What is a Turing Machine?

  Turing Machine is a machine that contains mutable state, executes sequences of
  simple instructions that read and write that state, and can pick different
  execution paths depending on the state (via conditional branch instructions.)

  This is a general expression, it is typically expanded as follows.

  The machine consists of an infinitely long tape, that has been divided up into
  cells. Each cell can contain either a 1, a 0 or an empty space. Above one cell
  of the tape is a head, which can either move left or right, and can read the
  symbols written in the cells. The head is also capable of erasing symbols and
  writing new symbols into the cells.

  ![Turing_machines_01.gif](Turing_machines_01.gif)

  The direction that the head moves, which values it erases, and which values it
  writes in, are dependent on a set of instructions provided to the machine.

  Different sets of instructions can be divided into **states.** States are like
  sub-routines and can themselves feature as part of instructions.

  For example:

  ### State 2

  - If 0 then erase
  - Write 1 then move right
  - Go to state 5

  ### State 5

  - If 1, then erase
  - Write 0 then move left
  - Go to state _n_

  Alan Turing proved that **any problem that is computable** can be computed by a
  Turing Machine using this simple system.

raw_content:
  ---
  tags:
    - theory-of-computation
    - Turing
  ---

  # Turing Machines

  ## What is a Turing Machine?

  Turing Machine is a machine that contains mutable state, executes sequences of
  simple instructions that read and write that state, and can pick different
  execution paths depending on the state (via conditional branch instructions.)

  This is a general expression, it is typically expanded as follows.

  The machine consists of an infinitely long tape, that has been divided up into
  cells. Each cell can contain either a 1, a 0 or an empty space. Above one cell
  of the tape is a head, which can either move left or right, and can read the
  symbols written in the cells. The head is also capable of erasing symbols and
  writing new symbols into the cells.

  ![Turing_machines_01.gif](Turing_machines_01.gif)

  The direction that the head moves, which values it erases, and which values it
  writes in, are dependent on a set of instructions provided to the machine.

  Different sets of instructions can be divided into **states.** States are like
  sub-routines and can themselves feature as part of instructions.

  For example:

  ### State 2

  - If 0 then erase
  - Write 1 then move right
  - Go to state 5

  ### State 5

  - If 1, then erase
  - Write 0 then move left
  - Go to state _n_

  Alan Turing proved that **any problem that is computable** can be computed by a
  Turing Machine using this simple system.


path: Twos_complement.md
sortable_path: Twos_complement.md
title: Two's complement
lead: ## Summary
checksum: 6fb7bb890f9352267a3cd807838ba5cc81c5affae5d5d6fc0528d34dde324d4d
created: 2025-10-01 13:31:02.52261772+00:00
modified: 2025-08-13 15:57:13.796077112+00:00
metadata: {"created":"Tuesday, March 19, 2024","id":"gktb","tags":["binary"]}
body:
  ## Summary

  - _Two's complement_ is a method for representing signed numbers (negative
    integers) in binary.

  - It is derived by inverting the values of an unsigned binary integer to create
    its signed equivalent.

  - A benefit is that hardware implementation of the binary arithmetic of signed
    and unsigned numbers can be handled in the same manner as unsigned numbers,
    requiring no additional handling. A drawback is that it halves the
    informational capacity of the given word length for the binary system.

  ## Detail

  ### Procedural steps

  Two's complement divides the available word length (see
  [[Binary_encoding|binary encoding]]) into two subsets: one for negative integrs
  and one for positive integers.

  Take the binary encoding of decimal five (`0101`). Its complement is `1011`.

  The procedure for deriving the complement is as follows.

  To derive the complement of an unsigned number:

  1. Take the unsigned number and invert its digits: `0` becomes `1`, `1` becomes
     `0`
  2. Add one

  ![](/img/unsigned-to-signed.png)

  To derive the unsigned equivalent of a signed number you invert the process but
  still make the smallest digit `1`:

  ![](/img/signed-to-unsigned.png)

  ### Formal expression

  $$
      2^n - x
  $$

  - where $x$ is the negative integer in binary that we wish to derive
  - where $n$ is the word length of the binary system in bits.

  Applied to the earlier example we have $2^4 -5$ which is:

  $$
      16 - 5 = 11
  $$

  When we convert the decimal `11` to binary we get `1011` which is identical to
  the signed version of the unsigned integer.

  We can confirm the correctness of the derviation by summing the signed and
  unsigned binary values. If this results in zeros (ignoring the overflow bit),
  the derivation is correct as the two values effectively cancel each other out:

  $$

    1011 + 0101 = 0000
  $$

  ### Advantages

  - The circuit implementation of arithmetic involving positive and negative
    integers is the same as the implementation of positive integers. There is no
    need for additional harware or special handling of the values.
  - This can be contrasted with the alternative approaches to signing numbers such
    as **signed magnitude representation** which uses certain bits as designators
    of negative/positive status.

  ### Limitations

  - Two's complement reduces the overall informational capacity of the given
    binary word length, effectively halving the total number of unique values.

  - In a 4-bit system instead of 16 total unique encodings of integers you have 8
    encodings for positive integers and 8 encodings for the their signed
    equivalent. For integers larger than denary 8 you would need to increase the
    bit length of the system

  - Consequently two's complement can necessitate larger overall word lengths.

  ## Related notes

  [[Signed_and_unsigned_numbers|signed_and_unsigned_numbers]],
  [[Binary_addition|binary addition]], [[Signed_magnitude_representation]]

raw_content:
  ---
  id: gktb
  tags:
    - binary
  created: Tuesday, March 19, 2024
  ---

  # Two's complement

  ## Summary

  - _Two's complement_ is a method for representing signed numbers (negative
    integers) in binary.

  - It is derived by inverting the values of an unsigned binary integer to create
    its signed equivalent.

  - A benefit is that hardware implementation of the binary arithmetic of signed
    and unsigned numbers can be handled in the same manner as unsigned numbers,
    requiring no additional handling. A drawback is that it halves the
    informational capacity of the given word length for the binary system.

  ## Detail

  ### Procedural steps

  Two's complement divides the available word length (see
  [[Binary_encoding|binary encoding]]) into two subsets: one for negative integrs
  and one for positive integers.

  Take the binary encoding of decimal five (`0101`). Its complement is `1011`.

  The procedure for deriving the complement is as follows.

  To derive the complement of an unsigned number:

  1. Take the unsigned number and invert its digits: `0` becomes `1`, `1` becomes
     `0`
  2. Add one

  ![](/img/unsigned-to-signed.png)

  To derive the unsigned equivalent of a signed number you invert the process but
  still make the smallest digit `1`:

  ![](/img/signed-to-unsigned.png)

  ### Formal expression

  $$
      2^n - x
  $$

  - where $x$ is the negative integer in binary that we wish to derive
  - where $n$ is the word length of the binary system in bits.

  Applied to the earlier example we have $2^4 -5$ which is:

  $$
      16 - 5 = 11
  $$

  When we convert the decimal `11` to binary we get `1011` which is identical to
  the signed version of the unsigned integer.

  We can confirm the correctness of the derviation by summing the signed and
  unsigned binary values. If this results in zeros (ignoring the overflow bit),
  the derivation is correct as the two values effectively cancel each other out:

  $$

    1011 + 0101 = 0000
  $$

  ### Advantages

  - The circuit implementation of arithmetic involving positive and negative
    integers is the same as the implementation of positive integers. There is no
    need for additional harware or special handling of the values.
  - This can be contrasted with the alternative approaches to signing numbers such
    as **signed magnitude representation** which uses certain bits as designators
    of negative/positive status.

  ### Limitations

  - Two's complement reduces the overall informational capacity of the given
    binary word length, effectively halving the total number of unique values.

  - In a 4-bit system instead of 16 total unique encodings of integers you have 8
    encodings for positive integers and 8 encodings for the their signed
    equivalent. For integers larger than denary 8 you would need to increase the
    bit length of the system

  - Consequently two's complement can necessitate larger overall word lengths.

  ## Related notes

  [[Signed_and_unsigned_numbers|signed_and_unsigned_numbers]],
  [[Binary_addition|binary addition]], [[Signed_magnitude_representation]]


path: Type_guarding_and_narrowing_in_TS.md
sortable_path: Type_guarding_and_narrowing_in_TS.md
title: Type narrowing and guarding
checksum: 490017e14d24edb1d0dbb214c845e7c0f56679e29856822fd555a7f4d46e88b5
created: 2025-10-01 13:31:02.552898841+00:00
modified: 2025-08-13 15:57:13.927326011+00:00
metadata: {"tags":["typescript"]}
lead:
  Type narrowing is the process of working out from a supertype like
  [any](Any.md) or [unknown](Unknown_type_in_TS.md) whic type the value should be in the
  x course of your code. This is necessary since a process will not necessarily
  always return or involve homogenous types.

body:
  Type narrowing is the process of working out from a supertype like
  [any](Any.md) or [unknown](Unknown_type_in_TS.md) whic type the value should be in the
  x course of your code. This is necessary since a process will not necessarily
  always return or involve homogenous types.

  Related to this is type guarding: ensuring that a value is of the suitable type
  as a factor of control flow. For instance using typeof to ensure that an input
  is numerical before proceeding with a function's logic.

raw_content:
  ---
  tags:
    - typescript
  ---

  # Type narrowing and guarding

  Type narrowing is the process of working out from a supertype like
  [any](Any.md) or [unknown](Unknown_type_in_TS.md) whic type the value should be in the
  x course of your code. This is necessary since a process will not necessarily
  always return or involve homogenous types.

  Related to this is type guarding: ensuring that a value is of the suitable type
  as a factor of control flow. For instance using typeof to ensure that an input
  is numerical before proceeding with a function's logic.


path: Type_hinting.md
sortable_path: Type_hinting.md
title: Type hinting in Python
checksum: 7e6a750a862c6677d8b50ec86ee2eab0fe82d4c821bddfb06e363d5324da6239
created: 2025-10-01 13:31:02.571430116+00:00
modified: 2025-08-13 15:57:13.820465617+00:00
metadata: {"tags":["python","data-types"]}
lead:
  With type hinting we can add type information to variables, functions, and
  classes. This is not enforced by the Python interpreter but can be used by
  external tools like `mypy` to check the code.

body:
  With type hinting we can add type information to variables, functions, and
  classes. This is not enforced by the Python interpreter but can be used by
  external tools like `mypy` to check the code.

  Prior to 3.10, to use types you would have to import the types you wanted `from`
  the `typing` module. With later versions you can use them directly without
  importing from `typing`.

  ## Basic syntax

  ### Variables

  ```py
  name: str = "John"
  age: int = 30
  ```

  ### Functions

  ```py
  def greet(name: str) -> str:
      return "Hello, " + name
  ```

  ### Classes

  // TODO: Add examples

  ## Complex data types

  We can type complex data types such as dictionaries, lists, tuples etc.

  The following function takes a tuple comprising two integers as a parameter and
  returns a list of ints:

  ```py
  def populate_pair_values(pair: tuple[int, int]) -> list[int]:
      return [i for i in range(pair[0], pair[1] + 1)]
  ```

  ## Custom dictionary types

  We can create custom dictionary types in the manner of TypeScript types and
  interfaces as follows:

  ```py
  class ArticleInfo(TypedDict):
      timestamp: str
      article_    link: str
  ```

  We could then use it thus:

  ```py
  def parse_articles() -> list[ArticleInfo]:
  ```

  ### Explicit type construction

  Sometimes, simply asserting that custom type will be returned by a function is
  not sufficient to pass the type checks. In these cases you need to explicitly
  construct the type within the function.

  For example this generates an error as Python sees the `tags` type as
  `[Dictionary]`, even though it matches the type:

  ```python
  class IGraphNode(TypedDict):
      id: str
      type: str

  ```

  ```py
  tags = [{"id": f"#{tag[0]}", "type": "tag"} for tag in tags]
  ```

  So you have to explicitly generate the type like this:

  ```python
  tags = [IGraphNode(id=f"#{tag[0]}", type="tag") for tag in tags]
  ```

  Then you will be able to type the return of the function as
  `-> list[IGraphNode]`

  ## Optional types

  `Optional` can be used to indicate that a variable can be `None` or the
  specified type.

  ```py
  from typing import Optional

  def find_index(numbers: list[int], target: int) -> Optional[int]:
      try:
          return numbers.index(target)
      except ValueError:
          return None
  ```

  The function above returns an `int` or `None`.

  Post 3.10, we don't need to use `Optional`, we can use a union to cover the
  `None` case. Refactoring the previous example:

  ```py
  def find_index(numbers: list[int], target: int) -> int | None:
      try:
          return numbers.index(target)
      except ValueError:
          return None
  ```

  ## Union types

  `Union` can be used to indicate that a variable can be one of several types.

  ```py
  from typing import Union

  def add(a: Union[int, float], b: Union[int, float]) -> Union[int, float]:
      return a + b
  ```

  The function above returns an `int` or `float`.

  > Note that in the case of `Optional` and `Union` the type is specified using
  > square brackets. This is a syntactic covnention and does not denote that the
  > return type is a list. Below is an example of how we would use type hinting to
  > indicate that the return type actually is a list:

  ```py
  from typing import List, Union
  def my_function() -> List[int]:
      return [1, 2, 3]

  # And to show a return type of a list of integers or floats:

  def my_function() -> List[Union[int, float]]:
      return [1, 2, 3]
  ```

  Post 3.10 we can use `|` syntax instead of importing `Union`. E.g:

  ```py

  def add(a: int | float, b: int | float) -> int | float:
      return a + b
  ```

raw_content:
  ---
  tags:
    - python
    - data-types
  ---

  # Type hinting in Python

  With type hinting we can add type information to variables, functions, and
  classes. This is not enforced by the Python interpreter but can be used by
  external tools like `mypy` to check the code.

  Prior to 3.10, to use types you would have to import the types you wanted `from`
  the `typing` module. With later versions you can use them directly without
  importing from `typing`.

  ## Basic syntax

  ### Variables

  ```py
  name: str = "John"
  age: int = 30
  ```

  ### Functions

  ```py
  def greet(name: str) -> str:
      return "Hello, " + name
  ```

  ### Classes

  // TODO: Add examples

  ## Complex data types

  We can type complex data types such as dictionaries, lists, tuples etc.

  The following function takes a tuple comprising two integers as a parameter and
  returns a list of ints:

  ```py
  def populate_pair_values(pair: tuple[int, int]) -> list[int]:
      return [i for i in range(pair[0], pair[1] + 1)]
  ```

  ## Custom dictionary types

  We can create custom dictionary types in the manner of TypeScript types and
  interfaces as follows:

  ```py
  class ArticleInfo(TypedDict):
      timestamp: str
      article_    link: str
  ```

  We could then use it thus:

  ```py
  def parse_articles() -> list[ArticleInfo]:
  ```

  ### Explicit type construction

  Sometimes, simply asserting that custom type will be returned by a function is
  not sufficient to pass the type checks. In these cases you need to explicitly
  construct the type within the function.

  For example this generates an error as Python sees the `tags` type as
  `[Dictionary]`, even though it matches the type:

  ```python
  class IGraphNode(TypedDict):
      id: str
      type: str

  ```

  ```py
  tags = [{"id": f"#{tag[0]}", "type": "tag"} for tag in tags]
  ```

  So you have to explicitly generate the type like this:

  ```python
  tags = [IGraphNode(id=f"#{tag[0]}", type="tag") for tag in tags]
  ```

  Then you will be able to type the return of the function as
  `-> list[IGraphNode]`

  ## Optional types

  `Optional` can be used to indicate that a variable can be `None` or the
  specified type.

  ```py
  from typing import Optional

  def find_index(numbers: list[int], target: int) -> Optional[int]:
      try:
          return numbers.index(target)
      except ValueError:
          return None
  ```

  The function above returns an `int` or `None`.

  Post 3.10, we don't need to use `Optional`, we can use a union to cover the
  `None` case. Refactoring the previous example:

  ```py
  def find_index(numbers: list[int], target: int) -> int | None:
      try:
          return numbers.index(target)
      except ValueError:
          return None
  ```

  ## Union types

  `Union` can be used to indicate that a variable can be one of several types.

  ```py
  from typing import Union

  def add(a: Union[int, float], b: Union[int, float]) -> Union[int, float]:
      return a + b
  ```

  The function above returns an `int` or `float`.

  > Note that in the case of `Optional` and `Union` the type is specified using
  > square brackets. This is a syntactic covnention and does not denote that the
  > return type is a list. Below is an example of how we would use type hinting to
  > indicate that the return type actually is a list:

  ```py
  from typing import List, Union
  def my_function() -> List[int]:
      return [1, 2, 3]

  # And to show a return type of a list of integers or floats:

  def my_function() -> List[Union[int, float]]:
      return [1, 2, 3]
  ```

  Post 3.10 we can use `|` syntax instead of importing `Union`. E.g:

  ```py

  def add(a: int | float, b: int | float) -> int | float:
      return a + b
  ```


path: Typing_built_in_React_hooks.md
sortable_path: Typing_built_in_React_hooks.md
title: Typing built-in React hooks
lead: ## `useState`
checksum: bf784ae842ad366733d7363e90c2b591524805c04410cb062cf6c9561b98564e
created: 2025-10-01 13:31:02.579529566+00:00
modified: 2025-08-13 15:57:13.87462458+00:00
metadata: {"tags":["typescript","react"]}
body:
  ## `useState`

  ```tsx
  const [amount, setAmount] = useState<number | string>(3);
  ```

  ### With a ustom type

  ```tsx
  interface IState {
    people: IPerson[];
  }

  interface IPerson {
    name: string;
    age: number;
  }

  const [people, setPeople] = useState<IState>({});

  // Alternative declaration

  interface IState {
    people: {
      name: string;
      age: number;
    }[];
  }

  const [people, setPeople] = useState<IState["people"]>({});
  ```

  ## `useReducer`

  Mostly the same. They key thing to type is the reducer function itself. You
  don't need to type the intitial state unless it is a pre-existing type, you can
  just use type assertion:

  ```jsx
  interface IAction {
    | { type: "increment"; payload: number }
    | { type: "decrement"; payload: string };
  }

  const initialState = { count: 0 };

  function reducer(state: typeof initialState, action: IAction) {
    switch (action.type) {
      case "increment":
        return { count: state.count + action.payload };
      case "decrement":
        return { count: state.count - Number(action.payload) };
      default:
        throw new Error();
    }
  }
  ```

  Then the syntax is as normal, i.e:

  ```js
  const [state, dispatch] = useReducer(reducer, initialState);
  ```

  ## `useEffect`

  > Limited potential for typing here as this hook does not return a value. See if
  > there is useful inferred typing from use and update.

  ## `useContext`

  We can use generics but mostly this is untyped:

  ```jsx
  type Theme = "light" | "dark";
  const ThemeContext = createContext < Theme > "dark";

  const App = () => (
    <ThemeContext.Provider value="dark">
      <MyComponent />
    </ThemeContext.Provider>
  );

  const MyComponent = () => {
    const theme = useContext(ThemeContext);
    return <div>The theme is {theme}</div>;
  };
  ```

  ## `useCallback` / `useMemo`

  You must only type the parameters that are passed to the callback, the return
  value will be inferred.

raw_content:
  ---
  tags:
    - typescript
    - react
  ---

  # Typing built-in React hooks

  ## `useState`

  ```tsx
  const [amount, setAmount] = useState<number | string>(3);
  ```

  ### With a ustom type

  ```tsx
  interface IState {
    people: IPerson[];
  }

  interface IPerson {
    name: string;
    age: number;
  }

  const [people, setPeople] = useState<IState>({});

  // Alternative declaration

  interface IState {
    people: {
      name: string;
      age: number;
    }[];
  }

  const [people, setPeople] = useState<IState["people"]>({});
  ```

  ## `useReducer`

  Mostly the same. They key thing to type is the reducer function itself. You
  don't need to type the intitial state unless it is a pre-existing type, you can
  just use type assertion:

  ```jsx
  interface IAction {
    | { type: "increment"; payload: number }
    | { type: "decrement"; payload: string };
  }

  const initialState = { count: 0 };

  function reducer(state: typeof initialState, action: IAction) {
    switch (action.type) {
      case "increment":
        return { count: state.count + action.payload };
      case "decrement":
        return { count: state.count - Number(action.payload) };
      default:
        throw new Error();
    }
  }
  ```

  Then the syntax is as normal, i.e:

  ```js
  const [state, dispatch] = useReducer(reducer, initialState);
  ```

  ## `useEffect`

  > Limited potential for typing here as this hook does not return a value. See if
  > there is useful inferred typing from use and update.

  ## `useContext`

  We can use generics but mostly this is untyped:

  ```jsx
  type Theme = "light" | "dark";
  const ThemeContext = createContext < Theme > "dark";

  const App = () => (
    <ThemeContext.Provider value="dark">
      <MyComponent />
    </ThemeContext.Provider>
  );

  const MyComponent = () => {
    const theme = useContext(ThemeContext);
    return <div>The theme is {theme}</div>;
  };
  ```

  ## `useCallback` / `useMemo`

  You must only type the parameters that are passed to the callback, the return
  value will be inferred.


path: UFW_firewall_management.md
sortable_path: UFW_firewall_management.md
title: UFW firewall management
checksum: 6d9fc7c6c7fd064004870b60232cb4190c34bc94e8eea3cd855cb02dba64091d
created: 2025-10-01 13:31:02.585026738+00:00
modified: 2025-08-13 15:57:13.83136883+00:00
metadata: {"created":"Sunday, February 16, 2025","tags":["servers","procedural","linux","firewalls"]}
lead:
  `ufw` (Uncomplicated Firewall) simplifies the process of setting up firewalls on
  your server, without directly using `iptables`.

body:
  `ufw` (Uncomplicated Firewall) simplifies the process of setting up firewalls on
  your server, without directly using `iptables`.

  ```sh
  sudo apt install ufw
  ```

  First we deny all incoming requests as the default:

  ```sh
  ufw default deny incoming
  ```

  And allow all outgoing requests as the default:

  ```sh
  ufw default allow outgoing
  ```

  Next we need to allow certain requests based on their port:

  | Port | Service | Rule  |
  | ---- | ------- | ----- |
  | 22   | SSH     | allow |
  | 80   | HTTP    | allow |
  | 443  | HTTPS   | allow |

  ```sh
  sudo ufw allow 22/tcp
  sudo ufw allow 80/tcp
  sudo ufw allow 443/tcp
  ```

  Enable:

  ```sh
  ufw enable
  ```

  Verify rules:

  ```sh
  ufw status --verbose
  ```

  ```

  ```

raw_content:
  ---
  tags: [servers, procedural, linux, firewalls]
  created: Sunday, February 16, 2025
  ---

  # UFW firewall management

  `ufw` (Uncomplicated Firewall) simplifies the process of setting up firewalls on
  your server, without directly using `iptables`.

  ```sh
  sudo apt install ufw
  ```

  First we deny all incoming requests as the default:

  ```sh
  ufw default deny incoming
  ```

  And allow all outgoing requests as the default:

  ```sh
  ufw default allow outgoing
  ```

  Next we need to allow certain requests based on their port:

  | Port | Service | Rule  |
  | ---- | ------- | ----- |
  | 22   | SSH     | allow |
  | 80   | HTTP    | allow |
  | 443  | HTTPS   | allow |

  ```sh
  sudo ufw allow 22/tcp
  sudo ufw allow 80/tcp
  sudo ufw allow 443/tcp
  ```

  Enable:

  ```sh
  ufw enable
  ```

  Verify rules:

  ```sh
  ufw status --verbose
  ```

  ```

  ```


path: USB-C.md
sortable_path: USB-C.md
title: USB-C
checksum: 9fd17c5cc79dec273849719ba1244026f36b2db8c7d5df229cd29e969721d067
created: 2025-10-01 13:31:02.592142591+00:00
modified: 2025-08-13 15:57:13.800922842+00:00
metadata: {"created":"Wednesday, March 12, 2025","tags":["hardware","usb"]}
lead:
  The "C" in the name refers to the shape of the _connector_ not the protocol that
  is used for the data transfer. (This is the case for all USB "types" which makes
  things confusing.)

body:
  The "C" in the name refers to the shape of the _connector_ not the protocol that
  is used for the data transfer. (This is the case for all USB "types" which makes
  things confusing.)

  Multiple protocols can use the same connector for input/output. For instance
  Thunderbolt can transfer over USB-C but not every port that can fit a USB-C is a
  Thunderbolt port. The device's internal hardware determines the protocols
  available.

  The USB-C connector can serve all of the following protocols (from slowest to
  fastest):

  - USB 1.0 (12mb/s)
  - USB 2.0 (480Mb/s)
  - USB 3.0 (5Gb/s)
  - USB 3.1 (10Gb/s)
  - USB 3.2 (20Gb/s)
  - USB 4 (40Gb/s)
  - Thunderbolt (40Gb/s)

  A USB-C connector is equipped to just charge a device as well as transfer data.

  The world is converging (often due to legislation) on USB-C as the global
  standard. It can be one connector for all existing protocols and future
  protocols (for about 15 years). Its physical design an form factor can support
  future advancements whilst using the same connector.

  Ultimately USB-C will be surpassed by more another yet more advanced connector
  and associated protocols but for the time being we can stop using the older
  models.

raw_content:
  ---
  tags: [hardware, usb]
  created: Wednesday, March 12, 2025
  ---

  # USB-C

  The "C" in the name refers to the shape of the _connector_ not the protocol that
  is used for the data transfer. (This is the case for all USB "types" which makes
  things confusing.)

  Multiple protocols can use the same connector for input/output. For instance
  Thunderbolt can transfer over USB-C but not every port that can fit a USB-C is a
  Thunderbolt port. The device's internal hardware determines the protocols
  available.

  The USB-C connector can serve all of the following protocols (from slowest to
  fastest):

  - USB 1.0 (12mb/s)
  - USB 2.0 (480Mb/s)
  - USB 3.0 (5Gb/s)
  - USB 3.1 (10Gb/s)
  - USB 3.2 (20Gb/s)
  - USB 4 (40Gb/s)
  - Thunderbolt (40Gb/s)

  A USB-C connector is equipped to just charge a device as well as transfer data.

  The world is converging (often due to legislation) on USB-C as the global
  standard. It can be one connector for all existing protocols and future
  protocols (for about 15 years). Its physical design an form factor can support
  future advancements whilst using the same connector.

  Ultimately USB-C will be surpassed by more another yet more advanced connector
  and associated protocols but for the time being we can stop using the older
  models.


path: USB_protocols_and_connectors.md
sortable_path: USB_protocols_and_connectors.md
title: USB protocols and connectors
checksum: 3ba7bc2c7eb1e116177febd1956dea73c7ee754dee929cde0a7fb473ec304f6d
created: 2025-10-01 13:31:02.601598194+00:00
modified: 2025-08-13 15:57:13.978537751+00:00
metadata: {"created":"Wednesday, March 12, 2025","tags":["protocols","USB"]}
lead:
  Different USB connectors can transfer according to different protocols. A
  connector is not always tied to a specific protocol. For instance each
  subsequent generation of connector is typically backwards-compatible with the
  protocol of the connector that it superseded.

body:
  Different USB connectors can transfer according to different protocols. A
  connector is not always tied to a specific protocol. For instance each
  subsequent generation of connector is typically backwards-compatible with the
  protocol of the connector that it superseded.

  Some connectors are limited to slower protocols but they have the example of
  being small which is a benefit for smaller devices. The USB-C connector is so
  good because it is both small and capable of supporting the most advanced
  protocols (USB-4 and Thunderbolt).

  The main types of USB connector:

  - USB-A/USB-B
  - USB-A/USB-B
  - Micro-USB ()
  - Mini-USB
  - USB-C

  The illustration below matches connector to the protocol supported:

  ![](../img/usb-illustration.png)

raw_content:
  ---
  tags: [protocols, USB]
  created: Wednesday, March 12, 2025
  ---

  # USB protocols and connectors

  Different USB connectors can transfer according to different protocols. A
  connector is not always tied to a specific protocol. For instance each
  subsequent generation of connector is typically backwards-compatible with the
  protocol of the connector that it superseded.

  Some connectors are limited to slower protocols but they have the example of
  being small which is a benefit for smaller devices. The USB-C connector is so
  good because it is both small and capable of supporting the most advanced
  protocols (USB-4 and Thunderbolt).

  The main types of USB connector:

  - USB-A/USB-B
  - USB-A/USB-B
  - Micro-USB ()
  - Mini-USB
  - USB-C

  The illustration below matches connector to the protocol supported:

  ![](../img/usb-illustration.png)


path: Union_types_in_TS.md
sortable_path: Union_types_in_TS.md
title: Union types
checksum: c821ae262bc36f541ebebc61d22c3d5bd95bb51fa3dd7b44522c80fe2956ee3f
created: 2025-10-01 13:31:02.610084232+00:00
modified: 2025-08-13 15:57:13.778915262+00:00
metadata: {"tags":["typescript"]}
lead:
  Like intersection types, union types allow us to combine the properties of
  existing primitive and custom types to make composites.

body:
  Like intersection types, union types allow us to combine the properties of
  existing primitive and custom types to make composites.

  Intersection types work well when you have complete knowledge of what form your
  data will take. This approach only works if you know in advance that your data
  will be and that it will share properties from more than one type. But what if
  you don't have this knowledge? You only know that it will be one of the three
  possible types? This is where union types become helpful. Instead of saying that
  event X will be of type A we say that event X may be one of types A or B or C.

  > A union type is a type formed from two or more other types, representing
  > values that may be any one of those types. We refer to each of these types as
  > the union’s members.

  ## Demonstration

  ```ts
  type Petrol = {
    maxSpeed: number;
    tankVolume: number; // unique property
  };

  type Electric = {
    maxSpeed: number;
    maxCharge: number; // unique property
  };

  type Car = Petrol | Electric;

  const tesla: Car = {
    maxSpeed: 120,
    maxCharge: 22,
  };
  ```

  - We have defined two custom types: `Petrol` and `Electric`
  - We have unified them to make a new custom type of `Car`
  - `tesla` is an instance of the union type

  ## Usage with functions

  Union type syntax is not limited to the declaration phase. You can also use them
  as parameters to functions or function return types:

  ```ts
  function generateCar(car: Petrol | Electric): void {
    console.log(`The car has a max speed of ${car.maxSpeed}`);
  }
  ```

  ## Unifying primitive and custom types

  We can also use unions with primitive types as well as custom types:

  ```ts
  function printId(id: number | string): void {
    console.log(`Your id is ${id}`);
  }
  ```

  We can also combine primitive and custom types in the same union:

  ```ts
  function printStringOrNumber(): string | Petrol {}
  ```

  ## Set theory: understanding errors

  In JavaScript we have the primitive types of string , number , boolean etc. In
  TS, these types are distinct and exclusive: if you declare that x : string you
  cannot redefine it as boolean or expect Boolean properties and methods to belong
  to a string type. For instance a string type is incapable of having as its value
  false.

  This is not true in the case of unions.

  With a union of `string | boolean` a value `x` could be `'banana'` or `true`. We
  are allowing for all values that are either from the set string or the set
  boolean. Logically this is equivalent to the OR operator: a value of type
  `string | boolean` could be one of infinite possible strings or true/false. With
  union types then, we radically widen the scope of possible values for the type.
  a

  These constraints don't just apply to the values themselves but the methods and
  properties you can use with the values. For instance a boolean doesn't have the
  property of `length`. Therefore if you have a union type of `string|boolean` you
  are going to get an error if you apply `length` to it since **the property must
  exist on both types that comprise the union**. The way round this is to use type
  narrowing and execute control flow based on the parameter that is passed to a
  function that is declared to be of this mixed type.

  Here is an example of this:

  ```ts
  function printId(id: number | string) {
    if (typeof id === "string") {
      // In this branch, id is of type 'string'
      console.log(id.toUpperCase());
    } else {
      // Here, id is of type 'number'
      console.log(id);
    }
  }
  ```

raw_content:
  ---
  tags:
    - typescript
  ---

  # Union types

  Like intersection types, union types allow us to combine the properties of
  existing primitive and custom types to make composites.

  Intersection types work well when you have complete knowledge of what form your
  data will take. This approach only works if you know in advance that your data
  will be and that it will share properties from more than one type. But what if
  you don't have this knowledge? You only know that it will be one of the three
  possible types? This is where union types become helpful. Instead of saying that
  event X will be of type A we say that event X may be one of types A or B or C.

  > A union type is a type formed from two or more other types, representing
  > values that may be any one of those types. We refer to each of these types as
  > the union’s members.

  ## Demonstration

  ```ts
  type Petrol = {
    maxSpeed: number;
    tankVolume: number; // unique property
  };

  type Electric = {
    maxSpeed: number;
    maxCharge: number; // unique property
  };

  type Car = Petrol | Electric;

  const tesla: Car = {
    maxSpeed: 120,
    maxCharge: 22,
  };
  ```

  - We have defined two custom types: `Petrol` and `Electric`
  - We have unified them to make a new custom type of `Car`
  - `tesla` is an instance of the union type

  ## Usage with functions

  Union type syntax is not limited to the declaration phase. You can also use them
  as parameters to functions or function return types:

  ```ts
  function generateCar(car: Petrol | Electric): void {
    console.log(`The car has a max speed of ${car.maxSpeed}`);
  }
  ```

  ## Unifying primitive and custom types

  We can also use unions with primitive types as well as custom types:

  ```ts
  function printId(id: number | string): void {
    console.log(`Your id is ${id}`);
  }
  ```

  We can also combine primitive and custom types in the same union:

  ```ts
  function printStringOrNumber(): string | Petrol {}
  ```

  ## Set theory: understanding errors

  In JavaScript we have the primitive types of string , number , boolean etc. In
  TS, these types are distinct and exclusive: if you declare that x : string you
  cannot redefine it as boolean or expect Boolean properties and methods to belong
  to a string type. For instance a string type is incapable of having as its value
  false.

  This is not true in the case of unions.

  With a union of `string | boolean` a value `x` could be `'banana'` or `true`. We
  are allowing for all values that are either from the set string or the set
  boolean. Logically this is equivalent to the OR operator: a value of type
  `string | boolean` could be one of infinite possible strings or true/false. With
  union types then, we radically widen the scope of possible values for the type.
  a

  These constraints don't just apply to the values themselves but the methods and
  properties you can use with the values. For instance a boolean doesn't have the
  property of `length`. Therefore if you have a union type of `string|boolean` you
  are going to get an error if you apply `length` to it since **the property must
  exist on both types that comprise the union**. The way round this is to use type
  narrowing and execute control flow based on the parameter that is passed to a
  function that is declared to be of this mixed type.

  Here is an example of this:

  ```ts
  function printId(id: number | string) {
    if (typeof id === "string") {
      // In this branch, id is of type 'string'
      console.log(id.toUpperCase());
    } else {
      // Here, id is of type 'number'
      console.log(id);
    }
  }
  ```


path: Unknown_type_in_TS.md
sortable_path: Unknown_type_in_TS.md
title: Unknown
checksum: 8c0f68fad261ac22cf13cf968e917e2308e19247798725d27b71e2be1a1a3da2
created: 2025-10-01 13:31:02.617468463+00:00
modified: 2025-08-13 15:57:13.872609663+00:00
metadata: {"tags":["typescript"]}
lead:
  You might think that a good use case for `any` is a scenario where you don't
  know in advance what the data type is going to be.

body:
  You might think that a good use case for `any` is a scenario where you don't
  know in advance what the data type is going to be.

  In order to mark that this is the case, you type the value as `any`. In fact,
  TypeScript provides a type for this specific scenario: `unknown`.

  Like `any`, `unknown` is equivalent to every type in TS (it is also a supertype)
  but it is deliberately inhibiting, in contrast to `any`. When you use `unknown`
  you have to use type narrowing to avoid it being returned. So if your code
  starts with `unknown`, you should filter out scenarios where it evaluates to
  each of the possible types otherwise if `unknown` is returned it will cause an
  error.

raw_content:
  ---
  tags:
    - typescript
  ---

  # Unknown

  You might think that a good use case for `any` is a scenario where you don't
  know in advance what the data type is going to be.

  In order to mark that this is the case, you type the value as `any`. In fact,
  TypeScript provides a type for this specific scenario: `unknown`.

  Like `any`, `unknown` is equivalent to every type in TS (it is also a supertype)
  but it is deliberately inhibiting, in contrast to `any`. When you use `unknown`
  you have to use type narrowing to avoid it being returned. So if your code
  starts with `unknown`, you should filter out scenarios where it evaluates to
  each of the possible types otherwise if `unknown` is returned it will cause an
  error.


path: Update_a_Mongo_document.md
sortable_path: Update_a_Mongo_document.md
title: Update a MongoDB document
lead: There are two methods for updating a document
checksum: 45e294432528520811c33744708746297fb71bfdc96726da610ef0146c820ba4
created: 2025-10-01 13:31:02.624881483+00:00
modified: 2025-08-13 15:57:14.057296653+00:00
metadata: {"tags":["mongo-db","node-js","mongoose","databases"]}
body:
  There are two methods for updating a document

  - query first
  - update first

  ## Query first document update

  With this approach we first execute a
  [query](Querying_a_Mongo_collection.md) to retrieve the document we
  want to edit and then make the change. We use the `findById` method to identify
  the document by its UUID and then `set` to update specified properties on the
  document. The `set` method is one of many operators that can be used to update
  values. For example there is also built-in operators for increment, renaming,
  multiplying values.

  Query first is best used when you want to secure the update with some prior
  logic or to validate. For example you may not want to update a course if it is
  listed as published. You could use a query to determine the publish status and
  then only update the entry if it returns `isPublished: false`.

  ```js
  async function updateCourseFromQuery(id) {
    const course = await Course.findById(id);
    if (!course) return;
    course.set({
      isPublished: true,
      author: "A new author",
    });

    // Instead of an object, we could also set the updated properties individually
    course.isPublished = true;
    course.author = "Biggs Volunksire";

    const result = course.save();
    console.log(result);
  }
  ```

  ## Update first document update

  With this approach we don't bother with a prior query. We are confident that the
  update is legitimate and do not need to first determine that certain conditions
  are met.

  To do this we directly use the `update` method, not `find`:

  ```js
  async function updateCourseFromQuery(id) {
    const result = await Course.update({ _id: id });
    $set: {
      // Invoke the set operator
      author: "Terry Nutile";
      isPublished: true;
    }
    console.log(result);
  }
  ```

  This function will just return some metadata about the update. It won't by
  default return the updated value. To do this use the `findByIdAndUpdate()`
  method instead of `update`:

  ```js
  async function updateCourseFromQuery(id) {
      const course = await Course.findByIdAndUpdate(id, {
      $set: { // Invoke the set operator
          author: 'Terry Nutile'
          isPublished: true
      }, {new: true}});
      console.log(result)
  }
  ```

  If we don't add `{new: true}`, it will return the document before the update.

  ### Updating multiple documents at once

  // Add: notes on this

raw_content:
  ---
  tags:
    - mongo-db
    - node-js
    - mongoose
    - databases
  ---

  # Update a MongoDB document

  There are two methods for updating a document

  - query first
  - update first

  ## Query first document update

  With this approach we first execute a
  [query](Querying_a_Mongo_collection.md) to retrieve the document we
  want to edit and then make the change. We use the `findById` method to identify
  the document by its UUID and then `set` to update specified properties on the
  document. The `set` method is one of many operators that can be used to update
  values. For example there is also built-in operators for increment, renaming,
  multiplying values.

  Query first is best used when you want to secure the update with some prior
  logic or to validate. For example you may not want to update a course if it is
  listed as published. You could use a query to determine the publish status and
  then only update the entry if it returns `isPublished: false`.

  ```js
  async function updateCourseFromQuery(id) {
    const course = await Course.findById(id);
    if (!course) return;
    course.set({
      isPublished: true,
      author: "A new author",
    });

    // Instead of an object, we could also set the updated properties individually
    course.isPublished = true;
    course.author = "Biggs Volunksire";

    const result = course.save();
    console.log(result);
  }
  ```

  ## Update first document update

  With this approach we don't bother with a prior query. We are confident that the
  update is legitimate and do not need to first determine that certain conditions
  are met.

  To do this we directly use the `update` method, not `find`:

  ```js
  async function updateCourseFromQuery(id) {
    const result = await Course.update({ _id: id });
    $set: {
      // Invoke the set operator
      author: "Terry Nutile";
      isPublished: true;
    }
    console.log(result);
  }
  ```

  This function will just return some metadata about the update. It won't by
  default return the updated value. To do this use the `findByIdAndUpdate()`
  method instead of `update`:

  ```js
  async function updateCourseFromQuery(id) {
      const course = await Course.findByIdAndUpdate(id, {
      $set: { // Invoke the set operator
          author: 'Terry Nutile'
          isPublished: true
      }, {new: true}});
      console.log(result)
  }
  ```

  If we don't add `{new: true}`, it will return the document before the update.

  ### Updating multiple documents at once

  // Add: notes on this


path: Update_an_SQL_table.md
sortable_path: Update_an_SQL_table.md
title: Updating an SQL table
checksum: 9645a802e36a0c8433bba7eea42e93ce70bebe748378d2896fd9a1f511bf77c5
created: 2025-10-01 13:31:02.638431524+00:00
modified: 2025-08-13 15:57:13.860092168+00:00
metadata: {"tags":["SQL","databases"]}
lead:
  We use the `ALTER` query to add, remove and otherwise change the structural
  properties of a table in the database.

body:
  We use the `ALTER` query to add, remove and otherwise change the structural
  properties of a table in the database.

  ## Add an additional field to existing table

  This adds a `price` field to the `sales` table. The `price` field accepts data
  of the type `real`.

  ```sql
  ALTER TABLE sales ADD price real;
  ```

raw_content:
  ---
  tags:
    - SQL
    - databases
  ---

  # Updating an SQL table

  We use the `ALTER` query to add, remove and otherwise change the structural
  properties of a table in the database.

  ## Add an additional field to existing table

  This adds a `price` field to the `sales` table. The `price` field accepts data
  of the type `real`.

  ```sql
  ALTER TABLE sales ADD price real;
  ```


path: Update_existing_data_in_SQL_table.md
sortable_path: Update_existing_data_in_SQL_table.md
title: Update existing data with the SQL UPDATE command
lead: We use `UPDATE` to modify existing records in our table.
checksum: 8d899c4df6b07aa260bf8286610be856b821f2fdc69aaebbd7557a866cf15cf5
created: 2025-10-01 13:31:02.645990026+00:00
modified: 2025-08-13 15:57:14.063555967+00:00
metadata: {"tags":["SQL","databases"]}
body:
  We use `UPDATE` to modify existing records in our table.

  ## Schematic syntax

  ```sql
  UPDATE [table_name]
  SET [field]
  WHERE [conditional expression/filter]
  ```

  ## Example

  ```sql
  UPDATE manufacturer
  SET url = '<http://www.hp.co.uk>'
  WHERE manufacturer_id = 4; --typically this will be the primary key as you are updating and existing record and need to identify it uniquely
  ```

  ## Update multiple fields

  ```sql
  UPDATE manufacturer
  SET url = '<http://www.apple.co.uk>',
      year_founded = 1977
  WHERE manufacturer_id = 2;
  ```

raw_content:
  ---
  tags:
    - SQL
    - databases
  ---

  # Update existing data with the SQL `UPDATE` command

  We use `UPDATE` to modify existing records in our table.

  ## Schematic syntax

  ```sql
  UPDATE [table_name]
  SET [field]
  WHERE [conditional expression/filter]
  ```

  ## Example

  ```sql
  UPDATE manufacturer
  SET url = '<http://www.hp.co.uk>'
  WHERE manufacturer_id = 4; --typically this will be the primary key as you are updating and existing record and need to identify it uniquely
  ```

  ## Update multiple fields

  ```sql
  UPDATE manufacturer
  SET url = '<http://www.apple.co.uk>',
      year_founded = 1977
  WHERE manufacturer_id = 2;
  ```


path: Useful_operators_in_SQL.md
sortable_path: Useful_operators_in_SQL.md
title: Useful operators in SQL
lead: ## True and false
checksum: 378ed8a4886205275eb46560a2a2acab4e6a008ca30aa641e5a9529d3500620c
created: 2025-10-01 13:31:02.651276996+00:00
modified: 2025-08-13 15:57:14.030544835+00:00
metadata: {"tags":["SQL","databases"]}
body:
  ## True and false

  > In SQL false results are represented by `0` and true results are represented
  > by `1`. Any zero value is false and any non-zero value is true.

  > When comparing operators we can use `SELECT` as a general "execute" function
  > for SQL to interpret

  ```sql
  SELECT 0 = 0 -- 1
  SELECT 0.0 = 0 -- 1
  SELECT 9 >= 12 -- 0
  ```

  ## Boolean operators

  SQL supports all Boolean operators:

  ```sql
  SELECT (9 > 7) AND (3 < 5) -- 1
  SELECT (3 > 1) OR (2 > 5) -- 1
  SELECT (3 > 1) XOR (5 > 1) -- 0
  ```

  We also have `IS` and `IS NOT` for testing Boolean values:

  ```sql
  SELECT (9 > 7) IS TRUE -- 1
  SELECT (9 > 7) IS NOT TRUE -- 0
  ```

  ## `NULL`

  - `NULL` is neither true (1) or false (0). It has no value.
  - An empty string (`""`) is **not** `NULL`

  ```sql
  SELECT 1 IS NULL -- 0
  SELECT 1 IS NOT NULL -- 1
  SELECT NULL IS NULL -- 1
  SELECT "" IS NULL -- 0
  ```

raw_content:
  ---
  tags:
    - SQL
    - databases
  ---

  # Useful operators in SQL

  ## True and false

  > In SQL false results are represented by `0` and true results are represented
  > by `1`. Any zero value is false and any non-zero value is true.

  > When comparing operators we can use `SELECT` as a general "execute" function
  > for SQL to interpret

  ```sql
  SELECT 0 = 0 -- 1
  SELECT 0.0 = 0 -- 1
  SELECT 9 >= 12 -- 0
  ```

  ## Boolean operators

  SQL supports all Boolean operators:

  ```sql
  SELECT (9 > 7) AND (3 < 5) -- 1
  SELECT (3 > 1) OR (2 > 5) -- 1
  SELECT (3 > 1) XOR (5 > 1) -- 0
  ```

  We also have `IS` and `IS NOT` for testing Boolean values:

  ```sql
  SELECT (9 > 7) IS TRUE -- 1
  SELECT (9 > 7) IS NOT TRUE -- 0
  ```

  ## `NULL`

  - `NULL` is neither true (1) or false (0). It has no value.
  - An empty string (`""`) is **not** `NULL`

  ```sql
  SELECT 1 IS NULL -- 0
  SELECT 1 IS NOT NULL -- 1
  SELECT NULL IS NULL -- 1
  SELECT "" IS NULL -- 0
  ```


path: Usenet.md
sortable_path: Usenet.md
title: Usenet
checksum: 2445b5be1ea783e6fd2700023528ec68df8054271bc568eedd5f0601ba5f3c69
created: 2025-10-01 13:31:02.65769347+00:00
modified: 2025-08-13 15:57:14.039417053+00:00
metadata: {"created":"Tuesday, November 12, 2024","tags":["networks","internet"]}
lead:
  A decentralised system of computer servers that allowed users to post messages
  to topic-based discussion groups called "newsgroups" which other users could
  read and reply to.

body:
  A decentralised system of computer servers that allowed users to post messages
  to topic-based discussion groups called "newsgroups" which other users could
  read and reply to.

  Arguably the first social network and the oldest internet network that is still
  operational.

  It was designed by staff at two separate universities. Both used Unix systems
  and wanted a way to communicate with each other.

  ## Technology

  Usenet comprised a [distributed network](385af4b4_Baran_distributed_networks.md)
  of dedicated servers that exchanged messages over the **Network News Transfer
  Protocol** (NNTP).

  A user would post a message to their local news server. The server would assign
  it a UUID and then share it with other peer servers that it was connected to,
  who would also share it, repeating the process. This created a ripple effect
  that would propagate the message over the global network. Because of the
  distributed nature of this network it meant that messages would have a high
  longevity and retention rate and that it was effectively impossible to delete a
  message once posted, since it would be stored across a multitude of servers.

  ## Relation to the Internet

  Usenet developed separately and in parallel with the Internet. Originally it did
  not operate over the Internet: servers would connect to each other over phone
  lines using the Unix-to-Unix Copy Protocol. In the late 1980s when the Internet
  became more widespread and not just limited to DARPA-adjacent universities,
  Usenet transitioned to running over
  [TCP/IP](Transport_Layer_of_Internet_Protocol.md) networks.

  ## Newsgroup hierarchy

  Usenet used a structured, dot-separated naming system for newsgroups that got
  more specific with each subdivision, e.g:

  - comp.sys.mac
  - sci.physics.particle
  - alt.music.nirvana

  Users would subscribe to the newsgroups that interested them and their
  newsreader (which connected to the servers) would only download messages from
  these groups.

  ## Influence

  - Created the concept of threaded discussions in its reply structure
  - Reddit appropriated the concept of topic-based communities in its sub-reddits
    and also used threading
  - RSS adopted Usenet's model of subscribing to specific information sources and
    getting updates: the concept of "pulling" new content from feeds mirrors how
    newsreaders worked
  - Distributed social networks like Mastodon/ActivityPub use a similar
    server-to-server propagation model that Usenet pioneered
  - [BitTorrent's](Torrenting.md) distributed file-sharing model has similarities
    to how binary files were distributed across Usenet servers

  ## "Eternal September"

  For most of its early history Usenet was limited to university students whose
  university was part of the network. Each September new students would join the
  university and this led to an influx of inexperienced Usenet users who didn't
  yet know the etiquette and culture. They would have to be schooled and inducted.

  In the mid-1980s Usenet became accessible to the general public as ISPs started
  granting access. In 1993-1994, AOL granted access to subscribers. This changed
  the nature of discussions from primarily academic and technical topics to
  include more general interest and entertainment content, leading to the creation
  of the "alt." hierarchy which allowed anyone to create new newsgroups without
  formal approval processes that were required in the traditional hierarchies.
  Hence it was "always September".

  This phrase is often used as a shorthand for the way in which the Internet and
  Web has moved from a niche community of technical users to a demotic public
  space with an attendant decline in perceived value.

raw_content:
  ---
  tags: [networks, internet]
  created: Tuesday, November 12, 2024
  ---

  # Usenet

  A decentralised system of computer servers that allowed users to post messages
  to topic-based discussion groups called "newsgroups" which other users could
  read and reply to.

  Arguably the first social network and the oldest internet network that is still
  operational.

  It was designed by staff at two separate universities. Both used Unix systems
  and wanted a way to communicate with each other.

  ## Technology

  Usenet comprised a [distributed network](385af4b4_Baran_distributed_networks.md)
  of dedicated servers that exchanged messages over the **Network News Transfer
  Protocol** (NNTP).

  A user would post a message to their local news server. The server would assign
  it a UUID and then share it with other peer servers that it was connected to,
  who would also share it, repeating the process. This created a ripple effect
  that would propagate the message over the global network. Because of the
  distributed nature of this network it meant that messages would have a high
  longevity and retention rate and that it was effectively impossible to delete a
  message once posted, since it would be stored across a multitude of servers.

  ## Relation to the Internet

  Usenet developed separately and in parallel with the Internet. Originally it did
  not operate over the Internet: servers would connect to each other over phone
  lines using the Unix-to-Unix Copy Protocol. In the late 1980s when the Internet
  became more widespread and not just limited to DARPA-adjacent universities,
  Usenet transitioned to running over
  [TCP/IP](Transport_Layer_of_Internet_Protocol.md) networks.

  ## Newsgroup hierarchy

  Usenet used a structured, dot-separated naming system for newsgroups that got
  more specific with each subdivision, e.g:

  - comp.sys.mac
  - sci.physics.particle
  - alt.music.nirvana

  Users would subscribe to the newsgroups that interested them and their
  newsreader (which connected to the servers) would only download messages from
  these groups.

  ## Influence

  - Created the concept of threaded discussions in its reply structure
  - Reddit appropriated the concept of topic-based communities in its sub-reddits
    and also used threading
  - RSS adopted Usenet's model of subscribing to specific information sources and
    getting updates: the concept of "pulling" new content from feeds mirrors how
    newsreaders worked
  - Distributed social networks like Mastodon/ActivityPub use a similar
    server-to-server propagation model that Usenet pioneered
  - [BitTorrent's](Torrenting.md) distributed file-sharing model has similarities
    to how binary files were distributed across Usenet servers

  ## "Eternal September"

  For most of its early history Usenet was limited to university students whose
  university was part of the network. Each September new students would join the
  university and this led to an influx of inexperienced Usenet users who didn't
  yet know the etiquette and culture. They would have to be schooled and inducted.

  In the mid-1980s Usenet became accessible to the general public as ISPs started
  granting access. In 1993-1994, AOL granted access to subscribers. This changed
  the nature of discussions from primarily academic and technical topics to
  include more general interest and entertainment content, leading to the creation
  of the "alt." hierarchy which allowed anyone to create new newsgroups without
  formal approval processes that were required in the traditional hierarchies.
  Hence it was "always September".

  This phrase is often used as a shorthand for the way in which the Internet and
  Web has moved from a niche community of technical users to a demotic public
  space with an attendant decline in perceived value.


path: User_Space.md
sortable_path: User_Space.md
title: User space
lead:
  User space is the portion of the main memory that the kernel allocates for user
  processes.

checksum: 9fda30f69fca66abe1243711ac15679517d775e629de2c0d9175dff66ed21cc5
created: 2025-10-01 13:31:02.694030365+00:00
modified: 2025-08-13 15:57:13.821085319+00:00
metadata: {"tags":["operating-systems"]}
body:
  User space is the portion of the main memory that the kernel allocates for user
  processes.

  There is a hierarchy within the user space with several different groups of user
  processes:

  <dl>

  <dt>Basic services</dt>
  <dd>Run at the bottom level and therefore are closest to ther kernel. Comprises small components that perform single, uncomplicated tasks. Examples include network configuration, the communication bus, and diagnostic logging</dd> 
  <dt>Utiility services</dt>
  <dd>The middle level: larger components such as mail, print, and database services</dd>
  <dt>Services controllable by the user</dt>
  <dd>The top level: complicated tasks that the user controls directly such as using a web-browser, editing a text file</dd>
  </dl>

  ## What is a user?

  > A user is an entity that can run processes and own files.

  At the level of users, a user is associated with a user name however the kernel
  does not manage usernames, it identifies users via numeric identifiers called
  **user IDs**.

  Users exist in order to support permissions and access boundaries. A user may
  terminate or modify the behaviour of its own processes (within limits) but
  cannot interfere with another user's proceses. In addition, users may own files
  and choose whether to share them with other users.

  ### Root user (superuser)

  A Linux system normally has a number of users in addition to the ones that
  correspond to the real human users. The most important is **root**. The root
  user _can_ terminate and alter another user's processes and access any file on
  the local system.

  A person who can operate as root is an administrator on a typical user system.

raw_content:
  ---
  tags:
    - operating-systems
  ---

  # User space

  User space is the portion of the main memory that the kernel allocates for user
  processes.

  There is a hierarchy within the user space with several different groups of user
  processes:

  <dl>

  <dt>Basic services</dt>
  <dd>Run at the bottom level and therefore are closest to ther kernel. Comprises small components that perform single, uncomplicated tasks. Examples include network configuration, the communication bus, and diagnostic logging</dd> 
  <dt>Utiility services</dt>
  <dd>The middle level: larger components such as mail, print, and database services</dd>
  <dt>Services controllable by the user</dt>
  <dd>The top level: complicated tasks that the user controls directly such as using a web-browser, editing a text file</dd>
  </dl>

  ## What is a user?

  > A user is an entity that can run processes and own files.

  At the level of users, a user is associated with a user name however the kernel
  does not manage usernames, it identifies users via numeric identifiers called
  **user IDs**.

  Users exist in order to support permissions and access boundaries. A user may
  terminate or modify the behaviour of its own processes (within limits) but
  cannot interfere with another user's proceses. In addition, users may own files
  and choose whether to share them with other users.

  ### Root user (superuser)

  A Linux system normally has a number of users in addition to the ones that
  correspond to the real human users. The most important is **root**. The root
  user _can_ terminate and alter another user's processes and access any file on
  the local system.

  A person who can operate as root is an administrator on a typical user system.


path: User_agent.md
sortable_path: User_agent.md
title: User agent
checksum: 8ebaa79e8318ce51042596b2b34faeec781367002c93c8f5a8007a60e6c8a553
created: 2025-10-01 13:31:02.702705395+00:00
modified: 2025-08-13 15:57:13.884685209+00:00
metadata: {"created":"Friday, December 20, 2024","tags":["world-wide-web"]}
lead:
  _User agent_ is the technical term for a web browser. So named because it acts
  on behalf of the user.

body:
  _User agent_ is the technical term for a web browser. So named because it acts
  on behalf of the user.

  When a browser makes a request to a web server it includes a header value called
  `User-Agent` that describes the browser.

raw_content:
  ---
  tags: [world-wide-web]
  created: Friday, December 20, 2024
  ---

  # User agent

  _User agent_ is the technical term for a web browser. So named because it acts
  on behalf of the user.

  When a browser makes a request to a web server it includes a header value called
  `User-Agent` that describes the browser.


path: User_management_in_Linux.md
sortable_path: User_management_in_Linux.md
title: User management
lead: ## Switch user
checksum: e5c96a6157409e2d25e0f6278475b4bb97323774450952af78f3ad3ea1369622
created: 2025-10-01 13:31:02.71108316+00:00
modified: 2025-08-13 15:57:13.865925081+00:00
metadata: {"tags":["Linux","procedural"]}
body:
  ## Switch user

  If already logged in as a user you can switch users with the command
  `su - [username]`.

  ## Login as root

  If you are logged in as a standard user, if you use `su -` without specifying a
  username, then it will assume you wish to log in as root.

  If you wish to login as root in the tty at startup, then use `root` as the
  username.

raw_content:
  ---
  tags:
    - Linux
    - procedural
  ---

  # User management

  ## Switch user

  If already logged in as a user you can switch users with the command
  `su - [username]`.

  ## Login as root

  If you are logged in as a standard user, if you use `su -` without specifying a
  username, then it will assume you wish to log in as root.

  If you wish to login as root in the tty at startup, then use `root` as the
  username.


path: User_management_on_AWS.md
sortable_path: User_management_on_AWS.md
title: User management and roles
lead: ## IAM
checksum: 173c1b3adbc04dfc09924b34e135e231e739c359cede8053c343a03bc5c2f4a8
created: 2025-10-01 13:31:02.717617162+00:00
modified: 2025-08-13 15:57:14.066364846+00:00
metadata: {"tags":["AWS"]}
body:
  ## IAM

  > Identity and Access Management

  An IAM user is an identity with long-term credentials that is used to interact
  with AWS in an account. An IAM login is different from a root login. The root
  user is the one who creates the AWS account and who has the power to delete it.
  The root user can create IAM logins so that other users can log in through
  dedicated IAM portals. The root user should also create an IAM login for himself
  and use that, set to administrator policies, for most activities, reserving root
  for the most consequential stuff like adding users etc.

  ## Users and user groups

  - Use to create users and access rights accross the AWS services
  - Users get permissions through policies (use template or create your own)
  - You can also create user groups and add users to them to simplify role
    attribution

  ## Roles

  Roles are adopted by _services_ rather than users. They run either periodically
  (from a `cronjob` say) or when they are hit from an API.

  This allows you to assign policies to a role that a service can adopt when
  running. A good example is a lambda. Lambdas often need access to other services
  such as an S3 bucket or a DynamoDB table. Say you have a service that combines a
  lambda with a DynamoDB database. You could assign a role to the lambda and it
  would have access to the database.

  ## Distinction between _principal_ and _identity_

  Both “principal” and “identity” refer to entities that can perform actions and
  interact with AWS resources. However, there is a subtle difference in their
  usage:

  > a principal is a specific type of entity that can take actions in AWS, while
  > an identity is the unique identifier associated with that principal.

  1. Principal: In the context of IAM policies, a principal represents the entity
     that is allowed or denied access to AWS rThe principal is specified in the
     policy statement as the entity to which the permissions are granted or
     deniedesources. It can be an IAM user, an IAM role, an AWS service. The
     principal is specified in the policy statement as the entity to which the
     permissions are granted or denied.
  2. Identity: An identity, on the other hand, is a broader term that encompasses
     both the principal and the authentication credentials associated with that
     principal. It refers to the entity’s unique identifier, such as an IAM user’s
     username or an IAM role’s ARN (Amazon Resource Name). An identity is used for
     authentication purposes to verify the entity’s identity and determine its
     permissions.

  ## Cognito

  > Amazon Cognito provides authentication, authorization, and user management for
  > your web and mobile apps.

  There are **user pools** and **identity pools**.

  User pools provide sign-up and sign-in options for your app users. Identity
  pools provide AWS credentials to grant your users access to other AWS services.

  You would use identity pools to manage access to API endpoints or S3 buckets.
  They can be authenticated or unauthenticated. For example if you want to use an
  image from S3 in a website you would authenticate for that, but you would
  explicitly have to specify that it is general access by applying the policy
  `AmazonS3ReadOnlyAccess`.

raw_content:
  ---
  tags:
    - AWS
  ---

  # User management and roles

  ## IAM

  > Identity and Access Management

  An IAM user is an identity with long-term credentials that is used to interact
  with AWS in an account. An IAM login is different from a root login. The root
  user is the one who creates the AWS account and who has the power to delete it.
  The root user can create IAM logins so that other users can log in through
  dedicated IAM portals. The root user should also create an IAM login for himself
  and use that, set to administrator policies, for most activities, reserving root
  for the most consequential stuff like adding users etc.

  ## Users and user groups

  - Use to create users and access rights accross the AWS services
  - Users get permissions through policies (use template or create your own)
  - You can also create user groups and add users to them to simplify role
    attribution

  ## Roles

  Roles are adopted by _services_ rather than users. They run either periodically
  (from a `cronjob` say) or when they are hit from an API.

  This allows you to assign policies to a role that a service can adopt when
  running. A good example is a lambda. Lambdas often need access to other services
  such as an S3 bucket or a DynamoDB table. Say you have a service that combines a
  lambda with a DynamoDB database. You could assign a role to the lambda and it
  would have access to the database.

  ## Distinction between _principal_ and _identity_

  Both “principal” and “identity” refer to entities that can perform actions and
  interact with AWS resources. However, there is a subtle difference in their
  usage:

  > a principal is a specific type of entity that can take actions in AWS, while
  > an identity is the unique identifier associated with that principal.

  1. Principal: In the context of IAM policies, a principal represents the entity
     that is allowed or denied access to AWS rThe principal is specified in the
     policy statement as the entity to which the permissions are granted or
     deniedesources. It can be an IAM user, an IAM role, an AWS service. The
     principal is specified in the policy statement as the entity to which the
     permissions are granted or denied.
  2. Identity: An identity, on the other hand, is a broader term that encompasses
     both the principal and the authentication credentials associated with that
     principal. It refers to the entity’s unique identifier, such as an IAM user’s
     username or an IAM role’s ARN (Amazon Resource Name). An identity is used for
     authentication purposes to verify the entity’s identity and determine its
     permissions.

  ## Cognito

  > Amazon Cognito provides authentication, authorization, and user management for
  > your web and mobile apps.

  There are **user pools** and **identity pools**.

  User pools provide sign-up and sign-in options for your app users. Identity
  pools provide AWS credentials to grant your users access to other AWS services.

  You would use identity pools to manage access to API endpoints or S3 buckets.
  They can be authenticated or unauthenticated. For example if you want to use an
  image from S3 in a website you would authenticate for that, but you would
  explicitly have to specify that it is general access by applying the policy
  `AmazonS3ReadOnlyAccess`.


path: Using_GraphQL_with_Node.md
sortable_path: Using_GraphQL_with_Node.md
title: Using GraphQL with Node.js
lead: ## Create a basic Express server
checksum: 685307a804f6e41606f8a1dacebf338a95dd7476b1496eced0506b8f73f5f9df
created: 2025-10-01 13:31:02.724316832+00:00
modified: 2025-08-13 15:57:13.967715796+00:00
metadata: {"tags":["graphql","node-js"]}
body:
  ## Create a basic Express server

  First we create a basic server in Node using Express:

  ```js
  import express from "express";

  app.get("/", (req, res) => {
    res.send("Graph QL test project");
  });

  app.listen(8080, () =>
    console.log("Running server on port localhost:8080/graphql")
  );
  ```

  ## Add GraphQL as middlewear

  Next we introduce GraphQL as a piece of Node.js
  [middlewear](Middleware_in_NodeJS.md), with the
  `app.use()` method.

  ```js
  import { graphqlHTTP } from "express-graphql";

  app.use(
    "/graphql",
    graphqlHTTP({
      schema: schema,
      rootValue: resolvers,
      graphiql: true,
    })
  );
  ```

  - `schema` is a reference to our GraphQL schema - the structure of the fields
    that define our server.
  - `rootValue` is a reference to our resolvers.
  - `graphiql` is the GUI tool that will be served from the GraphQL endpoint at
    `localhost:8080/graphql`. This tool enables us to interrogate our data using
    the defined schema and see what data we would get back from frontend queries.

raw_content:
  ---
  tags: [graphql, node-js]
  ---

  # Using GraphQL with Node.js

  ## Create a basic Express server

  First we create a basic server in Node using Express:

  ```js
  import express from "express";

  app.get("/", (req, res) => {
    res.send("Graph QL test project");
  });

  app.listen(8080, () =>
    console.log("Running server on port localhost:8080/graphql")
  );
  ```

  ## Add GraphQL as middlewear

  Next we introduce GraphQL as a piece of Node.js
  [middlewear](Middleware_in_NodeJS.md), with the
  `app.use()` method.

  ```js
  import { graphqlHTTP } from "express-graphql";

  app.use(
    "/graphql",
    graphqlHTTP({
      schema: schema,
      rootValue: resolvers,
      graphiql: true,
    })
  );
  ```

  - `schema` is a reference to our GraphQL schema - the structure of the fields
    that define our server.
  - `rootValue` is a reference to our resolvers.
  - `graphiql` is the GUI tool that will be served from the GraphQL endpoint at
    `localhost:8080/graphql`. This tool enables us to interrogate our data using
    the defined schema and see what data we would get back from frontend queries.


path: Using_SQLite_with_Python.md
sortable_path: Using_SQLite_with_Python.md
title: Using SQLite with Python
lead: SQLite is part of the core library, you do not require a separate package
checksum: 03ce3deea72df42fd5f2a60f72b37b791107c9ddc87f97b87fa48674925c6750
created: 2025-10-01 13:31:02.736593354+00:00
modified: 2025-08-13 15:57:13.7612834+00:00
metadata: {"created":"Thursday, November 07, 2024","tags":["python","databases","SQL","SQLite"]}
body:
  SQLite is part of the core library, you do not require a separate package

  ```py
  import sqlite3
  ```

  ## Connect to database

  Connect to an existing SQLite `.db` file or create a new database with the same
  command:

  ```py
  connection = sqlite3.connect("my_database.db")
  ```

  ## Disconnect from database

  ```python
  connection.close()
  ```

  ## Querying

  `connect` exposes a `cursor` entity. This is used for executing changes against
  the database file.

  ### Example: creating a table

  ```python
  CREATE_TAGS_TABLE = """
  CREATE TABLE IF NOT EXISTS tags (
      name TEXT PRIMARY KEY
  )
  """

  cursor = connection.cursor()
  cursor.execute(CREATE_TAGS_TABLE)
  cursor.commit()

  # Confirm changes

  cursor.execute("SELECT name from sqlite_master WHERE type='table'")
  print(cursor.fetchall())
  # (tags)
  ```

  Note that it is necessary to `commit` the changes. The cursor object just stores
  the operation in memory and does not execute until you specifically commit them.

  ## Prepared statements and parameterised queries

  ### Parameterised queries

  ```py
  username = "thomas"
  password = "123"
  cursor.execute("SELECT * FROM users WHERE username = ? AND password = ?",
              (username, password))
  ```

  Alternatively, pass the [tuple](Tuples_in_Python.md) directly:

  ```python
  login_data = ("thomas", "123")
  cursor.execute("SELECT * FROM users WHERE username = ? AND password = ?",
  login_data)
  ```

  ### Prepared statements

  > With prepared statements you have to pass in the parameters as a dictionary

  ```python
  user_data = {
      "username": "thomas",
      "email": "thomas@email.com"
  }

  cursor.execute("INSERT INTO users (username, email) VALUES (:username, :email)",
                 user_data)
  ```

  In the example above it so happens that we already have a dictionary we are
  working with. If you want to use a prepared statement and you don't have a
  dictionary already, you will need to construct one as part of the query, e.g:

  ```python
  cursor.execute("INSERT INTO table_name (value_one, value_two) VALUES (:value_one, :value_two)", { value_one: 'foo', value_two: 'bar'})
  ```

raw_content:
  ---
  tags: [python, databases, SQL, SQLite]
  created: Thursday, November 07, 2024
  ---

  # Using SQLite with Python

  SQLite is part of the core library, you do not require a separate package

  ```py
  import sqlite3
  ```

  ## Connect to database

  Connect to an existing SQLite `.db` file or create a new database with the same
  command:

  ```py
  connection = sqlite3.connect("my_database.db")
  ```

  ## Disconnect from database

  ```python
  connection.close()
  ```

  ## Querying

  `connect` exposes a `cursor` entity. This is used for executing changes against
  the database file.

  ### Example: creating a table

  ```python
  CREATE_TAGS_TABLE = """
  CREATE TABLE IF NOT EXISTS tags (
      name TEXT PRIMARY KEY
  )
  """

  cursor = connection.cursor()
  cursor.execute(CREATE_TAGS_TABLE)
  cursor.commit()

  # Confirm changes

  cursor.execute("SELECT name from sqlite_master WHERE type='table'")
  print(cursor.fetchall())
  # (tags)
  ```

  Note that it is necessary to `commit` the changes. The cursor object just stores
  the operation in memory and does not execute until you specifically commit them.

  ## Prepared statements and parameterised queries

  ### Parameterised queries

  ```py
  username = "thomas"
  password = "123"
  cursor.execute("SELECT * FROM users WHERE username = ? AND password = ?",
              (username, password))
  ```

  Alternatively, pass the [tuple](Tuples_in_Python.md) directly:

  ```python
  login_data = ("thomas", "123")
  cursor.execute("SELECT * FROM users WHERE username = ? AND password = ?",
  login_data)
  ```

  ### Prepared statements

  > With prepared statements you have to pass in the parameters as a dictionary

  ```python
  user_data = {
      "username": "thomas",
      "email": "thomas@email.com"
  }

  cursor.execute("INSERT INTO users (username, email) VALUES (:username, :email)",
                 user_data)
  ```

  In the example above it so happens that we already have a dictionary we are
  working with. If you want to use a prepared statement and you don't have a
  dictionary already, you will need to construct one as part of the query, e.g:

  ```python
  cursor.execute("INSERT INTO table_name (value_one, value_two) VALUES (:value_one, :value_two)", { value_one: 'foo', value_two: 'bar'})
  ```


path: Using_a_context_manager_in_Python.md
sortable_path: Using_a_context_manager_in_Python.md
title: Using a context manager in Python
checksum: a58b2ac56703de9c1998f08af72bdce53d72146fa6809c27427143c7caa0ddac
created: 2025-10-01 13:31:02.748485991+00:00
modified: 2025-08-13 15:57:13.938004112+00:00
metadata: {"created":"Sunday, March 16, 2025","tags":["python"]}
lead:
  For certain process that have a start and an end state such as opening and
  closing a file or connecting and disconnecting from a database, instead of
  having dedicated open/close, connect/disconnect handling, you can use a Context
  Manager as a form of "syntactic sugar".

body:
  For certain process that have a start and an end state such as opening and
  closing a file or connecting and disconnecting from a database, instead of
  having dedicated open/close, connect/disconnect handling, you can use a Context
  Manager as a form of "syntactic sugar".

  By using a Context Manager you can use a more Pythonic construction of e.g:

  ```py
  with open('/file-path') as file:
      ... # handle file operation
  ```

  ```py
  with open DatabaseService() as connection:
      ... # do database stuff
  ```

  When you use this approach it is an abstraction over `try...finally`. Meaning
  the clean-up `finally` operation will run automatically without you having to
  explicitly handle it.

  Some common processes such as file handling have the Context Manager built-in
  and you don't have to explicitly provision it.

  Other processes may lend themselves to `with` syntax but you have to do the
  configuration yourself.

  One such example is creating a database accessor. Below I have done this by
  using `__enter__` and `__exit__` methods on a database class:

  ```py
  class DatabaseService:
      def __init__(self, db_name, db_path):
          self.db_name = db_name
          self.db_path = db_path
          self.connection: Optional[sqlite3.Connection] = None

      def connect(self) -> Optional[sqlite3.Connection]:
          if self.connection is not None:
              return self.connection

          try:
              if not os.path.exists(self.db_path):
                  os.makedirs(self.db_path)
                  print("INFO Created database directory")
              self.connection = sqlite3.connect(f"{self.db_path}/{self.db_name}.db")
              self.connection.execute("PRAGMA foreign_keys = ON")
              return self.connection

          except Exception as e:
              raise Exception(f"ERROR Problem connecting to database: {e}")

      def disconnect(self) -> None:
          try:
              if self.connection is not None:
                  self.connection.close()
                  self.connection = None
          except Exception as e:
              raise Exception(f"ERROR Problem disconnecting from database: {e}")

      def __enter__(self) -> sqlite3.Connection:
          connection = self.connect()
          if connection is None:
              raise RuntimeError("Failed to establish database connection")
          return connection

      def __exit__(self) -> None:
          self.disconnect()
  ```

  Then I can use it like so:

  ```py
  with DatabaseService() as connection:
      cursor = connection.cursor()
      cursor.execute(...)
  ```

raw_content:
  ---
  tags: [python]
  created: Sunday, March 16, 2025
  ---

  # Using a context manager in Python

  For certain process that have a start and an end state such as opening and
  closing a file or connecting and disconnecting from a database, instead of
  having dedicated open/close, connect/disconnect handling, you can use a Context
  Manager as a form of "syntactic sugar".

  By using a Context Manager you can use a more Pythonic construction of e.g:

  ```py
  with open('/file-path') as file:
      ... # handle file operation
  ```

  ```py
  with open DatabaseService() as connection:
      ... # do database stuff
  ```

  When you use this approach it is an abstraction over `try...finally`. Meaning
  the clean-up `finally` operation will run automatically without you having to
  explicitly handle it.

  Some common processes such as file handling have the Context Manager built-in
  and you don't have to explicitly provision it.

  Other processes may lend themselves to `with` syntax but you have to do the
  configuration yourself.

  One such example is creating a database accessor. Below I have done this by
  using `__enter__` and `__exit__` methods on a database class:

  ```py
  class DatabaseService:
      def __init__(self, db_name, db_path):
          self.db_name = db_name
          self.db_path = db_path
          self.connection: Optional[sqlite3.Connection] = None

      def connect(self) -> Optional[sqlite3.Connection]:
          if self.connection is not None:
              return self.connection

          try:
              if not os.path.exists(self.db_path):
                  os.makedirs(self.db_path)
                  print("INFO Created database directory")
              self.connection = sqlite3.connect(f"{self.db_path}/{self.db_name}.db")
              self.connection.execute("PRAGMA foreign_keys = ON")
              return self.connection

          except Exception as e:
              raise Exception(f"ERROR Problem connecting to database: {e}")

      def disconnect(self) -> None:
          try:
              if self.connection is not None:
                  self.connection.close()
                  self.connection = None
          except Exception as e:
              raise Exception(f"ERROR Problem disconnecting from database: {e}")

      def __enter__(self) -> sqlite3.Connection:
          connection = self.connect()
          if connection is None:
              raise RuntimeError("Failed to establish database connection")
          return connection

      def __exit__(self) -> None:
          self.disconnect()
  ```

  Then I can use it like so:

  ```py
  with DatabaseService() as connection:
      cursor = connection.cursor()
      cursor.execute(...)
  ```


path: Using_arguments_with_Apollo_Client.md
sortable_path: Using_arguments_with_Apollo_Client.md
title: Using arguments with Apollo Client
lead: ## What are arguments?
checksum: 3d64509966807a581a0b74dcb400372cbc2441e505fe91173b05d7fed2473ed1
created: 2025-10-01 13:31:02.753678876+00:00
modified: 2025-08-13 15:57:13.83573845+00:00
metadata: {"tags":["graphql"]}
body:
  ## What are arguments?

  - An argument is a value you provide for a particular field in a query.
  - The schema must define the arguments that a field accepts (if any)
  - The resolver invokes a field's provided arguments to determine how to return
    specific data
  - Some example use cases of arguments:
    - retrieve specific objects
    - filter through a set of objects
    - transform a field's returned value

  ## Updated schema

  In order to demonstrate arguments we need to expand the
  [original schema](Apollo_Server.md#example-schema).

  Remember a Track is a group of modules that teaches about a specific topic. We
  are going to add:

  - `description` and `numberOfViews` fields to the original `Track` type
  - A new `Module` type
  - A field `modules` to the `Track` type that will be an array of type `Module`

  ```js
  type Query {
    tracksForHome: [Track!]!
  }

  type Track {
      id: ID!
          authorId: String! // They added this on the sly
      thumbnail: String
      length: Int
      modulesCount: Int
      description: String
      numberOfViews: Int
      modules: [Module!]!
  }

  type Module {
      id: ID!
          length: Int
  }

  type Author {
      id: ID!
      name: String!
      photo: String
  }
  ```

  ## Adding additional query

  > Currently we only have one query in the schema (`tracksForHome`) and this
  > returns an array of all the tracks. To demonstrate arguments we want to return
  > a specific track by its `id`. We are going to add a query that enables this.

  ```js
  type Query {
      track(id: ID!):  Track
  }
  ```

  This query will take an `id` as an argument and the resolver will match the `id`
  to a given track and return that `Track`.

  To define an argument for a field in the schema we add parentheses after the
  field name. In the parentheses we write the name of the argument and its type.
  If we have more than one argument we can separate them with commas.

  ## Create resolver for new query

  Now we have to create a resolver for our new `track` query. We will quickly run
  through the [server-side process](Apollo_Server.md).

  ### Update our `RESTDataSource`

  ```js
  class TrackAPI extends RESTDataSource {
    constructor() {...}
    getTracksForHome() {...}
    getAuthor(authorId) {...}

    getTrack(trackId){
      return this.get(`track/${trackId}`);
    }
  }

  ```

  ### Add resolver to `resolvers.js`

  ```js
  const resolvers = {
    Query: {
      tracksForHome: (_, __, { dataSources }) => {
        return dataSources.trackApi.getTracksForHome();
      },
      track: (_, { id }, { dataSources }) => {
        return dataSources.trackAPI.getTrack(id);
      },
    },
    Track: {
      author: ({ authorId }, _, { dataSources }) => {
        return dataSources.trackApi.getAuthor(authorId);
      },
    },
  };
  ```

  ### Making a query

  With the server changes complete, we can now issue a query with an argument from
  the client:

  ```js
  query track(id: 'xyz'){
    title
  }
  ```

  This will return the `title` field from the track with the specific id. This
  query complies with the schema since the `Track` type has the field `title`

  ### Resolver chains

  What about the following query:

  ```js
  query track(id: 'xyz'){
    title
    author {
      name
    }
  }
  ```

  It's not obvious how the resolver should respond to the nested query here since
  author name is not a field on `Track`, it is a field on `Author`:

  ```js
  type Track {
    id: ID!
      authorId: String!
    thumbnail: String
    length: Int
    modulesCount: Int
    description: String
    numberOfViews: Int
    modules: [Module!]!
  }

  type Author {
    id: ID!
    name: String!
    photo: String
  }
  ```

  This is where we use a **resolver chain**. Because `authorId` exists on `Track`
  we can use this to get the `name` part of the query. We already have a resolver
  for `author` under the `Track` resolver:

  ```js
  Track: {
      author: ({ authorId }, _, { dataSources }) => {
        return dataSources.trackApi.getAuthor(authorId);
      },
    },
  ```

  Notice that `authorId` is used in the place of the `parent` parameter. It
  already exists on the `Track` type that wraps the resolver. So this can be
  invoked to fulfill `author` and thereby access the author name from the graph.

  This process is also required for our extended schema. The `Track` type now has
  a `modules` field that comprises an array of the `Module` type.

  Here's a reminder of the `Module` type:

  ```js
  type Module {
      id: ID!
          length: Int
  }
  ```

  The modules for a `Track` can be sourced from the track `id`. So we can again
  add a resolver chain function to the resolvers file:

  ```js
  const resolvers = {
    Query: {
      tracksForHome: (_, __, { dataSources }) => {
        return dataSources.trackApi.getTracksForHome();
      },
      track: (_, { id }, { dataSources }) => {
        return dataSources.trackAPI.getTrack(id);
      },
    },
    Track: {
      author: ({ authorId }, _, { dataSources }) => {
        return dataSources.trackApi.getAuthor(authorId);
      },
      modules: ({ id }, _, { dataSources }) => {
        return dataSources.trackAPI.getTrackModules(id);
      },
    },
  };
  ```

  Once again we are using a property on the parent `Track` type to use as a
  parameter in the resolver function.

  This setup would enables like the following to be run:

  ```js
  query track(id: 'xyz'){
    title
    modules
  }
  ```

  The `id` parameter would be used by the `modules` resolver to return the array
  of the `Modules` type.

  ## Variables in arguments

  Instead of writing the following within our query constants on the client-side:

  ```js
  query GetTrack {
    track(id: 'xyz'){
      title
    }
  }
  ```

  We can make the code more reusable by using variables instead of the hardcoded
  `id` argument:

  ```js
  query GetTrack($trackId: ID!) {
    track(id: $trackId){
      title
    }
  }
  ```

  This way we do not need to write a new query constant every time we want to
  return a specific track.

  ## Send query with arguments using Apollo `useQuery`

  We can now write a proper query using the
  [useQuery hook](Apollo_Client.md#usequery-hook) from
  Apollo Client, with variables.

  First define our query constant:

  ```js
  export const GET_TRACK = gql`
    query GetTrack($trackId: ID!) {
      track(id: $trackId) {
        id
        title
        author {
          name
        }
        description
        modules {
          id
          title
          length
        }
      }
    }
  `;
  ```

  Then to employ in React:

  ```jsx
  const trackId = "xyz";

  const { loading, error, data } = useQuery(GET_TRACK, {
    variables: {
      id: trackId,
    },
  });
  ```

  Note that in contrast to the
  [simple example](Apollo_Client.md#query-constants)
  because we are using variables, we have to pass-in an additional options object
  with the query constant that specifies our variables.

raw_content:
  ---
  tags: [graphql]
  ---

  # Using arguments with Apollo Client

  ## What are arguments?

  - An argument is a value you provide for a particular field in a query.
  - The schema must define the arguments that a field accepts (if any)
  - The resolver invokes a field's provided arguments to determine how to return
    specific data
  - Some example use cases of arguments:
    - retrieve specific objects
    - filter through a set of objects
    - transform a field's returned value

  ## Updated schema

  In order to demonstrate arguments we need to expand the
  [original schema](Apollo_Server.md#example-schema).

  Remember a Track is a group of modules that teaches about a specific topic. We
  are going to add:

  - `description` and `numberOfViews` fields to the original `Track` type
  - A new `Module` type
  - A field `modules` to the `Track` type that will be an array of type `Module`

  ```js
  type Query {
    tracksForHome: [Track!]!
  }

  type Track {
      id: ID!
          authorId: String! // They added this on the sly
      thumbnail: String
      length: Int
      modulesCount: Int
      description: String
      numberOfViews: Int
      modules: [Module!]!
  }

  type Module {
      id: ID!
          length: Int
  }

  type Author {
      id: ID!
      name: String!
      photo: String
  }
  ```

  ## Adding additional query

  > Currently we only have one query in the schema (`tracksForHome`) and this
  > returns an array of all the tracks. To demonstrate arguments we want to return
  > a specific track by its `id`. We are going to add a query that enables this.

  ```js
  type Query {
      track(id: ID!):  Track
  }
  ```

  This query will take an `id` as an argument and the resolver will match the `id`
  to a given track and return that `Track`.

  To define an argument for a field in the schema we add parentheses after the
  field name. In the parentheses we write the name of the argument and its type.
  If we have more than one argument we can separate them with commas.

  ## Create resolver for new query

  Now we have to create a resolver for our new `track` query. We will quickly run
  through the [server-side process](Apollo_Server.md).

  ### Update our `RESTDataSource`

  ```js
  class TrackAPI extends RESTDataSource {
    constructor() {...}
    getTracksForHome() {...}
    getAuthor(authorId) {...}

    getTrack(trackId){
      return this.get(`track/${trackId}`);
    }
  }

  ```

  ### Add resolver to `resolvers.js`

  ```js
  const resolvers = {
    Query: {
      tracksForHome: (_, __, { dataSources }) => {
        return dataSources.trackApi.getTracksForHome();
      },
      track: (_, { id }, { dataSources }) => {
        return dataSources.trackAPI.getTrack(id);
      },
    },
    Track: {
      author: ({ authorId }, _, { dataSources }) => {
        return dataSources.trackApi.getAuthor(authorId);
      },
    },
  };
  ```

  ### Making a query

  With the server changes complete, we can now issue a query with an argument from
  the client:

  ```js
  query track(id: 'xyz'){
    title
  }
  ```

  This will return the `title` field from the track with the specific id. This
  query complies with the schema since the `Track` type has the field `title`

  ### Resolver chains

  What about the following query:

  ```js
  query track(id: 'xyz'){
    title
    author {
      name
    }
  }
  ```

  It's not obvious how the resolver should respond to the nested query here since
  author name is not a field on `Track`, it is a field on `Author`:

  ```js
  type Track {
    id: ID!
      authorId: String!
    thumbnail: String
    length: Int
    modulesCount: Int
    description: String
    numberOfViews: Int
    modules: [Module!]!
  }

  type Author {
    id: ID!
    name: String!
    photo: String
  }
  ```

  This is where we use a **resolver chain**. Because `authorId` exists on `Track`
  we can use this to get the `name` part of the query. We already have a resolver
  for `author` under the `Track` resolver:

  ```js
  Track: {
      author: ({ authorId }, _, { dataSources }) => {
        return dataSources.trackApi.getAuthor(authorId);
      },
    },
  ```

  Notice that `authorId` is used in the place of the `parent` parameter. It
  already exists on the `Track` type that wraps the resolver. So this can be
  invoked to fulfill `author` and thereby access the author name from the graph.

  This process is also required for our extended schema. The `Track` type now has
  a `modules` field that comprises an array of the `Module` type.

  Here's a reminder of the `Module` type:

  ```js
  type Module {
      id: ID!
          length: Int
  }
  ```

  The modules for a `Track` can be sourced from the track `id`. So we can again
  add a resolver chain function to the resolvers file:

  ```js
  const resolvers = {
    Query: {
      tracksForHome: (_, __, { dataSources }) => {
        return dataSources.trackApi.getTracksForHome();
      },
      track: (_, { id }, { dataSources }) => {
        return dataSources.trackAPI.getTrack(id);
      },
    },
    Track: {
      author: ({ authorId }, _, { dataSources }) => {
        return dataSources.trackApi.getAuthor(authorId);
      },
      modules: ({ id }, _, { dataSources }) => {
        return dataSources.trackAPI.getTrackModules(id);
      },
    },
  };
  ```

  Once again we are using a property on the parent `Track` type to use as a
  parameter in the resolver function.

  This setup would enables like the following to be run:

  ```js
  query track(id: 'xyz'){
    title
    modules
  }
  ```

  The `id` parameter would be used by the `modules` resolver to return the array
  of the `Modules` type.

  ## Variables in arguments

  Instead of writing the following within our query constants on the client-side:

  ```js
  query GetTrack {
    track(id: 'xyz'){
      title
    }
  }
  ```

  We can make the code more reusable by using variables instead of the hardcoded
  `id` argument:

  ```js
  query GetTrack($trackId: ID!) {
    track(id: $trackId){
      title
    }
  }
  ```

  This way we do not need to write a new query constant every time we want to
  return a specific track.

  ## Send query with arguments using Apollo `useQuery`

  We can now write a proper query using the
  [useQuery hook](Apollo_Client.md#usequery-hook) from
  Apollo Client, with variables.

  First define our query constant:

  ```js
  export const GET_TRACK = gql`
    query GetTrack($trackId: ID!) {
      track(id: $trackId) {
        id
        title
        author {
          name
        }
        description
        modules {
          id
          title
          length
        }
      }
    }
  `;
  ```

  Then to employ in React:

  ```jsx
  const trackId = "xyz";

  const { loading, error, data } = useQuery(GET_TRACK, {
    variables: {
      id: trackId,
    },
  });
  ```

  Note that in contrast to the
  [simple example](Apollo_Client.md#query-constants)
  because we are using variables, we have to pass-in an additional options object
  with the query constant that specifies our variables.


path: Utilities_operators_flags.md
sortable_path: Utilities_operators_flags.md
title: Utilities, operators, flags in Bash
checksum: 05c70aee147aa773a0a5398a3250697f6d1ff38a8998eeb9f614efd16dbb7e6c
created: 2025-10-01 13:31:02.778386977+00:00
modified: 2025-08-13 15:57:13.971670195+00:00
metadata: {"tags":["shell"]}
lead:
  The following are useful built-in utility methods that you can use for checking
  and validation in the course of your bash scripts.

body:
  The following are useful built-in utility methods that you can use for checking
  and validation in the course of your bash scripts.

  ## Flags

  ### Prevent newline

  Prevent bash from adding a new line after an echo:

  ```bash
  echo 'Your name is Thomas'
  echo 'and you are 33 years old'

  # Your name is Thomas
  # and you are 33 years old
  ```

  ```bash
  echo -n 'Your name is Thomas '
  echo 'and you are 33 years old'

  # Your name is Thomas and you are 33 years old
  ```

  ## Operators

  ### Mathematical

  ```bash
  -lt , -gt,
  ```

raw_content:
  ---
  tags:
    - shell
  ---

  # Utilities, operators, flags in Bash

  The following are useful built-in utility methods that you can use for checking
  and validation in the course of your bash scripts.

  ## Flags

  ### Prevent newline

  Prevent bash from adding a new line after an echo:

  ```bash
  echo 'Your name is Thomas'
  echo 'and you are 33 years old'

  # Your name is Thomas
  # and you are 33 years old
  ```

  ```bash
  echo -n 'Your name is Thomas '
  echo 'and you are 33 years old'

  # Your name is Thomas and you are 33 years old
  ```

  ## Operators

  ### Mathematical

  ```bash
  -lt , -gt,
  ```


path: Validating_Mongoose_schemas.md
sortable_path: Validating_Mongoose_schemas.md
title: Validating Mongoose schemas
lead: Let's say we have the following schema:
checksum: 847b00e5d4138313509f38e42afa01586c9e0fa1e3cd708baf6a709aafe2702a
created: 2025-10-01 13:31:02.784325332+00:00
modified: 2025-08-13 15:57:13.832544026+00:00
metadata: {"tags":["mongo-db","mongoose","node-js","databases"]}
body:
  Let's say we have the following schema:

  ```js
  const courseSchema = new mongoose.Schema({
    name: { type: String, required: true, minlength: 5, maxlength: 255 },
    author: String,
    tags: [String],
    data: { type: Date, default: Date.now }, // if unspecified, entry will default to current date
    isPublished: Boolean,
  });
  ```

  We set our validation criteria as the second property for each schema value. If
  it does not obtain, an exception will be thrown when we attempt to modify the
  database. This will most likely take the form of a Promise rejection.

  Note that this validation only occurs at the level of Mongoose. There is no
  mechanism for running the validation within the database itself. This is one of
  the ways that Mongo differs from an SQL database where you would be able to set
  the property of `required` for a cell in the table. If we didn't set any
  validation via Mongoose, Mongo would accept whatever we sent to it.

  What is the relationship between this Mongoose validation and the
  [Joi](Validation_in_NodeJS.md) validation that we
  use when validating API requests in Node/Express? They complement each other. We
  use Joi to validate the client request to the API. If this is valid, the process
  would then move onto the next stage which would be transforming the data from a
  REST request to a database request implemented through Mongoose, where a second
  layer of validation would appy. We would also have validation on the
  client-side.

  ## Validator functions

  Instead of simply applying a built-in validator like `required`, we can instead
  pass a function that validates agains the validation status of another property
  in the schema. For instance: make the `price` property required only if the
  entry has `true` for `isPublished`:

  ```js
  const courseSchema = new mongoose.Schema({
    ...
    isPublished: Boolean,
    price: {
      type: Number,
      required: function() {return this.isPublished}
    }
  });
  ```

  ## Enum validators

  An enum validator enables us to ensure that whatever value is passed when
  instantiating a document that is of a given schema type, it must be one of a
  finite set of values:

  ```js
  const anotherSchema = new mongoose.Schema({
    sex: {
      type: String,
      enum: ["male", "female"],
    },
  });
  ```

raw_content:
  ---
  tags:
    - mongo-db
    - mongoose
    - node-js
    - databases
  ---

  # Validating Mongoose schemas

  Let's say we have the following schema:

  ```js
  const courseSchema = new mongoose.Schema({
    name: { type: String, required: true, minlength: 5, maxlength: 255 },
    author: String,
    tags: [String],
    data: { type: Date, default: Date.now }, // if unspecified, entry will default to current date
    isPublished: Boolean,
  });
  ```

  We set our validation criteria as the second property for each schema value. If
  it does not obtain, an exception will be thrown when we attempt to modify the
  database. This will most likely take the form of a Promise rejection.

  Note that this validation only occurs at the level of Mongoose. There is no
  mechanism for running the validation within the database itself. This is one of
  the ways that Mongo differs from an SQL database where you would be able to set
  the property of `required` for a cell in the table. If we didn't set any
  validation via Mongoose, Mongo would accept whatever we sent to it.

  What is the relationship between this Mongoose validation and the
  [Joi](Validation_in_NodeJS.md) validation that we
  use when validating API requests in Node/Express? They complement each other. We
  use Joi to validate the client request to the API. If this is valid, the process
  would then move onto the next stage which would be transforming the data from a
  REST request to a database request implemented through Mongoose, where a second
  layer of validation would appy. We would also have validation on the
  client-side.

  ## Validator functions

  Instead of simply applying a built-in validator like `required`, we can instead
  pass a function that validates agains the validation status of another property
  in the schema. For instance: make the `price` property required only if the
  entry has `true` for `isPublished`:

  ```js
  const courseSchema = new mongoose.Schema({
    ...
    isPublished: Boolean,
    price: {
      type: Number,
      required: function() {return this.isPublished}
    }
  });
  ```

  ## Enum validators

  An enum validator enables us to ensure that whatever value is passed when
  instantiating a document that is of a given schema type, it must be one of a
  finite set of values:

  ```js
  const anotherSchema = new mongoose.Schema({
    sex: {
      type: String,
      enum: ["male", "female"],
    },
  });
  ```


path: Validation_in_NodeJS.md
sortable_path: Validation_in_NodeJS.md
title: Creating a RESTful API: Validation
checksum: 6f0c4e7f0af95a7522c442d4991bd06ab0a817dac6eabb19ed81afb036089759
created: 2025-10-01 13:31:02.8015669+00:00
modified: 2025-08-13 15:57:14.041669711+00:00
metadata: {"tags":["node-js","REST","APIs"]}
lead:
  We can provide server-side validation for our projects by using a **schema
  validator**. This is a program that declaratively parses the JSON values
  received as requests from the client. This makes it easy to systematically
  validate the data that we receive from any HTTP requests where the client sends
  a body to the endpoint.

body:
  We can provide server-side validation for our projects by using a **schema
  validator**. This is a program that declaratively parses the JSON values
  received as requests from the client. This makes it easy to systematically
  validate the data that we receive from any HTTP requests where the client sends
  a body to the endpoint.

  One of the most popular schema validators for NodeJS is
  [joi](joi).

  ## Demonstration

  Let's say we have a POST request that expects a single field as the body that
  must be a string and greater than two characters long. First we define our
  schema:

  ```js
  const schema = Joi.object({
    name: Joi.string().min(3).required(),
  });

  const { error } = schema.validate(req.body);
  ```

  The `schema` variable is an object whose keys should match those of the intended
  request body. Instead of actual values we provide Joi's in-built validators,
  concatenated as necessary. We then store the results of the validation in a
  variable.

  Next we add handling in the case of errors:

  ```js
  if (error) {
    error.details.map((joiErr) => res.status(400).send(joiErr.message));
    return;
  }
  ```

  We loop through the error array and return 400s as the response if they are
  found. If there are no errors, the Joi object will return `undefined`.

raw_content:
  ---
  tags:
    - node-js
    - REST
    - APIs
  ---

  # Creating a RESTful API: Validation

  We can provide server-side validation for our projects by using a **schema
  validator**. This is a program that declaratively parses the JSON values
  received as requests from the client. This makes it easy to systematically
  validate the data that we receive from any HTTP requests where the client sends
  a body to the endpoint.

  One of the most popular schema validators for NodeJS is
  [joi](joi).

  ## Demonstration

  Let's say we have a POST request that expects a single field as the body that
  must be a string and greater than two characters long. First we define our
  schema:

  ```js
  const schema = Joi.object({
    name: Joi.string().min(3).required(),
  });

  const { error } = schema.validate(req.body);
  ```

  The `schema` variable is an object whose keys should match those of the intended
  request body. Instead of actual values we provide Joi's in-built validators,
  concatenated as necessary. We then store the results of the validation in a
  variable.

  Next we add handling in the case of errors:

  ```js
  if (error) {
    error.details.map((joiErr) => res.status(400).send(joiErr.message));
    return;
  }
  ```

  We loop through the error array and return 400s as the response if they are
  found. If there are no errors, the Joi object will return `undefined`.


path: Validity_and_entailment.md
sortable_path: Validity_and_entailment.md
title: Validity and entailment
lead: ### Informal definition
checksum: c1ea50298640625de41a08e73e8852203b6a59a2f392299a43dc5ef7465fdb2b
created: 2025-10-01 13:31:02.815016062+00:00
modified: 2025-08-13 15:57:14.059007061+00:00
metadata: {"tags":["propositional-logic","logic"]}
body:
  ### Informal definition

  In order to say whether an argument is 'good' or 'bad' we must have criteria of
  evaluation. in logic there are different criteria of evaluation:

  - **Deductive validity**

    An **argument is deductively valid if and only if it is not possible for the
    premises to be true and the conclusion false**. Linking to consistency: it is
    not possible to consistently assert all of the premises but deny the
    conclusion.

  - **Inductive strength**

    We do not say that inductive arguments have 'validity' because despite
    inductive premises being true, the conclusion may be falsifiable. Therefore we
    say inductive 'strength' rather than 'validity'. An argument is inductively
    strong if and only if the conclusion is probably true given the premises.

  #### Demonstration

  The Socrates demonstration above is an example of deductive validity.

  The following is an example of an argument that is inductively strong:

  ```
  99% of deaf persons have no musical talent.
  Beethoven was deaf.
  ___________________________________________
  Beethoven had no musical talent.
  ```

  The test for a strong inductive argument is not whether the conclusion is true,
  rather it concerns the evidence the premises provide in support of the
  conclusion.

  > In propositional logic we are concerned solely with deductive validity or
  > invalidity.

  ### Formal definition

  > An argument is truth-functionally valid if and only if there is no
  > truth-assignment on which all the premises are true and the conclusion is
  > false.

  Linking this to derivation, we say:

  > In a system of derivation in propositional logic, an argument is valid if the
  > conclusion of the argument is derivable within the system of derivation from
  > the set consisting of the premises, and invalid otherwise.

  ### Demonstration

  The inference from the set ${P, P \rightarrow Q}$ to $Q$ is valid

  ### Truth-table

  | $P$ | $Q$ | $P \rightarrow Q$ | $P$ | $Q$ | Assessment |
  | --- | --- | ----------------- | --- | --- | ---------- |
  | T   | T   | T                 | T   | T   | Valid      |
  | T   | F   | F                 | T   | F   |            |
  | F   | T   | T                 | F   | T   |            |

  ## Entailment

  ### Informal definition

  Entailment as a concept is almost identical to validity. We say that a
  proposition is entailed by a set of propositions if it is not possible for every
  member of this set to be true and the proposition to be false.

  The difference with validity resides in the fact that the propositions are
  distinguished in terms of whether they are premises or a conclusion. So,
  technically, validity is a subclass of entailment. A case of entailment where we
  distinguish propositions in terms of whether they are premises or conclusions. A
  proposition may be entailed by a given set without that proposition being the
  _conclusion_ of the set and where the set is a syllogism.

  ### Formal definition

  > A finite set of sentences $\Gamma$ $\vdash$ $P$ if and only if there is no
  > truth-assignment in which every member of $\Gamma$ is true and $P$ is false.

  #### Informal demonstration

  ```
  It is raining.
  If it is raining then the pavement will be wet.
  The pavement is wet.
  ```

  #### Formal demonstration

  $$
    \{ P, P\rightarrow Q   \} \vdash Q
  $$

  #### Truth-table

  | $P$ | $Q$ | $P \rightarrow Q$ | $P$ | $Q$ | Assessment |
  | --- | --- | ----------------- | --- | --- | ---------- |
  | T   | T   | T                 | T   | T   | Valid      |
  | T   | F   | F                 | T   | F   |            |
  | F   | T   | T                 | F   | T   |            |

raw_content:
  ---
  tags:
    - propositional-logic
    - logic
  ---

  # Validity and entailment

  ### Informal definition

  In order to say whether an argument is 'good' or 'bad' we must have criteria of
  evaluation. in logic there are different criteria of evaluation:

  - **Deductive validity**

    An **argument is deductively valid if and only if it is not possible for the
    premises to be true and the conclusion false**. Linking to consistency: it is
    not possible to consistently assert all of the premises but deny the
    conclusion.

  - **Inductive strength**

    We do not say that inductive arguments have 'validity' because despite
    inductive premises being true, the conclusion may be falsifiable. Therefore we
    say inductive 'strength' rather than 'validity'. An argument is inductively
    strong if and only if the conclusion is probably true given the premises.

  #### Demonstration

  The Socrates demonstration above is an example of deductive validity.

  The following is an example of an argument that is inductively strong:

  ```
  99% of deaf persons have no musical talent.
  Beethoven was deaf.
  ___________________________________________
  Beethoven had no musical talent.
  ```

  The test for a strong inductive argument is not whether the conclusion is true,
  rather it concerns the evidence the premises provide in support of the
  conclusion.

  > In propositional logic we are concerned solely with deductive validity or
  > invalidity.

  ### Formal definition

  > An argument is truth-functionally valid if and only if there is no
  > truth-assignment on which all the premises are true and the conclusion is
  > false.

  Linking this to derivation, we say:

  > In a system of derivation in propositional logic, an argument is valid if the
  > conclusion of the argument is derivable within the system of derivation from
  > the set consisting of the premises, and invalid otherwise.

  ### Demonstration

  The inference from the set ${P, P \rightarrow Q}$ to $Q$ is valid

  ### Truth-table

  | $P$ | $Q$ | $P \rightarrow Q$ | $P$ | $Q$ | Assessment |
  | --- | --- | ----------------- | --- | --- | ---------- |
  | T   | T   | T                 | T   | T   | Valid      |
  | T   | F   | F                 | T   | F   |            |
  | F   | T   | T                 | F   | T   |            |

  ## Entailment

  ### Informal definition

  Entailment as a concept is almost identical to validity. We say that a
  proposition is entailed by a set of propositions if it is not possible for every
  member of this set to be true and the proposition to be false.

  The difference with validity resides in the fact that the propositions are
  distinguished in terms of whether they are premises or a conclusion. So,
  technically, validity is a subclass of entailment. A case of entailment where we
  distinguish propositions in terms of whether they are premises or conclusions. A
  proposition may be entailed by a given set without that proposition being the
  _conclusion_ of the set and where the set is a syllogism.

  ### Formal definition

  > A finite set of sentences $\Gamma$ $\vdash$ $P$ if and only if there is no
  > truth-assignment in which every member of $\Gamma$ is true and $P$ is false.

  #### Informal demonstration

  ```
  It is raining.
  If it is raining then the pavement will be wet.
  The pavement is wet.
  ```

  #### Formal demonstration

  $$
    \{ P, P\rightarrow Q   \} \vdash Q
  $$

  #### Truth-table

  | $P$ | $Q$ | $P \rightarrow Q$ | $P$ | $Q$ | Assessment |
  | --- | --- | ----------------- | --- | --- | ---------- |
  | T   | T   | T                 | T   | T   | Valid      |
  | T   | F   | F                 | T   | F   |            |
  | F   | T   | T                 | F   | T   |            |


path: Variable_indirection_in_Bash.md
sortable_path: Variable_indirection_in_Bash.md
title: Variable indirection
lead: > Write proper notes on this
checksum: 6682743efcc1a97b2ba0760899b1690e9af193e72e3df5a957904ab8e01726a5
created: 2025-10-01 13:31:02.821775095+00:00
modified: 2025-08-13 15:57:13.989124196+00:00
metadata: {"tags":["shell"]}
body:
  > Write proper notes on this

  ```sh
  function array_empty() {
      declare -n arr=$1
      # Proceed if array not empty:
      if [ ${#arr[@]} -gt 0 ]; then
          return 1 # array is not empty
      else
          return 0 # array is empty
      fi
  }

  my_array=(1 2 3)

  function push() {
      # $1 = stack name
      local stack_name="$1"
      if [ "${stack_name}" ]; then # stack exists
          if array_empty "${!stack_name}"; then
              echo "stack is empty"
          else
              echo "stack is not empty"
          fi
      else
          echo "Error: ${stack_name} does not exist."
      fi
  }

  push "my_array"

  ```

  In Bash scripting, ${!stack_name} is an example of indirect variable referencing
  or variable indirection. It's a way to use a variable whose name is stored in
  another variable.

  Let's break it down:

  `${stack_name}`: This syntax is used to reference the value of a variable. So if
  stack_name="my_array", then ${stack_name} would return "my_array".

  `${!stack_name}`: Adding the ! introduces indirection. Now, instead of getting
  the value of stack_name, you get the value of the variable whose name is stored
  in stack_name. So if stack_name="my_array" and my_array=(1 2 3), then
  ${!stack_name} would return (1 2 3).

  So, in the context of your script, `${!stack_name} allows you to pass the name
  of an array (as a string) to your push function, and then use that string to
  indirectly reference the actual array within the function.

raw_content:
  ---
  tags:
    - shell
  ---

  # Variable indirection

  > Write proper notes on this

  ```sh
  function array_empty() {
      declare -n arr=$1
      # Proceed if array not empty:
      if [ ${#arr[@]} -gt 0 ]; then
          return 1 # array is not empty
      else
          return 0 # array is empty
      fi
  }

  my_array=(1 2 3)

  function push() {
      # $1 = stack name
      local stack_name="$1"
      if [ "${stack_name}" ]; then # stack exists
          if array_empty "${!stack_name}"; then
              echo "stack is empty"
          else
              echo "stack is not empty"
          fi
      else
          echo "Error: ${stack_name} does not exist."
      fi
  }

  push "my_array"

  ```

  In Bash scripting, ${!stack_name} is an example of indirect variable referencing
  or variable indirection. It's a way to use a variable whose name is stored in
  another variable.

  Let's break it down:

  `${stack_name}`: This syntax is used to reference the value of a variable. So if
  stack_name="my_array", then ${stack_name} would return "my_array".

  `${!stack_name}`: Adding the ! introduces indirection. Now, instead of getting
  the value of stack_name, you get the value of the variable whose name is stored
  in stack_name. So if stack_name="my_array" and my_array=(1 2 3), then
  ${!stack_name} would return (1 2 3).

  So, in the context of your script, `${!stack_name} allows you to pass the name
  of an array (as a string) to your push function, and then use that string to
  indirectly reference the actual array within the function.


path: Variables_and_data_types_in_Bash.md
sortable_path: Variables_and_data_types_in_Bash.md
title: Variables and datatypes in Bash
lead: ## Types
checksum: fc6431165e27bec3e2955dbf547f4094db1ea5c48191596be901043599a966a2
created: 2025-10-01 13:31:02.827412994+00:00
modified: 2025-08-13 15:57:14.035126223+00:00
metadata: {"tags":["shell"]}
body:
  ## Types

  ## Variables

  We use the equality symbol to create a variable:

  ```bash
  stringVar="My first variable"
  floatVar="50.3"
  boolVar="true"
  ```

  As there is no typing in bash, the names of these variables are purely notional.

  To invoke a variable we use special brackets:

  ```bash
  echo ${stringVar} # My first variable
  echo ${floatVar} # 50.3
  echo ${boolVar} # true
  ```

  - Quotation marks at declaration are also not strictly necessary however they
    can help avoid bugs. Also serves as a reminder that every type is basically a
    string at the end of the day

  ## Common operations

  ### Check if a variable exists

  ```sh
  if [ -v variable_name ]; then
      echo "The variable exists."
  else
      echo "The variable does not exist."
  fi
  ```

  ### Check if a variable is set (defined)

  ```sh
  if [ -z "${variable_name+x}" ]; then
      echo "The variable does not exist."
  else
      echo "The variable exists."
  fi
  ```

  ## Parameter expansion

  Still not very clear on this.

raw_content:
  ---
  tags:
    - shell
  ---

  # Variables and datatypes in Bash

  ## Types

  ## Variables

  We use the equality symbol to create a variable:

  ```bash
  stringVar="My first variable"
  floatVar="50.3"
  boolVar="true"
  ```

  As there is no typing in bash, the names of these variables are purely notional.

  To invoke a variable we use special brackets:

  ```bash
  echo ${stringVar} # My first variable
  echo ${floatVar} # 50.3
  echo ${boolVar} # true
  ```

  - Quotation marks at declaration are also not strictly necessary however they
    can help avoid bugs. Also serves as a reminder that every type is basically a
    string at the end of the day

  ## Common operations

  ### Check if a variable exists

  ```sh
  if [ -v variable_name ]; then
      echo "The variable exists."
  else
      echo "The variable does not exist."
  fi
  ```

  ### Check if a variable is set (defined)

  ```sh
  if [ -z "${variable_name+x}" ]; then
      echo "The variable does not exist."
  else
      echo "The variable exists."
  fi
  ```

  ## Parameter expansion

  Still not very clear on this.


path: Variables_in_C.md
sortable_path: Variables_in_C.md
title: Variables in C
lead:
  ```c
  int age = 25
  ```

body:
  ```c
  int age = 25
  ```

checksum: 5a795d624f330f27608912116a4e66a003e6534c585d38e5adfe4b8d7df0ee99
created: 2025-10-01 13:31:02.832705916+00:00
modified: 2025-08-13 15:57:14.044030918+00:00
metadata: {"created":"Thursday, February 29, 2024 | 17:20","id":"r4i16p1x","tags":["C"]}
raw_content:
  ---
  id: r4i16p1x
  tags: [C]
  created: Thursday, February 29, 2024 | 17:20
  ---

  # Variables in C

  ```c
  int age = 25
  ```


path: View_IP_addresses.md
sortable_path: View_IP_addresses.md
title: View IP addresses
lead:
  Use `ip` package to view the [IP addresses](IP_addresses.md) associated with
  your current host:

checksum: eb10adeb1c2a55330524862e35dd667be3d04ac8a31d607e3bc2d3974587fcd1
created: 2025-10-01 13:31:02.840399372+00:00
modified: 2025-08-13 15:57:13.984630209+00:00
metadata: {"created":"Friday, January 10, 2025","tags":["networks","procedural","Linux"]}
body:
  Use `ip` package to view the [IP addresses](IP_addresses.md) associated with
  your current host:

  ```sh
  ip address show
  ```

  I am curently using ethernet on the computer I am writing this. This is listed
  as follows:

  ```
  enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
  link/ether 90:2e:16:53:98:b9 brd ff:ff:ff:ff:ff:ffinet
  192.168.0.3/24 brd 192.168.0.255 scope global dynamic noprefixroute enp0s31f6
      valid_lft 83353sec preferred_lft 83353sec
  ```

  The most important line is the third. This indicates my IP address
  (`192.168.0.3/24`) including its subnet mask.

raw_content:
  ---
  tags: [networks, procedural, Linux]
  created: Friday, January 10, 2025
  ---

  # View IP addresses

  Use `ip` package to view the [IP addresses](IP_addresses.md) associated with
  your current host:

  ```sh
  ip address show
  ```

  I am curently using ethernet on the computer I am writing this. This is listed
  as follows:

  ```
  enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
  link/ether 90:2e:16:53:98:b9 brd ff:ff:ff:ff:ff:ffinet
  192.168.0.3/24 brd 192.168.0.255 scope global dynamic noprefixroute enp0s31f6
      valid_lft 83353sec preferred_lft 83353sec
  ```

  The most important line is the third. This indicates my IP address
  (`192.168.0.3/24`) including its subnet mask.


path: Viewing_Docker_logs.md
sortable_path: Viewing_Docker_logs.md
title: Viewing Docker logs
lead: ## View logs from outside of container
checksum: e89d57428d7694c7f0036bcdc5bc281ca818928f424f16ebdaa177e3f25f80c6
created: 2025-10-01 13:31:02.853782253+00:00
modified: 2025-08-13 15:57:14.069410918+00:00
metadata: {"created":"Thursday, April 24, 2025","tags":["docker","procedural"]}
body:
  ## View logs from outside of container

  ```sh
  docker logs <container_name> OR <container_id>
  ```

  To view the logs in realtime, apend `-f` for `--follow`:

  ```sh
  docker logs -f <container_name> OR <container_id>
  ```

  ## View logs from within the container

  For example, to view nginx logs, first
  [enter into the container](./Enter_into_a_Docker_container.md) and then:

  ```sh
  cat /var/log/nginx/access.log
  cat /var/log/nginx/error.log
  ```

raw_content:
  ---
  tags: [docker, procedural]
  created: Thursday, April 24, 2025
  ---

  # Viewing Docker logs

  ## View logs from outside of container

  ```sh
  docker logs <container_name> OR <container_id>
  ```

  To view the logs in realtime, apend `-f` for `--follow`:

  ```sh
  docker logs -f <container_name> OR <container_id>
  ```

  ## View logs from within the container

  For example, to view nginx logs, first
  [enter into the container](./Enter_into_a_Docker_container.md) and then:

  ```sh
  cat /var/log/nginx/access.log
  cat /var/log/nginx/error.log
  ```


path: Viewing_remote_changes_without_merging.md
sortable_path: Viewing_remote_changes_without_merging.md
title: Viewing remote changes without merging
lead:
  ```
  git fetch origin/master
  git log origin/main
  git show origin/master
  ```

body:
  ```
  git fetch origin/master
  git log origin/main
  git show origin/master
  ```

checksum: 1a5f587f96e84a733ebdf522999e5efdca8e55052d1f846baa71e7e768081e7c
created: 2025-10-01 13:31:02.868703854+00:00
modified: 2025-08-13 15:57:13.798919974+00:00
metadata: {"tags":["git"]}
raw_content:
  ---
  tags: [git]
  ---

  # Viewing remote changes without merging

  ```
  git fetch origin/master
  git log origin/main
  git show origin/master
  ```


path: Views_in_relational_databases.md
sortable_path: Views_in_relational_databases.md
title: Views in relational databases
checksum: f48de8e2b132aa72d28eefba0fb3508f1baac92593009b7f0bd7be27e3a539ed
created: 2025-10-01 13:31:02.873290752+00:00
modified: 2025-08-13 15:57:13.930439416+00:00
metadata: {"tags":["databases"]}
lead:
  A view is a virtual table that is based on the result-set of a query. Although
  it doesn't actually exist in the same manner as an actual table it has exactly
  the same properties and functionality. Rather than adding the raw data manually,
  you derive it from logic applied to existing actual tables.

body:
  A view is a virtual table that is based on the result-set of a query. Although
  it doesn't actually exist in the same manner as an actual table it has exactly
  the same properties and functionality. Rather than adding the raw data manually,
  you derive it from logic applied to existing actual tables.

  This is a good way of dynamically generating data and is often employed as a way
  of creating mutations without changing the underlying data structure. In SQL,
  views can be created by combining `PRIMARY` and `FOREIGN` keys.

raw_content:
  ---
  tags:
    - databases
  ---

  # Views in relational databases

  A view is a virtual table that is based on the result-set of a query. Although
  it doesn't actually exist in the same manner as an actual table it has exactly
  the same properties and functionality. Rather than adding the raw data manually,
  you derive it from logic applied to existing actual tables.

  This is a good way of dynamically generating data and is often employed as a way
  of creating mutations without changing the underlying data structure. In SQL,
  views can be created by combining `PRIMARY` and `FOREIGN` keys.


path: VirtualMemory.md
sortable_path: VirtualMemory.md
title: Virtual memory
checksum: d53518d5476f53f2549469757a80180fa1c29f287964788ec6185e7fa28c3f86
created: 2025-10-01 13:31:02.878607215+00:00
modified: 2025-08-13 15:57:13.947725052+00:00
metadata: {"tags":["memory","Linux","kernel"]}
lead:
  Virtual memory is an abstracted and idealised representation of the physical
  memory capacity of a machine that is presented to user space for its memory
  operations.

body:
  Virtual memory is an abstracted and idealised representation of the physical
  memory capacity of a machine that is presented to user space for its memory
  operations.

  When an OS implements virtual memory, [processes](./Processes.md) in
  [user space](./User_Space.md) cannot directly read or write to the physical
  memory. Instead they execute memory operations against virtual memory and the
  [kernel](./The_kernel.md) translates these into the actual operations against
  the memory hardware.

  The main benefits:

  - User mode processes do not have to be concerned with the physical memory
    management
  - There is a buffer between user mode processes and physical memory, meaning
    that memory cannot be accidentally corrupted by other processes in user space.

  Because the physical memory is abstracted, it can be the case that the physical
  [memory addresses](./Memory_addresses.md) are non-contiguous or even distributed
  accross different hardware components (such as the
  [cache](./Register_and_cache_memory.md) and [swap](./Swap_space.md)). Despite
  this, the memory addresses will appear contiguous in virtual memory. Each user
  space process is presented with the same range of available memory addresses and
  the same total capacity.

  It is also possible for the kernel to present user space with an available
  virtual memory capcacity that exceeds the current physical capacity of the
  machine:

  > _It's possible for the kernel and all running processes to request more
  > byte<!--  -->s of virtual memory than the total size of RAM. In that
  > situation, the OS can move move bytes of memory to secondary storage to make
  > room in RAM for newly requested memory._

  _How Computers Really Work_ (2021) p.206

  ## Virtual memory and the kernel

  The kernel itself utilises virtual memory.

  During the [boot_process](Boot_process.md), it runs in physical memory mode
  however it rapidly creates a 1:1 mapping of physical to virtual memory for
  itself. This direct mapping allows the kernel to easily access any physical
  memory location as needed, whilst retaining the benefits of virtual memory.

  With its own virtual memory established, it can then build additional virtual
  memory structures for itself and user processes.

  While the kernel uses virtual memory it can also access physical memory at any
  time wherease user space processes can only ever access virtual memory.

  The kernel virtual memory has a different range of virtual addresses to work
  with than user space virtual memory.

  Unlike user space virtual memory, the kernel has access to everything running in
  kernel address space whereas processes in user address space are partitioned
  from each other with separate address spaces that cannot interact.

  In addition to being able to access its own virtual memory (and physical memory,
  as required) the kernel can also access any user processes' virtual address.

  This said, the virtual memory space of the kernel and the virtual memory space
  of the user processes are distinct. It is not the case that the kernel is
  superset of all available virtual memory. It can access user space virtual
  memory because it sets up the tables and locations, not because it is a subset
  of its own virtual memory.

  ![Virtual memory diagram](../img/virtual-memory-diagram.jpg)

raw_content:
  ---
  tags:
    - memory
    - Linux
    - kernel
  ---

  # Virtual memory

  Virtual memory is an abstracted and idealised representation of the physical
  memory capacity of a machine that is presented to user space for its memory
  operations.

  When an OS implements virtual memory, [processes](./Processes.md) in
  [user space](./User_Space.md) cannot directly read or write to the physical
  memory. Instead they execute memory operations against virtual memory and the
  [kernel](./The_kernel.md) translates these into the actual operations against
  the memory hardware.

  The main benefits:

  - User mode processes do not have to be concerned with the physical memory
    management
  - There is a buffer between user mode processes and physical memory, meaning
    that memory cannot be accidentally corrupted by other processes in user space.

  Because the physical memory is abstracted, it can be the case that the physical
  [memory addresses](./Memory_addresses.md) are non-contiguous or even distributed
  accross different hardware components (such as the
  [cache](./Register_and_cache_memory.md) and [swap](./Swap_space.md)). Despite
  this, the memory addresses will appear contiguous in virtual memory. Each user
  space process is presented with the same range of available memory addresses and
  the same total capacity.

  It is also possible for the kernel to present user space with an available
  virtual memory capcacity that exceeds the current physical capacity of the
  machine:

  > _It's possible for the kernel and all running processes to request more
  > byte<!--  -->s of virtual memory than the total size of RAM. In that
  > situation, the OS can move move bytes of memory to secondary storage to make
  > room in RAM for newly requested memory._

  _How Computers Really Work_ (2021) p.206

  ## Virtual memory and the kernel

  The kernel itself utilises virtual memory.

  During the [boot_process](Boot_process.md), it runs in physical memory mode
  however it rapidly creates a 1:1 mapping of physical to virtual memory for
  itself. This direct mapping allows the kernel to easily access any physical
  memory location as needed, whilst retaining the benefits of virtual memory.

  With its own virtual memory established, it can then build additional virtual
  memory structures for itself and user processes.

  While the kernel uses virtual memory it can also access physical memory at any
  time wherease user space processes can only ever access virtual memory.

  The kernel virtual memory has a different range of virtual addresses to work
  with than user space virtual memory.

  Unlike user space virtual memory, the kernel has access to everything running in
  kernel address space whereas processes in user address space are partitioned
  from each other with separate address spaces that cannot interact.

  In addition to being able to access its own virtual memory (and physical memory,
  as required) the kernel can also access any user processes' virtual address.

  This said, the virtual memory space of the kernel and the virtual memory space
  of the user processes are distinct. It is not the case that the kernel is
  superset of all available virtual memory. It can access user space virtual
  memory because it sets up the tables and locations, not because it is a subset
  of its own virtual memory.

  ![Virtual memory diagram](../img/virtual-memory-diagram.jpg)


path: Virtual_private_cloud.md
sortable_path: Virtual_private_cloud.md
title: Virtual Private Cloud
checksum: 34438b2d0b9aad65bd3698fdee839c0a140c5900550d8ed3f306cb75f0cd3306
created: 2025-10-01 13:31:02.922013845+00:00
modified: 2025-08-13 15:57:13.948624883+00:00
metadata: {"created":"Tuesday, February 20, 2024 | 08:31","id":"mdw5fe5a","tags":["AWS","networks"]}
lead:
  AWS VPC is used to create a virtual network. This is typically used in concert
  with [[zk/Elastic_Compute_Cloud]] to manage connections to a virtual server both
  privately and accross the internet.

body:
  AWS VPC is used to create a virtual network. This is typically used in concert
  with [[zk/Elastic_Compute_Cloud]] to manage connections to a virtual server both
  privately and accross the internet.

  You define a network address range and then create subnets for managing
  different connections and functionality. You use a public subnet for resources
  that must be connected to the internet and a private subnet for resources that
  are to remain isolated from the internet.

  The diagram below details a basic VPC configuration:

  ![](../img/aws-vpc-example.png)

  - Within a given AWS region we have created a VPC network.
  - This comprises public and private subnets
  - Both subnets host an EC2 instance
  - The public subnet has a bridge to the internet through the Internet Gateway
  - Both subnets have a routing table to manage requests and access

raw_content:
  ---
  id: mdw5fe5a
  tags: [AWS, networks]
  created: Tuesday, February 20, 2024 | 08:31
  ---

  # Virtual Private Cloud

  AWS VPC is used to create a virtual network. This is typically used in concert
  with [[zk/Elastic_Compute_Cloud]] to manage connections to a virtual server both
  privately and accross the internet.

  You define a network address range and then create subnets for managing
  different connections and functionality. You use a public subnet for resources
  that must be connected to the internet and a private subnet for resources that
  are to remain isolated from the internet.

  The diagram below details a basic VPC configuration:

  ![](../img/aws-vpc-example.png)

  - Within a given AWS region we have created a VPC network.
  - This comprises public and private subnets
  - Both subnets host an EC2 instance
  - The public subnet has a bridge to the internet through the Internet Gateway
  - Both subnets have a routing table to manage requests and access


path: Voltage.md
sortable_path: Voltage.md
title: Voltage
lead: ## Difference of potential and the transfer of energy
checksum: aad60b4e37e2791658c5676cb4e0c66e309094763d8e8260822ed70057858666
created: 2025-10-01 13:31:02.948054216+00:00
modified: 2025-08-13 15:57:14.052556719+00:00
metadata: {"tags":["physics","electricity"]}
body:
  ## Difference of potential and the transfer of energy

  We noted in the discussion of [current](Current.md) that current flows when
  there is a difference of potential between two points with negatively charged
  atoms at one point and positively charged atoms at the other.

  _Difference of potential_ is the same thing as voltage. Voltage is essential to
  current because it is the force that enables the current to flow.

  Without voltage there can be no current because in their natural state, the
  electrons in an atom are **in random motion with no direction. To produce a
  current, energy must be imparted to the electrons so that they all flow in the
  same direction.**

  Voltage is the application of this energy. Any
  [form of energy](Voltage_sources.md) that dislodges electrons from atoms can be
  used to produce current. Thus:

  > Voltage is the work required per coulomb to move a charge from one point to
  > another.

  ### Voltage exists even without current

  Given that voltage is the force that generates current, it would be natural to
  think that voltage only exists when a voltage source (such as a
  [battery](Cells_and_batteries.md)) is connected to a circuit. This however is
  not the case. Even if a 9V battery isn't connected to anything it still has a
  difference of potential of 9-volts accross its terminals. Remember voltage is
  _potential energy_ not just the actualisation of that energy.

  ## Voltage rise and voltage drops

  In circuits there are actually two types of voltage:

  1. Voltage rise
  2. Voltage drop

  ### Voltage rise

  When we introduce potential energy into a ciruit in the form of voltage, this is
  a voltage rise. The current flows from the negative terminal of the voltage
  source and returns to the positive terminal of the voltage source.

  A 12V battery connected to a circuit gives it a voltage rise of 12 volts.

  ### Voltage drop

  Voltage drop is the corrolary to voltage rise. It is the loss of energy that the
  electrons of the circuit current experience as a result of encountering
  resistance.

  As they move through the circuit the electrons encounter a **load** which is
  what we call resistance to the flow of current. As they run into this, they give
  up their energy. The relinquishing of energy happens in the form of a conversion
  of electrical energy to heat (hence why computers heat up). The amount lost is
  equal to the amount of energy imparted by the voltage rise.

  > The voltage drop in a circuit equals the the voltage rise of the circuit
  > because energy cannot be created or destroyed, only changed to another form.
  > When a voltage rise is converted to a voltage drop we say that **the energy
  > has been _consumed_ by the circuit**.

  #### Examples

  - If a 12V source is connected to a 12V lamp, the source supplies a 12V voltage
    rise and the lamp produces a 12V voltage drop.
  - If two identical 6V lamps are connected in series to the same 12V source, each
    lamp produces a 6V drop for a total of 12 volts
  - If two different lamps are connected in series to to a 12V source: a 3V and a
    9V lamp, the 9V lamp produces a drop of 9V and the 3V lamp produces a drop of
    3V. The sum of the voltage drops equals the voltage rise of 12 volts.

  These examples demonstrate that the voltage rise: voltage drop ratio always
  evens out.

  ### Kirchoff's Voltage Law

  The relationship between voltage rise and voltage drop is expressed in
  Kirchoff's Voltage Law:

  > The sum of the voltages around any closed loop in a circuit must be zero.

  The application of the Law is illustrated in the following diagram:

  ![](/img/voltage-drop.png)

  The explanation for the voltage drop at the positions $V^{A}$ and $V^{D}$ are
  obvious enough: they are at the beginning and end of the loop so are equal to
  the maximal voltage rise and minimal voltage drop, respectively.

  We can work out the voltage of the remaining voltage points by inverting
  [Ohm's Law](Ohms_Law.md): $V = I \times R$:

  For the voltage at $V^{B}$:

  $$
    0.5\textsf{mA} \times \textsf{4}k\Omega = 2 \textsf{V}
  $$

  For the voltage at $V^{C}$:

  $$
    0.5\textsf{mA} \times \textsf{6}k\Omega = 3 \textsf{V}
  $$

  The total drops (2V, 3V, 5V) are equal to the initial volatage rise (10V)

  ## Distinguishing _voltage_ from _electric field_

  It can be confusing that two different symbols often seemed to be used
  interchangeably when talking about voltage: $V$ and $E$. However, while they
  broadly point to the same phenomenon there is a difference in emphasis.

  - $V$ stands for volts or voltage conceived purely in terms of the difference in
    potential between two points: the positive and negative terminals
  - $E$ stands for electric field. This is the field that surround each electric
    charge and exerts force on all other charges in the field, attracting or
    repelling them. So it is more the physical process that the volatage
    represents rather than the numerical representation of the potential between
    the terminals.

raw_content:
  ---
  tags: [physics, electricity]
  ---

  # Voltage

  ## Difference of potential and the transfer of energy

  We noted in the discussion of [current](Current.md) that current flows when
  there is a difference of potential between two points with negatively charged
  atoms at one point and positively charged atoms at the other.

  _Difference of potential_ is the same thing as voltage. Voltage is essential to
  current because it is the force that enables the current to flow.

  Without voltage there can be no current because in their natural state, the
  electrons in an atom are **in random motion with no direction. To produce a
  current, energy must be imparted to the electrons so that they all flow in the
  same direction.**

  Voltage is the application of this energy. Any
  [form of energy](Voltage_sources.md) that dislodges electrons from atoms can be
  used to produce current. Thus:

  > Voltage is the work required per coulomb to move a charge from one point to
  > another.

  ### Voltage exists even without current

  Given that voltage is the force that generates current, it would be natural to
  think that voltage only exists when a voltage source (such as a
  [battery](Cells_and_batteries.md)) is connected to a circuit. This however is
  not the case. Even if a 9V battery isn't connected to anything it still has a
  difference of potential of 9-volts accross its terminals. Remember voltage is
  _potential energy_ not just the actualisation of that energy.

  ## Voltage rise and voltage drops

  In circuits there are actually two types of voltage:

  1. Voltage rise
  2. Voltage drop

  ### Voltage rise

  When we introduce potential energy into a ciruit in the form of voltage, this is
  a voltage rise. The current flows from the negative terminal of the voltage
  source and returns to the positive terminal of the voltage source.

  A 12V battery connected to a circuit gives it a voltage rise of 12 volts.

  ### Voltage drop

  Voltage drop is the corrolary to voltage rise. It is the loss of energy that the
  electrons of the circuit current experience as a result of encountering
  resistance.

  As they move through the circuit the electrons encounter a **load** which is
  what we call resistance to the flow of current. As they run into this, they give
  up their energy. The relinquishing of energy happens in the form of a conversion
  of electrical energy to heat (hence why computers heat up). The amount lost is
  equal to the amount of energy imparted by the voltage rise.

  > The voltage drop in a circuit equals the the voltage rise of the circuit
  > because energy cannot be created or destroyed, only changed to another form.
  > When a voltage rise is converted to a voltage drop we say that **the energy
  > has been _consumed_ by the circuit**.

  #### Examples

  - If a 12V source is connected to a 12V lamp, the source supplies a 12V voltage
    rise and the lamp produces a 12V voltage drop.
  - If two identical 6V lamps are connected in series to the same 12V source, each
    lamp produces a 6V drop for a total of 12 volts
  - If two different lamps are connected in series to to a 12V source: a 3V and a
    9V lamp, the 9V lamp produces a drop of 9V and the 3V lamp produces a drop of
    3V. The sum of the voltage drops equals the voltage rise of 12 volts.

  These examples demonstrate that the voltage rise: voltage drop ratio always
  evens out.

  ### Kirchoff's Voltage Law

  The relationship between voltage rise and voltage drop is expressed in
  Kirchoff's Voltage Law:

  > The sum of the voltages around any closed loop in a circuit must be zero.

  The application of the Law is illustrated in the following diagram:

  ![](/img/voltage-drop.png)

  The explanation for the voltage drop at the positions $V^{A}$ and $V^{D}$ are
  obvious enough: they are at the beginning and end of the loop so are equal to
  the maximal voltage rise and minimal voltage drop, respectively.

  We can work out the voltage of the remaining voltage points by inverting
  [Ohm's Law](Ohms_Law.md): $V = I \times R$:

  For the voltage at $V^{B}$:

  $$
    0.5\textsf{mA} \times \textsf{4}k\Omega = 2 \textsf{V}
  $$

  For the voltage at $V^{C}$:

  $$
    0.5\textsf{mA} \times \textsf{6}k\Omega = 3 \textsf{V}
  $$

  The total drops (2V, 3V, 5V) are equal to the initial volatage rise (10V)

  ## Distinguishing _voltage_ from _electric field_

  It can be confusing that two different symbols often seemed to be used
  interchangeably when talking about voltage: $V$ and $E$. However, while they
  broadly point to the same phenomenon there is a difference in emphasis.

  - $V$ stands for volts or voltage conceived purely in terms of the difference in
    potential between two points: the positive and negative terminals
  - $E$ stands for electric field. This is the field that surround each electric
    charge and exerts force on all other charges in the field, attracting or
    repelling them. So it is more the physical process that the volatage
    represents rather than the numerical representation of the potential between
    the terminals.


path: Voltage_sources.md
sortable_path: Voltage_sources.md
title: Voltage sources
lead: The following phenomena are the main sources that we use to generate voltage:
checksum: 88d7509073e3453a58af7a34f89f36e3de1f0212048f3255ad57489a3e2bf0a2
created: 2025-10-01 13:31:02.984506127+00:00
modified: 2025-08-13 15:57:13.882533183+00:00
metadata: {"tags":["physics","electricity"]}
body:
  The following phenomena are the main sources that we use to generate voltage:

  - friction
  - magnetism
  - chemicals
  - light
  - heat
  - pressure

  > In each case, energy in one state is transferred into energy as current. No
  > energy is lost or gained, it simply changes its form.

  ### Magentism (electrical generators)

  Magnetism is used as the voltage source in electrical generators and is by far
  the most common method of producing powerful and large currents at scale.

  If a conductive wire is passed through a magnetic field voltage will be produced
  so long as there is motion between the magnetic field and the conductor. A
  **generator** is a device that generates current in this manner. Generators
  themselves need to be powered. They can be powered by steam from a nuclear power
  plant, water, wind, coal or other fossil fuels.

  #### AC/DC

  Depending on how it is wired, a generator can produce **directed current** (DC)
  or **alternating current** (AC):

  - **Directed current**

    - The electrons flow in only one direction

  - **Alternating current**
    - The electrons flow in one direction and then the other

  ### Chemicals (cells and batteries)

  The chemical creation of current is the physics behind
  [batteries](Cells_and_batteries.md). Chemical current production produces
  currents on a smaller and less industrial scale than generators.

  A chemical cell consists in two dissimilar metals such as copper and zinc. We
  call these the **electrodes**. They are immersed in a salt, acid or alkaline
  solution. We call these the **electrolytes**. The electrolyte pulls the free
  electrons from the copper electrode which leaves it imbalanced with a positive
  charge. The zinc electrode attracts the free electrons from the electrolyte
  giving it a negative charge, thus a difference of potential is achieved.

  ### Light (photovoltaic cells)

  Solar energy can be converted to electrical energy through solar panels which
  are large collections of **photovoltaic cells**.

  When the surfaces of these cells are exposed to light, it dislodges electrons
  from their orbits around the surface atoms of the cell material. For each cell
  this only produces a very small amount of energy, therefore large quantities
  must be used.

raw_content:
  ---
  tags: [physics, electricity]
  ---

  # Voltage sources

  The following phenomena are the main sources that we use to generate voltage:

  - friction
  - magnetism
  - chemicals
  - light
  - heat
  - pressure

  > In each case, energy in one state is transferred into energy as current. No
  > energy is lost or gained, it simply changes its form.

  ### Magentism (electrical generators)

  Magnetism is used as the voltage source in electrical generators and is by far
  the most common method of producing powerful and large currents at scale.

  If a conductive wire is passed through a magnetic field voltage will be produced
  so long as there is motion between the magnetic field and the conductor. A
  **generator** is a device that generates current in this manner. Generators
  themselves need to be powered. They can be powered by steam from a nuclear power
  plant, water, wind, coal or other fossil fuels.

  #### AC/DC

  Depending on how it is wired, a generator can produce **directed current** (DC)
  or **alternating current** (AC):

  - **Directed current**

    - The electrons flow in only one direction

  - **Alternating current**
    - The electrons flow in one direction and then the other

  ### Chemicals (cells and batteries)

  The chemical creation of current is the physics behind
  [batteries](Cells_and_batteries.md). Chemical current production produces
  currents on a smaller and less industrial scale than generators.

  A chemical cell consists in two dissimilar metals such as copper and zinc. We
  call these the **electrodes**. They are immersed in a salt, acid or alkaline
  solution. We call these the **electrolytes**. The electrolyte pulls the free
  electrons from the copper electrode which leaves it imbalanced with a positive
  charge. The zinc electrode attracts the free electrons from the electrolyte
  giving it a negative charge, thus a difference of potential is achieved.

  ### Light (photovoltaic cells)

  Solar energy can be converted to electrical energy through solar panels which
  are large collections of **photovoltaic cells**.

  When the surfaces of these cells are exposed to light, it dislodges electrons
  from their orbits around the surface atoms of the cell material. For each cell
  this only produces a very small amount of energy, therefore large quantities
  must be used.


path: Weaving_the_Web_Berners_Lee.md
sortable_path: Weaving_the_Web_Berners_Lee.md
title: Weaving the Web (Berners-Lee, 1999)
checksum: 3761dfc6b81bf13c1e89e99bd3610c27737224fb4f5a3d5e66e13fd1ce071eb6
created: 2025-10-01 13:31:02.99895925+00:00
modified: 2025-08-13 15:57:13.841480926+00:00
metadata: {"created":"Monday, November 04, 2024","tags":["literature","ecopolsoc","world-wide-web"]}
lead:
  Tim did not really conceive of the possibility of the centralising and
  totalising effects of social media:

body:
  Tim did not really conceive of the possibility of the centralising and
  totalising effects of social media:

  _The Web's universality leads to a thriving richness and diversity. If a company
  claims to give access to the world of information, then presents a filtered
  view, the Web loses its credibility._ (p.143)

  _Happily, the Web is so huge that there's no way any one company can dominate
  it. All the human effort people and organisations have put in all over the world
  to create Web sites and home pages is astoundingly large, and most of the effort
  has to do with what's in the Web, not the software used to browse it. The Web's
  content, and thus value, will continue despite any one company's actions._
  (p.144)

  Apropos the "alt-right" etc:

  _The analogy of a global brain is tempting, because Web and brain both involve
  huge numbers of elements - neurons and Web pages - and a mixture of structure
  and apparent randomness. However, a brain has an intelligence that emerges on
  quite a different level from anything that a neuron could be aware
  of...[W]riters have contemplated an 'emergent property' arising from the mass of
  humanity and computers. But remember that such a phenomenon would have its own
  agenda. We would not as individuals be ware of it, let alone control it, any
  more than the neuron controls the brain._ (p.222)

raw_content:
  ---
  tags: [literature, ecopolsoc, world-wide-web]
  created: Monday, November 04, 2024
  ---

  # Weaving the Web (Berners-Lee, 1999)

  Tim did not really conceive of the possibility of the centralising and
  totalising effects of social media:

  _The Web's universality leads to a thriving richness and diversity. If a company
  claims to give access to the world of information, then presents a filtered
  view, the Web loses its credibility._ (p.143)

  _Happily, the Web is so huge that there's no way any one company can dominate
  it. All the human effort people and organisations have put in all over the world
  to create Web sites and home pages is astoundingly large, and most of the effort
  has to do with what's in the Web, not the software used to browse it. The Web's
  content, and thus value, will continue despite any one company's actions._
  (p.144)

  Apropos the "alt-right" etc:

  _The analogy of a global brain is tempting, because Web and brain both involve
  huge numbers of elements - neurons and Web pages - and a mixture of structure
  and apparent randomness. However, a brain has an intelligence that emerges on
  quite a different level from anything that a neuron could be aware
  of...[W]riters have contemplated an 'emergent property' arising from the mass of
  humanity and computers. But remember that such a phenomenon would have its own
  agenda. We would not as individuals be ware of it, let alone control it, any
  more than the neuron controls the brain._ (p.222)


path: Web_precursors.md
sortable_path: Web_precursors.md
title: cfbef1c4_web_precursors
lead: ## Early precursors to the commercial internet
checksum: f5f842b9d92e189c4cfc0cab9ff8cf03050a627d4e9daceee266e21473d8bf2c
created: 2025-10-01 13:31:03.004722051+00:00
modified: 2025-08-13 15:57:13.988550679+00:00
body:
  ## Early precursors to the commercial internet

  Prior to the commercial internet and the advent of the web, there were services
  in the 1980s which allowed home computers to connect to limited public networks.
  Example services: checking one's bank balance, paying bills, transferring money,
  booking holidays.

  (Presumably, the services would be hosted on a providers'
  [mainframe](Mainframe_computers.md) or mini-computer, which the consumer would
  access through something like time-sharing?)

  ![BT telephone modem](../img/modem-bt.png)

  The mechanism of connection was via a modem. A device that connected to your
  phone's cradle. Digital signals from your home computer would be converted into
  sound signals ("tones") that would travel through the phone transmitter to a
  receiver at the service provider which you would "dial-up" beforehand.

  In the US the main provider was Micronet in the 1970s, which became Compuserve
  in the 1980s and eventually AOL in the 1990s. In the UK there was Prestel
  provided by BT, first introduced in 1971.

  These services had an expensive subscription model and consequently uptake was
  limited.

  In addition there were bulletin board systems (BBS), typically administrated by
  hobbyists. Their personal computers acted as servers that other users would
  dial-up to access. They permitted the sharing of news and messages and later
  chat-rooms for realtime communication. Some of the biggest BBSs later became the
  first ISPs.

raw_content:
  ---
  tags: [fleeting, modems, computer-history, internet, bulletin-boards]
  created: Monday, October 14, 2024
  ---

  # cfbef1c4_web_precursors

  ## Early precursors to the commercial internet

  Prior to the commercial internet and the advent of the web, there were services
  in the 1980s which allowed home computers to connect to limited public networks.
  Example services: checking one's bank balance, paying bills, transferring money,
  booking holidays.

  (Presumably, the services would be hosted on a providers'
  [mainframe](Mainframe_computers.md) or mini-computer, which the consumer would
  access through something like time-sharing?)

  ![BT telephone modem](../img/modem-bt.png)

  The mechanism of connection was via a modem. A device that connected to your
  phone's cradle. Digital signals from your home computer would be converted into
  sound signals ("tones") that would travel through the phone transmitter to a
  receiver at the service provider which you would "dial-up" beforehand.

  In the US the main provider was Micronet in the 1970s, which became Compuserve
  in the 1980s and eventually AOL in the 1990s. In the UK there was Prestel
  provided by BT, first introduced in 1971.

  These services had an expensive subscription model and consequently uptake was
  limited.

  In addition there were bulletin board systems (BBS), typically administrated by
  hobbyists. Their personal computers acted as servers that other users would
  dial-up to access. They permitted the sharing of news and messages and later
  chat-rooms for realtime communication. Some of the biggest BBSs later became the
  first ISPs.

metadata:
  {"created":"Monday, October 14, 2024","tags":["fleeting","modems","computer-history","internet","bulletin-boards"]}


path: Web_sockets.md
sortable_path: Web_sockets.md
title: Web sockets
checksum: 6327ad24c4c2eb8a289adf5f07f71aef87627fc2b63b47302f070e087df7d875
created: 2025-10-01 13:31:03.018984009+00:00
modified: 2025-08-13 15:57:13.992002231+00:00
metadata: {"created":"Monday, April 14, 2025","tags":["networks","network-protocols","WebSocket"]}
lead:
  The WebSocket protocol provides a mechanism whereby data can continuously be
  shared between a client and a server in both directions.

body:
  The WebSocket protocol provides a mechanism whereby data can continuously be
  shared between a client and a server in both directions.

  This contrasts with standard HTTP whereby a server can only send data when a
  client specifically requests it and the communication channel is closed until
  such a request is made.

  Whereas HTTP works on the basis of a client-server architecture, WebSocket is an
  event-driven architecture. Under an event-driven architecture, _events_ trigger
  data flows rather than client requests.

  Examples of its application include instant messaging platforms where messages
  are sent in real time, stock trading platforms, social media feeds and
  cloud-based collaboration tools (e.g. Google Sheets). Hence WebSocket is best
  suited to applications where the immediacy and concurrency of server resolution
  is a factor.

  WebSocket was borne out of the limitations of HTTP. Prior to the creation of the
  WebSocket protocol, event-driven communication was attempted over HTTP through
  "long polling". Basically, the server sends a request to the server and if the
  response is not available, the server holds the request until the response is
  available and then returns it. Then, after an interval, the client sends the
  same request again and the process repeats. This is obviously inefficient.

  WebSocket is an [Application Layer](./Application_Layer_of_Internet_Protocol.md)
  protocol just like HTTP. HTTP is used as the initial connection mechanism but
  the resulting TCP connection is kept alive after the HTTP request completes,
  establishing the web socket. The standard HTTP handshake takes place but the
  HTTP request will include an 'Upgrade' header that indicates the client wants to
  establish a socket. The presence of this header results in the protocol being
  upgraded from HTTP to WebSocket.

  > This is possible because both HTTP and WebSocket are Application Layer
  > protocols that run on top of the same TCP connection.

  ## Encryption

  In the case of moving from HTTPS to WebSocket, the encryption of the former
  protocol is maintained. Although there is a protocol switch, the data is still
  being transferred on the same 443 port (used for HTTPS). Although the protocol
  has changed at the Application Layer, the encryption established at the TCP
  Layer is still in place.

  The HTTPS analog for WebSocket is WebSocketSecure (WSS). Thus HTTPS/WSS is the
  secure version of the HTTP/WS protocols.

raw_content:
  ---
  tags: [networks, network-protocols, WebSocket]
  created: Monday, April 14, 2025
  ---

  # Web sockets

  The WebSocket protocol provides a mechanism whereby data can continuously be
  shared between a client and a server in both directions.

  This contrasts with standard HTTP whereby a server can only send data when a
  client specifically requests it and the communication channel is closed until
  such a request is made.

  Whereas HTTP works on the basis of a client-server architecture, WebSocket is an
  event-driven architecture. Under an event-driven architecture, _events_ trigger
  data flows rather than client requests.

  Examples of its application include instant messaging platforms where messages
  are sent in real time, stock trading platforms, social media feeds and
  cloud-based collaboration tools (e.g. Google Sheets). Hence WebSocket is best
  suited to applications where the immediacy and concurrency of server resolution
  is a factor.

  WebSocket was borne out of the limitations of HTTP. Prior to the creation of the
  WebSocket protocol, event-driven communication was attempted over HTTP through
  "long polling". Basically, the server sends a request to the server and if the
  response is not available, the server holds the request until the response is
  available and then returns it. Then, after an interval, the client sends the
  same request again and the process repeats. This is obviously inefficient.

  WebSocket is an [Application Layer](./Application_Layer_of_Internet_Protocol.md)
  protocol just like HTTP. HTTP is used as the initial connection mechanism but
  the resulting TCP connection is kept alive after the HTTP request completes,
  establishing the web socket. The standard HTTP handshake takes place but the
  HTTP request will include an 'Upgrade' header that indicates the client wants to
  establish a socket. The presence of this header results in the protocol being
  upgraded from HTTP to WebSocket.

  > This is possible because both HTTP and WebSocket are Application Layer
  > protocols that run on top of the same TCP connection.

  ## Encryption

  In the case of moving from HTTPS to WebSocket, the encryption of the former
  protocol is maintained. Although there is a protocol switch, the data is still
  being transferred on the same 443 port (used for HTTPS). Although the protocol
  has changed at the Application Layer, the encryption established at the TCP
  Layer is still in place.

  The HTTPS analog for WebSocket is WebSocketSecure (WSS). Thus HTTPS/WSS is the
  secure version of the HTTP/WS protocols.


path: What_are_disks.md
sortable_path: What_are_disks.md
title: What are disks?
lead:
  A disk is a mass storage [block_device](Devices.md) which we
  can write to and read from.

checksum: e2afba648e34963a7b5c9f4a8b9bb857ce1540de0997c2f7291479de9d905c5b
created: 2025-10-01 13:31:03.035871863+00:00
modified: 2025-08-13 15:57:14.061851468+00:00
metadata: {"tags":["disks","computer-architecture"]}
body:
  A disk is a mass storage [block_device](Devices.md) which we
  can write to and read from.

  ## SCSI

  - Small Computer System Interface, responsible for handling disk access on most
    Linux systems.
  - It is a protocol that allows communicaton between printers, scanners and other
    peripherals in addition to harddisks.

  ## Disk schematic

  The following diagram represents the basic anatomy of a disk device.

  ![](/img/harddisk.png)

  - A disk is divided up into [partitions](Partitions.md)
    which are subsections of the overall disk. The kernel presents each partition
    as a [block device](Devices.md) as it would with an entire
    disk.
  - The disk dedicates a small part of its contents to a **partition table**: this
    defines the different partitions that comprise the total disk space.
  - The **filesystem** is a database of files and directories: this comprises the
    bulk of the partition and is what you interact with in
    [user space](User_Space.md) when reading and writing data.

raw_content:
  ---
  tags:
    - disks
    - computer-architecture
  ---

  # What are disks?

  A disk is a mass storage [block_device](Devices.md) which we
  can write to and read from.

  ## SCSI

  - Small Computer System Interface, responsible for handling disk access on most
    Linux systems.
  - It is a protocol that allows communicaton between printers, scanners and other
    peripherals in addition to harddisks.

  ## Disk schematic

  The following diagram represents the basic anatomy of a disk device.

  ![](/img/harddisk.png)

  - A disk is divided up into [partitions](Partitions.md)
    which are subsections of the overall disk. The kernel presents each partition
    as a [block device](Devices.md) as it would with an entire
    disk.
  - The disk dedicates a small part of its contents to a **partition table**: this
    defines the different partitions that comprise the total disk space.
  - The **filesystem** is a database of files and directories: this comprises the
    bulk of the partition and is what you interact with in
    [user space](User_Space.md) when reading and writing data.


path: What_can_the_ISP_see.md
sortable_path: What_can_the_ISP_see.md
title: What can the ISP see?
lead: ## Plain HTTP
checksum: 992a0ae00256f2ee6644480b697656300e6b8b22ecb34c4a05dae060081b24f2
created: 2025-10-01 13:31:03.066940955+00:00
modified: 2025-08-13 15:57:13.81359371+00:00
metadata: {"created":"Friday, December 13, 2024","tags":["internet","encryption","privacy","Tor"]}
body:
  ## Plain HTTP

  The ISP can see all content exchanged: DNS lookups, IP address of the server you
  connect to, requests and body content.

  ## Just HTTPS

  With HTTPS alone, the content of the messages is encrypted but the ISP can see:

  - DNS queries
  - IP addresses you connect to
  - Traffic patterns and metadata (when, how often, amount of data exchanged etc)

  ## HTTPS and encrypted DNS server

  With HTTPS and an encrypted DNS server (e.g. Quad9) the DNS queries are hidden
  but the IP addresses are not. So they could still derive your DNS lookups from
  the IPs you end up connecting to.

  ## HTTPS and VPN

  The ISP only sees the encrypted traffic to the VPN server. Your real IP and
  destination IP is hidden.

  However the VPN provider can potentially see DNS queries, depending on their
  policy. Although a good VPN provider will encrypt DNS lookups like an encrypted
  DNS server.

  The VPN can also still see what IPs you are connecting to. You are basically
  swiching trust from the ISP to the VPN provider. However reputable VPNs do not
  have your billing and identity info (and payment can use cryptocurrencies) and
  don't keep logs. Many also have policies about refusal to cooporate with
  authorities in different juristictions. Also the VPN doesn't know your real IP.

  ## HTTPS, VPN, and encrypted DNS

  DNS queries hidded from both ISP and VPN. Traffic is fully encrypted and routed
  through VPN.

  ## Fingerprinting

  Note that none of the above stops fingerprinting. Even with full encryption
  there are methods of determining online behaviour through encrypted packet
  analysis and usage patterns.

  A way round this is to use SOCKS proxies which add hops or to use TOR. TOR
  effectively solves the "have to trust VPN provider" problem.

  With TOR:

  - Traffic routed through 3+ nodes
  - Each node only knows previous and next hop
  - No single node knows both source and destination. Even Tor nodes can't see the
    full path

  It also uses:

  - Fixed packet sizes
  - Timing obfuscation
  - Traffic padding

  Which makes pattern analysis and fingerprinting much harder than just a VPN.

  Furthermore there is also the following protection from VPN weaknesses:

  - No single provider to trust/compromise
  - Decentralized network
  - Free to use and no account needed
  - No permanent exit node like with VPN

raw_content:
  ---
  tags: [internet, encryption, privacy, Tor]
  created: Friday, December 13, 2024
  ---

  # What can the ISP see?

  ## Plain HTTP

  The ISP can see all content exchanged: DNS lookups, IP address of the server you
  connect to, requests and body content.

  ## Just HTTPS

  With HTTPS alone, the content of the messages is encrypted but the ISP can see:

  - DNS queries
  - IP addresses you connect to
  - Traffic patterns and metadata (when, how often, amount of data exchanged etc)

  ## HTTPS and encrypted DNS server

  With HTTPS and an encrypted DNS server (e.g. Quad9) the DNS queries are hidden
  but the IP addresses are not. So they could still derive your DNS lookups from
  the IPs you end up connecting to.

  ## HTTPS and VPN

  The ISP only sees the encrypted traffic to the VPN server. Your real IP and
  destination IP is hidden.

  However the VPN provider can potentially see DNS queries, depending on their
  policy. Although a good VPN provider will encrypt DNS lookups like an encrypted
  DNS server.

  The VPN can also still see what IPs you are connecting to. You are basically
  swiching trust from the ISP to the VPN provider. However reputable VPNs do not
  have your billing and identity info (and payment can use cryptocurrencies) and
  don't keep logs. Many also have policies about refusal to cooporate with
  authorities in different juristictions. Also the VPN doesn't know your real IP.

  ## HTTPS, VPN, and encrypted DNS

  DNS queries hidded from both ISP and VPN. Traffic is fully encrypted and routed
  through VPN.

  ## Fingerprinting

  Note that none of the above stops fingerprinting. Even with full encryption
  there are methods of determining online behaviour through encrypted packet
  analysis and usage patterns.

  A way round this is to use SOCKS proxies which add hops or to use TOR. TOR
  effectively solves the "have to trust VPN provider" problem.

  With TOR:

  - Traffic routed through 3+ nodes
  - Each node only knows previous and next hop
  - No single node knows both source and destination. Even Tor nodes can't see the
    full path

  It also uses:

  - Fixed packet sizes
  - Timing obfuscation
  - Traffic padding

  Which makes pattern analysis and fingerprinting much harder than just a VPN.

  Furthermore there is also the following protection from VPN weaknesses:

  - No single provider to trust/compromise
  - Decentralized network
  - Free to use and no account needed
  - No permanent exit node like with VPN


path: What_is_memory.md
sortable_path: What_is_memory.md
title: What is memory ?
checksum: a8bdfe8e8fba49ccf433d7a0d90744c2b434915a5f1a4206998c53908a9d521a
created: 2025-10-01 13:31:03.074194441+00:00
modified: 2025-08-13 15:57:13.765647934+00:00
metadata: {"created":"Friday, July 12, 2024","tags":["memory"]}
lead:
  > A CPU is just an operator on memory. It reads its instructions and data from
  > the memory and writes back out to the memory. (Ward 2021)

body:
  > A CPU is just an operator on memory. It reads its instructions and data from
  > the memory and writes back out to the memory. (Ward 2021)

  When a [CPU](CPU_architecture.md) executes a program, it needs a place to store
  the program's **instructions** and **related data**. This is the role of memory.

  The data that comprises a program is a series of bits. The basic unit of memory
  storage is a **memory cell**: a circuit that can store a single bit.

  [RAM](./DRAM_and_SRAM_memory.md) memory is _volatile_ : the memory is only
  retained whilst the computer has a power supply and is wiped when the computer
  is rebooted. This contrasts with the memory of the harddisk which is
  non-volatile and is retained after a reboot.

  Programs that are executing are loaded into memory because the chips that
  comprise memory can read and store data much faster than the harddisk. It would
  be possible to run a program from the harddisk but it would be 500 - 1000 times
  slower than memory.

raw_content:
  ---
  tags: [memory]
  created: Friday, July 12, 2024
  ---

  # What is memory ?

  > A CPU is just an operator on memory. It reads its instructions and data from
  > the memory and writes back out to the memory. (Ward 2021)

  When a [CPU](CPU_architecture.md) executes a program, it needs a place to store
  the program's **instructions** and **related data**. This is the role of memory.

  The data that comprises a program is a series of bits. The basic unit of memory
  storage is a **memory cell**: a circuit that can store a single bit.

  [RAM](./DRAM_and_SRAM_memory.md) memory is _volatile_ : the memory is only
  retained whilst the computer has a power supply and is wiped when the computer
  is rebooted. This contrasts with the memory of the harddisk which is
  non-volatile and is retained after a reboot.

  Programs that are executing are loaded into memory because the chips that
  comprise memory can read and store data much faster than the harddisk. It would
  be possible to run a program from the harddisk but it would be 500 - 1000 times
  slower than memory.


path: Whole_numbers.md
sortable_path: Whole_numbers.md
title: The set of whole numbers
lead: We recall the set of whole numbers:
checksum: 55a213e7c8d3fd57417fb5291e2452abb12fdc19266c9a6c00130544e5b85789
created: 2025-10-01 13:31:03.093495158+00:00
modified: 2025-08-13 15:57:14.031733679+00:00
metadata: {"tags":["prealgebra"]}
body:
  We recall the set of whole numbers:

  $$ \mathbb{W} = {0, 1, 2, 3, ...} $$

  # The properties of $\mathbb{W}$

  > In mathematics, a **property** is any characteristic that applies to a given
  > set.

  ## The commutative property

  ### Addition

  When **adding** whole numbers, the placement of the addends does not affect the
  sum.

  Let **a**, **b** represent whole numbers, then:

  $$ a + b = b + a $$

  ### Multiplication

  When **multiplying** whole numbers the placement of the
  [multiplicands](Symbols-and-formal-conventions-80aeaf1872f94a0d97a2e8d07e3855bd)
  does not affect the
  [product](Symbols-and-formal-conventions-80aeaf1872f94a0d97a2e8d07e3855bd).

  Let **a, b** represent whole numbers, then:

  $$ a \cdot b = b \cdot a $$

  ### Subtraction

  **Subtraction** is not commutative, viz:

  $$ a - b \neq b - a $$

  ### Division

  Division is not commutative, viz:

  $$ a \div b \neq b \div a $$

  ## The associative property

  ### Addition

  When grouping symbols (parentheses, brackets, braces) are used with addition,
  the particular placement of the grouping symbols relative to each of the addends
  does not change the sum.

  Let **a**, **b, c** represent whole numbers, then:

  $$ (a + b) + c = a + (b + c) $$

  ### Multiplication

  Let **a, b, c** represent whole numbers, then:

  $$ a \cdot (b \cdot c) = (a \cdot b) \cdot c $$

  ### Subtraction

  Subtraction is not associative, viz:

  $$ (a - b) - c \neq a - (b - c) $$

  ### Division

  Division is not associative

  $$ (a \div b) \div c \neq a \div (b \div c) $$

  ## The property of additive identity

  If **a** is any whole number, then:

  $$ a + 0 = a $$

  We therefore call zero the additive identity: whenever we add zero to a whole
  number, the sum is equal to the whole number itself.

  ## The property of multiplicative identity

  If **a** is any whole number, then:

  $$ (a \cdot 1 = a) = (1 \cdot a = a) $$

  ## Multiplication by zero

  If **a** is any whole number, then:

  $$ (a \cdot 0 = 0) = (0 \cdot a = 0) $$

  ## Division by zero

  Division by zero is **undefined** but zero divided is zero.

raw_content:
  ---
  tags:
    - prealgebra
  ---

  # The set of whole numbers

  We recall the set of whole numbers:

  $$ \mathbb{W} = {0, 1, 2, 3, ...} $$

  # The properties of $\mathbb{W}$

  > In mathematics, a **property** is any characteristic that applies to a given
  > set.

  ## The commutative property

  ### Addition

  When **adding** whole numbers, the placement of the addends does not affect the
  sum.

  Let **a**, **b** represent whole numbers, then:

  $$ a + b = b + a $$

  ### Multiplication

  When **multiplying** whole numbers the placement of the
  [multiplicands](Symbols-and-formal-conventions-80aeaf1872f94a0d97a2e8d07e3855bd)
  does not affect the
  [product](Symbols-and-formal-conventions-80aeaf1872f94a0d97a2e8d07e3855bd).

  Let **a, b** represent whole numbers, then:

  $$ a \cdot b = b \cdot a $$

  ### Subtraction

  **Subtraction** is not commutative, viz:

  $$ a - b \neq b - a $$

  ### Division

  Division is not commutative, viz:

  $$ a \div b \neq b \div a $$

  ## The associative property

  ### Addition

  When grouping symbols (parentheses, brackets, braces) are used with addition,
  the particular placement of the grouping symbols relative to each of the addends
  does not change the sum.

  Let **a**, **b, c** represent whole numbers, then:

  $$ (a + b) + c = a + (b + c) $$

  ### Multiplication

  Let **a, b, c** represent whole numbers, then:

  $$ a \cdot (b \cdot c) = (a \cdot b) \cdot c $$

  ### Subtraction

  Subtraction is not associative, viz:

  $$ (a - b) - c \neq a - (b - c) $$

  ### Division

  Division is not associative

  $$ (a \div b) \div c \neq a \div (b \div c) $$

  ## The property of additive identity

  If **a** is any whole number, then:

  $$ a + 0 = a $$

  We therefore call zero the additive identity: whenever we add zero to a whole
  number, the sum is equal to the whole number itself.

  ## The property of multiplicative identity

  If **a** is any whole number, then:

  $$ (a \cdot 1 = a) = (1 \cdot a = a) $$

  ## Multiplication by zero

  If **a** is any whole number, then:

  $$ (a \cdot 0 = 0) = (0 \cdot a = 0) $$

  ## Division by zero

  Division by zero is **undefined** but zero divided is zero.


path: Why_computers_use_binary.md
sortable_path: Why_computers_use_binary.md
title: Why computers use binary
lead: ## Now we know how binary works, how does it relate to computing?
checksum: f1e48d431648ff5f081d982bd3cde7673eb114fa43c6c62f39f4d8e9f23817f3
created: 2025-10-01 13:31:03.141500757+00:00
modified: 2025-08-13 15:57:14.01682214+00:00
metadata: {"tags":["binary","computer-architecture"]}
body:
  ## Now we know how binary works, how does it relate to computing?

  The reason is straight forward: it is the simplest way on the level of pure
  engineering of representing numerical and logical values; both of which are the
  basic foundations of programming. An electronic circuit or transistor only needs
  to represent two states: on (1) and off (0) which corresponds to the switch on
  an electrical circuit. A single circuit representing the binary values of 1 and
  0:

  ![multi_on_off 1.gif](multi_on_off.gif)

  It would be much more complicated to have to represent ten different states
  under the decimal number system, although denary computers do exist.

  If we want more digits, we just need to add in more circuits, and we can
  represent as large a binary number as we need. We just need one switch for every
  digit we want to represent. The switches used in modern computers are so cheap
  and so small that billions can be fitted on a single circuit board.

  ![multiple_circuits.gif](multiple_circuits.gif)

  When we use the term 'switch' we actually mean the transistor components of a
  circuit. We don't need to know the physical details at this level but we can say
  that a transistor turns a current on and off. They can also be used to amplify
  the current.

  On the level of electrical engineering and in the subsequent examples using a
  light bulb and a simple circuit, the switch being OFF corresponds to the light
  being ON, this is because the switch breaks the current. So when it is unbroken,
  the current passes to the bulb. This also means that 1 corresponds to the switch
  being OFF and 0 corresponds to the switch being ON which can be confusing at
  first.

  ## From circuits to programs

  The following breaks down how we get from the binary number system to electrical
  circuits to computer programs:

  1. ”Data”= a piece or pieces of **information**

  1. In computing all data is represented in **binary form:**

     2.1. ”Binary” means that the value of a piece of data is exclusively one
     thing or another

     2.2. The values in binary code are exclusively 1 and 0 (either a piece of
     data is a 1 or it is a 0, it can never be both). Binary can also be expressed
     in logical terms where 1 is equal to `true` and 0 is equal to `false`

  1. The smallest piece of data is a **bit**

  1. We apply the binary number system to bits

  1. Thus by 1-4, **a bit is either 1 or 0**

  1. Binary form bears an isomorphic relation to switches on electrical circuits
     (the hardware of computers). Thus binary code can be _mapped onto circuits_.

     - Circuits are controlled by switches
     - A switch is a binary function: it is either on or off ”On/off”
       corresponding to 1/0.
     - In this way, switches control the flow of electric current

  1. Thus by the above, hardware (which is a physical structure made up of
     electric circuits) is **readily programmable in binary terms**

  1. Restatement of 6: in order for hardware to be programmed the program must
     provide the hardware with something that it can actually perform. The program
     must be formulated in binary code.

raw_content:
  ---
  tags:
    - binary
    - computer-architecture
  ---

  # Why computers use binary

  ## Now we know how binary works, how does it relate to computing?

  The reason is straight forward: it is the simplest way on the level of pure
  engineering of representing numerical and logical values; both of which are the
  basic foundations of programming. An electronic circuit or transistor only needs
  to represent two states: on (1) and off (0) which corresponds to the switch on
  an electrical circuit. A single circuit representing the binary values of 1 and
  0:

  ![multi_on_off 1.gif](multi_on_off.gif)

  It would be much more complicated to have to represent ten different states
  under the decimal number system, although denary computers do exist.

  If we want more digits, we just need to add in more circuits, and we can
  represent as large a binary number as we need. We just need one switch for every
  digit we want to represent. The switches used in modern computers are so cheap
  and so small that billions can be fitted on a single circuit board.

  ![multiple_circuits.gif](multiple_circuits.gif)

  When we use the term 'switch' we actually mean the transistor components of a
  circuit. We don't need to know the physical details at this level but we can say
  that a transistor turns a current on and off. They can also be used to amplify
  the current.

  On the level of electrical engineering and in the subsequent examples using a
  light bulb and a simple circuit, the switch being OFF corresponds to the light
  being ON, this is because the switch breaks the current. So when it is unbroken,
  the current passes to the bulb. This also means that 1 corresponds to the switch
  being OFF and 0 corresponds to the switch being ON which can be confusing at
  first.

  ## From circuits to programs

  The following breaks down how we get from the binary number system to electrical
  circuits to computer programs:

  1. ”Data”= a piece or pieces of **information**

  1. In computing all data is represented in **binary form:**

     2.1. ”Binary” means that the value of a piece of data is exclusively one
     thing or another

     2.2. The values in binary code are exclusively 1 and 0 (either a piece of
     data is a 1 or it is a 0, it can never be both). Binary can also be expressed
     in logical terms where 1 is equal to `true` and 0 is equal to `false`

  1. The smallest piece of data is a **bit**

  1. We apply the binary number system to bits

  1. Thus by 1-4, **a bit is either 1 or 0**

  1. Binary form bears an isomorphic relation to switches on electrical circuits
     (the hardware of computers). Thus binary code can be _mapped onto circuits_.

     - Circuits are controlled by switches
     - A switch is a binary function: it is either on or off ”On/off”
       corresponding to 1/0.
     - In this way, switches control the flow of electric current

  1. Thus by the above, hardware (which is a physical structure made up of
     electric circuits) is **readily programmable in binary terms**

  1. Restatement of 6: in order for hardware to be programmed the program must
     provide the hardware with something that it can actually perform. The program
     must be formulated in binary code.


path: Wildcards_in_SQL.md
sortable_path: Wildcards_in_SQL.md
title: Wildcards in SQL
lead:
  SQL does not use Regex. Instead it has a simpler glob-like syntax for carrying
  out string matching.

checksum: b761196c7c6f74fc89c73b5473e5d14148d6db91415904bc5456d190063aa12f
created: 2025-10-01 13:31:03.148277712+00:00
modified: 2025-08-13 15:57:13.823632161+00:00
metadata: {"tags":["SQL","databases"]}
body:
  SQL does not use Regex. Instead it has a simpler glob-like syntax for carrying
  out string matching.

  In order to signal that you wish to compare by a wildcard and not a value, you
  have to use the `LIKE` keyword. The actual wildcard operator is `%` .

  In an SQL statement, the `%` wild card will match any number of occurrences of
  any character. Any characters can appear before or after ‘MacBook’ and the
  record will still be returned:

  ```sql
  SELECT name, cores, release_date
  FROM model
  WHERE name LIKE '%MacBook%';
  ```

  This wildcard only filters characters that come after `Raspberry` :

  ```sql
  SELECT name, cores, release_date
  FROM model
  WHERE name LIKE 'Raspberry%';
  ```

raw_content:
  ---
  tags:
    - SQL
    - databases
  ---

  # Wildcards in SQL

  SQL does not use Regex. Instead it has a simpler glob-like syntax for carrying
  out string matching.

  In order to signal that you wish to compare by a wildcard and not a value, you
  have to use the `LIKE` keyword. The actual wildcard operator is `%` .

  In an SQL statement, the `%` wild card will match any number of occurrences of
  any character. Any characters can appear before or after ‘MacBook’ and the
  record will still be returned:

  ```sql
  SELECT name, cores, release_date
  FROM model
  WHERE name LIKE '%MacBook%';
  ```

  This wildcard only filters characters that come after `Raspberry` :

  ```sql
  SELECT name, cores, release_date
  FROM model
  WHERE name LIKE 'Raspberry%';
  ```


path: Williams_Tube_memory.md
sortable_path: Williams_Tube_memory.md
title: Williams Tube memory
checksum: cafd2be17ab55aa9487b07d6eb6cc4b0f994a323b35a2654af5858c733326885
created: 2025-10-01 13:31:03.156499299+00:00
modified: 2025-08-13 15:57:13.888548802+00:00
metadata: {"created":"Tuesday, September 17, 2024","tags":["computer-history","memory"]}
lead:
  - One of the early attempts to create RAM in computers based on the EDVAC
    stored-program architecture.

body:
  - One of the early attempts to create RAM in computers based on the EDVAC
    stored-program architecture.

  - Worked by displaying a grid of dots on a cathode-ray tube ("screen")

  - Due to the way CRTs work, this creates a small charge of static electricity
    over each dot. The charge at the location of each of the dots is read by a
    thin metal sheet just in front of the display.

  - Each dot position could be written to and read from and the pattern was
    constantly refreshed as the dots would fade over time.

  ![A Williams CRT tube](../img/williams-tube.jpg)

  ![Memory dot pattern from a Williams Tube](../img/williams-tube-dots.jpg)

raw_content:
  ---
  tags: [computer-history, memory]
  created: Tuesday, September 17, 2024
  ---

  # Williams Tube memory

  - One of the early attempts to create RAM in computers based on the EDVAC
    stored-program architecture.

  - Worked by displaying a grid of dots on a cathode-ray tube ("screen")

  - Due to the way CRTs work, this creates a small charge of static electricity
    over each dot. The charge at the location of each of the dots is read by a
    thin metal sheet just in front of the display.

  - Each dot position could be written to and read from and the pattern was
    constantly refreshed as the dots would fade over time.

  ![A Williams CRT tube](../img/williams-tube.jpg)

  ![Memory dot pattern from a Williams Tube](../img/williams-tube-dots.jpg)


path: With_open_in_Python.md
sortable_path: With_open_in_Python.md
title: With open in Python
checksum: 2de8f5b1da55138664ebf60cb770e8e97086f2fd1e888d75313b2c6ef83c26bc
created: 2025-10-01 13:31:03.169085679+00:00
modified: 2025-08-13 15:57:13.923973251+00:00
metadata: {"created":"Friday, October 25, 2024","tags":["python"]}
lead:
  We use the `open()` method to create a **file object** that allows us to read,
  write and append to files.

body:
  We use the `open()` method to create a **file object** that allows us to read,
  write and append to files.

  The general syntax is as follows:

  ```py
  file_object = open(<file_name>, <access_mode>)
  ```

  `<file_name>` is a path to the file you want to read, create or modify. The
  `<access_mode>` denotes the mode in which to open the file. The most frequently
  used are:

  - `r`
    - [read](Reading_files_in_Python.md)
  - `w`
    - [write](Writing_to_files_in_Python.md)
  - `a`
    - [append](Appending_to_files_in_Python.md)

  When we have finished with an I/O operation, such as reading from or writing to
  a file, we must call `file.close()` to terminate the process. This removes the
  reference to the file from memory.

  A more pythonic and concise way of reading files and closing them is to use
  `with...as` syntax. When this phrasing is used, a self-contained context is
  created for the I/O operation that closes the file automatically.

  ```py
  with open('filename.txt', 'r') as file:
      contents = file.read()
      print(contents)
  ```

raw_content:
  ---
  tags: [python]
  created: Friday, October 25, 2024
  ---

  # With open in Python

  We use the `open()` method to create a **file object** that allows us to read,
  write and append to files.

  The general syntax is as follows:

  ```py
  file_object = open(<file_name>, <access_mode>)
  ```

  `<file_name>` is a path to the file you want to read, create or modify. The
  `<access_mode>` denotes the mode in which to open the file. The most frequently
  used are:

  - `r`
    - [read](Reading_files_in_Python.md)
  - `w`
    - [write](Writing_to_files_in_Python.md)
  - `a`
    - [append](Appending_to_files_in_Python.md)

  When we have finished with an I/O operation, such as reading from or writing to
  a file, we must call `file.close()` to terminate the process. This removes the
  reference to the file from memory.

  A more pythonic and concise way of reading files and closing them is to use
  `with...as` syntax. When this phrasing is used, a self-contained context is
  created for the I/O operation that closes the file automatically.

  ```py
  with open('filename.txt', 'r') as file:
      contents = file.read()
      print(contents)
  ```


path: Working_with_CSVs_in_Python.md
sortable_path: Working_with_CSVs_in_Python.md
title: Working with CSVs in Python
lead: ## Core package
checksum: c766eccad8a6cb1d8f8a7115546fc6aa87c9e45ed09e22a331be092bbb2e2d56
created: 2025-10-01 13:31:03.204552542+00:00
modified: 2025-08-13 15:57:14.026791318+00:00
metadata: {"created":"Sunday, April 28, 2024","id":"sgtn","tags":["python","csv"]}
body:
  ## Core package

  ```py
  import csv
  ```

  ## Read and write to CSV

  ### Read

  Use standard Pythonic "read" syntax:

  ```py
  with open('./path.csv', mode="r") as csv_file:
      reader = csv.reader(csv_file)
  ```

  ### Parse values

  Having created the `reader` object, you can then loop through this as an
  iterable:

  ```py
  for row in reader:
      print(row)
  ```

  Will return something like:

  ```csv
  column something, column something else, ...
  ```

  Which we can individuate with:

  ```py
  print(row[0])
  # column something
  ```

  We can also parse the rows as a dictionary for easier individuation. We do this
  by creating a `DictReader` rather than the default `reader`:

  ```py
  ...
  dict_reader = csv.DictReader(csv_file)
  ```

  Now we can use the header row values to individuate particular columns.

  Say we have a CSV with the following headers:

  ```csv
  name, profession
  ```

  We can individuate thus:

  ```py
  for row in dict_reader
      name = row["name"]
  ```

  ### Write

  Use standard Pythonic "read" syntax:

  ```py
  with open('./new_csv_file.csv', mode="w") as csv_file:
      writer = csv.writer(csv_file)
  ```

  The above will create the file as well, it doesn't need to be pre-existing.

  This creates the writer object. To actually write a row:

  ```py
  some_list = ['thomas', 'engineer']
  for element in some_list:
      writer.writerow(element)
  ```

raw_content:
  ---
  id: sgtn
  tags: [python, csv]
  created: Sunday, April 28, 2024
  ---

  # Working with CSVs in Python

  ## Core package

  ```py
  import csv
  ```

  ## Read and write to CSV

  ### Read

  Use standard Pythonic "read" syntax:

  ```py
  with open('./path.csv', mode="r") as csv_file:
      reader = csv.reader(csv_file)
  ```

  ### Parse values

  Having created the `reader` object, you can then loop through this as an
  iterable:

  ```py
  for row in reader:
      print(row)
  ```

  Will return something like:

  ```csv
  column something, column something else, ...
  ```

  Which we can individuate with:

  ```py
  print(row[0])
  # column something
  ```

  We can also parse the rows as a dictionary for easier individuation. We do this
  by creating a `DictReader` rather than the default `reader`:

  ```py
  ...
  dict_reader = csv.DictReader(csv_file)
  ```

  Now we can use the header row values to individuate particular columns.

  Say we have a CSV with the following headers:

  ```csv
  name, profession
  ```

  We can individuate thus:

  ```py
  for row in dict_reader
      name = row["name"]
  ```

  ### Write

  Use standard Pythonic "read" syntax:

  ```py
  with open('./new_csv_file.csv', mode="w") as csv_file:
      writer = csv.writer(csv_file)
  ```

  The above will create the file as well, it doesn't need to be pre-existing.

  This creates the writer object. To actually write a row:

  ```py
  some_list = ['thomas', 'engineer']
  for element in some_list:
      writer.writerow(element)
  ```


path: Working_with_JSON_in_Python.md
sortable_path: Working_with_JSON_in_Python.md
title: Working with JSON in Python
lead: ## `json.loads()`
checksum: a741123a59d7b096529b7c0b930d9b2e0c7d25772c245a7108541f8fc8d29e26
created: 2025-10-01 13:31:03.210895028+00:00
modified: 2025-08-13 15:57:14.075928027+00:00
metadata: {"created":"Tuesday, June 25, 2024","tags":["python","JSON"]}
body:
  ## `json.loads()`

  Convert a string containing JSON data into the native Python dictionary object.
  Equivalent to `JSON.parse()` in JavaScript.

  Will error if the input string is not properly formatted JSON. This will be
  `json.JSONDecodeError`, a subclass of the `ValueError`
  [exception](./Error_handling_in_Python.md)

  ## `json.dumps()`

  Take a JSON-formatted dictionary and convert it into a string. The reverse of
  `json.loads()` and equivalent to `JSON.stringify()` in JavaScript.

  ## `json.dump()`

  Not to be confused with the above. Output a Python dictionary as JSON, for
  instance to create a `.json` file:

  ```py
  data = { "key": "value"}
  with open("some/directory/output.json", "w") as f:
      json.dump(data, f, indent=4)
  ```

raw_content:
  ---
  tags: [python, JSON]
  created: Tuesday, June 25, 2024
  ---

  # Working with JSON in Python

  ## `json.loads()`

  Convert a string containing JSON data into the native Python dictionary object.
  Equivalent to `JSON.parse()` in JavaScript.

  Will error if the input string is not properly formatted JSON. This will be
  `json.JSONDecodeError`, a subclass of the `ValueError`
  [exception](./Error_handling_in_Python.md)

  ## `json.dumps()`

  Take a JSON-formatted dictionary and convert it into a string. The reverse of
  `json.loads()` and equivalent to `JSON.stringify()` in JavaScript.

  ## `json.dump()`

  Not to be confused with the above. Output a Python dictionary as JSON, for
  instance to create a `.json` file:

  ```py
  data = { "key": "value"}
  with open("some/directory/output.json", "w") as f:
      json.dump(data, f, indent=4)
  ```


path: Working_with_directories_in_Python.md
sortable_path: Working_with_directories_in_Python.md
title: Working with directories in Python
lead: ## List files in a directory
checksum: a05534065f977b519f1cf4974936cbf4100d340585f1a18b64c34adea171038c
created: 2025-10-01 13:31:03.224601197+00:00
modified: 2025-08-13 15:57:13.868315248+00:00
metadata: {"created":"Friday, October 25, 2024","tags":["python","file-system","procedural"]}
body:
  ## List files in a directory

  ```py
  # Here we list the file exts of files in a dir
  import os
  from pathlib import Path

  with os.scandir("/a/directory") as dir:
      for entry in dir:
          print(Path(entry).suffix)
  ```

  ## List files without extension

  ```python

  file = "/some/file/with/markdown.md"
  return os.path.splitext(os.path.basename(file))[0]
  # markdown
  ```

  ## Create a directory

  ```py
  import os

  os.makedirs("/name/of/directory")
  ```

  ### Check if directory exists, remove if so

  ```py
  import os
  import shutil

  if os.path.isdir("/name/of/directory")
      shutil.rmtree(target_dir)
  ```

  ## Copy a nested directory from location to another

  ```py
  import shutil

  shutil.copytree("/source/dir", "/target/dir", dir_exist_ok=True)
  ```

raw_content:
  ---
  tags: [python, file-system, procedural]
  created: Friday, October 25, 2024
  ---

  # Working with directories in Python

  ## List files in a directory

  ```py
  # Here we list the file exts of files in a dir
  import os
  from pathlib import Path

  with os.scandir("/a/directory") as dir:
      for entry in dir:
          print(Path(entry).suffix)
  ```

  ## List files without extension

  ```python

  file = "/some/file/with/markdown.md"
  return os.path.splitext(os.path.basename(file))[0]
  # markdown
  ```

  ## Create a directory

  ```py
  import os

  os.makedirs("/name/of/directory")
  ```

  ### Check if directory exists, remove if so

  ```py
  import os
  import shutil

  if os.path.isdir("/name/of/directory")
      shutil.rmtree(target_dir)
  ```

  ## Copy a nested directory from location to another

  ```py
  import shutil

  shutil.copytree("/source/dir", "/target/dir", dir_exist_ok=True)
  ```


path: Working_with_numbers_in_Bash.md
sortable_path: Working_with_numbers_in_Bash.md
title: Working with numbers in Bash
lead: We distinguish:
checksum: d793b757573d5c77ece45d9b81efbc9a285dfe699f8221faf4f0ab9f095c96c5
created: 2025-10-01 13:31:03.229939093+00:00
modified: 2025-08-13 15:57:13.94041508+00:00
metadata: {"tags":["shell"]}
body:
  We distinguish:

  - **arithmetic expansion** `$(( ))`
    - returns the result of literal numbers that can then be stored in a variable
  - **artihmetic evaluation** `(( ))`
    - perform calculations on _existing_ variables

  An example of expansion:

  ```bash
  a=3
  ((a += 3))
  echo $a
  # 6
  ((a++))
  # 7
  ```

  > Note: we do not use a dollar-sign when referring to variables within
  > arithmetic evaluation, there is no need. If we do, we get an error. This is
  > because we are using an
  > [expansion](Expansions_and_substitutions.md),
  > therefore the variables are already being interpreted as variables.

  ## Declaring variables as integers

  It is good practice to safeguard against Bash treating numbers as strings to
  declare them as integers in addition to using arithmetic evaluation, e.g:

  ```bash
  declare -i b=3
  ```

  Whilst this isn't a strict type, it means we can do this:

  ```bash
  b=$b+4
  echo b
  # 7
  ```

  Without getting `3+4` in return

  ## No decimals in bash

  Bash does not support decimal calculations natively. This is what you'd get for
  example:

  ```bash
  echo $(( 1/3 ))
  # 0 (not 0.33)
  ```

  So work with decimals you should use `awk` or `bc` ("basic calculations").

  Example of using `bc`:

  ```bash
  declare -i c=1
  declare -i d=3
  e=$(echo "scale=3; $c/$d" | bc)
  # 0.333
  ```

  ## Random numbers

  Generate a pseudo-random number between 1 and 20:

  ```bash
  echo $(( 1 + RANDOM % 10))
  # 18
  ```

raw_content:
  ---
  tags:
    - shell
  ---

  # Working with numbers in Bash

  We distinguish:

  - **arithmetic expansion** `$(( ))`
    - returns the result of literal numbers that can then be stored in a variable
  - **artihmetic evaluation** `(( ))`
    - perform calculations on _existing_ variables

  An example of expansion:

  ```bash
  a=3
  ((a += 3))
  echo $a
  # 6
  ((a++))
  # 7
  ```

  > Note: we do not use a dollar-sign when referring to variables within
  > arithmetic evaluation, there is no need. If we do, we get an error. This is
  > because we are using an
  > [expansion](Expansions_and_substitutions.md),
  > therefore the variables are already being interpreted as variables.

  ## Declaring variables as integers

  It is good practice to safeguard against Bash treating numbers as strings to
  declare them as integers in addition to using arithmetic evaluation, e.g:

  ```bash
  declare -i b=3
  ```

  Whilst this isn't a strict type, it means we can do this:

  ```bash
  b=$b+4
  echo b
  # 7
  ```

  Without getting `3+4` in return

  ## No decimals in bash

  Bash does not support decimal calculations natively. This is what you'd get for
  example:

  ```bash
  echo $(( 1/3 ))
  # 0 (not 0.33)
  ```

  So work with decimals you should use `awk` or `bc` ("basic calculations").

  Example of using `bc`:

  ```bash
  declare -i c=1
  declare -i d=3
  e=$(echo "scale=3; $c/$d" | bc)
  # 0.333
  ```

  ## Random numbers

  Generate a pseudo-random number between 1 and 20:

  ```bash
  echo $(( 1 + RANDOM % 10))
  # 18
  ```


path: Writing_to_files_in_Python.md
sortable_path: Writing_to_files_in_Python.md
title: Writing to files in Python
lead: We create a file object with `open()` and use the `write` method:
checksum: 8449ca06c0ffa866293f7f3fb684a537bc7be485d9ca62c0c64350fad0555531
created: 2025-10-01 13:31:03.243928865+00:00
modified: 2025-08-13 15:57:13.876539033+00:00
metadata: {"created":"Friday, October 25, 2024","tags":["python","file-system","procedural"]}
body:
  We create a file object with `open()` and use the `write` method:

  ```py
  # Open file in write mode
  file = open("example.txt", "w")

  # Write some text to the file
  file.write("Hello, this is an example text written using Python.")

  # Close the file
  file.close()
  ```

  Alternatively we use `with open` which automatically closes the file:

  ```py

  with open("example.txt", "w") as file
      file.write('some lines')

  ```

  > Note that in the above example, if the file does not already exist, it will
  > create it. If it does exist, it will overwrite its contents with the new data.
  > So we use `write` to create new files as well as to write to existing files.

raw_content:
  ---
  tags: [python, file-system, procedural]
  created: Friday, October 25, 2024
  ---

  # Writing to files in Python

  We create a file object with `open()` and use the `write` method:

  ```py
  # Open file in write mode
  file = open("example.txt", "w")

  # Write some text to the file
  file.write("Hello, this is an example text written using Python.")

  # Close the file
  file.close()
  ```

  Alternatively we use `with open` which automatically closes the file:

  ```py

  with open("example.txt", "w") as file
      file.write('some lines')

  ```

  > Note that in the above example, if the file does not already exist, it will
  > create it. If it does exist, it will overwrite its contents with the new data.
  > So we use `write` to create new files as well as to write to existing files.


path: Zero_property_of_multiplication.md
sortable_path: Zero_property_of_multiplication.md
title: Zero property of multiplication
lead: **Let $a$ represent any member of $\mathbb{W}$ or $\mathbb{Z}$ then:**
body:
  **Let $a$ represent any member of $\mathbb{W}$ or $\mathbb{Z}$ then:**

  $$ a \cdot 0 = 0 $$

checksum: c547a0d6c08c9a206ef1caaadd2811d792cf21109b64772a860ee712d21a01f0
created: 2025-10-01 13:31:03.249936456+00:00
modified: 2025-08-13 15:57:13.795447749+00:00
metadata: {"tags":["prealgebra","theorems"]}
raw_content:
  ---
  tags:
    - prealgebra
    - theorems
  ---

  # Zero property of multiplication

  **Let $a$ represent any member of $\mathbb{W}$ or $\mathbb{Z}$ then:**

  $$ a \cdot 0 = 0 $$


path: Zip_function_in_Python.md
sortable_path: Zip_function_in_Python.md
title: zip
checksum: 0e7191e6c8fbb369de0479b6d8f0ebcd60161e6cf2f4f978eb878aa8436e54a1
created: 2025-10-01 13:31:03.255948607+00:00
modified: 2025-08-13 15:57:13.786776123+00:00
metadata: {"tags":["python","data-structures"]}
lead:
  The `zip` function returns a zip object when you pass in two iterator data types
  (typically two lists). This is an iterator of tuples where the first item in
  each passed iterator is paired together, and then the second item in each passed
  iterator are paired together, and so on.

body:
  The `zip` function returns a zip object when you pass in two iterator data types
  (typically two lists). This is an iterator of tuples where the first item in
  each passed iterator is paired together, and then the second item in each passed
  iterator are paired together, and so on.

  ## Example

  ```py
  a = ("John", "Charles", "Mike")
  b = ("Jenny", "Christy", "Monica", "Vicky")

  x = zip(a,b)

  print(x)
  # <zip object at 0x1514af34c340>

  # To display the resulting tuple in a readable format:

  print(tuple(x))

  (('John', 'Jenny'), ('Charles', 'Christy'), ('Mike', 'Monica'))
  ```

  As indicated above, if the input iterators are of equal lengths, Python will
  ignore the value in iterator B that doesn't have an equivalent index in iterator
  A.

  ## Real-life example

  ```py
  prompts = [
      "Enter title",
      "Enter author",
      "Enter publication date",
      "Enter year you read the book"
  ]

  keys = ["Title", "Author", "Publication_date", "Date_read"]

  book = {}

  for key, prompt in zip(keys,prompts):
      book[key] = input(prompt + ": ")

  ```

  Here I take the `prompts` and `keys` lists, collate them via `zip` and then loop
  through them to populate the `book` dictionary, which contains properties
  sourced from the `keys` list and values from the user's answers to the
  `prompts`.

raw_content:
  ---
  tags: [python, data-structures]
  ---

  # `zip`

  The `zip` function returns a zip object when you pass in two iterator data types
  (typically two lists). This is an iterator of tuples where the first item in
  each passed iterator is paired together, and then the second item in each passed
  iterator are paired together, and so on.

  ## Example

  ```py
  a = ("John", "Charles", "Mike")
  b = ("Jenny", "Christy", "Monica", "Vicky")

  x = zip(a,b)

  print(x)
  # <zip object at 0x1514af34c340>

  # To display the resulting tuple in a readable format:

  print(tuple(x))

  (('John', 'Jenny'), ('Charles', 'Christy'), ('Mike', 'Monica'))
  ```

  As indicated above, if the input iterators are of equal lengths, Python will
  ignore the value in iterator B that doesn't have an equivalent index in iterator
  A.

  ## Real-life example

  ```py
  prompts = [
      "Enter title",
      "Enter author",
      "Enter publication date",
      "Enter year you read the book"
  ]

  keys = ["Title", "Author", "Publication_date", "Date_read"]

  book = {}

  for key, prompt in zip(keys,prompts):
      book[key] = input(prompt + ": ")

  ```

  Here I take the `prompts` and `keys` lists, collate them via `zip` and then loop
  through them to populate the `book` dictionary, which contains properties
  sourced from the `keys` list and values from the user's answers to the
  `prompts`.


path: chmod.md
sortable_path: chmod.md
title: chmod
lead: Modify file permissions.
checksum: 4e19b65325eb68d2cf8b249bd6a45d56d00e455e6316b1dee6b1e260ab498884
created: 2025-10-01 13:31:03.262579423+00:00
modified: 2025-08-13 15:57:14.079140535+00:00
metadata: {"created":"Friday, April 11, 2025","tags":["permissions","Linux"]}
body:
  Modify file permissions.

  We can use symbols or numbers.

  For example:

  ```sh
  chmod +x filename
  ```

  To make a file executable.

  When we use numbers this is called an _absolute_ change, because all permission
  bits are being set at once in octal notation.

  Best just to memorise the most common sequences. Bear in mind that some only
  apply to files, rather than directories or executable programs.

  | Mode | Meaning                           | Applied to            |
  | ---- | --------------------------------- | --------------------- |
  | 644  | user: r/w, group/other: r         | files                 |
  | 600  | user: r/w, group/other: none      | files                 |
  | 755  | user: r/w/e, group/other: r/e     | directories, programs |
  | 700  | user: r/w/e, group/other: none    | directories, programs |
  | 711  | user: r/w/e, group/other: execute | directories           |

  ### Useful options

  `-v` → verbose: tell the user what `chmod` is doing

  `-r` → work recursively, i.e apply the action to directories as well as files

  > You can list the contents of a directory if it's readable but you can only
  > access a file in a directory if the directory is executable!

raw_content:
  ---
  tags: [permissions, Linux]
  created: Friday, April 11, 2025
  ---

  # chmod

  Modify file permissions.

  We can use symbols or numbers.

  For example:

  ```sh
  chmod +x filename
  ```

  To make a file executable.

  When we use numbers this is called an _absolute_ change, because all permission
  bits are being set at once in octal notation.

  Best just to memorise the most common sequences. Bear in mind that some only
  apply to files, rather than directories or executable programs.

  | Mode | Meaning                           | Applied to            |
  | ---- | --------------------------------- | --------------------- |
  | 644  | user: r/w, group/other: r         | files                 |
  | 600  | user: r/w, group/other: none      | files                 |
  | 755  | user: r/w/e, group/other: r/e     | directories, programs |
  | 700  | user: r/w/e, group/other: none    | directories, programs |
  | 711  | user: r/w/e, group/other: execute | directories           |

  ### Useful options

  `-v` → verbose: tell the user what `chmod` is doing

  `-r` → work recursively, i.e apply the action to directories as well as files

  > You can list the contents of a directory if it's readable but you can only
  > access a file in a directory if the directory is executable!


path: chown.md
sortable_path: chown.md
title: chown
lead: Change the owner of system files and directories.
checksum: bc23f326415f4488a594a19e8a733638e3357cbb13c17d7f68297a543f88c9da
created: 2025-10-01 13:31:03.272781006+00:00
modified: 2025-08-13 15:57:13.878394258+00:00
metadata: {"created":"Friday, April 11, 2025","tags":["permissions","Linux"]}
body:
  Change the owner of system files and directories.

  Basic syntax: `chown <owner:group> filename`.

  ```sh
  # Change file owner
  chown user1 doc.txt

  # Change owner and group
  chown user1:developers document.txt

  # Change only the group
  chown :staff shared_folder/

  # Do so recursively for dirs
  chown -R www-data:www-data /var/www/
  ```

  TODO: Partition into separate entry on groups:

  > When a user account in Linux is created the system also creates a group with
  > the same name as the user (known as the _primary group_ or _user private
  > group_).

  Because of the above, you will often change owndership to yourself with:

  ```sh
  chown thomas:thomas some_dir
  ```

  See groups:

  ```sh
  groups
  # thomas realtime docker input wheel adb plugdev
  ```

  When reassigning users and groups, it's safest to use the actual name. But each
  user/group also has a numeric representation, corresponding to UID:GID.

  The first regular, non-system user created on most Unix distributions is 1000
  but this isn't universal. Likewise his group will be 1000.

  See your UID/GID and the GIDs of the groups you're in:

  ```
  $ id
  uid=1000(thomas) gid=1000(thomas) groups=1000(thomas),959(realtime),966(docker),994(input),998(wheel),1001(adb),1002(plugdev)
  ```

raw_content:
  ---
  tags: [permissions, Linux]
  created: Friday, April 11, 2025
  ---

  # chown

  Change the owner of system files and directories.

  Basic syntax: `chown <owner:group> filename`.

  ```sh
  # Change file owner
  chown user1 doc.txt

  # Change owner and group
  chown user1:developers document.txt

  # Change only the group
  chown :staff shared_folder/

  # Do so recursively for dirs
  chown -R www-data:www-data /var/www/
  ```

  TODO: Partition into separate entry on groups:

  > When a user account in Linux is created the system also creates a group with
  > the same name as the user (known as the _primary group_ or _user private
  > group_).

  Because of the above, you will often change owndership to yourself with:

  ```sh
  chown thomas:thomas some_dir
  ```

  See groups:

  ```sh
  groups
  # thomas realtime docker input wheel adb plugdev
  ```

  When reassigning users and groups, it's safest to use the actual name. But each
  user/group also has a numeric representation, corresponding to UID:GID.

  The first regular, non-system user created on most Unix distributions is 1000
  but this isn't universal. Likewise his group will be 1000.

  See your UID/GID and the GIDs of the groups you're in:

  ```
  $ id
  uid=1000(thomas) gid=1000(thomas) groups=1000(thomas),959(realtime),966(docker),994(input),998(wheel),1001(adb),1002(plugdev)
  ```


path: fs.md
sortable_path: fs.md
title: fs module
checksum: acd2f1a9993165caa5ecf4a56b9e9dee29e00696a1f03582e784d576712099d4
created: 2025-10-01 13:31:03.282142899+00:00
modified: 2025-08-13 15:57:14.019262701+00:00
metadata: {"tags":["node-js"]}
lead:
  File System is an essential built-in module of Node that contains utility
  methods for working with files and directories.

body:
  File System is an essential built-in module of Node that contains utility
  methods for working with files and directories.

  Every method associated with `fs` has a _blocking_ and _asynchronous_
  implementation. The former obviously blocks the
  [event queue](Event_loop.md), the
  latter does not.

  The synchronous methods are useful to have in some contexts but in general and
  with real-world applications, you should be using the async implementation so as
  to accord with the single-threaded event-driven architecture of Node.

  ## Common I/0 methods

  ### Read directory

  Return a string array of all files in the current directory.

  ```js
  fs.readdir("./", function (err, files) {
    if (err) {
      console.error(err);
    } else {
      console.log(files);
    }
  });
  ```

  ### Read from file

  ```js
  fs.readFile("./lorem.md", "UTF-8", function (err, fileContents) {
    console.log(fileContents);
  });
  ```

  ### Write to file

  ```js
  let md = `A new file`;

  fs.writeFile("testFile.md", md.trim(), function () {
    console.log("File was created");
  });
  ```

  ### Appending to file

  ```js
  fs.appendFile("testFile.md", "new content");
  ```

  ### Create directory

  ```js
  if (!fs.existsSync("directory_name")) {
    fs.mkdir("directory_name", function (err) {
      if (err) {
        console.err(err);
      }
    });
  } else {
    console.warn("Directory already exists");
  }
  ```

  ### Rename and remove files

  ```js
  fs.rename("./filename.js", "./newname.js", function () {
    console.log("file renamed");
  });
  ```

  ```js
  // Remove file
  fs.unlink("./file-to-delete", function (err) {
    if (err) {
      console.error(err);
    }
  });
  ```

  ### Rename and remove directories

  ```js
  // Best to use synchronous method here
  fs.rmSync("/dir", { recursive: true, force: true });
  ```

  ## Streams

  See [Handling streams with fs](Streams_in_NodeJS.md)

raw_content:
  ---
  tags:
    - node-js
  ---

  # fs module

  File System is an essential built-in module of Node that contains utility
  methods for working with files and directories.

  Every method associated with `fs` has a _blocking_ and _asynchronous_
  implementation. The former obviously blocks the
  [event queue](Event_loop.md), the
  latter does not.

  The synchronous methods are useful to have in some contexts but in general and
  with real-world applications, you should be using the async implementation so as
  to accord with the single-threaded event-driven architecture of Node.

  ## Common I/0 methods

  ### Read directory

  Return a string array of all files in the current directory.

  ```js
  fs.readdir("./", function (err, files) {
    if (err) {
      console.error(err);
    } else {
      console.log(files);
    }
  });
  ```

  ### Read from file

  ```js
  fs.readFile("./lorem.md", "UTF-8", function (err, fileContents) {
    console.log(fileContents);
  });
  ```

  ### Write to file

  ```js
  let md = `A new file`;

  fs.writeFile("testFile.md", md.trim(), function () {
    console.log("File was created");
  });
  ```

  ### Appending to file

  ```js
  fs.appendFile("testFile.md", "new content");
  ```

  ### Create directory

  ```js
  if (!fs.existsSync("directory_name")) {
    fs.mkdir("directory_name", function (err) {
      if (err) {
        console.err(err);
      }
    });
  } else {
    console.warn("Directory already exists");
  }
  ```

  ### Rename and remove files

  ```js
  fs.rename("./filename.js", "./newname.js", function () {
    console.log("file renamed");
  });
  ```

  ```js
  // Remove file
  fs.unlink("./file-to-delete", function (err) {
    if (err) {
      console.error(err);
    }
  });
  ```

  ### Rename and remove directories

  ```js
  // Best to use synchronous method here
  fs.rmSync("/dir", { recursive: true, force: true });
  ```

  ## Streams

  See [Handling streams with fs](Streams_in_NodeJS.md)


path: http_in_Node.md
sortable_path: http_in_Node.md
title: http module
checksum: d74610fc301d67759fe76291d7e99eda1b918720fe38a694e5d28775c0b7ee25
created: 2025-10-01 13:31:03.302246303+00:00
modified: 2025-08-13 15:57:13.775996517+00:00
metadata: {"tags":["node-js"]}
lead:
  The HTTP Module allows us to create a web server that listens for HTTP requests
  on a given port. It is therefore perfect for creating backends for client-side
  JavaScript.

body:
  The HTTP Module allows us to create a web server that listens for HTTP requests
  on a given port. It is therefore perfect for creating backends for client-side
  JavaScript.

  ## Creating a server

  An HTTP server is another instance of an
  [event emitter](events.md)). It
  therefore has all the same methods as the `EventEmitter` class: `on`, `emit`,
  `addListener` etc. This demonstrates again how much of Node's core functionality
  is based on event emitters.

  _Creating a server_

  ```js
  const http = require("http");

  const server = http.createServer(); // Create server as emitter

  // Register functions to run when listener is triggered
  server.on("connection", (socket) => {
    console.log("new connection...");
  });

  server.listen(3000);
  console.log("Listening on port 3000");
  ```

  This server is functionally equivalent to a generic event emitter:

  ```js
  // Raise an event
  const emitter = new EventEmitter("messageLogged");

  // Register a listener
  emitter.on("messagedLogged", function () {
    console.log("The listener was called.");
  });
  ```

  Whenever a request is made to this server, it raises an event. We can therefore
  target it with the `on` method and make it execute a function when requests are
  made.

  If we were to start the server by running the file and we then used a browser to
  navigate to the port, we would see `new connection` logged every time we refresh
  the page.

  ### Sockets and `req, res`

  A socket is a generic protocol for client-server communication. Crucially it
  **allows simultaneous communication both ways**. The client can contact the
  server but the server can also contact the client. Our listener function above
  uses a socket as the callback function but in most cases this is quite
  low-level, not distinguishing responses from requests. It is more likely that
  you would initiate a `request, resource` architecture in place of a socket:

  ```js
  const server = http.createServer((req, res) => {
    if (req.url === "/") {
      res.write("hello");
      res.end();
    }
  });
  ```

  #### Return JSON

  Below is an example of using this architecture to return JSON to the client:

  ```js
  const server = http.createServer((req, res) => {
    if (req.url === "/products") {
      res.write(JSON.stringify(["shoes", "lipstick", "cups"]));
      res.end();
    }
  });
  ```

  ### Express

  In reality you would rarely use the `http` module directly to create a server.
  This is because it is quite low level and each response must be written in a
  linear fashion as with the two URLs in the previous example. Instead we use
  Express which is a framework for creating servers and routing that is an
  abstraction on top of the core HTTP module.

raw_content:
  ---
  tags:
    - node-js
  ---

  # `http` module

  The HTTP Module allows us to create a web server that listens for HTTP requests
  on a given port. It is therefore perfect for creating backends for client-side
  JavaScript.

  ## Creating a server

  An HTTP server is another instance of an
  [event emitter](events.md)). It
  therefore has all the same methods as the `EventEmitter` class: `on`, `emit`,
  `addListener` etc. This demonstrates again how much of Node's core functionality
  is based on event emitters.

  _Creating a server_

  ```js
  const http = require("http");

  const server = http.createServer(); // Create server as emitter

  // Register functions to run when listener is triggered
  server.on("connection", (socket) => {
    console.log("new connection...");
  });

  server.listen(3000);
  console.log("Listening on port 3000");
  ```

  This server is functionally equivalent to a generic event emitter:

  ```js
  // Raise an event
  const emitter = new EventEmitter("messageLogged");

  // Register a listener
  emitter.on("messagedLogged", function () {
    console.log("The listener was called.");
  });
  ```

  Whenever a request is made to this server, it raises an event. We can therefore
  target it with the `on` method and make it execute a function when requests are
  made.

  If we were to start the server by running the file and we then used a browser to
  navigate to the port, we would see `new connection` logged every time we refresh
  the page.

  ### Sockets and `req, res`

  A socket is a generic protocol for client-server communication. Crucially it
  **allows simultaneous communication both ways**. The client can contact the
  server but the server can also contact the client. Our listener function above
  uses a socket as the callback function but in most cases this is quite
  low-level, not distinguishing responses from requests. It is more likely that
  you would initiate a `request, resource` architecture in place of a socket:

  ```js
  const server = http.createServer((req, res) => {
    if (req.url === "/") {
      res.write("hello");
      res.end();
    }
  });
  ```

  #### Return JSON

  Below is an example of using this architecture to return JSON to the client:

  ```js
  const server = http.createServer((req, res) => {
    if (req.url === "/products") {
      res.write(JSON.stringify(["shoes", "lipstick", "cups"]));
      res.end();
    }
  });
  ```

  ### Express

  In reality you would rarely use the `http` module directly to create a server.
  This is because it is quite low level and each response must be written in a
  linear fashion as with the two URLs in the previous example. Instead we use
  Express which is a framework for creating servers and routing that is an
  abstraction on top of the core HTTP module.


path: journalctl.md
sortable_path: journalctl.md
title: journalctl
checksum: aa883484277709c341c68a33f6aa23a6062587c42eddd7176d85921fb94d2f31
created: 2025-10-01 13:31:03.320021842+00:00
modified: 2025-08-13 15:57:13.818073245+00:00
metadata: {"created":"Thursday, April 24, 2025","tags":["Linux","procedural","logs"]}
lead:
  We use `journalctl` to access [journald](./journald.md) logs. The command by
  itself outputs the entire log which will be huge and hard to scroll through. We
  can refine the results with modifiers.

body:
  We use `journalctl` to access [journald](./journald.md) logs. The command by
  itself outputs the entire log which will be huge and hard to scroll through. We
  can refine the results with modifiers.

  ### View logs for a specific process with pid

  ```bash
  journalctl _PID=1234
  ```

  ### View logs for a specific time period

  This can be really helpful since you can bracket the most recent events which
  will be more memorable.

  ```bash
  journalctl -S -1h
  ```

  ### View logs for a specfic systemd unit

  ```bash
  journalctl -u [unit_name] -e
  ```

  ### View boot logs

  ```bash
  journalctl -b
  ```

  #### Identify specific boot

  ```bash
  journalctl --list-boots

  ```

  ### List only kernel entries to the journal

  ```bash
  journalctl -k
  ```

  ### View logs in realtime

  Use `-f` for `--follow`:

  ```sh
  journalctl -f
  ```

raw_content:
  ---
  tags: [Linux, procedural, logs]
  created: Thursday, April 24, 2025
  ---

  # journalctl

  We use `journalctl` to access [journald](./journald.md) logs. The command by
  itself outputs the entire log which will be huge and hard to scroll through. We
  can refine the results with modifiers.

  ### View logs for a specific process with pid

  ```bash
  journalctl _PID=1234
  ```

  ### View logs for a specific time period

  This can be really helpful since you can bracket the most recent events which
  will be more memorable.

  ```bash
  journalctl -S -1h
  ```

  ### View logs for a specfic systemd unit

  ```bash
  journalctl -u [unit_name] -e
  ```

  ### View boot logs

  ```bash
  journalctl -b
  ```

  #### Identify specific boot

  ```bash
  journalctl --list-boots

  ```

  ### List only kernel entries to the journal

  ```bash
  journalctl -k
  ```

  ### View logs in realtime

  Use `-f` for `--follow`:

  ```sh
  journalctl -f
  ```


path: journald.md
sortable_path: journald.md
title: journald
checksum: 2a328b2eb38666790a9595c57e830b40361d7effd62c31e7a85fe7163406094b
created: 2025-10-01 13:31:03.338579788+00:00
modified: 2025-08-13 15:57:13.968867314+00:00
metadata: {"tags":["systems-programming","Linux","procedural"]}
lead:
  `journald` is a program that comes as default with [systemd](systemd.md). It is
  a service for collecting and storing system-level log data. I keeps a track of
  all [kernel](The_kernel.md) processes. It is invaluable when tracing the source
  of problems and errors that may arise on the system level. It keeps a track of
  all kernal processes.

body:
  `journald` is a program that comes as default with [systemd](systemd.md). It is
  a service for collecting and storing system-level log data. I keeps a track of
  all [kernel](The_kernel.md) processes. It is invaluable when tracing the source
  of problems and errors that may arise on the system level. It keeps a track of
  all kernal processes.

  ![](/img/journald.png)

raw_content:
  ---
  tags:
    - systems-programming
    - Linux
    - procedural
  ---

  # `journald`

  `journald` is a program that comes as default with [systemd](systemd.md). It is
  a service for collecting and storing system-level log data. I keeps a track of
  all [kernel](The_kernel.md) processes. It is invaluable when tracing the source
  of problems and errors that may arise on the system level. It keeps a track of
  all kernal processes.

  ![](/img/journald.png)


path: jq.md
sortable_path: jq.md
title: jq
lead: ## Remove property
checksum: 676854d84ae30e105cfd98da7b65640c36c42493225fcdd94a44f13d3be84d41
created: 2025-10-01 13:31:03.36238742+00:00
modified: 2025-08-13 15:57:13.941035281+00:00
metadata: {"tags":["json","shell"]}
body:
  ## Remove property

  Remove property from a JSON array of objects:

  ```json
  {
    "member_inputs": [
      {
        "name": "Thomas",
        "input_type": "select"
      },
      {
        "name": "Martha",
        "input_type": "text"
      }
    ]
  }
  ```

  ```sh
  jq '.member_details |= map(del(.input_type))' memberDetails.json
  ```

  > Add '-i' to modify the source file directly (in-place)

  > Multiple properties can be deleted simply by chaining,
  > e.g.`(del(.input_type, .another_property))`

raw_content:
  ---
  tags:
    - json
    - shell
  ---

  # jq

  ## Remove property

  Remove property from a JSON array of objects:

  ```json
  {
    "member_inputs": [
      {
        "name": "Thomas",
        "input_type": "select"
      },
      {
        "name": "Martha",
        "input_type": "text"
      }
    ]
  }
  ```

  ```sh
  jq '.member_details |= map(del(.input_type))' memberDetails.json
  ```

  > Add '-i' to modify the source file directly (in-place)

  > Multiple properties can be deleted simply by chaining,
  > e.g.`(del(.input_type, .another_property))`


path: lsof.md
sortable_path: lsof.md
title: lsof
lead:
  `lsof` stands for _list open files_. It lists opened files and the processes
  using them.

checksum: da5c29f66229b34260b9bfbe95bdfc9821e7fac0f246e526f512e6660ca925cb
created: 2025-10-01 13:31:03.372409835+00:00
modified: 2025-08-13 15:57:14.015926896+00:00
metadata: {"created":"Tuesday, March 04, 2025","tags":["shell","procedural"]}
body:
  `lsof` stands for _list open files_. It lists opened files and the processes
  using them.

  ## Directory

  ```sh
  lsof /home/thomas
  ```

  ```
  COMMAND    PID   USER  FD   TYPE DEVICE SIZE/OFF    NODE NAME
  pipewire  1025 thomas cwd    DIR  253,0     4096 7340035 /home/thomas
  wireplumb 1026 thomas cwd    DIR  253,0     4096 7340035 /home/thomas
  ```

  ## Pass in a pid

  ```sh
  lsof -p 1175
  ```

  ```
  COMMAND    PID   USER  FD      TYPE             DEVICE SIZE/OFF    NODE NAME
  at-spi-bu 1175 thomas cwd       DIR              253,0     4096 7340035 /home/thomas
  at-spi-bu 1175 thomas rtd       DIR              259,2     4096       2 /
  at-spi-bu 1175 thomas txt       REG              259,2    26760 1225154 /usr/lib/at-spi-bus-launcher
  ```

  ## Networking

  ### List programs using or listening to ports

  ```sh
  lsof -i
  ```

  ```
  COMMAND    PID   USER  FD   TYPE DEVICE SIZE/OFF NODE NAME
  signal-de 3250 thomas  39u  IPv4 149863      0t0  TCP archlinux:44304->ac88393aca5853df7.awsglobalaccelerator.com:https (ESTABLISHED)
  slack     3602 thomas  18u  IPv4 138442      0t0  TCP archlinux:57950->ec2-18-169-55-40.eu-west-2.compute.amazonaws.com:https (ESTABLISHED)
  firefox   6177 thomas 147u  IPv4 138842      0t0  TCP archlinux:cresco-control->93.243.107.34.bc.googleusercontent.com:https (ESTABLISHED)
  ```

  ### Filter by specific port

  ```sh
  lsof -i:22
  lsof -i4:22 # limit to IPv4 addresses

  ```

  ```
  COMMAND    PID   USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
  sshd     45448   root    3u  IPv4  640237      0t0  TCP *:ssh (LISTEN)
  sshd     45448   root    4u  IPv6  640239      0t0  TCP *:ssh (LISTEN)
  sshd    185660   root    4u  IPv4 2554538      0t0  TCP static.18.113.203.116.clients.your-server.de:ssh->0542cb12.skybroadband.com:57744 (ESTABLISHED)

  ```

raw_content:
  ---
  tags: [shell, procedural]
  created: Tuesday, March 04, 2025
  ---

  # lsof

  `lsof` stands for _list open files_. It lists opened files and the processes
  using them.

  ## Directory

  ```sh
  lsof /home/thomas
  ```

  ```
  COMMAND    PID   USER  FD   TYPE DEVICE SIZE/OFF    NODE NAME
  pipewire  1025 thomas cwd    DIR  253,0     4096 7340035 /home/thomas
  wireplumb 1026 thomas cwd    DIR  253,0     4096 7340035 /home/thomas
  ```

  ## Pass in a pid

  ```sh
  lsof -p 1175
  ```

  ```
  COMMAND    PID   USER  FD      TYPE             DEVICE SIZE/OFF    NODE NAME
  at-spi-bu 1175 thomas cwd       DIR              253,0     4096 7340035 /home/thomas
  at-spi-bu 1175 thomas rtd       DIR              259,2     4096       2 /
  at-spi-bu 1175 thomas txt       REG              259,2    26760 1225154 /usr/lib/at-spi-bus-launcher
  ```

  ## Networking

  ### List programs using or listening to ports

  ```sh
  lsof -i
  ```

  ```
  COMMAND    PID   USER  FD   TYPE DEVICE SIZE/OFF NODE NAME
  signal-de 3250 thomas  39u  IPv4 149863      0t0  TCP archlinux:44304->ac88393aca5853df7.awsglobalaccelerator.com:https (ESTABLISHED)
  slack     3602 thomas  18u  IPv4 138442      0t0  TCP archlinux:57950->ec2-18-169-55-40.eu-west-2.compute.amazonaws.com:https (ESTABLISHED)
  firefox   6177 thomas 147u  IPv4 138842      0t0  TCP archlinux:cresco-control->93.243.107.34.bc.googleusercontent.com:https (ESTABLISHED)
  ```

  ### Filter by specific port

  ```sh
  lsof -i:22
  lsof -i4:22 # limit to IPv4 addresses

  ```

  ```
  COMMAND    PID   USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
  sshd     45448   root    3u  IPv4  640237      0t0  TCP *:ssh (LISTEN)
  sshd     45448   root    4u  IPv6  640239      0t0  TCP *:ssh (LISTEN)
  sshd    185660   root    4u  IPv4 2554538      0t0  TCP static.18.113.203.116.clients.your-server.de:ssh->0542cb12.skybroadband.com:57744 (ESTABLISHED)

  ```


path: ps.md
sortable_path: ps.md
title: Processes (ps)
lead:
  `ps` allows us to control [user processes](The_kernel.md)
  from the shell.

checksum: db2045cff446875dd957c96c1b0b2dcaf4c16fed9ceb7c1f0f91146ea1f07297
created: 2025-10-01 13:31:03.38277077+00:00
modified: 2025-08-13 15:57:13.957023379+00:00
metadata: {"tags":["shell","Linux"]}
body:
  `ps` allows us to control [user processes](The_kernel.md)
  from the shell.

  The command in its most minimal application returns the following

  ```
    PID TTY       TIME     CMD
    2437 pts/2    00:00:01 zsh
    7112 pts/2    00:00:00 ps
  ```

  With the `-e` modifier we can list more processes:

  ```
   PID TTY          TIME CMD
        1 ?        00:00:05 systemd
        2 ?        00:00:00 kthreadd
        3 ?        00:00:00 rcu_gp
        4 ?        00:00:00 rcu_par_gp
        5 ?        00:00:00 netns
        7 ?        00:00:00 kworker/0:0H-events_highpri
        9 ?        00:00:00 mm_percpu_wq
       11 ?        00:00:00 rcu_tasks_kthread
       12 ?        00:00:00 rcu_tasks_rude_kthread
       13 ?        00:00:00 rcu_tasks_trace_kthread
       14 ?        00:00:08 ksoftirqd/0
       15 ?        00:03:20 rcu_preempt
       16 ?        00:00:00 rcub/0
       17 ?        00:00:00 migration/0
       18 ?        00:00:00 idle_inject/0
       20 ?        00:00:00 cpuhp/0
       21 ?        00:00:00 cpuhp/1
       22 ?        00:00:00 idle_inject/1
       23 ?        00:00:00 migration/1
  ```

  <dl>
  <dt>pid</dt>
  <dd>Process ID: every currently running process has a unique ID<dd>
  <dt>tty</dt>
  <dd>The terminal device where the process is running<dd>
  <dt>Time</dt>
  <dd>The amount of CPU time in minutes and seconds that the process has used so far. The total amount of time that the process has spent running instructions on the processor.<dd>
  <dt>cmd</dt>
  <dd>The command used to run the program. Note this can change during the running of the program.<dd>
  </dl>

  ## Modifiers

  <dl>
  <dt><code>ps x</code></dt>
  <dd>Show all running processes<dd>
  <dt><code>ps ax</code></dt>
  <dd>Show all processes not just the ones you, the current user, own<dd>
  <dt><code>ps u</code></dt>
  <dd>Show detailed info on processes<dd>
  <dt><code>ps w</code></dt>
  <dd>Show full command names<dd>
  </dl>

  ## Process termination

  The general schema is: `kill [pid]`. This allows for process clean-up. If this
  doesn't succeed you can force with `KILL [pid]` which will terminate the process
  immediately but is obviously more risky.

  We can also start/stop processes with modifiers on `kill`:

  - `kill -STOP pid`
  - `kill -CONT pid`

raw_content:
  ---
  tags:
    - shell
    - Linux
  ---

  # Processes (`ps`)

  `ps` allows us to control [user processes](The_kernel.md)
  from the shell.

  The command in its most minimal application returns the following

  ```
    PID TTY       TIME     CMD
    2437 pts/2    00:00:01 zsh
    7112 pts/2    00:00:00 ps
  ```

  With the `-e` modifier we can list more processes:

  ```
   PID TTY          TIME CMD
        1 ?        00:00:05 systemd
        2 ?        00:00:00 kthreadd
        3 ?        00:00:00 rcu_gp
        4 ?        00:00:00 rcu_par_gp
        5 ?        00:00:00 netns
        7 ?        00:00:00 kworker/0:0H-events_highpri
        9 ?        00:00:00 mm_percpu_wq
       11 ?        00:00:00 rcu_tasks_kthread
       12 ?        00:00:00 rcu_tasks_rude_kthread
       13 ?        00:00:00 rcu_tasks_trace_kthread
       14 ?        00:00:08 ksoftirqd/0
       15 ?        00:03:20 rcu_preempt
       16 ?        00:00:00 rcub/0
       17 ?        00:00:00 migration/0
       18 ?        00:00:00 idle_inject/0
       20 ?        00:00:00 cpuhp/0
       21 ?        00:00:00 cpuhp/1
       22 ?        00:00:00 idle_inject/1
       23 ?        00:00:00 migration/1
  ```

  <dl>
  <dt>pid</dt>
  <dd>Process ID: every currently running process has a unique ID<dd>
  <dt>tty</dt>
  <dd>The terminal device where the process is running<dd>
  <dt>Time</dt>
  <dd>The amount of CPU time in minutes and seconds that the process has used so far. The total amount of time that the process has spent running instructions on the processor.<dd>
  <dt>cmd</dt>
  <dd>The command used to run the program. Note this can change during the running of the program.<dd>
  </dl>

  ## Modifiers

  <dl>
  <dt><code>ps x</code></dt>
  <dd>Show all running processes<dd>
  <dt><code>ps ax</code></dt>
  <dd>Show all processes not just the ones you, the current user, own<dd>
  <dt><code>ps u</code></dt>
  <dd>Show detailed info on processes<dd>
  <dt><code>ps w</code></dt>
  <dd>Show full command names<dd>
  </dl>

  ## Process termination

  The general schema is: `kill [pid]`. This allows for process clean-up. If this
  doesn't succeed you can force with `KILL [pid]` which will terminate the process
  immediately but is obviously more risky.

  We can also start/stop processes with modifiers on `kill`:

  - `kill -STOP pid`
  - `kill -CONT pid`


path: python_advent_learnings.md
sortable_path: python_advent_learnings.md
title: Python advent learnings
lead: ### Construct paths without hard-coding absolute path
checksum: 41e623fc64d7bdeedbc4cb74740ba94f0d812a86c637bec43edfdfb62f90f6b5
created: 2025-10-01 13:31:03.402330014+00:00
modified: 2025-08-13 15:57:13.902912393+00:00
metadata: {}
body:
  ### Construct paths without hard-coding absolute path

  ```py
  puzzle_input_path = os.path.join(os.path.dirname(__file__), "data/test_input.txt")

  ```

  ### Read file by line and store in variable

  ```py
  puzzle_input_path = os.path.join(os.path.dirname(__file__), "data/test_input.txt")

  def getPuzzleInput(file_path):
      with open(file_path, "r") as file:
          return [line.strip() for line in file]


  puzzle_input = getPuzzleInput(puzzle_input_path)

  ```

  ### Conditional tests for members of arrays

  Check that all elements are greater than zero:

  ```py
  if all(element >= 0 for element in array):
      print('All elements greater than 0')
  ```

  Check if any element is less than 0:

  ```py
  if any(element < 0 for element in array):
      print('There is an element that is less than 0')
  ```

  ### Check for substring:

  ```py
  text = "latest test"
  if "test" in text:
      return True
  ```

  ### Control flow in loop

  ```py
  for counter_values in games:
      colour = counter_values[1]
      count = int(counter_values[0])
      if colour in initial_values.keys() and count > initial_values[colour]:
          break
  else:
      valid_games_count += game_number
  ```

  The use of the `else` clause here is a special feature of Python. The `else`
  clause after a `for` loop will normally only execute when the loop has finished
  iterating over the list, but not when the loop is terminated by a `break`
  statement.

  The `else` clause is executed if the `for` loop completes normally. If the
  `break` statement is executed (i.e., if any count in `counter_values` is greater
  than the corresponding count in `initial_values`), the `for` loop is terminated
  and the `else` clause is skipped.

  Here the combination of a `for` loop and the `break` statement creates
  `if, else` logic. If the break condition is not reached, then the `else` block
  runs for every iteration of the loop.

  ### Multiple loop in list comprehension

  ```py
  games = [item.strip().split() for game in games for item in game.split(",")]
  ```

raw_content:
  # Python advent learnings

  ### Construct paths without hard-coding absolute path

  ```py
  puzzle_input_path = os.path.join(os.path.dirname(__file__), "data/test_input.txt")

  ```

  ### Read file by line and store in variable

  ```py
  puzzle_input_path = os.path.join(os.path.dirname(__file__), "data/test_input.txt")

  def getPuzzleInput(file_path):
      with open(file_path, "r") as file:
          return [line.strip() for line in file]


  puzzle_input = getPuzzleInput(puzzle_input_path)

  ```

  ### Conditional tests for members of arrays

  Check that all elements are greater than zero:

  ```py
  if all(element >= 0 for element in array):
      print('All elements greater than 0')
  ```

  Check if any element is less than 0:

  ```py
  if any(element < 0 for element in array):
      print('There is an element that is less than 0')
  ```

  ### Check for substring:

  ```py
  text = "latest test"
  if "test" in text:
      return True
  ```

  ### Control flow in loop

  ```py
  for counter_values in games:
      colour = counter_values[1]
      count = int(counter_values[0])
      if colour in initial_values.keys() and count > initial_values[colour]:
          break
  else:
      valid_games_count += game_number
  ```

  The use of the `else` clause here is a special feature of Python. The `else`
  clause after a `for` loop will normally only execute when the loop has finished
  iterating over the list, but not when the loop is terminated by a `break`
  statement.

  The `else` clause is executed if the `for` loop completes normally. If the
  `break` statement is executed (i.e., if any count in `counter_values` is greater
  than the corresponding count in `initial_values`), the `for` loop is terminated
  and the `else` clause is skipped.

  Here the combination of a `for` loop and the `break` statement creates
  `if, else` logic. If the break condition is not reached, then the `else` block
  runs for every iteration of the loop.

  ### Multiple loop in list comprehension

  ```py
  games = [item.strip().split() for game in games for item in game.split(",")]
  ```


path: rsync.md
sortable_path: rsync.md
title: rsync
lead: ## Ordering
checksum: fc68ed7d91b56cb6a4ed0c5e05fd6954be5bb5213ea9895bb63648d96f2c897d
created: 2025-10-01 13:31:03.412085552+00:00
modified: 2025-08-13 15:57:13.848677618+00:00
metadata: {"created":"Saturday, April 26, 2025","tags":["file-transfer","Linux","procedural","disks"]}
body:
  ## Ordering

  ### Local to local

  ```
  SOURCE_DIR > TARGET_DIR
  ```

  ### Local to remote

  ```
  LOCAL_SOURC_DIR > REMOTE_TARGET_DIR
  ```

  ### Remote to local

  ```
  REMOTE_TARGET_DIR > LOCAL_SOURC_DIR
  ```

  ```sh
  rsync -a <dir_to_copy_to> <dir_to_copy_from>
  ```

  ## Expanding directories

  The following:

  ```sh
  rsync -a local_dir  target_dir
  ```

  Will create `/target_dir/local_dir` at the target. In other words it will nest
  the actual directory you are interested in within what you have named the
  target.

  To avoid this, add a slash to the source directory, viz:

  ```sh
  rysync -a local_dir/ target_dir
  ```

  Now, at the target, there will just be `local_dir`.

  ## Standard options I use

  ```sh
  rsync -vzP
  ```

  - verbose output

  - use compression (only really useful when running rysnc over a network)

  - display progress

  - preserve partially copied filed and resume if network connection interrupted

  ## Archive mode

  Use "archive mode" when specifically wanting to create a backup of a directory
  (i.e. for long term storage rather than immediate use).

  ```sh
  rsync -a
  ```

  Archive mode is an umbrella for the following flags:

  - `-r`: recursive

  - `l`: copy [symlinks](./Symlinks.md) as symlinks

  - `p`: preserve permissions

  - `t`: preserve times

  - `g`: preserve group

raw_content:
  ---
  tags: [file-transfer, Linux, procedural, disks]
  created: Saturday, April 26, 2025
  ---

  # rsync

  ## Ordering

  ### Local to local

  ```
  SOURCE_DIR > TARGET_DIR
  ```

  ### Local to remote

  ```
  LOCAL_SOURC_DIR > REMOTE_TARGET_DIR
  ```

  ### Remote to local

  ```
  REMOTE_TARGET_DIR > LOCAL_SOURC_DIR
  ```

  ```sh
  rsync -a <dir_to_copy_to> <dir_to_copy_from>
  ```

  ## Expanding directories

  The following:

  ```sh
  rsync -a local_dir  target_dir
  ```

  Will create `/target_dir/local_dir` at the target. In other words it will nest
  the actual directory you are interested in within what you have named the
  target.

  To avoid this, add a slash to the source directory, viz:

  ```sh
  rysync -a local_dir/ target_dir
  ```

  Now, at the target, there will just be `local_dir`.

  ## Standard options I use

  ```sh
  rsync -vzP
  ```

  - verbose output

  - use compression (only really useful when running rysnc over a network)

  - display progress

  - preserve partially copied filed and resume if network connection interrupted

  ## Archive mode

  Use "archive mode" when specifically wanting to create a backup of a directory
  (i.e. for long term storage rather than immediate use).

  ```sh
  rsync -a
  ```

  Archive mode is an umbrella for the following flags:

  - `-r`: recursive

  - `l`: copy [symlinks](./Symlinks.md) as symlinks

  - `p`: preserve permissions

  - `t`: preserve times

  - `g`: preserve group


path: scp.md
sortable_path: scp.md
title: scp
lead:
  ```sh
  scp host:/dir/*.sql.gz /local_machine/dir
  ```

checksum: 93d11d6905b9c936dbe59ef53c0c8e24c898e6bdc7a6a0c22da3241775e98bb5
created: 2025-10-01 13:31:03.427266284+00:00
modified: 2025-08-13 15:57:13.910491399+00:00
metadata: {"created":"Sunday, April 27, 2025","tags":["file-transfer","Linux","procedural","servers"]}
body:
  ```sh
  scp host:/dir/*.sql.gz /local_machine/dir
  ```

  > Obviously SSH from local to remote is a prerequisite

raw_content:
  ---
  tags: [file-transfer, Linux, procedural, servers]
  created: Sunday, April 27, 2025
  ---

  # scp

  ```sh
  scp host:/dir/*.sql.gz /local_machine/dir
  ```

  > Obviously SSH from local to remote is a prerequisite


path: systemd.md
sortable_path: systemd.md
title: systemd
checksum: 97593feb8a0b01ebe73a31ed767fe3fb51f286eb35ec06d79b9c6dfc1e3e9a59
created: 2025-10-01 13:31:03.438400944+00:00
modified: 2025-08-13 15:57:13.936728502+00:00
metadata: {"tags":["systems-programming","Linux","procedural","systemd","operating-systems"]}
lead:
  Once the [boot process](Boot_process.md) has completed and the bootloader has
  located the kernel and injected it into memory the first user space program
  runs: `init` (for _initialisation_). `init` is a [daemon](Daemons.md)
  [process](./Processes.md) that continues running until shutdown and is
  responsible for starting all the processes that are prerequisites for user
  space. For example: network connections, disk access, user logins etc. `init` is
  the parent of all processes: PID1. Whilst it does a lot of its work in quick
  succession at boot time it is not limited to the this stage of the lifescycle
  but runs continuously in reponse to new user events.

body:
  Once the [boot process](Boot_process.md) has completed and the bootloader has
  located the kernel and injected it into memory the first user space program
  runs: `init` (for _initialisation_). `init` is a [daemon](Daemons.md)
  [process](./Processes.md) that continues running until shutdown and is
  responsible for starting all the processes that are prerequisites for user
  space. For example: network connections, disk access, user logins etc. `init` is
  the parent of all processes: PID1. Whilst it does a lot of its work in quick
  succession at boot time it is not limited to the this stage of the lifescycle
  but runs continuously in reponse to new user events.

  On Linux systems `systemd` is used to implement `init` and therefore `systemd`
  will always be listed as PID1.

  `systemd` is directly accessible from user space and provides a straightforward
  way to enable/disable, start/stop system level processes

  > `systemd` can track individual service daemons after they start, and group
  > together multiple processes associated with a service, giving you more power
  > and insight into exactly what is running on the system _How Linux Works: Third
  > Edition_, Brian Ward 2021

  ## How systemd works

  ### Goal-directed units

  `systemd` works on the basis of _goals_. Each goal is system task defined as a
  **unit**. A unit contains instructions and a specification of dependencies to
  other units.

  When activating a unit, `systemd` first activates the dependencies and then
  moves onto the details of the unit itself. `init` as implemented by `systemd`
  does not follow a rigid sequence every time, initialising units in the same
  sequence and waiting for one to complete before starting another. Instead it
  activates units whenever they are ready. This, its parallelized nature, is one
  of the main advantages over previous programs that managed the `init` sequence
  on Linux (such as for example System V);

  ### Unit types

  Units are organised into **unit types**. The main types that run at boot time
  are as follows:

  - service units (`.service`)
    - control service daemons
  - socket units (`.socket`)
    - handle incoming network connection request locations
  - device units (`.device`)
    - disks and other devices
  - mount units (`.mount`)
    - handle the attachment of filesystems to the system
  - target units
    - control other units by organising them into groups

  For example, at boot, a target unit called `default.target` groups together a
  number of service and mount units as dependencies. These then run in a
  graph-like dependency structure where a unit that comes late in the boot process
  can depend on several previous units making earlier branches of a dependency
  tree join back together.

  ## systemd configuration files

  Units are managed via `systemd` configuration files.

  ### Configuration file locations

  System level `systemd` config files are located in the _system unit directory_
  at `/usr/lib/systemd/system`. You shouldn't change or manipulate these files or
  attempt to add new config files here since they will be overwritten by the
  system.

  ![](/img/systemd-global-files.png) _`systemd` global unit files_

  Local definitions that relate to the specific user and where the user herself
  can define units are located in the _system configuration_ directory:
  `/etc/systemd/system`.

  ![](/img/systemd-local-files.png)

  _`systemd` local unit files, specific to the currently logged-in user_

  ### Example files

  Below is the systemd service file for the creation of UUIDs:

  ```plain
  [Unit]
  Description=Daemon for generating UUIDs
  Documentation=man:uuidd(8)
  Requires=uuidd.socket

  [Service]
  ExecStart=/usr/sbin/uuidd --socket-activation
  Restart=no
  User=uuidd
  Group=uuidd
  ProtectSystem=strict
  ProtectHome=yes
  PrivateDevices=yes
  PrivateUsers=yes
  ProtectKernelTunables=yes
  ProtectKernelModules=yes
  ProtectControlGroups=yes
  MemoryDenyWriteExecute=yes
  ReadWritePaths=/var/lib/libuuid/
  SystemCallFilter=@default @file-system @basic-io @system-service @signal @io-event @network-io

  [Install]
  Also=uuidd.socket
  ```

  - The `Unit` section provides metadata about the unit including which `systemd`
    dependencies it has
  - `Service` constitutes the main specification for the unit
  - `Install` is the call to set the dependencies running before the `Service`
    functions are accessible.

  ## systemd operations: systemctl

  The `systemctl` command is the chief way of interacting with `systemd`. You use
  it to activate and deactivate services, list their status, reload the
  configuration and so.

  ### View the dependency graph

  `systemctl status` by itself will print a long list of units grouped by their
  dependency relations. It will also provide some metadata about the current
  systemd boot context.

  ### Viewing active units

  Below I have listed the running units pertaining to bluetooth:

  ```
  $ systemctl list-units | grep bluetooth
   sys-devices-pci0000:00-0000:00:14.0-usb3-3\x2d10-3\x2d10:1.0-bluetooth-hci0-hci0:3585.device                                   loaded active plugged   /sys/devices/pci0000:00/0000:00:14.0/usb3/3-10/3-10:1.0/bluetooth/hci0/hci0:3585
    sys-devices-pci0000:00-0000:00:14.0-usb3-3\x2d10-3\x2d10:1.0-bluetooth-hci0.device                                             loaded active plugged   /sys/devices/pci0000:00/0000:00:14.0/usb3/3-10/3-10:1.0/bluetooth/hci0
    sys-subsystem-bluetooth-devices-hci0.device                                                                                    loaded active plugged   /sys/subsystem/bluetooth/devices/hci0
    sys-subsystem-bluetooth-devices-hci0:3585.device                                                                               loaded active plugged   /sys/subsystem/bluetooth/devices/hci0:3585
    bluetooth.service                                                                                                              loaded active running   Bluetooth service
    bluetooth.target                                                                                                               loaded active active    Bluetooth Support
  ```

  ### List jobs

  Requests to activate, reactivate and restart units are called **jobs** in
  `systemd`. They can be thought of as unit state changes. Current jobs can be
  listed with `systemctl list-jobs`.

  This will most likely return `No jobs running` if the computer has been running
  for a while. Most jobs execute at boot.

  ### Enable/disable, start/stop units

  If a unit has dependencies it is necessary to _enable_ it before starting it.
  This installs the dependencies.

  ```bash
  systemctl enable mongodb.service
  Created symlink /etc/systemd/system/multi-user.target.wants/mongodb.service → /usr/lib/systemd/system/mongodb.service.
  ```

  Then we can start:

  ```
  systemctl start mongodb.service
  ```

  To stop the service:

  ```
  systemctl stop mongodb.service
  ```

  After this we should disable it, in order to remove any symbolic links it has
  created on the system as part of the install process:

  ```bash
  systemctl disable mongodb.service
  Removed "/etc/systemd/system/multi-user.target.wants/mongodb.service".
  ```

  ## Why use systemd over cron ?

  https://mark.stosberg.com/2016-08-26-rsnapshot-and-systemd/

raw_content:
  ---
  tags:
    - systems-programming
    - Linux
    - procedural
    - systemd
    - operating-systems
  ---

  # systemd

  Once the [boot process](Boot_process.md) has completed and the bootloader has
  located the kernel and injected it into memory the first user space program
  runs: `init` (for _initialisation_). `init` is a [daemon](Daemons.md)
  [process](./Processes.md) that continues running until shutdown and is
  responsible for starting all the processes that are prerequisites for user
  space. For example: network connections, disk access, user logins etc. `init` is
  the parent of all processes: PID1. Whilst it does a lot of its work in quick
  succession at boot time it is not limited to the this stage of the lifescycle
  but runs continuously in reponse to new user events.

  On Linux systems `systemd` is used to implement `init` and therefore `systemd`
  will always be listed as PID1.

  `systemd` is directly accessible from user space and provides a straightforward
  way to enable/disable, start/stop system level processes

  > `systemd` can track individual service daemons after they start, and group
  > together multiple processes associated with a service, giving you more power
  > and insight into exactly what is running on the system _How Linux Works: Third
  > Edition_, Brian Ward 2021

  ## How systemd works

  ### Goal-directed units

  `systemd` works on the basis of _goals_. Each goal is system task defined as a
  **unit**. A unit contains instructions and a specification of dependencies to
  other units.

  When activating a unit, `systemd` first activates the dependencies and then
  moves onto the details of the unit itself. `init` as implemented by `systemd`
  does not follow a rigid sequence every time, initialising units in the same
  sequence and waiting for one to complete before starting another. Instead it
  activates units whenever they are ready. This, its parallelized nature, is one
  of the main advantages over previous programs that managed the `init` sequence
  on Linux (such as for example System V);

  ### Unit types

  Units are organised into **unit types**. The main types that run at boot time
  are as follows:

  - service units (`.service`)
    - control service daemons
  - socket units (`.socket`)
    - handle incoming network connection request locations
  - device units (`.device`)
    - disks and other devices
  - mount units (`.mount`)
    - handle the attachment of filesystems to the system
  - target units
    - control other units by organising them into groups

  For example, at boot, a target unit called `default.target` groups together a
  number of service and mount units as dependencies. These then run in a
  graph-like dependency structure where a unit that comes late in the boot process
  can depend on several previous units making earlier branches of a dependency
  tree join back together.

  ## systemd configuration files

  Units are managed via `systemd` configuration files.

  ### Configuration file locations

  System level `systemd` config files are located in the _system unit directory_
  at `/usr/lib/systemd/system`. You shouldn't change or manipulate these files or
  attempt to add new config files here since they will be overwritten by the
  system.

  ![](/img/systemd-global-files.png) _`systemd` global unit files_

  Local definitions that relate to the specific user and where the user herself
  can define units are located in the _system configuration_ directory:
  `/etc/systemd/system`.

  ![](/img/systemd-local-files.png)

  _`systemd` local unit files, specific to the currently logged-in user_

  ### Example files

  Below is the systemd service file for the creation of UUIDs:

  ```plain
  [Unit]
  Description=Daemon for generating UUIDs
  Documentation=man:uuidd(8)
  Requires=uuidd.socket

  [Service]
  ExecStart=/usr/sbin/uuidd --socket-activation
  Restart=no
  User=uuidd
  Group=uuidd
  ProtectSystem=strict
  ProtectHome=yes
  PrivateDevices=yes
  PrivateUsers=yes
  ProtectKernelTunables=yes
  ProtectKernelModules=yes
  ProtectControlGroups=yes
  MemoryDenyWriteExecute=yes
  ReadWritePaths=/var/lib/libuuid/
  SystemCallFilter=@default @file-system @basic-io @system-service @signal @io-event @network-io

  [Install]
  Also=uuidd.socket
  ```

  - The `Unit` section provides metadata about the unit including which `systemd`
    dependencies it has
  - `Service` constitutes the main specification for the unit
  - `Install` is the call to set the dependencies running before the `Service`
    functions are accessible.

  ## systemd operations: systemctl

  The `systemctl` command is the chief way of interacting with `systemd`. You use
  it to activate and deactivate services, list their status, reload the
  configuration and so.

  ### View the dependency graph

  `systemctl status` by itself will print a long list of units grouped by their
  dependency relations. It will also provide some metadata about the current
  systemd boot context.

  ### Viewing active units

  Below I have listed the running units pertaining to bluetooth:

  ```
  $ systemctl list-units | grep bluetooth
   sys-devices-pci0000:00-0000:00:14.0-usb3-3\x2d10-3\x2d10:1.0-bluetooth-hci0-hci0:3585.device                                   loaded active plugged   /sys/devices/pci0000:00/0000:00:14.0/usb3/3-10/3-10:1.0/bluetooth/hci0/hci0:3585
    sys-devices-pci0000:00-0000:00:14.0-usb3-3\x2d10-3\x2d10:1.0-bluetooth-hci0.device                                             loaded active plugged   /sys/devices/pci0000:00/0000:00:14.0/usb3/3-10/3-10:1.0/bluetooth/hci0
    sys-subsystem-bluetooth-devices-hci0.device                                                                                    loaded active plugged   /sys/subsystem/bluetooth/devices/hci0
    sys-subsystem-bluetooth-devices-hci0:3585.device                                                                               loaded active plugged   /sys/subsystem/bluetooth/devices/hci0:3585
    bluetooth.service                                                                                                              loaded active running   Bluetooth service
    bluetooth.target                                                                                                               loaded active active    Bluetooth Support
  ```

  ### List jobs

  Requests to activate, reactivate and restart units are called **jobs** in
  `systemd`. They can be thought of as unit state changes. Current jobs can be
  listed with `systemctl list-jobs`.

  This will most likely return `No jobs running` if the computer has been running
  for a while. Most jobs execute at boot.

  ### Enable/disable, start/stop units

  If a unit has dependencies it is necessary to _enable_ it before starting it.
  This installs the dependencies.

  ```bash
  systemctl enable mongodb.service
  Created symlink /etc/systemd/system/multi-user.target.wants/mongodb.service → /usr/lib/systemd/system/mongodb.service.
  ```

  Then we can start:

  ```
  systemctl start mongodb.service
  ```

  To stop the service:

  ```
  systemctl stop mongodb.service
  ```

  After this we should disable it, in order to remove any symbolic links it has
  created on the system as part of the install process:

  ```bash
  systemctl disable mongodb.service
  Removed "/etc/systemd/system/multi-user.target.wants/mongodb.service".
  ```

  ## Why use systemd over cron ?

  https://mark.stosberg.com/2016-08-26-rsnapshot-and-systemd/


path: systemd_status.md
sortable_path: systemd_status.md
title: Get status of a specific unit
lead: Here I have requested the status of the currently running `mongodb` unit:
checksum: 9e8922e6a289c24e65832dd419b4392ebc960a9cbbd729bef55efb3fce5146ce
created: 2025-10-01 13:31:03.465940248+00:00
modified: 2025-08-13 15:57:13.989682737+00:00
body:
  Here I have requested the status of the currently running `mongodb` unit:

  ```
  $ systemctl status mongod
  mongodb.service - MongoDB Database Server
       Loaded: loaded (/usr/lib/systemd/system/mongodb.service; enabled; preset: disabled)
       Active: active (running) since Wed 2022-08-17 07:25:27 BST; 24h ago
         Docs: https://docs.mongodb.org/manual
     Main PID: 931 (mongod)
       Memory: 304.1M
          CPU: 2min 18.630s
       CGroup: /system.slice/mongodb.service
               └─931 /usr/bin/mongod --config /etc/mongodb.conf
  Aug 17 07:25:27 archbish systemd[1]: Started MongoDB Database Server.****
  ```

  In addition to the core info it tells us the unit type. In this case it is a
  service.

  We can also view the journal entry for the given unit. This provides you with
  its diagnostic log messages:

  ```
  journalctl --unit=mongodb.service
  - Boot b9565dfe8aca4d069143209b3aa84d8e --
  Aug 05 18:31:30 archbish systemd[1]: Started MongoDB Database Server.
  Aug 06 14:27:33 archbish systemd[1]: mongodb.service: Deactivated successfully.
  Aug 06 14:27:33 archbish systemd[1]: mongodb.service: Consumed 3min 17.598s CPU time.
  -- Boot 01922f84c3bd4b3a8f11824cf05f7320 --
  Aug 07 11:58:09 archbish systemd[1]: Started MongoDB Database Server.
  Aug 08 14:43:01 archbish systemd[1]: mongodb.service: Deactivated successfully.
  Aug 08 14:43:01 archbish systemd[1]: mongodb.service: Consumed 5min 28.760s CPU time.
  -- Boot e52b735e115c43bdad8c00462aaff395 --
  Aug 10 13:13:22 archbish systemd[1]: Started MongoDB Database Server.
  Aug 11 07:46:40 archbish systemd[1]: mongodb.service: Deactivated successfully.
  Aug 11 07:46:40 archbish systemd[1]: mongodb.service: Consumed 2min 16.629s CPU time.
  ```

  Each entry is organised around specific boots.

raw_content:
  ---
  id: v5f6xkdv
  tags:
    - systemd
    - systems-programming
    - Linux
    - procedural
  created: Thursday, February 29, 2024 | 18:19
  ---

  # Get status of a specific unit

  Here I have requested the status of the currently running `mongodb` unit:

  ```
  $ systemctl status mongod
  mongodb.service - MongoDB Database Server
       Loaded: loaded (/usr/lib/systemd/system/mongodb.service; enabled; preset: disabled)
       Active: active (running) since Wed 2022-08-17 07:25:27 BST; 24h ago
         Docs: https://docs.mongodb.org/manual
     Main PID: 931 (mongod)
       Memory: 304.1M
          CPU: 2min 18.630s
       CGroup: /system.slice/mongodb.service
               └─931 /usr/bin/mongod --config /etc/mongodb.conf
  Aug 17 07:25:27 archbish systemd[1]: Started MongoDB Database Server.****
  ```

  In addition to the core info it tells us the unit type. In this case it is a
  service.

  We can also view the journal entry for the given unit. This provides you with
  its diagnostic log messages:

  ```
  journalctl --unit=mongodb.service
  - Boot b9565dfe8aca4d069143209b3aa84d8e --
  Aug 05 18:31:30 archbish systemd[1]: Started MongoDB Database Server.
  Aug 06 14:27:33 archbish systemd[1]: mongodb.service: Deactivated successfully.
  Aug 06 14:27:33 archbish systemd[1]: mongodb.service: Consumed 3min 17.598s CPU time.
  -- Boot 01922f84c3bd4b3a8f11824cf05f7320 --
  Aug 07 11:58:09 archbish systemd[1]: Started MongoDB Database Server.
  Aug 08 14:43:01 archbish systemd[1]: mongodb.service: Deactivated successfully.
  Aug 08 14:43:01 archbish systemd[1]: mongodb.service: Consumed 5min 28.760s CPU time.
  -- Boot e52b735e115c43bdad8c00462aaff395 --
  Aug 10 13:13:22 archbish systemd[1]: Started MongoDB Database Server.
  Aug 11 07:46:40 archbish systemd[1]: mongodb.service: Deactivated successfully.
  Aug 11 07:46:40 archbish systemd[1]: mongodb.service: Consumed 2min 16.629s CPU time.
  ```

  Each entry is organised around specific boots.

metadata:
  {"created":"Thursday, February 29, 2024 | 18:19","id":"v5f6xkdv","tags":["systemd","systems-programming","Linux","procedural"]}


path: v8.md
sortable_path: v8.md
title: v8
checksum: b10a7303881a96d28e9623865c847d3ad3c39bf9056b9d8c1820f7236db2b385
created: 2025-10-01 13:31:03.474994329+00:00
modified: 2025-08-13 15:57:13.894280907+00:00
metadata: {"tags":["node-js"]}
lead:
  The v8 module is useful for providing information about the resource-usage of
  your Node.js apps, such as memory usage and capacity.

body:
  The v8 module is useful for providing information about the resource-usage of
  your Node.js apps, such as memory usage and capacity.

raw_content:
  ---
  tags:
    - node-js
  ---

  # v8

  The v8 module is useful for providing information about the resource-usage of
  your Node.js apps, such as memory usage and capacity.


path: Bluetooth_frequency_hopping.md
sortable_path: Bluetooth_frequency_hopping.md
title: Bluetooth frequency hopping
checksum: 405ac1d32edf093bbdc52a911d53996f40db94293dea36578bf582caa07d39b2
created: 2025-10-03 14:34:08.283158746+00:00
modified: 2025-10-03 09:30:41.188507177+00:00
metadata: {"tags":["bluetooth","networks","radio"]}
lead:
  Bluetooth transmits on the 2.5GHz frequency band. As this is an unlicenced band
  there can be interference to the signal and interception.

body:
  Bluetooth transmits on the 2.5GHz frequency band. As this is an unlicenced band
  there can be interference to the signal and interception.

  To get around this, bluetooth engages in **frequency hopping**. It rapidly
  switches its channel of transmission between 79 distinct frequences within the
  2.5GHz band where each channel occupies a bandwidth of 1MHz.

  Each available band is divided into time slots. This allows multiple devices to
  communicate simultaneously without interfering with each other (that much). The
  [master device](./Bluetooth_piconet.md) controls the sequencing and
  synchronisation of the time slots.

raw_content:
  ---
  tags:
    - bluetooth
    - networks
    - radio
  ---

  # Bluetooth frequency hopping

  Bluetooth transmits on the 2.5GHz frequency band. As this is an unlicenced band
  there can be interference to the signal and interception.

  To get around this, bluetooth engages in **frequency hopping**. It rapidly
  switches its channel of transmission between 79 distinct frequences within the
  2.5GHz band where each channel occupies a bandwidth of 1MHz.

  Each available band is divided into time slots. This allows multiple devices to
  communicate simultaneously without interfering with each other (that much). The
  [master device](./Bluetooth_piconet.md) controls the sequencing and
  synchronisation of the time slots.


path: Bluetooth_piconet.md
sortable_path: Bluetooth_piconet.md
title: Bluetooth piconet
checksum: 6b42882ddc8716cf3551f465e734efdb9d91996a7e1cdac513091da3e233505f
created: 2025-10-03 14:34:08.301586578+00:00
modified: 2025-10-01 13:38:41.934991438+00:00
metadata: {"tags":["bluetooth","networks"]}
lead:
  Bluetooth devices connect via a micro-network known as a "piconet". Devices
  connect to each other in a master/slave relationship (since renamed
  "central/peripheral").

body:
  Bluetooth devices connect via a micro-network known as a "piconet". Devices
  connect to each other in a master/slave relationship (since renamed
  "central/peripheral").

  Prior to the establishment of a piconet, devices remain in discoverable/scanning
  mode.

  At least one master and one slave is required to make a piconet however a master
  can have multiple slaves simultaneously (e.g. a computer with keyboard, mouse,
  speakers etc).

  The master controls the connection timing and frequency modulation (necessary
  for the Bluetooth protocol) and the slave synchronises its clock with the
  master's. A slave can have its own controls (e.g. a set of headphones with
  playback controls) but it is the master that orchestrates the action.

  What does this mean?

  Take a mouse as a slave that is sending the click data to the master it is
  paired with. The mouse's microcontroller controls the click and what it "means"
  (although this is handled at a level up, in the application layer) but _when_
  the data is processed is managed by the master. The master controls:

  - when devices can transmit
  - what frequency they use
  - the timing and synchronisation of the data exchanged

  So in other words, the mouse waits for the master to give it a timeslot once the
  click has occurred. Then it's data is handed off to the application responsible
  on the master device.

raw_content:
  ---
  tags:
    - bluetooth
    - networks
  ---

  # Bluetooth piconet

  Bluetooth devices connect via a micro-network known as a "piconet". Devices
  connect to each other in a master/slave relationship (since renamed
  "central/peripheral").

  Prior to the establishment of a piconet, devices remain in discoverable/scanning
  mode.

  At least one master and one slave is required to make a piconet however a master
  can have multiple slaves simultaneously (e.g. a computer with keyboard, mouse,
  speakers etc).

  The master controls the connection timing and frequency modulation (necessary
  for the Bluetooth protocol) and the slave synchronises its clock with the
  master's. A slave can have its own controls (e.g. a set of headphones with
  playback controls) but it is the master that orchestrates the action.

  What does this mean?

  Take a mouse as a slave that is sending the click data to the master it is
  paired with. The mouse's microcontroller controls the click and what it "means"
  (although this is handled at a level up, in the application layer) but _when_
  the data is processed is managed by the master. The master controls:

  - when devices can transmit
  - what frequency they use
  - the timing and synchronisation of the data exchanged

  So in other words, the mouse waits for the master to give it a timeslot once the
  click has occurred. Then it's data is handed off to the application responsible
  on the master device.


path: Generating_SSH_key_for_server_access.md
sortable_path: Generating_SSH_key_for_server_access.md
title: Generating SSH key for server access
lead: Using local machine generate new key:
checksum: 093d926800373ca5680243f44d0c83ae09be39cdb095d8baaebefee3a877a779
created: 2025-10-03 14:34:08.30992319+00:00
modified: 2025-10-03 14:56:53.803675226+00:00
metadata: {"tags":["servers"]}
body:
  Using local machine generate new key:

  ```sh
  ssh-keygen -t ed25519 -C "user@clientname" -f ~/.ssh/user_clientname
  ```

  This creates a public/private key pair on the local machine, viz:

  ```
  .ssh/user_clientname
  .ssh/user_clientname.pub
  ```

  Get the public key:

  ```sh
  cat .ssh/user_clientname.pub
  ```

  Add to the server's authorized keys:

  ```sh
  echo "PASTE_PUBLIC_KEY_HERE" >> ~/.ssh/authorized_keys
  ```

  Test by using the new public key to connect:

  ```sh
  ssh -i ~/.ssh/user_clientname user@servername
  ```

  Paste the private key into whatever client wants to access the server (e.g.
  Forgejo).

raw_content:
  ---
  tags:
    - servers
  ---

  # Generating SSH key for server access

  Using local machine generate new key:

  ```sh
  ssh-keygen -t ed25519 -C "user@clientname" -f ~/.ssh/user_clientname
  ```

  This creates a public/private key pair on the local machine, viz:

  ```
  .ssh/user_clientname
  .ssh/user_clientname.pub
  ```

  Get the public key:

  ```sh
  cat .ssh/user_clientname.pub
  ```

  Add to the server's authorized keys:

  ```sh
  echo "PASTE_PUBLIC_KEY_HERE" >> ~/.ssh/authorized_keys
  ```

  Test by using the new public key to connect:

  ```sh
  ssh -i ~/.ssh/user_clientname user@servername
  ```

  Paste the private key into whatever client wants to access the server (e.g.
  Forgejo).


path: LoRa_radio.md
sortable_path: LoRa_radio.md
title: LoRa radio
lead:
  LoRa (which stands for "long range") is a proprietary method of radio
  communication.

checksum: a34831be729657c9c3b9414856f213c298b88c860cbdac9f1836a4d8efbcbab7
created: 2025-10-03 14:34:08.315130318+00:00
modified: 2025-10-03 17:17:41.534025427+00:00
metadata: {"tags":["radio","LoRa"]}
body:
  LoRa (which stands for "long range") is a proprietary method of radio
  communication.

  It operates within the unlicenced ISM (Industrial, Scientific and Medical)
  bands. In Europe, the center frequency within the ISM for LoRa is 868MHz.

  It uses a specialised and characteristic technique to avoid interference in
  communication: **chirp-spread-spectrum modulation**. With this technique, radios
  "chirp", meaning they sweep over a range of frequencies over time. This is
  similar to a voice getting higher in pitch.

  This technique gives LoRa several beneficial properties.

  We know that in general, radio signals get weaker with distance. The weaker they
  become, the more susceptable they are to background noise. When a signal falls
  below the noise floor it is lost.

  Noise is noise, because it is random, without uniformity. A chirping signal
  however, has a clear pattern - it rises through both the frequency and time
  domain (it traverses different frequencies over a duration of time). This rising
  pattern can be explicitly filtered for which makes it detectable even amidst
  noise. This means even though a signal may be weak and far away, it can still be
  detected because of it's sweeping pattern, thus making LoRa long range.

  Moreover, a higher spreading rate results in a longer range of transmission and
  reception.

  Thus the key properties of LoRa are:

  - The signal is long range, travelling from 2-15km in urban areas and up to 50Km
    in rural areas

  - Receivers can pick up very weak signals

  - There is good resistance to noise interference since the receiver is trained
    to look for a pattern amidst the noise. Also, if there is interference at one
    frequency this is discounted once the data moves to the next frequency in the
    sweep.

  There are some trade-offs though.

  - A higher spreading rate results in a longer range but slower data transfer
    since each bit takes longer to transmit because it is spread accross more
    chirps.

  - A lower spreading rate results in faster data transfer (increased bandwidth)
    but reduced range

raw_content:
  ---
  tags:
    - radio
    - LoRa
  ---

  # LoRa radio

  LoRa (which stands for "long range") is a proprietary method of radio
  communication.

  It operates within the unlicenced ISM (Industrial, Scientific and Medical)
  bands. In Europe, the center frequency within the ISM for LoRa is 868MHz.

  It uses a specialised and characteristic technique to avoid interference in
  communication: **chirp-spread-spectrum modulation**. With this technique, radios
  "chirp", meaning they sweep over a range of frequencies over time. This is
  similar to a voice getting higher in pitch.

  This technique gives LoRa several beneficial properties.

  We know that in general, radio signals get weaker with distance. The weaker they
  become, the more susceptable they are to background noise. When a signal falls
  below the noise floor it is lost.

  Noise is noise, because it is random, without uniformity. A chirping signal
  however, has a clear pattern - it rises through both the frequency and time
  domain (it traverses different frequencies over a duration of time). This rising
  pattern can be explicitly filtered for which makes it detectable even amidst
  noise. This means even though a signal may be weak and far away, it can still be
  detected because of it's sweeping pattern, thus making LoRa long range.

  Moreover, a higher spreading rate results in a longer range of transmission and
  reception.

  Thus the key properties of LoRa are:

  - The signal is long range, travelling from 2-15km in urban areas and up to 50Km
    in rural areas

  - Receivers can pick up very weak signals

  - There is good resistance to noise interference since the receiver is trained
    to look for a pattern amidst the noise. Also, if there is interference at one
    frequency this is discounted once the data moves to the next frequency in the
    sweep.

  There are some trade-offs though.

  - A higher spreading rate results in a longer range but slower data transfer
    since each bit takes longer to transmit because it is spread accross more
    chirps.

  - A lower spreading rate results in faster data transfer (increased bandwidth)
    but reduced range


path: Create_a_deploy_user.md
sortable_path: Create_a_deploy_user.md
title: Create a deploy user
checksum: c7a438ddeb5cfacff1f1130c3c62536e8534b3b24c38a8520e73624583582462
created: 2025-10-05 18:47:25.516636492+00:00
modified: 2025-10-03 18:54:20.232072745+00:00
metadata: {"tags":["servers"]}
lead:
  When I want to run deploy operations on my server from a client that is not me
  (e.g. Forgejo) it is best to create a user for this purpose that has limited
  conditions - more restricted than my `/home` user.

body:
  When I want to run deploy operations on my server from a client that is not me
  (e.g. Forgejo) it is best to create a user for this purpose that has limited
  conditions - more restricted than my `/home` user.

  Create user on server:

  ```sh
  sudo useradd -m -s /bin/bash deploy
  ```

  > `-m` gives him a home directory which is necessary for him to have an `.ssh/`
  > directory. `/bin/bash` gives him the ability to run shell commands.

  Give permissions:

  ```sh
  sudo chown -R deploy:www-data /var/www
  sudo chmod -R 755 /var/www/
  ```

  This:

  - Adds the user ("deploy") to the group of the default web server user
    (`www-data`)
  - Gives deploy user full read/write/execute

  Then create an SSH key-pair for the deploy user following the steps at
  [Generating an SSH key for server access](./Generating_SSH_key_for_server_access.md)
  and add his public key to the `authorized_keys` file on the server.

  He'll first need his own `.ssh` directory however:

  ```sh
  sudo mkdir -p /home/deploy/.ssh
  sudo touch /home/deploy/.ssh/authorized_keys
  sudo chmod 700 /home/deploy/.ssh
  sudo chmod 600 /home/deploy/.ssh/authorized_keys
  sudo chown -R deploy:deploy /home/deploy/.ssh
  ```

  Then attempt to connect to ensure SSH access is working:

  ```sh
  ssh -i .ssh/deploy_self_host_server deploy@server_ip

  ```

  Now the SSH key associated with the deploy user can be used to remotely execute
  deployment functions on the server from any client.

  Next need to add deploy user's key to Forgejo eolas-api repo and test the action
  again

raw_content:
  ---
  tags:
    - servers
  ---

  # Create a deploy user

  When I want to run deploy operations on my server from a client that is not me
  (e.g. Forgejo) it is best to create a user for this purpose that has limited
  conditions - more restricted than my `/home` user.

  Create user on server:

  ```sh
  sudo useradd -m -s /bin/bash deploy
  ```

  > `-m` gives him a home directory which is necessary for him to have an `.ssh/`
  > directory. `/bin/bash` gives him the ability to run shell commands.

  Give permissions:

  ```sh
  sudo chown -R deploy:www-data /var/www
  sudo chmod -R 755 /var/www/
  ```

  This:

  - Adds the user ("deploy") to the group of the default web server user
    (`www-data`)
  - Gives deploy user full read/write/execute

  Then create an SSH key-pair for the deploy user following the steps at
  [Generating an SSH key for server access](./Generating_SSH_key_for_server_access.md)
  and add his public key to the `authorized_keys` file on the server.

  He'll first need his own `.ssh` directory however:

  ```sh
  sudo mkdir -p /home/deploy/.ssh
  sudo touch /home/deploy/.ssh/authorized_keys
  sudo chmod 700 /home/deploy/.ssh
  sudo chmod 600 /home/deploy/.ssh/authorized_keys
  sudo chown -R deploy:deploy /home/deploy/.ssh
  ```

  Then attempt to connect to ensure SSH access is working:

  ```sh
  ssh -i .ssh/deploy_self_host_server deploy@server_ip

  ```

  Now the SSH key associated with the deploy user can be used to remotely execute
  deployment functions on the server from any client.

  Next need to add deploy user's key to Forgejo eolas-api repo and test the action
  again


path: DHCP.md
sortable_path: DHCP.md
title: DHCP
checksum: 9cbff3d2862baaf6486887ec934e59ae55b982755df43c5c14ead2a1bd5b5dc7
created: 2025-10-06 18:26:26.104832207+00:00
modified: 2025-10-06 18:31:36.075142533+00:00
metadata: {"tags":["internet","networks"]}
lead:
  Stands for **Dynamic Host Configuration Protocol** and operates on consumer
  routers. This is a server protocol.

body:
  Stands for **Dynamic Host Configuration Protocol** and operates on consumer
  routers. This is a server protocol.

  Automatically assigns IP addresses and network configuration settings to devices
  when they connect to a network.

  Saves you from manually configuring each device on the network. Dynamically
  distributes:

  - [IP addresses](./IP_addresses.md)
  - [Subnet masks](./IP_addresses.md)
  - Default gateway addresses
  - DNS server addresses

  > Why was this pertinent to the configuration of my Pihole? DHCP on my router
  > will chose the default DNS server. I wanted to change the default DNS server
  > to use the address of the Pihole on the network. This change has to be made in
  > the DHCP settings.

raw_content:
  ---
  tags: [internet, networks]
  ---

  # DHCP

  Stands for **Dynamic Host Configuration Protocol** and operates on consumer
  routers. This is a server protocol.

  Automatically assigns IP addresses and network configuration settings to devices
  when they connect to a network.

  Saves you from manually configuring each device on the network. Dynamically
  distributes:

  - [IP addresses](./IP_addresses.md)
  - [Subnet masks](./IP_addresses.md)
  - Default gateway addresses
  - DNS server addresses

  > Why was this pertinent to the configuration of my Pihole? DHCP on my router
  > will chose the default DNS server. I wanted to change the default DNS server
  > to use the address of the Pihole on the network. This change has to be made in
  > the DHCP settings.


