feat: add full text search virt table for entries

README.md

@@ -1,15 +1,23 @@
 # eolas-db
 
-This CLI application parses entries in my
+A Python application that parses Markdown entries in my
 [zettelkasten](https://github.com/thomasabishop/eolas), extracts the content
-and metadata for each entry and inserts it into an SQLite database.
+and metadata and inserts it into an SQLite database.
 
-It takes two parameters:
+It is a constituent part of my knowledge management system comprising [eolas](https://forgejo.systemsobscure.net/thomasabishop/eolas),
+[eolas-api](https://forgejo.systemsobscure.net/thomasabishop/eolas-api), and [eolas-app](https://forgejo.systemsobscure.net/thomasabishop/eolas-app).
+
+Usage:
+
+```sh
+app.py --source="/my-zettelkasten-entries" --target="/tmp/zettelkasten-output-dir"
+
+```
 
 - `--source`
-  - A file reference to the location of the markdown files comprising the Zettelkasten
+  - The directory containing the Markdown files
 - `--target`
-  - A file reference designating where the SQLite file should be saved.
+  - The location where the resulting SQLite file should be save
 
 ## Local development
 
@@ -24,33 +32,18 @@ python3 src/app.py --source="/my-zettelkasten-entries" --target="/tmp/zettelkast
 
 ```sh
 source venv/bin/activate
-pyinstaller --onefile ${HOME}/eolas-db/src/app.py
+sudo pyinstaller --onefile ./src/app.py --distpath="/usr/local/bin/eolas-db"
 ```
 
-Will build to the `/dist` directory of this repo when run locally.
-
 ### Run executable
 
 ```sh
-${HOME}/eolas-db/dist/app --source="/my-zettelkasten-entries" --target="/tmp/zettelkasten-output-dir"
+/usr/local/bin/eolas-db/app --source="/my-zettelkasten-entries" --target="/tmp/zettelkasten-output-dir"
 ```
 
 ## ERM
 
 ![ERM diagram for eolas-db](./eolas-db-ERM.png)
 
-## Related projects
+> In addition to the tables listed in the diagram, there is a [full-text search](https://sqlite.org/fts5.html) virtual table applied to `entries`. This enables search queries against the text of every entry.
 
-### [eolas](https://forgejo.systemsobscure.net/thomasabishop/eolas)
-
-The repository for the Zettelkasten that this application reads from.
-
-### [eolas-api](https://forgejo.systemsobscure.net/thomasabishop/eolas-api)
-
-A NodeJS Express API which sources its data from database created by
-this application.
-
-### [eolas-app](https://forgejo.systemsobscure.net/thomasabishop/eolas-app)
-
-A web frontend for the application, deployed at
-[eolas-app.systemsobscure.net](https://eolas-app.systemsobscure.net).

src/services/table_service.py

@@ -68,10 +68,12 @@ class TableService(SqliteService):
                             },
                         )
 
-        print("TB-INFO Junction tables populated")
+    def __populate_fts_tables(self):
+        self._execute("INSERT INTO entries_fts(entries_fts) VALUES('rebuild')")
 
     def populate_tables(self, entries: list[IEntry]):
         self.__drop_tables()
         self.__create_tables()
         self.__populate_base_tables(entries)
         self.__populate_junction_tables(entries)
+        self.__populate_fts_tables()

src/sql/create_tables.py

@@ -32,9 +32,15 @@ CREATE TABLE IF NOT EXISTS entries_tags (
     PRIMARY KEY (entry_title, tag_name)
 )"""
 
+CREATE_ENTRIES_FTS_TABLE = """
+    CREATE VIRTUAL TABLE entries_fts USING fts5(title, body, content=entries, content_rowid=rowid)
+"""
+
+
 tables = [
     {"name": "entries", "create_statement": CREATE_ENTRIES_TABLE},
     {"name": "tags", "create_statement": CREATE_TAGS_TABLE},
     {"name": "backlinks", "create_statement": CREATE_BACKLINKS_TABLE},
     {"name": "entries_tags", "create_statement": CREATE_ENTRIES_TAGS_TABLE},
+    {"name": "entries_fts", "create_statement": CREATE_ENTRIES_FTS_TABLE},
 ]