ducklet
d49a241c0c
Our SQLite DB requires UTF-8 strings but the filenames from the file system could use any encoding. Ideally we'd try to find the right encoding, but we're already using `str.encode(errors="replace").decode()` in other places to handle filenames that don't use UTF-8 encoding and so this is a pragmatic solution that should work for all cases, even though we loose information - all non-utf8 chars will be converted to "?". An iteration on this could use an encoding detection mechanism. |
||
|---|---|---|
| metadex | ||
| scripts | ||
| .gitignore | ||
| metadex.ignore | ||
| poetry.lock | ||
| pyproject.toml | ||
| pyrightconfig.json | ||
| README.md | ||
| run | ||
Metadex
Metadex is a database for file metadata. It offers a simple and powerful CLI to scan your file system and find indexed files.
It supports path mapping, fast fuzzy searching, and multiple hosts.
Limited support exists for importing of metadata from external file lists, like for example from ls -R.
It's mostly useful to access file information for files stored remotely or offline. For example it could be used to keep track of the contents of backup DVDs or other cold storage.
Other use cases might also emerge.
All files are indexed with their last modified timestamp and file size.
Since searches are quite fast and flexible the database could even be useful to query local file information, or generate statistics, or graphs like a used space map.
Example usage
metadex scan ~ # Add all files from your home directory to the index.
metadex ls '*.xml' # List all .xml files.
metadex ls '.config/**.json' # List all .json config files.
metadex ls .git --type d # List all .git directories.