Skip to content

Optimise ledger parsing with caching #270

@wuzzapcom

Description

@wuzzapcom

I have a personal Beancount ledger with transactions spanning three years, so the total number of transactions to parse is around 10,000.
On my Raspberry Pi, even a basic query may take tens of seconds when using bean-query. My assumption is that parsing takes a significant amount of time and CPU resources, and that it would be worth optimizing this by using in-memory caching between different bean-query calls.

My idea is the following:

  • bean-query gets a new option --daemon

    • in this mode, the tool spawns a background process tied to a given ledger file

      • when any of the files change, the tool reparses them, possibly with some delay
    • the daemon keeps parsed transactions in memory and serves them to bean-query on request

  • when bean-query is called without --daemon, it looks for a running background daemon and requests the parsed transactions from it

This mode could be useful for server-side use cases such as Fava. I haven’t calculated the RAM usage, but even if it consumes 10 MB of RAM for a 1 MB ledger, that seems like a reasonable trade-off.

So my questions are:

  1. What do you think of this idea in general?
  2. Would you consider implementing it?
  3. If not, would you accept this feature as a pull request and merge it into master?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions