Repository navigation
Feature: Scheduler Refactor - Jobs Store, Separation of Concerns, and Jobs View #2360
Description
Summary
Refactor Vorta's scheduler to improve reliability, maintainability, and user visibility into scheduled and past jobs.
Goals
1. Add Persistent Jobs Store
Current state: EventLogModel only tracks executed jobs. Pause states and pending schedules are stored in-memory and lost on restart.
Goal: Add a database model to track:
- Pending/scheduled jobs (survive restarts)
- Skipped jobs with reasons (audit trail)
- Link between scheduled intent and execution results
2. Separate Scheduling, Execution, and State Management
Current state: VortaScheduler (532 lines) handles scheduling, execution triggering, and state management in one class with manual lock management and interleaved post-backup tasks.
Goal: Split into focused components:
- Scheduling: Calculate next run times, manage timers
- Execution: Create jobs, queue them, handle results, coordinate post-backup tasks
- State: Pause/resume persistence, job status tracking, crash recovery
3. Add Jobs View UI
Current state: Log page only shows past jobs. No view of pending jobs or filtering options.
Goal: New Jobs page showing:
- Pending (scheduled) and completed jobs in one view
- Filtering by profile, repository, job type, and status
- Skip reasons for jobs that didn't run
- Ability to cancel pending or re-queue failed jobs
4. Improve Reliability
Current issues:
- QTimer 32-bit limitation (~24.8 days) with imprecise workaround
- Race conditions between job checks and submission
- Post-backup tasks not guaranteed sequential
- DBus sleep/resume detection fails silently
Goal: More reliable scheduling that handles edge cases gracefully.
Out of Scope
- Cron expression support (e.g., "weekdays at 9am")
- Job dependencies/chaining
- Retry policies for failed jobs
Open Questions
- Should the Jobs view replace the existing Log page or be a separate tab?
- What filtering options are most important?
- How should crash recovery handle jobs that were running when the app crashed?
Related Files
Key files that will be affected:
src/vorta/scheduler.py- Current scheduler (to be refactored)src/vorta/store/models.py- Add new job modelsrc/vorta/borg/jobs_manager.py- Job queue managementsrc/vorta/views/schedule_tab.py- Add Jobs pagesrc/vorta/views/log_page.py- Existing log view
Activity
khadejaa23 commentedon Mar 11, 2026
Hi @m3nu,
I’m a backend-oriented engineer with a strong interest in software architecture, and I’m very interested in tackling the Scheduler Refactor (#2360) for GSoC 2026.
This project aligns perfectly with my focus on system design and persistent state management. To ensure my approach meets the project's standards, are there any smaller, related issues you’d recommend I start with to better familiarize myself with the current scheduler and database implementation, I’d really appreciate your guidance.
false200 commentedon Mar 15, 2026
Hi m3nu, I’ve opened #2438 as an initial Phase 1 contribution for #2360, focused on the persistent jobs store foundation for scheduled backups by adding a JobModel, persisting scheduled backup intent, recording skipped jobs with reasons, and linking scheduled execution results back to the stored job record.
false200 commentedon Mar 16, 2026
@khadejaa23, you mentioned interest in working on the scheduler refactor earlier.
I’ve opened PR #2438 implementing the initial persistent jobs store (JobModel + scheduled backup intent persistence) as a first step toward #2360.
Are you planning to work on the same part of the refactor as well?
khadejaa23 commentedon Mar 16, 2026
Thanks for reaching out and for the update on #2438!
Since you’ve already started with the persistent jobs store (JobModel), I’ll make sure to review your PR to stay aligned. To avoid any conflicts, I’ll focus on a different part of the scheduler refactor within #2360
khadejaa23 commentedon Mar 16, 2026
Since it’s still in the early stages and hasn't been merged yet, I’d like to collaborate on this part.
I’ll review your implementation of JobModel and perhaps suggest some enhancements or start working on the logic that connects this store to the actual scheduling triggers. This way, we can ensure the foundation is solid before moving fully into the rest of #2360.
false200 commentedon Mar 16, 2026
Sounds good. Feel free to review the PR and suggest any improvements if you spot something that could make the JobModel or persistence layer stronger.
false200 commentedon Mar 16, 2026
The remaining parts I planned for #2360 include separating scheduler responsibilities, improving the execution and recovery flow, adding restart/crash recovery, refining timer and wake/network handling, and later introducing the Jobs UI along with broader hardening and tests.
Since you mentioned working on the logic that connects the store to the scheduling triggers, that would fit naturally as the next step once the persistence layer is in place.
false200 commentedon Mar 17, 2026
Hi @khadejaa23, since the workload and architectural considerations for this part are quite large, it might be better if you implement it from scratch as part of the full scheduler refactor.
I’ve closed the PR I opened since it tries to cover multiple parts at once and might be harder to review or could potentially introduce loopholes. Feel free to go ahead and develop it as part of the refactor.
khadejaa23 commentedon Mar 17, 2026
Thank you for the update! I appreciate the trust. I agree that keeping the PRs focused will make the review process much smoother and avoid architectural loopholes.
false200 commentedon Mar 17, 2026
Great, sounds good! Have you started working on the initial parts yet?
khadejaa23 commentedon Mar 17, 2026
Yes, and work on the proposal concurrently
I'm currently drafting a formal proposal for the full refactor to ensure the design is solid before I push the first set of changes.
m3nu commentedon Mar 17, 2026
I reviewed one of the scheduler PRs and it didn't make sense as standalone commit. Needs to be part of a larger project.
false200 commentedon Mar 17, 2026
Yup, that one was mine. It had all of Phase 1 bundled into the same PR, which is probably what caused it to be a bit too large and hard to review as a standalone change.
4 remaining items
williamcancodee commentedon Mar 30, 2026
Hey @m3nu,
I'd love to give this a go. I had previously worked on #2404 , which you reviewed, so I'm already quite comfortable with the Vorta architecture and the underlying Python codebase. I have also been reading through the documentation and repository over the last few weeks for thorough familiarity.
I've just created a discussion thread with a proposal draft on how I intend to go about the implementation to ensure that I'm going the right way: #2485
This is my proposal draft: https://docs.google.com/document/d/1gGLGOfjKeasFIyNFPPbSgR6dfFR-fjYJpLeXPc0P8QI/edit?usp=sharing
Any feedback would be greatly appreciated
Thank you,
William
false200 commentedon Jun 9, 2026
Hi @m3nu, now that GSoC 2026 has concluded, is this issue still open for community contributions?
false200 commentedon Jun 9, 2026
Also, looking ahead to GSoC 2027, are you planning to expand the number of project slots? It looked like only one project was taken on this cycle, and I'd love to know if there'll be more opportunities to contribute under the program next year.
m3nu commentedon Jun 9, 2026
We have 2 GSoC contributors this year.
Likely no expansion of slots without additional maintainers.
m3nu commentedon Jun 9, 2026
This issue is part of GSoC for @ebuzerdrmz44
false200 commentedon Jun 9, 2026
Thanks for the clarification 👍
false200 commentedon Jun 9, 2026
oh cool, congrats @ebuzerdrmz44!
ebuzerdrmz44 commentedon Jul 13, 2026
@m3nu As we wrap up #2361, here is the PR breakdown for the next phase of our roadmap:
Right now,
set_timer_for_profilederives the next-run time on reload usingEventLogModeland profile settings, so we don't actually store it. Only pause states and skip reasons are kept strictly in-memory today. SinceEventLogModelalready serves as the backend for the Log page, building directly on top of it feels like the cleanest path.Proposed plan:
Total timeline is around 4 to 5 weeks. I'll stack the PRs so you can review one while I code the next.
One architectural question for you: I lean toward extending EventLogModel with a status field so all jobs share a single timeline (expanding the Log page into the Jobs view). This keeps a single source of truth and reuses the table model from #2361. Would you prefer this over a separate model and dedicated Jobs tab?
A couple of notes on how I'd handle open questions:
returncode=-1) and finalized on completion, a mid-run crash just leaves a -1 row. The scheduler's catch-up already ignores these, so scheduling self-heals on reload. We just need to mark leftover running rows as interrupted at startup instead of building complex recovery logic.If this looks solid, I'll start PR 1 as soon as #2361 is merged.
add JobModel for persistent scheduler job store (borgbase#2360)
add JobModel for persistent scheduler job store (borgbase#2360)
add JobModel: persistent scheduler job store (#2360, phase 1) (#2530)
detect resume from sleep without logind (#2360, phase 3) (#2537)
add SchedulerPauseModel: persist pauses across restarts (#2360, phase…
m3nu commentedon Sep 26, 2026
Closing, now that #2560 and #2565 are merged. The scheduler is split into scheduling, execution and state (#2558, #2560). Pauses and job records persist in the database (#2530, #2532). The Jobs page shows pending and recorded runs with skip reasons, and can be filtered by profile, repository and status (#2538, #2565). Timers longer than 24.8 days and resume detection without logind are fixed (#2547, #2537).
Two things are left out on purpose:
JobModel.Typehas only one member so far.JobsManagerhas no way to cancel a single job.Either can get its own issue if it's needed. Thanks @ebuzerdrmz44.