Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
5684249
Benchmark SQL batch result storage and retain the measurements
binaryfire Nov 20, 2026
61e3873
Plan duplicate-safe batch accounting with per-job results
binaryfire Nov 20, 2026
62c5e43
Document chain reuse and transaction-safe pruning
binaryfire Nov 20, 2026
144279c
Pin the batch storage comparison to its measured baseline
binaryfire Nov 20, 2026
eb22f0d
Share generation of missing table migrations
binaryfire Nov 20, 2026
81c776c
Generate the configured batch and results table migrations
binaryfire Nov 20, 2026
4e8c78d
Install the batch counters and per-job results schema
binaryfire Nov 20, 2026
fc09dc0
Count logical batch results atomically and preserve retry lookup IDs
binaryfire Nov 20, 2026
095790c
Cover batch lifecycle, queue identity and configuration contracts
binaryfire Nov 20, 2026
2bb9901
Update Horizon batch fixtures and verify retry identity
binaryfire Nov 20, 2026
f0c3747
Exercise overlapping batch updates and pruning on all SQL drivers
binaryfire Nov 20, 2026
7f299cb
Reproduce chain redelivery on the database queue
binaryfire Nov 20, 2026
869413b
Document duplicate-safe batches and Laravel porting requirements
binaryfire Nov 20, 2026
f4f45c0
Return the batch total at the accounting write
binaryfire Nov 20, 2026
27979c4
Retain the failed delivery UUID across Horizon retries
binaryfire Nov 20, 2026
c7f434f
Deliver completion callbacks for the batch that finished
binaryfire Nov 20, 2026
1435bb7
Exercise outstanding retry membership on fake batches
binaryfire Nov 20, 2026
18da1dc
Report cache migration creation only when a file was written
binaryfire Nov 20, 2026
24e8285
Document batch completion snapshots and retry IDs
binaryfire Nov 20, 2026
af70b02
Identify the exact baseline for the batch benchmark
binaryfire Nov 20, 2026
03201eb
Retry the outstanding batch delivery from the Horizon dashboard
binaryfire Nov 20, 2026
b60b41e
Clarify stale batch totals and missing-model retry precedence
binaryfire Nov 20, 2026
934aed8
Clarify the batch repository extension point
binaryfire Nov 20, 2026
0f951ed
Identify the unindexed benchmark variant and batch key representation
binaryfire Nov 20, 2026
b444364
Wait for the parent update when testing a PostgreSQL recovery
binaryfire Nov 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
438 changes: 51 additions & 387 deletions docs/plans/todo/batch-success-counting.md

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions skeletons/hypervel/config/queue.php
Original file line number Diff line number Diff line change
Expand Up @@ -182,15 +182,15 @@
| Job Batching
|--------------------------------------------------------------------------
|
| The following options configure the database and table that store job
| batching information. These options can be updated to any database
| connection and table which has been defined by your application.
| Here you may configure the database connection and tables that store
| your job batches and the results of individual jobs in each batch.
|
*/

'batching' => [
'database' => env('DB_CONNECTION', 'sqlite'),
'table' => 'job_batches',
'results_table' => 'job_batch_results',
],

/*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,12 @@ public function up(): void
$table->integer('total_jobs');
$table->integer('pending_jobs');
$table->integer('failed_jobs');
$table->json('failed_job_ids');
$table->integer('outstanding_failed_jobs');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: This only changes the original create migration, so applications that already ran it will not gain this column; the new results migration only creates the child table. DatabaseBatchRepository now reads and writes outstanding_failed_jobs, causing batch operations on those schemas to fail; add a forward migration that updates existing batch tables and failure data.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At skeletons/hypervel/database/migrations/0001_01_01_000005_create_job_batches_table.php, line 21:

<comment>This only changes the original create migration, so applications that already ran it will not gain this column; the new results migration only creates the child table. `DatabaseBatchRepository` now reads and writes `outstanding_failed_jobs`, causing batch operations on those schemas to fail; add a forward migration that updates existing batch tables and failure data.</comment>

<file context>
@@ -18,11 +18,12 @@ public function up(): void
             $table->integer('pending_jobs');
             $table->integer('failed_jobs');
-            $table->json('failed_job_ids');
+            $table->integer('outstanding_failed_jobs');
             $table->mediumText('options')->nullable();
             $table->integer('cancelled_at')->nullable();
</file context>

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hypervel 0.4 uses a fresh schema. Its batch and result migrations create both tables in their final form. Applications porting a retained Laravel database should follow the framework-table instructions in the porting guide.

$table->mediumText('options')->nullable();
$table->integer('cancelled_at')->nullable();
$table->integer('created_at');
$table->integer('finished_at')->nullable();
$table->index(['finished_at', 'created_at']);
});
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
<?php

declare(strict_types=1);

use Hypervel\Database\Migrations\Migration;
use Hypervel\Database\Schema\Blueprint;
use Hypervel\Support\Facades\Schema;

return new class extends Migration {
/**
* Run the migrations.
*/
public function up(): void
{
Schema::create('job_batch_results', function (Blueprint $table) {
$table->string('batch_id');
$table->char('job_id_hash', 32);
$table->longText('retry_id')->nullable();
$table->primary(['batch_id', 'job_id_hash']);

if (in_array(Schema::getConnection()->getDriverName(), ['pgsql', 'sqlite'], true)) {
$table->index('batch_id')->whereNotNull('retry_id');
}
});
}

/**
* Reverse the migrations.
*/
public function down(): void
{
Schema::dropIfExists('job_batch_results');
}
};
6 changes: 3 additions & 3 deletions skeletons/react-starter-kit/config/queue.php
Original file line number Diff line number Diff line change
Expand Up @@ -182,15 +182,15 @@
| Job Batching
|--------------------------------------------------------------------------
|
| The following options configure the database and table that store job
| batching information. These options can be updated to any database
| connection and table which has been defined by your application.
| Here you may configure the database connection and tables that store
| your job batches and the results of individual jobs in each batch.
|
*/

'batching' => [
'database' => env('DB_CONNECTION', 'sqlite'),
'table' => 'job_batches',
'results_table' => 'job_batch_results',
],

/*
Expand Down
6 changes: 6 additions & 0 deletions src/bus/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,10 @@ Hypervel does not include Laravel's DynamoDB batch repository because DynamoDB i

`DatabaseBatchRepository::setConnection()` is intentionally omitted. The repository is shared for the worker lifetime, so mutating its connection would race across coroutines. Configure `queue.batching.database` instead; each repository operation resolves that connection when it runs.

Batch repositories count by logical job ID. A failure carries its delivery UUID as `$retryId` through `recordFailedJob()` and `incrementFailedJobs()` when it differs from the counting ID. Implement `getFailedJobIds()` for retry lookup IDs and `hasFailedJob()` for membership by logical ID. Batch constructors and factories accept a nullable failed-ID list for [lazy loading](https://hypervel.org/docs/queues#inspecting-batches).

Overrides of `Batch::invokeCallbacks()` must accept the optional trailing `?int $expectedTotalJobs` argument. It skips a completion callback group when the snapshot it would receive includes jobs added since the accounting write.

Subclasses customizing `DatabaseBatchRepository` accounting should override the counting methods. Laravel's protected `updateAtomicValues()` hook is omitted.

Ported from: https://github.com/laravel/framework/tree/13.x/src/Illuminate/Bus
Loading