Importing Large CSV Files with Laravel Job Batching

Importing a large CSV in a single web request is a recipe for timeouts and exhausted memory. The reliable approach is to stream the file, split it into chunks, and process those chunks as queued jobs inside a batch. Laravel's job batching then gives you progress, completion callbacks, and failure handling for free. Let us build a customer import for the PHP Architect CRM.

Prerequisites

Job batching needs a job_batches table and a queue driver that supports it, such as Redis or the database driver.

php artisan make:queue-batches-table
php artisan migrate

Step 1: Stream the File with LazyCollection

The trick to not blowing your memory budget is to never load the whole file at once. LazyCollection reads the file line by line, and chunk() groups rows without materializing everything.

use Illuminate\Support\LazyCollection;

LazyCollection::make(function () use ($path) {
    $handle = fopen($path, 'r');
    $header = fgetcsv($handle);

    while (($row = fgetcsv($handle)) !== false) {
        yield array_combine($header, $row);
    }

    fclose($handle);
})->chunk(500);

Each element is now an associative array keyed by the CSV header, and we get them 500 at a time.

Step 2: A Job That Imports One Chunk

Create a batchable job. The Batchable trait gives it access to the batch it belongs to, and lets the batch cancel it if a sibling fails.

namespace App\Jobs;

use App\Models\Customer;
use Illuminate\Bus\Batchable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;

class ImportCustomerChunk implements ShouldQueue
{
    use Batchable, Queueable;

    public function __construct(public array $rows) {}

    public function handle(): void
    {
        if ($this->batch()?->cancelled()) {
            return;
        }

        $records = collect($this->rows)->map(fn ($row) => [
            'name' => $row['name'],
            'email' => strtolower(trim($row['email'])),
            'created_at' => now(),
            'updated_at' => now(),
        ])->all();

        Customer::upsert($records, uniqueBy: ['email'], update: ['name', 'updated_at']);
    }
}

Using upsert() means re-running the import will not create duplicate customers, which makes the whole operation safely repeatable.

Step 3: Dispatch the Batch

Now tie it together. We map each chunk to a job and hand the array to Bus::batch(), attaching callbacks for progress, completion, and failure.

use App\Jobs\ImportCustomerChunk;
use Illuminate\Support\Facades\Bus;
use Illuminate\Support\LazyCollection;
use Throwable;

$jobs = LazyCollection::make(function () use ($path) {
    $handle = fopen($path, 'r');
    $header = fgetcsv($handle);
    while (($row = fgetcsv($handle)) !== false) {
        yield array_combine($header, $row);
    }
    fclose($handle);
})
->chunk(500)
->map(fn ($chunk) => new ImportCustomerChunk($chunk->all()));

$batch = Bus::batch($jobs)
    ->name('Customer import')
    ->allowFailures()
    ->then(fn () => logger()->info('Import finished cleanly.'))
    ->catch(fn ($batch, Throwable $e) => logger()->error('Import chunk failed', ['error' => $e->getMessage()]))
    ->finally(fn ($batch) => logger()->info('Import batch done.'))
    ->dispatch();

Because LazyCollection is lazy all the way through, the file is still only read chunk by chunk as the jobs are created, so even a million-row file never lives in memory all at once.

Step 4: Report Progress to the User

The dispatch returns a Batch object with an id. Store it, then poll the batch for progress from your frontend.

$batch = Bus::findBatch($batchId);

return response()->json([
    'progress' => $batch->progress(),      // 0–100
    'processed' => $batch->processedJobs(),
    'total' => $batch->totalJobs,
    'failed' => $batch->failedJobs,
    'finished' => $batch->finished(),
]);

A small polling endpoint like this is all you need to drive a progress bar while the import runs in the background.

Don't Forget the Worker

None of this happens until a queue worker is running. In development that is php artisan queue:work, and in production you will want a process manager like Supervisor, or Laravel Horizon if you are on Redis, to keep workers alive and give you a dashboard.

Wrapping Up

The pattern generalizes far beyond CSV imports. Any large, chunkable workload, such as recalculating balances, regenerating thumbnails, or sending a newsletter, fits the same shape: stream the source, chunk it, dispatch batchable jobs, and track progress through the batch. Once you have built it once, timeouts on bulk operations stop being something you worry about.