PHP was built to answer one web request and then stop. Most PHP applications eventually need to do something that does not fit that model. Merge ten thousand patient records. Rebuild a year of financial reports. Process a big import file. Trying to do this inside the same web request that started it is how you get timeouts, and users who refresh the page halfway through, sending the same job twice by accident. Here is a plan for handling this kind of work properly.
The moment a task might take a long time, or you are not sure how long, the right move is to check the input quickly, hand the actual work to a separate process, and send back a message saying the job has been accepted. The screen can then check on progress or wait for an update. A simple PHP command line process started from the web request, set up so it keeps running even if the web request itself times out, is an easy and solid way to do this. You do not need a full job queue system to get started.
A user can trigger the same job twice by accident. A double click before a button turns off. A retry after a weak network connection. Write every background job so that running it twice with the same input causes no harm. Check whether the end result is already reached before doing the work, instead of assuming the job only ever runs once. Planning for this from the start costs far less than discovering it after payroll runs twice by mistake.
If a job takes ten minutes, someone will ask if it is done after five. Save the job's status, queued, running, finished, or failed, along with a time and a short note, into a table the screen can check. Do not leave the only record of progress inside a server log file that only an engineer can read.
A job that fails quietly is worse than one that fails loudly, because whoever is watching believes the work actually happened. Catch every failure and save enough detail to understand it without running the job again. If the job changes data, make sure a failure partway through does not leave things half changed. Wrap actual units of work in a single transaction, and make sure the job can either pick up safely where it left off, or roll all the way back. Never leave it silently half done.
If a background job only finishes once you raise its memory limit to several gigabytes, that is usually telling you to process the data in smaller batches instead of loading it all at once. Smaller batches limit the damage from a bug, and they let you show actual progress, such as three thousand of ten thousand records done, instead of leaving everyone staring at a blank screen.
Follow these five steps and most of the painful background job stories we have heard from other teams simply will not happen to you.
Maybeach Tech designs background processing systems that hold up under actual production data, not just demo data. Get in touch and tell us about your bottleneck.
