James Sinclair

Simple Parallel Processing in PHP with pcntl_fork()

Written by James Sinclair on the 19th September 2013

For a recent project I needed to process some code in parallel. A colleague pointed me to the pcntl_fork() function, and it turns out it’s not quite as scary and complicated as I thought. So I turned the simple use case into a generic function. You use it something like this:

<?php
// A function to run
$makeDir = function($a) {
    shell_exec('mkdir '.shellescapearg($a));
}
 
// An array to process
$dirnames = array('a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k');
 
// Run the process in parallel.
processParallel($makeDir, $dirnames, 8);

The actual function is below:

<?php
/**
 * Process in Parallel.
 *
 * Run a function (with no return result) on each item in an array in parallel.
 * Note: This function is only useful if order is not important, and you don't
 * need any return values from the function (i.e. no inter-process communication).
 *
 * @param mixed   $func  A closure function to apply to each item in parallel.
 * @param array   $arr   The array to apply function to.
 * @param integer $procs Number of processes to run in parallel.
 *
 * @return void
 */
function processParallel($func, array $arr, $procs=4)
{
    // Break array up into $procs chunks.
    $chunks   = array_chunk($arr, ceil((count($arr) / $procs)));
    $pid      = -1;
    $children = array();
    foreach ($chunks as $items) {
        $pid = pcntl_fork();
        if ($pid === -1) {
             die('could not fork');
        } else if ($pid === 0) {
            // We are the child process. Pass a chunk of items to process.
            array_walk($items, $func);
            exit(0);
        } else {
            // We are the parent.
            $children[] = $pid;
        }
    }
 
    // Wait for children to finish.
    foreach ($children as $pid) {
        // We are still the parent.
        pcntl_waitpid($pid, $status);
    }
}

Free Cheat Sheet

Ever forget which JavaScript array method does what? Let the Civilised Guide to JavaScript Array Methods gently guide you to the right one. It’s free for anyone who subscribes to receive updates.

Acquire your copy
  • Let me know your thoughts via the Twitters
  • Subscribe to receive updates via the electronic mail system
  • Essays
  • Web Development
  • Book
  • Stories
  • About
© 2023 James Sinclair. Subscribe via the electronic mail.