yii2的Console定时任务执行多进程10w+万条模拟数据插入

2022-07-27 16:19:38 浏览数 (2)

版本介绍: yii2 版本
代码语言:javascript复制
zhengniu@zhengdembp:~/basic$     ./yii

This is Yii version 2.0.15.1.

The following commands are available:
php版本 注:要有pcntl扩展(php -m查询是否有此扩展)
代码语言:javascript复制
PHP 7.3.2 (cli) (built: Feb 14 2019 10:08:45) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.2, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.3.2, Copyright (c) 1999-2018, by Zend Technologies

mysql 数据库(test1) 数据表

代码语言:javascript复制
mysql> desc test1;
 ------------ ------------- ------ ----- --------- ---------------- 
| Field      | Type        | Null | Key | Default | Extra          |
 ------------ ------------- ------ ----- --------- ---------------- 
| id         | int(11)     | NO   | PRI | NULL    | auto_increment |
| name       | varchar(32) | NO   |     |         |                |
| createTime | datetime    | NO   |     | NULL    |                |
 ------------ ------------- ------ ----- --------- ---------------- 

具体代码:

代码语言:javascript复制
<?php
/**
 * Created by PhpStorm.
 * User: zhengniu
 * Date: 2019/3/27
 * Time: 8:22 PM
 */

namespace appcommands;

use yiiconsoleController;

class BatchController extends Controller
{
    const TOTAL = 100000; //总条数
    const NUM = 10; //进程数
    protected $per;//分页

    /**
     * 模拟并发请求,10万次写入数据库
     * 拆分为10个进程,每个进程处理一万条插入
     */
    public function actionIndex()
    {
        if (!function_exists("pcntl_fork")) {
            die("pcntl extention is must !");
        }
        $this->per = self::TOTAL / self::NUM;
        $child = [];
        echo 'start ' . microtime(true) . PHP_EOL;
        for ($i = 1; $i <= self::NUM; $i  ) {
            $pid = pcntl_fork();
            if ($pid == -1) {
                die('fork error');
            }
            if ($pid > 0) {//主进程
                $child[] = $pid;
            } else if ($pid == 0) {//子进程
                $link = mysqli_connect('localhost', 'root', '123456', 'test');
                $start = ($i - 1) * $this->per   1;
                $end = $start   $this->per;
                for ($j = $start; $j < $end; $j  ) {
                    $time = date('Y-m-d H:i:s',time());
                    $sqlTpl = 'INSERT INTO %s (`name`,`createTime`) VALUES (%s, "%s")';
                    $sql = sprintf($sqlTpl,'test1',$j, $time);
                    mysqli_query($link, $sql);
                }
                mysqli_close($link);
                $id = getmypid();
                echo 'child ' . $id . ' finished ' . microtime(true) . PHP_EOL;
                exit(0);
            }
        }
        while (count($child)) {
            foreach ($child as $k => $pid) {
                $res = pcntl_waitpid($pid, $status, WNOHANG);
                if (-1 == $res || $res > 0) {
                    unset($child[$k]);
                }
            }
            sleep(1);  
        }
        echo 'end ' . microtime(true) . PHP_EOL;
    }
}

命令行进入项目根目录 (如下我的是basic目录)

代码语言:javascript复制
zhengniu@zhengdembp:~/basic$     ls
LICENSE.md       commands         config           helper           modules          tests            web
README.md        composer.json    controllers      mail             requirements.php vendor           yii
assets           composer.lock    createSql        models           runtime          views            yii.bat

执行 ./yii controllerName/methodName

查看插入10w条数据用时:

代码语言:javascript复制
zhengniu@zhengdembp:~/basic$     ./yii batch/index
start 1553696914.2219
child 3059 finished 1553696919.0115
child 3061 finished 1553696919.0741
child 3060 finished 1553696919.08
child 3053 finished 1553696919.1264
child 3055 finished 1553696919.1543
child 3058 finished 1553696919.2006
child 3054 finished 1553696919.215
child 3052 finished 1553696919.2192
child 3056 finished 1553696919.2406
child 3057 finished 1553696919.2828
end 1553696919.2875

查看数据库是否插入:

代码语言:javascript复制
mysql> select count(id) from test1;
 ----------- 
| count(id) |
 ----------- 
|    100000 |
 ----------- 
1 row in set (0.03 sec)</code></pre>
说明:
代码语言:javascript复制
<?php
 
$pid = pcntl_fork();
//父进程和子进程都会执行下面代码
if ($pid == -1) {
    //错误处理:创建子进程失败时返回-1.
     die('could not fork');
} else if ($pid) {
     //父进程会得到子进程号,所以这里是父进程执行的逻辑
     pcntl_wait($status); //等待子进程中断,防止子进程成为僵尸进程。
} else {
     //子进程得到的$pid为0, 所以这里是子进程执行的逻辑。
}
 
?>

0 人点赞