airflow安装教程（local模式）

2022-09-16 14:23:16 浏览数 (1)

环境介绍

​

OS：centos 7.6

python：3.6.8

pip：21.2.4

具体安装

pip配置阿里云清华源

代码语言：javascript复制
mkdir -p $HOME/.config/pip/

vim $HOME/.config/pip/pip.conf

新增如下内容：

代码语言：javascript复制
[global]

index-url = https://pypi.tuna.tsinghua.edu.cn/simple

执行安装：

代码语言：javascript复制
export AIRFLOW_HOME=/data/airflow

AIRFLOW_VERSION=2.2.2

export PYTHON_VERSION=3.6

CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"

pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"

pip install virtualenv

启动airflow

代码语言：javascript复制
airflow standalone

报错

代码语言：javascript复制
[root@hadoop1 ~]# airflow standalone
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 5, in <module>
    from airflow.__main__ import main
  File "/usr/local/lib/python3.6/site-packages/airflow/__init__.py", line 34, in <module>
    from airflow import settings
  File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 35, in <module>
    from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf  # NOQA F401
  File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 1129, in <module>
    conf.validate()
  File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 224, in validate
    self._validate_config_dependencies()
  File "/usr/local/lib/python3.6/site-packages/airflow/configuration.py", line 278, in _validate_config_dependencies
    f"error: sqlite C library version too old (< {min_sqlite_version}). "
airflow.exceptions.AirflowConfigException: error: sqlite C library version too old (< 3.15.0). See https://airflow.apache.org/docs/apache-airflow/2.2.2/howto/set-up-database.html#setting-up-a-sqlite-database

解决方法：

参考：Set up a Database Backend — Airflow Documentation (apache.org)，重新配置sqlite

代码语言：javascript复制
wget https://www.sqlite.org/src/tarball/sqlite.tar.gz
tar xzf sqlite.tar.gz
cd sqlite/
export CFLAGS="-DSQLITE_ENABLE_FTS3 
    -DSQLITE_ENABLE_FTS3_PARENTHESIS 
    -DSQLITE_ENABLE_FTS4 
    -DSQLITE_ENABLE_FTS5 
    -DSQLITE_ENABLE_JSON1 
    -DSQLITE_ENABLE_LOAD_EXTENSION 
    -DSQLITE_ENABLE_RTREE 
    -DSQLITE_ENABLE_STAT4 
    -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT 
    -DSQLITE_SOUNDEX 
    -DSQLITE_TEMP_STORE=3 
    -DSQLITE_USE_URI 
    -O2 
    -fPIC"
export PREFIX="/usr/local"
LIBS="-lm" ./configure --disable-tcl --enable-shared --enable-tempstore=always --prefix="$PREFIX"
make
make install

将

代码语言：javascript复制
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

新增到/etc/profile中，并执行source /etc/profile生效

再次启动airflow

代码语言：javascript复制
airflow standalone

可以看到：

代码语言：javascript复制
 webserver | ____________       _____________
 webserver | ____    |__( )_________  __/__  /________      __
 webserver | ____  /| |_  /__  ___/_  /_ __  /_  __ _ | /| / /
 webserver | ___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 webserver | _/_/  |_/_/  /_/    /_/    /_/  ____/____/|__/
 webserver | [2021-12-03 03:36:04,583] {dagbag.py:500} INFO - Filling up the DagBag from /dev/null
 webserver | [2021-12-03 03:36:04,851] {manager.py:512} WARNING - Refused to delete permission view, assoc with role exists DAG Runs.can_create Admin
 webserver | [2021-12-03 03:36:07 -0500] [5428] [INFO] Starting gunicorn 20.1.0
 webserver | [2021-12-03 03:36:07 -0500] [5428] [INFO] Listening at: http://0.0.0.0:8080 (5428)
 webserver | [2021-12-03 03:36:07 -0500] [5428] [INFO] Using worker: sync
 webserver | [2021-12-03 03:36:07 -0500] [5433] [INFO] Booting worker with pid: 5433
 webserver | [2021-12-03 03:36:07 -0500] [5434] [INFO] Booting worker with pid: 5434
 webserver | [2021-12-03 03:36:07 -0500] [5435] [INFO] Booting worker with pid: 5435
 webserver | [2021-12-03 03:36:07 -0500] [5436] [INFO] Booting worker with pid: 5436

进入$AIRFLOW_HOME查看admin密码

代码语言：javascript复制
cat $AIRFLOW_HOME/standalone_admin_password.txt

但是请注意，SQLite只适合测试场景，不能使用其作为生产环境部署使用，生产环境部署建议使用Postgres或者MySQL。

0 0 投票数

文章评分

本文为从大数据到人工智能博主「xiaozhch5」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。

原文链接：https://cloud.tencent.com/developer/article/2109287

腾讯云开发者社区 sqlite android 数据库

0 人点赞