简介: Anaconda,中文大蟒蛇,是一个开源的Python发行版本,其包含了conda、Python等180多个科学包及其依赖项。
Anaconda3
Anaconda,中文大蟒蛇,是一个开源的Python发行版本,其包含了conda、Python等180多个科学包及其依赖项。我当初安装这个是为了学习人工智能和算法。
1、安装
1、命令操作
安装好之后,可以使用命令conda,可以使用一系列命令参数,conda --help 或 conda -h 、 conda --version 或conda -V
例如,使用conda info命令如下所示信息:
代码语言:javascript复制$xxx:~ xxx$ conda info
active environment : None
shell level : 0
user config file : /Users/xxx/.condarc
populated config files : /Users/xxx/.condarc
conda version : 4.8.2
conda-build version : 3.18.11
python version : 3.7.6.final.0
virtual packages : __osx=10.15.7
base environment : /Users/xxx/opt/anaconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /Users/xxx/opt/anaconda3/pkgs
/Users/xxx/.conda/pkgs
envs directories : /Users/xxx/opt/anaconda3/envs
/Users/xxx/.conda/envs
platform : osx-64
user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.6 Darwin/19.6.0 OSX/10.15.7
UID:GID : 501:20
netrc file : None
offline mode : False
2、导出所有存在依赖
1)导出所有依赖包到记事本
conda list -e > [fileName(.txt)]
例如,conda list -e > /usr/local/data/conda-lib-98.txt
1、Kafka 集群防火墙
2、运行情况,或者执行命令,查看status
3、手动执行消费者或生产者,订阅消息,查看消费情况
问题记录FQA
Q1、ModuleNotFoundError: No module named 'pymysql'
Q1:执行python程序,出现 ``ModuleNotFoundError: No module named 'pymysql'
A1:
1)查找确认依赖库
初步判定是安装阿naconda3缺少pymysql依赖包,执行命令`` conda list `` 分别查看85服务器和96服务器anaconda3的已经安装插件库,依赖包如下所示,85服务器已安装插件库295个插件,96服务器已安装插件库288个插件
2)查找、下载依赖库
85插件所在路径, /data/_software/_install_pack/_python/,anaconda3的home路径,/data/_software/python/anaconda3/
86插件所在路径,, /data/_software/_pack/_python/,anaconda3的home路径,/data/_software/python/anaconda3/
96插件所在路径,/data/_software/_package/,anaconda3的home路径,/data/_software/python/anaconda3/
经过插件列表比对并排查,发现96服务器缺失插件列表如下所示
85服务器有whl插件源件:
- PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
- bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
- confluent_kafka-1.1.0-cp37-cp37m-manylinux1_x86_64.whl
- paramiko-2.6.0-py2.py3-none-any.whl
- PyMySQL-0.9.3-py2.py3-none-any.whl
- cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
85服务器未找到whl插件源件(其中avro-python3使用python安装,cx-oracle使用pip安装):
- avro-python3=1.8.2=pypi_0
- cx-oracle=8.0.0=pypi_0
先下载所需要的插件包,whl格式本质上是一个压缩包,里面包含了py文件,以及经过编译的pyd文件,使得可以在不具备编译环境的情况下,选择合适自己的python环境进行安装,简单来说,适合内网环境。
其次有些时候在内网环境中并不可以使用pip命令来下载依赖包,这就出现了需要手动安装依赖包的情况,由于网络不通,采取手动安装插件库,查找85服务器是否存在插件库或网络下载插件库再传给96服务器。经过检查,发现原来85服务器已经下载插件。在85服务器找到所依赖的插件,打包上传到96服务器。
3)安装依赖库
安装插件的命令使用pip,此命令在已经内置在anaconda3的依赖插件中,具体位置为home/lib/python3.7/site-packages。
- 安装已有whl文件
安装bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
代码语言:javascript复制$ pip install bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
Processing ./bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
Requirement already satisfied: cffi>=1.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt==3.1.7) (1.12.3)
Requirement already satisfied: six>=1.4.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt==3.1.7) (1.12.0)
Requirement already satisfied: pycparser in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cffi>=1.1->bcrypt==3.1.7) (2.19)
Installing collected packages: bcrypt
Successfully installed bcrypt-3.1.7
$ pip install confluent_kafka-1.1.0-cp37-cp37m-manylinux1_x86_64.whl
Processing ./confluent_kafka-1.1.0-cp37-cp37m-manylinux1_x86_64.whl
Installing collected packages: confluent-kafka
Successfully installed confluent-kafka-1.1.0
安装PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
先安装PyNaCl,再安装paramiko
代码语言:javascript复制$ pip install PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
Processing ./PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
Requirement already satisfied: cffi>=1.4.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from PyNaCl==1.3.0) (1.12.3)
Requirement already satisfied: six in /data/_software/python/anaconda3/lib/python3.7/site-packages (from PyNaCl==1.3.0) (1.12.0)
Requirement already satisfied: pycparser in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cffi>=1.4.1->PyNaCl==1.3.0) (2.19)
Installing collected packages: PyNaCl
Successfully installed PyNaCl-1.3.0
安装paramiko-2.6.0-py2.py3-none-any.whl
代码语言:javascript复制$ pip install paramiko-2.6.0-py2.py3-none-any.whl
Processing ./paramiko-2.6.0-py2.py3-none-any.whl
Requirement already satisfied: bcrypt>=3.1.3 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from paramiko==2.6.0) (3.1.7)
Requirement already satisfied: pynacl>=1.0.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from paramiko==2.6.0) (1.3.0)
Requirement already satisfied: cryptography>=2.5 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from paramiko==2.6.0) (2.7)
Requirement already satisfied: six>=1.4.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko==2.6.0) (1.12.0)
Requirement already satisfied: cffi>=1.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko==2.6.0) (1.12.3)
Requirement already satisfied: asn1crypto>=0.21.0 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cryptography>=2.5->paramiko==2.6.0) (0.24.0)
Requirement already satisfied: pycparser in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko==2.6.0) (2.19)
Installing collected packages: paramiko
Successfully installed paramiko-2.6.0
安装PyMySQL-0.9.3-py2.py3-none-any.whl
代码语言:javascript复制$ pip install PyMySQL-0.9.3-py2.py3-none-any.whl
Processing ./PyMySQL-0.9.3-py2.py3-none-any.whl
Installing collected packages: PyMySQL
Successfully installed PyMySQL-0.9.3
安装cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
代码语言:javascript复制$ pip install cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
Processing ./cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
Installing collected packages: cx-Oracle
Successfully installed cx-Oracle-8.0.0
安装未有whl文件的tar.gz文件,进入到arvo_python3所在目录
代码语言:javascript复制cd /data/_software/_package
解压arvo_python3-1.8.2.tar.gz,会自动创建并解压到arvo_python3-1.8.2
代码语言:javascript复制tar -zxvf arvo_python3-1.8.2.tar.gz
进入arvo_python3-1.8.2包
代码语言:javascript复制cd arvo_python3-1.8.2
执行命令python setup.py install
代码语言:javascript复制$ python setup.py install
running install
running bdist_egg
running egg_info
writing avro_python3.egg-info/PKG-INFO
writing dependency_links to avro_python3.egg-info/dependency_links.txt
writing top-level names to avro_python3.egg-info/top_level.txt
reading manifest file 'avro_python3.egg-info/SOURCES.txt'
writing manifest file 'avro_python3.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/avro
copying avro/__init__.py -> build/lib/avro
copying avro/datafile.py -> build/lib/avro
copying avro/io.py -> build/lib/avro
copying avro/ipc.py -> build/lib/avro
copying avro/protocol.py -> build/lib/avro
copying avro/schema.py -> build/lib/avro
copying avro/tool.py -> build/lib/avro
copying avro/txipc.py -> build/lib/avro
copying avro/HandshakeRequest.avsc -> build/lib/avro
copying avro/HandshakeResponse.avsc -> build/lib/avro
copying avro/VERSION.txt -> build/lib/avro
copying avro/LICENSE -> build/lib/avro
copying avro/NOTICE -> build/lib/avro
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/__init__.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/datafile.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/io.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/ipc.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/protocol.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/schema.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/tool.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/txipc.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/HandshakeRequest.avsc -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/HandshakeResponse.avsc -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/VERSION.txt -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/LICENSE -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/NOTICE -> build/bdist.linux-x86_64/egg/avro
byte-compiling build/bdist.linux-x86_64/egg/avro/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/datafile.py to datafile.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/io.py to io.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/ipc.py to ipc.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/protocol.py to protocol.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/schema.py to schema.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/tool.py to tool.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/txipc.py to txipc.cpython-37.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
installing scripts to build/bdist.linux-x86_64/egg/EGG-INFO/scripts
running install_scripts
running build_scripts
creating build/scripts-3.7
copying and adjusting scripts/avro -> build/scripts-3.7
changing mode of build/scripts-3.7/avro from 600 to 755
creating build/bdist.linux-x86_64/egg/EGG-INFO/scripts
copying build/scripts-3.7/avro -> build/bdist.linux-x86_64/egg/EGG-INFO/scripts
changing mode of build/bdist.linux-x86_64/egg/EGG-INFO/scripts/avro to 755
copying avro_python3.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying avro_python3.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying avro_python3.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying avro_python3.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
avro.__pycache__.__init__.cpython-37: module references __file__
avro.__pycache__.ipc.cpython-37: module references __file__
creating dist
creating 'dist/avro_python3-1.8.2-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing avro_python3-1.8.2-py3.7.egg
creating /data/_software/python/anaconda3/lib/python3.7/site-packages/avro_python3-1.8.2-py3.7.egg
Extracting avro_python3-1.8.2-py3.7.egg to /data/_software/python/anaconda3/lib/python3.7/site-packages
Adding avro-python3 1.8.2 to easy-install.pth file
Installing avro script to /data/_software/python/anaconda3/bin
Installed /data/_software/python/anaconda3/lib/python3.7/site-packages/avro_python3-1.8.2-py3.7.egg
Processing dependencies for avro-python3==1.8.2
Finished processing dependencies for avro-python3==1.8.2
4)检查确认依赖库
执行安装新插件后,96服务器插件列表为295个插件
执行命令 conda list -e > [输出插件列表到文件.txt]
Q2、sshpass命令没有找到
Q2:执行python程序,提示sshpass命令没有找到
A2:
1)下载 sshpass-1.06.tar.gz
有网络情况下,执行命令
代码语言:javascript复制wget http://sourceforge.net/projects/sshpass/files/latest/download -O sshpass.tar.gz
2)解压
解压sshpass-1.06.tar.gz到当前目录(自动创建目录并写入解压文件)
代码语言:javascript复制tar -zxvf sshpass-1.06.tar.gz
3)编译
进入sshpass-1.06目录,进行编译
代码语言:javascript复制./configure
4)安装
执行安装命令
代码语言:javascript复制sudo make install
5)其他参考
https://linux.cn/article-8086-1.html
Q3、No such file or directory
Q3、执行python程序,控制台打印出来以下错误信息
代码语言:javascript复制Traceback (most recent call last):
File "test_oracle_lyc.py", line 4, in <module>
db = cx_Oracle.connect("xxx.xxx.xxx.xxx", "username", "password", "orclName", charset='utf8' )
cx_Oracle.DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libnsl.so.1: cannot open shared object file: No such file or directory". See https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html for help
A3:
1)python、oracle依赖包
经过排查,发现缺少oracle_client,安装oracle-instantclient19.3-basic-19.3.0.0.0-1.x86_64.rpm
2)安装
使用root用户执行安装命令,安装到默认路径
代码语言:javascript复制rpm -Uvh oracle-instantclient19.3-basic-19.3.0.0.0-1.x86_64.rpm
3)环境变量
使用root用户登录进去配置相关环境变量(这个好像不用配置也可以,我在官方网站看到18 以下数据库客户端才需要配置环境变量,以及手动配置lib库地址和bin目录,现在我们使用的版本为19,已经自动创建libaio以及其他依赖包)。vim /etc/profile,输入环境变量信息
代码语言:javascript复制export ORACLE_HOME=/usr/lib/oracle/19.3/client64
export ORABIN=/usr/lib/oracle/19.3/client64/bin
export LD_LIBRARY_PATH=/usr/lib/oracle/19.3/client64/lib
export PATH=$PATH:$ORACLE_HOME/bin
让环境变量立即生效
代码语言:javascript复制source /etc/profile
4)安装系统依赖包
在root用户下,安装libnsl.so.1,进入到目录查看系统依赖包,由于我们都是使用的64bit操作系统,所有的软件工具也都是64bit
代码语言:javascript复制cd /usr/lib64
查看(ls)是否有libnsl依赖包,在85服务器上,我查询到libnsl* 的依赖包为
代码语言:javascript复制libnsl.so.2
libnsl.so.2.0.0
查看(ll)是否有libnsl依赖包,在85服务器上,我查询到libnsl* 的依赖包为
代码语言:javascript复制lrwxrwxrwx 1 root root 15 Mar 15 2020 libnsl.so.2 -> libnsl.so.2.0.0
-rwxr-xr-x 1 root root 96480 Mar 15 2020 libnsl.so.2.0.0
然后进行拷贝,软连接设置
代码语言:javascript复制cp -r libnsl.so.2 libnsl.so.1
再次执行查看操作
代码语言:javascript复制lrwxrwxrwx 1 root root 15 Jan 23 16:40 libnsl.so.1 -> libnsl.so.2.0.0
lrwxrwxrwx 1 root root 15 Mar 15 2020 libnsl.so.2 -> libnsl.so.2.0.0
-rwxr-xr-x 1 root root 96480 Mar 15 2020 libnsl.so.2.0.0
5)切换测试
新打开一个窗口,使用appuser登录,或者重新登录到服务器,然后编写一个连接oracle的测试脚本,脚本如下
代码语言:javascript复制import cx_Oracle
db = connect("userName", "password", "IP:Port/serviceName", "UTF-8")
print(db.version)
保存代码为 test_oracle.py
执行python test_oracle.py,如果打印出来数据库服务器的版本号,则说明已经连接并成功读取数据。
总结
笔记属于个人的喜好,但是形成了自我的风格就是博文了。希望在记录自己博文道路越走越远。
我正在参与2023腾讯技术创作特训营第三期有奖征文,组队打卡瓜分大奖!