Presto+yanagishima部署

2022-06-29 15:03:31 浏览数 (1)

1 部署Presto

1.1 下载Presto

Presto下载页面 https://prestodb.io/docs/current/installation/deployment.html

下载 presto-server-0.273.3.tar.gz后上传到服务器并解压

代码语言：javascript复制
tar -zxvf presto-server-0.273.3.tar.gz -C /opt
mv /opt/presto-server-0.273.3 /opt/presto

1.2 修改配置

创建目录

代码语言：javascript复制
cd /opt/presto
mkdir  etc/

创建配置文件

代码语言：javascript复制
vi etc/node.properties
代码语言：javascript复制
    node.environment=production
    node.id=ffffffff-ffff-ffff-ffff-ffffffffffff
    node.data-dir=/opt/presto/data
代码语言：javascript复制
vi etc/jvm.config
代码语言：javascript复制
-Xmx16G
-XX: UseG1GC
-XX:G1HeapRegionSize=32M
-XX: UseGCOverheadLimit
-XX: ExplicitGCInvokesConcurrent
-XX: HeapDumpOnOutOfMemoryError
-XX: ExitOnOutOfMemoryError
-XX: CMSClassUnloadingEnabled
-XX: AggressiveOpts
-XX:OnOutOfMemoryError=kill -9 %p
-XX:ReservedCodeCacheSize=150M
代码语言：javascript复制
vi etc/config.properties
代码语言：javascript复制
    coordinator=true
    node-scheduler.include-coordinator=true
    http-server.http.port=8080
    query.max-memory=5GB
    query.max-memory-per-node=1GB
    query.max-total-memory-per-node=2GB
    discovery-server.enabled=true
    discovery.uri=http://10.1.13.193:8080
代码语言：javascript复制
vi etc/log.properties
代码语言：javascript复制
    com.facebook.presto=INFO
代码语言：javascript复制
mkdir etc/catalog
vi etc/catalog/jmx.properties
代码语言：javascript复制
    connector.name=jmx

1.3 创建不同的数据源

代码语言：javascript复制
vi mysql.properties
代码语言：javascript复制
connector.name=mysql
connection-url=jdbc:mysql://10.1.13.199:3306?useSSL=false
connection-user=root
connection-password=paaaswd
代码语言：javascript复制
vi oracle.properties
代码语言：javascript复制
connector.name=oracle
connection-url=jdbc:oracle:thin:test1/pass$123@//10.1.13.223:1521/orcl
代码语言：javascript复制
vi mongodb.properties
代码语言：javascript复制
connector.name=mongodb
mongodb.seeds=10.1.13.12:27017
mongodb.schema-collection=admin
代码语言：javascript复制
vi sqlserver.properties
代码语言：javascript复制
connector.name=sqlserver
connection-url=jdbc:sqlserver://10.1.13.13:1433
connection-user=sa
connection-password=passwd

vi redis.properties

代码语言：javascript复制
connector.name=redis
redis.table-names=schema1.table1,schema1.table2
redis.nodes=10.1.13.14:6379

1.4 启动Presto

代码语言：javascript复制
cd /opt/presto/bin
./launcher start

1.5 访问Presto页面

浏览器访问 http://PrestoIP:8080

2 部署yanagishima

2.1 下载解压yanagishima

下载yanagishima20.zip 上传服务器并解压

代码语言：javascript复制
mkdir /opt/yanagishima
cd /opt/yanagishima
unzip yanagishima-20.zip

2.2 修改配置文件

代码语言：javascript复制
cd /opt/yanagishima/conf
代码语言：javascript复制
vi yanagishima.properties
代码语言：javascript复制
# yanagishima web port.
jetty.port=7080
# 30 minutes. If presto query exceeds this time, yanagishima cancel the query.
presto.query.max-run-time-seconds=1800
# 1GB. If presto query result file size exceeds this value, yanagishima cancel the query.
presto.max-result-file-byte-size=1073741824
# you can specify freely. But you need to specify same name to presto.coordinator.server.[...] and presto.redirect.server.[...] and catalog.[...] and schema.[...]
presto.datasources=your-presto
auth.your-presto=false
# presto coordinator url
presto.coordinator.server.your-presto=http://10.1.13.193:8080
# almost same as presto coordinator url. If you use reverse proxy, specify it
presto.redirect.server.your-presto=http://10.1.13.193:8080
# presto catalog name
catalog.your-presto=hive
# presto schema name
schema.your-presto=default
# if query result exceeds this limit, to show rest of result is skipped
select.limit=500
# http header name for audit log
audit.http.header.name=some.auth.header
use.audit.http.header.name=false
# limit to convert from tsv to values query
to.values.query.limit=500
# authorization feature
check.datasource=false
sql.query.engines=presto
# 1GB. If hive query result file size exceeds this value, yanagishima cancel the query.
hive.setup.query.path.your-hive=/opt/yanagishima/conf/hive_setup_query_your-hive

创建hive_setup_query_your-hive

代码语言：javascript复制
touch /opt/yanagishima/conf/hive_setup_query_your-hive

2.3 启动yanagishima

代码语言：javascript复制
cd /opt/yanagishima
nohup ./yanagishima-start.sh >y.log 2>&1 &

2.4 访问yanagishima页面

浏览器访问http://yanagishimaIP:7080 在此界面可以进行多表查询

http tcp/ip hive

0 人点赞