Impala
默认是以impala
这个超级用户运行服务,执行DML
和DDL
操作的,要实现不同用户之间细粒度的权限控制,需要与Sentry
整合。Sentry
是Apache
下的一个开源项目,它基于RBAC
的授权模型实现了权限控制,Impala
与它整合以后,就能实现不同用户之间在应用层的权限认证,从而控制用户的DML
、DDL
、DCL
操作权限。Sentry
为确保数据安全,提供了一个统一平台,可以使用现有的Hadoop Kerberos
实现安全认证,同时,通过Hive
或Impala
访问数据时可以使用同样的Sentry
协议。本文会对Sentry
进行简单的介绍并演示Impala Sentry
整合后的实际效果。
Sentry介绍
Apache Sentry
是Cloudera
公司发布的一个用于权限控制的Hadoop
开源组件,已于2016年3月顺利从孵化器毕业,成为Apache
顶级项目。它基于RBAC
的授权模型实现了细粒度的权限控制,Sentry
目前可以与Apache Hive
、Hive Metastore
/HCatalog
、Apache Solr
、Impala
和HDFS
(仅限于Hive
表数据)整合实现权限控制。以下是Sentry
以及它与Hadoop
其他组件整合的一张概览图:
这张概览图的成员可以按身份分为两部分:
1、Sentry服务组件:
Sentry Server
: 服务提供层。它基于RPC
协议实现,主要负责管理权限数据,提供了安全的查询和保存元数据的RPC
接口Data Engine
:数据引擎层。它有两个职责:一是负责加载Sentry
插件,二是拦截所有访问资源的客户端(如Hive
或Impala
)请求,并转发到Sentry Plugin
中进行权限验证Sentry Plugin
:权限认证层。这是Sentry
授权的核心组件,负责判定从数据处理层获取的权限信息与服务提供层已保存的权限信息是否匹配Policy Metadata
:数据存储层。负责权限数据的存储,Sentry
支持使用ini
文件和关系型DB
来存储权限数据。当使用ini
文件时,这个文件可以存在于本地路径或者HDFS
中,基于文件的方式在使用程序修改过程中会存在资源竞争,不利于维护;当使用关系型DB
时,Sentry
将权限信息持久化到DB
中,并为应用层提供API
接口方便创建、查询、更新和删除这些数据。Sentry
可以使用很多后端的数据库,例如MySQL
、Postgres
等等,它使用ORM
库DataNucleus
来完成持久化操作。
2、Sentry使用者组件:
以Impala、Hive、Solr
为代表的各个组件组成了Sentry
使用者组件,在Sentry
中,这些组件都是以客户端的身份调用Sentry
服务的。
简单地讲,Sentry
是用一种类似C/S
架构的方式来向外提供服务,所有使用Sentry
的组件都可以被视为一个Sentry
客户端,使用RPC
协议来与Sentry Server
端交互。使用了Sentry
之后,这些客户端grant/revoke
管理的权限完全被Sentry
接管,grant/revoke
的执行也完全在Sentry
中实现。对于所有引擎的授权信息也存储在由Sentry
设定的统一的数据库中,这样所有引擎的权限就实现了集中管理。
Sentry
授权包括以下几种角色:
- 资源。可以是
Server、Database、Table
或者URL
(例如:HDFS
或者本地路径)。Sentry1.5
中支持对列进行授权 - 权限。授权访问某一个资源的规则
- 角色。角色是一系列权限的集合
- 用户和组。一个组是一系列用户的集合。
Sentry
的组映射是可以扩展的。默认情况下,Sentry
使用Hadoop
的组映射(可以是操作系统组或者LDAP
中的组)。Sentry
允许你将用户和组进行关联,你可以将一系列的用户放入到一个组中。Sentry
不能直接给一个用户或组授权,需要将权限授权给角色,角色可以授权给一个组而不是一个用户
安装Sentry Server
环境
Sentry
版本:1.5.1-cdh5.16.1
JDK
版本:jdk1.8.0_212
Maven
版本:apache-maven-3.6.1
Impala
版本:2.12.0-cdh5.16.1
Hadoop
版本:hadoop-2.6.0-cdh5.16.1
编译安装Sentry Server
接下来使用maven
编译生成Sentry
安装包文件。步骤如下:
1、下载源码:
代码语言:javascript复制git clone https://github.com/cloudera/sentry.git
切换到1.5.1-cdh5.16.1
这个tag
:
git checkout -b cdh5.16.1-release cdh5.16.2-release
源码结构:
2、编译打包:
代码语言:javascript复制mvn -Dmaven.test.skip=true clean package
编译打包完成后,生成的Sentry
安装包在下图所标识的目录下:
3、设置环境变量:
解压Sentry
压缩包到指定目录下,同时下载hadoop-2.6.0-cdh5.16.1.tar.gz并解压,编辑/etc/profile
,设置Hadoop
和Sentry
环境变量:
HADOOP_HOME=/data/sentry/hadoop-2.6.0-cdh5.16.1
HADOOP_LIBEXEC_DIR=${HADOOP_HOME}/libexec
SENTRY_HOME=/data/sentry/apache-sentry-1.5.1-cdh5.16.1-bin
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HADOOP_LIBEXEC_DIR:$SENTRY_HOME/bin:$PATH
4、配置sentry-site.xml
:
转到Sentry
解压目录的conf
文件夹下,修改sentry-site.xml
配置文件:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>sentry.service.server.rpc-address</name>
<value>hadoop21-test1-rgtj5-tj1</value>
</property>
<property>
<name>sentry.service.server.rpc-port</name>
<value>8038</value>
</property>
<property>
<name>sentry.service.admin.group</name>
<value>hadoop</value>
</property>
<property>
<name>sentry.service.allow.connect</name>
<value>hadoop</value>
</property>
<property>
<name>sentry.store.group.mapping</name>
<value>org.apache.sentry.provider.common.HadoopGroupMappingService</value>
</property>
<property>
<name>sentry.service.reporting</name>
<value>JMX</value>
</property>
<property>
<name>sentry.service.web.enable</name>
<value>true</value>
</property>
<property>
<name>sentry.service.web.port</name>
<value>51000</value>
</property>
<property>
<name>sentry.service.web.authentication.type</name>
<value>NONE</value>
</property>
<property>
<name>sentry.verify.schema.version</name>
<value>true</value>
</property>
<property>
<name>sentry.service.security.mode</name>
<value>none</value>
</property>
<property>
<name>sentry.store.jdbc.url</name>
<value>jdbc:mysql://localhost:3306/sentry_test?useSSL=false</value>
</property>
<property>
<name>sentry.store.jdbc.driver</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>sentry.store.jdbc.user</name>
<value>root</value>
</property>
<property>
<name>sentry.store.jdbc.password</name>
<value>123456</value>
</property>
</configuration>
5、创建MySQL
数据库表:
CREATE DATABASE `sentry_test` /*!40100 DEFAULT CHARACTER SET utf8 */;
6、初始化Sentry
数据库表:
将mysql-connector-java-5.1.47.jar
放到Sentry
解压目录的lib
文件夹下,然后执行以下命令创建Sentry
数据库表:
sentry --command schema-tool --conffile ${SENTRY_HOME}/conf/sentry-site.xml --dbType mysql --initSchema
显示以下信息表示连接到数据库并初始化数据库表成功:
代码语言:javascript复制Sentry store connection URL: jdbc:mysql://localhost:3306/sentry_test?useSSL=false
Sentry store Connection Driver : com.mysql.jdbc.Driver
Sentry store connection User: root
Starting sentry store schema initialization to 1.5.0-cdh5-2
Initialization script sentry-mysql-1.5.0-cdh5-2.sql
Connecting to jdbc:mysql://localhost:3306/sentry_test?useSSL=false
Connected to: MySQL (version 5.6.24-72.2-log)
Driver: MySQL Connector Java (version mysql-connector-java-5.1.47 ( Revision: fe1903b1ecb4a96a917f7ed3190d80c049b1de29 ))
Transaction isolation: TRANSACTION_REPEATABLE_READ
Autocommit status: true
No rows affected (0.006 seconds)
No rows affected (0.001 seconds)
No rows affected (0.002 seconds)
No rows affected (0.001 seconds)
No rows affected (0.001 seconds)
No rows affected (0.001 seconds)
No rows affected (0.004 seconds)
No rows affected (0.001 seconds)
No rows affected (0.001 seconds)
No rows affected (0.001 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.003 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.016 seconds)
No rows affected (0.007 seconds)
No rows affected (0.006 seconds)
No rows affected (0.007 seconds)
No rows affected (0.006 seconds)
No rows affected (0.005 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.005 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.006 seconds)
No rows affected (0.006 seconds)
No rows affected (0.003 seconds)
No rows affected (0.005 seconds)
No rows affected (0.002 seconds)
No rows affected (0.004 seconds)
1 row affected (0.002 seconds)
No rows affected (0.003 seconds)
No rows affected (0.007 seconds)
No rows affected (0.005 seconds)
No rows affected (0.004 seconds)
No rows affected (0.005 seconds)
No rows affected (0.005 seconds)
No rows affected (0.005 seconds)
No rows affected (0.006 seconds)
No rows affected (0.005 seconds)
No rows affected (0.002 seconds)
No rows affected (0.006 seconds)
No rows affected (0.002 seconds)
No rows affected (0.004 seconds)
No rows affected (0.003 seconds)
No rows affected (0.004 seconds)
No rows affected (0.005 seconds)
No rows affected (0.003 seconds)
No rows affected (0.006 seconds)
No rows affected (0.006 seconds)
No rows affected (0.006 seconds)
No rows affected (0.003 seconds)
No rows affected (0.003 seconds)
No rows affected (0.006 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.002 seconds)
No rows affected (0.006 seconds)
No rows affected (0.005 seconds)
No rows affected (0.004 seconds)
No rows affected (0.006 seconds)
No rows affected (0.002 seconds)
No rows affected (0.005 seconds)
No rows affected (0.005 seconds)
No rows affected (0.005 seconds)
Closing: 0: jdbc:mysql://localhost:3306/sentry_test?useSSL=false
Initialization script completed
Sentry schemaTool completed
7、运行Sentry
命令,启动Sentry
服务端:
nohup sentry --command service --conffile ${SENTRY_HOME}/conf/sentry-site.xml>sentry.out 2>&1 &
在浏览器输入以下地址访问Sentry Web UI
,验证是否安装成功:
http://localhost:51000/
Web UI
如下图所示:
Impala Sentry整合
1、引入Sentry依赖
将apache-sentry-1.5.1-cdh5.16.1-bin/lib
目录下相关jar
拷贝到/usr/lib/impala/lib
目录下,或者使用如下命令建立Sentry jar
包的软链接也行:
#!/bin/bash
SENTRY_HOME=/data/impala/apache-sentry-1.5.1-cdh5.16.1-bin
sudo rm -rf /usr/lib/impala/lib/sentry-*.jar
sudo ln -s $SENTRY_HOME/lib/sentry-binding-hive-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-binding-hive.jar
sudo ln -s $SENTRY_HOME/lib/sentry-core-common-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-common.jar
sudo ln -s $SENTRY_HOME/lib/sentry-core-model-db-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-model-db.jar
sudo ln -s $SENTRY_HOME/lib/sentry-core-model-kafka-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-model-kafka.jar
sudo ln -s $SENTRY_HOME/lib/sentry-core-model-search-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-model-search.jar
sudo ln -s $SENTRY_HOME/lib/sentry-policy-common-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-common.jar
sudo ln -s $SENTRY_HOME/lib/sentry-policy-db-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-db.jar
sudo ln -s $SENTRY_HOME/lib/sentry-policy-kafka-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-kafka.jar
sudo ln -s $SENTRY_HOME/lib/sentry-policy-search-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-search.jar
sudo ln -s $SENTRY_HOME/lib/sentry-provider-cache-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-cache.jar
sudo ln -s $SENTRY_HOME/lib/sentry-provider-common-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-common.jar
sudo ln -s $SENTRY_HOME/lib/sentry-provider-db-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-db-sh.jar
sudo ln -s $SENTRY_HOME/lib/sentry-provider-file-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-file.jar
最终的Sentry jar
包依赖如下:
lrwxrwxrwx 1 root root 90 Jul 6 11:00 sentry-binding-hive.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-binding-hive-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 89 Jul 6 11:00 sentry-core-common.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-common-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 91 Jul 6 11:00 sentry-core-model-db.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-model-db-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 94 Jul 6 11:00 sentry-core-model-kafka.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-model-kafka-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 95 Jul 6 11:00 sentry-core-model-search.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-model-search-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 91 Jul 6 11:00 sentry-policy-common.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-common-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 87 Jul 6 11:00 sentry-policy-db.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-db-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 90 Jul 6 11:00 sentry-policy-kafka.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-kafka-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 91 Jul 6 11:00 sentry-policy-search.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-search-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 92 Jul 6 11:00 sentry-provider-cache.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-cache-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 93 Jul 6 11:00 sentry-provider-common.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-common-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 89 Jul 6 11:00 sentry-provider-db-sh.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-db-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root 91 Jul 6 11:00 sentry-provider-file.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-file-1.5.1-cdh5.16.1.jar
注意:使用sentry
和Impala
的CDH
版本一定要对应上,比如我这里使用的是Impala
版本是CDH5.16.1
,那么sentry
的也需要是,否则会因为jar
版本问题导致Impala
启动过程中抛出一些异常,比如:
java.lang.NoClassDefFoundError: org/apache/sentry/provider/cache/SentryPrivilegeCache
如果不知道Impala
依赖的一些外部组件的版本,可以在Impala
源码的Impala/bin/impala-config.sh
里找到,如Impala cdh5-2.12.0_5.16.1版本的依赖信息在该配置文件里定义如下:
# Versions of Hadoop ecosystem dependencies.
# ------------------------------------------
export CDH_MAJOR_VERSION=5
export IMPALA_HADOOP_VERSION=2.6.0-cdh5.16.1
unset IMPALA_HADOOP_URL
export IMPALA_HBASE_VERSION=1.2.0-cdh5.16.1
unset IMPALA_HBASE_URL
export IMPALA_HIVE_VERSION=1.1.0-cdh5.16.1
unset IMPALA_HIVE_URL
export IMPALA_SENTRY_VERSION=1.5.1-cdh5.16.1
unset IMPALA_SENTRY_URL
export IMPALA_PARQUET_VERSION=1.5.0-cdh5.16.1
export IMPALA_LLAMA_MINIKDC_VERSION=1.0.0
unset IMPALA_LLAMA_MINIKDC_URL
export IMPALA_KITE_VERSION=1.0.0-cdh5.16.1
2、创建sentry-site.xml
将apache-sentry-1.5.1-cdh5.16.1-bin/conf
目录下的sentry-site.xml.service.template
文件拷贝到/etc/impala/conf
目录下:
# 拷贝
cp apache-sentry-1.5.1-cdh5.16.1-bin/conf/sentry-site.xml.service.template /etc/impala/conf/
# 重命名
cd /etc/impala/conf/
mv sentry-site.xml.service.template sentry-site.xml
编辑sentry-site.xml
为以下内容:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- WARNING!!! This file is provided for documentation purposes ONLY! -->
<!-- WARNING!!! You should copy to sentry-site.xml and make modification instead. -->
<configuration>
<!--Sentry Server端口-->
<property>
<name>sentry.service.client.server.rpc-port</name>
<value>8038</value>
</property>
<!--Sentry Server服务器地址-->
<property>
<name>sentry.service.client.server.rpc-addresses</name>
<value>hadoop21-test1-rgtj5-tj1</value>
</property>
<!--客户端连接Sentry Server超时时间,以毫秒为单位,默认为200000毫秒-->
<property>
<name>sentry.service.client.server.rpc-connection-timeout</name>
<value>200000</value>
</property>
<!--权限存储方式:数据库或者ini配置文件-->
<property>
<name>sentry.hive.provider.backend</name>
<value>org.apache.sentry.provider.db.SimpleDBProviderBackend</value>
</property>
<!--权限认证方式,支持Kerberos认证,设置为none表示不启用认证 -->
<property>
<name>sentry.service.security.mode</name>
<value>none</value>
</property>
</configuration>
3、启用权限认证
编辑/etc/default/impala
配置文件,修改如下两个配置启用Sentry
权限认证:
- 修改
IMPALA_CATALOG_ARGS
选项,增加-sentry_config=/etc/impala/conf/sentry-site.xml
配置 - 修改
IMPALA_SERVER_ARGS
选项,增加-sentry_config=/etc/impala/conf/sentry-site.xml
和-server_name=sentryserver
配置
配置文件最终内容如下:
代码语言:javascript复制IMPALA_CATALOG_SERVICE_HOST=hadoop21-test1-rgtj5-tj1
IMPALA_STATE_STORE_HOST=hadoop21-test1-rgtj5-tj1
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/data/log/impala
IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} -sentry_config=/etc/impala/conf/sentry-site.xml"
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}"
IMPALA_SERVER_ARGS="
-log_dir=${IMPALA_LOG_DIR}
-catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST}
-state_store_port=${IMPALA_STATE_STORE_PORT}
-use_statestore
-state_store_host=${IMPALA_STATE_STORE_HOST}
-be_port=${IMPALA_BACKEND_PORT}
-kudu_master_hosts=hadoop21-test1-rgtj5-tj1:7051,hadoop20-test1-rgtj5-tj1:7051,hadoop22-test1-rgtj5-tj1:7051,hadoop-bi06-test1-rgtj5-tj1:7051,hadoop-bi07-test1-rgtj5-tj1:7051
-sentry_config=/etc/impala/conf/sentry-site.xml
-server_name=sentryserver"
ENABLE_CORE_DUMPS=true
# LIBHDFS_OPTS=-Djava.library.path=/usr/lib/impala/lib
# MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
# IMPALA_BIN=/usr/lib/impala/sbin
# IMPALA_HOME=/usr/lib/impala
# HIVE_HOME=/usr/lib/hive
# HBASE_HOME=/usr/lib/hbase
# IMPALA_CONF_DIR=/etc/impala/conf
# HADOOP_CONF_DIR=/etc/impala/conf
# HIVE_CONF_DIR=/etc/impala/conf
# HBASE_CONF_DIR=/etc/impala/conf
4、重启Impala服务,验证权限
重启Impala
服务:
sudo service impala-state-store restart
sudo service impala-catalog restart
sudo service impala-server restart
打开impala-shell
,验证权限配置是否成功,具体操作如下:
(1)切换到hadoop
用户,打开impala-shell
,创建一个admin
角色:
[hadoop21-test1-rgtj5-tj1:21000] > create role admin_role;
Query: create role admin_role
Fetched 0 row(s) in 0.35s
(2)为admin
角色赋予超级权限:
[hadoop21-test1-rgtj5-tj1:21000] > GRANT ALL ON SERVER sentryserver TO ROLE admin_role;
Query: GRANT ALL ON SERVER sentryserver TO ROLE admin_role
Query submitted at: 2019-07-06 10:40:11 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=15475b39691bd167:66c1403300000000
Fetched 0 row(s) in 0.13s
(3)将admin
角色授权给hadoop
用户组:
[hadoop21-test1-rgtj5-tj1:21000] > GRANT ROLE admin_role TO GROUP hadoop;
Query: GRANT ROLE admin_role TO GROUP hadoop
Query submitted at: 2019-07-06 10:41:53 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=434bb908587eaf31:65887a5a00000000
Fetched 0 row(s) in 0.11s
(4)创建一个test
库和test
表,并插入测试数据:
[hadoop21-test1-rgtj5-tj1:21000] > create database test;
Query: create database test
Fetched 0 row(s) in 0.29s
[hadoop21-test1-rgtj5-tj1:21000] > use test;
Query: use test
[hadoop21-test1-rgtj5-tj1:21000] > CREATE TABLE test(x INT, y STRING) STORED AS PARQUET;
Query: CREATE TABLE test(x INT, y STRING) STORED AS PARQUET
Fetched 0 row(s) in 0.16s
[hadoop21-test1-rgtj5-tj1:21000] > INSERT INTO test VALUES (1, 'one'), (2, 'two'), (3, 'three');
Query: INSERT INTO test VALUES (1, 'one'), (2, 'two'), (3, 'three')
Query submitted at: 2019-07-06 11:18:33 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=ce4e7f66f1209531:641f39a900000000
Modified 3 row(s) in 5.47s
因为hadoop
用户是超级管理员并拥有ALL
的权限,因此执行以下SELECT
语句便能很快看到我们刚插入的数据:
[hadoop21-test1-rgtj5-tj1:21000] > select * from test;
Query: select * from test
Query submitted at: 2019-07-06 11:19:50 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=34e4b5594e3d0c6:8cfb1acb00000000
--- -------
| x | y |
--- -------
| 1 | one |
| 2 | two |
| 3 | three |
--- -------
Fetched 3 row(s) in 1.87s
接着我们切换到root
用户,运行impala-shell
,对我们刚刚创建的test
库进行操作:
[hadoop21-test1-rgtj5-tj1:21000] > use test;
Query: use test
ERROR: AuthorizationException: User 'root' does not have privileges to access: test.*.*
提示root
用户没有操作test
库的权限,至此,说明Sentry
权限认证已经生效。
各种授权操作语法如下:
代码语言:javascript复制创建角色:CREATE ROLE <role name>
组分配角色:GRANT ROLE <role name> TO GROUP <group name>
服务级赋权:GRANT <ALL|SELECT|UPDATE> ON SERVER <server name> TO ROLE <role name>
数据库赋权:GRANT <ALL|SELECT|UPDATE> ON DATABASE <database name> TO ROLE <role name>
表赋权:GRANT <ALL|SELECT|UPDATE> ON TABLE <database name>.<table name> TO ROLE <role name>
字段权限:GRANT SELECT(column name)ON TABLE <table name> TO ROLE <role name>;
回收组权限:REVOKE ROLE <role name> FROM GROUP <group name>
回收字段权限:REVOKE SELECT <column name> ON TABLE <table name> FROM ROLE <role name>;
回收数据库权限:REVOKE <ALL|SELECT|UPDATE> ON DATABASE <database name> FROM ROLE <role name>
查看某个角色的权限:SHOW GRANT ROLE <role name>
各种查看命令:
SHOW ROLES;
SHOW CURRENT ROLES;
SHOW ROLE GRANT GROUP <group name>;
SHOW GRANT ROLE <role name>;
SHOW GRANT ROLE <role name> on OBJECT <object name>;
总结
1、Impala
服务的权限安全,认证(Kerberos/LDAP
)是第一步,授权(Sentry
)是第二步。如果要启用授权,必须先启用认证。本文在测试过程中不启用认证而只启用Sentry
授权,强烈不建议在生产系统中这样使用,因为如果没有用户认证,授权没有任何意义形同虚设,用户可以随意使用任何超级用户登录Impala
,并不会做密码校验。
2、Impala
是不区分底层存储用户的,Sentry
控制的只是Impala
应用层的操作权限,底层操作HDFS
的还是impala
用户,也就是启动impalad
的用户。不区分底层存储用户主要是因为C
的libhdfs
在Hadoop2
时还不支持doAs
。
3、Impala
中的授权处理过程类似于Hive
中的授权处理过程,主要的区别在于权限信息的缓存。Impala
的Catalog
服务管理并缓存数据库schema
元数据和Sentry
权限元数据,并将其传播到所有Impala Server
节点。因此,Impala
中的授权验证在本地进行,而且速度更快。可以用下图进行概括:
参考资料
官方资料:
- Sentry官方文档
- Sentry Tutorial
博客文章:
- Apache sentry架构分析-(与hive、hdfs集成)
- Apache Sentry架构介绍
- 基于Sentry实现数据访问权限控制
- Hive记录-Impala jdbc连接hive和kudu参考
- Impala权限控制
Hadoop实操公众号:
- 如何在CDH未启用认证的情况下安装及使用Sentry
- 0648-6.2.0-配置Senty服务
- 如何使用Sentry管理Hive外部表权限
- 如何使用Sentry管理Hive外部表(补充)
- 如何使用java代码通过JDBC访问Sentry环境下的Hive
- 使用Java代码通过JDBC连接只启用Sentry的Impala异常分析