hive动态分区相关

2022-11-13 12:59:29 浏览数 (1)

1.动态分区操作 –动态分区插数 set hive.exec.dynamic.partition=true; set hive.exec.dynamic.partition.mode=nonstrict; set hive.exec.max.dynamic.partitions=100000; set hive.exec.max.dynamic.partitions.pernode=100000;

INSERT overwrite table 目标表名 partition(分区字段) select 表字段,分区字段 from 源表;

2.但是如果遇到想批量插入某些分区的数据的情况,可不可以限制分区范围动态分区插入数据呢?下面来测试一下。

1)建两张测试表。 create table db_test.test_dtfq (a string comment ‘’,b bigint comment ‘’)partitioned by (ds string) STORED AS orc; create table db_test.test_dtfq_bak (a string comment ‘’,b bigint comment ‘’)partitioned by (ds string) STORED AS orc; 2)造几条测试数据。 insert into db_test.test_dtfq partition(ds=‘2020-07-21’) values(‘1’,1); insert into db_test.test_dtfq_bak partition(ds=‘2020-06-01’) values(‘4’,4); insert into db_test.test_dtfq_bak partition(ds=‘2020-06-21’) values(‘2’,2); insert into db_test.test_dtfq_bak partition(ds=‘2020-06-31’) values(‘3’,3); 3)动态分区插入数据。 set hive.exec.dynamic.partition=true; set hive.exec.dynamic.partition.mode=nonstrict; set hive.exec.max.dynamic.partitions=100000; set hive.exec.max.dynamic.partitions.pernode=100000;

INSERT overwrite table test_dtfq partition(ds) select a ,b , ds from db_test.test_dtfq_bak where ds<=‘2020-06-21’ ; 4)执行查看结果。 select * from db_test.test_dtfq;

结果显而易见,可以限制某些分区动态插入数据。噢耶!!!

0 人点赞