Author: kwu
【解决】hive动态增加partitions不能超过100的问题,全量动态生成partitions超过100会出现如下异常:
The maximum number of dynamic partitions is controlled by hive.exec.max.dynamic.partitions and hive.exec.max.dynamic.partitions.pernode. Maximum was set to: 100解决100限制,可设置如下参数:set hive.exec.dynamic.partition.mode=nonstrict;SET hive.exec.max.dynamic.partitions=100000; SET hive.exec.max.dynamic.partitions.pernode=100000; insert overwrite table ods.cms_entity PARTITION (DAY)select ENTITY_ID ,ENTITY_NAME ,ENTITY_DESC ,ENTITY_TYPE ,ENTITY_PID ,ENTITY_TIME ,ENTITY_PRIORITY ,ENTITY_STATUS ,ENTITY_CHANNEL ,ENTITY_EDITOR ,ENTITY_TEMPLATE ,ENTITY_URL ,ENTITY_CATEGORY ,ENTITY_PARAM ,ENTITY_SHORTNAME ,ENTITY_SUBTYPE ,ENTITY_COMPDELAY ,dayfrom stage.cms_entity_by_day;
设置动态partitions为100000,该语句插入几年按天的partitions超过1000个,,成功执行语句。
版权声明:本文为博主原创文章,未经博主允许不得转载。
转动心中的期待,血在澎湃,吃苦流汗算什么。