Driver类是对
org.apache.hadoop.hive.ql.processors.CommandProcessor.java
接口的实现,重写了run方法,定义了常见sql的执行方式.
publicclassDriverimplementsCommandProcessor
具体的方法调用顺序:
run—>runInternal—>(createTxnManager+recordValidTxns)—–>compileInternal—>compile–analyzer(BaseSemanticAnalyzer)—>execute
其中compile和execute是两个比较重要的方法:
compile用来完成语法和语义的分析,,生成执行计划
execute执行物理计划,即提交相应的mapredjob
通过打印perflog可以看到Driver类的简单地时序图:
下面来看下Driver类的几个常用的方法实现:
1)createTxnManager 用来获取目前设置的用于实现lock的类,比如:
org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager
2)checkConcurrency 用来判断当前hive设置是否支持并发控制:
booleansupportConcurrency=conf.getBoolVar(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY);
主要是通过判断hive.support.concurrency参数,默认是false
3)getClusterStatus 调用JobClient类的getClusterStatus方法来获取集群的状态:
publicClusterStatusgetClusterStatus()throwsException{ClusterStatuscs;try{JobConfjob=newJobConf(conf,ExecDriver.class);JobClientjc=newJobClient(job);cs=jc.getClusterStatus();}catch(Exceptione){e.printStackTrace();throwe;}LOG.info(“Returningclusterstatus:”+cs.toString());returncs;}
4)getSchema //返回表的schema信息
5)
doAuthorization/doAuthorizationV2/getHivePrivObjects
用来在开启权限验证情况下对sql的权限检测操作
6)
getLockObjects/acquireReadWriteLocks/releaseLocks
都是和锁相关的方法 ,其中getLockObjects用来获取锁的对象(锁的路径,锁的模式等),最终返回一个包含所有锁的list,acquireReadWriteLocks用来控制获取锁,releaseLocks用来释放锁:
getLockObjects:privateList<HiveLockObj>getLockObjects(Databased,Tablet,Partitionp,HiveLockModemode)throwsSemanticException{List<HiveLockObj>locks=newLinkedList<HiveLockObj>();HiveLockObjectDatalockData=newHiveLockObjectData(plan.getQueryId(),String.valueOf(System.currentTimeMillis()),”IMPLICIT”,plan.getQueryStr());if(d!=null){locks.add(newHiveLockObj(newHiveLockObject(d.getName(),lockData),mode));//数据库层面的锁returnlocks;}if(t!=null){//表层面的锁locks.add(newHiveLockObj(newHiveLockObject(t.getDbName(),lockData),mode));locks.add(newHiveLockObj(newHiveLockObject(t,lockData),mode));mode=HiveLockMode.SHARED;locks.add(newHiveLockObj(newHiveLockObject(t.getDbName(),lockData),mode));returnlocks;}if(p!=null){//分区层面的锁locks.add(newHiveLockObj(newHiveLockObject(p.getTable().getDbName(),lockData),mode));if(!(pinstanceofDummyPartition)){locks.add(newHiveLockObj(newHiveLockObject(p,lockData),mode));}//Alltheparentsarelockedinsharedmodemode=HiveLockMode.SHARED;//Fordummypartitions,onlypartitionnameisneededStringname=p.getName();if(pinstanceofDummyPartition){name=p.getName().split(“@”)[2];}StringpartialName=””;String[]partns=name.split(“/”);intlen=pinstanceofDummyPartition?partns.length:partns.length-1;Map<String,String>partialSpec=newLinkedHashMap<String,String>();for(intidx=0;idx<len;idx++){Stringpartn=partns[idx];partialName+=partn;String[]nameValue=partn.split(“=”);assert(nameValue.length==2);partialSpec.put(nameValue[0],nameValue[1]);try{locks.add(newHiveLockObj(newHiveLockObject(newDummyPartition(p.getTable(),p.getTable().getDbName()+”/”+p.getTable().getTableName()+”/”+partialName,partialSpec),lockData),mode));partialName+=”/”;}catch(HiveExceptione){thrownewSemanticException(e.getMessage());}}locks.add(newHiveLockObj(newHiveLockObject(p.getTable(),lockData),mode));locks.add(newHiveLockObj(newHiveLockObject(p.getTable().getDbName(),lockData),mode));}returnlocks;}
acquireReadWriteLocks调用了锁具体实现类的acquireLocks方法
releaseLocks调用了锁具体实现类的releaseLocks方法
7)
放下一处烦恼,收获一个惊喜;放下一种偏见,收获一种幸福;