Posts: 40
Registered: ‎02-01-2017

Spark t bit error when trying to overwrite table using .parquet

I receive this error when I try to overwrite a table in apache spark.


Machine generated alternative text:
User class threw exceptiom org.apache hadoop securityAccessControIExceptiom Permission denied by sticky bit: user—...expl.db"
at org.apache nadoop fidts server. namenode DefaultAuthorizationProvider c heckStickyBit(DefaultAuthorizationProvider java:3B7)
at org.apache nadoop fidts server. name node. DefaultAuthcrizationProvider c heckPermission( 159)
at org.apache nadoop fidts server. name node F S PermissionChecker c heckPermissicn(F S 152)
at org.apache hadoop hdts.server_namenode_FSDirectory_checkPermission(FSDirectory java:352g)
at org.apache hadoop hdts.server_namenode_FSNamesystem chec kPermission(FSNamesystem java:BE50)
at org.apache hadoop hdts.server_namenode_FSNamesystem
at org.apache hadoop hdts.server_namenode_FSNamesystem deletelnt(
at org.apache hadoop hdts.server_namenode_FSNamesystem delete(
at org.apache hadoop
at org.apache hadoop hdts.server_namenode_AuthorizationProviderProxyCIientProtocoLdelete(AuthorizationProviderProxyClientProtocoLjava:30B)
at org.apache hadoop hdts.protocoIPB.ClientNamenodeProtocolServerSideTransIatorPB_delete(
at org.apache hadoop hdts.protocoI.proto.ClientNamenodeProtocoIProtosSClientNamenodeProtocolS2 callBIockingMethod(
at org.apache hadoop ipc_ProtobufRpcEngineSServerSProtoaufRpclnvoker.calI(Protoö
at org.apache hadoop 1073)
at org.apache hadoop
at org.apache hadoop
at doPrivileged(Native Method)
at javax_security auth
at org.apache hadoop security_userGroupIntormation
at org.apache hadoop 2214)



I am using spark 1.6 and cdh 5.7.2 

I try to overwrite using a hiveContext and persisting the df


hiveContext.sql(s"msck repair table $capiTbl")