Member since
09-10-2016
14
Posts
6
Kudos Received
0
Solutions
01-22-2016
12:01 PM
Hi, Just tried that: [root@sandbox ~]# su - hdfs
[hdfs@sandbox ~]$ hdfs dfs -chown -R root:hdfs /user/root
but I get: chown: `/user/root': No such file or directory
... View more
01-21-2016
01:57 AM
1 Kudo
Hi Artem, I have tried with another directory and I get the following error... sqoop import \ --connect "jdbc:mysql://127.0.0.1:3306/test" \ --username hue --password 1111 \ --table testtbl \ --target-dir /user/guest/mysqlimport2 \ --driver com.mysql.jdbc.Driver Warning: /usr/hdp/2.3.2.0-2950/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/01/21 01:52:31 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.3.2.0-2950
16/01/21 01:52:31 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consi
der using -P instead.
16/01/21 01:52:31 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appro
priate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to o
rg.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should b
e used next time.
16/01/21 01:52:31 INFO manager.SqlManager: Using default fetchSize of 1000
16/01/21 01:52:31 INFO tool.CodeGenTool: Beginning code generation
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/
impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4
j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/01/21 01:52:32 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM testtbl AS t WHERE
1=0
16/01/21 01:52:32 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM testtbl AS t WHERE
1=0
16/01/21 01:52:32 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.3.2.0-2950/hadoop-mapre
duce
Note: /tmp/sqoop-root/compile/89e6906e14bcf45371dcde0a398899e1/testtbl.java uses or overrides a depreca
ted API.
Note: Recompile with -Xlint:deprecation for details.
16/01/21 01:52:33 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/89e6906e14bcf4
5371dcde0a398899e1/testtbl.jar
16/01/21 01:52:33 INFO mapreduce.ImportJobBase: Beginning import of testtbl
16/01/21 01:52:34 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM testtbl AS t WHERE
1=0
16/01/21 01:52:35 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.co
m:8188/ws/v1/timeline/
16/01/21 01:52:35 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/10.0.2.
15:8050
16/01/21 01:52:35 ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.
security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user/root/.staging
":hdfs:hdfs:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:31
9)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:29
2)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionCheck
er.java:213)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.chec
kPermission(RangerHdfsAuthorizer.java:300)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionCheck
er.java:190)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738
)
at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3896)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:984)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(Client
NamenodeProtocolServerSideTranslatorPB.java:622)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.
callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.jav
a:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2137)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2133)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2131)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.
java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:3010)
at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2978)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:1047)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:1043)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1043)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1036)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:144)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:196)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:169)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Pe
rmission denied: user=root, access=WRITE, inode="/user/root/.staging":hdfs:hdfs:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:31
9)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:29
2)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionCheck
er.java:213)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.chec
kPermission(RangerHdfsAuthorizer.java:300)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionCheck
er.java:190)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738
)
at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3896)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:984)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(Client
NamenodeProtocolServerSideTranslatorPB.java:622)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.
callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.jav
a:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2137)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2133)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2131)
at org.apache.hadoop.ipc.Client.call(Client.java:1427)
at org.apache.hadoop.ipc.Client.call(Client.java:1358)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy9.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodePr
otocolTranslatorPB.java:558)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:1
87)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy10.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:3008)
... 27 more
... View more
01-21-2016
12:36 AM
1 Kudo
Hi, Thank you Neeraj and jkotireddy. I have tried the sqoop import statement with the driver line but still I get an error (pasted below). I will update the mysql driver and try again - but is there any reason why it still doesn't work with the driver line included? sqoop import \ --connect "jdbc:mysql://127.0.0.1:3306/test" \ --username hue --password 1111 \ --table testtbl \ --target-dir /user/guest/mysqlimport \ --driver com.mysql.jdbc.Driver Warning: /usr/hdp/2.3.2.0-2950/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/01/21 00:29:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.3.2.0-2950
16/01/21 00:29:12 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consi
der using -P instead.
16/01/21 00:29:12 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appro
priate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to o
rg.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should b
e used next time.
16/01/21 00:29:12 INFO manager.SqlManager: Using default fetchSize of 1000
16/01/21 00:29:12 INFO tool.CodeGenTool: Beginning code generation
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/
impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4
j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/01/21 00:29:13 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM testtbl AS t WHERE
1=0
16/01/21 00:29:13 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM testtbl AS t WHERE
1=0
16/01/21 00:29:13 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.3.2.0-2950/hadoop-mapre
duce
Note: /tmp/sqoop-root/compile/5e5baec496fc20389f12c27fbc094cd5/testtbl.java uses or overrides a depreca
ted API.
Note: Recompile with -Xlint:deprecation for details.
16/01/21 00:29:15 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/5e5baec496fc20
389f12c27fbc094cd5/testtbl.jar
16/01/21 00:29:15 INFO mapreduce.ImportJobBase: Beginning import of testtbl
16/01/21 00:29:15 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM testtbl AS t WHERE
1=0
16/01/21 00:29:16 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.co
m:8188/ws/v1/timeline/
16/01/21 00:29:16 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/10.0.2.
15:8050
16/01/21 00:29:17 ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.
mapred.FileAlreadyExistsException: Output directory hdfs://sandbox.hortonworks.com:8020/user/guest/mysq
limport already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.ja
va:146)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:266)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:196)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:169)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
... View more
01-20-2016
05:39 PM
2 Kudos
Hi, Hoping someone can advise. I am playing around with Sqoop. I can list the databases using: sqoop list-databases --connect jdbc:mysql://127.0.0.1:3306 --username hue --password 1111 And I can list the tables: sqoop list-tables --connect "jdbc:mysql://127.0.0.1:3306/test" --username hue --password 1111 However, when I try an import, I get an error: sqoop import \ --connect "jdbc:mysql://127.0.0.1:3306/test" \ --username hue --password 1111 \ --table testtbl \ --target-dir /user/guest/mysqlimport The error is below. I am not sure why this code is causing an error. Does anyone have any ideas? Regards Rev Warning: /usr/hdp/2.3.2.0-2950/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/01/20 17:38:31 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.3.2.0-2950
16/01/20 17:38:31 WARN tool.BaseSqoopTool: Setting your password on the command-line is
insecure. Consider using -P instead.
16/01/20 17:38:31 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultse
t.
16/01/20 17:38:31 INFO tool.CodeGenTool: Beginning code generation
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/hadoop/lib/slf4j-log4j12-1.7.10.
jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/zookeeper/lib/slf4j-log4j12-1.6.
1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/01/20 17:38:31 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tes
ttbl` AS t LIMIT 1
16/01/20 17:38:31 ERROR manager.SqlManager: Error reading from database: java.sql.SQLExc
eption: Streaming result set com.mysql.jdbc.RowDataDynamic@4b1aa70c is still active. No
statements may be issued when any streaming result sets are open and in use on a given c
onnection. Ensure that you have called .close() on any active streaming result sets befo
re attempting more queries.
java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@4b1aa70c is st
ill active. No statements may be issued when any streaming result sets are open and in u
se on a given connection. Ensure that you have called .close() on any active streaming r
esult sets before attempting more queries.
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931)
at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569)
at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524)
at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003)
at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602)
at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:
286)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java
:241)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1845)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
16/01/20 17:38:31 ERROR tool.ImportTool: Encountered IOException running import job: jav
a.io.IOException: No columns to generate for ClassWriter
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
... View more
Labels:
- Labels:
-
Apache Sqoop
01-20-2016
05:35 PM
Thanks Shivaji, I managed to get it working...
... View more
01-14-2016
12:31 AM
Hi, Hoping someone can help. I am trying to insert data from a csv file stored on the sandbox (within /root) to a mysql database table. Below I have bullet pointed the steps I have taken and the error I am getting. - I have a file called countrycodes.csv stored in the /root folder of the sandbox. This contains the data that I want to insert into the database. - I have created a mysql database called forHadoop. - I have granted privileges to user hue by: Grant all privileges on forHadoop.* to hue@localhost identified by '1111'; - I can log into this database as hue by typing: mysql -u hue -p Then type the password 1111 and I am in. - I have created a table called countrycodes within the database forHadoop. - I want to populate the table countrycodes with the data from /root/countrycodes.csv.
To do this I type: mysql forHadoop --local_infile=1 -u hue -p
Then type my password 1111 - But when I type the below load statement I get an error message. LOAD DATA LOCAL INFILE ‘/root/countrycodes.csv’ into table forHadoop.countrycodes fields terminated by ‘,’ LINES TERMINATED BY ‘\r\n’; - The error message is:
ERROR 1044 (42000): Access denied for user 'hue'@'localhost' to database 'n’;'
Any idea as to what I am doing wrong? Why is access denied fro user hue?
... View more
Labels:
- Labels:
-
Cloudera Hue
01-13-2016
08:49 PM
Hi, Hoping someone can advise. I am running a few lines of Pig code where I get an error. When I run the four lines of code below in Pig it works fine: allairports = LOAD 'default.airport' USING org.apache.hive.hcatalog.pig.HCatLoader(); gbairports = filter allairports by iso_country == 'GB'; gbcols = foreach gbairports generate $13, $3, $6; dump gbcols;
However, when I replace the last line (as below) with a store statement, I get an error. allairports = LOAD 'default.airport' USING org.apache.hive.hcatalog.pig.HCatLoader(); gbairports = filter allairports by iso_country == 'GB'; gbcols = foreach gbairports generate $13, $3, $6; store gbcols into '/user/hue/tutorials' using pigstorage(','); The error is below. Does anyone know why I am getting this? WARNING: Use "yarn jar" to launch YARN applications.
16/01/13 20:33:06 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/01/13 20:33:06 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
16/01/13 20:33:06 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2016-01-13 20:33:06,502 [main] INFO org.apache.pig.Main - Apache Pig version 0.15.0.2.3.2.0-2950 (rexported) compiled Sep 30 2015, 19:39:20
2016-01-13 20:33:06,502 [main] INFO org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/hue/appcache/application_1452469761257_0020/container_e03_1452469761257_0020_01_000002/pig_1452717186500.log
2016-01-13 20:33:07,183 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2016-01-13 20:33:07,294 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020
2016-01-13 20:33:08,734 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-01-13 20:33:08,770 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-01-13 20:33:08,823 [main] INFO hive.metastore - Connected to metastore.
2016-01-13 20:33:09,127 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse:
<file script.pig, line 9, column 46> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1735)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1443)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:387)
at org.apache.pig.PigServer.executeBatch(PigServer.java:412)
at org.apache.pig.PigServer.executeBatch(PigServer.java:398)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:502)
at org.apache.pig.Main.main(Main.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by:
<file script.pig, line 9, column 46> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7782)
at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1669)
at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
... 17 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:677)
at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1336)
... 25 more
2016-01-13 20:33:09,131 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /hadoop/yarn/local/usercache/hue/appcache/application_1452469761257_0020/container_e03_1452469761257_0020_01_000002/pig_1452717186500.log 2016-01-13 20:33:09,153 [main] INFO org.apache.pig.Main - Pig script completed in 2 seconds and 779 milliseconds (2779 ms)
... View more
Labels:
- Labels:
-
Apache Pig
01-11-2016
07:48 PM
Hi Artem, sure I understand. If I take the example database xademo (which I can see when viewing from Hive in Ambari) - how can I use sqoop to list this database?
... View more