Sqoop exception: ERROR sqoop.Sqoop: Got exception running Sqoop: java.awt.HeadlessException: No X11 DISPLAY variable was set, but this program performed an operation which requires it.

[#1]

I am ingesting data into Hive using Sqoop. I'm connecting to DB2 for iSeries using the jt400.jar jdbc driver.

Here is the log of the error I'm getting. I appreciate help on how to solve it.

Warning: /usr/hdp/2.4.2.0-258/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
/usr/hdp/2.4.2.0-258/hbase/conf/hbase-env.sh: line 15: export: ` -Djava.net.preferIPv4Stack=true -XX:+PrintGCTimeStamps -verbose:gc -XX:CMSInitiatingOccupancyFraction=70 -XX:+PerfDisableSharedMem -XX:+PrintGCDetails -Xloggc:/var/log/hbase/gc.log-201803161634 -XX:+UseConcMarkSweepGC -XX:ParallelGCThreads=8 -XX:+PrintGCDateStamps -XX:+UseCMSInitiatingOccupancyOnly': not a valid identifier
18/03/16 16:34:53 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.0-258
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-258/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.4.2.0-258/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/03/16 16:34:55 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
18/03/16 16:34:55 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
18/03/16 16:34:56 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
18/03/16 16:34:56 INFO tool.CodeGenTool: Beginning code generation
18/03/16 16:34:57 ERROR sqoop.Sqoop: Got exception running Sqoop: java.awt.HeadlessException: 
No X11 DISPLAY variable was set, but this program performed an operation which requires it.
java.awt.HeadlessException: 
No X11 DISPLAY variable was set, but this program performed an operation which requires it.
	at java.awt.GraphicsEnvironment.checkHeadless(GraphicsEnvironment.java:207)
	at java.awt.Window.<init>(Window.java:535)
	at java.awt.Frame.<init>(Frame.java:420)
	at java.awt.Frame.<init>(Frame.java:385)
	at com.ibm.as400.access.ToolboxSignonHandler.displayMessage(ToolboxSignonHandler.java:336)
	at com.ibm.as400.access.ToolboxSignonHandler.displayMessage(ToolboxSignonHandler.java:323)
	at com.ibm.as400.access.ToolboxSignonHandler.userIdUnknown(ToolboxSignonHandler.java:302)
	at com.ibm.as400.access.AS400.promptSignon(AS400.java:2997)
	at com.ibm.as400.access.AS400.signon(AS400.java:4252)
	at com.ibm.as400.access.AS400.connectService(AS400.java:1342)
	at com.ibm.as400.access.AS400JDBCConnection.setProperties(AS400JDBCConnection.java:3383)
	at com.ibm.as400.access.AS400JDBCDriver.prepareConnection(AS400JDBCDriver.java:1407)
	at com.ibm.as400.access.AS400JDBCDriver.initializeConnection(AS400JDBCDriver.java:1244)
	at com.ibm.as400.access.AS400JDBCDriver.connect(AS400JDBCDriver.java:395)
	at java.sql.DriverManager.getConnection(DriverManager.java:571)
	at java.sql.DriverManager.getConnection(DriverManager.java:215)
	at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:885)
	at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
	at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:744)
	at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
	at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
	at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
	at org.apache.sqoop.manager.SqlManager.getColumnTypesForQuery(SqlManager.java:234)
	at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:304)
	at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1845)
	at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
	at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:244)

[#2]

@Carol Elliott

May be you can try using this option to see if this works:

f you're on the main display, then export DISPLAY=:0.0 or if you're using csh or tcsh setenv DISPLAY :0.0

[#3]

@mbalakrishnan thanks for posting a suggestion.

I added a line to my script like this:

export DISPLAY=$HOSTNAME:0.0

Now I am getting this result:

Exception in thread "main" java.lang.InternalError: Can't connect to X11 window server using 'myservername:0.0' as the value of the DISPLAY variable.

at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)

at sun.awt.X11GraphicsEnvironment.access$200(X11GraphicsEnvironment.java:65)

at sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:110)

at java.security.AccessController.doPrivileged(Native Method)

at sun.awt.X11GraphicsEnvironment.<clinit>(X11GraphicsEnvironment.java:74)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:195)

at java.awt.GraphicsEnvironment.createGE(GraphicsEnvironment.java:102)

at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:81)

at java.awt.Window.initGC(Window.java:467)

at java.awt.Window.init(Window.java:487)

at java.awt.Window.<init>(Window.java:536)

at java.awt.Frame.<init>(Frame.java:420)

at java.awt.Frame.<init>(Frame.java:385)

at com.ibm.as400.access.ToolboxSignonHandler.displayMessage(ToolboxSignonHandler.java:336)

at com.ibm.as400.access.ToolboxSignonHandler.displayMessage(ToolboxSignonHandler.java:323)

at com.ibm.as400.access.ToolboxSignonHandler.userIdUnknown(ToolboxSignonHandler.java:302)

at com.ibm.as400.access.AS400.promptSignon(AS400.java:2997)

at com.ibm.as400.access.AS400.signon(AS400.java:4252)

at com.ibm.as400.access.AS400.connectService(AS400.java:1342)

at com.ibm.as400.access.AS400JDBCConnection.setProperties(AS400JDBCConnection.java:3383)

at com.ibm.as400.access.AS400JDBCDriver.prepareConnection(AS400JDBCDriver.java:1407)

at com.ibm.as400.access.AS400JDBCDriver.initializeConnection(AS400JDBCDriver.java:1244)

at com.ibm.as400.access.AS400JDBCDriver.connect(AS400JDBCDriver.java:395)

at java.sql.DriverManager.getConnection(DriverManager.java:571)

at java.sql.DriverManager.getConnection(DriverManager.java:215)

at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:885)

at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)

at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:744)

at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)

at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)

at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)

at org.apache.sqoop.manager.SqlManager.getColumnTypesForQuery(SqlManager.java:234)

at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:304)

at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1845)

at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)

at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)

at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)

at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)

at org.apache.sqoop.Sqoop.run(Sqoop.java:148)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)

at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)

at org.apache.sqoop.Sqoop.main(Sqoop.java:244)

[#4]

@Carol Elliott,

I think the issue is related to the jt400.jar which is launching java awt windows to lauch a pop-up for login. I don't this this would work for you in this case.

As you are connecting to DB2, I would suggest to use DB2 JDBC driver for ISeries, specifically these two jars: Type 4 JDBC driver: db2jcc4.jar and a license file: db2jcc_license_*xxx* .jar.

IBM article link which talks about the same: http://www-01.ibm.com/support/docview.wss?uid=swg21959125

[#5]

@Carol Elliott,

1. 1.Remote server needs to direct its output to our local machine via DISPLAY environment variable.
   -echo $DISPLAY -> localhost:10.0(With putty, $DISPLAY is automatically set to point at display 10 or above when
   X-forwarding )
   - write in local ssh configuration file : Host remote.host.name
   ForwardX11 yes
   2.We at our end , local machine need to accept the connection after authenticating it -authentication done in two ways
   -using ssh , which forwards x-connection
   -making an entry in host mechanism
   -the magic cookie mechanism
2. Hence ask the remote server to direct its output to you as until they direct it , you cant see the sqoop commands on your display running at remote server.