About haeusler

haeusler · ‎02-06-2019

As John described the package seems to only provide a broken symlink. Any other hints on this topic would be highly appreciated @Jay Kumar SenSharma It seems that the issue still persists for John as well, as you can see in the follow up question: https://community.hortonworks.com/questions/232464/libhdfs-problems.html

haeusler · ‎06-28-2017

Thanks a lot @mungeol heo - that setting does the trick. Do you happen to know whether hive.session.id should be globally unique? On first tests, it seems to work to reuse the same hive.session.id across multiple yarn applications, but it feels a bit dangerous as the setting is named "id".

haeusler · ‎04-24-2017

When executing a simple "hdfs dfs -cat" on a file, I get a NullPointerException (running on HDP 2.4.2): hdfs --loglevel TRACE dfs -cat '/filename' [...] 17/04/24 18:06:39 TRACE ipc.ProtobufRpcEngine: 1: Call -> prd-hmaster-01-tor/172.22.5.37:8020: getBlockLocations {src: "/filename" offset: 0 length: 1342177280} 17/04/24 18:06:39 DEBUG ipc.Client: IPC Client (1293677337) connection to prd-hmaster-01-tor/172.22.5.37:8020 from hdfs sending #1 17/04/24 18:06:39 DEBUG ipc.Client: IPC Client (1293677337) connection to prd-hmaster-01-tor/172.22.5.37:8020 from hdfs got value #1 17/04/24 18:06:39 TRACE ipc.ProtobufRpcEngine: 1: Exception <- prd-hmaster-01-tor/172.22.5.37:8020: getBlockLocations {org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException } cat: java.lang.NullPointerException 17/04/24 18:06:39 DEBUG ipc.Client: stopping client from cache: org.apache.hadoop.ipc.Client@62656be4 [...] Unfortunately, the NameNode logs do not show a stack trace: 2017-04-24 18:06:39,199 WARN ipc.Server (Server.java:logException(2292)) - IPC Server handler 99 on 8020, call org.apache.hadoop.hdfs.protocol.ClientProtocol.getBlockLocations from 172.22.5.39:17182 Call#1 Retry#0 java.lang.NullPointerException Any hints on how to debug this issue would be much appreciated. Thanks a lot Michael

haeusler · ‎04-07-2017

@Artem Ervits followup question: https://community.hortonworks.com/questions/93819/naming-hive-tez-session-not-individual-dag.html

haeusler · ‎04-07-2017

Setting "hive.query.name" will influence the query name that is shown for an individual DAG on the TEZ UI. But how do we influence the YARN application name of a complete TEZ session that is shown on the YARN UI?

haeusler · ‎04-03-2017

In case you want to leverage structured results from HDFS commands or further reduce latency / overhead, also have a look at "snakebite", which is a pure python implementation of HDFS client functionality: https://github.com/spotify/snakebite https://community.hortonworks.com/articles/26416/how-to-install-snakebite-in-hdp.html

haeusler · ‎03-29-2017

@Artem Ervits maybe you can help here once more? Setting "hive.query.name" will influence the query name that is shown for an individual DAG on the TEZ UI. But how do we influence the YARN application name of a complete TEZ session that is shown on the YARN UI?

haeusler · ‎03-07-2016

When testing Hive queries in HDP 2.4.0 Sandbox, I noticed that simple queries without a from clause fail. In the Ambari Hive View, they fail with "Error while processing statement: FAILED: Hive Internal Error: java.lang.NullPointerException(null)" On the command line, they fail with the following stacktrace: hive> SELECT 42; FAILED: Hive Internal Error: java.lang.NullPointerException(null) java.lang.NullPointerException at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:109) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:270) at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:321) at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:214) at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) When removing the "org.apache.atlas.hive.hook.HiveHook" from "hive.exec.post.hooks" this query works: hive> select 42; OK 42 Time taken: 6.483 seconds, Fetched: 1 row(s)

haeusler · ‎03-03-2016

Thanks for the quick fix @glupu

haeusler · ‎03-02-2016

It seems to depend on whether you enter 127.0.0.1:8888 or localhost:8888 in your browser: 127.0.0.1:8888 shows port 2222 localhost:8888 shows port 22 Screenshot attached.

Online	Offline
Last Visited	‎04-07-2020 09:53 AM

Member Since	‎02-18-2016 01:18 PM
Last Visited	‎04-07-2020 09:53 AM
Posts	24
Kudos received	9

Cloudera Community

Re: libhdfs missing

Re: Naming Hive TEZ session (not individual DAG)

How to debug a NullPointerException on "hdfs dfs -...

Re: Naming TEZ-Hive session

Naming Hive TEZ session (not individual DAG)

Re: Interacting with Hadoop HDFS using Python code...

Re: Naming TEZ-Hive session

Hive queries without from clause seem to fail when...

Re: HDP 2.4.0 Sandbox: documentation typo regardin...

Re: HDP 2.4.0 Sandbox: documentation typo regardin...