Member since
10-24-2016
11
Posts
1
Kudos Received
0
Solutions
11-27-2017
03:00 PM
I have 3 nodes nifi cluster as below: node 1: 32GB, 8 cores node 2: 128GB, 32 cores node 3: 256GB, 64 cores Could someone please help to explain how the load distribution works in nifi cluster? will coordinator manager distribute more loads to node 3 since it is more powerful? If yes, how do I confirm that? because I didn't set number of threads for each nodes, default value are used for all 3 nodes. And why a RPG group is needed in nifi cluster? I googled but couldn't find answer for that.
... View more
Labels:
10-25-2017
03:13 AM
I'm wondering is there a way to merge few data loader to 1 powerful loader so the resources can be shared among the data loader.
... View more
Labels:
07-24-2017
02:48 AM
Command: select distinct .... from <database> error: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1497551165160_3893578_1_00, diagnostics=[Vertex vertex_1497551165160_3893578_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: summary initializer failed, vertex=vertex_1497551165160_3893578_1_00 [Map 1], java.io.IOException: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider executed above SQL command in Hive, and saw the job stuck in initializing stage, and an error saying: Adapter operation failed: 500: failed to fetch result from proxy. The command was working few weeks back, but not now. Any insight on this issue would be appreciated
... View more
- Tags:
- hadoop
- Hadoop Core
Labels:
04-28-2017
03:00 AM
Just curious why the data is missing in HDFS directory, and seeing the error below when trying to get the partition list from the table. Error occurred executing hive query: Error while compiling statement: FAILED: SemanticException java.io.FileNotFoundException: File hdfs://KSD01/tera/WW=0/test=JM does not exist. but the partition can be found in the result of command :"show partitions <table_name>" command
... View more
- Tags:
- Hadoop Core
- HDFS
- Hive
Labels:
11-22-2016
03:55 AM
@Sunile Manjee, sorry I'm new to ranger security, what is the command to check the access?
... View more
11-22-2016
03:33 AM
error received when trying to create UDF function: Code: ADD JAR /home/nbg/mlut.jar create temporay function mlut as 'hive.udf.mlut' Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [sbig] does not have [CREATE] privilege on [mlut] with kerbores installed Tried few workaround, but failed: 1.create a temporary tmp folder, and chmod to 777 2. Add the java file to tmp file, and list jars (confirmed file has been added) 3. But still failed.
... View more
Labels:
10-31-2016
03:20 AM
[Hortonworks][HiveODBC] (34) Error from Hive: Bad Status: HTTP/1.1 500 Server Error.
... View more
Labels:
10-24-2016
03:01 PM
I have an external hive table in Hadoop with few partitioned key. For example, 10 columns and 2 partitioned key. I was trying to pull data from Hadoop by using Tableau Hortonwork ODBC driver, the query time is very long. Is there a way to enhance the performance in Hadoop/Tableau?
... View more
Labels:
10-24-2016
02:34 PM
1 Kudo
In python or others programming code, we can use "print" command to check whether the variable is correctly set or not. Is there any library or function in nifi that serve same functionality as "print"? I'm wondering what is the better debugging process in Nifi? I'm new to Nifi, I spent most of times to figure out what's wrong in the nifi processor group, because sometimes nifi doesn't throw any error, and the processor group just hang in there. The processor group is working correctly after restarted the nifi. I googled, and can't find any related debugging documentation. Any advice would be greatly appreciated.
... View more
Labels:
10-24-2016
07:37 AM
I want nifi to use kerberos principal while running the Hive script in Hadoop with Kerbores set up. Is there a way to automate renew Kerbores ticket? Nifi executestream process group stopped working after the Kerbores ticket expired. The process continue after executing command "kinit <username>". current nifi.properties file: nifi.kerberos.krb5.file=/etc/krb5.conf nifi.kerberos.service.principal=<username> nifi.kerberos.keytab.location=/home/username.keytab nifi.kerberos.authetication.expiration=12 hours
... View more
Labels: