I was looking into the Open ID way of user authentication for Nifi Registry. I see that we can implement OpenID for Nifi but it is not available for Nifi Registry? Is there a way we can implement Open ID authentication for NiFi registry also? ... View more

In one of our data nodes the root partition is almost 98% full and I see that there is nothing much I can clean up. I saw below tar.gz files in the root partition: rw-r--r-- 1 root root 196M /usr/hdp/2.5.0.0-1245/hadoop/mapreduce.tar.gz r--r--r-- 1 root root 378M /usr/hdp/2.5.0.0-1245/oozie/oozie-sharelib.tar.gz -rw-r--r-- 1 root root 117M /usr/hdp/2.5.0.0-1245/pig/pig.tar.gz -rwxr-xr-x 1 root root 121M /usr/hdp/share/hst/smartsense-activity-explorer-1.3.0.0-22.tar.gz Will there be any impact if I move these tar.gz files to a different lodcation so that we can have more space in the root partition? ... View more

Hi All, I am facing a wierd problem today. We noticed that the space in the root partition was full. The /var/log/nifi is configured in the same partition. I cleared almost all the nifi logs, however within a minute they are appearing back and making root partition full. What might be the cause for this ? How do I get it resolved? I have a standalone nifi instance ... View more

Hi All, I have a requirement where in I have to setup a new HDP cluster which will be replica of our one another cluster. WHat I am planning is to export the Ambari Blueprint and use the same blue print to setup another server. What are the right steps for me to do and what else I need to take care to get this task done? ... View more

I have a requirement where in I have to maintain a cache of about 200,000 records. I want to use H2 database as a cache. If any body have used H2 as a cache please let me know the steps. ... View more

Hi All, I have taken a Snapshot in my Dev Environment and Exported to PreProd environment so that I can clone a new table from snapshot or else I can restore it. I exported the snapshot from Dev to Pre Prod using below command: /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot snapshot_table_name -copy-to hdfs://NAME_NODE:8020/hbase -mappers 5 My problem is when I try to clone the table from this exported snapshot I get error Unknown Snapshot and also this exported snapshot is not showing up when I hit list_snapshot commad in Hbase shell. I can see the exported snapshot in hdfs using below command as I gave the hdfs path as hdfs://NAME_NODE:8020/hbase: hadoop fs -ls /hbase However all the other snapshots which I created in Pre Prod are present in the below location: hadoop fs -ls /apps/hbase/data How can I resolve this issue? Is there a problem in the way I exported the Snapshot? Do i have to export the snapshot and give the path as hdfs://NAME_NODE:8020/apps/hbase/data ? Request for assistance. ... View more

Hi All, I have a requirement in which we have a table which has about 100 Million records. On this table we want to enable Salting. Inorder to enable the salting I would have to recreate the table which has 100 Million records. Can any one suggest what would be the best way to migrate these 100 million records to a different place, recreate the table with SALTING and put the data back into the SALTED table. Also I am reading in the official documentation that "There are some cautions and difference in behavior you should be aware about when using a salted table." Can anyone guide what could be the possible difference in behavior we can see? ... View more

I have created an INDEX on my tables using below DDL: CREATE INDEX my_index ON my_table (col1, col2, col3 , EVENT DESC) INCLUDE (col4 , col5); However the EVENT column is not sorted in descending order. Is there anything I am missing? Have I created the Index correctly? One thing I wanted to highlight was my EVENT column datatype is VARCHAR....is that making EVENT column not to sort? ... View more

Hi, I have a requirement in which I want the event column to be indexed. However I wanted to know if it is possible to ORDER the event column in Descending order something like below: CREATE INDEX my_index ON my_table (EVENT ORDER BY DESC) INCLUDE (v2); My Base table has EVENT column but it is not ordered, I am thinking if indexing EVENT column in Descending order will have me get away with the ORDERBY clause and reduce query time. Request for comments. ... View more

Hi , I am trying to configure nifi Execute Process to connect to MSSQL and execute the store procedure . I followed this link https://community.hortonworks.com/questions/26170/does-executesql-processor-allow-to-execute-stored.html where in @M. Mashayekhi has provided the steps how he connected to MSSQL and executed the Stored Procedure. @Mashayekhi , I wanted to know how your execute process configuration screen looks like and also please let me know if any additional client tools are required to be installed so that I can execute the stored procedure. Thank you. ... View more

Hi All, we have a HBase/Phoenix Table, wherein we have created a covered index in improve read performance. There is a requirement that we create one more covered index on the same HBase/Phoenix Table. Will that cause any performance issues on the table due to two indexes on the same table? ... View more

We have an Phoenix table in which records are inserted by Nifi workflow. Our table and primary key looks something like below: PRIMARY KEY COL1 COL2 COL3 COL4 COL5 COL1 | COL2 | COL3 We faced some issues in Nifi and |COL2|COL3 was inserted as primary keys for all rows without COL1 value. Now our primary key looks like this -> ( |COL2 | COL3) . Is there any upsert statement to update existing primary key with COL1 value so that out primary key become like this --> COL1|COL2|COL3 ? @Sergey Soldatov @Josh Elser ... View more

Some of the users who access our cluster are trying to access HDFS. However when they try accessing HDFS they get permission denied error on production cluster. Below are the steps I am planning to implement in Ranger : 1. Since Ranger plugin is enabled, I will add a policy. 2. Mention the path which I need to provide access to 3. Add there userid's and type of access and save. However problem is the users whom I want to add are not showing up in the list of users. Can we go ahead and manually add the users in Ranger and then provide access? Request you guys help to validate it. or is there anything else I have to take care before doing these? ... View more

Couple of Days Ago I observed that Nifi was not processing any records and it was like hanged. When I saw the bootstrap.log I saw this message "Apache nifi is running at PID () but not responding to ping requests". The issue got fixed by restarting NiFi service, however what could be the possible reason for this issue , it happened twice in our cluster. What can be done so that issue dont reoccur again. Request for comments. ... View more

HI All, I have ORACLE as my Hive Database and I am getting belwo error in ORACLE DB : "ORA-00060: Deadlock detected. See Note 60.1 at My Oracle Support for Troubleshooting ORA-60 Errors. More info in file /apps/oracle/diag/rdbms/hive" WHat could be the possible reasons for this...how to resolve this? It is creating trace files of very large sizes and sending alerts ... View more