Is Solr removed from HDP install? I see the solr component listed for 2.5.3. So wondering why it has been removed in 2.6 ... View more

We are planning an upgrade of our HDF. We are currently in 2.1.2 We heard about SAM - Stream Analytics Manager - a new component in latest version of HDF. Which HDF version is best to upgrade for this? Ideally, 1. We want the new HDF version to have a stable version of SAM 2. The entire edition should be less buggy 3. Ideally no config/data loss (or) minimal config change only. Thanks, ... View more

We are Currently having HDP version 2.5.3. We have 2.6.0, 2.6.1 and 2.6.2 in front of us. Which is the best version to upgrade? The following points are important to us: 1. Configuration and Data should be unchanged to a large extent 2. New version must be less buggy. We had a tough time with Zeppelin on 2.5.3. Looking out for some experience from people who already done it. ... View more

We have a Spark SQL Thrift Server running with Kerberos enabled. While Kerberos helps in authenticate users, I am not sure how I could authorize who could use what tables. We use "Ranger" and it has "hive" policies. I think this is more for "HS2" access. Does this also protect access via Spark-SQL Thrift server? ... View more

How do I list contents of a Remote non-kerberized cluster (say NKB) from a kerberized cluster (KB) So, I am here sitting on "KB" edge node and saying "hdfs dfs -ls hdfs://remotenn:8020/tmp" And all I get is Java "EOFException" Now, I try "hdfs dfs -Dipc.client.fallback-to-simple-auth-allowed=true -ls hdfs://remotenn:8020/tmp" No change in Behavior (Same EOFException). 17/06/28 02:25:00 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.getFileInfo over null. Not retrying because try once and fail. java.io.EOFException: End of File Exception between local host is: "xxx.com/aa.bb.cc.dd"; destination host is: "yyyy.com":8020; : java.io.EOFException; For more details see: http://wiki.apache.org/hadoop/EOFException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:801) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:765) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1556) at org.apache.hadoop.ipc.Client.call(Client.java:1496) at org.apache.hadoop.ipc.Client.call(Client.java:1396) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233) at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:816) ....... and so on My other question is how does the "unsecure" (NKB) cluster treat my credential as ? What user name will it map it. The "auth_to_local" field is set to DEFAULT in the NKB cluster. Can some1 help? ... View more

We added a new user in our AD and Ranger is not syncing it at all. What could be the potential reason? I ambari-synced the user and the user is now reflected in Ambari users.... but the same user is not visible in Ranger for giving permissions. It has been more than 2 days and the sync has not happened. What should I do? How do I make it see? ... View more

We need to copy data from a non-kerberized (NK) cluster to a kerberized one (K). I am registering entities in "K" cluster. I was able to register the "K" cluster with the falcon server running on "K" cluster without any problems. While registering the "NK" cluster to the falcon server running on "K" , I run into multiple issues. Falcon seems to expect a "kerberos" principal by default. It is not clear if this property is for the "K" cluster or for the "NK" cluster..... Right now, I am using the namenode prinicipal in "K" cluster itself (not sure if this is right). And, while registering the NK entity, an error message says "Server asks us to fall back to SIMPLE auth, but this client is configured to only allow secure connections". ... So , I just added a property like this: " <property name="ipc.client.fallback-to-simple-auth-allowed" value="true" />". But this does not help either. Can some1 help how to do this with Falcon. Also, please kindly point me to some nice tutorials on Falcon. Thank you. ... View more

Whenever I restart NiFi, I see this Warning followed by an exception stacktrace. When I look at the "crc" file, it is actually owned by "root" and is not readable by anyone else.... So, what do we do? Why is it like this? Is this a problem? 2017-05-25 19:48:31,503 WARN [main] org.apache.hadoop.fs.FSInputChecker Problem opening checksum file: file:/etc/ranger/DEV_HDF_nifi/cred.jceks. Ignoring exception: java.io.FileNotFoundException: /etc/ranger/DEV_HDF_nifi/.cred.jceks.crc (Permission denied) ......huge stacktrace... 2017-05-25 19:48:32,295 INFO [main] o.a.ranger.plugin.util.PolicyRefresher PolicyRefresher(serviceName=DEV_HDF_nifi): found updated version. lastKnownVersion=-1; newVersion=12 2017-05-25 19:48:32,354 INFO [main] o.a.r.plugin.util.RangerResourceTrie resourceName=nifi-resource; optIgnoreCase=true; optWildcard=true; wildcardChars=*?; nodeCount=21; leafNodeCount=3; singleChildNodeCount=17; maxDepth=11; evaluatorListCount=2; wildcardEvaluatorListCount=2; evaluatorListRefCount=19; wildcardEvaluatorListRefCount=19 Should I be worried about Security? ... View more