Solved

NULL

Hi, Reading the latest NiFi manual from the Hortworks web site: http://docs.hortonworks.com/HDPDocuments/HDF1/HDF-1.2.0.1/bk_AdminGuide/content/index.html (this uses NiFi version 0.6.0.) The manual talks about creating a 'state' directory you need to use, when you want to use "embedded zookeeper' for a clustered NiFi environment: When using an embedded ZooKeeper, the ./conf/zookeeper.properties file has a property nameddataDir. By default, this value is set to./state/zookeeper. If more than one NiFi node is running an embedded ZooKeeper, it is important to tell the server which one it is. This is accomplished by creating a file named myid and placing it in ZooKeeper's data directory. The contents of this file should be the index of the server as specific by theserver.<number>. So for one of the ZooKeeper servers, we will accomplish this by performing the following commands: cd $NIFI_HOME mkdir state mkdir state/zookeeper echo 1 > state/zookeeper/myidWhen I started looking arou

I am new to HDFS Transparent encryption feature. I am using cloudera CDH 5.4.9 and trying to use hdfs encryption in following manner.# I am using Java KeyStore and configured KMS service and integrated HDFS to use this JAVA KMS# Create two usersuseradd -m user1passwd user1useradd -m user2passwd user2 # As a user1 perform following operations# Create key and create user1 encryption zone# link user1 zone to created keysu user1hadoop key create user1key1hadoop fs -mkdir /tmp/user1zone1su hdfshdfs crypto -createZone -keyName user1key1 -path /tmp/user1zone1 # verify zone is createdhdfs crypto -listZones # create file with user1 credential and put into user1 encryption zoneecho "Hello World" > /tmp/helloWorld.txthadoop fs -put /tmp/helloWorld.txt /tmp/user1zone1hadoop fs -cat /tmp/user1zone1/helloWorld.txtHello World # Verify if file is encryptedsu hdfshadoop fs -cat /.reserved/raw/tmp/zone1/helloWorld.txtT▒▒6▒5▒▒7̼[ # Now login as another usersu user2hadoop fs -cat /tmp/user1zone1/helloW

I am looking deepen my understanding on type of storage disk used for data nodes. outside of single point of failure (SAN box goes down) are they any other reason not to use SAN storage on data nodes? Spindle dedicated on SAN? Is that even possible? How is performance san vs das (dedicated attached storage)? Any insights you can share would be appreciated.

Hello, Hiveserver2 is non-responsive and the log reveals below error. Cluster is Kerberos enabled and there is no HA on Hive. hive.cluster.delegation.token.store.class - org.apache.hadoop.hive.thrift.ZooKeeperTokenStoreAmbari V 2.0HDP V 2.2.4.2 2016-01-19 23:20:18,466 ERROR [HiveServer2-Handler-Pool: Thread-159]: metadata.Hive (Hive.java:getDelegationToken(2910)) - java.lang.UnsupportedOperationException: getDelegationToken() can be called only in thrift (non local) mode at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDelegationToken(HiveMetaStoreClient.java:1664) at sun.reflect.GeneratedMethodAccessor72.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90) at com.sun.proxy.$Proxy12.getDelegationToken(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.ge

Whats the difference between the two?

I use a kerberized cluster and once in a while I notice following error in my zookeeper client logs:   15/11/15 15:46:53 ERROR client.ZooKeeperSaslClient: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Connection reset)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state. 15/11/15 15:46:53 ERROR zookeeper.ClientCnxn: SASL authentication with Zookeeper Quorum member failed: javax.security.sasl.SaslException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Connection reset)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state.   So, I had following doubt with this:   It is showing

Is there a way to configure a non-default replication factor for a HDFS directory such that all future files and sub-directories in that directory use that specific replication factor? Currently, we are using a work around of running a daemon process to set the replication factor for all files in the required directory. Is there a better way to do this? while true; do hdfs dfs -setrep -w 2 /tmp/ sleep 30done I see at one point there was this JIRA https://issues.apache.org/jira/browse/HDFS-199 opened but is blocked by this JIRA [https://issues.apache.org/jira/browse/HADOOP-4771]

Is there a way of restricting access to an HBase coprocessor in a multi tenant environment ? What should i be taking into consideration when using coprocessor ?

When I try to download some of the protected stuff, such as Cloudera Express or Quickstart VM, I have to submit some form. The thing is form submission is broken due to the absence of some geoplugin. More details on the attached image (I erased unnecessary info). Is this a bug or I'm doing something wrong?

Hello,Just a question - when I see a list of posts, some author names are appearing in blue and some in black. What is the logic and difference between these? Thanks,Martin

Hi, I am setting up a Hadoop cluster and I am having trouble with Zookeeper health tests. Our devops team has closed all ports in the machines except those used by zookeeper, namely: 2181, 3181, and 4181. That makes the Zookeeper canary test to fail. If we open all ports in the machines, the canary test passes. I have seen two types in problems in the logs: WARN org.apache.zookeeper.server.quorum.Learner: Unexpected exception, tries=2, connecting to myhostname/10.2.3.23:3181java.net.ConnectException: Connection refused. Which is odd, because port 3181 is open. and: INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /10.2.2.11:50561INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x14ba997bb9d0e82 with negotiated timeout 30000 for client /10.2.2.11:50561 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /10.2.2.11:50561 which had sessionid 0x14ba997bb9d0e82WARN com.cloudera.cmf.event.publ

Hello,   I  want to do a folder that can be accessed in r, w,x by all the users of the group 'developer' that I created. I also want that all the folders/files created by these users in this folder can be accessed in r,w,x by the users from this group.  All the other users should access the folder in r and x.   What should I do?   I tried:   - to create my folder with these rights: drwxrwxr-t+   - the folder that I created is owned by admin who is part of the developer group.   All I think is missing is to add the default group of my users the developer group : do something like hadoop def -usermod -G developer Peter But this is not working!      Thank you!    

Hi, I like the site, it had lots of information, that is easy to find and query.Downloads and community provide all that is needed to get started with the cloudera platforms. One suggestion:Once a user is "signed in", and he / she wants to download a package, the same registration info is required to fill in before the download is enabled.That information is available right ? Would be nice if you can leave out the extra form for signed in users.  Kind regards,Ken.

This has been bugging me for a long time now, why does Cloudera's website not have any link to my user profile?  I have spent a lot of time searching your site and I can't find that link.  The only way I can get to it is from emails I received.  It's very annoying.

Hello, We are trying to execute a MapReduce job to write to HBASE using OOZIE Workflow. We have implemented Kerberos in our Cluster and that is when these issues started.... The Job is failing after 10 minutes with an error: We had a similar issue with beeline not able to run HBASE Commands and the solution was - I think the issue is that HBase does not allow hive to impersonate users.  So you'll need to setup hive as a proxy user in HBase.  Can you try the following: - Go to the HDFS service configuration in CM. - Go to Service-Wide->Advanced and add the following to "Cluster-wide Configuration Safety Valve for core-site.xml": I have added similar entry for OOZIE - HDFS Service-Wide Advanced properties and add the following to Cluster-wide ConfigurationSafety Valve for core-site.xml Here is what i have in the logs - Oozie Launcher failed, finishing Hadoop job gracefully

 I am trying to make my first post to the commmity and am limited to 100 characters in chrome, firefox and IE.  <textarea class="lia-form-question-field-input lia-form-input-vertical" id="lia-questionField" name="questionField">Type your question here...</textarea> That can't be right. Thanks, Tod

Hi there! I am new to Cloudera and signed up as Cloudera partner. I have created my cloudera account and able to login to Partner Portal. However once I go to the e-learning page, for example http://cloudera.com/content/cloudera/en/training/library/hadoop-essentials.html, the page requires me to fill the registration form again, even if I already sign in the site. I just dont want to fill the form everytime I attent the training courses, any way to work around this problem? Thank you!

Failed to execute query: SELECT Id, Type, Name, Domains__c, Partner_Status__c, Reseller__c, Software_Vendor_ISV__c, Hardware_Vendor_IHV__c, Training_Delivery_Partner__c, System_Integrator__c , Portal_User_Creation__c, Entitlement_Count__c, KB_Allowed__c, Auto_Provision__c, Portal_User_Profile__c, Partner_Application_Status__c FROM Account WHERE Id = '00180000011IbSRAA0' Cannot serve request to /content/support/en/downloads.html on this server ApacheSling/2.2 (Day-Servlet-Engine/4.1.24, Java HotSpot(TM) 64-Bit Server VM 1.6.0_34, Linux 2.6.32-358.el6.x86_64 amd64)