Hi All, My cluster is on HDP 2.5.3 and we have multiple jobs running every 2-5 minutes frequency. So there will be 8-10 K job id's generated in a day with job history. Actually I wanted a log of job which was 3 days older & I was not able to find since MapReduce JHS was storing 20,000 Job ID's, how do I increase this to maximum number? ... View more

Hi All, Is there a way to use RHEL 7 custom AMI on AWS with Cloudbreak as I can currently see only Amazon Linux is being supported? Is there any way that I can use RHEL7/Centos7 on AWS using Cloudbreak? Thanks in advance. ... View more

We are trying to setup the Kerberos on our Dev environment using Exising AD and we will also need to do similarly in Production cluster later. What are the challenges that we might face after having necessary prerequisites in handy also? Note: We do not have Hortonworks Support, what are issues that might occur if we do it on our Production cluster without enterprise support? ... View more

Hi All, Earlier we were not able to install HDF on top of HDP, as per latest document (https[:]//docs.hortonworks[.]com/HDPDocuments/HDF3/HDF-3.0.2/bk_release-notes) we can install HDF on top of existing HDP cluster. My HDP version: 2.6.4.0, Ambari 2.6.1.3 I need to install NiFi on my cluster through HDF, but the document refers to install the databases on MySQL 5.5.x or PostgreSQL 9.5 later versions on Ambari Server node but I have MySQL (5.1.73) and PostgreSQL(8.4.20) on the same Server which supports my current HDP environment. Now how can I proceed further to install? Can someone please brief the steps here to proceed further? Do we really require installing these DB servers or databases to install NiFi from HDF? Thanks in Advance. ... View more

Hi, We are currently on HDP 2.5.3 running on AWS EC2 instance type m4.4xlarge with 16 CPUs and 64 GB memory. We need to change the Instance type from m4.4xlarge to m4.10xlarge (40 CPUs and 160 GB memory). Changing the Instance types does not harm any EBS volumes or data that is residing but are there any other issues post changing the instance type? If yes, what kind of issues we might face? If anyone has already succeeded, please brief me the steps prior to changing the instance type. Appreciate for response. Thank you. ... View more

All, I have set up PAM authentication and successfully authenticated with Linux users but I see this error in hiveserver2.log, can someone help with this error. 2017-07-24 03:55:46,111 ERROR [HiveServer2-Handler-Pool: Thread-5643]: transport.TSaslTransport (TSaslTransport.java:open(315)) - SASL negotiation failure javax.security.sasl.SaslException: Error validating the login [Caused by javax.security.sasl.AuthenticationException: Error authenticating with the PAM service: sshd] at org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(PlainSaslServer.java:109) at org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:539) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:283) at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: javax.security.sasl.AuthenticationException: Error authenticating with the PAM service: sshd at org.apache.hive.service.auth.PamAuthenticationProviderImpl.Authenticate(PamAuthenticationProviderImpl.java:46) at org.apache.hive.service.auth.PlainSaslHelper$PlainServerCallbackHandler.handle(PlainSaslHelper.java:106) at org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(PlainSaslServer.java:102) ... 8 more 2017-07-24 03:55:46,112 ERROR [HiveServer2-Handler-Pool: Thread-5643]: server.TThreadPoolServer (TThreadPoolServer.java:run(296)) - Error occurred during proces sing of message. java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Error validating the login at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.thrift.transport.TTransportException: Error validating the login at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) ... 4 more Though there are no issues with Hive authentication but why I'm getting these errors? ... View more