Member since
12-14-2016
58
Posts
1
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
267 | 04-19-2017 05:49 PM | |
271 | 04-19-2017 11:43 AM | |
500 | 04-19-2017 09:07 AM | |
616 | 03-26-2017 04:20 PM | |
1755 | 02-03-2017 04:44 AM |
06-24-2019
02:49 PM
Hi All, We have a three node Nifi Cluster which is in backend and haproxy (LB) in frontend with a defined port for client to send request to the LB which will hit Nifi cluster in backend in round robin method. We have a built a pipeline which has "HandleHTTPRequest" --> intermediate processing using multiple processors | "HandleHTTPResponse". Client application may send multi part payload with different file extension and within the pipeline we parse and process according to business logic and move it to Network/Central Drive. When client application submits parallel payloads, we are being hit by below error on "HandleHTTPRequest" processor. I have tried to increase concurrent tasks for "HandleHTTPRequest" from "1 to 10" and all nodes on our cluster can handle the increase in tasks as we have enough CPU. Also we have increased the container queue size from "50 to 80" Failed to receive content from HTTP Request from <host> due to org.eclipse.jetty.io.EofException: Early EOF: org.eclipse.jetty.io.EofException: Early EOF
org.eclipse.jetty.io.EofException: Early EOF
at org.eclipse.jetty.server.HttpInput$3.getError(HttpInput.java:1138)
at org.eclipse.jetty.server.HttpInput$3.noContent(HttpInput.java:1126)
at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:313)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
at org.eclipse.jetty.util.ReadLineInputStream.read(ReadLineInputStream.java:131)
at org.eclipse.jetty.util.MultiPartInputStreamParser.parse(MultiPartInputStreamParser.java:769)
at org.eclipse.jetty.util.MultiPartInputStreamParser.getParts(MultiPartInputStreamParser.java:493)
at org.eclipse.jetty.server.MultiParts$MultiPartsUtilParser.<init>(MultiParts.java:121)
at org.eclipse.jetty.server.Request.newMultiParts(Request.java:2410)
at org.eclipse.jetty.server.Request.getParts(Request.java:2333)
at org.eclipse.jetty.server.Request.getParts(Request.java:2319)
at org.apache.nifi.processors.standard.HandleHttpRequest.onTrigger(HandleHttpRequest.java:575)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Thanks in Advance, Ram
... View more
Labels:
09-04-2018
06:46 AM
Hi All, My cluster is on HDP 2.5.3 and we have multiple jobs running every 2-5 minutes frequency. So there will be 8-10 K job id's generated in a day with job history. Actually I wanted a log of job which was 3 days older & I was not able to find since MapReduce JHS was storing 20,000 Job ID's, how do I increase this to maximum number?
... View more
Labels:
08-22-2018
03:42 PM
Hey I was using CB 2.4.0 and found that I can upgrade to CB 2.7.1 which worked like a charm with Centos 7 and Thanks much @Harald Berghoff & @mmolnar. Cheers, Ram
... View more
08-22-2018
01:04 PM
Thanks for the reply @Harald Berghoff, Yes AWS supports RHEL but i'm unable to enable RHEL 7 with Cloubbreak. I have already followed the above link earlier and tried below steps. 1. I created a Custom Image and it is ready in my AMI's. 2. I created the Image Catalog with custom setting for RHEL 7 Custom AMI and Cloudbreak is unable to recognize. (But custom Amazon Linux worked well with custom image catalog) Can you help here? Regards, Ram
... View more
08-22-2018
10:48 AM
Hi All, Is there a way to use RHEL 7 custom AMI on AWS with Cloudbreak as I can currently see only Amazon Linux is being supported? Is there any way that I can use RHEL7/Centos7 on AWS using Cloudbreak? Thanks in advance.
... View more
05-09-2018
12:03 PM
We are trying to setup the Kerberos on our Dev environment using Exising AD and we will also need to do similarly in Production cluster later. What are the challenges that we might face after having necessary prerequisites in handy also? Note: We do not have Hortonworks Support, what are issues that might occur if we do it on our Production cluster without enterprise support?
... View more
Labels:
05-08-2018
09:23 AM
Can you explain in detail as how you solved, facing similar issue now. Thanks in advance.
... View more
03-26-2018
05:24 PM
Hi All, Hortonworks ODBC tools works for most of the 3rd party tools but it does not work for JMP (SAS) tool, do we have any other alternative? I wanted to use Zookeeper Quorum url for HiveServer2 HA connectivity in JMP tool. Thanks in advance.
... View more
03-12-2018
09:44 AM
Hi @sapin amin, sorry for the delay in reply, I have managed to install HDF using Management Pack (tar.gz). Also NiFi is installed successfully. Ambari is using default which is Postgres 8.4.20 and MySql 5.1.73 is being used by Hive, Oozie and Ranger. (Both PSql and MySql are running on Ambari Server node) Now do I need to install Schema Registry & SAM on new MySql version on a different nodes ? Thanks, Ram
... View more
02-21-2018
04:08 PM
Hi All, Earlier we were not able to install HDF on top of HDP, as per latest document (https[:]//docs.hortonworks[.]com/HDPDocuments/HDF3/HDF-3.0.2/bk_release-notes) we can install HDF on top of existing HDP cluster. My HDP version: 2.6.4.0, Ambari 2.6.1.3 I need to install NiFi on my cluster through HDF, but the document refers to install the databases on MySQL 5.5.x or PostgreSQL 9.5 later versions on Ambari Server node but I have MySQL (5.1.73) and PostgreSQL(8.4.20) on the same Server which supports my current HDP environment. Now how can I proceed further to install? Can someone please brief the steps here to proceed further? Do we really require installing these DB servers or databases to install NiFi from HDF? Thanks in Advance.
... View more
11-03-2017
05:03 AM
Hi, We are currently on HDP 2.5.3 running on AWS EC2 instance type m4.4xlarge with 16 CPUs and 64 GB memory. We need to change the Instance type from m4.4xlarge to m4.10xlarge (40 CPUs and 160 GB memory). Changing the Instance types does not harm any EBS volumes or data that is residing but are there any other issues post changing the instance type? If yes, what kind of issues we might face? If anyone has already succeeded, please brief me the steps prior to changing the instance type. Appreciate for response. Thank you.
... View more
Labels:
11-03-2017
04:57 AM
Hi Pavan, Did you manage to change the Instance Type? Even we are upgrading it from m4.4xlarge to m4.10xlarge, let me know if you have faced any issues post doing this. If it was successful can you post the brief steps. Thanks. -Ram Charan
... View more
07-24-2017
08:06 AM
All, I have set up PAM authentication and successfully authenticated with Linux users but I see this error in hiveserver2.log, can someone help with this error. 2017-07-24 03:55:46,111 ERROR [HiveServer2-Handler-Pool: Thread-5643]: transport.TSaslTransport (TSaslTransport.java:open(315)) - SASL negotiation failure
javax.security.sasl.SaslException: Error validating the login [Caused by javax.security.sasl.AuthenticationException: Error authenticating with the PAM service: sshd]
at org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(PlainSaslServer.java:109)
at org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:539)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:283)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.security.sasl.AuthenticationException: Error authenticating with the PAM service: sshd
at org.apache.hive.service.auth.PamAuthenticationProviderImpl.Authenticate(PamAuthenticationProviderImpl.java:46)
at org.apache.hive.service.auth.PlainSaslHelper$PlainServerCallbackHandler.handle(PlainSaslHelper.java:106)
at org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(PlainSaslServer.java:102)
... 8 more
2017-07-24 03:55:46,112 ERROR [HiveServer2-Handler-Pool: Thread-5643]: server.TThreadPoolServer (TThreadPoolServer.java:run(296)) - Error occurred during proces
sing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Error validating the login
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException: Error validating the login
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
... 4 more
Though there are no issues with Hive authentication but why I'm getting these errors?
... View more
- Tags:
- Hive
- hiveserver2
- pam
Labels:
07-21-2017
01:55 PM
Will you be able to post more information and logs that are available?
... View more
07-21-2017
01:50 PM
Hi All, I'm on HDP 2.3.6 with Spark 1.5.2. Since R does not support sparklyr for Spark 1.5.2. So I downloaded 2.0.1 Spark and i'm running it as Standalone to test it out before distributing it other nodes. So I have connected to Hive Metastore, where Managed tables are accessible through this Spark perfectly, but I'm unable to do simple select for S3 tables. I'm using this config in Spark-Shell and same way for Spark-Sql using spark.properties. Spark Thrift server was started from /sbin/. Also hadoop-aws and aws-sdk jars are in place. val hc = sc.hadoopConfiguration
hc.set("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem")
hc.setInt("fs.s3a.connection.maximum", 100)
hc.set("fs.s3a.awsAccessKeyId", "OUR ACCESS KEY")
hc.set("fs.s3a.awsSecretAccessKey", "OUR SECRET KEY")
Spark.Properties for Spark Sql spark.hadoop.fs.s3a.impl org.apache.hadoop.fs.s3a.S3AFileSystem
spark.driver.extraClassPath /home/ram/spark-2.0.1/jars/hadoop-aws-2.7.1.2.3.6.0-3796.jar:/home/ram/spark-2.0.1/jars/aws-java-sdk-1.11.164.jar
spark.hadoop.fs.s3a.access.key OUR ACCESS KEY
spark.hadoop.fs.s3a.secret.key OUR SECRET KEY
So here's the error, scala> import org.apache.spark.sql.SQLContext
scala> val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc);
scala> val df_1=sqlContext.sql("select * from win.win_vi1_data")
scala> df_1.show()
java.lang.NoSuchMethodError: org.apache.hadoop.security.ProviderUtils.excludeIncompatibleCredentialProviders(Lorg/apache/hadoop/conf/Configuration;Ljava/lang/Class;)Lorg/apache/hadoop/conf/Configuration;
at org.apache.hadoop.fs.s3a.S3AFileSystem.getAWSAccessKeys(S3AFileSystem.java:352)
at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:160)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:258)
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:229)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:199)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
at org.apache.spark.rdd.UnionRDD$$anonfun$1.apply(UnionRDD.scala:84)
at org.apache.spark.rdd.UnionRDD$$anonfun$1.apply(UnionRDD.scala:84)
at scala.collection.parallel.AugmentedIterableIterator$class.map2combiner(RemainsIterator.scala:115)
at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2combiner(ParVector.scala:62)
at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1054)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1051)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.internal(Tasks.scala:159)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinTask.doJoin(ForkJoinTask.java:341)
at scala.concurrent.forkjoin.ForkJoinTask.join(ForkJoinTask.java:673)
at scala.collection.parallel.ForkJoinTasks$WrappedTask$class.sync(Tasks.scala:378)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:443)
at scala.collection.parallel.ForkJoinTasks$class.executeAndWaitResult(Tasks.scala:426)
at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:56)
at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:958)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIterableLike.scala:953)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:152)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
scala> :quit How do I get rid of this error, I have tried all the way but no luck. Any help, Appreciated!!
... View more
07-04-2017
07:41 AM
Thanks a lot @Mushtaq, this helped me.
... View more
05-04-2017
09:50 AM
Hi, How do we connect to Hive using presto://? can you please help me out in changing the DB from SQLite to Hive/MySql? Thanks, Ram
... View more
05-04-2017
09:45 AM
Hi, How do we connect to Hive using presto://? can you please help me out in changing the DB from SQLite to Hive/MySql? Thanks, Ram
... View more
05-04-2017
07:41 AM
Thanks Rajesh for the reply, this is on AWS EC2 instance installed with HDP 2.5.3 ! Let me know if you know any workaround for the same ! Cheers, Ram
... View more
05-03-2017
05:19 AM
Please configure full hostname as per ubuntu, add the ip-addresses & hostnames in hosts list file. Later try registering in Ambari.
... View more
05-02-2017
06:30 AM
Hi, Have you configured FQDN or are you using Private DNS provided by AWS? In case if you are using ip or Private DNS, please configure FQDN and add all the ip address and FQDN in /etc/hosts in each node! Cheers, Ram
... View more
04-27-2017
11:43 AM
References worked for me by replacing /dev/random to /dev/../dev/urandom in java.security file !
... View more
04-27-2017
11:24 AM
Hello @Sami Ahmad, You will not have access to internet but will have access to docs.hortonworks.com ! Cheers, Ram
... View more
04-27-2017
11:17 AM
Hi Midhun, You've also created one more similar thread ! Delete either of the question, it'd be good for people who are following your question or answer !
... View more
04-27-2017
11:12 AM
Thanks Sanjeev,
we have Symlinked hdp/ under usr directory to /opt/usr/hdp earlier successfully !! We are going good now ! Cheers, Ram
... View more
04-24-2017
01:21 PM
Hi @Varun R From the above logs, I can see you are running the query from Ambari Views? If that is correct then you have to tweak the below changes : Services > HDFS > Configs > Advanced core-site hadoop.proxyuser.ambari.groups=*
hadoop.proxyuser.ambari.hosts=* Since you're using admin user, apply the below settings: su - hdfs
hadoop fs -mkdir /user/admin
hadoop fs -chown admin:hadoop /user/admin Cheers, Ram
... View more
04-24-2017
12:43 PM
@NatcWaro: Incase if Forward and Reverse DNS is not configured, you've to follow above process which you have mentioned about listing of ip and hostnames in /etc/hosts! Let me know if that works or not?
... View more
04-24-2017
07:10 AM
Were you able to check Forward and Reverse DNS mapping? If DNS isn't configured, add the list of ip-addresses with hostnames in /etc/hosts !
... View more
04-23-2017
01:51 PM
Hi @Natc Waro, Check if you have configured ssh properly, ssh localhost or your <FQDN>
if it does not work then
cat id_rsa.pub >> authorized_keys
ssh localhost
Can you please check the permissions of ~/.ssh and authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys Also let me know if you are providing FQDN or IP addresses during installation process ? Cheers, Ram
... View more
04-23-2017
01:28 PM
hi @Mathi Murugan, Hope this article helps you, which also helped me tuning up our cluster ! https://community.hortonworks.com/content/kbentry/22419/hive-on-tez-performance-tuning-determining-reducer.html Cheers, Ram
... View more