I want latest file from hdfs dir.I am using below programm,but it get me all files into dataframe.I want to filter latest file and read only latest file.I am using scala val  fname=spark.read.csv("hdfs://ndwns001.ndw.leidos.com/automation_test/oozie/output/Bigdata_Counts/") ... View more

@Joe_Jim  By default, the broker binds to localhost. Can you share your server.properties and logs? I am of the opinion you need to update the server.properties listeners=PLAINTEXT://<IP:_TO_CHANGE>:9092 ... View more

Hi,     The issue has been resolved. I crossed checked my command and I was not passing the client certificate due to that I was getting this error.   Thanks to all for your support. ... View more

@SushantRao where is the source code for Cloudera Manager and all the other CDH7 components ? Is there documentation available to locate and build the packages ? ... View more

Thanks for your response. Sorry for my the late response, but I wanted to pass this along in case someone runs into this issue. So I did some more testing and what I found is that I had an incompatible postgresql-jdbc driver. So I was upgrading from 9.6 to 10.x and it appears the jdbc driver I was using was pretty old and incompatible with version 10.x. Once I upgraded the postgresql-jdbc driver everything worked just fine. Here is what I did to resolve my issue:   wget https://jdbc.postgresql.org/download/postgresql-42.2.9.jar mv /usr/share/java/postgresql-jdbc.jar /usr/share/java/postgresql-jdbc.jar.old  (backing up the old driver) cp ~/postgresql-42.2.9.jar /usr/share/java/postgresql-jdbc.jar ambari-server setup --jdbc-db=postgres --jdbc-driver=/usr/share/java/postgresql-jdbc.jar     ... View more

@WilsonLozano,   As this thread is older and was marked 'Solved back in August of 2019 you would have a better chance of receiving a resolution by starting a new thread. This will also provide the opportunity to provide details specific to your environment, version of CDH, etc. that could aid others in providing a more accurate answer to your question.      ... View more

@saivenkatg55    That could be a memory issue on your cluster. Can you share the below config set spark.executor.memory set yarn.nodemanager.resource.memory-mb set yarn.scheduler.maximum-allocation-mb Here are some links to help How to calculate node and executors memory in Apache Spark after adjusting that share the new output   ... View more

@asmarz, As this is an older thread which was previously marked 'Solved', you would have a better chance of receiving a resolution by starting a new thread. This will also provide the opportunity to provide details specific to your environment (for example, what happened once you added the affected user accounts with "normal" command add user on all nodes)  that could aid others in providing a more relevant, accurate answer to your question.     ... View more

Dear Jay   what to say   excellent answer , you are really one of the best here  ... View more