Q&A

I am getting a server action failed error when ambari tries to create keytabs. I have no idea what is causing this error. I am using KDC admin to install the service. Please help. @Geoffrey Shelton Okot

When running ./nifi.sh status command occasionally the response is that nifi is running but not responding to ping requests.   Yet when I open cmd and ping the node I get a normal response and the response time is not out of line with the other nodes.   Why is that?

Good morning everybody, I have the following JSON return:{"RegistrosRecebidos" : 1,"RegistrosImportados" : 0,"Erros" : [ "Erro registro 0: The statement has been terminated.\r\nString or binary data would be truncated in table 'Ober_HML.dbo.DB_WS_IMP_CLIENTE', column 'NOME_FANTASIA'. Truncated value: 'ADLER PELZER PERNAMBUCO IND. E COMERCIO '." ]} By default from our supplier, it comes with some formatting. That in Nifi gives the following error: Follow my flow:  How can I do to solve this problem in Nifi? I am trying to use ReplaceText for this but I have not been successful. Thanks.

I have identified that some of my Ranger logs are coming in in gz format (but without the extension). Is there a way of extracting the log data within the Nifi flow?

I am trying to collect data with NiFi using ListSFTP+FetchSFTP. But in the configuration ListSFTP works correctly but I have problems with the configuration of FetchSFTP. I tried with ${path}/${filename} still the problem persists,I also used only the data path, but that doesn't work either . the problem also I have a folder of several files.

Hello everebody! Pls i need to your advise. I need to allows a number of flow-files together into a single flow-file with use general key.For example, i have two flow-files with next attributes:manager_name = John Dowcostumer_info = Matt Damoncostumer_location = LA and manager_name = John Dowcostumer_info = Tony Starkcostumer_location = LA Next, I need to recieve single flow-file with count result of the flow-files with general key = John DowFinally single flow-file whith attributes:manager_name = John Dowtotal = 2 How i can do it in nifi? 

How do I upload multiple nifi templates at once?Nifi only allows to upload one single template/xml file at a time. How do I upload all the templates in  bulk?

I am new to nifi and I want nifi logs to roll daily with no retention policyIs this configuration correct?Another question was do we need to restart nifi after updating logback.xml file?   <appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"><!---<file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app.log</file>--><rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"><fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app_%d.log</fileNamePattern><maxFileSize>1GB</maxFileSize><!-- keep 30 log files worth of history --><maxHistory>1</maxHistory></rollingPolicy><immediateFlush>true</immediateFlush><encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"><pattern>%date %level [%thread] %logger{40} %msg%n</pattern></encoder></appender>      &nbs

Hi Guys, I am trying to migrate an ORC table from one cluster to another. The export and import does not throw any error and I could see folders and files getting created. But the imported table does not contain any data. Table is getting created with all the correct columns. The same works properly for table stored as textfile. Hive version is 3.1. Please provide your inputs.  Regards,Magudeswaran R.

Recently, MapReduce job sometimes failed, the details as below: after check map tasks , the log like below: Log Type: syslogLog Upload Time: Sun Apr 25 13:54:17 +0800 2021Log Length: 55072021-04-25 13:51:01,806 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties2021-04-25 13:51:01,893 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).2021-04-25 13:51:01,893 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started2021-04-25 13:51:01,895 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:2021-04-25 13:51:01,895 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1618548626214_99981, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@732c2a62)2021-04-25 13:51:02,182 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.

I have installed NiFi and MiNiFi on a remote unix server. When I run the command: tail -f nifi-app.logI get all repositories: content_repository, database_repository, flowfile_repository, provenance_repository for NiFi.quamd I run the same command for MiNiFi: tail -f minifi-app.log, get three repositories: content_repository, flowfile_repository, provenance_repository.  the installation was successful. I run the command: http://ip:8080. I can't get the NiFi interface, I'm told that this page is not accessible.I do the same thing for MiNiFi and I don't get  interface.this page is not accessible. i need help    

Hi may I know what version of Python does the new 6.3.3 version of Cloudera supported?

Hi,I have configured Ranger with AD for user and group sync but not able to see the users in the Ranger. Ldapsearch  is working fine and able to retrieve the users from group hdpadmin. Please find below snap for the user configuration.Common ConfigUser ConfigGroup Config  

Ranger Installation fails as ranger fileset is missing in the repo.I dont see any ranger-admin package in the repositoryhttps:/archive.cloudera.com/p/HDP/3.x/3.1.4.0/centos7-ppc/ranger/ raise RuntimeError(message) RuntimeError: Failed to execute command '/usr/bin/yum -y install ranger_3_1_4_0_315-admin', exited with code '1', message: 'https://archive.cloudera.com/p/HDP/3.x/3.1.4.0/centos7-ppc/ranger/ranger_3_1_4_0_315-admin-1.2.0.3.1.4.0-315.ppc64le.rpm: [Errno 14] Error 404 - The requested URL returned error: 404 Not Found Trying other mirror. To address this issue please refer to the below knowledge base article

Hi , I am trying to pull impala running query with status /services/Impala/impalaQueries#'?filter=(query_state=RUNNING) but it also displaying all Finished query detials. I need to pull information of Only RUNNING query which duration exceed with certain time period. Can some one help me. ThanksAt

--driver-java-options %%JAVA_OPTIONS%% --driver-memory 6g \--master dse://?workpool=default \--deploy-mode cluster \--supervise \--conf spark.executor.extraJavaOptions=%%EXECUTOR_JAVA_OPTIONS%% --total-executor-cores 6 --executor-memory 6g --num-executors 6 \--conf spark.driver.extraClassPath=${CLASSPATH} --conf spark.executor.extraClassPath=${CLASSPATH} \%%ADDITIONAL_SPARK_SUBMIT_CONFIGS%% \using the above configuration.I am trying to consolidate driver and worker node logs in console or single file and new to spark application.Can you please provide your adivse

Hi Team, Can someone please help me to connect hive through datagrip. Regards.Raul

I have a question about regionservers that go to bad health when I am writing data from Spark.My question is: how do I know (in what logs to look exactly) what the cause is of the bad health?BACKGROUND:My spark job processes a month of some data and writes to hbase. This runs 1/month. For most months there are no problems.For some months (probably with slightly higher traffic), I notice the regionservers go into bad health. The master notices this and when the server goes down, it moves all regions to another regionserver, and then it becomes healthy again. But as my writing is going on, the same happens to other regionservers and eventually my spark job fails.I am confident my error in write is not due to corrupt data (like impossible UTC time or so), since sometimes this happens and then I clearly see in my spark logs "caused by value out of bounds". Now I see21/03/17 15:54:22 INFO client.AsyncRequestFutureImpl: id=5, table=sometable, attempt=10/11, failureCount=2048ops, last excepti

I'm sending data from MiNiFi to the NiFi Cluster. Is there any way to keep MiNiFi backpressure data in the disk rather than the memory? I've also disabled the Swap feature for increasing performance. I'm worried about consuming lots of the memory by MiNiFi. Nifi version: 1.8.0MiNiFi version: 0.5.0 java

Can any one suggest ranger policy migration from one cluster to another cluster between two kerberised clusters. Through export and import we can do migrate ranger policies but can any one explain detailed steps involved in this export and import process.Thanks and Advance 

Hi,  I've been using cloudera manager for the last one year with running of 20+ nodes. Recently i started to see heap memory size issue in Service monitor roles. I've increased from 3 to 4, then 4 to 5, and then 5 to 6 GB.But still i sometime get the service monitor crashed and restarted. During the time, the entire dashboard seems bad. What i need to do here to fix the issue?.  Logs are2021-04-26 16:10:34,938 WARN com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 20583ms: GC pool 'G1 Young Generation' had collection(s): count=2 time=182ms, GC pool 'G1 Old Generation' had collection(s): count=1 time=20877ms2021-04-26 16:11:34,862 WARN com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 19870ms: GC pool 'G1 Young Generation' had collection(s): count=2 time=131m

Hi everyone, I just installed virtualbox ad hortonworks sanbox.  I want to copy files between my local host and Sandbox using SCP.  I used this command and getting errors. [root@sandbox-hdp ~]# scp -P 2222 root@sandbox-hdp.hortonworks.com:/root/saketh/sample1.txt C:\Users\satvi\Desktop\test\sample1.txt ssh: connect to host sandbox-hdp.hortonworks.com port 2222: Connection refused root@sandbox-hdp ~]# scp -P 4200 root@sandbox-hdp.hortonworks.com:/root/saketh/sample1.txt C:\Users\satvi\Desktop\sample1.txt ssh_exchange_identification: Connection closed by remote host  ThanksSaketh 

I have 5 datanode where impala-demons was installed and in all it shows me this error:**************************************************************************************log4j:ERROR Could not instantiate appender named "redactorForRootLogger".E0426 13:25:49.818779 12933 RequestPoolService.java:257] Unable to stop AllocationFileLoaderService after failed start.Java exception follows:java.lang.IllegalStateExceptionat com.google.common.base.Preconditions.checkState(Preconditions.java:492)at org.apache.impala.util.FileWatchService.stop(FileWatchService.java:136)at org.apache.impala.util.RequestPoolService.stopInternal(RequestPoolService.java:281)at org.apache.impala.util.RequestPoolService.start(RequestPoolService.java:255)F0426 13:25:49.820109 12933 request-pool-service.cc:148] RuntimeException: org.apache.impala.yarn.server.resourcemanager.schedule$CAUSED BY: AllocationConfigurationException: Could get past last queue placement rule without assigning. Impalad exiting.******************

Hi all,I've run few oozie workflow through Hue. But once the jobs finished, I cannot see the logs anymore. I'm wondering whether this is normal behaviour or I can config something on yarn to make the logs available even after job finishes.  yarn.log-aggregation-enable is set to enable on the cluster Using this command is fine. yarn logs -applicationId <Application ID>. Only cannot see in the hue web ui after the job finished  Thanks