Solved

Hi, In my process I'm triggering some emails at certain points. My problem is that the PutEmail processor just keep dying frequently (even two or three times per day)!!! Nifi Version 1.14.0  When processor dyes I have to terminate and restart it. This is the dump of the last crash:"Timer-Driven Process Thread-1" Id=68 RUNNABLE (in native code)at java.base@11.0.2/java.net.SocketInputStream.socketRead0(Native Method)at java.base@11.0.2/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)at java.base@11.0.2/java.net.SocketInputStream.read(SocketInputStream.java:168)at java.base@11.0.2/java.net.SocketInputStream.read(SocketInputStream.java:140)at com.sun.mail.util.TraceInputStream.read(TraceInputStream.java:102)at java.base@11.0.2/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)at java.base@11.0.2/java.io.BufferedInputStream.read(BufferedInputStream.java:271)- waiting on java.io.BufferedInputStream@40399b64at com.sun.mail.util.LineInputStream.

Hi,  I see the in Cloudera Manager in Kafka Status the diagram for "Total Bytes Received Across Kafka Brokers". What does it mean exactly? Is the pure input without replication, or is replication counted on it and the pure input would be only e.g. 1/3 for replication factor 3   Best regards Jaro

I have the Flow File which has Duplicate Column i want to pick the Column threw index Number,is it possible to do with Query Record or any Processor Note: Column Will change with every new Flow File coming 

Hello, I have my NiFi application running on 1 server. And I need to connect to users SFTP server and need to execute few commands like cd, rm, rename etc on remote server. And to connect to that server I need to fulfill 2 step authentication (username/password and key authentication) So if i ssh and provide key from shell script which will be mounted in docker. For that i will have to add the key needs to be installed in all nodes and then needs to be added to users sftp server. It does not seem to be best solution I there any other way where  I can use NiFi processor to execute these commands to remote sftp server which needs to be connected through 2 step authentication Any advise will be very helpful Thank you.Shalaka

Hello,We were getting ‘DatabaseError: ORA-01017: invalid username/password; logon denied’ post our db upgrade, to fix that we have reset the hue password, which worked and that issue got fixed.Unfortunately one of our guys deleted a couple of logs from /var/log/hue. We have recreated runcpserver.log with the necessary permission, and restart Hue server, yet we fail to see the file being populated (file is still empty). Request assistance / guidance in fixing this issue. ThanksWert

Hello,How do I check the location where Kerberos is writing logs?I checked the location which is mentioned in krb5.conf (default = FILE:/var/log/krb5libs.logkdc = FILE:/var/log/krb5kdc.logadmin_server = FILE:/var/log/kadmind.log) however the log files mentioned in this location are empty. Am I checking an incorrect location? Secondly logs on Hue Server & KT renewer is not getting updated / current. Any help/guidance is appreciated.

Hello, I recently discovered the CM Rest API and how I can restart services and obtain some information just using the Linux console and not using the CM UI. I'm looking for some examples to delete and create the Impala Catalog Server and State Store (in a diferent node) by only using this feature.  I did not find any information about it in the internet, just some poor youtube videos to do simple actions. I'm using CDP 7.1.6 CM 7.3.1 API v43 This is the Cloudera documentation I could get from my installation http://my-CMhost.com:7180/static/apidocs/resource_RolesResource.html   I would really appreciate if anyone used this and could share some examples 🙂  

I have a use case where I want to Replace Value From My AttributeLike I have a Attribute Like :- 1. Header: A,B,C,D,E2. Index: 3Now I Want to replace the index Attribute Value from My header Attribute value with same Position like if I have 3 in my index attribute so it will replace the third aliment from my header attribute or if it's 5 so it should change the 5th one from header and make a new Attribute something like thisHeader_2 :- A,B,3,D,EBased on index attribute number it's needs to replaceThanks In Advance 

We upgraded from 7.1.6 to 7.1.7 GA and had similar issue and now we are trying to upgrade to hotfix and facing same issue and a bit unclear what fixed the first issue ! We copied the following files to /opt/cloudera/parcel-repo 8549720983 Jan 29 00:10 CDH-7.1.7-1.cdh7.1.7.p74.21057765-el7.parcel40 Jan 30 00:44 CDH-7.1.7-1.cdh7.1.7.p74.21057765-el7.parcel.sha164 Jan 30 00:44 CDH-7.1.7-1.cdh7.1.7.p74.21057765-el7.parcel.sha256 When we check parcels in CM we see the following Error CDH 77.1.7-1.cdh7.1.7.p74.21057765Unavailable Error for parcel CDH-7.1.7-1.cdh7.1.7.p74.21057765-el7.parcel : No valid hash file found.We tried copying the SHA files again ...compared bits with another download and still same error . Anything we are missing ? Is there supposed to be a "torrent" file ? Couldnt see it in archive .   

Hi ,After accessing the hbase master ui link in production , under hbck we see there are 967 regions under ophan regions in filesystem. Can someone please help us as to how can we clear them. Thanks.@gsharma 

Can someone suggest regarding nifi 1.15.3 setup in plain Linux. As I am able to setup 1.13 in Linux but not latest version As seems like some pre-request is needed on this version respect of security/keystore. If already Discussed share me the link.

hive has --database option to select database.--database <databasename> Specify the database to use What is the beeline equivalent of the same?

I am following the instructions to run hive tpch https://github.com/hortonworks/hive-testbench.git I am using Cloudera Enterprise Trial 7.3.1 I solved the initial error(https://community.cloudera.com/t5/Support-Questions/Error-while-running-hive-tpch-Exception-in-thread-quot-main/m-p/336879/thread-id/232418/highlight/false#M232465 ) and tried to rerun the tpch-setup.sh. I am running into following error. + echo 'TPC-H text data generation complete.'TPC-H text data generation complete.+ echo 'Loading text data into external tables.'Loading text data into external tables.+ runcommand 'beeline -u jdbc:hive2://techmach.com:10000/default -i settings/load-flat.sql -f ddl-tpch/bin_flat/alltables.sql -d DB=tpch_text_2 -d LOCATION=/tmp/tpch-generate/2'+ '[' X '!=' X ']'+ beeline -u jdbc:hive2://techmach.com:10000/default -i settings/load-flat.sql -f ddl-tpch/bin_flat/alltables.sql -d DB=tpch_text_2 -d LOCATION=/tmp/tpch-generate/2+ i=1+ total=8+ test 2 -le 1000+ SC

What is the beeline equivalent of hive cli -d option?-d,–define <key=value>Variable substitution to apply to Hive commands.

HI,  I am having an issue in LDAP integration with NIFI version 1.15.3.I can able to login the admin screen, i added policy for "modify component". but when i drag and drop the process or process group into the screen. it appear in read only ( with doted line). in the log it is saying"Unable to find access policy for write on /process-groups/2740a80a-017f-1000-a4b3-70d6580cdb38. Returning Not Found response"  My configuration belowlogin-identity-providers.xml   <provider> <identifier>ldap-provider</identifier> <class>org.apache.nifi.ldap.LdapProvider</class> <property name="Authentication Strategy">SIMPLE</property> <property name="Manager DN">cn=admin,dc=ae,dc=test,dc=com</property> <property name="Manager Password">a@psswd</property> <property name="TLS - Keystore"></property> <property name="TLS - Keystore Password"></p

I am following the instructions to run hive tpch https://github.com/hortonworks/hive-testbench.git  I am using Cloudera Enterprise Trial 7.3.1 I am running into following error. [root@test_mach hive-testbench]# ./tpch-setup.sh 3+ '[' X3 = X ']'+ '[' X = X ']'+ DIR=/tmp/tpch-generate+ '[' 3 -eq 1 ']'+ hdfs dfs -mkdir -p /tmp/tpch-generate+ hdfs dfs -ls /tmp/tpch-generate/3/lineitemls: `/tmp/tpch-generate/3/lineitem': No such file or directory+ '[' 1 -ne 0 ']'+ echo 'Generating data at scale factor 3.'Generating data at scale factor 3.+ cd tpch-gen+ hadoop jar target/tpch-gen-1.0-SNAPSHOT.jar -d /tmp/tpch-generate/3/ -s 3WARNING: Use "yarn jar" to launch YARN applications.Exception in thread "main" java.lang.IllegalAccessError: class org.apache.hadoop.hdfs.web.HftpFileSystem cannot access its superinterface org.apache.hadoop.hdfs.web.TokenAspect$TokenManagementDelegatorat java.lang.ClassLoader.defineClass1(Native Method)at java.lang.ClassLoader.defineClass(ClassL

While looking into the Spark unsupported features in CDP private cloud, i can see below statement "Using the JDBC Datasource API to access Hive or Impala is not supported"Can you please explain this? If JDBC datasource is not supported, can't we access hive? Is this true?Is there any fix for this.Appreciate your quick response. 

Hi guys, i am trying some hive partitions but i am getting the following errors. kindly help figure out the problem. 0: jdbc:hive2://svdt5hadoopms02.bigdatadev.sa> ALTER TABLE customer_transaction_extract DROP IF EXISTS PARTITION (date='date=2019-11-13') PURGE;Error: Error while compiling statement: FAILED: ParseException line 1:67 cannot recognize input near 'date' '=' ''date=2019-11-13'' in drop partition statement (state=42000,code=40000)0: jdbc:hive2://svdt5hadoopms02.bigdatadev.sa> ALTER TABLE customer_transaction_extract DROP IF EXISTS PARTITION (date='date="2019-11-13"') PURGE;Error: Error while compiling statement: FAILED: ParseException line 1:67 cannot recognize input near 'date' '=' ''date="2019-11-13"'' in drop partition statement (state=42000,code=40000)0: jdbc:hive2://svdt5hadoopms02.bigdatadev.sa> ALTER TABLE customer_transaction_extract DROP IF EXISTS PARTITION (date=`2019-11-13`) PURGE;Error: Error while compiling statement: FAILED: ParseException line 1:

Hi everyone! I hope all is well. I have this json  {    "Data": {        "ID": "09878666",        "DATE": "2022-01-01",        "ARRAY_ONE": [            {                "NAME": "test_1",                "details": [                    {                        "address": "123123"                    }                ]            },            {                "NAME": "test_2",                "details": [               

Dear All, On one particular cluster we have spark 2.3 running(CDH 5.15.1). can it be possible to only upgrade spark to 2.4 ? by remaining on the same CDH version(5.15.1). will it compatible with existing service ?. kindly suggest  

Hello!I'm using CM api v19I'm trying to get only critical alerts which I'm looking at right now (RED ones).Tried https://cloudera.github.io/cm_api/apidocs/v19/path__events.html  /api/v19/events?query=category==HEALTH_EVENT;severity=CRITICAL   The output consists of too many not relevant events. What should I do to get only critical events displayed on the dashboard?Thank you.

Hello,a question regarding application design in NiFi. We have a REST resource which provides data changes since a unix timestamp, which is passed inside a json structure to be sent as http POST.The major question is how and where to keep the state of the last unix timestamp and how this state is accessible to operation.We should be able to reset the timestamp to any value to repeat calls. Also we should be able set increment, which es added to the timestmap in the next call. Similary, we have a related issue ina use case, which access a database. We generate a query in GenerateTableFetch and set the property "Maximum-value Columns".Here the state is maintened by the processor, but it pretty much unclear how to set the state to specific stating value. To clear the state is obviously not appropriate. Best regardsJaro

Hi, Company I'm working for is paying subscription to Cloudera and I need to use Cloudera support on their behalf.  When I created my cloudera account with company mail I don't see any subscription or asset. My colleague did the same and his account was paired automatically.  What's proper way how to link my account to company I'm working for ? Can I trigger this manually ?  Thank you for any help. 

Hello, Apache druid cannot be added as a separate service in CDP but I can see its binaries under the CDH-7.1.7 parcel folder. Is Druid available and supported in CDP? 

Hello all, I am trying to connect to Kafka from my Flink flow. I am using Flink version 1.14.3 and Kafka connector version: flink-connector-kafka-0.11_2.11:jar:1.11.6 (latest version in Maven repo). I am using FlinkKafkaConsumer011 in my code to create Kafka consumer to consume my kafka topics. However, when running Flink and deploying my flow, I see the below error thrown in logs: java.lang.NoSuchMethodError: org.apache.flink.api.common.functions.RuntimeContext.getMetricGroup()Lorg/apache/flink/metrics/MetricGroup;at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.run(FlinkKafkaConsumerBase.java:762)at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:67)at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:323) Appreciate any help on this. - Prabu.