Solved

Hello!I installed Nifi, Nifi toolkit.I have an error "Unknown user with identity 'CN=admin, OU=people, DC=nifi, DC=com'. Contact the system administrator." or "Unable to locate initial admin CN=admin,OU=people,DC=nifi,DC=com to seed policies".Ldap config -> ldap.pngnifi.properties:# security properties #nifi.sensitive.props.key=nifi.sensitive.props.key.protected=nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSLnifi.sensitive.props.provider=BCnifi.sensitive.props.additional.keys=nifi.security.keystore=./conf/keystore.jksnifi.security.keystoreType=jksnifi.security.keystorePasswd=C952nQSnvjyqA38F4Z5WHB80qzzCGPjB8qCrfIUTXWsnifi.security.keyPasswd=C952nQSnvjyqA38F4Z5WHB80qzzCGPjB8qCrfIUTXWsnifi.security.truststore=./conf/truststore.jksnifi.security.truststoreType=jksnifi.security.truststorePasswd=0f9v05dN9t15lVRklQ3B6jGphb5fbtfYuEKjefQyb7Anifi.security.needClientAuth=yesnifi.security.user.authorizer=managed-authorizernifi.security.user.login.identity.provider=ldap-provide

Hi,Can someone please tell me what this error means when trying to connect to hiveserver2 via beeline and how to solve this? [root@test1 ~]# beelineBeeline version 1.2.1000.2.6.1.0-129 by Apache Hivebeeline> !connect jdbc:hive2://test.domain.com:10000/test;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2Connecting to jdbc:hive2://test.domain.com:10000/test;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2Enter username for jdbc:hive2://test.com:10000/test;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: nameEnter password for jdbc:hive2://test.domain.com:10000/test;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2: ******************21/03/01 10:16:28 [main-SendThread(test.com:10000)]: WARN zookeeper.ClientCnxn: Session 0x0 for server test.domain.com/10.49.0.42:10000, unexpected error, closing socket connection and attempting reconnectjava.io.IOException: Packet len67108864 is out of range!at org.apache.zookeeper.ClientCnxnSocke

Hello, I have a 18.04 bionic cluster using the Cloudera Manager 7.1.3 and CDH 7.1.3 parcels.   I need to install CDS3 (Spark 3.0.0) on my cluster but I can't find any parcels for the bionic version, only xenial, wheezy and trusty.     Does it exist? Can you also publish it for bionic?   Thank you !!

Hi,I've enabled HA successfully. My directory for the three journal node is not the same accross that 3 journal node. There are alert of required JournalNode Default Group box. Is it safe to I put anything here? as my journal node location is not the same? It is only for managing that 3 journal node loaction right? Thanks 

Hi,I was wondering if there is anyway to upgrade our HIVE from an existing HDP cluster?So we have deployed previously a HDP cluster, but the hortonworks public url is currently not accessible, so I was wondering if there was any other way to upgrade our HIVE. Can someone help please point me in the right direction? Thanks,

Hello. I'm a data analyst who was unfortunately tasked to fix an issue with hive. I triedRestarting all HDFS Components - successRestarting all Hive Components - failed.Restart HiveServer2 has failed 1 / 4 nodes have failed. The node that failed is not the master nodeOther components was started succesfully.

Hi,I have json flowfile,  [{"prediction":"Test2"},{"prediction":"Test2"}]. I am want extract the 1st part of the array ie prediction":"Test2  my destination is flowfile-attribute using EvaluateJsonPath . I have added a custom attribute mlresult --->$.prediction[0], to extract the "Test2" into mlresult, but I am getting empty string  for the mlresults for the flow file attribute section. flow file : [{"prediction":"Test2"},{"prediction":"Test2"}]EvaluateJsonPath configuration:     How to get the value(only one)  Test2 to mlresult? Thanks--Murali   

Hi,Can someone please help me and let me know what I did wrong or am I missing something? [root@sandbox ~]# yarn log -applicationId application_1614345689259_0002Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=trueJava HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0Error: Could not find or load main class log[root@sandbox~]#  Thanks,

sqoop import --connect "jdbc:jtds:sqlserver://BJHBLSNR:1496;useNTLMv2=true;domain=BST;databaseName=HTG" --username DM -P  --table PROVIDER_STG --split-by PROV_CODE_BK --target-dir /test/dd -m 1    ERROR tool.ImportTool: Import failed: org.apache.hadoop.security.AccessControlException: Permission denied: user=cloudera_user, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x  

Hi, I tried looking in to the community but could not get proper answer for this question. How can I change the block size for the existing files in HDFS? I want to increase the block size. I see the solution as distcp and I understood that we have to use distcp to move the files, folders and subfolders to a new temporary location with new block size and then remove the files, folders, etc for which block size has to be increased and copy the files from temporary location back to original location. The above methodology might have side effects such as overhead of HDFS by adding duplicate copies of the files and change in permissions while copying files from temporary location and etc. Is they any way which is efficient enough to replace the existing files with the same name and same privileges but with increased block size? Thanks to all for your time on this question.

Zookeeper 3-node cluster in docker (DockerHub zookeeper:3.6.1). Individual services deployed as stack. docker-compose.yml:zookeeper1: image: zookeeper:3.6.1 hostname: "zookeeper1" environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=0.0.0.0:2888:3888;2181 server.2=zookeeper2:2888:3888;2181 server.3=zookeeper3:2888:3888;2181 ALLOW_ANONYMOUS_LOGIN: "yes" ZOO_4LW_COMMANDS_WHITELIST: "*" ZOO_STANDALONE_ENABLED: "false" ZOO_INIT_LIMIT: 20 ZOO_SYNC_LIMIT: 10 networks: - kafka-net volumes: - zookeeper-data:/data/ - zookeeper-datalog:/datalog/ zookeeper2: image: zookeeper:3.6.1 hostname: "zookeeper2" environment: ZOO_MY_ID: 2 ZOO_SERVERS: server.1=zookeeper1:2888:3888;2181 server.2=0.0.0.0:2888:3888;2181 server.3=zookeeper3:2888:3888;2181 ALLOW_ANONYMOUS_LOGIN: "yes" ZOO_4LW_COMMANDS_WHITELIST: "*" ZOO_STANDALONE_ENABLED: "false" ZOO_INIT_LIMIT: 20 ZOO_SYNC_LIMIT: 10 networks:

Hello, I'm using a HDP 3.1 Kerberized HDP and I've a problem when using "regenerating keytabs" button : I've 16 regenerated keytabs and 5 keytabs not regenerated. The 5 keytabs that are not updated :- ambari-infra-sol.service.keytab- hive.service.keytab- smokeuser.headless.keytab- spnego.service.keytab- yarn.service.keytab I have no error in the logs regarding problem with keytab generation or keytab deployment. I can restart all the services and everything is working fine but how can I have ALL keytabs regenerated ? thanks for your help,    

Hi,I recently installed ranger by building it from source.We also have ambari installed and was wondering if anyone can help redirect me to somewhere that explains how to manage Ranger using Ambari. Thanks,

I need your advice, I need to insert data into salesforce but I have a lookup to do to verify the existence of the data before adding it.Me, I had considered creating a tempo database (PosgresSql) in order to put the contents of the salesforce object being the object of lookup to verify first and then to make the join to update the contents. in salesforce.1- NIFI to PosgresSql2- Salesforce to PosgreSql3- Join data to SalesforceWhat do you think? what do you think is the best method?

Hello,I am trying to download a nifi template which has around 260 processors. Whenever I try to download the template in chrome browser, the downloading stops after 320kb and throws me an error which says "Failed - Network Error". Since there are around 260 processors, it is understandable that the template size is considerably big however I can download other smaller templates from the same Nifi instance. I even tried downloading the template with other browsers but the error is same.Is there any configurations which needs to be done from the server side? Any insights are appreciated@mburgess @bbende @TimothySpann 

Hi all,we are executing pyspark and spark-submit to kerberized CDH 5.15v from remote airflow docker container not managed by CDH CM node, e.g. airflow container is not in CDH env. Versions of hive, spark and java are the same as on CDH. There is a valid kerberos ticket before executing spark-submit or pyspark.Python script: from pyspark.sql import SparkSession, functions as Fspark = SparkSession.builder.enableHiveSupport().appName('appName').getOrCreate()sa_df=spark.sql("SELECT * FROM lnz_ch.lnz_cfg_codebook") Error is: To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /__ / .__/\_,_/_/ /_/\_\ version 2.3.0 /_/Using Python version 3.6.12 (default, Oct 13 2020 21:45:01)SparkSession available as 'spark'.>>> from pyspark.sql import SparkSession, functions as F>>> spark = SparkSession.builder.enableHiveSupport().app

Hi, I am having trouble getting the data provenance displaying in a clustered environment in Nifi. I have set all the required access policies and still the data provenance is empty. Any suggestions at this point is very much appreciated. 

What is the default character set used within Nifi while extracting from Oracle using executeSQL ?I have a use case of 2 database in 2 different character sets, was just curious, is there any way to identify the incoming character set & then over-ride it based on the target ?

Hi all,I am curious to know if the BDR schedules sync deletes as well? Meaning, if some files are deleted from the source, will the next BDR run delete those files from the destination as well to make the source and destination exact copies?Thanks!

Hi,  I have Cloudera Enterprise Data Hub Edition trial version installed in my environment. So I am getting an message like below when I login into my cloudera manager. You have 3 days left to try Cloudera Enterprise Data Hub Edition.After that, you will need a valid license, or you can revert to using Cloudera Express. Your cluster will remain unaffected.So will I be able to login to my cloudera manager after the license expire?,will this effect my Hadoop cluster?How can I revert to using Cloudera Express? Thanks You!

Hi, we are running a HDP 3.1 production cluster with 2 Master Nodes and 54 Data Nodes;Spark version is 2.3 and yarn is the cluster manager. Each data node has about 250GB RAM and 56 Cores. We are using a combination of Nifi and Spark to set up a Hive DWH as follows:Nifi picks input csv files from source and loads them to HDFS in parquet formatSpark jobs picks these files and loads them to hive managed tablesAnother set of Spark jobs aggregates the data in the hive managed tables to hourly and daily hive managed tables.I noticed that my Spark jobs were very slow at times when writing to hive. I ran a test where i ran the same spark-submit job several times and it took between 10-13 minutes (fast) to 35-40 minutes (slow) to run the same job with the exact same parameters. My spark submit job: sudo /usr/hdp/3.1.0.0-78/spark2/bin/spark-submit --files /etc/hive/3.1.0.0-78/0/hive-site.xml,/etc/hadoop/3.1.0.0-78/0/core-site.xml,/etc/hadoop/3.1.0.0-78/0/hdfs-site.xml --driver-class-p

Hi,          I've pretty much read through all the articles and questions about this, but i'm still having issues. the ldap user is definitely there and i can query through ldap port:389.  I've set up the HUE ldap to authenticate against my openldap server, it works great. But i couldn't get the impala ldap setup to work. I'm using CDH 6.3.    I configured impala ldap settings through CM gui, services restarted okay after the config   Advanced Configuratoin snippet(safety valve) :--ldap_passwords_in_clear_ok=true Enable LDAP Authentication: checked LDAP URL: ldap://myldapFQDN:389 LDAP BaseDN: ou=users,dc=ldap,dc=xxx,dc=com     Error when using cloudera impala jdbc driver to connect, it worked with AuthMech=0 before I enabled LDAP drv <- RJDBC::JDBC("com.cloudera.impala.jdbc4.Driver", "/opt/jars/ImpalaJDBC4.jar")con <- dbConnect(drv, "jdbc:impala://impalaDaemonIP:21050;AuthMech=3;UID=myuser;PWD=mypassword") Error in .jca

Hello allI have task and not find solutions, have to comparing configuration between cluster (CDH 6.1.1) and everything i found shows using Apache Ambari but i dont have this tool and must using different way. Do you know this tool/script??

Hello All,I need your support , I have Cloudera version 5.15 connected to Data source Oracle 11.2.0.3 and Database Team want to Upgrade it to Oracle 19.3C, can someone tell me if My cloudera version will still compatible with the new Oracle DB version or not ? I searched lot in google but didn't find what I want. Thank you in Advance

Hello,I had my CCA122 exam during which I was not able to solve two questions due to repo was not accessible. I tried manually and even from browser, I was getting not found and 403 for HDP and ambari repo. Same was informed and noted by the proctor.As per Cloudera exam policy, if any technical issue is encountered my exam should be reset but they have marked me failed.What could I do in such situation? As per my understanding, Repo should be managed by Cloudera during exam