Solved

Hello All, We would like to install a cluster CDP 7.1 on our DEV environment composed of 8 nodes :3 Masters3 Slaves1 Edge1 Cloudera-manager What are the best practices for distributing the services (Spark,Hbase,Zookeeper...) on the different nodes of the cluster? Thanks for your feedback.

Hi All, When we Load data into Hive table from HDFS, it deletes the file from source directory(HDFS) is there a way we can keep the file in the source directory and load the data into hive table as well. I used the below query; LOAD DATA INPATH 'source_file_path' OVERWRITE INTO TABLE TABLENAME;

Hello, I'm using Ubuntu 16 and Cloudera 6.2.1. I need to startup a couple other agents only after CDH HDFS has completely started upon OS start, which can take several minutes. I am unsure how to do this. Anyone have any pointers?   Thank you.

Hello All, I could see error messages being displayed every minute on top most corner of Processor groups.The errors being displayed is regarding Processor FETCHSFTP trying to fetch an old file which doesn't exist in source, I don't want that error to be displayed. Could you please let me know what settings to be changed. Thank you in advance

Hello All and @Shu_ashu,@MattWhoI have a CSV file with a time column "Apr 15, 2020 02:02:16.388267000 Paris, Madrid (heure d’été)".... "Apr 15, 2020 02:02:16.986879000 Paris, Madrid (heure d’été)"...etc.Then I to convert the field time using UpdateRecord processor by the following code:${field.value:substring(0,28):toDate("MMM dd,yyyy HH:mm:ss.SSSSSS"):format("yyyy-MM-dd HH:mm:ss.SSSSSS")}But after conversion, the output I get is:nbtimedelta12020-04-15 02:08:44.0002670.022020-04-15 02:18:42.0008790.598612 But the output should be:nbtimedelta1Apr 15, 2020 02:02:16.3882670.02Apr 15, 2020 02:02:16.9868790.598612 So, I always get 000 before the milliseconds. As per the example above, I got 000267 instead of 388267 and 000879 instead of 986879.Why I am getting the 000 instead of the real value? How can I modify me code to resolve this problem? Any help on this issue will be highly appreciated.

How should I install impala-udf-devel package? Is it installed along with another services?

Hi,i read that usualy the result should be ready in few hours, but i am still waiting for.My exam was yestarday on 3 May 2020 at 10:00 a.m CEST, more then 1 day passed. Just to be informed, it is normal?  If not, what i should do?  Thanks in advance. 

Hello,I would like to filter some activities for 15 days, but not the last 15 (for example: from the 1st to the 15th and today is the 28th). how would I filter this in Cloudera Navigator?

Hello Team, Could you please help with the list of tools that are available in the exam and link of the documentation which can be used in the exam. ThanksSarath

I want the json split on the "DATA_MESSAGE", but note that some of them have multiple logEvents and should be included regardless of the number of them.  There are no outer "[ ]" in these flowfiles so $ or $.* doesn't work properly.{    "messageType": "DATA_MESSAGE",    "owner": "540804524916",    "logGroup": "/awg-p-deploy/aws/ec2/syslogs",    "logStream": "540804524916/i-01918034325752bbd/ip-10-170-16-11.xxxxxxx.xxx.org/var/log/audit/audit.log ",    "subscriptionFilters": ["awg-p-deploy-kinesis-destination-subscription-SubscriptionFilter-SOC692HYHJN8"],    "logEvents": [{            "id": "35604816732673909304179019342583609767225770179215556608",            "timestamp": 1596575200322,            "message": "type=SYSCALL ms

Hi everyone,Let say my existing Cloudera manager is managing a 10 node cluster. What happen if my cloudera manager server is down?(my cloudera management service is on diffrerent server and I'm using external database for the setup).  Can I immediately setup a new server, reinstall cloudera manager here, and manage the existing cluster without too much effort on the configuration and setup? I'm worried about the TLS setting, keberos setting,etc. Can it be preserve and work the way it was before the cloudera manager is replaced I believe the data in the HDFS should be safe right?  Thanks

While trying to install HDP 2.6.5 sandbox on windows 10 with wmare, after starting the vm with 4cpu and 10g ram, i can login the ambari but all service can't be started I have tried "restart all" service or restart zookeeper via the account: raj_ops, but no luck.  

I have installed apache metron version 0.7.2 using cloud installation, it created 6 nodes  and everything running fine but when i move to Metron Management UI it asks me for the password. I tried "Metron/Metron", "metron/metron", "admin/admin" etc but still unable to login. I tried metron user guides to find default password for metron management UI & metron alerts  UI but not able to find any document that gives or guides to set default password for metron.Is there anyone who tried this? Can anyone help me for default password for metron management UI & metron alerts  UI.

I have 2 NIFI instance on differents servers with 1 NGINX reverse proxy in front.It should go like this:NGINX Reverse proxy > proxy.lan/nifi1 goes to the NIFI 1 instance on nifi1.lanNGINX Reverse proxy > proxy.lan/nifi2 goes to the NIFI 2 instance on nifi2.lan NGINX reverse proxy runs on 192.168.1.114:80 NIFI runs on 192.168.1.153:8080 My configuration file nginx.conf:server {listen 80server_name 192.168.1.114;location /nifi1 {proxy_set_header X-ProxyScheme "http";proxy_set_header X-ProxyHost "192.168.1.114";proxy_set_header X-ProxyPort "80";proxy_set_header X-ProxyContextPath "/nifi1";proxy_pass http://192.168.1.153:8080;}} This is my NIFI nifi.properties file:nifi.web.proxy.content.path=/nifi1nifi.web.proxy.host=192.168.1.114:80 When accessing http://192.168.1.114/nifi1 , I will get the message "Did you mean /nifi" but the page looks off. It's missing the styling of the page. I get redirected after 5 secs to http://192.168.1.114/nifi1/nifi  an

Hi,I have created a post and I got few replies but suddenly I have my posted marked as spam. I tried to create new post but still I could see all my questions are marking as spam  

Hello everyone,   we setup a Cloudera Environment which inherits a DataHub of type "7.1.0 - Data Engineering: Apache Spark, Apache Hive, Apache Oozie". We managed to connect to HIVE via a JDBC connection from our local machines. But so far we were not able to connect from CML to HIVE via JDBC I use the JayDeBeApi as follows:       conn_hive = jaydebeapi.connect('org.apache.hive.jdbc.HiveDriver', 'jdbc:hive2://dataengineering-master0.......:443/;ssl=1;transportMode=http;httpPath=dataengineering/cdp-proxy-api/hive;AuthMech=3;', \ {'UID': "user_name", 'PWD': "password"}, '/home/cdsw/drivers/HiveJDBC41.jar',)       The error message is       TypeError: Class org.apache.hive.jdbc.HiveDriver is not found       I set the environment variable CLASSPATH to '/home/cdsw/drivers/HiveJDBC41.jar' which is were the jar actually rests. Hence I wanted to check if JAVA_HOME is set correctly and yes, there the env. variable is set to '/us

Dear All,I am encountering issue with LDAP integration.I have completed LDAP (ldap-provider) and Certificate configurations according to documentation.I added IU certificate in NiFi (keystore, truststore etc.) and configured other pieces of the MS AD LDAP integration (authorizers.xml, login-identity-providers.xml and nifi.properties).I logged on NiFi from HTTPS UI with initial admin (admin1) and assigned the policies one of the LDAP users (nifiadmin) which is located on MS AD LDAP.I checked LDAP user (nifiadmin) from NiFi UI it is exist in the NiFi. It seems Ok. I added all screenshots (nifi_policies.jpg) about that.When I try to login initial admin (admin1) there is no error:nifi-user.log:2020-08-13 10:46:43,544 INFO [main] o.a.n.a.FileUserGroupProvider Users/Groups file loaded at Thu Aug 13 10:46:43 MSK 20202020-08-13 10:46:43,684 INFO [main] o.a.n.a.FileAccessPolicyProvider Authorizations file loaded at Thu Aug 13 10:46:43 MSK 20202020-08-13 11:21:28,051 INFO [NiFi Web Server-22] o.

Hello all,   We are seeing the next error in our infra-solr server logs (hortonworks HDP-3.1.4.0, infra-solr 0.1.0).   2020-09-23T16:33:00,258 [qtp2039810346-189141] ERROR [c:audit_logs s:shard2 r:core_node10 x:audit_logs_shard2_replica_n9] org.apache.solr.common.SolrException (SolrException.java:148) - org.apache.solr.common.SolrException: ERROR: [doc=697665cf-bc6b-4ada-a94e-e14ede1ba18c] Error adding field 'result'='true' msg=For input string: "true"at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:218)at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:102)at org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:967)at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:341)at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:288)at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:235)at or

Hi everyone, How do you guys install/manage Druid through Cloudera Manager? Been trying to find the parcel or CSD file for installation without any luck.    Is this option coming soon or you just install druid directly on the servers. I understand that Druid was one of the components that can be install on Ambari previously. Is this going to be the same on Cloudera Data Platform or Cloudera CDH version 5 or 6.   Thank you very much.

If I do echo $JAVA_HOME I get /usr/lib/jvm/java-8-openjdk-amd64 as a response, yet I cannot get nifi to see it when it starts, always gives this:nifi.sh: JAVA_HOME not set; results may varyJava home:NiFi home: /opt/nifi-1.12.0 This is on a clean Ubuntu 18.04 installation, having followed these instructions: https://techexpert.tips/apache-nifi/apache-nifi-installation-ubuntu-linux/At this point really not sure what to do.

I am running the command from hdfs user hdfs@master>sqoop import-all-tables --connect jdbc:mysql://10.11.11.15:6306/siki_asmet?serverTimezone=Asia/Jakarta --username micronics -P --hive-import --warehouse-dir /warehouse/siki --hive-database siki_ods --exclude-tables "Sheet1$" --m 1; When i run the above command, i get the following error:  FAILED: SemanticException Line 1:17 Invalid path ''hdfs://master.lpjk.com:8020/warehouse/siki/_asdamkindo_personal_ska_pendidikan'': No files matching path hdfs://master.lpjk.com:8020/warehouse/siki/_asdamkindo_personal_ska_pendidikan (state=42000,code=40000) But when i try to run the command again, it said file already exists. 

I'm using a PutSNS processor which puts JSON (from API call) into SNS, in some cases JSON is more than 256KB, and I get "Size exceeds AWS SNS Limit of 256KB". Is there a way to chunk this into smaller sets?

Hi, I have below JSON file with multiples image's details{    "images": [{        "id": "C2934",        "ds_folder_id": "759838458734"    },    {        "id": "C8343",        "folder_id": "3942879137986"    }]}. I have id of the image and have to update the folder_id with some value. For Example:id is C8343 and the new folder_id is 2984584735then I have to update folder_id of the JSON file with new one. New JSON would be{    "images": [{        "id": "C2934",        "ds_folder_id": "759838458734"    },    {        "id": "C8343",        "folder_id": "2984584735"    }]}. Please help me to perform above in NiFi.

Hi Community, I'm looking for the training materials for CDH.I think there's no public materials about CDH. They seems to be for CDP, new products.(https://www.cloudera.com/about/training.html )I also tried to check the partner portal but it's 404 link. https://www.cloudera.com/partner-portal/training.html I have to upgrade CDH v5.16 to v6.3.3 and I'd like to learn the whole functions about CDH.(the ones written in Japanese are better if possible but I'm OK with them in English) Best regards,Takuya Kohama

Hello.I have activity monitor in green state.In logs I have smth like this:ps Sorry for quality, have no internet access in my cluster.But when I open navigator web ui I see this: