Solved

Hi Support Team, I have JSON input for Nifi flow with some special characters. Could someone help me with how to remove special characters following payload? we would need only value with array and double-quotes.input json: - {"TOT_NET_AMT" : "[\"55.00\"]","H_OBJECT" : "File","H_GROSS_AMNT" : "[\"55.00\"]","TOT_TAX_AMT" : "[9.55]"}  Expect Result :  {"TOT_NET_AMT" : "55.00","H_OBJECT" : "File","H_GROSS_AMNT" : "55.00","TOT_TAX_AMT" : "9.55"}

Hello , I have this error when executing select count(*) on Hive on Tez , could you guys help me?      

I had a table that it vanished after an issue. I created again the table and then recover the partitions in order to read again the files that were in hdfs hive direcotry. Although all of a sudden now, it throws me the below error: File 'path/data.0.parq' has an incompatible Parquet schema for column 'db.table.parameter_11'. Column type: STRING, Parquet schema: optional int64 amount [i:10 d:1 r:0]  

Hi Team, I have a scenario if an Participant status  Filed value is 'Inactive' then i should update some of the columns data in the file with NULL/empty. Could you please help me in how can we achieve this? 

我在一个启用了kerberos的CDH6.3集群里，提交一个flink1.9版本程序到yarn，需消费集群外一个0.8版本的未启用安全认证的kafka，需要如何设置他不使用安全的方式去连接这个集群外kafka的zookeeper呢？谢谢。 org.apache.flink.shaded.zookeeper.org.apache.zookeeper.client.ZooKeeperSaslClient - Client will use GSSAPI as SASL mechanism.org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - Opening socket connection to server ark1.analysys.xyz/10.2.33.189:2181. Will attempt to SASL-authenticate using Login Context section 'Client'org.apache.flink.shaded.zookeeper.org.apache.zookeeper.client.ZooKeeperSaslClient - An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state.

Hello Everyonei am running one YARN job on daily basis ,but today its running little bit slow than yesterday, on the same node other peoples also running the same job but they didn't face any issue regarding the slowness job,only my job is running slow and its a daily running jobwhat could be the possible reason behind this? thanks in advance !!

Hello experts!Trying to add the host to an existing SSL-protected demo CDP 7.1.6 cluster (on IBM Power servers, ppc64le).The "Install Agents" wizard step fails with the error "Installation failed. Failed to copy installation files."When looking into the "Details" link, the following is printed there:/tmp/scm_prepare_node.gvqeyBBtFailed to generate certificates for cldr62.ibmcc.ru/opt/cloudera/cm-agent/bin/certmanager is present on both the Cloudera Manager host and on the machine to be added.Can someone please recommend how to resolve this issue?Cloudera Manager logs do not seem to reveal anything useful: 2021-08-12 16:14:24,928 INFO scm-web-499421:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/add-hosts-wizard/install2021-08-12 16:14:24,936 INFO scm-web-499421:com.cloudera.server.cmf.node.NodeConfiguratorService: Creating request with id 162021-08-12 16:14:24,937 INFO scm-web-499421:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing

Issue summary: In cdh6.2.1，with kerberos and sentry enabled, we are getting issues when using statement "insert overwrite" to insert data into new partitions of parttioned table, the sql statement, and detailed metastore and hiveserver2 logs are attached. Note:1. Event though errors are throwed (and show partions xxx will not dispaly the new partition), the underneath hdfs directory and files for the corresponding partition are created successfully.2. after errors throwed for the "insert overwrite" statement, we can use msck repair tablexxx to fix the hive metastore data for the talbe, and after that, we can use "show partitions" to dispaly the new created partition successfully, and use "select xx" to query the new inserted partition data successfully:3. This happens for both static mode partitioning and dynamic mode partitoning；( as long as you are inserting data to new partitions）4. if you are using insert overwrite to insert data into an existing partion (the partition can be eithe

Hi, I am new to using Cloudera Cloudera Data Visualization tool (public)... wanted to know if anyone knows if it is possible to set only a "From" date filter. What I have is; -Data that predicts outcomes from now to a set number times, eg. predict the next 10 events from now.- The predicted events happen over hours- What I want to do is the ability to set only the FROM date filter and then have the vis return all the values from that point At the moment, I have to set a FROM and TO date. Setting the TO date means I only ever gets result TO that point. Thanks

 I have installed Nifi locally and recieved the below error, could some one please let me know what need to be done.   It would be great help if provided in detailed.thanks!

Hello Team, I have the following file to load on Kudu.1829;BN=0;UNIT=VOLUME_ALL;IN=0;TC=0;TCC=0;CT=;FU=1000001;CU=54274;FB=61701;FL=ugw9811_3828500385_360_271530=5126742111750858;U=23059268534;SI=6;SG=1;SR=7;SN=BROWSING;SC=BROWSING;BS=60342256;BR=2581143;TU=2021-04-27 14:02:47;TF=2021-04-27 00:00:00;TA=2021-04-27 14:02:47;TB=2021-04-27 00:00:00;TE=2021-04-27 14:02:47;TS=1619517767;D=16292;R=151;E=0;UDR_cu=0;UDR_fb=BROWSING;DCM=0;UP=Prepaid;ST=BROWSING;MSISDN=23059268534;APN=orange;SGSN=196.192.13.113;GGSN=196.192.13.113;IMSI=617010014925066;BU1=23292;BN=0;UNIT=VOLUME_ALL;IN=0;TC=0;TCC=62923399;CT=;FU=1000000;CU=3586;FB=61701;FL=ugw9811_3828490275_312_87990=5126752111750858;U=23059268534;SI=6;SG=1;SR=7;SN=BROWSING;SC=BROWSING;BS=0;BR=0;TU=2021-04-27 14:02:47;TF=2021-04-27 00:00:00;TA=2021-04-27 14:02:47;TB=2021-04-27 00:00:00;TE=2021-04-27 14:02:47;TS=1619517767;D=16292;R=151;E=0;UDR_cu=0;UDR_fb=BROWSING;DCM=0;UP=Prepaid;ST=BROWSING;MSISDN=23059268534;APN=orange;SGSN=196.192.13.113

I have to implement one scenario using Nifi flow. 1. There is one attribute from upstream processor like    KeyValue : KGPYT2. I have to find whether KeyValue is unique in the system or not , by validation from Database.   If it is not found in the database, the flowfile should be proceed to success and   if the value is present in database, it should be proceed to failure. Please suggest me how to proceed in this case. thanks!

Hello guys,I just wanted to know that what are the issues in hive and spark that mostly faces in production , and how to figure out this.could anyone tell me??

One of the JSON values needs to find the string last element. how to get the particular value in attribute with the last element.{"TOT_NET_AMT" : "55.00","H_OBJECT" : "File","H_GROSS_AMNT" : "55.00,58.00","TOT_TAX_AMT" : "9.55"} expected value - {"TOT_NET_AMT" : "55.00","H_OBJECT" : "File","H_GROSS_AMNT" : "58.00","TOT_TAX_AMT" : "9.55"} Much appreciate it in advance.

Hi Community Team. I´m configuring Distcp between secure clusters in different kerberos realms.Source= HDP 2.6.5Target= CDP Private Cloud Base 7.1.5 I am setting up, but in the process I have doubts as to how these 2 steps are done, I add the links that show little detail1)First doubt (what is the step by step of this):https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/scaling-namespaces/topics/hdfs-distcp-truststore-properties.html 2)Second doubt (what is the step by step of this):https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/scaling-namespaces/topics/hdfs-distcp-set-hadoop-conf.html thanking the help

Team , Any idea on the migration process from HDP 2.x version to CDP version . RegardsBharad

While trying to create a case to send it to support, it shows me this message : "Access to page is restrictedAccessing the requested page requires special permissions "  what shall i do ?

Hello boys!Trying to get the free version of the CDH to be able to present a solution in the company, but I can't download it.I'm copying and pasting link tamplate https://igor.ufrn@gmail.com:MyPassword@archive.cloudera.com/cdh6/6.3.2-patch4071/ubuntu1804/apt/ And I get the answer:"403 Forbidden (varnish) the provided credentials were incorrect"What should I do?Thank you very much for the support.

I am trying to install Cloudera in AWS for test purposes and the first obstacle I ran into was when I tried to run the cloudera-manager-installer.bin which I downloaded from https://archive.cloudera.com/p/cm7/7.3.1/cloudera-manager-installer.bin. The problem is that when you run this it fails with "https://archive.cloudera.com/cm7/7.3.1/redhat7/yum/repodata/repomd.xml: [Errno 14]  Error 404 - The requested URL returned error: 404 Not Found" and this seems like a bug because the file is there but in a diff location ie https://archive.cloudera.com/p/cm7/7.3.1/redhat7/yum/repodata/repomd.xml (please note the "p" in working link .  How do we fix this ? Looks like installer is looking in wrong location ? 

Hi Folks,Can anyone tell me how to check how many users are connected to the hive server2?And  along with can we see which user is connected ?

Hello Everyone,How can we renew the kerberos ticket from both MIT as well as AD kerberoswhats the step?   Thanks in AdvanceNilesh

Hi experts,We recently changed the Ip address of our ambari in our dev enviornment.The cluster seems to be up and running and working properly, however, ambari is not recognizing which namenode is active and which is passive. Also some of the users are unable to access the ambari hive view.This is the error message when trying to access the hive view via ambari. USER HOME CheckMessage: test01.dmicorp.com:50070: No route to host (Host unreachable) Any help is much appreciated. Thanks,

Hi Community, I am trying to open the Hive weburi and getting below error HTTP ERROR 403Problem accessing /hiveserver2.jsp. Reason: java.lang.IllegalArgumentException  Weburi is configured in system with all required parameters. pls advice.  ThanksAmit

Hello, I need your advice regarding what seems like a strange behavior  in the cluster I'm using. Running Spark (2.4, Cloudera) on Yarn the configuration calls the set up of 10 executors:  spark = (SparkSession .builder.master("yarn") .config("spark.executor.cores", "12") .config("spark.executor.memory", "8G") .config("spark.num.executors", "10") .config("spark.driver.memory", "6G") .config("spark.yarn.am.memoryOverhead", "6G") .config("spark.executor.memoryOverhead", "5G") .config("spark.driver.memoryOverhead", "5G") .config("spark.sql.hive.convertMetastoreOrc", "true") .config("spark.executor.heartbeatInterval", "60s") .config("spark.network.timeout", "600s") .config("spark.driver.maxResultSize", "2g") .config("spark.driver.cores","4") .config("spark.executor.extraClassPath", "-Dhdp.version=current") .config("spark.debug.maxToStringFields", 200) .config("spark.sql.catalogImplementation", "hive") .config("spark.memory.f

One of of my data-node won't start. The error log on {HADOOP_LOG}/hdfs/{datanode}.err shown that the port of dfs.datanode.http.address already in use. We use custom port for  dfs.datanode.http.address. When I run netstat -lpn | grep 1022, no service run on it. While for {datanode}.log file is not updated.  Initializing secure datanode resourcesisSaslEnabled:trueOpened streaming server at /0.0.0.0:1019java.net.BindException: Address already in use /0.0.0.0:1022 at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:461) at sun.nio.ch.Net.bind(Net.java:453) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78) at org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.getSecureResources(SecureDataNodeStarter.java:158) at org.a