Solved

Hi when i'm performing calculations for "Importo" attribute, it computing correctly , but since it is an array i'm getting repetitive of  "Importo" : "." for each item level.  Can you help in get rid of this unwanted  "Importo" : "." values.INPUT:{"FatturaElettronicaBody": {"DatiGenerali": {"DatiGeneraliDocumento": {"Numero": "130889","Data": "31.10.2023","Causale": "22122013260655367-000180 DEL 20/12/2022","Divisa": "EUR","ImportoTotaleDocumento": "25369.31","TipoDocumento": "${TipoDocumento}"},"DatiDDT": [{"NumeroDDT": "287421-30.10.2023"},{"NumeroDDT": " 287419-30.10.2023"},{"NumeroDDT": " 287423-30.10.2023"},{"NumeroDDT": " 287422-30.10.2023"},{"NumeroDDT": " 287420-30.10.2023"}]},"DatiBeniServizi": {"DettaglioLinee": {"AltriDatiGestionali": {"RiferimentoTesto": "22122013260655367-000180 DEL 20/12/2022","RiferimentoData": "22122013260655367-000180 DEL 20/12/2022","TipoDato": "INTENTO"},"NumeroLinea": [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,"addnewline_1"],"Quant

Hi, i have cluster with knox installed and kerberos autentication enabled, i want to access the knox admin ui to make changes in the topologies, the thing is that i dont really know what is the admin-ui credentials, i tried using my admin from the kerberos but it doesn't work. my credentials from the cloudera manager also dont work. Everything is installed in the same cluster so i wanted to see if someone hava any idea what im doing wrong or  any idea of what credencials am i supposed to use.Thanks in advance

I started DistributedMapCacheClientService on Controller Services.when i try to start FetchDistributedMapCache processor got an error "AnnotatedConnectException: Connection refused".Do you have any comments on this situation?@MattWho 

HelloI was able to install CDS (spark) 3.3 with CDP 7.1.7-SP2. However, I am not able to do it with CDP 7.1.7.Can someone confirm the latest spark supported bu following CDP(s):1. CDP 7.1.7 - ??2. CDP 7.1.7 SP1 - ??3. CDP 7.1.7 SP1 - CD 3.3 works here.

How to use FFmpeg in NIFI . I want to convert Video to Tiff .

I have a flow that queries data from a source table and loads it into an iceberg table. The source data includes a Date type which gets read in as a string. The destination table has the same field where the schema is also a Date. When I try to use the PutIceberg processor, I get the following error:org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Failed Conversion of Field [null] from String [2023-10-30] to LocalDate- Caused by: java.lang.NumberFormatExceptionThe flow file is Avro and I'm using an AvroReader service for the PutIceberg. Here's a sample of the Avro:[ {"scan_id" : "1698665963-82DD686DBC1979E6FF6C5443BA82456621DB5004","queue_seq_id" : 558745163,"flight_airline_code" : "AC","flight_number" : 8663,"flight_departure_date" : "2023-10-30","scan_airport_code" : "YHZ","scan_checkpoint_name" : "Transborder","scan_stage_name" : "S2 - Screening Lines","manual_entry" : "N","wait_time_pax_queue" : "Main Queue","wait_time_seconds" : 93,"included_in_counts" : "

Trying to load and parse YARN application metrics and store it for performance analysis by using REST api call. Data returned is a single json entry with multiple application entries nested in a complex format. Can someone suggest or provide correct DDL how to map all data points to a hive external table since JsonSerde can parse and tabulate it?Sample json data format: {"apps":{"app":[{"id":"application_282828282828_12717","user":"xyz","name":"xyz-4b6bdae2-1a0c-4772-bd8e-0d7454268b82","queue":"root.users.dummy","state":"finished","finalstatus":"succeeded","progress":100.0,"trackingui":"history","trackingurl":"http://dang:8088/proxy/application_282828282828_12717/","diagnostics":"session stats:submitteddags=1, successfuldags=1, faileddags=0, killeddags=0\n","clusterid":282828282828,"applicationtype":"aquaman","applicationtags":"ABC,xyz_20221107070124_2beb5d90-24c7-4b1b-b977-3c9af1397195,userid=dummy","priority":0,"startedtime":1667822485626,"launchtime":1667822485767,"finishe

I am new to Apache NiFi and  have a fresh apache NiFi instance on my local system and i want to create 3 users and authenticate them with openID/google oAuth from their email ID( instead of seeing the usual NiFi login screen, i want to see the google authentication window where i can use my email id to login).i tried to follow the bryan bende's article on it but it feels very outdated for now. (https://bryanbende.com/development/2017/10/03/apache-nifi-openid-connect)please help me with this and provide me with a step-by-step breakdown on how can i achieve this? Thanks

I have a problem like this.I am using ApacheNifi nifi-1.23.0 version.Generate flowFile with GenerateFlowFile processor as follows.When I view the content, I see that it is as I shared below.As far as I understand, it mixes the content I triggered with a different content.What could be the reason for this?    

[root@b7-40 ~]# hiveSLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1000.24102687/jars/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1000.24102687/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]WARNING: Use "yarn jar" to launch YARN applications.SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1000.24102687/jars/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1000.24102687/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]

【Problem】After upgrading Cloudera Manager to 7.11.3, attempting to reboot the cluster causes an error in the Lily HBase Indexer of the Key-Value Store Indexer.【Use】CDP Private Cloud Base 7.1.8 (Will upgrade to 7.1.9 later)OS: RHEL 7.6Java: OpenJDK 11.0.8Python: 3.8.12【Occurrence procedure】Before upgrading, I installed Python 3.8 on all hosts that contained Cloudera Manager Agents.The installation was done according to the following documentation:https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/installation/topics/cdpdc-cm-install-python-3-rhel-custom-location.htmlThen, according to the following documentation, Cloudera Manager was upgraded from 7.7.1 to 7.11.3.https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/upgrade-cdp/topics/ug_cm_upgrade.htmlVerified that the Server and Agent were upgraded successfully.Therefore, I proceeded with the steps in Step 5: After You Upgrade, "Perform Post-Upgrade Steps".When I rebooted the cluster in step 4, I encountered an error with the

i generated a .p12 certificate using tls-toolkit by using following command :- ./tls-toolkit.sh standalone -n 'localhost' -B admin1admin -C 'CN=admin,OU=lending'when i try to upload this certificate in keychain access, it shows invalid password and after trying multiple time with the correct password, an error is thrown "mac verification failed during pkcs12 import (wrong password?)". please help me in solving this issue.

I want to have multiple users in Nifi. I don't want to use LDAP because I don't have a LDAP. I don't want to use Kerberos because I don't have a Kerberos. I don't want to use Apache Knox because I don't have a Apache Knox. I don't want to integrate with the thing because I don't have the thing. I just want to have 3 users, each with their own username and password. I've gotten to the point where I have three users but I have no idea what password to use for the non-admin users.  

HelloI'm running Nifi toolkit v1.12.1 with JDK 8I'm running into a problem when running the following commandcli.sh nifi pg-list -p /nifi.properties --verboseI can do awget --certificate=./my.pem --private-key=./my.key --ca-certificate=./myca.pem https://xxx/nifi-api/flow/process-groups/rootand get a correct outputCan you help me get the corret output with the toolkit ?Is it possible to log the toolkit requests / responses ?The stack trace I get with the toolkit is: org.apache.nifi.toolkit.cli.api.CommandException: Error executing command 'pg-list' : nullat org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:65)at org.apache.nifi.toolkit.cli.impl.command.AbstractPropertyCommand.execute(AbstractPropertyCommand.java:74)at org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processCommand(CommandProcessor.java:252)at org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processGroupCommand(CommandProcessor.java:233)at org.apa

Hi all Getting below error, when trying to start hive metastore: Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.fs.FsTracer.get(Lorg/apache/Hadoop/conf/Configuration;)Lorg/Apache/htrace/core/Tracer;  Details are as under Hive version: Apache hive 3.1.3      Remote metastore: postgres version 12 Schematool init created all relevant tables without any error. Only when trying to start metastore services facing error. Checked for htrace core incubating jar versions also. Htrace version details across cluster are attachedapache-hive-3.1.3-bin/lib/htrace-core-3.2.0-incubating.jarhadoop-3.3.4/share/hadoop/common/lib/htrace-core-3.2.0-incubating.jarhadoop-3.3.4/share/hadoop/yarn/timelineservice/lib/htrace-core-3.2.0-incubating.jarhbase-2.4.14/lib/htrace-core4-4.2.0-incubating.jarhbase-2.4.14/lib/client-facing-thirdparty/htrace-core4-4.2.0-incubating.jar Request your help.Thanks

Hi,I am running NIFI in Kubernetes cluster with 3 nodes. After the upgrade of NIFI to 1.23.1 version, most of the time, the JSON messages or CSV files are corrupted something like below, which is causing serious production issues to my workflows. Actual message:{"RequestPeriod":"morning","TRANSMISSION_TYPE":"","ReturnDatetime":"2022-02-07T13:00:00+01:00", "PickupDatetime":"2022-02-07T08:00:00+01:00"}Corrupted message:1:00"}{"RequestPeriod":"morning","TRANSMISSION_TYPE":"","ReturnDatetime":"2022-02-07T13:00:00+01:00", "PickupDatetime":"2022-02-07T08:00:00+0 Additional screenshots:OUTPUT: This is become an URGENT issue now. Any support would be highly appreciated.

I want to configure Apache Spark for collecting metrics using Jolokia. For that I am able to do in standalone but not in Cloudera Data Platform as it requires to edit the conf directory in which bigdata.ini file is present. I want to add the following lines in it:- [Spark-Master]stats: http://127.0.0.1:7777/jolokia/read Do anyone have any idea that how can we configure this in Cloudera? 

Even after passing the executor memory and driver memory as 1 Gb still it is getting maximum heap size rounded to 512 MB spark-shell --conf spark.executor.memory=1g --conf spark.driver.memory=1gWarning: Maximum heap size rounded up to 512 MB Anything that is missing?

I ran into an interesting situation using the Impala external table. In short, I used "create external table" statement but ended up with a table like a managed one. Here are details.Step 1: creating an external tablecreated external table testdb1.table1 ( fld1 STRING, fld2 STRING ) PARTITIONED BY ( loaddate INT ) STORED AS PARQUET tblproperties('parquet.compress'='SNAPPY','transactional'='false');Step 2: adding partitions and loading data files.alter table testdb1.table1 add if not exists partition (loaddate=20231018);load data inpath '/mytestdata/dir1' into table testdb1.table1 partition (loaddate=20231018);Step 2 shows that this table1 behaves exactly like a managed table. Files at /mytestdata/dir1 are moved to hdfs warehouse path warehouse/tablespace/external/hive/testdb1.db/table1/loaddate=20231018 path. If I drop this partition 20231018, the directory at warehouse/tablespace/external/hive/testdb1.db/table1/loaddate=20231018 is removed.So what exactly is the difference between a m

I am new to cloudera rest api.I try to upgrade the cloudera runtime with rest api instead of the web-ui for automation.  I am able to download the CDH with rest api.  curl -X POST -u admin:admin "http://{url}:7180/api/v44/clusters/{cluster-name}/parcels/products/CDH/versions/7.1.8-1.cdh7.1.8.p0.30990532/commands/startDownload" I am stuck on it ask for upgrade the cluster when it finish the download. From the Web-UI. it is a series of process.   I can skip all the checkup step(I understand the risks) and upgrade the cluster. How to do that with the rest api?Thank you very much!! 

I added some new nodes to my cluster and it works fine. Then I add Spark Gateway roles to all the new nodes. We're using Yarn to manage and distribute Spark work.Does adding Spark Gateway roles to new nodes enough to make Yarn think like "Hey there are some new nodes here, let's distribute some containers and work to these new nodes"? Or do I have to add Yarn Gateway roles to these new nodes too?How to make sure that Yarn will use these new nodes when executing jobs to reduce the overall workload of my cluster  

In case of imgae, only one single value(number) in second queue(name: matched).  How can I insert data(value in queue) into table use SQL?"insert into table values ( queue )"I want insert 1321 into table in this case.I mean i want make like as "insert into table values (1321)"Which variable or parameter can I use?insert into table values ( ${???????})Thank you for your answer.

I found the old thread that is exactly similar to the issue I am having .https://community.cloudera.com/t5/Support-Questions/SQOOP-import-of-quot-image-quot-data-type-into-hive/m-p/376825#M243019column data type in source [sql server] : imagecolumn value in source : 0xFFffDeDJBF.......dDf.      --> ( A hexadecimal value )column data type in hive: string column value in hive: FFffDeDJBF.......dDf (0x is not retained)Also I have tried binary data type in hive. yet the value would not match with what oracle has. Is there a data type hive table can handle and retain the  'Image' datatype orginal value in SQL source while sqoop importing into the hive table ?  

Hi, when I'm trying to run NiFi (both for versions 1.20.0 and 1.21.0) with JDK 1.8.0_202 on a Windows 64bit machin I keep on receiving the same error message:2023-04-23 01:00:45,455 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi... 2023-04-23 01:00:45,455 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: C:\nifi\NIFI-1~1.0 2023-04-23 01:00:45,455 INFO [main] org.apache.nifi.bootstrap.Command Command: C:\java\JDK18~1.0_2\bin\java.exe -classpath C:\nifi\NIFI-1~1.0\.\conf;C:\nifi\NIFI-1~1.0\.\lib\javax.servlet-api-3.1.0.jar;C:\nifi\NIFI-1~1.0\.\lib\jcl-over-slf4j-2.0.6.jar;C:\nifi\NIFI-1~1.0\.\lib\jetty-schemas-5.2.jar;C:\nifi\NIFI-1~1.0\.\lib\jul-to-slf4j-2.0.6.jar;C:\nifi\NIFI-1~1.0\.\lib\log4j-over-slf4j-2.0.6.jar;C:\nifi\NIFI-1~1.0\.\lib\logback-classic-1.3.5.jar;C:\nifi\NIFI-1~1.0\.\lib\logback-core-1.3.5.jar;C:\nifi\NIFI-1~1.0\.\lib\nifi-api-1.20.0.jar;C:\nifi\NIFI-1~1.0\.\lib\nifi-framework-api-1.20.0.jar;C:\nifi\NIFI-1~1.0\.\lib\nifi-nar-utils-1.

I ran into an issue last week, which seems to be an issue related to the Cloudera ODBC Driver for Impala. First I thought it was an issue with pyodbc, but after submitting the issue on their github we figured out that it was a driver issue. For reference: pyodbc github issue 1197 Simply put, executing query1 and query2 should produce the same results, but it doesn't: query1 = """SELECT col1, col2 as col2_orig, (CASE WHEN col2 IS NOT NULL THEN col2 ELSE 0 END) AS col2_1, (CASE WHEN col2 IS NULL THEN 0 ELSE col2 END) AS col2_2 FROM sandbox.jk_test"""query2 = f"""WITH wrapped_table AS ({query1}) SELECT * FROM wrapped_table""" It results in two different sets: ['col1', 'col2_orig', 'col2_1', 'col2_2']['1 ', '1 ', '1 ', '1 ']['2 ', 'None ', 'None ', '0 ']['col1', 'col2_orig', 'col2_1', 'col2_2']['1 ', '1 ', '1 ', '1 ']['2 ', 'None ', '0 ', '0 '] The 2nd result is the expected