Solved

Hello.Is there a way in ranger policies to  add permissions for users that are only in both of domain group1 and domain group2 (logical AND between groups). Because there is always a logical OR between users and groups in ranger policies??

Hi All,Could you please help in getting below value to be splitted across the array , the value outside of arrray Input:{"DatiRiepilogo": {"Natura": "N3.5","AliquotaIVA": ["22.00","5.00"],"ImponibileImporto": ["215.43","6.70"],"Imposta": ["47.39","0.34"]}} SPEC:[{"operation": "shift","spec": {"DatiRiepilogo": {"Natura":{"*":{"@(2,Natura)": "FatturaElettronicaBody.DatiBeniServizi.DatiRiepilogo[&1].Natura"}},"AliquotaIVA": {"@": "FatturaElettronicaBody.DatiBeniServizi.DatiRiepilogo[&1].AliquotaIVA","*": {"@": "FatturaElettronicaBody.DatiBeniServizi.DatiRiepilogo[&1].AliquotaIVA"}},"ImponibileImporto": {"@": "FatturaElettronicaBody.DatiBeniServizi.DatiRiepilogo[&1].ImponibileImporto","*": {"@": "FatturaElettronicaBody.DatiBeniServizi.DatiRiepilogo[&1].ImponibileImporto"}},"Imposta": {"@": "FatturaElettronicaBody.DatiBeniServizi.DatiRiepilogo[&1].Imposta","*": {"@": "FatturaElettronicaBody.DatiBeniServizi.DatiRiepilogo[&1].Imposta"}}}}}] Expected

Hello everyone,I am currently facing an issue with Cloudera Manager on my CDH 6.3.4 installation. I have noticed that my licenses are no longer valid, which is preventing me from accessing the Cloudera Manager graphical interface. However, I want to emphasize that I can still use the API to manage and control the various services.I have attempted to restart Cloudera Manager and my nodes, but the problem persists. I am not sure why my licenses have become invalid, as they were functional until recently.Can you please advise me on the best approach to resolve this issue? Are there specific steps I should follow to update or reactivate my licenses? Has anyone else encountered this issue with CDH 6.3.4 and could share their experience?I appreciate your assistance and guidance in advance. If you need more information to help me address this problem, please feel free to ask.Best regards.

Hi Team,I have below requirement to convert the SQL server queries into hive. But i am not sure how to achive that as there is no pivot and unpivot available in hive.could you please help me with the hive equivalent queries for the below SQL queries? Query 1-----------------SELECT x,y,zFROM dbo.tbl1 UNPIVOT (split FOR check IN (1, y2, y3, y4, y5, y6, y7, y8, y9, y10)) AS splnpivotQuery 2----------------SELECT *FROM(SELECT a,b,c FROM tbl2) SRCPIVOT (SUM(TOTAL_AMOUNT) FOR cbn_TYPE IN ([ONE TQ FOUR], [going loss], [COSTS LEAVING team sales], [profit], [check money])) AS PIV;Query 3--------------NULLIF (ISNULL(abc.Tc, 0) + ISNULL(abc.YR, 0), 0); Thanks in advance,Wasim 

Hi all, I would like to execute "INSERT OVERWRITE" from managed table to Hive Transaction Table which is EMRFS. What I tried was to first create managed table with s3 location (CSV), and executed the following: INSERT OVERWRITE TABLE tobe_tblSELECT * FROM asis_tbl LIMIT 10; but I got error as below:...Total jobs = 1Launching Job 1 out of 1FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask... I noticed that there was an error saying it is writing in Avro not ORC. What commands or configurations could I make to properly INSERT OVERWRITE to Hive Transaction Table?

I m trying to use HWC (HiveWarehouseConnector) with Spark- to append to an existing Hive ACID table- to overwrite an existing Hive ACID table- to append to a new Hive ACID table that does not exist before- to overwrite to a new Hive ACID table that does not exist beforeWriting to a new Hive ACID table works but writing to an existing table does not work using append/overwrite. Please let me know how to use HWC to append/ overwrite existing ACID tables. List of steps that I followed in CDP 7.1.7 SP1 single node cluster is given below:1) Ran kinit username2) Ran these below commands in beeline hive client command line.drop database testDatabase cascade;drop database tpcds_bin_partitioned_orc_1000 cascade;create database tpcds_bin_partitioned_orc_1000;use tpcds_bin_partitioned_orc_1000;create managed table web_sales(ws_sold_time_sk bigint, ws_ship_date_sk bigint) stored as orc;insert into web_sales values(80000,1), (80001,2), (80002,3);create database testDatabase;3)I prepared a

I'm using the Zeppelin 0.10.1 container. The first launch of the container is successful.If you stop the container and start it again, it does not startcontainer log on startzeppelin | INFO [2023-10-09 16:09:59,673] ({main} ZeppelinLocationStrategy.java[locate]:44) - Load configuration from /opt/zeppelin/conf/zeppelin-site.xmlzeppelin | INFO [2023-10-09 16:09:59,681] ({main} ZeppelinLocationStrategy.java[locate]:44) - Load configuration from /opt/zeppelin/conf/zeppelin-site.xmlzeppelin | INFO [2023-10-09 16:09:59,796] ({main} ZeppelinConfiguration.java[create]:135) - Server Host: 0.0.0.0zeppelin | INFO [2023-10-09 16:09:59,796] ({main} ZeppelinConfiguration.java[create]:139) - Server Port: 8088zeppelin | INFO [2023-10-09 16:09:59,796] ({main} ZeppelinConfiguration.java[create]:141) - Context Path: /zeppelin | INFO [2023-10-09 16:09:59,797] ({main} ZeppelinConfiguration.java[create]:142) - Zeppelin Version: 0.10.1zeppelin | INFO [2023-10-09 16:09:59,821] ({main} Log.java[initialized]:16

Hi, we are trying to create an alert to get notification when no row was inserted into a given HBase table in given period of time.  The following command was used: IF (SELECT total_write_requests_rate_across_regionservers WHERE serviceName=$SERVICENAME AND max(total_write_requests_rate_across_regionservers ) = 0 AND entityName = "hbase" AND category = "SERVICE") DO health:badbut the name of HBase table is missing. The `htableName` property is mentioned under the https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_dg_tsquery.html#concept_d2g_k11_dk__section_cdx_sbp_3l chapter, but it does not work - the command return empty results after putting the `htableName` attribute to WHERE condition. Could I please to an advice, how to get a notification when entities are not inserted to the HBase table?

I need some inputs on relationship definitions in a custom processor code:1. Do I need to define all three(success, failure, original) relationships in processor class?2. Do I need to handle all three relationships(using session.transfer(....))?3. Why do I need to transfer original flowfile as it has been processed?

When a try this script, i have  this error raise Exception('No LDAP URL was specified') on  hue/apps/useradmin/src/useradmin/ldap_access.py at master · cloudera/hue · GitHub . So i decided to read  the  following lines : ldap_url = ldap_config.LDAP_URL.get()username = ldap_config.BIND_DN.get()password = desktop.conf.get_ldap_bind_password(ldap_config)ldap_cert = ldap_config.LDAP_CERT.get()search_bind_authentication = ldap_config.SEARCH_BIND_AUTHENTICATION.get()if ldap_url is None:raise Exception('No LDAP URL was specified')Its means that ldap_url is None, so i tried to prind ldap_config and i have :"<class 'desktop.lib.conf.BoundContainerWithGetAttr'>(config=<desktop.lib.conf.ConfigSection object at 0x7f5b80fb0250>, bind_to={'auth': {'jwt': {}}, 'database': {}, 'query_database': {}, 'task_server': {}, 'session': {}, 'knox': {}, 'smtp': {}, 'websockets': {}, 'raz': {}, 'ozone': {}, 'kerberos': {}, 'tracing': {}, 'metrics': {}, 'slack':

I have a usecase which involves creating a new Record(actually several records) from a record in input flowfile, and writing it in output flowfile.PS: I would use avro reader to read input and avro writer to write into output.What is idiomatic way of doing it in nifi?

Hi,I want to read data from Ranger Rest API ...assets/accessAudit...Works fine but need to set the startDate like I can do it in the Ranger WebUI in the Audit part.Is there any docu where to find how to do that? Example: ...server:port/service/assets/accessAudit?startDate=06/02/2023&resourceType=@table&accessType=select&sortBy=eventTime&sortType=descending&excludeServiceUser=falseIn this example the startDate does not work, only getting the data from the actual day.If I use the startDate in the Ranger WebUI it works fine. BRRoland

I want to transfer data from HDP 2.6 with kerberos authentication via spark to CDP with  kerberos authentication. I am trying to run 2 kerberos ticket in a server but not able to run. One kerberos ticket will be used for reading data from HDP and second kerberos ticket for writing data in CDP cluster. Anyone has any better idea apart from spark please let me know

Hello,We have several Data Hub Clusters on CDP - PC, currently conducting health checks manually, which is time consuming and repetitive. Would like to know if there are some automated scripts / ways, we could get this task completed which would give us the health status of nodes, Cloudera services running on them.Any assistance / guidance is appreciated.

Hello,trying to setup zeppelin + livy-on-spark3.Livy-on-spark2 is up and rspark3.sql runs fine, any spark3.pyspark code returns error "Interpreter died:".What could be wrong?

Hello Experts,I have a requirement where I need to re-load my custom processors nar without restarting Nifi.I tried with nifi.nar.library.autoload.directory option, If I freshly place new nar first time in autoload directory it loads automatically.But if I replace the nar in autoload directory again with some enhancements, it does not get reloaded in nifi.I tried with changing nar version, with that nifi loaded that new version nar but that resulted in 2 different versions processors in Nifi (it does not unload the pre version nar)Is there any any "auto reload" feature for custom nar or something like unload prev version and load new version automatically without restart?Any help/suggestion is much appreciated.ThanksMahendra

Hi Support,Would like to ask is that possible to set the Global_parameter in processor configuration via nifi rest api instead of using UI ? 

I developed a custom hook on CDH (6.2. X), implement the HiveDriverRunHook interface and ExecuteWithHookContext interface. It runs smoothly, but when I migrate the hook to CDP (7.1. X), I report an error and cannot execute it. I suspect it was caused by the Tez engine, but I cannot solve it and need help，thanks very much

 

Hello Team,I am trying to use the underscore "_" in the column name example "_name" but the create statement is failing. i also wanted to use the reserved keywords example "transform" as column name but that is also not working.I tried setting the values for keyword and for identifiers while running command still it is giving me error when running via beeline  Following are the commands and the error that i am getting  :Command : sudo -u hive beeline -u "jdbc:hive2://cdp1:2181,cdp2:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;;principal=hive/_HOST@hdpcluster.test" -n hive --showHeader=false --outputformat=tsv2 -e "SET hive.support.quoted.identifiers=column;use jayesh; create external table testkeyword1( `transform` string);";error : Error: Error while compiling statement: FAILED: ParseException line 1:45 cannot recognize input near ')' '<EOF>' '<EOF>' in column type (state=42000,code=40000)------------------------------------------

Hi all, Have hit a snag when trying to setup a Ozzie workflow with combination of CTE and Insert Statement, not sure why, please help me look at the code, thanks.  set hive.exec.dynamic.partition=true;set hive.exec.dynamic.partition.mode=nonstrict;insert into table A partition(date)with test as(select col1 as name1 , col2 as name2, col3 as name3 from table B),select * from test   

Hi, i need to know how to create a code in both PIG and Hive for this data: I need to obtain the sum of the newcases for the continents, as well as which country has had the highest number of cases in each continent and their number. somethig like: For pig i was trying somethig like: Any one knows how to do it? Thanks.

Hi everyone, So I've inherited a kerberized Cloudera cluster and I'm learning as I go. Right now I'm trying to get Airflow to work with our Spark jobs but without success. As I understand Airflow was installed by our OS team only after the cluster was configured by Cloudera. It runs on our edge node from where we run our jobs. Basically I'm using bash operators for my test DAG with the following tasks: Task 1:Kinit the user that is running the script:"echo 'password' | kinit user@domain" Task 2:Download some files from some location. Task 3:spark-submit /path/to/script.py  Task 1 and 2 work fine, but task 3 fails with the following: py4j.protocol.Py4JJavaError: An error occurred while calling o32.csv.: org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.  Available:[TOKEN, KERBEROS] I am a bit confused by this as I am am authenticating the user as a f

Hello Experts,After changing MIT Kerberos to AD Kerberos and Regenerating all the Kerberos credentials in CM the zookeeper, YARN etc. is not starting. There is an error about the Active Directory samaccount not able to login as the zookeeper principal. I checked that the principals are created in the AD OrgUnit for Cloudera. And the $ kinit -kt zookeeper.keytab zookeeper/redacted@ADREALM on the linux servers works fine.  Any thoughts how to fix? SERVICE_TYPEZOOKEEPER SEVERITYCRITICAL STACKTRACEjavax.security.sasl.SaslException: Problem with callback handler [Caused by javax.security.sasl.SaslException: redacted@ADREALM.COM is not authorized to connect as zookeeper/redacted@ADREALM.COM] at com.sun.security.sasl.gsskerb.GssKrb5Server.doHandshake2(GssKrb5Server.java:333) at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:161) at org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer.authenticate(SaslQuorumAuthServer.java:

Hi Experts, We are on 7.1.6 CDP version and exposed to CVE-2022-25168.  Versions affected: 2.0.0 to 2.10.1, 3.0.0-alpha to 3.2.3, 3.3.0 to 3.3.2 To address this vulnerability, we are planning to upgrade to CDP 7.1.7 as I got this information from one of the posts in the cloudera community. My question is does anyone did this upgrade before from 7.1.6 to 7.1.7 to eliminate this CVE. If yes, then how easy is it do we need to make changes in the code level also or we can do the upgrade from the GUI by just downloading, distributing and activating the new version.   Please help with your suggestion and if possible, please share the documentation for this upgrade. Thanks