Find answers, ask questions, and share your expertise
Recently active
Hi All, We are enabling hbase hook in apache atlas 2.2.0 , we followed below steps ,https://atlas.apache.org/2.0.0/Hook-HBase.html After that , when we start to start hbase , failing with below error.ROD.2022-09-05 14:02:23,481 INFO [master/vmazewdranger01:16000:becomeActiveMaster] atlas.ApplicationProperties: Setting solr.wait-searcherproperty 'true'2022-09-05 14:02:23,481 INFO [master/vmazewdranger01:16000:becomeActiveMaster] atlas.ApplicationProperties: Setting index.search.map-name property 'false'2022-09-05 14:02:23,482 INFO [master/vmazewdranger01:16000:becomeActiveMaster] hook.AtlasHook: Failed to load application propertiesorg.apache.atlas.AtlasException: Failed to load application propertiesat org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:150)at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:103)at org.apache.atlas.hook.AtlasHook.<clinit>(AtlasHook.java:80)at org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.<in
Hi, I have the requirement where I am converting the base64 Image content to attribute and after that I am preparing the payload including the base64 Image content. But the problem is while converting the base64 Image content (approx. 6-7 MB size ) it is getting truncated in the attribute Value. Can you please tell me what is the length limit of attribute value in NIFI.Appreciate your help. Thank you.
I've added ranger as service in Cloudera manager, but when I click 'audit' from the menu i get the following error :
Hello,I am trying to create an Operational DB and getting the error in Subject. Appreciate any inputs.Thanks in advance.
Hi Cloudera community, I have been using MapR for sometime and have some regular data pipelines in place. I want to move out from MapR to Couldera. My clarification with the experts here is, I want to retain the system & data architecture created in MapR(servers, nodes, clusters, data storage) and implement couldera on top of hadoop HDFS. Is this possible? basically i want to replace MapR with Couldera without disturbing my existing hardware and data storage setup. Looking forward for some sharing.
Problem: I have set up a listSFTP processor to run between 2am to 2pm everyday. But the files are not being picked up by the processor. Existing setup and configuration: The files are generated at the SFTP server at 12 am to 1am every day. ListSFTP ConfigurationSchedule: listing strategy: tracking timestampsother tracking configuration When I start this processor, it runs as expected for 1 day or 2 and after that, the files are not picked. Is it the tracking time window of 3 hours that is affecting the file being listed?
Hello,I have a question about Cloudera Data Platform and SDX :Is data sharing available accros my company only?If I want to share a specific file or a specific table in my datawarehouse with my client outside my company (who has his own account in Cloudera), is it possible? Regards,Mohamed
Need your expertise on this. I would like to know concerning about disabling the ansible tower job isolation security bubble .I am aware that there is a security bubble integration with Ansible Tower and CyberArk using Central credential provider (CCP) approach .However, with the credential provider (CP) approach, this security bubble integration will be bypassed
I have a GenerateFlowFile processor that sends a flowFile to the ExecuteScript processor. In the ExecuteScript processor, I added a new attribute and value to the incoming flowFIle. But this updated flowFIle is not output to the next processorIncoming FlowFile content from GenerateFlowFile {'name':'myName'} ExecuteScript script body var flowFile = session.get();if (flowFile != null) { try { flowFile = session.putAttribute(flowFile, 'gender', 'male'); session.transfer(flowFile, REL_SUCCESS); } catch (e) { session.transfer(flowFile, REL_FAILURE); }} Output to next processor {'name':'myName'} Am I missing something??
I have NIFI single Node and for my learning purpose I'm trying to implement SiteToSiteBulletinReporyingTask (with assuming this will work for single node)I configured StandardRestrictedSSLContextService as belowNote - same keystore and truststore used inside nifi-properties I configured SiteToSiteBulletinReportingTask as belowNote - the destination URL is the same URL that nifi is currently running. Now I'm facing below issues -I'm not receiving any data to the bulletin port. I'm getting thebelow warning message in nifi-app.log (couldn't find resource to fix this)The issue is I'm getting the below warning and, to be honest, I don't have a clear idea about this warning, and also2022-10-23 17:57:09,618 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Certificate for <xxx.xxx.xx.xxx> doesn't match any of the subject a
I have a NiFi flow that reads file from local drive and writes in another folder. But due to some reason it is not reading files on regular basis. I can't find anything on the log. The log file is printed as if nothing was scheduled for that particular time. It's simply skipping the timing and continuing. I'm using Apache NiFi version 1.11.4 Screenshot of the Flow. Screenshot of Scheduling tabScreenshot of the GetFile Processor Scheduling tab. I have scheduled it for 08:05 on daily basis. Screenshot of properties tabScreenshot of GetFile Processor Properties tab. This NiFi is hosted on Windows 2012 server and this problem is only occurring in production environment. Although all the configuration is same in all the environment. I'm also attaching the sample flow template in case if anyone want to try it out. Sample_flow.xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?><template encoding-version="1.3"> <
The current cluster has 7 tablet servers and 448 CPU cores. It is planned to set Hash partitions to cooperate with Range partitions. The Range partition divides the ten year data into 120 partitions by month. How many Hash partitions based on other fields is appropriate? In addition, the main read/write scenario is a large number of reads/writes based on the current month's data.
when I input:start-all.shin command line, I can open localhost:50070 successfully.However, if I click on the overview tab, nothing is shown on overview subpage.But other subpages work well, like datanode subpage:localhost:8088 also looks well: I don't know whether this case is normal or there's something wrong with my overview page.
I'm trying to run listenHTTP processor for listening to a webhook notification. I'm relatively new to this processor. When I initialize, I observe java.net.bind exception. Address already in use. nifi url: http://localhost:8080/nifi/ (it has a public IP but we use NATing) Existing configuration Error Observed
Have been getting the below exception "ReplaceText[id=3fa43c98-5a5a-1c87-8e34-911a06de02aa] Processing failed: org.apache.nifi.processor.exception.ProcessException: IOException thrown from ReplaceText[id=3fa43c98-5a5a-1c87-8e34-911a06de02aa]: java.io.FileNotFoundException: /opt/nifi-1.17.0/content_repository/466/1666653401958-112082 (No space left on device) - Caused by: java.io.FileNotFoundException: /opt/nifi-1.17.0/content_repository/466/1666653401958-112082 (No space left on device)" It seems like the disk is full which is not allowing the processors to update or modify the data. I notice that the logs directory on all the cluster have taken all the space. Each node has 64G of storage space and log directory has taken almost 55G of space. I recently notice my disk is getting full very quickly. I remove the logs and the repository and restart nifi and that seems to work but then again after a few hours it got filled up. I finally notice that the logs direct
So I am using SplitXml processor to split flowfile. After some transformations I am merging those flowfiles back together using MrgeContent. After mergning I have noticed that not all of the flow files merged together. e.g. I have splitted my flowfile on 100 parts and after merging I've got 2 files with 50 records of my initial flow file each. When i set Run schedule option to 10 seconds instead of 1 it merged all files together. But what if i don't know how many flowfiles i will get after split and how much time it will take for merge processor to merge it? I use flowfile name as Correlation attribute name. Thank you beforehand!
Hello community, well i cannot put data from local to hdfs due to this error.how can i solve it please?!
There are many same errors showing at back-end and nothing return to front-end.error message as below: 2022-10-31T01:42:15.569Z, RpcRetryingCaller{globalStartTime=1667180497142, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.CallQueueTooBigException: Call queue is full on XXXXX,16020,1663311727745, too many items queued ? env: OpenShift container + Hbase
I am unable to complete hue installation due to the following errorUnexpected error: Unable to verify database connection #Hue
running the following from hiveSET hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;SET hive.support.concurrency=true;INSERT INTO hello_acid partition (load_date='2016-03-01') VALUES (1, 1);getting this error SQL Error [30041] [42000]: Error while processing statement: FAILED: Execution Error, return code 30041 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Failed to create Spark client for Spark session e73c125d-e40a-49fb-b2e0-28c28ab1ff60_0: java.lang.RuntimeException: spark-submit process failed with exit code 1 and error ? Hive has Spark configured as dependency Have this settings in hive-site.xml <property><name>hive.spark.client.connect.timeout</name><value>30000ms</value></property><property><name>hive.spark.client.server.connect.timeout</name><value>300000ms</value></property> Some Other settings Java Configuration Options for HiveSer
Do table statistics get replicated as part of hive BDR replication ?
Hi We use Cloudera in the environment without internet access These URLs added on Parcel Repository & Network Settings :https://archive.cloudera.com/accumulo-c5/parcels/latest/https://archive.cloudera.com/accumulo6/6.1/parcels/https://archive.cloudera.com/kafka/parcels/latest/http://archive.cloudera.com/kudu/parcels/latest/Someone know how to remove these URLs using the CM API ? Best Regards,
Hello, As part of a class I am in, we have tasks to edit the configuration files to include or change certain properties. I have found numerous articles detailing the importance of changing these properties but cannot find any explaining HOW TO change properties, either through the CLI or the web interface. We are using Hortonworks Hadoop Sandbox HDP through a virtual machine. For instance, in one of our tasks, we are to configure wire encryption. I understand that I need to edit the core-site.xml configuration file to include the property hadoop.rpc.protection=privacy But I do not understand how to do so. I have tried going into the CLI, and the furthest I can make it is to change the directory to etc/hadoop/conf where the file is located. I can also view the file properties using the vi command. But how can I add a property to the file, or replace the value if it an existing property. I have searche
Hi all, I would like to change the name of objects and add values to the following JSON file.[ { "Ean": 4052269532309, "Bestand": 0 }, { "Ean": 4052269532316, "Bestand": 0 }, { "Ean": 4052269532323, "Bestand": 0 }]The result of my specification is not what I expect.JOLT Script:[ { "operation": "shift", "spec": { "*": { "Ean": "gtin", "ean": "gtin", "EAN": "gtin", "EanCode": "gtin", "EAN-Code": "gtin", "EANCode": "gtin", "Ausdr1": "gtin", "Bestand": "qty", "Menge": "qty", "Stock": "qty", "stock": "qty", "Verfügbare Menge (Kennzahl)": "qty", "Verfügbare Menge": "qty" } } }, { "operation": "modify-default-beta", "spec": { "update_date": "2022-11-03", "supplier_name": "demo", "file_name": "test" } }]I get this result:{ "gtin" : [ 4052269532309, 4052269532316, 4052269532323 ], "qty" : [ 0, 0, 0 ], "update_date" : "2022-11-03",
Hi,I have Spark program written in scala which is continuously consuming data from Kafka topic processing and aggregating data. The program uses structured streaming and is supposed to insert data in Hive (non- acid table) partitioned based on 'startdate' field from the data.I tried below to insert data to Hive table. val query = stream_df_agg_tr1.writeStream.format("orc").partitionBy("startdate").option("checkpointLocation","<>:8020/user/sm/hivecheckpt8").outputMode(OutputMode.Append).start("<>/warehouse/tablespace/external/hive/activecore.db/applicationstats5minute")query.awaitTermination() I tried by specifying the table path using .option("path",<>) as well. I have below properties set (tried in spark program and as spark-shell arguments) ."spark.hadoop.hive.exec.dynamic.partition=true" "spark.hadoop.hive.exec.dynamic.partition.mode=nonstrict" Still, I do not see the partitions getting created automatically. Once I tried with one date-based pa
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.