Find answers, ask questions, and share your expertise
Recently active
Hello Experts, I am new to Flink CEP and trying to test basic things - In below code my expectation is all the input should matched in patter and should print as matched result. But somehow nothing is matching ('matechedStream.print()') any idea about the reason ?Any suggestion/help would be much appreciated. package com.o9.flink;import com.o9.flink.asyncio.DemandSupply;import org.apache.flink.cep.CEP;import java.util.ArrayList;import java.util.List;import java.util.Map;import org.apache.flink.cep.PatternStream;import org.apache.flink.cep.functions.PatternProcessFunction;import org.apache.flink.cep.pattern.Pattern;import org.apache.flink.cep.pattern.conditions.SimpleCondition;import org.apache.flink.streaming.api.datastream.DataStream;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;import org.apache.flink.streaming.api.windowing.time.Time;import org.apache.flink.util.Collector;public class DemandSupplyPattern { public static void main(String[]
Hi all , is Apache Druid supported as as a stand alone service in the Cloudera data flow (CDF)
Hello,first of all I apologize for posting this subject here,because I couldn't find an email address to contact the support directly.At this moment I am unable to ask support questions and I have the following messageIf anyone has had this problem please help me.thank you.
Hello,I am working on one of the tasks in Nifi and I am stuck on of the issue, the issue is I am getting the Json data from jolt transforms processor and also I am getting the Json data from input port processor now the issue is I need to combine the data from two processor based on id’s. How to combined the data from two processors based on common id ? Will you please help me out.Thanks,Sathish
The following code what it does is to transform a string to json and then send them to kafka.But it is sending some blank messages.Does anyone know how I can control what is sent, that is, what comes empty not to send it to kafka. This is code:import sysimport tracebackimport stringimport java.iofrom org.apache.commons.io import IOUtilsfrom java.nio.charset import StandardCharsetsfrom org.apache.nifi.processor.io import StreamCallbackfrom org.apache.nifi.processor import Processorfrom org.apache.nifi.processor import Relationshipfrom org.apache.nifi.components import PropertyDescriptorfrom org.apache.nifi.processor.util import StandardValidatorsfrom org.apache.nifi.logging import ComponentLogimport datetimeimport jsonclass PyStreamCallback(StreamCallback): def __init__(self, log): self.log = log def process(self, inputStream, outputStream): text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)&n
Hello, I could use some help... I'm using GetHTTP and InvokeHTTP processors for reading certain indices in a elasticsearch database. The URL to this database is a HTTPS, my question is the following: Is it possible to run these processors leaving the SSL Context Service property as "No value" when you insert in the URL property a HTTPs? If I do this I get the following error (InvokeHTTP processor): InvokeHTTP[id=0e07115a-102e-1182-be33-467ed7f3f784] Yielding processor due to exception encountered as a source processor: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target- Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target- Caused by: sun.security.provid
Hi, Could you please help me with the correct URL format to connect to Sybase using the DBCP connection pool of NiFi? The URL format that I have tried is - jdbc:sybase:Tds://host:port I keep getting this error - failed to create Poolable Connection Factory. Incorrect URL format. Thank you in advance.
Dear Experts,We have some files in our HDFS server and want to get these files and put them into Isilon. We use GetHDFS and PutHDFS for this purpose. But, we get the lease and lock errors, as shown in the picture below. How can we fix these problems?Regards
On CDP Private Cloud 7.1.7, and running without rack awareness (an intentional choice due to some of the older servers running with smaller data disks), I am (as would be expected) seeing this spammed to the namenode logs: 2022-08-24 09:41:50,253 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Not enough replicas was chosen. Reason:{NO_REQUIRED_STORAGE_TYPE=1} Because I'm not logging INFO messages, I can eliminate those entries with something like: hadoop daemonlog -setlevel nnserverhost:9870 org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy WARN What's the best way to persist that across a restart?
Good morning,trying to install Cloudera Manager agents in a RH7.9 (this should be supported as seen in the Cloudera support matrix)I realized that this is not supported ¿!!!?The packages installed in the server are newer as expected (they are from rh7.9) and the cloudera manager is looking for older versions...What crazy is having a support matrix if this is not applied any idea? downgrade dependencies are not considered.
Hi , I'm trying to create a host and after enabling auto-tls .browser shows that certificate is invalid and not secure. I got failed to receive heartbeat from agent in the step of installing agent.
Hi ,We are having four nodes CDP 7.1.7 SP1 cluster. Currently , when a normal hive user creates a database, the database ownership is getting assigned to hive:supergroup in /external/hive which is default. We want the /external/hive database ownership to be assigned to the user who is creating a database. For example:User "Test " is creates one database, the ownership should be like:drwxr-xr-x - test supergroup 0 2022-07-05 17:35 /warehouse/tablespace/exteranl/hive/test_db Kindly let us know if this is possible.
Hi Cloudera gurús, This is my CDP. 3 Master Nodes+3 Worker Nodes HA enabled and testing it. Here is the issue: when I shut down Master 2 some queries are randomly failing showing this: # impala-shell -i haproxy-server.com -q "use dbschema; select * from table_foo limit 10;"Starting Impala Shell without Kerberos authenticationWarning: live_progress only applies to interactive shell sessions, and is being skipped for now.Opened TCP connection to haproxy-server.com:21000Connected to haproxy-server.com:21000Server version: impalad version 3.4.0-SNAPSHOT RELEASE (build 0cadcf7ac76ecec87d9786048db3672c37d41c6f)Query: use dbschemaQuery: select * from table_foo limit 10Query submitted at: 2022-03-23 11:28:37 (Coordinator: http://worker1:25000)ERROR: ImpalaRuntimeException: Unable to initialize the Kudu scan nodeCAUSED BY: AnalysisException: Unable to open the Kudu table: dbschema.table_fooCAUSED BY: NonRecoverableException: cannot complete before timeout: KuduRpc(method=GetTableSchema, tablet=
Hello,i'm using a small 3Master+3workers cluster [Cloudera Manager 7.3.1 (Runtime 7.1.6)]Postgresql HA configured (not embedded)The thing is that I need to restart a tablet server if hte CM host is down, and obviously I can't do it using the CM.is there any way to restart these Kudu tablet servers without CM?NOTE: I've already tried sudo service kudu-tserver stopbutFailed to stop kudu-tserver.service: Unit kudu-tserver.service not loaded.
Im trying to run Apache NiFi on my laptop. I can start the run-nifi.bat But when I try to access http://localhost:8443/. "This page isn’t working localhost didn’t send any data. ERR_EMPTY_RESPONSE"On the same laptop, I'm accessing different NiFi instances running on AWS. With different ports (Production).System: Windows10Please help to connect
Hi Team,we are trying to integrate kafka and hbase into atlas 2.2.0we are not using embedded kafka and hbase , seperately installed in same machine.kafka version : 2.13-2.5.0atlas version : 2.2.0/home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh --topic Maestrotopic3>>>>> /home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh>>>>> /home/ranger/apache-atlas-sources-2.2.0Using Kafka configuration directory [/data/kafka_2.13-2.5.0/config]Log file for import is /var/log/atlas/import-kafka.logPicked up _JAVA_OPTIONS: -Xms1024m -Xmx1024m -XX:MaxPermSize=512mOpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/data/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/data/kafka_2.13-2.5.0/libs/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/St
We are using Nifi TailFile to tail multiple files in a directory and state location is set to "remote". It looks like Nifi is keeping the file state even after the file is deleted. is there a way to remove deleted file from the state, manually or automatically?
Hi all, I am new to Hive, was told below parameter used to improve hive performance, if i were to set and run those code sequence as below, does the sequence matters and correlated to each other? Do we need to put up these code whenever run the query? Or execute once will be sufficient? Thanks. Set hive.exec.parallel = true;set hive.vectorized.execution.enabled = true;set hive.vectorized.execution.reduce.enabled = true;set hive.cbo.enable=true;set hive.compute.query.using.stats=true;set hive.stats.fetch.column.stats=true;set hive.stats.fetch.partition.stats=true;set mapred.compress.map.output = true;set mapred.output.compress= true;SET hive.exec.dynamic.partition = true;SET hive.exec.dynamic.partition.mode = nonstrict;set hive.auto.convert.join = false;set hive.exec.max.dynamic.partitions=100000;set hive.exec.max.dynamic.partitions.pernode=10000;
my cluster is CDH 5.16.2 Please help ##log## [xxx@hdpwxxx1.true.care]:/reserve1>hdfs dfs -put 30.img /tmp/41.img22/08/12 17:49:55 INFO hdfs.DFSClient: Exception while adding a blockorg.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.NotReplicatedYetException): Not replicated yet: /tmp/41.img._COPYING_at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:3688)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3477)at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:694)at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.addBlock(AuthorizationProviderProxyClientProtocol.java:219)at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:507)at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.
Hi guys:It's possible to run Oracle stored proc that return a cursor as SP parameter using only NiFi processors?
I'm installing CDH 6.2 on rhel7 and using Oracle19C as metastore and backend db for all components wherever required. All services start normally, however when trying to login to hue, the page throws error. CDH Parcel Version: CDH-6.2.0-1.cdh6.2.0.p0.967373Oracle DB: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - ProductionOracle Client: 19.3.0.0.0 Tried with three cx_Oracle versions: cx_Oracle-5.2.1-py2.7 / cx_Oracle-5.3-py2.7 / cx_Oracle-7.2.0-py2.7 Environment:Request Method: GETRequest URL: http://<URL>:8888/hue/accounts/login?next=/Django Version: 1.11Python Version: 2.7.5Installed Applications:['django.contrib.auth', 'django_openid_auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.staticfiles', 'django.contrib.admin', 'django_extensions', 'babeldjango', 'desktop', 'axes', 'about', 'beeswax', 'filebrowser', 'help', 'impala', 'jobbrowser', 'jobsub', 'metastore', 'oozie', 'pig', 'proxy', 'rdbms'
hello cloudera community, we identified that there is tablet_id with different volume among the cluster's tablet servers there is tablet_id with 700MB on one tablet server and the same tablet_id with 190MB on another tablet server how can we equalize this volumetry between tablet_id on all tablet servers we believe that with this different volumetry between the tablet_id, some hosts are using a lot of memory, practically 90% of memory of what was defined. 1 - we are using cloudera express 5.16.x2 - a rebalance has already been performed on kudu recently
hello cloudera community, we have installed livy directly on a cloudera cluster host and we are now trying to connect to livy now using kerberos but we are getting the following error: <html><head><meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/><title>Error 401 </title></head><body><h2>HTTP ERROR: 401</h2><p>Problem accessing /sessions. Reason:<pre> Authentication required</pre></p><hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.24.v20180605</a><hr/></body></html> how can we make this connection to retest livy? we are testing the connection with python3 script as follows: import json, pprint, requests, textwrapfrom requests_kerberos import HTTPKerberosAuthhost='http://localhost:8998'data = {'kind': 'spark'}headers = {'Requested-By': 'livy','Content-Type': 'application/json'}auth=HTTPKerberosA
Hey everyone,I want to make an API request for every single day in a period. I have defined a start date and an end date in the Generate Flow File Procesorthe api structure is as followshttps://example.com/interfaces?session=${SessionId}&date_start=${since}&date_end=${since}and now I want to loop with route on attribute and update attribute to request all days in this time period.now the question how can I, the attribute of the start date always increase by one day. with the following expression in the update attribute processore I get an empty field as result${since:toNumber():plus(86400000):format('yyyy-MM-dd')}ResultDoes anyone have an idea what adjusted we
I know Hive only provide equi join. For example, below sql statement.select *from A join B on A.c1 = B.c2where 1=1; But I want to execute Like join Query in Hive. For example, below sql statement.select *from A join B on A.c1 like B.c2where 1=1; Please let me know if you know the solution in Hive.
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.