Member since
06-26-2015
502
Posts
121
Kudos Received
113
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
228 | 09-20-2022 03:33 PM | |
749 | 09-19-2022 04:47 PM | |
392 | 09-11-2022 05:01 PM | |
492 | 09-06-2022 02:23 PM | |
604 | 09-06-2022 04:30 AM |
08-11-2022
03:58 AM
@rafy Try this: ${DATE_TIME:toDate('yyyy-MM-dd HH:mm:ss.SSSSSSSSS'):format('dd/MM/yyyy HH:mm:ss.SSSSSS')} Cheers, André
... View more
08-11-2022
03:48 AM
1 Kudo
@romi , You can try to connect to some NiFi API endpoint. If it succeed, it's an indication that the service is up. For example: $ curl -k "https://mynifi:8443/nifi-api/access"
{"accessStatus":{"status":"UNKNOWN","message":"Access Unknown: Certificate and Token not found."}} If the command above returned anything different, you could send an alert. Cheers, André
... View more
08-10-2022
01:25 AM
Then the service will be unavailable until you recover at least one of them.
... View more
08-09-2022
02:52 PM
@noekmc , How are you restarting Cloudera Manager? This restart must be done from the command line, on the CM host: sudo systemctl restart cloudera-scm-server Cheers, André
... View more
08-09-2022
02:38 PM
@Big-dAta , A quorum in a ZooKeeper ensemble is a group of servers with more than half of the total number of servers in the ensemble. So for an ensemble of 5 servers, the quorum is 3 servers. As long as there are 3 servers available an election can always be carried out. So, in your example, the 4 remaining servers will talk to each other and decide who will be the new leader. Cheers, André
... View more
08-08-2022
11:25 PM
@ho_ddeok , Hive 1 is a very old version and and equi-join was probably the only option available back then. As @nramanaiah mentioned, you should consider upgrading to the latest version, where this statement works. Nevertheless, you can try the following alternative query. I'm not sure if it works, though, since I don't have a Hive 1 environment to test this: select *
from A cross join B
where A.c1 like B.c2; Cheers, André
... View more
08-08-2022
07:12 PM
1 Kudo
@RB764 , Ok, I see. In the case of running JS code on the browser client, it makes sense to see the errors that you are seeing. That's exactly what CORS restrictions are designed for. Are you loading this script from a file locally on your computer or is this part from a web application that you have running on a server? If this is local to your computer, I would recommend that you use some other language to interact with NiFi, since running this locally on a browser has limitations. In Python, for example, you can use the nipyapi module, which makes it really easy to interact with the NiFi API: from nipyapi import config, security
from nipyapi.nifi.apis.flow_api import FlowApi
config.nifi_config.host = 'https://<<server host:port>>/nifi-api'
security.set_service_ssl_context(service='nifi', ca_file='<</path/to/truststore.pem>>')
security.service_login(service='nifi', username='<<user>>', password='<<pwd>>')
api = FlowApi()
summary = api.get_cluster_summary() If what you are trying to do is part of a web application, the interaction between application and NiFi should happen on the server side instead of the browser. This would also eliminate any sort of CORS issues. Cheers, André
... View more
08-07-2022
10:41 PM
1 Kudo
@RB764 , What are you using to access the NiFi REST API. Could you give us an example of what your API call looks like? It's possible to access the API from a client in a different domain without any reconfiguration. Cheers, André
... View more
08-07-2022
03:14 PM
@chitrarthasur , "running" is not the state of the connection, but its name. Your "RouteOnAttribute_EndpointRunning" must have a user-defined relationship called "running". Cheers, André
... View more
08-04-2022
05:28 AM
I don't think there is. At least, I can't think of one.
... View more
08-04-2022
12:34 AM
Sorry, I'm out of ideas.
... View more
08-03-2022
07:33 PM
1 Kudo
Ahh! Good catch, @ChuckE !! So simple!
... View more
08-03-2022
04:24 PM
@AmiJhones , I've answer a similar question before: https://community.cloudera.com/t5/Support-Questions/Json-Jolt-to-Remove-All-Nulls-from-Json/td-p/336008 Cheers, André
... View more
08-03-2022
03:51 PM
@samrathal , Could you please install the gcloud CLI, if you haven't done it yet, and run the following command to test your credential file? ./google-cloud-sdk/bin/gcloud auth login --cred-file /home/apache/nifi/gcp.json Please let me know the results. Cheers, André
... View more
08-03-2022
04:56 AM
@ftrg , You don't need the tar.gz file. You can install using the -bin.zip file (https://dlcdn.apache.org/nifi/1.17.0/nifi-1.17.0-bin.zip). Just download this file and unzip it with: unzip nifi-1.17.0-bin.zip Cheers, André
... View more
08-03-2022
04:51 AM
@AbhishekSingh , The CaptureChangeMySQL processor acts as a MySQL replication slave. It requires the REPLICATION_SLAVE privilege in MySQL. Following the steps in the documentation and create a separate user to use in your processor. Cheers, André Cheers, André
... View more
08-02-2022
11:55 PM
Could you run these commands and try again: chown nifi:nifi /home/apache/nifi/gcp.json
chmod 400 /home/apache/nifi/gcp.json Also, would you be able to take a screenshot of the content of your gcp.json, mask out the sensitive values and post it here? I'd like to check the format of your credentials file. Cheers, Andr'e
... View more
08-02-2022
10:45 PM
@samrathal , Could you please run this command below and report the results? namei -l /home/apache/nifi/gcp.json Cheers, André
... View more
08-02-2022
10:43 PM
I'm also curious about the performance difference. If you could report your results here, I'd appreciate! Cheers, André
... View more
08-02-2022
10:20 PM
@ChuckE , You can use a JoltTransformRecord processor to perform the conversion to JSON and, at the same time, add the root node back to it. The JOLT specification I used is this: {
"*": "a.&"
} Cheers, André
... View more
08-02-2022
09:56 PM
@samrathal It seems that you set two properties in your controller service: "Service Account JSON File" and also "Service Account JSON". You should specify only one. If you are using the file one, click on the the other property and delete its contents. Make sure it shows "No value set". Then try again. Cheers, André
... View more
08-02-2022
09:50 PM
2 Kudos
@SandyClouds , You don't need a MySQL replication server. What I meant is that the CaptureChangeMySQL act as a replication client, and needs the same privileges as one. I tested the exact same commands that you specified above and I was able to connect to the database from the command line: [root@cdp ~]# mysql -u replication -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 8.0.26 Source distribution
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> I was also able to configure the CaptureChangeMySQL processor to capture changes without any other privileges being granted: Which version of MySQL are you using? There seems to be some issue with your user creation. The connection from the command line should work. What was the command you used to connect to the db using the replication user? Could you please provide the configuration of your processor? Cheers, André
... View more
08-01-2022
02:57 PM
@ChuckE Do your XML messages conform to defined a schema? Cheers, André
... View more
08-01-2022
06:22 AM
1 Kudo
@Brenigan , 1. It depends on the context and the level of &n. In the example above, &1 return the element in the transports array (e.g. "PUSH"), while &2 returns the numeric index of that element in the array (e.g. 0). 2. &4 and &2 are numeric array indexes. outer[&4] means that the output will be in the &4 position of an array called outer. That element of the array will have and attribute called inner and the &2 position of the inner array will have two attributes, t and etc, with the specified values. Cheers, André
... View more
08-01-2022
01:18 AM
What's the version of your MySQL database? It seems you're using a version that's not compatible with Sentry. Cheers, André
... View more
08-01-2022
12:06 AM
Could you please provide the full stack trace? The one above is incomplete.
... View more
08-01-2022
12:03 AM
Unfortunately, this doesn't tell much. Any luck with the Sentry logs?
... View more
07-31-2022
06:35 PM
@AbhishekSingh , 1. No, the NiFi Toolkit is not mandatory. If you're familiar with TLS certificates and NiFi administration you can set up and manage the cluster without it. The toolkit makes some of the activities easier, though. 2. What do mean by " flowfile change restriction "? Cheers, André
... View more
07-31-2022
06:33 PM
@Domo , I'm not familiar with Cassandra to answer the question. I see, though, that there's a Jira for the upgrade of the Cassandra driver to version 4, but that hasn't got traction yet. You might want to enquire in the Jira itself. Cheers, André
... View more
07-31-2022
06:29 PM
@AbhishekSingh , The JSON datatype in MySQL is stored in an optimized binary format. When you do CDC you only get the binary changes associated with that column. If you are processing this within MySQL, you have functions to deal with the JSON datatype. In NiFi, though, I'm afraid this is not possible. If you want to get the JSON changes as text you should convert the column data type to a string-type. Cheers, André
... View more