Member since
07-19-2018
613
Posts
101
Kudos Received
117
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 5688 | 01-11-2021 05:54 AM | |
| 3812 | 01-11-2021 05:52 AM | |
| 9488 | 01-08-2021 05:23 AM | |
| 9289 | 01-04-2021 04:08 AM | |
| 38620 | 12-18-2020 05:42 AM |
05-20-2020
11:40 AM
1 Kudo
@vikrant_kumar24 1. Yes, sorry for some reason they didnt get saved in that version of the template. I will update. You should see or set it like this: 2. Yes, you will need to enter the Remote Url, and maybe more properties. You also need to confirm that the json content in flow file is correct. Mine is just an example. I do not know what service you are connect too. That service may require username, passwords, tokens, headers, etc. The configuration for InvokeHttp will depend on these. If you prefer to share more details about the service you are connecting in PM we can discuss in private. Once you have the response from InvokeHttp, if the response is json, you will again do EvaluateJson, grab the values you want for the new Long/Lat, and replace or overwrite the existing attributes and continue downstream with your flow. 3. Converting CSV to JSON is pretty typical. The CSV->JSON Method you are a working on is just fine. You should be able to link the flowfile from that, to EvaluateJson in the flow. My example is just a demo to show you the concepts of EvaluateJson, UpdateAttribute, and AttributesToJson.
... View more
05-18-2020
04:55 AM
1 Kudo
Great answer many thanks. For the record, I had the data in a FlowFile. I needed a bit of regex magic to remove the quotation marks after unescaping.
... View more
05-16-2020
05:33 AM
@johndcal A namespace is not required within the avro schema source in Schema Registry. In the context of avro spec. In order to create an avro schema in the Schema Registry, you have to send the first call to create the schema entity. The next call is then to add the actual avro schema to the existing entity. This is just the behavior of the Schema Registry. You can find some lessons I created in how to use the registry: https://community.cloudera.com/t5/Community-Articles/Using-the-Schema-Registry-API/ta-p/286194 I also have an article showing how to fully automate the creation of Avro Schemas from CSV file (column name and data type) using the Schema Registry, Hive, and NiFI: https://community.cloudera.com/t5/Community-Articles/How-to-automate-creation-of-Avro-and-Hive-Schemas-using-NiFi/ta-p/293183 If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven @ DFHZ
... View more
05-13-2020
08:27 AM
@satishjan1 The initial question is asking about setting the hostname. The information you reference is telling you to do that, but for a different operating system. My first response was telling you how to do it for RHEL. For your next question, you do not have to set the hostname in /etc/sysconfig/network, you have to do it the way required for your operating system. See Above. The hostname must be set, and persist after reboot. If you do not set the hostname before installing the cluster you will have unmentionable problems with services and components later on down the road.
... View more
05-09-2020
09:27 AM
@michaelli You can use the CM API to fetch the service configurations by executing below command from terminal # curl -u admin_uname:admin_pass "http://cm_server_host:7180/api/v19/cm/deployment" The password should be present under the name : hive_metastore_database_password Refer: https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/cm_intro_api.html#concept_dnn_cr5_mr__section_ztf_lyt_xr Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
04-30-2020
02:07 PM
Thanks for the revert mate. Much Appreciated.
... View more
04-28-2020
05:15 AM
@Damian_S You cannot delete and recreate a hadoop cluster in this manner. The data in the metastore is very specific to the previous installation. A proper migration will back up the required artifacts from the original cluster, and migrate them to a new existing ambari cluster that has already completed the initial cluster installation.
... View more
04-28-2020
05:09 AM
@san_re The configuration of Remote Url in InvokeHTTP should be as follows: /j_security_check?j_username=${j_username}&j_password=${j_password} Notice the credential values are passed to the url with the ?. This is the only change in the processor config. Here is a demo flow: GenerateFlowFile - starts flow for testing UpdateAttribute - sets user, pass,and mime type attributes needed in InvokeHttp InvokeHttp - sends POST You can find this template on my GitHub to inspect the entire flow: https://github.com/steven-dfheinz/NiFi-Templates/blob/master/InvokeHttp_Demo.xml
... View more