Member since
07-19-2018
613
Posts
100
Kudos Received
117
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3211 | 01-11-2021 05:54 AM | |
2292 | 01-11-2021 05:52 AM | |
6144 | 01-08-2021 05:23 AM | |
5671 | 01-04-2021 04:08 AM | |
26229 | 12-18-2020 05:42 AM |
08-17-2018
12:00 PM
@Errico
Malatesta good job, glad you were able to work it out!
... View more
08-15-2018
02:39 PM
@seninus glad you got it working. Please click accept on the main answer please, it helps close the question and gives me some reputation points. ;O)
... View more
08-06-2018
01:48 PM
@Steven Matison Hi Steven, Thanks For Reply.
1. Kafka Broker: hostname:6667 - many times documentation shows a different port, use 6667=== Used same port 6667 2. Topic Name: Mine is a variable, you can enter anything here: "test", etc====Entered fixed topic 3. Kafka Key: Not required you can leave it empty.====Its empty
We are using Kerberized Kafka. Keytab and principal have defined in properties. PFA for properties. nifi-property.png
Thanks, Gitanjali
... View more
07-23-2018
11:55 AM
1 Kudo
First, congrats on getting started with NiFi and not being afraid to tackle one of the harder processors: ExecuteScript. My advice is to start small, get a working script that maybe doesnt do what you want, but does actually complete and return SUCCESS in your NiFi flow. From there you can start building it up and testing step by step. I also recommend that you tail the NIFI Log during all of your work so that you can see any errors or information from each test/step. My last peice of advice for an advanced script would be to manually execute your script to ensure that its logic is working outside of the execution from within NiFi. You may find the following links very helpful: https://community.hortonworks.com/articles/75032/executescript-cookbook-part-1.html https://community.hortonworks.com/articles/75545/executescript-cookbook-part-2.html https://community.hortonworks.com/articles/75545/executescript-cookbook-part-3.html If my answer is helpful, please choose ACCEPT.
... View more
02-13-2019
09:20 AM
Why do we synch users into Apache ranger? It would be great if someone can explain the reason along with one example.
... View more
07-20-2018
11:10 AM
@Geoffrey Shelton Okot those are the correct repo files and I realized what I had done wrong. Thanks!
... View more
09-20-2018
03:07 AM
Nevermind, used the PDF docs (https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.2.0/installing-hdf/hdf-installing-hdf.pdf) with new VMs and all is good now.
... View more
08-01-2018
11:31 AM
To properly troubleshoot elasticsearch you first need to make sure that elasticsearch is actually running correctly. Go to QuickLinks and open Elasticsearch Health. Status must be green. Tail your elasticsearch master node log file while restarting to see if there are any issues: tail -f /var/log/elasticsearch/elasticsearch.log Additionally you can edit /etc/elasticsearch/log4j2.properties and set logger.action.level to debug for more verbose logging. Based on the log output, you will likely need to adjust config settings in Advanced elastic-site from Ambari. Here are my settings for a Master Node + 2 Data Nodes: bootstrap_memory_lock
true cluster_name
elasticsearch cluster_routing_allocation_disk_threshold_enabled
true cluster_routing_allocation_disk_watermark_high
0.99 cluster_routing_allocation_disk_watermark_low
.97 cluster_routing_allocation_node_concurrent_recoveries
4 discovery_zen_fd_ping_interval
15s discovery_zen_fd_ping_retries
5 discovery_zen_fd_ping_timeout
60s discovery_zen_ping_timeout
3s expected_data_nodes
0 gateway_recover_after_data_nodes
1 http_cors_enabled
"true" http_port
9200 index_merge_scheduler_max_thread_count
5
index_number_of_replicas
2 index_number_of_shards
4
index_refresh_interval
1s index_translog_flush_threshold_size
5g indices_cluster_send_refresh_mapping
false indices_fielddata_cache_size
25% indices_memory_index_buffer_size
10% indices_memory_index_store_throttle_type
none masters_also_are_datanodes
"true" network_host
[ 0.0.0.0 ]
network_publish_host
[] path_data
"/hadoop/elasticsearch/es_data" recover_after_time
15m threadpool_bulk_queue_size
3000 threadpool_index_queue_size
1000 transport_tcp_port
9300
zen_discovery_ping_unicast_hosts
[ "fqdn.hostname1.com", "fqdn.hostname2.com", "fqdn.hostname3.com" ]
... View more
08-17-2018
11:52 AM
@Guven
Guvenal this is a great custom processor, you should create a HOWTO Article here about how to use it and or create one about how to make Custom Processors.
... View more
- « Previous
- Next »