Member since
01-05-2017
9
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1878 | 07-24-2017 11:02 AM |
05-22-2018
09:04 PM
Hello, We need to develop a NiFi dataflow that have a lot of sources that come to the cluster by different ways, most of them by querying different tables of different databases.
For simplicity drawing the dataflow, we're thinking in some processor (or combination of) that we can query different tables using data coming from a flowfile (that includes information about de database controller). Is it possible to do it with the current processors? Do you think it's possible to develop a custom processor that it has this characteristics? Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
09-18-2017
01:38 PM
I advanced in the cluster creation and the alerts disappeared. Thank you very much for your help @Jay SenSharma
... View more
09-15-2017
03:30 PM
It works, thank you! This solves one issue. I can advance in the deployment with the Ambari Cluster Install Wizard. But, I still have the alerts with the old name of the cluster: 14 sep 2017 15:29:20,717 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu
14 sep 2017 15:29:20,717 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu
14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu
14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu
14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu
... View more
09-14-2017
02:04 PM
@Jay SenSharma It works! Thank you! But I still have the alerts with the name of the old cluster. Do you know how to resolve this, please? 14 sep 2017 15:29:20,717 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu
14 sep 2017 15:29:20,717 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu
14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu
14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu
14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu
... View more
09-14-2017
11:43 AM
The first option I tried yesterday and It didn't work. So, we go with the other option. Here you can see the results of the queries $ psql -U ambari ambari
Contraseña para usuario ambari:
psql (9.2.21)
Digite «help» para obtener ayuda.
ambari=> SELECT repo_version_id, stack_id, version, display_name FROM repo_version;
repo_version_id | stack_id | version | display_name
-----------------+----------+---------+--------------
(0 filas)
ambari=> SELECT * FROM clusters;
cluster_id | resource_id | upgrade_id | cluster_info | cluster_name | provisioning_state | security_type | desired_cluster_state | desired_stack_id
------------+-------------+------------+--------------+--------------+--------------------+---------------+-----------------------+------------------
(0 filas)
ambari=> SELECT * FROM cluster_version;
id | repo_version_id | cluster_id | state | start_time | end_time | user_name
----+-----------------+------------+-------+------------+----------+-----------
(0 filas)
ambari=> SELECT * FROM host_version;
id | repo_version_id | host_id | state
----+-----------------+---------+-------
(0 filas)
We have 0 rows.
... View more
09-14-2017
11:00 AM
Thanks Jay for your answer. I have a little advance. Now, I have “Success” status but the wizard tells “ Please wait while the hosts are being checked for potential problems.” ambari-confirm-hosts-success-but-waiting.png And I found continuously in /var/log/ambari-server/ambari-server.log 14 sep 2017 12:46:49,483 WARN [ambari-action-scheduler] ExecutionCommandWrapper:185 - Unable to lookup the cluster by ID; assuming that there is no cluster and therefore no configs for this execution command: Cluster not found, clusterName=clusterID=-1
14 sep 2017 12:46:49,484 WARN [ambari-action-scheduler] ExecutionCommandWrapper:185 - Unable to lookup the cluster by ID; assuming that there is no cluster and therefore no configs for this execution command: Cluster not found, clusterName=clusterID=-1
14 sep 2017 12:46:49,484 WARN [ambari-action-scheduler] ActionScheduler:316 - Exception received
java.lang.NullPointerException
at org.apache.ambari.server.actionmanager.Stage.getStartTime(Stage.java:630)
at org.apache.ambari.server.actionmanager.ActionScheduler.processHostRole(ActionScheduler.java:1065)
at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:461)
at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
at java.lang.Thread.run(Thread.java:745)
The alerts with the name of the first cluster (cordisclu) are still appearing.
... View more
09-14-2017
09:50 AM
I had an
issue installing HDF with Ambari and accidentally I closed Ambari UI. When I
try to continue with the cluster, the previous configuration was lost and it’s
impossible to pass the “Confirm Hosts” phase. The hosts are in “Preparing”
Status always without advance. ambari-confirm-hosts-preparing.png Additionally,
I found in /var/log/ambari-server/ambari-server.log 1 ) Alerts with the name of the first
cluster (that it wasn’t finished) cordisclu. Now the cluster will have a new
name. 14 sep 2017 10:59:00,526 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu
14 sep 2017 10:59:00,526 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert infra_solr for an invalid cluster named cordisclu
14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu
14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,527 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu
14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,528 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert logsearch_ui for an invalid cluster named cordisclu
14 sep 2017 10:59:00,528 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu
14 sep 2017 10:59:00,528 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,528 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu
14 sep 2017 10:59:00,529 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu
14 sep 2017 10:59:00,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,530 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,529 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,530 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu
14 sep 2017 10:59:00,530 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu
14 sep 2017 10:59:00,531 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu
14 sep 2017 10:59:00,531 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu
14 sep 2017 10:59:01,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_collector_process for an invalid cluster named cordisclu
14 sep 2017 10:59:01,528 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_collector_autostart for an invalid cluster named cordisclu
14 sep 2017 10:59:01,528 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu
14 sep 2017 10:59:01,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu
14 sep 2017 10:59:01,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu
14 sep 2017 10:59:01,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu
14 sep 2017 10:59:01,530 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu
14 sep 2017 10:59:01,530 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_collector_hbase_master_process for an invalid cluster named cordisclu
14 sep 2017 10:59:10,526 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert grafana_webui for an invalid cluster named cordisclu
I found a reference that does not work in my case (https://community.hortonworks.com/questions/41553/unable-to-process-alert-for-an-invalid-cluster-nam.html) I think that
deleting cache and data information in the ambari-agent folder (/var/lib/ambari-agent/) and restarting ambari-agent could be the
solution, but this does not resolve my main issue. 2) Error when executing bootstrap 14 sep 2017 10:54:16,453 INFO [Thread-1112] BSRunner:189 - Kicking off the scheduler for polling on logs in /var/run/ambari-server/bootstrap/7
14 sep 2017 10:54:16,454 INFO [Thread-1112] BSRunner:372 - Error executing bootstrap Cannot create /var/run/ambari-server/bootstrap
14 sep 2017 10:54:16,455 ERROR [Thread-1112] BSRunner:441 - java.io.FileNotFoundException: /var/run/ambari-server/bootstrap/7/srvifsidsp01.xxxxxxxxxxx.done (No existe el fichero o el directorio)
14 sep 2017 10:54:16,456 WARN [Thread-1112] BSRunner:401 - File does not exist: /var/run/ambari-server/bootstrap/7/sshKey Note: "No existe el fichero o el directorio" means "File or directory does not exist" in Spanish. More information: OS: CentOS 7 Ambari version: 2.5.1.0
... View more
Labels:
- Labels:
-
Apache Ambari
08-25-2017
07:24 AM
I had the same problem with PublishKafka_0_10 processor and I found that there is an open issue in JIRA https://issues.apache.org/jira/browse/NIFI-3957 Meanwhile I solved the problem controlling the size of the FlowFile before with RouteOnAttribute
... View more
07-24-2017
11:02 AM
1 Kudo
Hi, I had the same issue and I resolved it incluiding proxy information -Dhttps.proxyHost=proxy_name -Dhttps.proxyPort=xxxx when bootstrap-storage.sh call the class ${TABLE_INITIALIZER_MAIN_CLASS} (functions dropTables, createTables, checkStorageConnection).
... View more