About juan_vares

juan_vares · ‎05-22-2018

Hello, We need to develop a NiFi dataflow that have a lot of sources that come to the cluster by different ways, most of them by querying different tables of different databases. For simplicity drawing the dataflow, we're thinking in some processor (or combination of) that we can query different tables using data coming from a flowfile (that includes information about de database controller). Is it possible to do it with the current processors? Do you think it's possible to develop a custom processor that it has this characteristics? Thanks

juan_vares · ‎09-18-2017

I advanced in the cluster creation and the alerts disappeared. Thank you very much for your help @Jay SenSharma

juan_vares · ‎09-15-2017

It works, thank you! This solves one issue. I can advance in the deployment with the Ambari Cluster Install Wizard. But, I still have the alerts with the old name of the cluster: 14 sep 2017 15:29:20,717 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu 14 sep 2017 15:29:20,717 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu 14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu 14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu 14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu

juan_vares · ‎09-14-2017

@Jay SenSharma It works! Thank you! But I still have the alerts with the name of the old cluster. Do you know how to resolve this, please? 14 sep 2017 15:29:20,717 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu 14 sep 2017 15:29:20,717 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu 14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu 14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu 14 sep 2017 15:29:38,717 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu

juan_vares · ‎09-14-2017

The first option I tried yesterday and It didn't work. So, we go with the other option. Here you can see the results of the queries $ psql -U ambari ambari Contraseña para usuario ambari: psql (9.2.21) Digite «help» para obtener ayuda. ambari=> SELECT repo_version_id, stack_id, version, display_name FROM repo_version; repo_version_id | stack_id | version | display_name -----------------+----------+---------+-------------- (0 filas) ambari=> SELECT * FROM clusters; cluster_id | resource_id | upgrade_id | cluster_info | cluster_name | provisioning_state | security_type | desired_cluster_state | desired_stack_id ------------+-------------+------------+--------------+--------------+--------------------+---------------+-----------------------+------------------ (0 filas) ambari=> SELECT * FROM cluster_version; id | repo_version_id | cluster_id | state | start_time | end_time | user_name ----+-----------------+------------+-------+------------+----------+----------- (0 filas) ambari=> SELECT * FROM host_version; id | repo_version_id | host_id | state ----+-----------------+---------+------- (0 filas) We have 0 rows.

juan_vares · ‎09-14-2017

Thanks Jay for your answer. I have a little advance. Now, I have “Success” status but the wizard tells “ Please wait while the hosts are being checked for potential problems.” ambari-confirm-hosts-success-but-waiting.png And I found continuously in /var/log/ambari-server/ambari-server.log 14 sep 2017 12:46:49,483 WARN [ambari-action-scheduler] ExecutionCommandWrapper:185 - Unable to lookup the cluster by ID; assuming that there is no cluster and therefore no configs for this execution command: Cluster not found, clusterName=clusterID=-1 14 sep 2017 12:46:49,484 WARN [ambari-action-scheduler] ExecutionCommandWrapper:185 - Unable to lookup the cluster by ID; assuming that there is no cluster and therefore no configs for this execution command: Cluster not found, clusterName=clusterID=-1 14 sep 2017 12:46:49,484 WARN [ambari-action-scheduler] ActionScheduler:316 - Exception received java.lang.NullPointerException at org.apache.ambari.server.actionmanager.Stage.getStartTime(Stage.java:630) at org.apache.ambari.server.actionmanager.ActionScheduler.processHostRole(ActionScheduler.java:1065) at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:461) at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310) at java.lang.Thread.run(Thread.java:745) The alerts with the name of the first cluster (cordisclu) are still appearing.

juan_vares · ‎09-14-2017

I had an issue installing HDF with Ambari and accidentally I closed Ambari UI. When I try to continue with the cluster, the previous configuration was lost and it’s impossible to pass the “Confirm Hosts” phase. The hosts are in “Preparing” Status always without advance. ambari-confirm-hosts-preparing.png Additionally, I found in /var/log/ambari-server/ambari-server.log 1 ) Alerts with the name of the first cluster (that it wasn’t finished) cordisclu. Now the cluster will have a new name. 14 sep 2017 10:59:00,526 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu 14 sep 2017 10:59:00,526 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert infra_solr for an invalid cluster named cordisclu 14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu 14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,527 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu 14 sep 2017 10:59:00,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,528 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert logsearch_ui for an invalid cluster named cordisclu 14 sep 2017 10:59:00,528 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu 14 sep 2017 10:59:00,528 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,528 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu 14 sep 2017 10:59:00,529 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu 14 sep 2017 10:59:00,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,530 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,529 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,530 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu 14 sep 2017 10:59:00,530 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu 14 sep 2017 10:59:00,531 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu 14 sep 2017 10:59:00,531 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu 14 sep 2017 10:59:01,527 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_collector_process for an invalid cluster named cordisclu 14 sep 2017 10:59:01,528 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_collector_autostart for an invalid cluster named cordisclu 14 sep 2017 10:59:01,528 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ambari_agent_disk_usage for an invalid cluster named cordisclu 14 sep 2017 10:59:01,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert kafka_broker_process for an invalid cluster named cordisclu 14 sep 2017 10:59:01,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert nifi_status for an invalid cluster named cordisclu 14 sep 2017 10:59:01,529 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert zookeeper_server_process for an invalid cluster named cordisclu 14 sep 2017 10:59:01,530 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_monitor_process for an invalid cluster named cordisclu 14 sep 2017 10:59:01,530 ERROR [alert-event-bus-1] AlertReceivedListener:480 - Unable to process alert ams_metrics_collector_hbase_master_process for an invalid cluster named cordisclu 14 sep 2017 10:59:10,526 ERROR [alert-event-bus-2] AlertReceivedListener:480 - Unable to process alert grafana_webui for an invalid cluster named cordisclu I found a reference that does not work in my case (https://community.hortonworks.com/questions/41553/unable-to-process-alert-for-an-invalid-cluster-nam.html) I think that deleting cache and data information in the ambari-agent folder (/var/lib/ambari-agent/) and restarting ambari-agent could be the solution, but this does not resolve my main issue. 2) Error when executing bootstrap 14 sep 2017 10:54:16,453 INFO [Thread-1112] BSRunner:189 - Kicking off the scheduler for polling on logs in /var/run/ambari-server/bootstrap/7 14 sep 2017 10:54:16,454 INFO [Thread-1112] BSRunner:372 - Error executing bootstrap Cannot create /var/run/ambari-server/bootstrap 14 sep 2017 10:54:16,455 ERROR [Thread-1112] BSRunner:441 - java.io.FileNotFoundException: /var/run/ambari-server/bootstrap/7/srvifsidsp01.xxxxxxxxxxx.done (No existe el fichero o el directorio) 14 sep 2017 10:54:16,456 WARN [Thread-1112] BSRunner:401 - File does not exist: /var/run/ambari-server/bootstrap/7/sshKey Note: "No existe el fichero o el directorio" means "File or directory does not exist" in Spanish. More information: OS: CentOS 7 Ambari version: 2.5.1.0

juan_vares · ‎08-25-2017

I had the same problem with PublishKafka_0_10 processor and I found that there is an open issue in JIRA https://issues.apache.org/jira/browse/NIFI-3957 Meanwhile I solved the problem controlling the size of the FlowFile before with RouteOnAttribute

juan_vares · ‎07-24-2017

Hi, I had the same issue and I resolved it incluiding proxy information -Dhttps.proxyHost=proxy_name -Dhttps.proxyPort=xxxx when bootstrap-storage.sh call the class ${TABLE_INITIALIZER_MAIN_CLASS} (functions dropTables, createTables, checkStorageConnection).

Online	Offline
Last Visited	‎07-16-2018 06:22 PM

Date Registered	‎01-05-2017 09:51 AM
Last Visited	‎07-16-2018 06:22 PM
Total Messages Posted	9
Total Kudos Received	1

Re: Error starting registry in Ambari (HDF 3.0)

NiFi Processor multi query using incoming connecti...

Re: Ambari: Status "Preparing" in confirmation of ...

Re: Ambari: Status "Preparing" in confirmation of ...

Re: Ambari: Status "Preparing" in confirmation of ...

Re: Ambari: Status "Preparing" in confirmation of ...

Re: Ambari: Status "Preparing" in confirmation of ...

Ambari: Status "Preparing" in confirmation of host...

Re: NiFi PublishKafka_0_10 processor

Re: Error starting registry in Ambari (HDF 3.0)