Member since
03-03-2016
29
Posts
21
Kudos Received
0
Solutions
07-14-2016
12:44 PM
@Ryan Cicak Thanks for your reply. Still I need your opinion and help. I want to go step by step and suppose I only want to install MIT KDC. 1. Install MIT KDC on all 3 nodes as per the hortonworks link you given in earlier answer. Question: Is there any impact on login on 3 nodes, does earlier ppk files work in similar way for all different users including ec2user ? Question: We have few more tools and applications (like Informatica) running on same server. Does it require any changes ? Question: Any impact on Tools connection running on other server/cluster. Does any changes required any where?
What would happen in worst scenario and how to recover from that.
... View more
07-10-2016
08:19 AM
@emaxwell @Sagar Shimpi @pankaj singh @Ryan Cicak Thanks for your answers. I have few queries it would be
great if you can clarify my doubt. As per you answers it is clear that we can configure our
Ambari server without LDAP/Active Directory. And as per answers I need to use
MIT KDC or Free IPA services. We are using AWS cluster, do I need to follow
any other specific process for implementation because we are always using ppk
file for login because of firewall restriction or settings. I understood both MIT KDC and Free IPA services
are 2 different approaches for enabling Kerberos. If I choose MIT KDC then I need to install a new MIT KDC as per link (install_a_new_mit_kdc). Then JCE as per link (installing_the_jce) And Finally Automated setup (the_kerberos_wizard_automated_setup) If I choose freeipa then Need to configure freeipa as per link (ambari-freeipa-service) I belive then I do not require to install MIT
KDC. Then JCE as per link (installing_the_jce) And Finally Manual setup (kerberos_wizard_manual_setup) After these steps what should be my next step as I mentioned
earlier we are using ppk file for login. Those ppk file will work correctly after this or do I need
to configure something else ?? What would be the backup plan if something goes wrong, any
suggestions ??
... View more
06-08-2016
03:02 PM
1 Kudo
Is it possible to implement kerberization in Ambari cluster without LDAP/Active Directory. We do not have access to Active Directory thus can not use the same. And if we implement kerberos without Active Directory then what are all benefits which I enjoy later.
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
06-06-2016
04:37 AM
@Joy Thanks for the details. But unfortunatelly I can not use gmail as its blocked by proxy.
... View more
06-06-2016
04:36 AM
@emaxwell Thanks for your reply. So it means I need to set up PostFix email server on our XYZ server and add that XYZ in relay server. So my XYZ server send email to my external SMTP (relay server) and it will processed further.
... View more
06-03-2016
06:06 PM
Hello All, I need to setup Alert email in Ambari. Now as per initial requirement I need SMTP server with user id and password. I talked to my network team they told me they will add 1 of my cluster server in relay server which allow that server to send email through firewall. Now my question is what all I need to do in server ? Means which software I need to install on that server and what are the steps. Hope you count this as a valid question. 🙂
... View more
Labels:
- Labels:
-
Apache Hadoop
03-19-2016
12:55 PM
2 Kudos
@Artem Ervits Thanks. It is working now.
... View more
03-19-2016
08:03 AM
2 Kudos
I am trying to fetch twitter feed in using hortonworks ambari cluster. But getting below error: 19 Mar 2016 03:54:49,969 WARN [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels:486) - Could not configure channel MemChannel due to: Component has no type. Cannot configure. MemChannel
org.apache.flume.conf.ConfigurationException: Component has no type. Cannot configure. MemChannel
at org.apache.flume.conf.ComponentConfiguration.configure(ComponentConfiguration.java:76)
at org.apache.flume.conf.channel.ChannelConfiguration.configure(ChannelConfiguration.java:32)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:470)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:334)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.access$000(FlumeConfiguration.java:212)
at org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:126)
at org.apache.flume.conf.FlumeConfiguration.<init>(FlumeConfiguration.java:108)
at org.apache.flume.node.PropertiesFileConfigurationProvider.getFlumeConfiguration(PropertiesFileConfigurationProvider.java:193)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:94)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
19 Mar 2016 03:54:49,971 WARN [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid:336) - Agent configuration for 'TwitterAgent' does not contain any valid channels. Marking it as invalid.
19 Mar 2016 03:54:49,971 WARN [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration.validateConfiguration:127) - Agent configuration invalid for agent 'TwitterAgent'. It will be removed.
19 Mar 2016 03:54:49,971 INFO [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration.validateConfiguration:140) - Post-validation flume configuration contains configuration for agents: []
19 Mar 2016 03:54:49,972 WARN [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.getConfiguration:138) - No configuration found for this host:TwitterAgent
19 Mar 2016 03:54:49,981 INFO [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:138) - Starting new configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
19 Mar 2016 03:54:50,177 INFO [conf-file-poller-0] (org.apache.hadoop.metrics2.sink.flume.FlumeTimelineMetricsSink.configure:76) - Context parameters { parameters:{node=masbidw2.usa.corp.ad:6188, type=org.apache.hadoop.metrics2.sink.flume.FlumeTimelineMetricsSink} }
19 Mar 2016 03:54:50,186 INFO [conf-file-poller-0] (org.apache.hadoop.metrics2.sink.flume.FlumeTimelineMetricsSink.start:59) - Starting Flume Metrics Sink flume-conf.jpgflume-env.jpg Also download cloudera source jar file but not luck. Please help
... View more
Labels:
- Labels:
-
Apache Flume
03-10-2016
12:54 PM
1 Kudo
We have 3 server clusters and I am start exploring HIVE. I have already done basis things now for more hands on experience what are the all things I should do. If possible please provide links from where I can download data for processing.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Pig
-
Apache Sqoop
03-04-2016
05:02 AM
1 Kudo
Thanks @ebergenholtz. Query is now returning a data. I was running
wrong query. Now query is returning value but no lineage data found message is showing in input and output but in details it is showing it is import from sqoop. Please help.firstresult.jpgdetails.jpginput.jpg If possible, can you please help me to understand Atlas more
what all the things I can do with Atlas and exactly what configuration, tables,
flow I need to create so it give me desired output. Any suggested tutorial.
... View more