Cloudera Community
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
vincentvc
user rank
Explorer
My Accepted Solutions
Show All

Re: [Flume] Remote spoolDir source monitoring

by Explorer in Support Questions
‎04-24-2017 07:43 PM
‎04-24-2017 07:43 PM
Hi all,  fixed the problem with  agent.sources.spooldir_source1.interceptors=spool_interceptors agent.sources.spooldir_source1.interceptors.spool_interceptors.type=static agent.sources.spooldir_source1.interceptors.spool_interceptors.key=channelselector agent.sources.spooldir_source1.interceptors.spool_interceptors.value=spool ... View more

[Flume] Remote spoolDir source monitoring

by Explorer in Support Questions
‎04-19-2017 08:04 PM
‎04-19-2017 08:04 PM
As topic said, I want to use spoolDir source to monitor folder remotely.   there are some approaches I have tried. 1. Using spoolDir source and kafka sink in client side, then consume that certain topic in the master side then to HDFS.   2. Using spoolDir source and avro sink in client side, then listen from master side with selector.   I have tried approach 1 and it works fine. However, from my perspective the approach 2 is more direct and fast. And my objective is do some multiplexing header mappings in my master side just like:   agent1.sources.avro_source1.selector.type=multiplexing agent1.sources.avro_source1.selector.header=selector agent1.sources.avro_source1.selector.mapping.spool1=ch1 agent1.sources.avro_source1.selector.mapping.spool2=ch2 agent1.sources.avro_source1.selector.mapping.spool3=ch3 However, unlike taildir source there is setting of headers.<filegroupName>.<headerKey>, Spooldir seems don't have such direct setting.    May I know am I missing some configuration parameter of Spooldir to acheive my goal? or is there any alternatives I may try?   Thanks everyone     ... View more
Labels:

Re: [CDH 5.8 Kerberos] Generate Missing Credential...

by Explorer in Support Questions
‎08-16-2016 01:53 AM
‎08-16-2016 01:53 AM
Thanks, you are right. I just discovered that there are two kadmin packages installed for unknown reason. Maybe it is because I changed the PATH variable once and installed the kadmin in other path where is different from the default path setting in CM. I solved the problem with correcting the PATH variable and reinstalling the package. Once again, thank you for you help    ... View more

Re: [CDH 5.8 Kerberos] Generate Missing Credential...

by Explorer in Support Questions
‎08-16-2016 12:44 AM
‎08-16-2016 12:44 AM
Yes, It means that I will use some environment variables which are set in bashrc.   Also, by running ldd on kadmin from script and outside, I found that there is differences between their results.   Command line - root linux-vdso.so.1 => (0x00007ffdb1bad000) libkadm5clnt_mit.so.10 => /usr/local/lib/libkadm5clnt_mit.so.10 (0x00007fd13679f000) libkrb5.so.3 => /usr/local/lib/libkrb5.so.3 (0x00007fd1364ce000) libk5crypto.so.3 => /usr/local/lib/libk5crypto.so.3 (0x00007fd13629d000) libcom_err.so.3 => /usr/local/lib/libcom_err.so.3 (0x00007fd136099000) libkrb5support.so.0 => /usr/local/lib/libkrb5support.so.0 (0x00007fd135e8d000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd135ab2000) libgssrpc.so.4 => /usr/local/lib/libgssrpc.so.4 (0x00007fd135895000) libgssapi_krb5.so.2 => /usr/local/lib/libgssapi_krb5.so.2 (0x00007fd13564c000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fd135430000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd13522c000) /lib64/ld-linux-x86-64.so.2 (0x0000564c783cb000) CM linux-vdso.so.1 => (0x00007fff078ce000) libss.so.2 => /lib/x86_64-linux-gnu/libss.so.2 (0x00007fa458c31000) libkadm5clnt_mit.so.9 => /usr/lib/x86_64-linux-gnu/libkadm5clnt_mit.so.9 (0x00007fa458a1e000) libkrb5.so.3 => /usr/local/lib/libkrb5.so.3 (0x00007fa45874c000) libk5crypto.so.3 => /usr/local/lib/libk5crypto.so.3 (0x00007fa45851c000) libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fa458318000) libkrb5support.so.0 => /usr/local/lib/libkrb5support.so.0 (0x00007fa45810b000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa457d46000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa457b42000) libgssrpc.so.4 => /usr/local/lib/libgssrpc.so.4 (0x00007fa457924000) libgssapi_krb5.so.2 => /usr/local/lib/libgssapi_krb5.so.2 (0x00007fa4576db000) libcom_err.so.3 => /usr/local/lib/libcom_err.so.3 (0x00007fa4574d7000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fa4572bb000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa45709d000) /lib64/ld-linux-x86-64.so.2 (0x000055aa9bce6000)' ... View more

Re: [CDH 5.8 Kerberos] Generate Missing Credential...

by Explorer in Support Questions
‎08-15-2016 06:33 PM
‎08-15-2016 06:33 PM
Thanks for your help. I have tried the full command of CM is trying to run and it perfectly generated the principle of kafka/sb-node1.example.com@ACME.COM.    Also, I have to tell you that I am setting all the kerberos environment as a root user. When I try to print out $USER in the gen_credentials.sh, it shows the User name of "cloudera-scm". I don't know whether it is the cause of the problem. Hope it helps. ... View more

Re: [CDH 5.8 Kerberos] Generate Missing Credential...

by Explorer in Support Questions
‎08-15-2016 02:09 AM
‎08-15-2016 02:09 AM
Thank you for your reply.   Per your request, my environment works with no error with the test command you provided. It successfully adds the principle test/admin to keytab test-admin.keytab with kadmin.local.    Also, It works fine in part of authenticating as principal test/admin@MyRealmName with keytab test-admin.keytab.    Please let me know if you want to know what console log printed while running your test command       ... View more

Re: [CDH 5.8 Kerberos] Generate Missing Credential...

by Explorer in Support Questions
‎08-15-2016 12:39 AM
‎08-15-2016 12:39 AM
dpkg -s libkrb5support0 | grep Version Version: 1.12+dfsg-2ubuntu5.2   dpkg -s libc6 | grep Version Version: 2.19-0ubuntu6.9   it seems that the version is ideal  ... View more

Re: [CDH 5.8 Kerberos] Generate Missing Credential...

by Explorer in Support Questions
‎08-14-2016 07:36 PM
‎08-14-2016 07:36 PM
Thanks for your reply. The version of krb5-kdc and krb5-admin-server is exactly "1.12+dfsg-2ubuntu5.2". Also, I guess that I didn't upgrade my any packages to other versions.   So I guess the repositories is somehow changed. And I restore the repositories with the steps with this link http://askubuntu.com/questions/124017/how-do-i-restore-the-default-repositories. However, I still got the same error while generating Missing Credentials.     ... View more

[CDH 5.8 Kerberos] Generate Missing Credentials Er...

by Explorer in Support Questions
‎08-11-2016 06:41 PM
‎08-11-2016 06:41 PM
Hi,   I am trying to enable kerberos in CDH 5.8 with os of ubuntu 14.04  All are fine until I try to do the part of Generate Missing Credentials Please find the error.   /usr/share/cmf/bin/gen_credentials.sh failed with exit code 127 and output of << + export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games + PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games + export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/sbin + PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/sbin + export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/sbin:/usr/sbin + PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/sbin:/usr/sbin + export KRB5_CONFIG=/var/krb5kdc/krb5.conf + KRB5_CONFIG=/var/krb5kdc/krb5.conf + export KRB5_KDC_PROFILE=/var/krb5kdc/kdc.conf + KRB5_KDC_PROFILE=/var/krb5kdc/kdc.conf + export DEFKTNAME=FILE:/etc/krb5.keytab + DEFKTNAME=FILE:/etc/krb5.keytab + CMF_REALM=ACME.COM + KEYTAB_OUT=/tmp/cmf6659933762842902135.keytab + PRINC=kafka/sb-node1.example.com@ACME.COM + MAX_RENEW_LIFE=432000 + KADMIN='kadmin -k -t /tmp/cmf622833402687775807.keytab -p cloudera-scm/admin@ACME.COM -r ACME.COM' + RENEW_ARG= + '[' 432000 -gt 0 ']' + RENEW_ARG='-maxrenewlife "432000 sec"' + '[' -z /var/krb5kdc/krb5.conf ']' + echo 'Using custom config path '\''/var/krb5kdc/krb5.conf'\'', contents below:' + cat /var/krb5kdc/krb5.conf + kadmin -k -t /tmp/cmf622833402687775807.keytab -p cloudera-scm/admin@ACME.COM -r ACME.COM -q 'addprinc -maxrenewlife "432000 sec" -randkey kafka/sb-node1.example.com@ACME.COM' kadmin: relocation error: /usr/lib/x86_64-linux-gnu/libkadm5clnt_mit.so.9: symbol k5_buf_len, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference   ... View more
Contact Me
Online
Offline
Last Visited
‎06-07-2017 02:41 AM
Community Statistics
Member Since ‎08-10-2016 10:49 PM
Last Visited ‎06-07-2017 02:41 AM
Posts 10