Member since
10-10-2017
13
Posts
4
Kudos Received
0
Solutions
10-29-2018
09:18 AM
1 Kudo
Greetings Doug, I just ran into this same issue and resolved it. I wanted to give you some context first. The Telemetry Publushing Service is not a necessary component of a Cloudera cluster deploy - it is to supplement an add-on service called "Workload Experience Manager or Workload XM". You can safely delete the service and it will not affect cluster operations If Workload XM is a functionality you wish to leverage then you will need to take a couple steps to enable the Telemetry Publisher to send telemetry to Cloudera. Here is the Guide I Followed: https://www.cloudera.com/documentation/wxm/latest/topics/wxm_setup.html Step 1. Get Altus Credentials Step 2. Add Altus Credentials to Cloudera Manager Step 3. Add the Telemetry Publisher Service Role Basically you need to log into your Altus account and generate a key pair for WXM to leverage - you will then need to add this account in Cloudera Manager as an external account under the altus tab. That can be done here. https://altus.cloudera.com/wxm/home.html#/ Next in Cloudera Manager: Administration -> External Accounts -> click "Altus Credentials" tab and add the account. You may need to configure the account as your Altus account in Administration -> Settings -> "Telemetry Altus Account" The docs don't say this but I had to restart Cloudera Manager to get this setting to take effect - I could not start the Telemetry Publisher until after I restarted Cloudera Manager To access Workload XM, perform the following steps: Log in to the Workload XM console: wxm.cloudera.com/
... View more
10-10-2017
09:03 AM
1 Kudo
I am able to do this by running (or adding this to your ~/.bashrc) export PATH=/opt/cloudera/parcels/Anaconda/bin:$PATH After this conda, pip, and python commands all defaulted to the Anaconda version.
... View more
03-14-2017
03:56 AM
1 Kudo
I have found a solution to this provided by another user here: https://community.hortonworks.com/questions/20719/sqoop-to-sql-server-with-integrated-security.html Basically if you switch to the jtds driver which you can download here: http://jtds.sourceforge.net/ Per Rajendra Manjunath " Sqoop SQL Server data import to HDFS worked with manual parametric the authentication(using windows credential) with added parameter on the SQL Server JDBC driver, as integrated security is not supported by the SQL driver as of now due to the Kerberos authentication(Delegated tokens distributed over cluster while running MR job). So we need to pass the windows authentication with password and with the integrated security disabled mode to import the data to the system. As normal SQL server driver does not support, so I had used the jtds.jar and the different driver class to pull the data to the Hadoop Lake. Sample Command I tried on the server as follows, sqoop import --table Table1 --connect "jdbc:jtds:sqlserver://<Hostname>:<Port>;useNTLMv2=true;domain=<WindowsDomainName>;databaseName=XXXXXXXXXXXXX" \ --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX --password 'XXXXXXX' \ --verbose --target-dir /tmp/33 -m 1 -- --schema dbo " Here are some examples that worked for me: # List databases sqoop list-databases --connect "jdbc:jtds:myactivedirectorydomain.com" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX -P # List tables sqoop list-tables --connect "jdbc:jtds:myactivedirectorydomain.com;databaseName=DATABASENAMEHERE" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username jmiller.admin -P # Pull data example sqoop import --table TABLENAMEHERE --connect "jdbc:jtds:myactivedirectorydomain.com;databaseName=DATABASENAMEHERE" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX -P --fields-terminated-by '\001' --target-dir /user/XXXXX/20170313 -m 1 -- --schema dbo Note* In the above example you need to change the username to your username and database name in the list-tables or pull to the one you need (note the AD account you use will require access to the data).
... View more
03-14-2017
03:55 AM
I have found a solution to this provided by another user here: https://community.hortonworks.com/questions/20719/sqoop-to-sql-server-with-integrated-security.html Basically if you switch to the jtds driver which you can download here: http://jtds.sourceforge.net/ Per Rajendra Manjunath " Sqoop SQL Server data import to HDFS worked with manual parametric the authentication(using windows credential) with added parameter on the SQL Server JDBC driver, as integrated security is not supported by the SQL driver as of now due to the Kerberos authentication(Delegated tokens distributed over cluster while running MR job). So we need to pass the windows authentication with password and with the integrated security disabled mode to import the data to the system. As normal SQL server driver does not support, so I had used the jtds.jar and the different driver class to pull the data to the Hadoop Lake. Sample Command I tried on the server as follows, sqoop import --table Table1 --connect "jdbc:jtds:sqlserver://<Hostname>:<Port>;useNTLMv2=true;domain=<WindowsDomainName>;databaseName=XXXXXXXXXXXXX" \ --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX --password 'XXXXXXX' \ --verbose --target-dir /tmp/33 -m 1 -- --schema dbo " Here are some examples that worked for me: # List databases sqoop list-databases --connect "jdbc:jtds:myactivedirectorydomain.com" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX -P # List tables sqoop list-tables --connect "jdbc:jtds:myactivedirectorydomain.com;databaseName=DATABASENAMEHERE" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username jmiller.admin -P # Pull data example sqoop import --table TABLENAMEHERE --connect "jdbc:jtds:myactivedirectorydomain.com;databaseName=DATABASENAMEHERE" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX -P --fields-terminated-by '\001' --target-dir /user/XXXXX/20170313 -m 1 -- --schema dbo Note* In the above example you need to change the username to your username and database name in the list-tables or pull to the one you need (note the AD account you use will require access to the data).
... View more
03-14-2017
03:55 AM
I have found a solution to this provided by another user here: https://community.hortonworks.com/questions/20719/sqoop-to-sql-server-with-integrated-security.html Basically if you switch to the jtds driver which you can download here: http://jtds.sourceforge.net/ Per Rajendra Manjunath " Sqoop SQL Server data import to HDFS worked with manual parametric the authentication(using windows credential) with added parameter on the SQL Server JDBC driver, as integrated security is not supported by the SQL driver as of now due to the Kerberos authentication(Delegated tokens distributed over cluster while running MR job). So we need to pass the windows authentication with password and with the integrated security disabled mode to import the data to the system. As normal SQL server driver does not support, so I had used the jtds.jar and the different driver class to pull the data to the Hadoop Lake. Sample Command I tried on the server as follows, sqoop import --table Table1 --connect "jdbc:jtds:sqlserver://<Hostname>:<Port>;useNTLMv2=true;domain=<WindowsDomainName>;databaseName=XXXXXXXXXXXXX" \ --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX --password 'XXXXXXX' \ --verbose --target-dir /tmp/33 -m 1 -- --schema dbo " Here are some examples that worked for me: # List databases sqoop list-databases --connect "jdbc:jtds:myactivedirectorydomain.com" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX -P # List tables sqoop list-tables --connect "jdbc:jtds:myactivedirectorydomain.com;databaseName=DATABASENAMEHERE" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username jmiller.admin -P # Pull data example sqoop import --table TABLENAMEHERE --connect "jdbc:jtds:myactivedirectorydomain.com;databaseName=DATABASENAMEHERE" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX -P --fields-terminated-by '\001' --target-dir /user/XXXXX/20170313 -m 1 -- --schema dbo Note* In the above example you need to change the username to your username and database name in the list-tables or pull to the one you need (note the AD account you use will require access to the data).
... View more
03-14-2017
03:54 AM
I have found a solution to this provided by another user here: https://community.hortonworks.com/questions/20719/sqoop-to-sql-server-with-integrated-security.html Basically if you switch to the jtds driver which you can download here: http://jtds.sourceforge.net/ Per Rajendra Manjunath " Sqoop SQL Server data import to HDFS worked with manual parametric the authentication(using windows credential) with added parameter on the SQL Server JDBC driver, as integrated security is not supported by the SQL driver as of now due to the Kerberos authentication(Delegated tokens distributed over cluster while running MR job). So we need to pass the windows authentication with password and with the integrated security disabled mode to import the data to the system. As normal SQL server driver does not support, so I had used the jtds.jar and the different driver class to pull the data to the Hadoop Lake. Sample Command I tried on the server as follows, sqoop import --table Table1 --connect "jdbc:jtds:sqlserver://<Hostname>:<Port>;useNTLMv2=true;domain=<WindowsDomainName>;databaseName=XXXXXXXXXXXXX" \ --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX --password 'XXXXXXX' \ --verbose --target-dir /tmp/33 -m 1 -- --schema dbo " Here are some examples that worked for me: # List databases sqoop list-databases --connect "jdbc:jtds:myactivedirectorydomain.com" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX -P # List tables sqoop list-tables --connect "jdbc:jtds:myactivedirectorydomain.com;databaseName=DATABASENAMEHERE" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username jmiller.admin -P # Pull data example sqoop import --table TABLENAMEHERE --connect "jdbc:jtds:myactivedirectorydomain.com;databaseName=DATABASENAMEHERE" --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver --username XXXXX -P --fields-terminated-by '\001' --target-dir /user/XXXXX/20170313 -m 1 -- --schema dbo Note* In the above example you need to change the username to your username and database name in the list-tables or pull to the one you need (note the AD account you use will require access to the data).
... View more
03-13-2017
08:49 PM
1 Kudo
I have found a solution to this provided by another user here: https://community.hortonworks.com/questions/20719/sqoop-to-sql-server-with-integrated-security.html Basically if you switch to the jtds driver which you can download here: http://jtds.sourceforge.net/ Per Rajendra Manjunath " Sqoop SQL Server data import to HDFS worked with manual
parametric the authentication(using windows credential) with added parameter on
the SQL Server JDBC driver, as integrated security is not supported by the SQL
driver as of now due to the Kerberos authentication(Delegated tokens
distributed over cluster while running MR job). So we need to pass the windows authentication with password and
with the integrated security disabled mode to import the data to the system. As
normal SQL server driver does not support, so I had used the jtds.jar and the
different driver class to pull the data to the Hadoop Lake. Sample Command I tried on the server as follows, sqoop import --table Table1 --connect
"jdbc:jtds:sqlserver://<Hostname>:<Port>;useNTLMv2=true;domain=<WindowsDomainName>;databaseName=XXXXXXXXXXXXX"
\ --connection-manager org.apache.sqoop.manager.SQLServerManager
--driver net.sourceforge.jtds.jdbc.Driver --username XXXXX --password 'XXXXXXX'
\ --verbose --target-dir /tmp/33 -m 1 -- --schema dbo " Here are some examples that worked for me: # List
databases sqoop
list-databases --connect
"jdbc:jtds:sqlserver://databasehostname.yourdomain.com:1433;useNTLMv2=true;domain=myactivedirectorydomain.com"
--connection-manager org.apache.sqoop.manager.SQLServerManager --driver
net.sourceforge.jtds.jdbc.Driver --username XXXXX -P # List
tables sqoop
list-tables --connect "jdbc:jtds:sqlserver://databasehostname.yourdomain.com:1433;useNTLMv2=true;domain=myactivedirectorydomain.com;databaseName=DATABASENAMEHERE"
--connection-manager org.apache.sqoop.manager.SQLServerManager --driver
net.sourceforge.jtds.jdbc.Driver --username jmiller.admin -P # Pull
data example sqoop import --table
TABLENAMEHERE --connect
"jdbc:jtds:sqlserver://databasehostname.yourdomain.com:1433;useNTLMv2=true;domain=myactivedirectorydomain.com;databaseName=DATABASENAMEHERE"
--connection-manager org.apache.sqoop.manager.SQLServerManager --driver
net.sourceforge.jtds.jdbc.Driver --username XXXXX -P --fields-terminated-by '\001'
--target-dir /user/XXXXX/20170313 -m 1 -- --schema dbo Note* In the above
example you need to change the username to your username and database name in
the list-tables or pull to the one you need (note the AD account you use will
require access to the data).
... View more
03-13-2017
07:04 PM
Greetings Rajendra, You have saved my life, your solution works!!! We have spent two years on this thinking there was no way to get Sqoop working, I want to buy you a drink!!
... View more
02-09-2017
06:39 PM
Definately has not been resolved. It's something Sqoop can't currently do.
... View more