Member since
03-31-2017
21
Posts
3
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1697 | 08-18-2017 05:43 AM | |
1022 | 05-19-2017 11:54 PM |
10-04-2017
10:32 PM
You can find the GC time in LLAP monitor link. for example: http://XXX-hddb-01-b:15002/
... View more
08-18-2017
05:43 AM
Thank you @Sonu Sahi for the response. I checked your colleague's blog. He is using password file but not the password alias (credential provider) which is saving password in keystore instead of file. Any way I finally figure out how to make it work. In case some one else need this in the future, I paste my solution here: The option file: import -connect
"jdbc:sqlserver://host:1433;database=DB1" Script for Sqoop job defination: sqoop job -Dhadoop.security.credential.provider.path=jceks://hdfs/user/root/mysql.password.jceks --create sqoop_test -- options-file db.props --username sqoop_ro --password-alias sqoop_ro.passwd --target-dir /XXX --query 'select ....' This works for me. Thanks
... View more
08-17-2017
07:10 PM
I'm able to use credential provider to encrypt password. Also be able to use it with sqoop command. But I can't work it out if I want to put connection string in option files. Anybody can give me some clue ? Thanks
... View more
- Tags:
- Data Processing
- Sqoop
Labels:
05-23-2017
07:24 PM
Same issue with script init_nifi_in_src_data.sh. Command not found. It's a pain. You don't have this issue ? @Sonu Sahi
... View more
05-23-2017
05:40 PM
I'm able to open Nifi UI by http://localhost:19090/nifi/ . Maybe I can ignore the script error ?
... View more
05-23-2017
05:36 PM
Yes, I'm able to log on http://127.0.0.1:9080 and http://127.0.0.1:18888. Can't open http://sandbox-hdf.hortonworks.com:19090/nifi Have issue with the tutorial script . It says virtualBox command not found. @Sonu Sahi
... View more
05-23-2017
12:02 AM
Hi @Sonu Sahi . Any idea with below error? I tried this script in both virtualBox and Vmware. Neither works. In VirtualBox, it complains vboxmanage not found. Thanks [root@sandbox-hdf ~]# ./auto_scripts/launch_ambari_ui.sh
HDF Sandbox Virtual Environment Selection Menu
'Docker' - is my environment 'VirtualBox' - is my environment
Enter the 'virtual environment' for HDF Sandbox: VirtualBox You entered HDF Sandbox runs on: VirtualBox
Start VirtualBox VM's Docker HDF Sandbox...
./auto_scripts/virtualbox-scripts/virtualbox_sandbox_hdf.sh: line 3: vboxmanage: command not found
Starting Ambari Services, may take 2.5 minutes...
... View more
05-19-2017
11:54 PM
2 Kudos
Finally I figured out the reason. I downloaded the HDF sandbox vmware image. When I click the file to open, it pop up and automatically opened by virtualbox which can't work correctly for the network part. Realized this, I turned off and opened in VMWare. Issue gone. Thanks for your help.
... View more
05-19-2017
11:04 PM
Thank you @Sonu Sahi for your response. I'm actually also using virtual box. Attached is my screenshot: NOt sure what's wrong. My network configuration also NAT.
... View more
05-19-2017
05:46 PM
Yes, the VM setup is using NAT, After I started the VM. Before press ALT+F5, The VMware says "please open a browser's address field: http://10.0.2.15:18888/. However I got "connection time out " error. I press ALT+F5 to log in the Vmware as root user. Then tried "ssh -p 2222 root@127.0.0.1" it doesn't work. @Sonu Sahi Could you help check if you did the same but got different result ? Thanks
... View more
05-18-2017
10:42 PM
1 Kudo
I have been able to connect to HDP sanbox by ssh -p 2222. But with the new HDF sandbox, I was stuck again with the "connection refused" error. After I brought up the sandbox, I tried the URL "http://10.0.2.15:18888/" but just got timeout error. Any idea ? @mqureshi
... View more
Labels:
05-18-2017
08:44 PM
I removed the original VM from the virtualBOX and restarted the HDF VM. Then the error seems gone ? Not sure why removing another VMware from virtualBOX can hellp though. Thanks @mqureshi
... View more
05-18-2017
07:11 PM
I have downloaded the right HDF sandbox and confirmed my downloading has the same MD5 checksum. After I started the virtual machine, I keep getting below error, Any idea ? rejecting I/O to offline devices
@mqureshi
... View more
05-18-2017
05:57 PM
Issue with HDF sandbox: After starting the virtual machine, I got lines of errors saying "sd 2:0:0:0: rejecting I/O to offline device"
... View more
Labels:
05-18-2017
05:00 PM
You are right, it should be HDF sandbox. Seems I downloaded wrong VMware. Will redownload and try again. Thanks.
... View more
05-18-2017
04:51 PM
@mqureshi ;
The tutorial can be found at https://hortonworks.com/hadoop-tutorial/learning-ropes-apache-nifi/#section_3. First I logged in the Vmware, then I used ssh into container by port 2222.
... View more
05-16-2017
11:35 PM
Not sure if hortornworks support team really run these command when they prepare the Nifi tutorial. it doesn't work for me. vboxmanage is not found in the sandbox. Open command doesn't work either. Same issue with the other script launch_nifi_ui.sh [root@sandbox ~]# ./auto_scripts/launch_ambari_ui.sh
HDF Sandbox Virtual Environment Selection Menu
'Docker' - is my environment
'VirtualBox' - is my environment Please enter 'Docker' or 'VirtualBox'.
HDF Sandbox Virtual Environment Selection Menu
'Docker' - is my environment You entered HDF Sandbox runs on: VirtualBox
Start VirtualBox VM's Docker HDF Sandbox... ./auto_scripts/virtualbox-scripts/virtualbox_sandbox_hdf.sh: line 3: vboxmanage: command not found
Starting Ambari Services, may take 2.5 minutes.. .
Launch Ambari UI...
./auto_scripts/launch_ambari_ui.sh: line 34: open: command not found
... View more
04-14-2017
09:55 PM
@thussain, It does logged some java error in hivemetastore.log. Please ese attachment hivemetastore.txt
... View more
04-14-2017
05:59 PM
I'm running a hive POC. We have a combined big table which has > 2300 columns and 44million rows. I submitted the command "analyze table TABXXX COMPUTE STATISTICS for COLUMNS;" It failed with below messages in the end: Map 1: 79(+1)/80 Reducer 2: 0(+1)/1 Map 1: 79(+1)/80 Reducer 2: 0(+1)/1 Map 1: 79(+1)/80 Reducer 2: 0(+1)/1 Map 1: 79(+1)/80 Reducer 2: 0(+1)/1
Map 1: 80/80 Reducer 2: 0(+1)/1 Map 1: 80/80 Reducer 2: 1/1 FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exe
Anybody can give me some hint how to make it work ?
... View more
03-31-2017
06:32 PM
Thanks. The upper case of first letter for java type works. --map-column-java isactive=Integer,etl_isactive=Integer. Execution successfully: INFO mapreduce.ImportJobBase: Retrieved 88281 records.
... View more
03-31-2017
05:03 PM
I'm importing data from our SQL server database to HDFS with Sqoop. Some tables have bit data type in SQL Server. By default, sqoop will convert the bit column like 1->true 0 -> false. We want the data be converted to int type 1 or 0. So I tried the option --map-column-java. For example sqoop ... --map-column-java isactive=int; and even --map-column-java isactive=string, Neither works. I always got error : ERROR orm.ClassWriter: No ResultSet method for Java type integer ERROR tool.ImportTool: Imported Failed: No ResultSet method for Java type integer. Seems in Hortonworks platform, sqoopt doesn't support this option?
... View more
Labels: