@Neil Morin Can you check the Virtualbox network rules to see if port forwarding is properly setup? If you ssh/scp to 127.0.0.1 on port 22, that defaults to your local computer. If port forwarding is not setup to forward that port to the VM, then you will get a connection refused. Network -> Advanced dropdown -> Port Forwarding ... View more

@rajdip chaudhuri I used your warehouse.csv and your load command. While it does appear to finish successfully, this is what I see at the end: ImportTsv Bad Lines=5 File Input Format Counters Bytes Read=590 File Output Format Counters Bytes Written=0 As you can see there were 5 bad lines, which is the total line count in your file. That means the command ran, but there was a problem with the data. It took a little bit of effort to find the issue, but the problem was that your csv file has an extra , at the end of the lines. Here is an example from your file: 1,AAAAAAAABAAAAAAA,Conventional childr,977787,651,6th ,Parkway,Suite 470,Fairview,Williamson County,TN,35709,United States,-5, It should look like this: 1,AAAAAAAABAAAAAAA,Conventional childr,977787,651,6th ,Parkway,Suite 470,Fairview,Williamson County,TN,35709,United States,-5 Notice that I removed the trailing comma. Now the data was loaded: ImportTsv Bad Lines=0 File Input Format Counters Bytes Read=585 File Output Format Counters Bytes Written=0 And here is the scan: hbase(main):011:0> scan 'warehouse' ROW COLUMN+CELL 1 column=mycf:w_city, timestamp=1482521720833, value=Fairview 1 column=mycf:w_country, timestamp=1482521720833, value=United States 1 column=mycf:w_county, timestamp=1482521720833, value=Williamson County 1 column=mycf:w_gmt_offset, timestamp=1482521720833, value=-5 1 column=mycf:w_state, timestamp=1482521720833, value=TN 1 column=mycf:w_street_name, timestamp=1482521720833, value=6th 1 column=mycf:w_street_number, timestamp=1482521720833, value=651 1 column=mycf:w_street_type, timestamp=1482521720833, value=Parkway 1 column=mycf:w_suite_number, timestamp=1482521720833, value=Suite 470 1 column=mycf:w_warehouse_id, timestamp=1482521720833, value=AAAAAAAABAAAAAAA 1 column=mycf:w_warehouse_name, timestamp=1482521720833, value=Conventional childr 1 column=mycf:w_warehouse_sq_ft, timestamp=1482521720833, value=977787 1 column=mycf:w_zip, timestamp=1482521720833, value=35709 2 column=mycf:w_city, timestamp=1482521720833, value=Fairview 2 column=mycf:w_country, timestamp=1482521720833, value=United States 2 column=mycf:w_county, timestamp=1482521720833, value=Williamson County 2 column=mycf:w_gmt_offset, timestamp=1482521720833, value=-5 2 column=mycf:w_state, timestamp=1482521720833, value=TN 2 column=mycf:w_street_name, timestamp=1482521720833, value=View First 2 column=mycf:w_street_number, timestamp=1482521720833, value=600 2 column=mycf:w_street_type, timestamp=1482521720833, value=Avenue 2 column=mycf:w_suite_number, timestamp=1482521720833, value=Suite P 2 column=mycf:w_warehouse_id, timestamp=1482521720833, value=AAAAAAAACAAAAAAA 2 column=mycf:w_warehouse_name, timestamp=1482521720833, value=Important issues liv 2 column=mycf:w_warehouse_sq_ft, timestamp=1482521720833, value=138504 2 column=mycf:w_zip, timestamp=1482521720833, value=35709 3 column=mycf:w_city, timestamp=1482521720833, value=Fairview 3 column=mycf:w_country, timestamp=1482521720833, value=United States 3 column=mycf:w_county, timestamp=1482521720833, value=Williamson County 3 column=mycf:w_gmt_offset, timestamp=1482521720833, value=-5 3 column=mycf:w_state, timestamp=1482521720833, value=TN 3 column=mycf:w_street_name, timestamp=1482521720833, value=Ash Laurel 3 column=mycf:w_street_number, timestamp=1482521720833, value=534 3 column=mycf:w_street_type, timestamp=1482521720833, value=Dr. 3 column=mycf:w_suite_number, timestamp=1482521720833, value=Suite 0 3 column=mycf:w_warehouse_id, timestamp=1482521720833, value=AAAAAAAADAAAAAAA 3 column=mycf:w_warehouse_name, timestamp=1482521720833, value=Doors canno 3 column=mycf:w_warehouse_sq_ft, timestamp=1482521720833, value=294242 3 column=mycf:w_zip, timestamp=1482521720833, value=35709 4 column=mycf:w_city, timestamp=1482521720833, value=Fairview 4 column=mycf:w_country, timestamp=1482521720833, value=United States 4 column=mycf:w_county, timestamp=1482521720833, value=Williamson County 4 column=mycf:w_gmt_offset, timestamp=1482521720833, value=-5 4 column=mycf:w_state, timestamp=1482521720833, value=TN 4 column=mycf:w_street_name, timestamp=1482521720833, value=Wilson Elm 4 column=mycf:w_street_number, timestamp=1482521720833, value=368 4 column=mycf:w_street_type, timestamp=1482521720833, value=Drive 4 column=mycf:w_suite_number, timestamp=1482521720833, value=Suite 80 4 column=mycf:w_warehouse_id, timestamp=1482521720833, value=AAAAAAAAEAAAAAAA 4 column=mycf:w_warehouse_name, timestamp=1482521720833, value=Bad cards must make. 4 column=mycf:w_warehouse_sq_ft, timestamp=1482521720833, value=621234 4 column=mycf:w_zip, timestamp=1482521720833, value=35709 5 column=mycf:w_city, timestamp=1482521720833, value=Fairview 5 column=mycf:w_country, timestamp=1482521720833, value=United States 5 column=mycf:w_county, timestamp=1482521720833, value=Williamson County 5 column=mycf:w_gmt_offset, timestamp=1482521720833, value= 5 column=mycf:w_state, timestamp=1482521720833, value=TN 5 column=mycf:w_street_name, timestamp=1482521720833, value= 5 column=mycf:w_street_number, timestamp=1482521720833, value= 5 column=mycf:w_street_type, timestamp=1482521720833, value= 5 column=mycf:w_suite_number, timestamp=1482521720833, value= 5 column=mycf:w_warehouse_id, timestamp=1482521720833, value=AAAAAAAAFAAAAAAA 5 column=mycf:w_warehouse_name, timestamp=1482521720833, value= 5 column=mycf:w_warehouse_sq_ft, timestamp=1482521720833, value= 5 column=mycf:w_zip, timestamp=1482521720833, value=35709 5 row(s) in 0.3110 seconds ... View more

@Huahua Wei You should be able to use just defaults,noatime as ext4 mount options. For the most part, data loss concerns revolve around improper shutdown of the servers (hard reboots, etc). Data loss is always a concern when a server is improperly shutdown. The option data=ordered is the default and provides the best balance between performance and protection. If you need more data protection you can use data=journaled. Out of the box, ext4 generally doesn't require as many tuning options as ext3 does. Most people tend to use defaults,noatime, but you can use defaults,noatime,data=journaled if you are concerned about data protection. Be aware there are likely performance impact. ... View more

@Yurii Tymchii The issue you are seeing is the sandbox uses a start script (/etc/init.d/startup_script) to start the services, but it really just calls a chain of other scripts. Even though you have stopped Spark 1.6, when the sandbox starts up again it makes the call to start Spark 1.6 automatically. I haven't yet found exactly where Spark is starting. As I said, the chain of script calling is a little more complex than I expected. What I do in my sandboxes is to create a couple of scripts that use the Ambari REST API to start/stop extra services. I'm not using Flume, so I created a script called "stop_flume.sh" on my sandbox. After I run the startup_script, I then run the stop_flume.sh script. Here is my script: USERNAME=admin PASSWORD=admin CLUSTER=Sandbox echo "Stopping FLUME" # Turn off maintenance mode curl -u $USERNAME:$PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Turn OFF Maintenance Mode for FLUME via REST"}, "Body": {"ServiceInfo": {"maintenance_state": "OFF"}}}' http://`hostname -f`:8080/api/v1/clusters/$CLUSTER/services/FLUME sleep 5 # Stopping a service is actually setting the state to INSTALLED curl -u $USERNAME:$PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Start FLUME via REST"}, "Body": {"ServiceInfo": {"state": "INSTALLED"}}}' http://`hostname -f`:8080/api/v1/clusters/$CLUSTER/services/FLUME sleep 5 # Turn on maintenance mode curl -u $USERNAME:$PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Turn OFF Maintenance Mode for FLUME via REST"}, "Body": {"ServiceInfo": {"maintenance_state": "ON"}}}' http://`hostname -f`:8080/api/v1/clusters/$CLUSTER/services/FLUME ... View more