To do incremental imports, you need to add a few more arguments to your command. Take a look at the following description from the sqoop docs: https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_incremental_imports Inside your shell script, I’m not sure how you’re setting the values of your named parameters, unless you have more code that is mapping the positional parameters to local variables. When using the lastmodified mode, it requires that a field in your table to be a timestamp, which will be updated with the current timestamp during update. Additionally, the update requires a check column, which can be of type timestamp also. ... View more

Hello! This is definitely not a single point of failure in a Kafka cluster. Let me quote from the Kafka doc: "Each partition has one server which acts as the "leader" and zero or more servers which act as "followers". The leader handles all read and write requests for the partition while the followers passively replicate the leader. If the leader fails, one of the followers will automatically become the new leader. Each server acts as a leader for some of its partitions and a follower for others so load is well balanced within the cluster." And there is a nice explanation of how leader election is performed: https://kafka.apache.org/documentation#design_replicatedlog "Kafka takes a slightly different approach to choosing its quorum set. Instead of majority vote, Kafka dynamically maintains a set of in-sync replicas (ISR) that are caught-up to the leader. Only members of this set are eligible for election as leader. A write to a Kafka partition is not considered committed until all in-sync replicas have received the write. This ISR set is persisted to ZooKeeper whenever it changes. Because of this, any replica in the ISR is eligible to be elected leader." ... View more

You’re seeing the effects of the sandbox running as a docker container in a VM on your host. In the image that you uploaded, you have ssh’d into the docker container, which runs in its own network, so that is why you are unable to reach the site at the IP address. However, you should be able to use localhost and get a response from the sandbox. Before starting the tutorial, with the new sandbox, you should be able to access http://localhost:8888, which will bring you to a to the Sandbox start page. I suggest clicking into the Advanced HDP page. From there you will see links to Ambari and a web-based command line. ... View more

Unfortunately, not yet. At this point, Atlas is best used with Hive, capturing the following events: create database create table/view, create table as select load, import, export DMLs (insert) alter database alter table (skewed table information, stored as, protection is not supported) alter view Also, Atlas works with Sqoop, capturing events from the hive import command. ... View more