Member since
11-01-2018
4
Posts
0
Kudos Received
0
Solutions
12-14-2018
06:11 AM
The same can be peformed in hive using concat_ws('.',from_unixtime(cast(epochmillis/1000 as BIGINT),'yyyy-MM-dd HH:mm:ss'),cast(floor(epochmillis % 1000) as STRING)) to get the timestamp with milliseconds. Is this efficient way of doing it ?
... View more
11-01-2018
11:29 AM
Please post this query in a new thread. this is not related to the post. To answer your query. Try these steps: Use --direct path, it helps improve performance. Select appropriate --split-by key that is of numerical type (like INT) & also is continuous. This ensures the mappers get appx. equal chunk of data to process & thus makes full use of parallel processing. Try increasing the --fetch-size , by default it is 1000. Try using --boundary-query [sqoop handles this internally but if that is not efficient then you can override it with this boundary query].
... View more