Sqoop time transferred

[#1]

Hello guys, I'm importing a table to Hbase via Sqoop import, and i want to do some experimentations and measure the time written, based on some options in Hbase like Number of regions, Max. file size, pre-splitting, others... Can i rely on the number (315.1576 seconds) that i'm showing in this picture:

Thanks

[#2]

You can time your command and compare numbers if you do not trust the number reported by Sqoop MR Import job 🙂

time IMPORT_COMMAND (Linux)

[#3]

Thanks for the suggestion @vrodionov.

So, there is no other way to measure the time that sqoop spends to import data to hbase?

[#4]

What I suggested is to compare both times. If they close enough, than you can rely on both. If there is a significant discrepancy than I would go with Unix timing.

[#5 · Accepted solution]

You can view the job completion details if u launch <resourcemangerhost>:8080/cluster/apps/FINISHED

look for the respective Mapreduce associated to the sqoop job.

[#6]

@rkandula

Shouldn't be <resourcemanagerhost>:8088/cluster/apps/FINISHED ?

[#7]

sorry typo, 8088 is the port

[#8]

yup, i figured out, but thanks @rkandula, i tested it and it worked. But now i have 3 types of different times:

214.51 seconds from shell

real 4m40.985s from Linux command

user 0m19.121s from Linux command

sys 0m5.181s from Linux command

2m50 seconds elapsed time from mapreduce job

What time should i consider about the real time of writing to hbase table?

[#9]

i am not sure about unix command,what is the time duration u see in resourcemanager url

[#10]

I see elapsed time 2m50 seconds and in the bottom : Aggregate Resource Allocation:53607 MB-seconds, 213 vcore-seconds