Member since
05-05-2016
147
Posts
223
Kudos Received
18
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3678 | 12-28-2018 08:05 AM | |
3626 | 07-29-2016 08:01 AM | |
2996 | 07-29-2016 07:45 AM | |
6930 | 07-26-2016 11:25 AM | |
1370 | 07-18-2016 06:29 AM |
07-06-2016
01:22 PM
1 Kudo
Error Message is below:- hbase(main):027:0* delete_get_some
NameError: undefined local variable or method `output' for #<Object:0x714863bd>
... View more
07-06-2016
12:43 PM
4 Kudos
I am using HDP2.4 sandbox and I have a Hive external table on top of HBase, my application suppose to load data from Hive. Below JRuby program runs once in a day to give me custom filter on my INT column values. import org.apache.hadoop.hbase.HBaseConfiguration
import org.apache.hadoop.hbase.client.HTable
import org.apache.hadoop.hbase.client.Get
import org.apache.hadoop.hbase.util.Bytes
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Result;
import java.util.ArrayList;
def delete_get_some()
var_table = "tiny_hbase_table1"
htable = HTable.new(HBaseConfiguration.new, var_table)
rs = htable.getScanner(Bytes.toBytes("o"), Bytes.toBytes("kingdomKey"))
output = ArrayList.new output.add "ROW\t\t\t\t\t\tCOLUMN\+CELL"
rs.each { |r| r.raw.each { |kv|
row = Bytes.toInt(kv.getRow)
fam = kv.getFamily
ql = Bytes.toString(kv.getQualifier)
ts = kv.getTimestamp
val = Bytes.toInt(kv.getValue)
rowval = Bytes.toInt(kv.getRow)
output.add "#{row} #{ql} #{val}"
}
}
output.each {|line| puts "#{line}\n"}
end
delete_get_some
But i am getting "NoMethodError: undefined method `add' for nil:NilClass" error while running this program. Suggestions or advice please on this issue...
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache HBase
07-06-2016
10:56 AM
5 Kudos
If YARN cluster dont have capacity to launch AM then Hive CLI waits indefinitely to launch an AM. Tez have its own optimization step where it pre-allocate resources saving the user the overhead of launching a AM everytime a new query launch from same session. But when it lanches using mr it dont have any condition called preallocate and works successfully.
... View more
07-06-2016
10:01 AM
3 Kudos
Hi Alex McLintock, It raise semantic exception which means failed during parsing your query. You cant use CTAS if you have duplicate columns in table. Another options suggested by Benjamin Leonhardi is good if you know your duplicate columns candidate list.
... View more
07-06-2016
09:41 AM
Thanks for a great tip, we can also specify multiple columns like this tb2.`(id1)?+(id2)?+.+`
... View more
07-06-2016
06:19 AM
3 Kudos
SQL Server’s limit on the row size is applicable even when you join 2 tables and select a big row. Alternative solution if you dont want to change row size is you can Import the sqlserver tables into Hadoop as individual external tables. Then join them in the Hadoop side and populate actual table.
... View more
07-05-2016
02:16 PM
6 Kudos
I think kafka->Storm->Cassandra is batter as similar architecture worked very well in our use case... https://www.linkedin.com/pulse/real-time-data-ingestion-easy-simple-co-dev-neeraj-sabharwal
... View more
07-05-2016
12:09 PM
4 Kudos
I have downloaded HDP2.4 sandbox(Oracle Virtual Box). All components are running perfectly, when i add NiFi service using Ambari it failed to install. Attached is the error.txt and output.txt file. I have checked my /etc/host file having proper host name with correct IP. I am logged in using admin in Ambari also seems good.
... View more
Labels:
07-05-2016
11:20 AM
2 Kudos
After the initial import, subsequent imports can leverage SQOOP’s native support for “Incremental Import” by using the “check-column”, “incremental” and “last-value” parameters. http://hortonworks.com/blog/four-step-strategy-incremental-updates-hive/
... View more
07-05-2016
11:10 AM
5 Kudos
How big is the table? If it is very big and not updating then you can use sqoop for this task otherwise another option could be to use use below template for that uses ExecuteScript with Groovy to issue a SQL query and produce a flowfile containing a CSV representation of the results. Follow template here https://gist.github.com/mattyb149/9945663ae9ae5dcb1ddb43d21553204d
... View more