Member since
07-19-2016
3
Posts
0
Kudos Received
0
Solutions
02-17-2017
03:47 PM
I've two external tables in Hive that have exactly the same schema and I want to import the data from these two tables into a single table. I can do a UNION ALL from two tables but it seems to be very slow. Example show below. Is there a better way to do this? Thanks. CREATE TABLE MYTABLE
ROW FORMAT SERDE "org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe"
STORED AS RCFILE
AS
SELECT row_number() OVER(), *
FROM
(
SELECT * FROM x
UNION ALL
SELECT * FROM Y
) T2
... View more
Labels:
- Labels:
-
Apache Hive
08-03-2016
03:28 PM
I'm trying to get the messages using a GetKafka processor but I'm not able to get it. I tested consuming messages using Kafka command line consumer and it works. I was also able to use PutKafka processor successfully to put the messages in the topic. Attached my settings where I set the Zookeeper Connection String and Topic Name. When I run the flow, I don't see any errors in the processors. I see an exception in nifi-app.log:
2016-08-03 09:34:33,722 WARN [70e1df87-6097-4ed0-9a40-7e36f9be6921_mydomain.com-1470231250839-1fbd0cfe-leader-finder-thread] kafka.client.ClientUtils$ Fetching topic metadata with correlation id 0 for topics [Set(test)] from broker [id:0,host:DataSlave1.mydomain.com,
port:9092] failed
java.nio.channels.ClosedByInterruptException: null
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) ~[na:1.8.0_101]
at sun.nio.ch.SocketChannelImpl.poll(SocketChannelImpl.java:957) ~[na:1.8.0_101] Versions: Kafka (0.8): 2.10-0.8.2.1, Nifi: 0.7.0 Am I missing anything? Also tested with Kafka 0.10 and had the same problem. Thanks.
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache NiFi