Reply
New Contributor
Posts: 1
Registered: ‎03-13-2018

ERROR hdfs.HDFSEventSink: process failed JDBC Channel

[ Edited ]

All,

I am unable to ingest avro file compressionCodec = Snappy using avro client. Please see below Flume config file , please help me to resolve the issue

 

a1.sources = avro
a1.sinks = hd
a1.channels = jdbc


a1.sources.avro.type = avro
a1.sources.avro.bind = localhost
a1.sources.avro.port = 11112

a1.sinks.hd.type = hdfs
a1.sinks.hd.hdfs.path = /user/cloudera/problem7
a1.sinks.hd.hdfs.fileSuffix = .avro
a1.sinks.hd.hdfs.fileType = DataStream
a1.sinks.hd.serializer = avro_event
a1.sinks.hd.serializer.compressionCodec = snappy
a1.sinks.hd.serializer = org.apache.flume.serialization.AvroEventSerializer.Builder


a1.channels.jdbc.type = jdbc


a1.sources.avro.channels = jdbc
a1.sinks.hd.channel = jdbc

 

commands :

 

flume-ng agent -n a1 -f <</path/configfile>>

flume avro-client -H localhost  -p 11112 -F <<path/Avrocompressed file >

Announcements