About TimothySpann

TimothySpann · ‎08-11-2017

kerberos is a tougher, but without kerberos: set configuration: must have access to hadoop files /etc/hadoop/conf/core-site.xml,/etc/hadoop/conf/hdfs-site.xml set a directory

TimothySpann · ‎08-11-2017

+1 for an OrcRecordSetWriter

TimothySpann · ‎08-11-2017

It's easiest to run your commands in a shell script. What I do is create a shell script, make sure it has all the permissions you need and set any CLASSPATH, JAVA_HOME and other environment variables. Then I just call it like /opt/mydir/myshell.sh This is 777 and will call my java via java -jar ... Build a jar and run it that way. Or wrap it in a custom processor. https://community.hortonworks.com/articles/73811/trigger-sonicpi-music-via-apache-nifi.html

TimothySpann · ‎08-11-2017

The files are really small too as seen above and don't seem complete. I have done that conversion before with no issues. I am wondering if this is related to having a bunch of null fields.

TimothySpann · ‎08-11-2017

java -jar avro-tools-1.8.2.jar getschema test.avro log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Exception in thread "main" java.io.IOException: Not a data file. at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:105) at org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97) at org.apache.avro.tool.DataFileGetSchemaTool.run(DataFileGetSchemaTool.java:47) at org.apache.avro.tool.Main.run(Main.java:87) at org.apache.avro.tool.Main.main(Main.java:76) No errors until I try to convert that file to ORC or if I download it and look at it in avro tools. 1 Jungnickel Rd Ganado��&2017-08-04 15:26:54MovementBFAK@WE�=@�"ƖX�&2017-08-04 15:06:561243142403�TX&2017-08-04 14:15:31 77962%

TimothySpann · ‎08-11-2017

1. Make sure NiFi and SAM are the same version HDF 2. Make sure kafka message is converted properly to Avro. 3. Restart Kafka, SAM, NiFi 4. Make sure all the settings are proper.

TimothySpann · ‎08-10-2017

If you have a id that increments you can use QueryDatabaseTable. CDC will add Postgresql next, Hana is closed source so that connector will be tougher.

TimothySpann · ‎08-08-2017

https://github.com/hortonworks/registry/blob/master/examples/schema-registry/avro/src/main/java/com/hortonworks/registries/schemaregistry/examples/avro/KafkaAvroSerDesApp.java

TimothySpann · ‎08-08-2017

Deleting or marking invalid for schemas is a good idea, I have mentioned to the team as I would like to do so as well. I have some I created by mistake.

TimothySpann · ‎08-07-2017

An exception with message [More than one 'CURRENT' version for topology id: 1] was thrown while processing request. Only 1 application and only one version. Save was slow. Now won't restart. Anything to do but delete?

Online	Offline
Last Visited	‎05-20-2024 05:42 PM

Member Since	‎01-07-2019 11:58 AM
Last Visited	‎05-20-2024 05:42 PM
Posts	1,973
Kudos received	1122

Cloudera Community

Re: Has anyone tried NiFi consuming (JMSConsume) f...

Re: NiFi Crash after runing chain of lookups

Re: Recommend approach for listening to RSS Feed i...

Re: NiFi ListenFTP Processor Default Data Port

Re: Nifi: Kafka Producer with Avro format in both ...

Re: IoT Example in Apache NiFi: Consuming and Pr...

Re: NiFI ConvertRecord : AvroRecordSetWriter Produ...

Re: Unable to execute ExecuteStreamCommand process...

Re: NiFI ConvertRecord : AvroRecordSetWriter Produ...

NiFI ConvertRecord : AvroRecordSetWriter Producing...

Re: SAM error: com.hortonworks.registries.schemare...

Re: Change Data Capture for Hana View using NiFi

Re: SAM application: "Unknown protocol id [12] rec...

Re: Delete schema from Schema Registry

SAM Weird Error