Member since
07-19-2016
91
Posts
10
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2698 | 08-12-2016 05:05 PM |
06-27-2017
04:22 PM
1 Kudo
Hello, Is there a processor help with converting a string(Datetime format: 2017-01-18 13:28:17) to long(133434444545) in a csv field? Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
06-26-2017
06:30 PM
Hello, Let's say I have a csv file called User.csv: id,firstName,lastName,email,gender 1,alvin,jin,aj@gmail.com,m
2,bob,trump,bt@gmail.com,m 3,alice,trump,at@gmail.com,f I'd like to use each row as a message value, and the file name as message key. All messages from one file will use the same key, which is "User.csv" in above example. However, it seems the "Message Key Field" property in PublishKafkaRecord can only be specified from one of the fields in the record. Is there anyway to use one of the flowfile attributes as message key, e.g. filename? Besides, I found PublishKafka processor's "Message Key" property can accept flowfile attributes, but it doesn't have Schema registry service.
I was wondering why PublishKafkaRecord_0_10 is designed in this way? Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
06-23-2017
02:09 PM
Yes, after add this dependency. It works. Thanks.
... View more
06-22-2017
02:00 PM
Hello, Since our downstream applications use Confluent Schema Registry, I implemented a custom ConfluentSchemaRegistry service by referring the HWXSchemaRegistry. I put the .nar file into NIFI_HOME/lib folder. I can find this custom service in the Controller Service panel. However, when I'd like to inject this service to the "Schema Registry" property of CSVReader or AvroReader, there are only two default schema registries to choose, without the one I added. Please see the attached picture. How to make my custom schema registry service available for the other services to inject? In other words, how to make it available in the drop-down list in the attached picture? BTW, I am using Apache NiFi v1.3. Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
06-12-2017
06:26 PM
Hi @Bryan Bende, Thank you for your response. I am more interested in PublishKafkaRecord_0_10 + an external schema registry. Since we have already deployed ConfluentSchemaRegistry used by some other apps, we don't want to switch to HortonworksSchemaRegistry. We want NiFi workflow to share avro schema with the other apps. In this case, do we have to implement a Confluent-based schema registry control service? or any easier solutions? Thanks.
... View more
06-12-2017
06:17 PM
Thanks. I did try it. It seems we can't simply use ConfluentSchemaRegistry to replace HortonworksSchemaRegistry in the control service.
... View more
06-12-2017
04:11 PM
1 Kudo
Hi guys, I'd like to know the difference between Hortonworks schema registry vs. Confluent schema registry. Are they compatible? In other words, may I directly point the ConfluentSchemaRegistry URL to the HortonworksSchemaRegistry control service? Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Schema Registry
06-08-2017
03:28 PM
1 Kudo
Hello, I'd like to publish avro message to a kafka topic, with the schema registered in avro schema registry. However, I didn't find any KafkaProducer processors have a "Schema Registry URL". Please note my downstream is not in NiFi. It can be a independent java app implemented the Kafka consumer client api,
which consumes the avro message and connects to Schema registry to retrieve the schema. It seems in v1.2 the schema registry is integrated within NiFi, is it? How does the other apps share schemas with NiFi
? Any documents or examples for using schema registry with Kafka processors in NiFi? Thanks.
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache NiFi
-
Schema Registry
06-06-2017
08:36 PM
Hi @Matt Clarke, Does this one address=([a-zA-Z0-9 ,]+) mean NiFi can find the column named "address"? I tried it, but no luck. Would you please upload an example? Thanks.
... View more
05-29-2017
07:20 PM
I tried to run below command to mount a volume for the content repository. However, it always after a few minutes. $ docker run -it --rm -v ~/nifi-content:/opt/nifi/nifi-1.2.0/content_repository -p 8080-8081:8080-8081 apache/nifi:1.2.0 It seems like file permission issue related with uid. Would you please suggest how to make the docker container work? Thanks.
... View more