JSON from Twitter (flat) getting error on InferAvroSchema

TimothySpann — Sun, 18 Aug 2019 10:50:38 GMT

2016-11-08 19:35:09,121 WARN [Timer-Driven Process Thread-9] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding ConvertJSONToAvro[id=10091199-14c4-1ad1-fdc6-9c4abcfa4349] due to uncaught Exception: java.lang.IllegalArgumentException: Schemas for JSON files should be record
2016-11-08 19:35:09,123 WARN [Timer-Driven Process Thread-9] o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.IllegalArgumentException: Schemas for JSON files should be record
 at org.kitesdk.shaded.com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) ~[na:na]
 at org.kitesdk.data.spi.filesystem.JSONFileReader.initialize(JSONFileReader.java:84) ~[na:na]
 at org.apache.nifi.processors.kite.ConvertJSONToAvro$1.process(ConvertJSONToAvro.java:144) ~[na:na]
 at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2370) ~[nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
 at org.apache.nifi.processors.kite.ConvertJSONToAvro.onTrigger(ConvertJSONToAvro.java:139) ~[na:na]
 at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
 at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) ~[nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
 at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
 at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
 at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.0.0.2.0.0.0-579.jar:1.0.0.2.0.0.0-579]
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_101]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_101]
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_101]
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_101]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_101]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_101]
 at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]

Error on inferAvroSchema from JSON file.

Re: JSON from Twitter (flat) getting error on InferAvroSchema

TimothySpann — Wed, 09 Nov 2016 04:00:16 GMT

It got a different error, when I removed that attribute. I also turned off use internal attributes into convert to JSON. Maybe be internal fields are an issue.

Re: JSON from Twitter (flat) getting error on InferAvroSchema

mburgess — Wed, 09 Nov 2016 04:11:22 GMT

Avro doesn't like the dots in the attribute names (which become field names), perhaps you could rename them (with UpdateAttribute or at the source processor(s)) using underscores (or other valid characters like alphanumeric characters, see the link for the rules).

question JSON from Twitter (flat) getting error on InferAvroSchema in Archives of Support Questions (Read Only)

JSON from Twitter (flat) getting error on InferAvroSchema

Re: JSON from Twitter (flat) getting error on InferAvroSchema

Re: JSON from Twitter (flat) getting error on InferAvroSchema