Member since
12-10-2015
4
Posts
2
Kudos Received
0
Solutions
04-07-2016
10:46 AM
This schema corresponds to an array of records (which in turn has arrays), but not an array of array?
... View more
03-27-2016
04:43 AM
1 Kudo
Greetings! I
was trying to write some sample Avro schemas, and was unable to create
one for an array of array.
Sample json - {
"testarray":[[1,2,3][4,5,6] ... ]
}
My best try to create the schema -
{
"name": "testarray",
"type": {
"type":"array",
"items": {
"type": {
"type":"array",
"items": {
"name": "temp",
"type":"long"
}
}
}
}
} But the schema parser throws this error - --- Exception in thread "main" org.apache.avro.SchemaParseException: No type: {"name":"testarray","type":{"type":"array","items":{"type":{"type":"array","items":{"name":"temp","type":"long"}}}}} at org.apache.avro.Schema.getRequiredText(Schema.java:1372) at org.apache.avro.Schema.parse(Schema.java:1231) at org.apache.avro.Schema$Parser.parse(Schema.java:1031) at org.apache.avro.Schema$Parser.parse(Schema.java:996) at com.flipkart.de.App.main(App.java:54) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) Process finished with exit code 1
---
Is there something wrong with the schema I've written, or is this not supported in Avro?
Any help would be much appreciated.
... View more
03-07-2016
05:14 AM
Thanks Artem!
I had to add 'http://repo.hortonworks.com/content/groups/public/' apart from the 'http://repo.hortonworks.com/content/repositories/releases/' repo to resolve for the following error I encountered - [ERROR] Failed to execute goal on project funnel: Could not resolve dependencies for project com.yahoo.hive.udf:funnel:jar:1.0: The following artifacts could not be resolved: org.mortbay.jetty:jetty:jar:6.1.26.hwx, org.mortbay.jetty:jetty-util:jar:6.1.26.hwx: Failure to find org.mortbay.jetty:jetty:jar:6.1.26.hwx in http://repo.hortonworks.com/content/repositories/releases/ was cached in the local repository, resolution will not be reattempted until the update interval of HDP has elapsed or updates are forced -> [Help 1] One another issue I face is that the code makes use of Java 8 features, while the sandbox Java version is "1.7.0_95". Should I consider upgrading Java in my Sandbox? or should I rewrite the code to make it compatible with Java 7?
... View more
03-07-2016
04:44 AM
1 Kudo
I need to use the funnel Hive UDFs from here - https://github.com/yahoo/hive-funnel-udf But I'm not sure if I should build the jar with HDP specific libraries? I was hoping if someone could help me with the details of what all needs to be changed in the dependency configuration for the build?
... View more
Labels:
- Labels:
-
Apache Hive