Posts: 25
Registered: ‎01-30-2014
Accepted Solution

extractAvroPaths nested structure

I'm having trouble extracting a nested structure from my avro data.




    "username" : "alex"

    "date" : "21-08-2014"

    "attachments" : [

        "documents" : [


                  "title": "test"

                  "tags" : [ "a", "b", "c" ]



                  "optional1" : "test2"

                  "title" : "test2"

              } ],

        "context" : "school"




Extracting the paths with Avro:



{ extractAvroPaths {

     flatten : true

     paths : {

         /my_user : /username       # this works fine

         # all three of these result in the same error message, with different patemeters

         /my_attachments : /attachments[]

         /my_documents : /attachments[]/documents[]

         /my_contexts : /attachments[]/documents[]/context





Results in the following error message:

com.typesafe.config.ConfigExceptionWrongType: morph-solr.conf: 30: Cannot concatenate object or list with a non-object-or-list, ConfigString("/my_attachments") and SimpleConfigList([]) are not compatible.


Eventualy I would like to map the fields to a solr index. So if its possible to extract the nested structures, the followup question would be how to map those to a solr schema, but lets take it one step at a time :)

Posts: 25
Registered: ‎01-30-2014

Re: extractAvroPaths nested structure

Srry too quick on the post trigger: this was just a matter of quoting the mapping:


 /my_attachments : "/attachments[]" 


The question remains how to map the structure to a solr index, but will post that in the appropriate section.

Cloudera Employee
Posts: 146
Registered: ‎08-21-2013

Re: extractAvroPaths nested structure

Try to use syntax with quotes, like so :

/my_contexts : "/attachments[]/documents[]/context"