Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

extractAvroPaths nested structure

SOLVED Go to solution
Highlighted

extractAvroPaths nested structure

Contributor

I'm having trouble extracting a nested structure from my avro data.

 

_attachment_body=[

{

    "username" : "alex"

    "date" : "21-08-2014"

    "attachments" : [

        "documents" : [

              {

                  "title": "test"

                  "tags" : [ "a", "b", "c" ]

              },

              {

                  "optional1" : "test2"

                  "title" : "test2"

              } ],

        "context" : "school"

    ]

}

 

Extracting the paths with Avro:

 

...

{ extractAvroPaths {

     flatten : true

     paths : {

         /my_user : /username       # this works fine

         # all three of these result in the same error message, with different patemeters

         /my_attachments : /attachments[]

         /my_documents : /attachments[]/documents[]

         /my_contexts : /attachments[]/documents[]/context

     }

  }

}

 

Results in the following error message:

com.typesafe.config.ConfigExceptionWrongType: morph-solr.conf: 30: Cannot concatenate object or list with a non-object-or-list, ConfigString("/my_attachments") and SimpleConfigList([]) are not compatible.

 

Eventualy I would like to map the fields to a solr index. So if its possible to extract the nested structures, the followup question would be how to map those to a solr schema, but lets take it one step at a time :)

1 ACCEPTED SOLUTION

Accepted Solutions

Re: extractAvroPaths nested structure

Contributor

Srry too quick on the post trigger: this was just a matter of quoting the mapping:

 

 /my_attachments : "/attachments[]" 

 

The question remains how to map the structure to a solr index, but will post that in the appropriate section.

2 REPLIES 2

Re: extractAvroPaths nested structure

Contributor

Srry too quick on the post trigger: this was just a matter of quoting the mapping:

 

 /my_attachments : "/attachments[]" 

 

The question remains how to map the structure to a solr index, but will post that in the appropriate section.

Re: extractAvroPaths nested structure

Expert Contributor
Try to use syntax with quotes, like so :

/my_contexts : "/attachments[]/documents[]/context"

Wolfgang.