Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

extractAvroPaths nested structure

SOLVED Go to solution

extractAvroPaths nested structure

Contributor

I'm having trouble extracting a nested structure from my avro data.

 

_attachment_body=[

{

    "username" : "alex"

    "date" : "21-08-2014"

    "attachments" : [

        "documents" : [

              {

                  "title": "test"

                  "tags" : [ "a", "b", "c" ]

              },

              {

                  "optional1" : "test2"

                  "title" : "test2"

              } ],

        "context" : "school"

    ]

}

 

Extracting the paths with Avro:

 

...

{ extractAvroPaths {

     flatten : true

     paths : {

         /my_user : /username       # this works fine

         # all three of these result in the same error message, with different patemeters

         /my_attachments : /attachments[]

         /my_documents : /attachments[]/documents[]

         /my_contexts : /attachments[]/documents[]/context

     }

  }

}

 

Results in the following error message:

com.typesafe.config.ConfigExceptionWrongType: morph-solr.conf: 30: Cannot concatenate object or list with a non-object-or-list, ConfigString("/my_attachments") and SimpleConfigList([]) are not compatible.

 

Eventualy I would like to map the fields to a solr index. So if its possible to extract the nested structures, the followup question would be how to map those to a solr schema, but lets take it one step at a time :)

1 ACCEPTED SOLUTION

Accepted Solutions

Re: extractAvroPaths nested structure

Contributor

Srry too quick on the post trigger: this was just a matter of quoting the mapping:

 

 /my_attachments : "/attachments[]" 

 

The question remains how to map the structure to a solr index, but will post that in the appropriate section.

2 REPLIES 2

Re: extractAvroPaths nested structure

Contributor

Srry too quick on the post trigger: this was just a matter of quoting the mapping:

 

 /my_attachments : "/attachments[]" 

 

The question remains how to map the structure to a solr index, but will post that in the appropriate section.

Highlighted

Re: extractAvroPaths nested structure

Expert Contributor
Try to use syntax with quotes, like so :

/my_contexts : "/attachments[]/documents[]/context"

Wolfgang.