question Re: Morphlines - converting a byte array field to a UTF-8 string in Archives of Support Questions (Read Only)

Morphlines - converting a byte array field to a UTF-8 string

hadooptom — Tue, 30 Sep 2014 13:51:41 GMT

I've got a record in Morphlines that includes a byte array field. I want to convert that to a UTF-8 string, i.e. the equivalent of String(field, "UTF-8) in Java.

I can see the readClob command exists, but that works on a whole record rather than a single field. Is there an alternative?

Appreciate that I should have stored my data differently, but it's not my format and that's what ETL tools (morphlines) are for! 😉

Thanks in advance.

Re: Morphlines - converting a byte array field to a UTF-8 string

whosch — Tue, 30 Sep 2014 18:42:15 GMT

You can try the java command, like so:

java {
code: """
String str = new String((byte[]) record.getFirstValue("myInputField"), "UTF-8");
record.put("myOutputField", str);
return getChild().process(record); // pass record to next command in chain
"""
}

Wolfgang.

Re: Morphlines - converting a byte array field to a UTF-8 string

hadooptom — Tue, 30 Sep 2014 20:19:41 GMT

Perfect, thanks for this.

I was also having a look at something like:

{ setValues { _attachment_body : "@{myInputField}" } }
{ readClob { charSet : "UTF-8" } }

Out of interest, are there any disadvantages with that approach?

Thanks.

Re: Morphlines - converting a byte array field to a UTF-8 string

whosch — Tue, 30 Sep 2014 20:37:15 GMT

That would work fine as well.

Wolfgang.

Re: Morphlines - converting a byte array field to a UTF-8 string

sergey.sheypak566881637 — Mon, 20 Oct 2014 23:04:09 GMT

Hi, what kind of source you use to get attachment_body? Is it possible to use HttpSource as a source and morphlinesolrsink to process accepted payload (xml data)

Re: Morphlines - converting a byte array field to a UTF-8 string

PoojaK — Thu, 12 Jul 2018 06:07:16 GMT

While reading parquet file, How to convert Parquet DECIMAL datatype to String.