Member since
02-08-2017
20
Posts
1
Kudos Received
0
Solutions
03-29-2017
10:58 AM
Hi Everyone, I am planning for HCA - HORTONWORKS CERTIFIED ASSOCIATE Certification in somtime. I have been referring to the exam objective as mentioned below - examobjectives-hcassociate.pdf I need some tips to make more solid foundation on my preparation for this exam. Considering there is no sample test or question available , I am struggling to understand the depth , I need to go through for each topic. Considering the exam broad defines itself into below categorties as mentioned in examobjectives-hcassociate.pdf 1. Data Access 2. Data Management 3. Data Governance and Workflow 4. Operations 5. Security My query is whether we need to just focus on objective component - What each component does and How it does ? (first two sections) or whether we need to see and understand all the tutorials , blogs and go in depth. Considering it being very vast and covering many Ecosystem component , understanding this will be really helpful. Thanks, Ankur
... View more
Labels:
- Labels:
-
Apache Hadoop
02-13-2017
08:39 AM
Hi @mqureshi, I should have been specific before. My requirement is that the big xml files are coming in real time and it is needed to be ingested through Nifi to covert into avro format. I had attached some of the xmls for your reference. Kindly have a look at those and advise. I have been reading and found --> 1. TransformXML processor - convert xml to json format easily , but it requires us to know XSLT format. Kindly advise. Thanks, Ankur
... View more
02-12-2017
05:52 PM
Hi Friends, Needed a help here. I was able to covert one of xml file lying in the zip - trafficLocs_data_for_simulator.zip to avro schema by able to define its structure in EvaluteXPath (image attached for reference). Many thanks to @mqureshi for his help in solving my last question. Now I want to understand , how we handle bigger xmls , do we need to define its structure in EvaluateXPath completely or is there is a simpler way to handle this? How we handle conversion of these big xml's into avro which really exist in the real life. Please advise. Attached some xmls for your reference. Many Thanks, Ankur evaluate-xpath.png xmls.zip
... View more
02-12-2017
09:25 AM
Hi @mqureshi , Many thanks brother. I am pleased to say it is working now. You have really been helpful. Now , I want to solve some bigger questions , the xml file lying in the zip - trafficLocs_data_for_simulator.zip was a small xml. Hence we are able to define its structure in EvaluteXPath , suppose we are having bigger xmlxmls.zip How we process these big xml's which really exist in the real life. Please advise. Attached some xmls for your reference. Many Thanks, Ankur
... View more
02-11-2017
11:52 PM
Hi @mqureshi, Many thanks for your reply. I had actually tried this option of using infer avro schema before but did not work. I wanted to check whether you have used XML file as a source or CSV file. My requirement is to convert the XML file into Avro format. If you see tutorials on Nifi , I had picked up one xml file lying in the zip - trafficLocs_data_for_simulator.zip. I am attaching some snapshots also warning message which is coming. Please have a look. 1-entire-flow.gif2-getfile.gif4-evaluatexpath.gif7-inferavroschema.gifwarning-message.png Thanks, Ankur
... View more
02-11-2017
05:47 PM
Hi @mqureshi, Many thanks for the reply. I have removed space but it is still showing the same issue Failed to Parse Record Schema. The Record schema is mentioned below - {
"type" : "record",
"name" : "first",
"namespace" : "xml",
"fields" : [{"name" : "Direction_of_Travel", "type" : "string" },
{ "name" : "Latitude", "type" : "float" },
{ "name" : "Longitude", "type" : "float" },
{ "name" : "Vehicle_ID", "type" : "integer" },
{ "name" : "Vehicle_Speed","type" : "integer" }]
} Please advise. Thanks, Ankur
... View more
02-11-2017
01:18 PM
Hi Friends, Needed a help here. I have picked up one of the xmls lying in the trafficLocs_data_for_simulator.zip. My objective is to convert XML to Avro format. The tutorials has shown us how to convert XML to JSON format by using AttributesToJSON processor. I am facing issue while configuring ConvertJSONToAvro processor. I have used the Record Schema as mentioned below - {
"type": "record",
"name" : "first",
"namespace" : "xml",
"fields" : [{"name" : "Direction_of_Travel", "type" : "string" },
{ "name" : "Latitude", "type" : "float" },
{ "name" : "Longitude", "type" : "float" },
{ "name" : "Vehicle_ID", "type" : "integer" },
{ "name" : "Vehicle_Speed"," type" : "integer" }]
}
I am getting red alert symbol mentioning it as Failed to Parse Record Schema. Any pointers will help. Thanks, Ankur K
... View more
Labels:
- Labels:
-
Cloudera DataFlow (CDF)