About kshashi205

kshashi205 · ‎05-18-2017

also another issue we have : we need to get back the xmls from the avro, whenever we required. Is there any solution for this.Please let me know. Thanks in Advance.

kshashi205 · ‎05-17-2017

Use case Description: We are receiving the xml from the source and the expected xmls per day is around 1lakh. we thought is merge all the xmls of a day and convert to avro. sample data: extended schema xml xsd: <?xml version="1.0"?> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element name="employee" type="fullpersoninfo"/> <xs:complexType name="personinfo"> <xs:sequence> <xs:element name="firstname" type="xs:string"/> <xs:element name="lastname" type="xs:string"/> </xs:sequence> </xs:complexType> <xs:complexType name="fullpersoninfo"> <xs:complexContent> <xs:extension base="personinfo"> <xs:sequence> <xs:element name="address" type="xs:string"/> <xs:element name="city" type="xs:string"/> <xs:element name="country" type="xs:string"/> </xs:sequence> </xs:extension> </xs:complexContent> </xs:complexType> </xs:schema> Issue: i have used databricks jar to convert xml to avro it worked for simple xml, but it didnt work for schema extended xml. Is there any workaround to convert this type of xml to avro

kshashi205 · ‎01-24-2017

Thanks for replying, In our cluster, the SSL is disabled. Now i executed a command, which curl directly to the ResourceManager. COMMAND: curl -v -X POST "http://xxxxxxxxxx.xxxxx.xxxxxxxx.net:8088/ws/v1/cluster/apps/new-application" ERROR: * About to connect() to xxxxxxxxxx.xxxxx.xxxxxxxx.net port 8088 (#0) * Trying 53.244.194.222... connected * Connected to xxxxxxxxxx.xxxxx.xxxxxxxx.net (53.244.194.222) port 8088 (#0) > POST /ws/v1/cluster/apps/new-application HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-suse-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8j zlib/1.2.7 libidn/1.10 > Host: xxxxxxxxxx.xxxxx.xxxxxxxx.net:8088 > Accept: */* > < HTTP/1.1 403 Forbidden < Cache-Control: no-cache < Expires: Tue, 24 Jan 2017 07:40:09 GMT < Date: Tue, 24 Jan 2017 07:40:09 GMT < Pragma: no-cache < Expires: Tue, 24 Jan 2017 07:40:09 GMT < Date: Tue, 24 Jan 2017 07:40:09 GMT < Pragma: no-cache < Content-Type: application/json < X-Frame-Options: SAMEORIGIN < Transfer-Encoding: chunked < Server: Jetty(6.1.26.hwx) < * Connection #0 to host xxxxxxxxxx.xxxxx.xxxxxxxx.net left intact * Closing connection #0 The default static user cannot carry out this operation. is there any other issue ..i need to fix up?

kshashi205 · ‎01-23-2017

Trying to execute the spark program using YARN RESTAPI in HORTONWORKS 2.5 i have followed this link: https://community.hortonworks.com/content/kbentry/28070/starting-spark-jobs-directly-via-yarn-rest-api.html Issue iam facing is : In the 5step: Request an application ID from YARN curl -ikvu "knoxuser:knoxpwd" -X POST "https://$KNOX_SERVER:8443/gateway/default/resourcemanager/v1/cluster/apps/new-application" Error: curl -ikvu "knoxuser:knoxpwd" -X POST "https://$KNOX_SERVER:8443/gateway/default/resourcemanager/v1/cluster/apps/new-application" * About to connect() to $KNOX_SERVER port 8443 (#0) * Trying 53.244.194.234... connected * Connected to $KNOX_SERVER (53.244.194.234) port 8443 (#0) * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs/ * SSLv3, TLS handshake, Client hello (1): * error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol * Closing connection #0 curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol where do i find the url or i am going wrong in the url part in the command.. Please let me know, it would be great help to me.

Online	Offline
Last Visited	‎05-19-2017 07:48 AM

Date Registered	‎08-25-2016 04:12 AM
Last Visited	‎05-19-2017 07:48 AM
Total Messages Posted	6

Re: converting extended schema xml to avro

converting extended schema xml to avro

Re: How to request a applicationID using YARN REST...

How to request a applicationID using YARN REST API