am having csv file data like this as shown below example 1,"Air Transport International, LLC",example,city i have to load this data in hive like this as shown below 1,Air Transport InternationalLLC,example,city but actually am getting like below?? 1,Air Transport International, LLC,example,city how to solve this problem please give me solution thanks with regards, swathi.T ... View more

Hi friends, In python how to control configuration files am having two configuration file one is by default configuration another real time config file if the real time config file having some object details some defined as empty if any defined values are empty it should take from default configure json file otherwise it not empty should take from current config json file?? step1: am having on config json file as shown below: [{ "parser_config": { "vista": { "Parser_info": { "module_name": "A", "class_name": "a", "standardformat1": { "transaction": "filename", "terminal": "filenme2", "session": "filename3" } } }, "Agilis": { "Parser_info": { "module_name": "B", "class_name": "b", "standardformat1": { "transaction": "filename", "terminal": "filenme2", "session": "filename3" } } } }, "merger_processor_config": { "Merger": { "standardformat1": { "vista": { "file_names": ["transaction", "terminal"], "parser_output_join_columns": ["TERMINALID"] }, "commander": { "file_names": ["transaction", "terminal"], "parser_output_join_columns": ["TERMINALID"] } }, "source_merge_join_columns": [ "TERMINALID", "BUSINESSDATE", "TXNSEQNO" ], "input_path": "/tenantShortName/yyyyMMdd/", "primary_datatype": "Vista", "module_name": "A", "class_name": "a", "merged_output": "true", "merger_output_filepath": "/customer1/MergerOutput/" }, "Processor": { "terminal_transaction_fact_processor": { "module_name": "nme1", "class_name": "nme2", "usecase_processor": { "usecase1": { "module_name": "nme1", "class_name": "nme2" } } } } }, "path_details": { "parent_path": "wasbs://XXXX@XXXXXstorage.blob.core.windows.net/tenantShortName/" }, "db_details": { "datawarehouse_url": "", "datawarehouse_username": "", "datawarehouse_password": "" } }] step2: am calling that configuration file objects here import json from pprint import pprint with open('Config.json') as data_file: data = json.load(data_file) #val = data["parser_config"] for val in data: print"parser_config =====",val["parser_config"]["vista"] step3: If am having another swathi_configure json file in that file am having few values of objects as shown in below [{ "parser_config": { "vista": { "Parser_info": { "module_name": "A", "class_name": "a", "standardformat1": { "transaction": "filename", "terminal": "filenme2", "session": "filename3" } } }, "path_details": { "parent_path": "wasbs://XXXX@XXXXXstorage.blob.core.windows.net/tenantShortName/" }, "db_details": { "datawarehouse_url": "", "datawarehouse_username": "", "datawarehouse_password": "" } }] step4: In the config file having only three objects parser_config,path_details,db_details remaining objects are not existed not existed fields to be from default config file if any objects not existed in current config file it should go for default config file and collects like a trace back. If the current config file having objects it should take current of objects in from current config json file how to write a code to control on config.json& swathi_configure.json please help today i need to complete this task give me how to solve this problem Thanks in advance swathi.tukkaraju ... View more

please help me how to access HDFS &b hive tables in (Horton works) Spark environment using python code how Acess the data sets into Spark i need step by step in details please help me how to do this scenario i need to finish this task by tomorrow please help me thanks with regards swathi.T ... View more