Member since
03-21-2018
13
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2367 | 03-31-2018 05:26 PM |
04-16-2019
04:31 AM
@Marcos Da Silva , we are having the same problem and could you please elaborate on the UDF you are talking about and if possible give the code snippet.
... View more
04-16-2019
04:31 AM
@Marcos Da Silva , we are having the same problem and could you please elaborate on the UDF you are talking about and if possible give the code snippet.
... View more
06-07-2018
05:48 PM
I am trying to use PutElasticsearchHttpRecord processor to input data into ElasticSearch using an API (port 9200) as PutElastic does use 9300 (TransportClient). I am having issue with Record Reader. I am trying to how to define the Json input file format. I am guessing I should use JasonTreeReader service for the below input json file. But how to I define the Schema Text. I am using Use 'Schema Text' Property for Schema Access Strategy, No Value Set for Schema Registry, and CustomSchema for Schema Name. *********** {"meta":{"hits":77312},"data":[{"id":"263687-86963068584","author_id":"9938-904748691663097856","site_id":"9938","channel":"earned","collaboration_status":null,"collaboration_user":null,"content":null,"copyright":null,"crawled_at":null,"date":null,"extra_properties":{"restricted_usage":true},"human_review_status":"reviewed","infused_at":"2017-10-27T11:31:22+00:00","influence":782,"interactions":{},"language":null,"license":null,"location":null,"meta":{"mention_list":["hsbc","hsbc_uk","lisa_lynch"],"token_list":["@lisa_lynch","@hsbc","@hsbc_uk","blatantly","#hacked"],"hashtag_list":["hacked"]},"metric_list":[],"native_id":"923821581314297858","parent_id":"86963041385","review":{"date":"2017-10-27T08:11:54+00:00","status":"reviewed"},"root_id":null,"sentiment":"neutral","synthesio_rank":9.633,"tags":[{"value":"bram","sentiment":null}],"title":null,"type":"post","url":"http:\/\/twitter.com\/Exploited_bot\/status\/923821581314297858#904748691663097856_923821404369358848","twitter":{"type":"retweet","retweeted_status":{"date":"2017-10-27T07:59:00Z","sentiment":"neutral","hashtag_list":["hacked"],"legacy":{"idProfile":42866066,"access":"F","idLanguage":2,"idSite":9938,"idSiteType":13,"idTone":2,"json":{"twitter_entities":{"user_mentions":[{"screen_name":"HSBC","id_str":"467368287","name":"HSBC"},{"screen_name":"HSBC_UK","id_str":"2922732233","name":"HSBC UK"}]}},"idSource":42866066,"idCountry":15},"author":{"username":"lisa_lynch","full_name":"Lisa Lynch","followers":"281","picture_url":"\/\/pbs.twimg.com\/profile_images\/2320382779\/8actjkaj2mgbnh7rw1lz_normal.jpeg","bio":"happiness is the key to success","updated_on":"2018-06-06 19:41:48","id":"42866066","native_id":"42866066","url":"https:\/\/twitter.com\/lisa_lynch","location":{"country":"GBR","state":"England","county":"London","city":"London","latitude":null,"longitude":null},"current_statistics":null,"extra_properties":null,"demographics":null,"deleted":false,"statistics_history":null},"language":"eng","source":{"location":[],"id":"9938-42866066"},"title":"@HSBC @HSBC_UK has blatantly been #Hacked","type":"post","token_list":["@hsbc","@hsbc_uk","blatantly","#hacked"],"url":"http:\/\/twitter.com\/lisa_lynch\/status\/923821404369358848#42866066","synthesio_rank":6,"site":{"type_id":"13","name":"Twitter","id":"9938","url":"http:\/\/www.twitter.com"},"twitter":[],"mention_list":["hsbc","hsbc_uk"],"location":{"country":"GBR","author_location":false},"id":"86963041385"}}},{"id":"263687-86963083523","site_id":"9938","channel":null,"collaboration_status":null,"collaboration_user":null,"content":null,"copyright":null,"crawled_at":null,"date":null,"extra_properties":{"restricted_usage":true},"human_review_status":null,"infused_at":null,"influence":null,"interactions":{},"language":null,"license":null,"location":null,"meta":{},"metric_list":[],"native_id":null,"parent_id":null,"review":null,"root_id":null,"sentiment":null,"synthesio_rank":null,"tags":[{"value":"bra","sentiment":null}],"title":null,"type":null,"url":null,"twitter":null,"deleted":true},{"id":"263687-86963103695","author_id":"9938-9624102","site_id":"9938","channel":"earned","collaboration_status":null,"collaboration_user":null,"content":null,"copyright":null,"crawled_at":null,"date":null,"extra_properties":{"restricted_usage":true},"human_review_status":"reviewed","infused_at":"2017-10-27T11:31:21+00:00","influence":459,"interactions":{"total_interactions":0,"updated_at":"2017-10-28T12:18:04.129104075Z","earned_media_value":0,"engagement_rate":0,"next_update":"2100-01-01T00:00:00Z","twitter":{"favorites":0,"updated_at":"2017-10-28T12:18:04.129104075Z","retweets":0}},"language":null,"license":null,"location":null,"meta":{"mention_list":["amandarazzano","chase","citizensone","jamiejamz85"],"token_list":["@amandarazzano","@jamiejamz85","@citizensone","@chase","decided","decline","apple","pay","purchase","minutes","buying","order","confirmation","idk","happens","3rd"]},"metric_list":[{"date":"2017-10-28T12:18:04.129104075Z","name":"retweets","value":0},{"date":"2017-10-28T12:18:04.129104075Z","name":"favorites","value":0},{"date":"2017-10-28T12:18:04.129104075Z","name":"total_interactions","value":0}],"native_id":"923821796918362112","parent_id":null,"review":{"date":"2017-10-27T08:11:49+00:00","status":"reviewed"},"root_id":null,"sentiment":"neutral","synthesio_rank":8.633,"tags":[{"value":"brac","sentiment":null}],"title":null,"type":"post","url":"http:\/\/twitter.com\/michaelweissman\/status\/923821796918362112#9624102_923818417353871360","twitter":{"interactions":{"favorites":0,"updated_at":"2017-10-28T12:18:04.129104075Z","retweets":0},"type":"tweet"}},{"id":"263687-86963092039","author_id":"9938-163603505","site_id":"9938","channel":"earned","collaboration_status":null,"collaboration_user":null,"content":null,"copyright":null,"crawled_at":null,"date":null,"extra_properties":{"restricted_usage":true},"human_review_status":"reviewed","infused_at":"2017-10-27T11:31:22+00:00","influence":551,"interactions":{"total_interactions":0,"updated_at":"2017-10-28T12:18:03.494161551Z","earned_media_value":0,"engagement_rate":0,"next_update":"2100-01-01T00:00:00Z","twitter":{"favorites":0,"updated_at":"2017-10-28T12:18:03.494161551Z","retweets":0}},"language":null,"license":null,"location":null,"meta":{"mention_list":["chase","jpmorgan"],"token_list":["@jpmorgan","@chase","poured","$20","billion","into","extreme","fossil","fuels","2014-2016","stop","#bankingonclimatechange"],"url_list":["https:\/\/www.ran.org\/banking_on_climate_change?utm_medium=social&utm_campaign=eg_eneg&utm_content=partner_social_june_2017"],"hashtag_list":["bankingonclimatechange"]},"metric_list":[{"date":"2017-10-28T12:18:03.494161551Z","name":"retweets","value":0},{"date":"2017-10-28T12:18:03.494161551Z","name":"favorites","value":0},{"date":"2017-10-28T12:18:03.494161551Z","name":"total_interactions","value":0}],"native_id":"923821720208887808","parent_id":null,"review":{"date":"2017-10-27T08:11:50+00:00","status":"reviewed"},"root_id":null,"sentiment":"positive","synthesio_rank":9.233,"tags":[{"value":"brac","sentiment":null}],"title":null,"type":"post","url":"http:\/\/twitter.com\/Shriz100\/status\/923821720208887808#163603505","twitter":{"interactions":{"favorites":0,"updated_at":"2017-10-28T12:18:03.494161551Z","retweets":0},"type":"tweet"}},{"id":"263687-86963082476","site_id":"9938","channel":null,"collaboration_status":null,"collaboration_user":null,"content":null,"copyright":null,"crawled_at":null,"date":null,"extra_properties":{"restricted_usage":true},"human_review_status":null,"infused_at":null,"influence":null,"interactions":{},"language":null,"license":null,"location":null,"meta":{},"metric_list":[],"native_id":null,"parent_id":null,"review":null,"root_id":null,"sentiment":null,"synthesio_rank":null,"tags":[{"value":"brac","sentiment":null}],"title":null,"type":null,"url":null,"twitter":null,"deleted":true},{"id":"264167-87209949943","site_id":"2965406","channel":"earned","collaboration_status":null,"collaboration_user":null,"content":null,"copyright":null,"crawled_at":null,"date":"2017-10-27T08:00:00+00:00","extra_properties":{},"human_review_status":"reviewed","infused_at":"2017-10-31T15:29:52+00:00","influence":555615,"interactions":{},"language":"eng","license":null,"location":{"country":"CHN"},"meta":{"token_list":["smarcent","q50","kid","smart","watch","anti","lost","gps","tracker","smartwatch","oled","screen","clock"]},"metric_list":[],"native_id":null,"parent_id":null,"review":{"date":"2017-10-31T06:51:51+00:00","status":"reviewed"},"root_id":null,"sentiment":"neutral","synthesio_rank":5,"tags":[{"value":"wea","sentiment":null}],"title":"Smarcent Q50 Kid Smart Watch Anti Lost GPS Tracker Smartwatch OLED Screen Clock","type":null,"url":"http:\/\/www.aliexpress.com\/item\/Q50-GPS-Kid-Smart-Watch-Anti-Lost-GPS-Tracker-Smartwatch-OLED-Screen-SOS-Call-Safe-with\/32740138314.html?ws_ab_test=searchweb0_0,searchweb201602_5_10152_10065_10151_10344_10068_10130_9983_10345_10324_10342_10547_10325_10343_10546_10340_10341_10548_10545_10541_10084_10083_10307_10178_10060_10155_10154_10056_10055_10539_10538_10537_10312_10536_10059_10313_10314_10534_10533_100031_10103_10073_10102_10142_10107-9983,searchweb201603_2,ppcSwitch_5&btsid=3ac2b40e-2e49-4d3"},{"id":"264167-87191811866","source_id":"21711-188205787","author_id":"21711-609063984","site_id":"21711","channel":"earned","collaboration_status":null,"collaboration_user":null,"content":"Aakash Smart *WATCH OUR spoof COMEDY ON \"KBC\" , YOU WILL DEFINITELY LOVE IT click link\ud83d\udc49 **https:\/\/youtu.be\/xQ1SVjHHu9E** AUR ACHA LAGE TO PLZ SUBSCRIBE \"SB KI MASTI\" APKA EK SUBSCRIBE HAMARE LIYE BOHOT HOGA\u200b* plz suport us to grow","copyright":null,"crawled_at":null,"date":"2017-10-27T08:00:00+00:00","extra_properties":{},"human_review_status":"reviewed","infused_at":"2017-12-19T22:54:41+00:00","influence":4757130,"interactions":{},"language":"eng","license":null,"location":{"country":"IND"},"meta":{"attachment_list":[{"id":"OL3yCJEm4OM","thumbnail":"https:\/\/i.ytimg.com\/vi\/OL3yCJEm4OM\/hqdefault.jpg","type":"video","url":"http:\/\/www.youtube.com\/embed\/OL3yCJEm4OM"}],"token_list":["vines","yaar","anmulle","aakash","smart","watch","spoof","comedy","kbc","definitely","love","click","link","\ud83d\udc49","aur","acha","lage","plz","subscribe","masti","apka","subscribe","hamare","liye","bohot","hoga","plz","suport","grow"],"url_list":["https:\/\/youtu.be\/xq1svjhhu9e**"]},"metric_list":[],"native_id":null,"parent_id":"87191811863","review":{"date":"2017-10-30T23:24:38+00:00","status":"reviewed"},"root_id":"86923810015","sentiment":"neutral","synthesio_rank":5,"tags":[{"value":"wea","sentiment":null}],"title":"BB Ki Vines- | Yaar Anmulle |","type":"comment","url":"https:\/\/www.youtube.com\/watch?v=OL3yCJEm4OM&lc=z23ptnvyjvzuvt0hk04t1aokgaxssbqdlvoyjaciwzvkbk0h00410#z23ptnvyjvzuvt0hk04t1aokgaxssbqdlvoyjaciwzvkbk0h00410.1509091224127460#609063984_86923810015"},{"id":"264167-86963103695","author_id":"9938-9624102","site_id":"9938","channel":"earned","collaboration_status":null,"collaboration_user":null,"content":null,"copyright":null,"crawled_at":null,"date":null,"extra_properties":{"restricted_usage":true},"human_review_status":"reviewed","infused_at":"2017-12-19T23:10:32+00:00","influence":459,"interactions":{"earned_media_value":0,"engagement_rate":0,"next_update":"2100-01-01T00:00:00Z","total_interactions":0,"updated_at":"2017-10-28T12:18:04.129104075Z","twitter":{"favorites":0,"retweets":0,"updated_at":"2017-10-28T12:18:04.129104075Z"}},"language":null,"license":null,"location":null,"meta":{"mention_list":["amandarazzano","chase","citizensone","jamiejamz85"],"token_list":["@amandarazzano","@jamiejamz85","@citizensone","@chase","decided","decline","apple","pay","purchase","minutes","buying","order","confirmation","idk","happens","3rd"]},"metric_list":[{"date":"2017-10-28T12:18:04.129104075Z","name":"retweets","value":0},{"date":"2017-10-28T12:18:04.129104075Z","name":"favorites","value":0},{"date":"2017-10-28T12:18:04.129104075Z","name":"total_interactions","value":0}],"native_id":"923821796918362112","parent_id":null,"review":{"date":"2017-10-31T15:43:57+00:00","status":"reviewed"},"root_id":null,"sentiment":"neutral","synthesio_rank":8.633,"tags":[{"value":"otha","sentiment":null}],"title":null,"type":"post","url":"http:\/\/twitter.com\/michaelweissman\/status\/923821796918362112#9624102_923818417353871360","twitter":{"interactions":{"favorites":0,"retweets":0,"updated_at":"2017-10-28T12:18:04.129104075Z"},"type":"tweet"}},{"id":"264167-86963100947","author_id":"9938-2910690584","site_id":"9938","channel":null,"collaboration_status":null,"collaboration_user":null,"content":null,"copyright":null,"crawled_at":null,"date":null,"extra_properties":{"restricted_usage":true},"human_review_status":null,"infused_at":null,"influence":null,"interactions":{},"language":null,"license":null,"location":null,"meta":{},"metric_list":[],"native_id":null,"parent_id":null,"review":null,"root_id":null,"sentiment":null,"synthesio_rank":null,"tags":[{"value":"wea","sentiment":null}],"title":null,"type":null,"url":null,"twitter":null,"deleted":true},{"id":"264167-86975413641","site_id":"220766","channel":"earned","collaboration_status":null,"collaboration_user":null,"content":"Spanje. Meer bepaald: Cataloni\u00eb. Kan iemand mij bij zijn gezond verstand nu uitleggen waarom ze daar zich als een bende klein kinderen gedragen? De ene roept dat hij onafhankelijk wil zijn. De ander roept terug dat hij da tniet mag. En hij haalt er de schoolpestkop bij die iedereen die tegenspreekt hardhandig de kop in slaat. Ik ben maar een Domme Belg, maar ik begin me toch stilletjesaan de vraag te stellen of we niet allemaal traag maar zeker ons verstand aan het verliezen zijn. We lijken allemaal op kleine kinderen die om het luidst roepen dat we zelfstandig willen zijn. Tegelijkertijd roepen we ook dat onze regeringen bedrijven te veel macht geven. Verbonden Beiden zijn met elkaar vebonden. Want\u00a0 bedrijven hebben nu al enorm veel macht (en geld) en zullen in de toekomst enkel meer macht (en geld) krijgen. Landen, daarentegen, worden steeds kleiner en kleiner. Vlaanderen, Cataloni\u00eb, andere Spaanse gebieden, Noord-Itali\u00eb, Friesland en nog zovele andere plaatsen binnen Europa. Mensen komen letterlijk op straat omdat ze geloven dat ze het geld dat ze betalen aan belastingen, v\u00e9\u00e9l beter zelf kunnen beheren. De volgende dag komen ze dan weer op straat omdat ze het niet vinden kunnen dat multinationals belastingsafspraken maken met bepaalde landen zodat ze overal anders geen of nauwelijks belastingen betalen. Geld In beide gevallen gaat het, hoe kan het ook anders, om geld. En in beide gevallen roepen vaak dezelfde mensen dat het \u2018allemaal niet eerlijk is\u2019. Geld van het \u2018rijke\u2019 deel van het land (want laten we eerlijk zijn, die willen vaak onafhankelijkheid) mag niet meer naar het arme deel gaan. Tegelijkertijd willen we dat multinationals ook belastingen betalen. Wij moeten dat toch niet alleen doen? De overeenkomsten in beide gevallen? Ik. Ik. Ik. En waar hebben we dat nog gehoord? Top 10 Wie de moeite doet de tien rijkste landen ter wereld op te zoeken, komt tot enkele verrassende conclusies. Het mag misschien geen verrassing zijn dat Luxemburg ook tot \u00e9\u00e9n van de rijkste landen behoort. De 10 rijkste bedrijven hebben we ook op een rijtje gezet. Denk aan bedrijven zoals Shell (op nummer 10). Of JP Morgan Chase op 3, China Construction Bank op 2 en ICBC op 1 (Industrial and Commercial Bank of China Limited). Mag het verbazen dat de top 3 uit banken bestaat? Maar het wordt nog interessanter wanneer we de 10 machtigste landen en bedrijven op een rijtje zetten. Want wat\u00a0 blijkt? Dat heel wat bedrijven een heel stuk m\u00e9\u00e9r verdienen dan verschillende landen samen. Niet echt een hoopgevende vaststelling. Oplossing Wie dus graag onafhankelijkheid wil, kan dat krijgen. Maar je zal er wel een hoge prijs voor moeten betalen, wan tuiteindelijk zullen bedrijven het voor het zeggen hebben \u2013 en niet wij. Wil je dat voorkomen, dan zal je daar toch iets tegenover moeten zetten. Ofwel kies je voor onafhankelijkheid, ofwel voor een fair belastingssysteem, want een bedrijf dat kan kiezen, gaat altijd naar het land waar het het beste voordeel kan krijgen. De enige manier waarop je dat faire belastingssysteem gaat krijgen, is door voor Europa te kiezen. Ja, die [...]","copyright":null,"crawled_at":null,"date":"2017-10-27T08:00:00+00:00","extra_properties":{"excerpt":{"content":"Spanje. Meer bepaald: Cataloni\u00eb. Kan iemand mij bij zijn gezond verstand nu uitleggen waarom ze daar zich als een bende klein kinderen gedragen? De ene roept dat hij onafhankelijk wil zijn. De ander roept terug dat hij da tniet mag. En hij haalt er de "}},"human_review_status":"reviewed","infused_at":"2017-12-19T23:10:32+00:00","influence":93,"interactions":{},"language":"nld","license":null,"location":{"country":"BEL"},"meta":{"token_list":["domme","belg","112","weet","eigenlijk","willen","spanje","bepaald","cataloni\u00eb","gezond","verstand","uitleggen","waarom","bende","klein","kinderen","gedragen","ene","roept","onafhankelijk","ander","roept","terug","tniet","mag","haalt","schoolpestkop","iedereen","tegenspreekt","hardhandig","kop","slaat","domme","belg","begin","stilletjesaan","vraag","stellen","allemaal","traag","zeker","verstand","verliezen","lijken","allemaal","kleine","kinderen","luidst","roepen","zelfstandig","willen","tegelijkertijd","roepen","regeringen","bedrijven","macht","geven","verbonden","beiden","elkaar","vebonden","bedrijven","enorm","macht","geld","zullen","toekomst","enkel","macht","geld","krijgen","landen","daarentegen","steeds","kleiner","kleiner","vlaanderen","cataloni\u00eb","spaanse","gebieden","noord-itali\u00eb","friesland","zovele","plaatsen","binnen","europa","mensen","komen","letterlijk","straat","geloven","geld","betalen","belastingen","v\u00e9\u00e9l","beter","beheren","volgende","dag","komen","straat","vinden","multinationals","belastingsafspraken","maken","bepaalde","landen","zodat","overal","anders","nauwelijks","belastingen","betalen","geld","beide","gevallen","anders","geld","beide","gevallen","roepen","vaak","dezelfde","mensen","allemaal","eerlijk","geld","rijke","deel","land","laten","eerlijk","willen","vaak","onafhankelijkheid","mag","arme","deel","tegelijkertijd","willen","multinationals","belastingen","betalen","moeten","overeenkomsten","beide","gevallen","gehoord","top","moeite","tien","rijkste","landen","ter","wereld","zoeken","enkele","verrassende","conclusies","mag","misschien","verrassing","luxemburg","\u00e9\u00e9n","rijkste","landen","behoort","rijkste","bedrijven","rijtje","gezet","denk","bedrijven","shell","nummer","morgan","chase","china","construction","bank","icbc","industrial","and","commercial","bank","china","limited","mag","verbazen","top","banken","bestaat","interessanter","wanneer","machtigste","landen","bedrijven","rijtje","zetten","blijkt","bedrijven","stuk","m\u00e9\u00e9r","verdienen","verschillende","landen","samen","hoopgevende","vaststelling","oplossing","graag","onafhankelijkheid","krijgen","hoge","prijs","moeten","betalen","wan","tuiteindelijk","zullen","bedrijven","zeggen","voorkomen","tegenover","moeten","zetten","ofwel","kies","onafhankelijkheid","ofwel","fair","belastingssysteem","bedrijf","kiezen","land","beste","voordeel","krijgen","enige","manier","waarop","faire","belastingssysteem","krijgen","europa","kiezen"]},"metric_list":[],"native_id":null,"parent_id":null,"review":{"date":"2017-10-31T15:43:57+00:00","status":"reviewed"},"root_id":null,"sentiment":"negative","synthesio_rank":5,"tags":[{"value":"otha","sentiment":null}],"title":"De Domme Belg 112 : Weet nog iemand wat we eigenlijk willen ?","type":null,"url":"https:\/\/computertaal.info\/2017\/10\/27\/domme-belg-112-weet-nog-iemand-we-eigenlijk-willen\/"}],"included":{"author":[{"id":"9938-904748691663097856","native_id":"904748691663097856","site_id":9938,"url":null,"location":null,"extra_properties":{},"demographics":{},"deleted":false,"full_name":null,"picture_url":null,"username":null,"metric_list":{"date":null,"name":"followers","value":null},"type":"user"},{"id":"9938-9624102","native_id":"9624102","site_id":9938,"url":null,"location":null,"extra_properties":{},"demographics":{},"deleted":false,"full_name":null,"picture_url":null,"username":null,"metric_list":{"date":null,"name":"followers","value":null},"type":"user"},{"id":"9938-163603505","native_id":"163603505","site_id":9938,"url":null,"location":null,"extra_properties":{},"demographics":{},"deleted":false,"full_name":null,"picture_url":null,"username":null,"metric_list":{"date":null,"name":"followers","value":null},"type":"user"},{"id":"21711-609063984","native_id":"UCXMOIIPN26qYDis9_AfLL3A","site_id":21711,"url":"https:\/\/www.youtube.com\/channel\/UCXMOIIPN26qYDis9_AfLL3A","location":{"country":"ZZZ"},"extra_properties":{},"demographics":{},"deleted":false,"full_name":"SHUBHAM SINGH RAJPUT","picture_url":"\/\/yt3.ggpht.com\/-EZvRZiSooRs\/AAAAAAAAAAI\/AAAAAAAAAAA\/d9CLTPIhpK0\/s240-c-k-no-mo-rj-c0xffffff\/photo.jpg","username":"UCXMOIIPN26qYDis9_AfLL3A","type":"user","metric_list":{"date":"2018-01-20 19:21:12","name":"subscribers","value":419}},{"id":"9938-2910690584","native_id":null,"site_id":9938,"url":null,"location":null,"extra_properties":{},"demographics":{},"deleted":false,"full_name":null,"picture_url":null,"username":null,"metric_list":{"date":null,"name":"followers","value":null},"type":"user"}],"site":[{"id":"9938","site_type_id":"13","communities_list_id":[],"name":"Twitter","url":"http:\/\/www.twitter.com","logo":null,"synthesio_rank":null},{"id":"2965406","site_type_id":"15","communities_list_id":[],"name":"Aliexpress","url":"http:\/\/www.aliexpress.com","logo":null,"synthesio_rank":null},{"id":"21711","site_type_id":"14","communities_list_id":[],"name":"YouTube","url":"http:\/\/www.youtube.com","logo":null,"synthesio_rank":null},{"id":"220766","site_type_id":"9","communities_list_id":["15"],"name":"Computertaal","url":"http:\/\/www.computertaal.info","logo":null,"synthesio_rank":null}],"source":[{"id":"21711-188205787","native_id":"UCqwUrj10mAEsqezcItqvwEw","site_id":21711,"url":"https:\/\/www.youtube.com\/channel\/UCqwUrj10mAEsqezcItqvwEw","location":{"country":"IND"},"extra_properties":{},"demographics":{},"deleted":false,"full_name":"BB Ki Vines","picture_url":"\/\/yt3.ggpht.com\/-V7WGpnvPvtI\/AAAAAAAAAAI\/AAAAAAAAAAA\/BI3_YxcidJs\/s240-c-k-no-mo-rj-c0xffffff\/photo.jpg","username":"UCqwUrj10mAEsqezcItqvwEw","type":"user","metric_list":{"date":"2018-01-30 19:08:09","name":"subscribers","value":5925738}}]}}
... View more
Labels:
06-07-2018
05:34 PM
I have added Accept and Authorization in UpdateAttribute processor before InovkeHTTP processor and in the Attributes to Send property (in InovkeHTTP) - I have used .*
... View more
06-04-2018
02:18 PM
@Felix Albani Thank you very much. that worked perfectly. I have one more question. What if I have one more header such as -H "Authorization: Bearer 27ffdfada34982-1123qwdq2" . How do I send it ? Should I be adding one more property and set the name as 'Authorization and Value as Bearer 27ffdfada34982-1123qwdq2 ? or Can I specify more than one attribute in AttributestoSend in InvokeHTTP processor ? If yes, should they be seperated by comma or space or something else ?
... View more
06-01-2018
04:39 PM
I have curl command as below. curl -s -XPOST -H "Accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d 'username=usco&password=asdl2342&client_id=dwer234324-2sd2-32dwq&scope=read' 'https://rest.abc.com/security' I am trying to replicate this in Nifi. I am trying to figure out how and where the above options to be provided in InvokeHTTP processor. I have gone through the articles and some mentioned I have to define it in body of flow file. But I couldn't understand what exactly goes into body and what goes into extra attributes (if I need to add) in InvokeHTTP processsor.
... View more
Labels:
04-05-2018
08:24 PM
I have a flow file (Jason file with Z records) and I am using a split processor to divide it into Z flow files. I am doing so as I need to do a look up. Once I split them X flow files go on to the left and Y flow files goes to right (where X+Y = Z). Now I have to merge these Y flow files into one flow file. How do I do that ? And once that is done I need to do some processing and then I need to combine that 1 flow file with X flow files to make it 1 flow file with 'Z' records. Any idea on how to achieve it with wait and notify processor ?
... View more
Labels:
03-31-2018
05:26 PM
1 Kudo
I am able to resolve this. var flowFile = session.get();
if (flowFile != null) {
var StreamCallback = Java.type("org.apache.nifi.processor.io.StreamCallback")
var IOUtils = Java.type("org.apache.commons.io.IOUtils")
var StandardCharsets = Java.type("java.nio.charset.StandardCharsets")
var JString = Java.type("java.lang.String")
var TokenAttr = flowFile.getAttribute('token_value')
var JsonTokenAttr = JSON.parse(TokenAttr) ; flowFile = session.write(flowFile,
new StreamCallback(function(inputStream, outputStream) {
var text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
var obj = JSON.parse(text)
var accounts = obj['customer-accounts'];
var convertedAccounts = accounts.map(function(account, index) {
account['account_number'] = JsonTokenAttr.Tokens[index];return account;
});
var newObj = {'customer-accounts' : convertedAccounts }
var builder = new JString()
outputStream.write(JSON.stringify(newObj, null, '\t').getBytes(StandardCharsets.UTF_8))
})) session.transfer(flowFile, REL_SUCCESS)
}
... View more
03-29-2018
03:34 AM
I am trying to write and execute a Java script in Nifi. Below is all I have defined in the body of Execute script processor. token_value is an attribute and the value for it is - {
"Tokens": [
"12345acdb4567",
"98765acdb1111"
]
} Flow File Content is - { "customer-accounts": [ { "company_identifier": "074", "product_code": "CCD", "account_number": "222222222222", "relationship_code": "JOF", "primary_ownership_indicator": "Y" }, { "company_identifier": "674", "product_code": "CCD", "account_number": "111111111111", "relationship_code": "EOD", "primary_ownership_indicator": "N" } ] } Script I am using is as below. I am not familiar with JavaScript. Trying to get this to work. ScriptBody : var flowFile = session.get() ; if (flowFile != null) {
var TokenAttr = flowFile.getAttribute('token_value') ;
} function convert(flowFile, TokenAttr) {
let accounts = flowFile['customer-accounts'];
let convertedAccounts = accounts.map((account, index) => {
account['account_number'] = TokenAttr.Tokens[index];
return account;
});
return {
'customer-accounts' : convertedAccounts
}
} flowFile = session.write(flowFile,console.log(JSON.stringify(convert(flowFile,TokenAttr)))) session.transfer(flowFile, REL_SUCCESS) your help will be appreciated. Thanks in advance.
... View more
Labels:
03-27-2018
10:26 PM
Hello, In our flow there will be an attribute in which we will have Jason data. And that data doesn't exist in actual flow file content. And I would like to extract values of certain fields of the Jason data of that attribute. Like EvaluateJasonPath works on flow file content, is there a processor that works on Attributes which has Jason data ? One Idea I have is to swap the content into attribute and and attribute data into flow file content and do what ever I want to do and swap it back. But one concern I have is keep the flow file content in attribute as it will be stored on memory. Thanks in advance for the help -Raja
... View more
Labels:
03-23-2018
03:19 PM
Hi @Abdelkrim Hadjidj Thank you very much for the quick reply. Now I did go through the update processor. The issue I am unable to figure out is, how to reference the array of Attribute values in update processor. To my initial question there are two records in Jason file and 2 values in attribute "Attr_X". So I want the first value of "atct_no" which is 123456789 to be replaced by 2020202020, and in the same way 234567891 to be replaced by 1010101010. Thank you, Raja
... View more
03-22-2018
08:51 PM
I have a flow file content like below {
"atct": [
{
"flag": "Y",
"atct_no": 123456789,
"debr" : "EAR"
},
{
"flag": "Y",
"atct_no": 234567891,
"debr" : "EYE"
}
]
} And then I have an attribute Attr_X which has value - [ "2020202020", "1010101010"] I would like to replace "atct_no" values in flow file content and I need it to look like (if possible with out splitting the flow file) { "atct": [ { "flag": "Y", "atct_no": 2020202020, "debr" : "EAR" }, { "flag": "Y", "atct_no": 1010101010, "debr" : "EYE" } ] }
... View more
Labels:
03-21-2018
04:13 AM
I have a sample Jason as provided. I have to check the values for "po_bo" and if it is 'DEF' then send the records to relationship (1) if the value not DEF send it to other relation ship (2). For the records that are sent to the relationship (1) we need extract the list of "an_tn" values and send it to a web service and the result will be encrypted values in an array in Jason format (result will be in the same order that you sent to web service). Now we need to replace the actual "an_tn" values with encrypted values. And then merge it with the values that we sent to relationship(2) in the beginning. Finally it should look like the original that file that was provided to us but the "an_tn" values encrypted.
{
"pc_mi": {
"etvn": {
"etvn_hub": "Prod Msg",
"etvn_subr": "System",
"etvn_tiar": "True",
"etvn_itni_itdy": "False"
},
"partition": [
{
"bo_id": "11",
"po_bo": "ABC",
"an_tn": "1111111111111",
"nwr_ck": "T"
},
{
"bo_id": "11",
"po_bo": "DEF",
"an_tn": "2222222222222",
"nwr_ck": "T"
},
{
"bo_id": "11",
"po_bo": "DEF",
"an_tn": "3333333333333",
"nwr_ck": "T"
}
]
}
}
... View more
- Tags:
- nifi-processor
- split
Labels: