About blimbu

blimbu · ‎12-11-2017

@Oleg Ruchovets Hi Oleg, I have a similar situation. I am loading an orc formatted file into HDFS to which the Hive table's Location is pointing to. In my case not all my values are null, If there are 33 files in the hdfs location where I stored it as orc format, I see 33 rows with data in the Hive table but I also see another 10 rows where all null in the hive table. Have you found a solution for your issue yet.

blimbu · ‎05-15-2017

@Matt Clarke I Matt today when I ran the same process flow somehow I get the same but different error in the log file as follows: 2017-05-15 17:01:27,115 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Timer-Driven Process Thread-7] ERROR org.apache.nifi.processors.standard.GetHTTP - GetHTTP[id=a2b91b66-100d-115b-caa1-ad4dfe1342f1] received status code 401:Unauthorized from https://sharepoint site 2017-05-15 17:01:27,144 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Timer-Driven Process Thread-6] WARN org.apache.http.impl.auth.HttpAuthenticator - NTLM authentication error: Credentials cannot be used for NTLM authentication: org.apache.http.auth.UsernamePasswordCredentials

blimbu · ‎05-15-2017

@Ahmad Debbas Hi Ahmad I have a similar scenario where there is a need ingest data from Sharepoint into HDFS. How were you able to implement this. Could you please share a snapshot of your nifi dataflow.

blimbu · ‎05-12-2017

I am trying to connect to a secured sharepoint site using a GetHTTP processor and get the following is the error logs. Please help!! 2017-05-12 13:16:47,790 ERROR [NiFi logging handler] org.apache.nifi.StdErr [NiFi Web Server-7240] INFO org.apache.nifi.web.security.NiFiAuthenticationFilter - Attempting request for (eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJjbj1CTGltYnUsb3U9SW5zdGl0dXRpb24sb3U9UGVvcGxlLGRjPW1kYW5kZXJzb24sZGM9ZWR1IiwiaXNzIjoiTGRhcFByb3ZpZGVyIiwiYXVkIjoiTGRhcFByb3ZpZGVyIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiQkxpbWJ1Iiwia2lkIjozLCJleHAiOjE0OTQ2MTY5OTgsImlhdCI6MTQ5NDYwOTc5OH0.cV2-fJJjiuJNnK8qVQcbJFT0PI_lDU-UfBFxKko-Ddo) GET https://dcdrlhadoop1a:8085/nifi-api/flow/process-groups/015b100c-a2b9-1b66-630f-71e1ddfae949 (source ip: 172.18.58.232)

blimbu · ‎05-11-2017

I am using a GetHTTP processor to get files from a secured SharePoint site. I created a SSL Context Service using the keystore and truststore jks files from the nifi properties. But when I run this processor I get the error: ERROR org.apache.nifi.processors.standard.GetHTTP - GetHTTP[id=015b100d-a2b9-1b66-7de9-4b67826a6492] received status code 401:Unauthorized from "the secured sharepoint site"

blimbu · ‎03-01-2017

@Matt Thank you it was very helpful.

blimbu · ‎03-01-2017

Hi All, I am using a QueryDatabase Table processor to get data from a sql server database.My requirement is to get the data from the sql server source and put this data in HDFS. The table in sqlserver is a small table which has 240 rows so I was expecting it would generate not more than 240 flow files. But when I run this processor I see that It keeps on producing a lot more than 240 flowfiles. How do I make sure it only grabs the 240 rows and not continuously get the data over and over again. I used a merge content to merge the flowfiles, used a convertAvroToJSON processor and used PutHDFS processor. When I checked the file in HDFS it seems like it is repeating data.

blimbu · ‎01-12-2017

Hi Timothy, this is the following error i get: ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.ListSFTP java.lang.OutOfMemoryError: Java heap space

blimbu · ‎01-12-2017

It seems to me that it get stuck in the first processor itself for a long time because i don't see any data being pushed over to the next processor fetchsftp; but I don't see any errors.

blimbu · ‎01-11-2017

There is a need to load 3 terabyte of historical unix files into hdfs. I am using listsftp, fetchsftp, update attribute and puthdfs processors for this. There are 16 directories with 3 subdirectories each with 350 subdirectories each. I have set the search recursively to true in the listsftp. The dataflow works for a smaller dataset when i point to a specific directory/subdirectory/subdirectory but when i try to do for the whole parent directory the listsftp processor doesn't perform. This is a one time historical load. Is there a way i could only process one directory/subdirectory/subdirectory at one time. Has anyone come across this issue. Thank you for your help. ,

Online	Offline
Last Visited	‎01-22-2018 09:59 PM

Member Since	‎01-11-2017 10:50 PM
Last Visited	‎01-22-2018 09:59 PM
Posts	17
Kudos received	2

Cloudera Community

Re: HIVE ORC table returns NULLs

Re: GetHTTP processor Error 401:unauthorized

Re: HDFS-Sharepoint

Re: GetHTTP processor Error 401:unauthorized

GetHTTP processor Error 401:unauthorized

Re: QueryDatabase Table keeps on fetching data fro...

QueryDatabase Table keeps on fetching data from sq...

Re: Listsftp taking a long time,

Re: Listsftp taking a long time,

Listsftp taking a long time,