Hi @Tom McCuch Great article, I have one question - when we say "ARCHIVE, which has high storage density (petabyte of storage) but little compute power" Is its a different kind of HDD - can you please give examples how it is different from the regular HDD. Thanks, Avijeet ... View more

Hi @Andrew Grande I tried to run the reconfigure.groovy - however the search api can't get anything, I tried name, uuid Caught: groovyx.net.http.HttpResponseException: Not Found groovyx.net.http.HttpResponseException: Not Found at groovyx.net.http.RESTClient.defaultFailureHandler(RESTClient.java:263) at groovyx.net.http.HTTPBuilder$1.handleResponse(HTTPBuilder.java:503) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1070) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1044) at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:515) ... View more

Hi @Andrew Grande is the nifi-api-deploy works fine for nifi-1.0.0? I was trying it got this error - can you pls suggest what's wrong. Thanks. 2016/12/02 10:57:45:199 UTC [DEBUG] BasicClientConnectionManager - Get connection for route {}->http://localhost:8080 2016/12/02 10:57:45:219 UTC [DEBUG] DefaultClientConnectionOperator - Connecting to localhost:8080 2016/12/02 10:57:45:249 UTC [DEBUG] RequestAddCookies - CookieSpec selected: best-match 2016/12/02 10:57:45:265 UTC [DEBUG] RequestAuthCache - Auth cache not set in the context 2016/12/02 10:57:45:265 UTC [DEBUG] RequestTargetAuthentication - Target auth state: UNCHALLENGED 2016/12/02 10:57:45:266 UTC [DEBUG] RequestProxyAuthentication - Proxy auth state: UNCHALLENGED 2016/12/02 10:57:45:266 UTC [DEBUG] DefaultHttpClient - Attempt 1 to execute request 2016/12/02 10:57:45:267 UTC [DEBUG] DefaultClientConnection - Sending request: GET /nifi-api/controller/process-groups/root/process-group-references HTTP/1.1 2016/12/02 10:57:45:267 UTC [DEBUG] wire - >> "GET /nifi-api/controller/process-groups/root/process-group-references HTTP/1.1[\r][\n]" 2016/12/02 10:57:45:269 UTC [DEBUG] wire - >> "Accept: */*[\r][\n]" 2016/12/02 10:57:45:269 UTC [DEBUG] wire - >> "Host: localhost:8080[\r][\n]" 2016/12/02 10:57:45:269 UTC [DEBUG] wire - >> "Connection: Keep-Alive[\r][\n]" 2016/12/02 10:57:45:269 UTC [DEBUG] wire - >> "[\r][\n]" 2016/12/02 10:57:45:269 UTC [DEBUG] headers - >> GET /nifi-api/controller/process-groups/root/process-group-references HTTP/1.1 2016/12/02 10:57:45:269 UTC [DEBUG] headers - >> Accept: */* 2016/12/02 10:57:45:269 UTC [DEBUG] headers - >> Host: localhost:8080 2016/12/02 10:57:45:269 UTC [DEBUG] headers - >> Connection: Keep-Alive 2016/12/02 10:57:45:271 UTC [DEBUG] wire - << "HTTP/1.1 404 Not Found[\r][\n]" 2016/12/02 10:57:45:273 UTC [ [ERROR] HTTP call failed. Status code: HTTP/1.1 404 Not Found: The specified resource could not be found. Caught: java.lang.AssertionError: Terminated script execution. Expression: null java.lang.AssertionError: Terminated script execution. Expression: null ... View more

Thanks @Bryan Bende, makes sense. quick follow up. So can we say we can have one cluster (say 5 node) - we can put the nifi + KAFKA (one broker in each node) in same nodes, I believe HDF cluster would be doing same way, so each node would have a nifi instance + KAFKA broker + storm supervisor. Also as NIFI doesn't do replication, it would not be disk crash proof. please correct if wrong. ... View more

Hi @Bryan Bende Great Artcile! I have one question - KAFKA is a disk-bound distributed system I believe NIFI as well disk bound as it acts a reliable-buffer similar to KAFKA + all the productivity ui based Is there a strong reason to go for both NIFI + KAFKA what are the reliability features of NIFI, is it machine/disk crash proof? The persistent/replay features of KAFKA are mostly not there in NIFI? pls correct me if wrong. ... View more

@Ned Shawa please suggest if HIVE needs to be started with ACID and compactor on? or any other property. Thanks. While creating the table got this error, on HDP2.5 java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:default.OLYMPICS cannot be declared transactional because it's an external table) Also got errors with the inferred AVRO schema How does Infer schema works in the flow, does it keep inferring for every 10 records, is that a good approach? shoudn't we use convertCSVToAVRO by providing a avsc file created by Kite. Thanks. ... View more