About wjsandman

wjsandman · ‎08-21-2019

HW/Cloudera support has responded with "As per Apache NiFi documentation, EncryptContent process 1.9.0 does not supports DSA 1024 bit algorithm for encryption." and a link to the Apache EncryptContent processor Problem is we have a number of clients using DSA 1024 under pgp/gpg and we'd really prefer to encrypt with NiFI. Does anyone ( Andy Lopresto are you listening?) know if this is a fundamental incompatibility with DSA, a legal issue (see RSA patent fights of the 2000's), or simply an algorithm that has yet to be implemented?

wjsandman · ‎08-21-2019

wjsandman · ‎06-10-2019

We recently upgraded from HDF 3.2.0 to 3.4.1 - In the process Ambari Metrics Collector failed to start. We've seen this before and most times /.../hbase/hbase.id and hbase.version have been deleted. This hbase-ams won't start. Having kept s copy of these files after past issues, We replaced the files and restarted AMC which started, however we seem to have lost all historical data. We did not delete any hbase data for files. Is it common for the hbase files to get deleted? If replacing the hbase.id and hbase.version is incorrect, what's the fix? Might there be a way to recover the historical metric data?

wjsandman · ‎02-20-2019

I believe the ask was can we add "Custom Fields" eg; new fields other that the defined "valid" fields. How can I add pat or time stamp etc?

wjsandman · ‎09-25-2018

I'm having this exact issue also. Ever get a resolution?

wjsandman · ‎09-25-2018

Could you share the details of you HDP upgrade? I'm suing Ambari to manage HDF (Nifi) and am having a few issues getting the services (zookeeper at this point) to come up

wjsandman · ‎09-25-2018

I am in the process of performing an inplace upgrade RHEL6.7 to RHEL7.5. The OS has bee completed. Postgres has been completed. Python has been completed. TLS in ambari as bee set to v1.2 redhat7 repositories have been defined I am presently experiencing issue with the Zookeeper client install. I'll let you know how this plays out.

wjsandman · ‎09-18-2018

And probably should have found this earlier https://community.hortonworks.com/questions/178140/how-to-configure-nifi-to-maximize-the-usage-of-sys.html

wjsandman · ‎09-18-2018

OK Answer part of my own question In the Nifi interface NOT AMBARI configs.... Controller Settings; Maximum Event Driven Thread Count default: 10 - pushed up to 36 Maximum Timer Driven Thread Count default 5 - pushed up to 15 Running new flows. Will report how this works.

wjsandman · ‎09-17-2018

In Nifi my active thread count on one of our clusters won't go above 30. How can I increase the number of Active Threads? I have two clusters, that SEEM to be configured the same. On one the Active Thread count in Nifi won't go above 30. In the other we've seen as many as 70 Active Threads. Memory is the same in each cluster, but one cluster has more cores. Is this a setting or a function of available resources (possibly cores)?

wjsandman · ‎09-06-2018

There is a known issue in the PutGCSObject if running behind a secured network and accessing GCS through a proxy. The Proxy Host and Proxy Port added to the PutGCSObject will proxy the object itself, but DOES NOT proxy the GCPCredentialsControllerService, the service that authenticates your connection using yous Service AccountJSON file. At this time, this is a Community Supported Processor, and this issue exists in HFD 3.2 (Nifi v.1.7).

wjsandman · ‎08-16-2018

I agree on deleting the hbase data, however it's the only fix I've found so far. No argument from me. "medical kit logo" - can you clarify what do you mean by this?" --- Turns out it's the maintenance mode indicator. AMC was in maintenance mode. I'd never used Maintenance Mode and was unfamiliar with the indicator. I get the bozo award this week... Sigh.

wjsandman · ‎08-16-2018

OK OK... I'm a moron... Medical logo kit... Ambari Maintenance mode. but the corruption question stands.

wjsandman · ‎08-16-2018

We’re running Hortonworks Data Flow (HDF) Nifi actually. We have recently experienced Ambari Metrics Collector (AMC) HBase corruption. As per the multitude of posted fixes we deleted hbase and hbase-tmp and restarted. Great AMC is working again. But now I’ lost my history. This has happened 3-4 times on different HDF clusters In addition in this instance Ambari displays what appears to be a medical kit logo suggesting there are still issues. Is there any good way to recover from this corruption? Hbase hbc is not of much use although is does show that /hbase does not exist in ZooKeeper even on good clusters. For the moment I’d like to get the suitcase to go away. Long term does anyone have suggestions on avoiding AMC corruption?

wjsandman · ‎05-17-2018

Heads up folks! The above responses are valid for previous version of HDF. For Ambari 2.6.1 with NIFI 1.5.0.3.1.1.0-35 or better the location changed to inside the Ambari interface. Please find Ambari GUI --> NIFI ---> Configs ----> Advanced nifi-toolkit-env ---> NIFI Toolkit Java Options I've found restarting the ambari-server and ambari-agents is no longer necessary as ambari-server pushes these options to the toolkit at runtime, overiding the configurations in the script and the configuration file.

wjsandman · ‎05-17-2018

Heads up folks! The above responses are valid for previous version of HDF. For Ambari 2.6.1 with NIFI 1.5.0.3.1.1.0-35 or better the location changed to inside the Ambari interface. Please find Ambari GUI --> NIFI ---> Configs ----> Advanced nifi-toolkit-env ---> NIFI Toolkit Java Options

wjsandman · ‎12-04-2017

I'm experiencing the same issue in 5.7.0. Can you offer any detail on the issue and why we might be sseing it in 5.7.0?

wjsandman · ‎11-18-2016

Harsh, In this thread you stated " but do look into if your users have begun creating too many tiny files as it may hamper their job performance with overheads of too many blocks (and thereby, too many mappers)." Too may tiny files is in the eye of the beholder if those files are what get you paid. I'm also seeing a block issue on wo of our nodes, but a rebalance to 10% has no effect. I've rebalanced to 8% and it improves, but I suspect we're running into a small files issue.

wjsandman · ‎09-30-2016

This is apparently a know issue with JDK 8u91 and 8u92 and Oozie's TomCat environment. See JIRA OOZIE-2553 https://issues.apache.org/jira/browse/OOZIE-2533 Per the JIRA JDK 8u90 seeems to be fine, and there's a kludged work-a-round html page if you're in clined to implement it. (See the JIRA) or... you could always downgrade your Java to 8u90.

wjsandman · ‎06-13-2016

After an upgrade to CDH 5.7.0 we are experiencing the same behavior. Did anyone have a fix/update for this issue?

Online	Offline
Last Visited	‎09-19-2019 12:10 PM

Date Registered	‎02-05-2016 01:38 PM
Last Visited	‎09-19-2019 12:10 PM
Total Messages Posted	20
Total Kudos Received	1

Re: Error while launzing oozie web console : java....

Re: What is the issue with EncryptContent that is ...

What is the issue with EncryptContent that is does...

Upgraded from HDF 3.2.0 to 3.4.1 - Ambari Metrics ...

Re: Is there any way in NiFi to add custom fields ...

Re: No package found for zookeeper_${stack_version...

Re: Impact of OS upgrade from RHEL 6-->RHEL7 for h...

Re: RHEL 6 to RHEL 7 Upgrade

Re: In Nifi my active thread count on one of our c...

Re: In Nifi my active thread count on one of our c...

In Nifi my active thread count on one of our clust...

Re: NiFi PutGCSObject Error access token

Re: Ambari Metrics Collector HBase Corruption

Re: Ambari Metrics Collector HBase Corruption

Ambari Metrics Collector HBase Corruption

Re: NiFi won't start throwing "java.lang.OutOfMemo...

Re: NiFi not starting in HDF Sandbox 3.1, cannot l...

Re: Sentry Kerberos Issue

Re: DATA_NODE_BLOCK_COUNT threshold 200,00 block(s...

Re: Error while launzing oozie web console : java....

Re: Hive reports manifest security exception after...