Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar
Master Guru

Using cloudbreak to launch HDF (specifically NiFi), an error may prevent launching an instance. In the logs (/usr/hdf/current/nifi/logs/nifi-app.log) the following error is produced:

pache.nifi.processors.hive.PutHiveStreaming could not be instantiated
java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.hive.PutHiveStreaming could not be instantiated
        at java.util.ServiceLoader.fail(ServiceLoader.java:232)
        at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
		....
      
Caused by: org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
        at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:239)
....

This is generally caused by by snappy library extracting to /tmp (generally defined in java.io.tmpdir).

To fix this, do one of the following

  • Grant nifi user access to /tmp
  • Create a tmp directory where nifi user has access. For example, create tmp directory /home/tmp
    • Then go into ambari and find parameter
      Template for bootstrap.conf
    • Scroll past all the java.arg.* arguments
    • Add the following
    java.arg.snappy=-Dorg.xerial.snappy.tempdir=/usr/hdf/current/nifi/tmp
    

That's it!

10,925 Views
Comments
avatar
Contributor

@sunile.manjee

Thank you. You posted this just in time. I had this same issue today and following your 2nd option fixed it.

avatar
Master Guru

Super! glad I can help

avatar
Expert Contributor

This isn't limited to cloud installations, any server where /tmp is mounted noexec will have this issue and that is generally considered a security best practice. HDF 3.1.0 did not have this issue

avatar
New Contributor

granted permissions to temp dir, added the snappy temp dir parameter. However, the same error shows up while launching the nifi.

avatar
New Contributor

The error reported in bootstrap log file was

2018-10-08 18:06:37,982 ERROR [NiFi logging handler] org.apache.nifi.StdErr Caused by: java.lang.UnsatisfiedLinkError: /tmp/snappy-1.0.5-libsnappyjava.so: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /tmp/snappy-1.0.5-libsnappyjava.so)