FileSystem create exception IllegalArgumentException: "TLSv1

[#1]

CDH 5.7.3 with services HBase, HDFS, YARN, ZooKeeper, Kerberos, TLS/SSL on services, Key Trustee KMS, KeyTrustee Server Cluster all services running green

When making a the call

              org.apache.hadoop.fs.FileSystem.create(testFile, 1) // 1 being replication

Results in an IOException

              message [java.lang.IllegalArgumentException: "TLSv1], cause java.lang.IllegalArgumentException: "TLSv1]

Any idea of what this may mean?

Thanks

[#2]

Here is the full stack trace

java.io.IOException: java.lang.IllegalArgumentException: "TLSv1
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:491)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:776)
at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388)
at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1381)
at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1483)
at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1468)
at org.apache.hadoop.hdfs.DistributedFileSystem$7.doCall(DistributedFileSystem.java:451)
at org.apache.hadoop.hdfs.DistributedFileSystem$7.doCall(DistributedFileSystem.java:444)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:459)
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:387)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:909)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:890)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:816)
at com.attivio.commandline.hadoopchecker.AbstractHadoopChecker.testHDFS(AbstractHadoopChecker.java:128)
at com.attivio.commandline.hadoopchecker.HDFSChecker.internalRun(HDFSChecker.java:26)
at com.attivio.commandline.hadoopchecker.AbstractHadoopChecker.runTest(AbstractHadoopChecker.java:95)
at com.attivio.slider.commandline.hadoopchecker.HadoopChecker.runTest(HadoopChecker.java:31)
at com.attivio.slider.commandline.hadoopchecker.HadoopChecker.internalRun(HadoopChecker.java:66)
at com.attivio.commandline.hadoopchecker.AbstractHadoopChecker.runTest(AbstractHadoopChecker.java:95)
at com.attivio.commandline.hadoopchecker.AbstractHadoopChecker.run(AbstractHadoopChecker.java:90)
at com.attivio.commandline.CommandLineRunner.execute(CommandLineRunner.java:163)
at com.attivio.commandline.CommandLineRunner.execute(CommandLineRunner.java:186)
at com.attivio.commandline.CommandLineRunner.main(CommandLineRunner.java:182)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:62)
at com.install4j.runtime.launcher.UnixLauncher.main(UnixLauncher.java:57)
Caused by: java.lang.IllegalArgumentException: "TLSv1
at sun.security.ssl.ProtocolVersion.valueOf(ProtocolVersion.java:187)
at sun.security.ssl.ProtocolList.convert(ProtocolList.java:84)
at sun.security.ssl.ProtocolList.<init>(ProtocolList.java:52)
at sun.security.ssl.SSLSocketImpl.setEnabledProtocols(SSLSocketImpl.java:2504)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:463)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.jav...)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:190)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:483)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:478)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:478)
... 29 more

[#3]

Kinit needed? Maybe? Seems like a kerberos heavy stack trace.

[#4 · Accepted solution]

From the stray quote in your exception, it would seem like something in your client program (the com.install4j.runtime.launcher.UnixLauncher class invocation script, the com.attivio.commandline.CommandLineRunner, etc.) has incorrectly set the JDK used system property of "https.protocols" to a value of "TLSv1" with the quotes partially included.

if there is an override of "https.protocols" required on your JDK (such as is needed for using TLSv1.2 in JDK7 environments [1]) then please modify it to pass the value correctly without the quote wrapped around it.

For ex. this is a good way to set it:

-Dhttps.protocols=TLSv1.2

But this isn't gonna work:

-Dhttps.protocols="TLSv1.2"

From your error, if I were to guess, it seems to instead be set to something like:

-Dhttps.protocols="TLSv1,2", which gives rise to the literal error of java.lang.IllegalArgumentException: "TLSv1

[1] - https://blogs.oracle.com/java-platform-group/entry/diagnosing_tls_ssl_and_https

X-Refs: http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/sun/net/www/protocol/ht... and http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/sun/security/ssl/Protoc...