Solved

Hi, I am trying to run nexgtech/knox as a container in a pod in a kubernetes cluster and i am getting the below   Exception in thread "main" java.lang.reflect.InvocationTargetExceptionat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.knox.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:68)at org.apache.knox.gateway.launcher.Invoker.invoke(Invoker.java:39)at org.apache.knox.gateway.launcher.Command.run(Command.java:99)at org.apache.knox.gateway.launcher.Launcher.run(Launcher.java:75)at org.apache.knox.gateway.launcher.Launcher.main(Launcher.java:52)Caused by: java.lang.NoSuchFieldError: DEFAULT_XML_TYPE_ATTRIBUTEat org.eclipse.persistence.jaxb.compiler.MappingsGenerator.setupInheritance(MappingsGenerator.java:2431)at org.eclipse.

  Hello Folks,    I am trying to debug Ambari alerts , I am not getting logs generated in /var/log/ambari-server/ directory.   Ambari Version 2.5.1.0   HDP -  1) Tried ambari-server restart  2) enabled debug mode mentioned in    tried steps mentioned in  https://community.cloudera.com/t5/Support-Questions/Not-able-to-receive-emails-via-Ambari-Alerts/td-p/122948   http://mail-archives.apache.org/mod_mbox/ambari-user/201504.mbox/%3CA0F1A095-D4F9-47CE-BEC7-12C4EB54E5EC@hortonworks.com%3E   @nsabharwal- Could you please provide some ways to debug      

Hello,  I create a new cluster with relaxed security and disabled TLS on all services. I'm able to connect on Hive/Impala using JDBC Connection (Squirrel), with no user and password using this conection string:  Impala: jdbc:impala://Impala_Daemon_Host:21050/default. Impala: jdbc:hive2://Hive_Gateway_Host:10000/default   Now I activated TLS, and create a user on SO (newuser1).  Now, I'm trying to connect using the same JDBC connection, and using the new user and password, with no sucess.   Any advice are welcome.  

Hello All,   I would need to use Path filter for searching subfolders starting by AX. AX1/ AX2/ XY1/ XY2/   I tried this in path filter: AX.*/   Is that correct? Thanks everybody

Is there a way to run spark-submit (spark v2.3.2 from HDP 3.1.0) while in a virtualenv? Have situation where have python file that uses python3 (and some specific libs) in a virtualenv (to isolate lib versions from rest of system). I would like to run this file with /bin/spark-submit, but attempting to do so I get... [me@myserver tests]$ source ../venv/bin/activate; /bin/spark-submit sparksubmit.test.py File "/bin/hdp-select", line 255 print "ERROR: Invalid package - " + name ^SyntaxError: Missing parentheses in call to 'print'. Did you mean print("ERROR: Invalid package - " + name)?ls: cannot access /usr/hdp//hadoop/lib: No such file or directoryException in thread "main" java.lang.IllegalStateException: hdp.version is not set while running Spark under HDP, please set through HDP_VERSION in spark-env.sh or add a java-opts file in conf with -Dhdp.version=xxx at org.apache.spark.launcher.Main.main(Main.java:118)# also tried...(ve

Dear All.   I having issue with creating hive table with hdfs data in Azure HDInsight cluster.   i am able to create hive table from BLOB Storage but not with HDFS Data.   as below example code for taking data from blob storage:- hive> CREATE EXTERNAL TABLE IF NOT EXISTS tsvtab (> name string,> region_code int,> sal int,> add string> )> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'> STORED AS TEXTFILE; load data inpath 'wasb://hadoophdinsightigi-2019-10-21t07-33-15-078z@hadooighdistorage.blob.core.windows.net/user/sshuser/data.tsv' into table tsvtab;   below taking data from hdfs but not working:-   hive> CREATE EXTERNAL TABLE IF NOT EXISTS tsvtab (> name string,> region_code int,> sal int,> add string> )> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'> STORED AS TEXTFILE; load data inpath 'wasb://hadoophdinsightigi-2019-10-21t07-33-15-078z@hadohdistorage.blob.core.windows.net//user/HadoopPOCDir/data.tsv' into ta

Hi, We are currently using merge content processor to merge kafka messages with "minimum number of entries" = 500. when 500 messages reached it will merge as single file. That use case works fine. whenever end of the day at 11:59, is there any queue pending in merge content processor of kafka messages needs to dynamically pushed before start to the new date. Kindly help me out from this kind of use case. Workflow :kafka_consumer -> merge_content_prcessor -> puthdfs@mburgess 

Hello,I am working with 3 nodes CFM cluster. the version of CFM is 1.0.1.  The below pic shows the steps: the below pic is config of ExecuteSQL:the below pic is lookup service:the below is the config of dbcpconnetionpool:The behavior is the ExcuteSQL processor will execute very slowly seem hanged.  but sometime execute very fast. I believe that is database issue. So my questions is:1. what is the impact? is dbcp connections pool issue of config? or is the dbcp connections pool lookup wrong behavior?  2. I'm afraid the impact of this will be magnified. when increased database connection pool to 500+.Could help me to answer my questions?ThanksPaul

Hi Team, Greetings....!!My cluster license got expired and still it was showing "Cloudera Enterprise ".Is any problem using after license expiration ?Can some one suggest me how to downgrade to express edition ? Thanks,Vinod

Hey, so I want to create a flow that creates a putemail trigger when NO files are present in a hdfs path for a certain date range: my current flow is GETHDFS->  ROUTEONATTRIBUTE(has the proper date matching criteria) -->MATCHED(should be getting files here) -->UNMATCHED(when this outputs zero I want to receive an email alert)    RouteOnAttribute Logic: ${Path:substring(0,6):equals(${now():toNumber():minus(86400000):format('yyyyMM')})}   Pretty much if I find no files within the current month(last 20 days) then I need to receive an email alert. How do I get an alert when there are no files that fit the criteria?    @Former Member  Thank you matt sorry for yet another question.

I tried to install HortonWorks HDP Sandbox with Docker on a Windows10 Machine, as described here: https://www.cloudera.com/tutorials/sandbox-deployment-and-install-guide/3.html   I executed the commandsh docker-deploy-hdp30.shin a Git Bash console window, having admin rights. It seems that pulling the image worked but I got following error:   C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: An unexpected error occurred while sharing drive.   =>How can I resolve that? Does the docker image expect that a certain drive exists?   Some more output from the console:   ... 4b9047c5fbbb: Download complete951bdea65c93: Pull complete4b9047c5fbbb: Pull complete773156407aae: Pull completed8524176841d: Pull completeDigest: sha256:42e4cfbcbb76af07e5d8f47a183a0d4105e65a1e7ef39fe37ab746e8b2523e9eStatus: Downloaded newer image for hortonworks/sandbox-proxy:1.0docker.io/hortonworks/sandbox-proxy:1.0+ '[' hdp == hdf ']'+ '[' hdp == hdp ']'+

Apache Nifi PutHiveQL How do I extract error message into flowfile attribute when failure occurs?

Hi,  We are running CDH 5.14.0 version of cloudera.There are multiple warnings coming related to log data directory for only one Data node( hadoop-slave-1). please see warnings.   for the same node,root / dir space is showing critical. can anyone suggest what files i should delete from "/ ,so that i can clear these alarms .   Thanks Devendra    

java.lang.ClassNotFoundException    error while running/usr/hcp/2.0.0.0-4/metron/bin/threatintel_taxii_load.sh SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/usr/hdp/3.1.4.0-315/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/usr/hdp/3.1.4.0-315/phoenix/phoenix-5.0.0.3.1.4.0-315-server.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]Exception in thread "main" org.codehaus.jackson.map.JsonMappingException: org.apache.metron.stellar.dsl.VariableResolver (through reference chain: org.apache.metron.dataloads.extractor.ExtractorHandler["extractor"])at org.codehaus.jackson.map.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:345)at org.codehaus.jackson.map.deser.SettableBeanProperty._throwAsIOE(SettableBeanProper

Nifi DatadogReportingTask attribute "API Key" is neither sensitive nor it supports env variables. This field is a sensitive key and I dont want it to show the attribute value on the UI.   Please let me know how can this be achieved? Also what is the reason to not keep this field sensitive or "expression language compatible".

Attempting to set LDAP/AD users for Ranger (v1.2.0) following the docs (https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/configuring-ranger-authe-with-unix-ldap-ad/content/configuring_ranger_authentication_with_unix_ldap_or_ad.html) and this older video (https://www.youtube.com/watch?v=2aZ9GBhCOhA), but when looking at the Ranger Users tab in the Ranger UI, seeing only the original Unix users. In the UI I see...  Looking at the usersync logs, near the tail I see: ....06 Dec 2019 14:21:51 INFO LdapUserGroupBuilder [UnixUserSyncThread] - LdapUserGroupBuilder initialization started06 Dec 2019 14:21:51 INFO LdapUserGroupBuilder [UnixUserSyncThread] - LdapUserGroupBuilder initialization completed with -- ldapUrl: ldap://172.18.4.42:389, ldapBindDn: UID=hwldap,OU=Users,OU=HortonworksUsers,DC=ucera,DC=local, ldapBindPassword: ***** , ldapAuthenticationMechanism: simple, searchBase: dc=hadoop,dc=apache,dc=org, userSearchBase: [dc=ucera,dc=local], userSearchScope: 2, userObj

hi there,   I have created a S2S reporting task to the same instance of nifi:    After enabing it, the port "Bulletin" can get error message however the bulletin board raise warining the message continueously  :   2019-12-06 20:35:13,274 WARN [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@7a616d8a Unable to refresh Remote Group's peers due to null   Why and any idea to fix it?

Hello,  I install impala/kudu on HDP 2.6.2.2:  impalad version 2.7.0-cdh5-IMPALA_KUDU-cdh5 RELEASE (build 48f1ad385382cd90dbaed53b174965251d91d088)  With impala-shell i created database "imp_kudu" with single table "test_table" When I try to open JDBC connection for "ImpalaJDBC4.jar" driver using url: jdbc:impala://myserver.com:21050/imp_kudu;AuthMech=0 I have the following error Cloudera][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: Operation timed out (Connection timed out). What kind of problem does this indicate and how can I resolve it?

Hi,I am trying to remove all new lines from base64encoded string using replaceText processor.Properties for replaceText processor is as follows:Search Value : \\nReplacement Value:$1Character Set: UTF-8Maximum Buffer Size: 10MBReplacement Strategy: Regex ReplaceEvaluation Mode: Line-by-Line But this not working.  

I had installed HDP 3.0 which has Spark 2.3.1. I was running sql commands in Zeppelin notebook successfully but suddenly Windows update restarted the machine. When I opened the Zeppelin notebook again and ran the first command %spark2val hiveContext = new org.apache.spark.sql.SparkSession.Builder().getOrCreate() I got following error logs: -    java.lang.NullPointerException at org.apache.thrift.transport.TSocket.open(TSocket.java:209) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apac

Hi All, when using the MergeContent processor I'm getting an avro.SchemaParseException due to $ characters in a nested attribute name ($.Snapshot.Events.['$values'][0].SportsUrn) in a json file. I have tried to fix it with the UpdateAttribute processor, but it won't work. What would be the best way to handle this exception? Your support is highly appreciated! Thanks

I need to know how I can increase the RAM size (Java Heap Size) of my NameNode.

Hi    I am trying to do an incremental load from API using the API end_time tag.    Trying to use the below syntax in RouteAttribute processor to fetch only records updated till yesterday night - 23:59:59  Could you tell me what is the exact syntax for checking end_time here?  ${end_time:lt(${now():toNumber():minus(86400000) })}

HI All,can somebody let me know which version of HDP supports Spark 2.4? and is that good idea to go to Spark 2.4 instead of 2.3?My Current versions: HDP 2.6.3.0Apache Ambari 2.6.0.0 HDFS 2.7.3 YARN 2.7.3MapReduce2 2.7.3Spark2 2.2.0Any suggestions are welcome.Regards,Manjunath P N

https://docs.cloudera.com/documentation/enterprise/6/6.1/topics/admin_ha_hiveserver2.html#concept_u4b_c5d_wv My cluster version is Cloudera CDH 6.1 Express Edition. I try set Hive HA.I added new instance.And I set the address on HiveServer2 Load Balancer property.But Load Balancer server didn't open proxy port.Doesn't Managed cluster start proxy server?Do I have to configure proxy?Managed cluster means Enterprise Edition?If that is, Can I use L4 instead of haproxy?