Solved

I tried creating a simple table , i am not sure what the error is.Can some one please help me.CREATE TABLE stocks_eod (> stockticker STRING,> tradedate INT,> openprice FLOAT,> highprice FLOAT,> lowprice FLOAT,> volume BIGINT> ) ROW FORMAT DELIMTED FIELDS TERMINATED BY ','; NoViableAltException(232@[])at org.apache.hadoop.hive.ql.parse.HiveParser.tableRowFormat(HiveParser.java:36025)at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6865)at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:4122)at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1786)at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1152)at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:211)at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:171)at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:447)at org.apache.hadoop.hive.ql.Driver.compile(Driver.java

I am new to this. i've tried installing ambari and on the first try, i was able to start the ambari-server. but when i tried again, an error occurred.Using python /usr/bin/pythonStarting ambari-serverAmbari Server running with administrator privileges.Organizing resource files at /var/lib/ambari-server/resources...Ambari database consistency check started...Server PID at: /var/run/ambari-server/ambari-server.pidServer out at: /var/log/ambari-server/ambari-server.outServer log at: /var/log/ambari-server/ambari-server.logWaiting for server start.........Unable to determine server PID. Retrying.........Unable to determine server PID. Retrying.........Unable to determine server PID. Retrying...ERROR: Exiting with exit code -1.REASON: Ambari Server java process died with exitcode 255.the /var/log/ambari-server/ambari-server.out gave me:Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0Jul 10, 2019 10:02:23 PM com.google.inject.assistedinj

Hi, I'm using the Impala function "show table stats": show table stats table1;  I get following resultsbusiness_datetec_execution_date#Rows#FilesSizeBytes CachedCache ReplicationFormat13/05/202013/05/2020 20:08000BNOT CACHEDNOT CACHEDPARQUET14/07/202015/07/2020 16:396116.08KBNOT CACHEDNOT CACHEDPARQUET15/07/202016/07/2020 16:077702114.68KBNOT CACHEDNOT CACHEDPARQUET16/07/202020/07/2020 16:50272141.88KBNOT CACHEDNOT CACHEDPARQUET17/07/202021/07/2020 12:38290145.33KBNOT CACHEDNOT CACHEDPARQUET20/07/202020/07/2020 12:16379151.10KBNOT CACHEDNOT CACHEDPARQUET21/07/202022/07/2020 12:45156131.50KBNOT CACHEDNOT CACHEDPARQUET24/07/202024/07/2020 15:15278140.26KBNOT CACHEDNOT CACHEDPARQUETTotal 24318340.84KB0B  However if I sum up the rows in column "#Rows" I get 2151...!Am I'm missing something? How is that possible?ThanksThanks

Can anyone tell me what I'm doing wrong. I'm getting the following error from the table creation below -  FAILED: ParseException line 8:37 mismatched input 'TREX' expecting ) near ''\\', bp.step_parameter_name) \nWHERE prm.data_source = '' in function specification")  CREATE TABLE new_table AS SELECT DISTINCTprm.source AS source,bp.process_stage AS process_stageFROM table1 AS prmLEFT JOIN table2 AS bpON (prm.parameter = CONCAT(bp.step_path, '\\', bp.step_parameter_name))WHERE (prm.data_source = 'TDATA' OR prm.data_source = 'Docs')

Hello we have secured Nifi cluster with 3 nodes. When we click to list or empty queue on connection, there is error message Node is unable to fulfill this request due to: Unable to view the data for Processor with ID 904e5254-8129-32ce-967e-03876fc61926. Contact the system administrator. Returning Forbidden response We grant user policy to view and modify data, but no success. Admin user got the same error.message Regards Kamil 

I'm facing an issue during the upgrade of HDP 3.1.0.0-78 to 3.1.4.0-315 on Ubuntu 18The upgrade process is not able to restart the datanodes.I get the error java.lang.RuntimeException: Cannot start datanode because the configured max locked memory size (dfs.datanode.max.locked.memory) of 2147483648 bytes is more than the datanode's available RLIMIT_MEMLOCK ulimit of 16777216 bytes.I don't understand why this error happens. The datanodes were well started before the starting of the upgrade process and the system setting RLIMIT_MEMLOCK hasn't been changed.Thanks in advance for your help  

HelloI would like to knowHow to reset Cloudera Manager Admin Password from the Backend DatabaseDatabase version: Mysql 5.7.29CM version: 6.3.xAuthentication method  for the "Authentication Backend Order": External then Cloudera Manager Thanks in advanced!

Hi all,can I install an HDFS datanode role in machine with installed HDFS NameNode or into machine with SecondaryNameNode?I use CDH 6.3.2 Thanks,Francesco

Hello everyone,I'm trying to install the CDP trial version and the wizard is taking hours to finish. I'm not an expert, but the log seems okay. I did the command below to keep tracking the log.  tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log and I got these as a result and never ends... 2020-04-20 02:29:15,382 INFO avro-servlet-hb-processor-0:com.cloudera.server.common.AgentAvroServlet: (3 skipped) AgentAvroServlet: heartbeat processing stats: average=3ms, min=1ms, max=196ms.2020-04-20 02:30:15,422 INFO avro-servlet-hb-processor-0:com.cloudera.server.common.AgentAvroServlet: (3 skipped) AgentAvroServlet: heartbeat processing stats: average=3ms, min=1ms, max=196ms.2020-04-20 02:31:15,453 INFO avro-servlet-hb-processor-0:com.cloudera.server.common.AgentAvroServlet: (3 skipped) AgentAvroServlet: heartbeat processing stats: average=3ms, min=1ms, max=196ms.2020-04-20 02:32:15,502 INFO avro-servlet-hb-processor-0:com.cloudera.server.common.AgentAvroServ

I am running an Hive query (my execution engine is Tez) on an external table with gz files. The error message i get is "java.io.EOFException: Unexpected end of input stream". I have done a check on HDFS for corrupt files and it returns a healthy status. Anyone come across a similar issue?Caused by: java.io.EOFException: Unexpected end of input streamat org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:145)at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85)at java.io.InputStream.read(InputStream.java:101)at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)at org.apache.hadoop.mapred.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:208)at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:246)at org.apache.hadoop.mapred.LineRecordReader.next

I am running Hortonworks Sandbox HDP 2.6.5 on VirtualBox. I have increased the size of my virtual hard disk (.vdi) to 500 GB. However, when I login to Ambari and view the size of my disk, it shows 106 GB only. What should I do to increase the HDFS capacity from 106 GB to 500 GB?

Team , We have upgraded NIFI from 1.5 to 1.11 and trying to configure ranger authorization at group level instead of user level . But its not working . Below are the properties i have given in authorizers.xml <userGroupProvider><identifier>ldap-user-group-provider</identifier><class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class><property name="Authentication Strategy">LDAPS</property><property name="Manager DN">CN=xxxx,OU=Users,OU=Standard Users,OU=Users and Computers,OU=Atlanta Corporate,OU=CCI,DC=CORP,DC=COX,DC=com</property><property name="Manager Password">xxxxx</property><property name="TLS - Keystore">/etc/nifi/conf/signed-keystore.jks</property><property name="TLS - Keystore Password">xxx</property><property name="TLS - Keystore Type">JKS</property><property name="TLS - Truststore">/etc/nifi/conf/coxca-truststore.jks</property><property name=

yesterday i deleted some topics in my kafka with below command ./kafka-topics.sh — zookeeper localhost:2181 — delete — topic <topic_name> with the configuration of delete.topic.enable=true after 20 hrs i checked those topics still not deleted. how can we force delete those topics. and why it will take this much time to delete the topic?

We are doing spark-submit from airflow (added it as a custom parcel into CDP 7.1)Airflow is built with python 3 however default python version on CDP is python2. As a result during spark-submit getting this issue: WARN net.ScriptBasedMapping: Exception running /etc/hadoop/conf.cloudera.yarn/topology.py 10.228.86.42ExitCodeException exitCode=1: File "/opt/cloudera/parcels/Airflow-1.10.10-python3.7.7_1.2.3/lib/python3.7/site.py", line 177 file=sys.stderr) ^SyntaxError: invalid syntax Added PYSPARK_DRIVER_PYTHON and PYSPARK_PYTHON to spark-defaults as well as spark-env.sh pointing to python3. Also added spark.yarn.appMasterEnv.PYTHONHASHSEED = 0 however the problem remains. As soon as python version is being changed to python3 on the workers (basically the only available python becomes python 3) spark-submit starts working. I was wondering if there is something I am missing. Thanks

Hi, I am running cdh 5.14.0 cluster version both on Prod and DR cluster. Both cluster are Kerberos and TLS enabled with the same kdc server. Could anyone guide me on how to do BDR between Kerberos enabled clusters. As I was not able to find any proper guide on the Cloudera website.   Please help me or provide any docs related to this. Appreciate your help.   Thanks  pdev

Hello, I am trying to install Apache Metron in a cluster created with Ambari and HDP 2.6.5.0. When I select Metron as a service to install, I get stuck when trying to get to the next step, the next button shows as always disabled. This is not happening with any other service.   I followed this guides: Github community.cloudera.com My environment specs are: Java JDK 1.8 in all hosts Metron RPM packages in /localrepo in all hosts root user in all hosts Ambari server 2.7.5.0 6 Centos 7 host : metron1 (9.6 GB, 4 cores, 100G SSD) NameNode ZooKeeper Server Kafka Broker Zeppelin Notebook metron2 (9.6 GB, 4 cores, 100G SSD) SNameNode App Timeline Server ResourceManager History Server ZooKeeper Server Nimbus Kafka Broker metron3 (15.5 GB, 4 cores, 500G SSD) HBase Master Kafka Broker Elasticsearch Master metron4 (11.6 GB, 3 cores, 50G SSD) HBase Master Oozie Server Nimbus DRPC Server Storm UI Server Kafka Broker metron5 (11.6 GB, 3 cores, 50G SSD) Kafka Broker metron6 (15

Is the a configuration file for Cloudera Management Service?I'd like to have the service roles' configuration stored in repository. This way, service configuration can be shared between environments without the need to change the configuration via CM ui or rest API on each environment.

Hi there, I'm facing this problem for 2 days now and maybe you guys can find it.I'm not the only one facing this issue but none of the given solutions fixed my problem. Hive metastore is running and hiveserver2 is not saying connection refused I will put my exact error messages here too.   Connection failed on host localhost:10000 (Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/alerts/alert_hive_thrift_port.py", line 204, in execute ldap_password=ldap_password) File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/hive_check.py", line 84, in check_thrift_port_sasl timeout_kill_strategy=TerminateStrategy.KILL_PROCESS_TREE, File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/ambari-age

How is data quality managed?Which tool should be used for Hadoop security - Apache Ranger or Apache Sentry?

Need to download hdp-3.1.5 and ambari-2.7.5 but it says I need to register.Does one needs to be a paid customer of Cloudera to download the latest versions?

Hi,being pretty new to Nifi, I am strugeling to extract interface monitoring data Assuming a very simple ingest flow like:HttpListener -> KafkaPublisher How can I get throughput information, i.e. number of records put (e.g.) every minute to a topic? There is NifiSummary -> Processors -> Status History, which might be useful. Can the statistics be accessed programatically and how? Best regardsJaro

Newbie question, apologies. We have a need to backup a Kafka cluster, so that we can restore to a given point in time (as far as possible according to backup granularity) in case of problems, e.g. bad data. Replication would not help here, since bad data could be replicated.Does anyone out there have such a use case, and did you solve it (with Cloudera or open-source tools)?Thanks in advance.

I will deploy CDH 6.3.3 and it has Hadoop 3.0.0 , Is there any way to use later version of Hadoop like 3.1.2 as more stable instead and still being supported with Cloudera CDH 6.3.3 ?

I am using PostgreSQL for the Cloudera Manager Server and other services that use databases. I'm using a RHEL 7.4 Azure VM and following the official documentation -https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/prepare_cm_database.htmlI have completed step 4(Install Databases). In Step 5 : Set up the Cloudera Manager Database, on executing the scm_prepare_database.sh script I'm getting the following output-  [root@machine1 ~]# sudo /opt/cloudera/cm/schema/scm_prepare_database.sh postgresql scm scmEnter SCM password:JAVA_HOME=/usr/java/jdk1.8.0_181-clouderaVerifying that we can write to /etc/cloudera-scm-serverCreating SCM configuration file in /etc/cloudera-scm-serverExecuting: /usr/java/jdk1.8.0_181-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera

hi all we have HDP 2.6.4 cluster with 365 data-nodes ( workers machines )  we intend to add to this cluster 67 new data-nodes , but for now these new data-nodes should not be active , in spite the Linux OS is up and machine is up so is it possible to disable HDFS and YARN on these new machines?  what is the preferred way?  for example - decommission ? , or just maintenance mode? or maybe both them?  the target is to avoid from spark jobs to run on the new data-nodes machines and disable the HDFS Temporarily