Solved

i want change the following property in cluster-env.xml: <property> <name>recovery_enabled</name> <value>true</value> <description>Auto start enabled or not for this cluster.</description> <on-ambari-upgrade add="true"/> </property> <property> <name>recovery_type</name> <value>FULL</value> <description>Auto start type.</description> <on-ambari-upgrade add="true"/> </property> <property> <name>recovery_lifetime_max_count</name> <value>1024</value> <description>Auto start lifetime maximum count of recovery attempt allowed per host component. This is reset when agent is restarted.</de$ <on-ambari-upgrade add="true"/> </property> <property> <name>recovery_max_count</name> <value>1</value> <description>Auto start maximum count of recovery attempt al

 无法安装cloudera Manager代理程序包 我试图从本地存储库安装cm6.1.0和cdh6.1.0。 系统是Centos7.2  下载包：cloudera-manager-agent-6.1.0-769885.el7.x86_64.rpm | 25 MB 00:00:01正在运行事务检查运行事务测试事务检查错误：来自安装了Cloudera管理器 - 代理 - 6.1.0-769885.el7.x86_64的文件/usr/lib/systemd/system/supervisord.service与来自程序包监事，3.1.3-3.el7的文件冲突错误摘要-------------

Hi, i am sorry if  i am postiing this where it should not be posted.New to impala/hive , new to this community site but i need to populate an empty table of one field (date) with dates (from 1990-01-01 till 2099-12-31) and  the  "while" loop does not seem to be working here. Any suggestions,or links  you guys can provide please?Thanks,Tom

I am getting below error when loading Json data from Hive table of type TEXTFILE to ORC type Hive table. Please share your thoughts to overcome from this issue. Thanks.Input sample file TestFile1.json: {"jsondata":[{ "id":1, "m":"Edward the Elder", "cty":"Uited Kigdom", "hse":"House of Wessex", "yrs":"899925" },{ "id":2, "m":"Edward the Elder", "cty":"Uited Kigdom", "hse":"House of Wessex", "yrs":"899925" }]}Cluster version: HDP 2.6.5.0-292TEXFILE Hive Table schema:CREATE EXTERNAL TABLE IF NOT EXISTS TestJson1 (jsondata array<struct<id:int,nm:varchar(30),cty:varchar(30),hse:varchar(30),yrs:varchar(20)>>) ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe' Location '/data/3rdPartyData/Hive/TestJson1';Load json into above table: load data local inpath '/data/Hive/TestFile1.json' overwrite into table TestJson1;ORC Hive table schema:CREATE EXTERNAL TABLE IF NOT EXISTS TestJson1ORC(jsondata array<struct<id:int,nm:varchar(30),cty:varchar(30),hse:varchar(30),yrs:varc

I am getting an error when querying Hive table over JSON Data. I think the root cause would be JSON SerDe. Should I have to change Hive native JSON SerDe to any other JSON SerDe? Please adviseError: Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start token not found where expected

what problem?

Hello The documentation for upgrading/migrating the JDK shows installing Java OpenJDK 1.8 using the package manager and therefore from the OS repositories, which for Redhat/CentOS 7 is: su -c yum install java-1.8.0-openjdk-devel Does this mean that the whichever version (1.8.0.191 at time of writing) and all future versions of OpenJDK 1.8 are supported for Cloudera Manager and CDH 6.x? The above command is taken from the documentation here: https://www.cloudera.com/documentation/enterprise/upgrade/topics/ug_jdk8.html Thanks,JG

hello everyone,I have a query regarding EvaulateJSONPath processor configuration.whats the significance of Execution in Scheduling of EvaulateJSONPath.I have observed when I select Primary node instead of All nodes, Queue builds up as shown in image and stays the same, dosen't decrease , but if I select All nodes Queue gradually becomes 0.Please suggest.screen-shot-2019-01-17-at-60906-pm.png

Hello Team,   I am considering HA configuration of Kerberized CDH cluster.   Is it possible to avoid SPoF of Load Balancer when constructing Load Balancer such as HAProxy and configuring HA of HiveServer2?For example, is it possible to deploy two Load Balancer with Active-Active and specify two host and port for setting(HiveServer2 Load Balancer property)?Or, should deploy Active and Stanby of Load Balancer be constructed, and when Active fails, should I change the setting(HiveServer2 Load Balancer property) to refer to Stanby? Impalad thinks the same way.   I refer to the following procedure.  * https://www.cloudera.com/documentation/enterprise/6/6.0/topics/admin_ha_hiveserver2.html#concept_u4b_c5d_wv * https://www.cloudera.com/documentation/enterprise/6/6.0/topics/impala_proxy.html#proxy   thank you for reading.  

Hi,   I have upgraded my CM/CDH to 6.1 and every time I login into Hue I receive this error:   Solr server could not be contacted properly: HTTPConnectionPool(host='localhost', port=8983): Max retries exceeded with url: /solr/admin/info/system?user.name=hue&doAs=mpanahi&wt=json (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))   The problem is I have never had Solr installed nor configured inside Hue. The Solr is None in Hue inside CM and there is no such thing that refers to [search] or [dashboard] inside my hue_safety_valve.ini so I don't know why it tries to connect to something without me setting it up.   Does anyone have any idea why in the new Hue it tries to connect to Solr without being it installed or selected?   Many thanks,  

Hi all, I have NiFi behind a nginx server proxy, and when I try upload template I recieve HTTP 403. If I access directly I can upload template. Below are my config file nginx.upstream nifi-app { ip_hash; server node-01:8443; server node-02:8443;}server { listen 443 ssl; server_name nifi-server; ssl_certificate /etc/nginx/conf.d/cert.pem; ssl_certificate_key /etc/nginx/conf.d/key.pem; location /nifi { proxy_pass https://nifi-app; proxy_set_header X-ProxyScheme https; proxy_set_header X-ProxyHost myhost.com; proxy_set_header X-ProxyPort 443; proxy_set_header X-ProxyContextPath /; }}Any help will be appreciated

Hi,I ´m not able to create 2 nodes cluster. I´m getting the next error at start:Infrastructure creation failed. Reason: Following tag values are not well formatted: [aws.xxxxxx_xx@xxxxxxxxx.com]('x' obviously are not the real content, it is a valid mail with simple characters.)

Hi,   Could you please help me out by giving answer for the below questions   If  HDFS services goes down in kafka cluster, what will happen to kafka service and data   Regards, R.

Hi, Could you please help me out by giving answer for the below questions? 1. What is command to set queue name for hive job 2. Which of the files in the input directory will be skipped by hive job?A._SUCCESSB.  IGNOREC. Input.metadataD.   .skip

How do I remove a currency symbol from a string and convert it into an int in Hive? For example, how do I change '124$' into 124?

Issue: Loading data into Hive ORC table is infinite, I should manually kill the load process.I am trying to load data into ORC Hive table from another Hive TEXTFILE table. Since the source files are TXT/Json, loading data first into TEXT table and then trying to load into ORC table.Cluster: HDP 2.6.5-292Hive version: 1.2.1000.2.6.5.0-292Here is the Hive TEXTFILE table schema:Create external table if not exists TEXTTable(ID bigint, DOCUMENT_ID bigint, NUM varchar(20), SUBMITTER_ID bigint, FILING string, CODE varchar(10), RECEIPTNUM varchar(20))row format delimited fields terminated by '|' Location '/data/3rdPartyData/Hive/ TEXTTable 'TBLPROPERTIES ('skip.header.line.count'='1');Load Data into TEXTFILE table:load data local inpath '/data/TextFile.txt' overwrite into table TEXTTable;Here is the Hive ORC table schema:Create external table if not exists ORCTable(ID bigint, DOCUMENT_ID bigint, NUM varchar(20), SUBMITTER_ID bigint, FILING TIMESTAMP, CODE varchar(10), RECEIPTNUM varchar(20))

I have nifi cluster summarry json data like this;[{ "nodeId": "29bed24c-6d73-4652-930b-0065cad4ef66", "address": "b1518aec7e38", "apiPort": 8080, "status": "CONNECTED", "heartbeat": "01/16/2019 16:24:32 UTC", "roles": ["Primary Node", "Cluster Coordinator"], "activeThreadCount": 0, "queued": "0 / 0 bytes", "events": [{ "timestamp": "01/16/2019 15:21:50 UTC", "category": "INFO", "message": "Received first heartbeat from connecting node. Node connected." }, { "timestamp": "01/16/2019 15:21:48 UTC", "category": "INFO", "message": "Connection requested from existing node. Setting status to connecting." }], "nodeStartTime": "01/16/2019 15:20:44 UTC" }, { "nodeId": "22aofpp4-87rf-asf4-930b-0065cad4ef66", "address": "b67uf98tkl5", "apiPort": 8080, "status": "CONNECTED", "heartbeat": "01/16/2019 16:24:32 UTC", "roles": [], "activeThreadCount": 0, "queued": "0 / 0 bytes", "events": [{ "timestamp": "01/16/2019 15:21:50 UTC", "category": "INFO", "message": "Received first heartbeat f

If any node is down, can nifi create new node automatically? The unused node will be ready to use. When a node is down(fail, disconnect or etc.), I just want to use this new node instead of failed node. Can nifi do this automatically? Is this possible?Thank you.

Hello, i am new to atlas and i am wondering if there is a solution making atlas expose, in addition to a dataset metadata, a sample data of this dataset ( 10 first rows for example).for a purpose of data governance, the problem i am solving is that the metadata could not be so comprehensive functionally, and reading a sample data will make more sens to explain a dataset content.if , actually, there is not a solution please help me find were should i invest my effort ( inner development within atlas, or treat the problem using a third party accessing hive,scoop or hbase directly to get the sample data)thanks in advance.

Does anyone know what are the changes we need to use the https://issues.cloudera.org/browse/HUE-7407 changed code effectively. We have in our user_filter:|(memberof=CN=admin_group,OU=OUname,DC=DC,DC=DC,DC=com)(memberof=CN=nonadmin_group,OU=OUname,DC=DC,DC=DC,DC=com) Ideally we want to someway set all the poeple in the admin group as super_user's for hue.It would be of great help because currently when we rehydrate our AWS EMR which has HUE enabled all the user's gets removed and when someone logins to the new AWS EMR they will become super_user.I have circumvented this issue temporarly  by login a user in the bootstrap action.   

I'm currently upgrading from HDP 2.6.3 to HDP 3.1.0, following the docs.On my test cluster, upgrading ambari-infra-solr went fine, but on our main cluster running the backup stage failed. I restarted the ambari-infra-solr server, but it doesn't seem to have come back properly - loading the web UI shows a few GUI elements and a big "loading" spinning. Logging seems broken too - while the server is running, nothing appears in solr.log and there are log4j configuration warnings in stdout.I added a second ambari-infra-solr server, and it sees the collections but can't access the shards.Long story short, I don't particularly mind losing the content of the collections - is there an easy way to drop them and start from scratch?Thanks

Hello,When installing new HDP version in Ambari, the following error is displayed: "Cluster <cluster_name> upgrade is in progress. Cannot install packages."I had tried to use instruction like this https://community.hortonworks.com/content/supportkb/177629/errorcluster-xyz-upgrade-is-in-progress-cannot-ins.htmlBut not successfully. How can i work around this problem?

We would like to take advantage of Oracle JDK 11.1.0. So, Will it be possible to use Oracle JDK 11.1.0 with HDP 3.1.0 ?.Anyone have any success stories on this ?.Please share your experiences?

Please tell me the requirements of Impala.In the HW requirements of Catalog Server and Impalad, the CPUs are listed as well, are they correct?Does Catalog Server also require CPU, Min: 4 - Recommended: 16?I referred to the following page. * Hardware Requirements  * impala   * https://www.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_hardware_requirements.html#concept_usf_qln_3bb * Impala  * Impala Requirements   * https://www.cloudera.com/documentation/enterprise/6/6.0/topics/impala_prereqs1.html#prereqs_hardware

we have ambari cluster with 176 datanodes machines ( workers )each datanode have the following component:capture.pngDataNode / HDFSMetrics Monitor / Ambari MetricsNodeManager / YARNhow to restart all these component on all datanode machines by API* because it will take time to restart all datanode component manual as ( access each datanode machine from ambari and perform --> host actions --> restart all )