Solved

I have a table with 38M rows which is being updated daily. The table is in orc format and it is managed table.I want to create partitions on that table but I couldn't find a way to alter existing non-partitioned table to create partitions in that table. I tried searching all over the google but the only option I was able to find was Create a new partitioned table and insert data into it from the old table. I was wondering if there was any way to create partitions on an existing table. My HIVE version is 3.1.Thanks & Regards,Kartik

Hello,My client wants to delegate to certain users (of his company and external) the opening of support cases, how can it be configured?Thanks in advance, Iñigo Perez

I ran the the command ambari-server setup, after which I select accordingly. I am prompt to select the java and selected custom JDK and enter the JAVA_HOME but I was shown the error below.

I've a Hbase mapped Hive table which retrieves rows when I run select * from my_table;But when I execute select count(*) from my_table; the output is 0.What must be the issue in this case ?

I get Error 404. Please see the attached pic

Basically, I am trying to analyze SQL server log files using Hive.The layout is the log file ischar(23) - for timestampchar(1) - spacechar(12) - sourcethe rest of the row, and the length varies. The row delimiter is CR+LF. Below are some entries in the log.2019-05-28 07:29:55.03 Server      UTC adjustment: -7:00 2019-05-28 07:29:55.03 Server      (c) Microsoft Corporation. 2019-05-28 07:29:55.03 Server      All rights reserved. 2019-05-28 07:29:55.03 Server      Server process ID is 3368.There are several posts here regarding fixed-width column layout. But in my case, the last column is identified not by the width but by the row delimiter.

Due to changes in our networking and firewall requirements we had to switch from the hortonworks public repositories to the local repositories (artifactory). After making this change we are seeing the following exception when we try to "Start All" (from Ambari) Which triggers the installation of the client packages and they are failing with the following exception.Essentially similar error is thrown when we try to install any new package after this change.OS: Ubuntu 16.04No package found for hadoop-${stack_version}-client(hadoop-(\d|-)+-client$)2019-05-29 15:47:30,684 - The repository with version 2.6.5.0-292 for this command has been marked as resolved. It will be used to report the version of the component which was installedCommand failed after 1 tries

Hello. I'm a new guy using Coludera.I have very simple question about a log. I need to check every single minute's resource's log such as cpu usage, mem usage, disk I/O etc,but it's hard to find that kinds of data in log files.To be more specific, I want to utilize resource logs as data frame shapefor recognizing whether the server is going well or not. so, can I obtain resource data ? Aplogies for this stupid question. Appreciate your response.  

I have a json flowfile with below content{"name":"abc"}I want the output should be {"name":"abc","attributes":{"id":"12233","map":"Y"}}How to achieve the same?

I've successfully cloned and built the NIFI system. It built cleanly. Though initially I experienced a few random unit test failures. That was solved (it was actually a network configuration issue on my end that caused this).The instructions I followed are here:https://nifi.apache.org/quickstart.html The built system is in nifi-assembly/target. I want to run the primary NIFI backend application in the debugger (not the web app, but rather the backend). I have the nifi set of maven projects loaded into eclipse.Can someone provide me information that would assist in getting the app up and running in the debugger?For example:What project contains the main entry point?Aside from running the main application, how do I launch the web app? I'm assuming it is launched separately from the main application.

Hi, I was about to give my exam on CCA175, but the Cloudera team canceled my exam schedule hours before my exam and they sent me an email stating some technical issues they are facing. It's been 3 days I am waiting to give the exam and looks like the technical issues are not resolved yet. The technical team is not giving me any ETA and I don't see my exam voucher in my examlocal account. I never expected this from a reputed company like Cloudera. Who can help me getting back my exam voucher and letting me schedule my exam?

Hello All, I am looking for some best practices or recommendations to set a best possible value for rlimit_fds (Maximum Process File Descriptors) property. Currently, it is set to default i.e. 32768 and we are getting File Descriptor Threshold alerts. We would first like to look for a best possible value for rlimit_fds. Is there a formulae or a practice or few checks that can be performed to determine a best value? Thankssnm1523

Hello, We have been facing issues with our CM Service Monitor, for the past couple of days it fails and the starts to work again, we also get the below alerts repeatedly : Concerning : The last metrics aggregation run duration is 25.4 second(s). Warning threshold: 10 second(s). Concerning : Average time spent paused was 27.9 second(s) (46.47%) per minute over the previous 5 minute(s). Warning threshold: 30.00%. There are no configuration changes done on this node, attached is the SM log file.5:48:14.372 AM INFO JvmPauseMonitor Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 2543ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=2718ms5:48:14.373 AM ERROR SafeAvroResponderServlet Error procesing Avro requestorg.mortbay.jetty.EofException at org.mortbay.jetty.HttpGenerator.flush(HttpGenerator.java:791) at org.mortbay.jetty.AbstractGenerator

HiWhen i do a SELECT null; at cli hive i get a good answer -> NULLwhen i do this in beeline , i get Error: Error while compiling statement: FAILED: SemanticException [Error 10004]: Line 1:7 Invalid table alias or column reference 'null': (possible column names are: ) (state=42000,code=10004)this is working on one of my clusters but not in one of them (hdp 2.6.1 Connected to: Apache Hive (version 2.1.0.2.6.1.0-129)Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129)Transaction isolation: TRANSACTION_REPEATABLE_READBeeline version 1.2.1000.2.6.1.0-129 by Apache Hive)i need this to work, since when i query INSERT OVERWRITE TABLE schema.customerSELECT CUST_ACCOUNTSotherschema.CUST_ACCOUNT_ID,NULL AS ACCOUNTFROM xxxx ;i really appreciate any help

Hi, i am new with hadoop,I have the following errors when i try to run the command hdfs dfs -ls on a specific directory:[t_hdhusr@node01 ~]$ hdfs dfs -ls /user/T_HDHUSRlog4j:ERROR setFile(null,true) call failed.java.io.FileNotFoundException: /var/log/hadoop/t_hdhusr/hadoop-mapreduce.jobsummary.gz (No such file or directory)at java.io.FileOutputStream.open0(Native Method)at java.io.FileOutputStream.open(FileOutputStream.java:270)at java.io.FileOutputStream.<init>(FileOutputStream.java:213)at java.io.FileOutputStream.<init>(FileOutputStream.java:133)at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)at org.apache.log4j.config.PropertySetter.setProperties(PropertySett

hi,I've two clusters running on hdp 3.1, both consists of single node and kerbeorized as well. I am trying to send a single file using distcp but failed.got error(pictures attached)even unable to list hdfs directories of other cluster on either of the cluster.Thanks.

Is there any way to parameterize Nifi DBConnection Services..? I want to use the same nifi flow to extract the data from different DataBases with different set of parameters (for example tables and data types) so i am wondering is there any way to pass polling services just like any other parameter value as a variable..?

Hi, I have a very simple question. Can hive dB have multiple ranger policies?In any circumstance can you restrict having multiple ranger policies?

Content of stock2.json file{"myid":"0001","mytype":"donut"}stock2.json file is in the path of '/warehouse/tablespace/external/hive/haijintest.db/stock2';Below is the table creation script.CREATE EXTERNAL TABLE stock4_json (myjson struct<myid:string, mytype:string>)ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe' LOCATION '/warehouse/tablespace/external/hive/haijintest.db/stock2';Below is the query that returned null value.0: jdbc:hive2://> select myjson.myid from stock4_json;19/05/27 20:20:18 [103d2343-eb59-4072-8a07-5469b2112093 main]: WARN optimizer.SimpleFetchOptimizer: Table haijintest@stock4_json is external table, falling back to filesystem scan.OK+-------+| myid  |+-------+| NULL  |+-------+1 row selected (0.232 seconds)0: jdbc:hive2://> select * from stock4_json;OK+---------------------+| stock4_json.myjson  |+---------------------+| NULL                |+---------------------+1 row selected (0.242 sec

how to configure hivserver2 in superset.

Hi all,We have HDP version 3.1 , we are trying to integrate cluster with corporate Active directory for kerberos , we have setup CA certificate as instruction given on ambari/hdp communities ,we have below required details with us ..AD Service account / passwordRealm nameLdap server with portDC=in , dc =xyz ,dc=com.we we have gid down may article on it but same issue we are facingbut facing issue with below error messageError message: Failed to connect to KDC - Failed to communicate with the Active Directory at ldaps://adldap.x.x.com: simple bind failed: adldap.x.xcom:636 Make sure the server's SSL certificate or CA certificates have been imported into Ambari's truststore.can someone has all relevant steps which need to be follow to integrate hadoop cluster with AD for kerberos and for domain authentication

My exam for CCA 175 was scheduled on 03/03/2019 19:30:00 IST. The exam did not load on examslocal.com as it was not ready by cloudera. The chat support in examslocal suggested me to drop an email to cloudera team. As soon as i got this instruction, I dropped an email to certification@cloudera.com regarding the issue, but there is no proper response till now. All I get is, an automatic reply, which says, cloudera team is having a tech summit and would be busy between 25th feb- 3rd march, but it's been 7th march now. I am supposed to share a certificate of completion of CCA 175 before 12th march, otherwise, i'd be in trouble.Any suggestions are welcome. Thanks.

I want to read a MS SQL database. I'm using a QueryDatabaseTable to make de query and then SplitAvro, because the output of QueryDatabaseTable is an avro file. But my problem is that the query does not arrive to SplitAvro processor. Can someone help me with the configuration of this processors?qdt.PNGsplitavro.PNG

Hi, I need some help with JOLT-syntax.JSON-Input{  "nummer": "1073",  "table": [    {      "datum": "20180529",      "zn": 1    },    {      "datum": "20180530",      "zn": 2    },    {      "datum": "20190522",      "zn": 3    }  ]}JOLT-Specification[{    "operation": "shift",    "spec": {      "nummer": "Nummer",      "table": {        "*": {          "zn": "Positionen.[].ZeileNr",          "datum": "Positionen.[].Datum"        }      }    }}]JSON-Output{    "Nummer": "1073",    "Positionen": [{        "ZeileNr": 1    }, {        "Datum": "20180529"    }, {      &nbs

We have ten millions image and video files, are looking for efficient ways to store them in Hadoop (HDFS ...), and analyze them with tools available in the Hadoop ecosystem. I understand HDFS prefer big files. These image files are small, they are under ten megabytes. Please advise. Thanks very much!