Solved

Query:select 2gusage,count(2gusage) from demo group by 2gusage;Output:MID 765153 HIGH 18095461 LOW 119069472 NULL 25997075I tried the below query to find the count of NULL valuesQueryselect count(*) from demo where 2gusage is 'NULL';Output0Kindly help me out with the query to find the count of NULL values

I have enabled kerberos authentication on my cluster. After that when namenode is not starting due Checksum error. I am using cloudera 6.0.1 on Suse 12 OS hosted on GCP Failed to start namenode.org.apache.hadoop.security.KerberosAuthException: failure to login: for principal: hdfs/nn.asia-south1-c.c.cdh-cluster2.internal@ASIA-SOUTH1-C.C.CDH-CLUSTER2.INTERNAL from keytab hdfs.keytab javax.security.auth.login.LoginException: Checksum failedat org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1843)at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1211)at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1007)at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:313)at org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser(NameNode.java:694)at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:713)

Hi,I'm making a sandbox on AWS. I have 3 instances of t2.medium (2 vCPUs, 4GB RAM), with 32 GB disks. Everything was set up to pass the installation. Instances are running RHEL 7.6 (AWS image). I didn't touch much - just enabled passwordless login for root, added mysql to "master" node for hive. I'm getting no warnings or issues during installation.After installation, during 'startup phase' I get the following log. The "only" error I see is related to port 50070 being in use, but this is not the case - at least not before or after installation (on all three hosts actually).Obviously, almost nothing else starts afterwards.Here's the full log:stderr: <script id="metamorph-23723-start" type="text/x-placeholder"></script>Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py", line 361, in <module> NameNode().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/s

I am not able to launch the Ambari home page because an Oracle agreement page keeps coming up as a pop-up. What is the solution to this?

I have Nifi cluster(two instances). I generate urls using "GenerateFlowFile", then split it using "SplitText". For example;-> "GenerateFlowFile" is generate this;url1url2url3url4-> In "SplitText"(split count 2);url1url2---------url3url4-> I want to send these two flowfiles to other nifi instances in order. For this, i am using round robin. But i always want to go in the same order. What I want;url1, url2 ---> node1url3, url4 ---> node2url1, url2 ---> node1url3, url4 ---> node2url1, url2 ---> node1url3, url4 ---> node2-> But data is being sent this wayurl1, url2 ---> node1url3, url4 ---> node2url3, url4 ---> node1url1, url2 ---> node2How can I send same data to the same Nifi node?

I was able to open ambari and see it is installed but I couldn't find.

I am not able to see new users from linux into ranger ui.below is the error log of usersync.logplease help to resolve this.03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - GROUP: hadoop03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - GROUP: users03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - USER: aapuser03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - GROUP: aapuser03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - USER: hdfs03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - GROUP: hadoop03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - GROUP: hdfs03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - GROUP: hue03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - USER: yarn03 Jan 2019 12:02:15 DEBUG PolicyMgrUserGroupBuilder [UnixUserS

Sqoop import fails on empty table We have a batch job that looks into Teradata table and offloads delta records to HIVE based on a timestamp column. We havent used SQOOP incremental mode, Instead we have setup views on top of teradata tables and a joined to a control table to get the delta records. Once the offload is completed, We update the control table with MAX of timestamp record.   However, We are stuck with below issue: Import failed: com.teradata.connector.common.exception.ConnectorException: Input source table is empty   Could someone assist on the same? Regards, SV

We have a Hive External table where a couple of columns have XML data in them. The XML structure is valid but some of the records contain "?<" or "?>" in the data of an element. As you can see, the XML itself is valid as the value is quoted but Hive xpath is unhappy. Trying to figure out if there's a better way to escape out the data elements or have xpath ignore the extra "<". Example:<Root><data1><data2 id="4" value="622" name="CoID" /><data2 id="9999" value="Company XYZ" name="Company Name" /><data2 id="9999" value="2222345=0000?<" name="OfferId" /></data1></Root>Query:selectxpath_string(xml_column, '//Root [@name="OfferId"]/@value')from table;Here is the Error that we receive:Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 284; The value of attribute "value" associated with an element type "Root" must not contain the '<' character.  at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)  at org

I am facing connection issue for ranger usersync. because of that my users and groups are not able to sync in ranger.below is the error log of usersync.log02 Jan 2019 17:54:38 DEBUG PolicyMgrUserGroupBuilder [UnixUserSyncThread] - ==> PolicyMgrUserGroupBuilder.buildGroupList02 Jan 2019 17:56:45 ERROR UserGroupSync [UnixUserSyncThread] - Failed to initialize UserGroup source/sink. Will retry after 120000 milliseconds. Error details:com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection timed out (Connection timed out) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151) at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:104) at com.sun.jersey.api.client.Client.handle(Client.java:648) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)

HelloI've been setting up my secured nifi cluster with the help of the nifi-tls-toolkit in standalone mode. which outputs the nifi.properties file along with keystores and the certificate. I tried to use the tls-toolkit in client/server mode, which outputs the same keystores and certificate. but instead of outputting the nifi.properties file it outputs a config.json containing the keystore passwords,types etc. in a json file. In the nifi docs (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#client-server) nothing is really mentioned about the config.json file, accept configuration of the tls-toolkit where to output it (client) /read it from (server). and if i choose to use this method (client/server) the only valid solution for configuration of the nifi security properties is to copy the json fields into the nifi.properties of the node the certificate was created for (manually or via script).The main question i'm interested in is what is this config.json file?, is

Hi there!I set up standalone NIFI 1.8.0 and now try to communicate with it using node through his api.I am trying to deploy template which already exist on nifi (made it through GUI) : POST to "www.localhost:8080/nifi-api/process-groups/{THE_TEMPLATE_ID}/template-instance" but recieving error 415 status : media type not supported.I cant figure out what is worng with the request.I did not set up any authentication, but still when i tried to make any change like creating processor i am getting error 405 : not allowed. How is that possible? love for some help,john

I am new to Ambari ... It looks like it is not allowed to manually edit configuration files like /etc/hive/conf/hive-site.xml. I tried to make some changes but after I re-started Hive via Ambari - all my changes are gone. Can somebody comment on this and point me to some Amabari "best practices" documentation?

Hello,I have used Ambari 2.7.3 to setup an HDP 3.1.0 + HDF 3.3.0 cluster, and am trying to configure Knox proxy functionality but am having problems...the Knox admin UI doesn't seem to respect the setting of 30 minute sessions with the ShiroProvider. I have verified that the configs in Ambari all specify sessionTimeout of 30, which is supposed to be 30 minutes, but I get logged-out almost immediately when trying to use the admin UI. If I click down into the default read-only topologies, and then try to navigate back into the providers, I have to re-login, even though it's on;y been like 30 seconds.Any idea what the problem might be?Thanks in advance,Doug

I am using 3.0.0.0-1634. The annoying part is that the issue is random.The same job runs fine and suddenly it fails with this issue and next time some other job might fail after couple of days which had successfully ran earlier and which will run properly in future. There is no other jobs running during that time on the cluster. 18/12/29 20:25:21 INFO mapreduce.Job: Running job: job_1546013184089_004618/12/29 20:49:47 INFO mapreduce.Job: Job job_1546013184089_0046 running in uber mode : false 18/12/29 20:49:47 INFO mapreduce.Job: map 0% reduce 0%18/12/29 20:49:47 INFO mapreduce.Job: Job job_1546013184089_0046 failed with state FAILED due to: Application application_1546013184089_0046 failed 2 times due to ApplicationMaster for attempt appattempt_1546013184089_0046_000002 timed out. Failing the application.18/12/29 20:49:47 INFO mapreduce.Job: Counters: 018/12/29 20:49:47 ERROR crawl.DeduplicationJob: DeduplicationJob: java.io.IOException: Job failed! In Resource manager in Di

hi allwe have ambari cluster with HDP version 2.6.0when we try to start the metrics collector from ambari , we get the following logs:2019-01-02 06:35:16,406 ERROR org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher: hconnection-0x4bdeaabb-0x1680d44c1810001, quorum=master02.sys32.com:61181, baseZNode=/ams-hbase-unsecure Received unexpected KeeperException, re-throwing exceptionorg.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /ams-hbase-unsecure/table/SYSTEM.CATALOG at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:354) at org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(ZKUtil.java:622)2019-01-02 06:35:18,256 INFO org.apache.zookeeper.ClientCnxn: Opening sock

hi allwe try to start the ambari metrics and we get the following logs:2019-01-02 06:35:16,406 ERROR org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher: hconnection-0x4bdeaabb-0x1680d44c1810001, quorum=master02.sys32.com:61181, baseZNode=/ams-hbase-unsecure Received unexpected KeeperException, re-throwing exceptionorg.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /ams-hbase-unsecure/table/SYSTEM.CATALOG at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:354) at org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(ZKUtil.java:622)and also2019-01-02 06:35:18,256 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master02.sys32.com/101.23.98.60

unable to login with root user using hadoop password, I never changed the password

I tried to add another host via cloudera manager . I followed the below steps 1. Hosts - > Add hosts -> provided the ip address -> CM asked me to create a host template but i gave none and finised . 2. Found the host in good health . 3. Cluster -> HDFS -> Instances -> Add -> I added a Datanode with the new hosts.   (Now the Datanode i added was not starting , When i search for logs via UI i dont find any . I searched for logs in logs/datanode folder , but there was no Datanode folder in Var .)   4. Cluster -> Deploy client Configuration -> Completed -> Still the Datanode was not starting .   Please help .   Thanks Bala

i am trying to deploy client configuration on hbase , and it always fails , i am having 1 master , and 2 workers  , and here are the logs:     Generate client configuration.Successfully generated client configuration.   Run 3 steps in parallelCompleted 2/3 steps successfully       Cleanup client configs that failed to deploy.Successfully cleaned up client configs that failed to deploy.   Can't open /var/run/cloudera-scm-agent/process/740-deploy-client-config/hbase-conf/hive-env.sh: No such file or directory.+ '[' '!' -e /etc/hbase ']'+ cd /etc/hbase+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'+ '[' -e /etc/hbase/conf.cloudera.hbase/taskcontroller.cfg ']'+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'+ '[' -e /etc/hbase/conf.cloudera.hbase/container-executor.cfg ']'+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executo

Hi everyone! Is there a way where I can download all the videos on the Cloudera OnDemand app so that I can view it while commuting, instead having to download 1 by 1? Thanks!

In stanalone and secured NiFi 1.7.0 using oauth 2.0 as authentication mechanism. I try to use the SiteToSiteProvenanceReportingTask The property 'Transport Protocol' of Remote Process Group is 'HTTP'. I get the error message: Attached is the snapshot of the Reporting Task configurations: I get the below error in bulletin and logs: Unable to refresh Remote Group's peers due to response code 403:Forbidden with explanation: null 2018-11-12 06:20:27,984 INFO [Timer-Driven Process Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling SiteToSiteProvenanceReportingTask[id=<id>] to run The Site to Site properties nifi.properties are as below: nifi.remote.input.host=<hostname>, nifi.remote.input.secure=true, nifi.remote.input.socket.port=10443, nifi.remote.input.http.enabled=true, nifi.remote.input.http.transaction.ttl=30 sec nifi.remote.contents.cache.expiration=30 secs How to resolve it? Thanks for your help. Poonam

So I have 4 740xd Servers with 80 Threads on each and 384GB of ram each. How should I best handle heap for each nifi VM on the server? Option 1: 1, VM 80 cpus, NiFi set to 300GB of Ram for heap? And have the 4 nifis clustered. Option 2: 10, VMs 8 cpus each, NiFi set to 32GB of Ram for heap for each VM, These NiFis would also be clustered, across the Virtualized 740 and across the 4 hardware 740s Note: all these files going through these nifis are less than 4GB, and 90% of them are under 10MB ALSO what generation should I use for Java Garbage Collection? Attached are two pictures explaining my options. John

We're looking to automate the shutdown/restart the processor instead of manually stopping through Nifi UI. We don't want to bring down everything through nifi.sh stop, instead looking to bring down a specific processor.

Hi All, I have problem with kerberbos util command (ktutil), on client machine i created a new principal with add_entry and it's work fine but when i try to init ticket with kinit it's failed with message : "principal don't existe on kerberos database". when i create principal with kadmin.local on krb-server and try to init ticket on client it's work. any suggestion pleasee ?? Thanks for your help. Best Regards suggestion