Member since
07-16-2015
177
Posts
28
Kudos Received
19
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
9539 | 11-14-2017 01:11 AM | |
54634 | 11-03-2017 06:53 AM | |
3554 | 11-03-2017 06:18 AM | |
11716 | 09-12-2017 05:51 AM | |
1378 | 09-08-2017 02:50 AM |
04-24-2022
01:25 AM
No it is not. You get the exact same error as before. Does anybody know how to actually fix this broken program?
... View more
04-23-2022
08:42 PM
What kind of an answer is this? Where is the help and how is this done? Nothing about this software is in a functioning working condition in any way.
... View more
07-12-2021
11:56 AM
Hi Tony, I will reach out via private message
... View more
11-05-2020
09:25 AM
We have opened a ticket to the cloudera support. They told below After reviewing the HS2 logs we see GC Pauses which go up to a minute which is causing hive zookeeper sessions to get timed out.The GC messages are likely triggering a ZooKeeper bug condition discussed here - https://issues.apache.org/jira/browse/ZOOKEEPER-2323
... View more
02-27-2020
06:45 AM
I can see a function called MONTH NAME . https://impala.apache.org/docs/build/html/topics/impala_datetime_functions.html#datetime_functions__month
... View more
10-04-2019
12:02 PM
I am having the same error but I didn't understand the solution, can you please explain it. Thank you.
... View more
07-26-2019
02:18 PM
> Is there any option to find empty directory using HDFS command Directly? You can get a list/find empty directories using the 'org.apache.solr.hadoop.HdfsFindTool'. And using the hdfs tool to check/test if _a_ directory is empty, you can use -du or -test; please see the FileSystemShell [0] test
Usage: hadoop fs -test -[defsz] URI
Options:
-d: f the path is a directory, return 0.
-e: if the path exists, return 0.
-f: if the path is a file, return 0.
-s: if the path is not empty, return 0.
-r: if the path exists and read permission is granted, return 0.
-w: if the path exists and write permission is granted, return 0.
-z: if the file is zero length, return 0.
Example:
hadoop fs -test -e filename du
Usage: hadoop fs -du [-s] [-h] [-x] URI [URI ...]
Displays sizes of files and directories contained in the given directory or the length of a file in case its just a file.
Options:
The -s option will result in an aggregate summary of file lengths being displayed, rather than the individual files. Without the -s option, calculation is done by going 1-level deep from the given path.
The -h option will format file sizes in a “human-readable” fashion (e.g 64.0m instead of 67108864)
The -x option will exclude snapshots from the result calculation. Without the -x option (default), the result is always calculated from all INodes, including all snapshots under the given path.
The du returns three columns with the following format:
size disk_space_consumed_with_all_replicas full_path_name
Example:
hadoop fs -du /user/hadoop/dir1 /user/hadoop/file1 hdfs://nn.example.com/user/hadoop/dir1
Exit Code: Returns 0 on success and -1 on error. [0] https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html
... View more
06-25-2019
09:39 AM
2 Kudos
We use Percona distro of MySQL for a MySQL cluster. We use Haproxy to connect to the cluster. Our actual connect is to localhost:port that Haproxy proxies to mysql:port All of our DB connection configurations specify localhost:3306
... View more
06-06-2019
11:09 PM
I added above values and that was causing https to shutdown. After deleting those values , it started and working fine now. Thanks @Harsh J for your reply.
... View more
05-31-2019
10:29 AM
Not helpful yet, but promising... PIVOT keyword is reserved for future use! https://www.cloudera.com/documentation/enterprise/6/6.2/topics/impala_reserved_words.html
... View more
04-08-2019
11:03 AM
i am also facing same error , may i know where you increased the memory
... View more
08-13-2018
02:19 PM
This issue was resolved by following the instructions in this site: http://vijayjt.blogspot.com/2016/02/how-to-connect-to-kerberised-chd-hadoop.html We need to copy the Java JCE unlimited strength policy files and the krb5.conf file under jdk/jre/lib/security folder where SQL Developer is installed. After this the Hive connection via Kerberos was successful.
... View more
07-11-2018
02:08 PM
how did you stored logs in local? can you please provide me the way you did it.
... View more
01-16-2018
01:33 AM
Hi, It's been a while ! If I remember correctly, we did not find any solution back then (with CDH5.3.0) - at least other than recreating the collection and re-indexing the data. But after upgrading the CDH version using a version of Solr supporting the "ADDREPLICA" and "DELETEREPLICA" functions in the API you can add an other replica and then delete the one which is down. regards, mathieu
... View more
01-12-2018
09:18 AM
2 Kudos
You can use PURGE option to delete data file as well along with partition mentadata but it works only in INTERNAL/MANAGED tables ALTER TABLE table_name DROP [IF EXISTS] PARTITION partition_spec PURGE; External Tables have a two step process to alterr table drop partition + removing file ALTER TABLE table_name DROP [IF EXISTS] PARTITION partition_spec; hadoop fs -rm -r <partition file path>
... View more
12-13-2017
12:05 AM
I think the keytab you used has expired. Try to kinit a new keytab for your code, and issue should be solved
... View more
11-29-2017
06:26 AM
Hi. Yes, jobs are scheduled as Oozie Coordinators. The problem is that job description say that the job will be executed at every even hour, but job starts executing at every odd hour. Do you know any other way to fix this, since this happened on our production environment and "reinstalling" would take a lot of time which could lead to downtime? Thank you in advance.
... View more
11-21-2017
08:32 PM
@saranvisa. Thanks for reply. We are using Hadoop 2.6 and mapreduce.map.memory.mb and mapreduce.reduce.memory.mb is 1 Gb resp. Also mapreduce.map.java.opts and mapreduce.reduce.java.opts is around 800MB. In the logs I can see below error as well. Exit code: 1 Stack trace: ExitCodeException exitCode=1: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 1 Can you please help me ? Thanks, Priya
... View more
11-15-2017
03:19 AM
Is there any way to integrate with cloudera infracture. I mean setting up the slider inside the cloudera infrastrure and make solr to run in yarn contianer
... View more
11-13-2017
11:31 AM
I continued the resolution of this issue in another thread specific to the error: ls: Operation category READ is not supported in state standby The solution is marked on that thread, however, a quick summary was that I needed to add the Failover Controller role to a node in my cluster, enable Automatic Failover, and then restart the cluster for it all to kick in.
... View more
11-13-2017
09:14 AM
Great ! Nice debug
... View more
11-10-2017
01:11 AM
First : - Why do you suspect that Hiveserver2 need to be up and running for impala query to works ? Did you observe that when running Hiveserver2 impala queries worked ? By the way, Hivemetastore is running right ? - Second : You should go look at hue server log files. You should see a better error. regards, mathieu
... View more
11-08-2017
05:12 PM
Thanks, can you share out some more details? such as workflow, scripts
... View more
11-06-2017
07:28 PM
@mathieu.d, Thanks for reply, will try this and let you know.
... View more
11-03-2017
06:53 AM
1 Kudo
The timestamp column is not "suitable" for a partition (unless you want thousands and thousand of partitions). What is suitable : - is to create an Hive table on top of the current not partitionned data, - create a second Hive table for hosting the partitionned data (the same columns + the partition column), - eventualy load the data from the first table to the second one using a query that will "parse" the timestamp column and extract what should be a suitable value for the partition column (for example the year or the year-and-the-month, ...). Example : INSERT INTO TABLE my_partitioned_table PARTITION (part_col_name) SELECT *, year(to_date(my_timestamp_column)) FROM my_not_partitioned_table; You don't have to put the partition value in the insert statement if you enable dynamic partition in Hive. set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict; And on your sample it's not working properly because you didn't parse the timestamp column, you use it as is. Each unique value will create a partition. For a timestamps, it's almost each value that is unique.
... View more
11-03-2017
06:38 AM
Alternatively you could search around "yarn queue" and ressource allocation. This will not "restrict" the number of mappers or reducers but this will control how many can run concurrently by giving access to only a subset of the available resources.
... View more
11-03-2017
06:22 AM
Thanks for quick response. Your answer is fine, but I am looking something specific for python-spark code. Thanks, Palak
... View more