About drgenious

drgenious · ‎09-24-2021

I am trying to learn the basics in cloudera and Impala and by reading this documentation : https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_cluster_util_custom.html#concept_jp4_4bh_hx I saw that with tsquery I can monitor Impala. I want to ask : 1) where can I run these kind of queries? 2) where can I find the attributes like category and clusterName in cloudera? Thanks in advance.

drgenious · ‎09-20-2021

How can I see the memory % that is used right now in Impala? Also I want to check the whole memory size

drgenious · ‎08-08-2021

Hi @ChethanYM , 1)I have created in impala. 2)result CREATE TABLE db.table ( ... ) PARTITIONED BY ( ... ) SORT BY ( ... ) STORED AS PARQUET LOCATION 'hdfs://path/' TBLPROPERTIES ('DO_NOT_UPDATE_STATS'='true', 'STATS_GENERATED'='TASK', 'impala.lastComputeStatsTime'='1627655409', 'numRows'='0', 'parquet.compression'='snappy', 'totalSize'='0') 3) I have listed the files and they seem ok 4)I can access it but returns again 0 results 5)It is not failing the query. It can not read the partitions as it did before.

drgenious · ‎08-07-2021

Hi, I have a table in Impala and it had some files in its location in hdfs. All of a sudden it can not read the files. I tried to invalidate and recover the partitions but nothing happened. I also moved the files from this folder into a new's folder path but it couldn't read them too. What can I do ? UPDATE Tried to repair the table in hive set hive.msck.repair.batch.size=1; set hive.msck.path.validation=ignore; msck REPAIR TABLE db.table but received this error : Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

drgenious · ‎08-04-2021

I have a table and when I am trying to query it through imapala it throws me this error: File 'hdfs:/path/table/1_data.0.parq' has an incompatible Parquet schema for column 'db.table.parameter_11'. Column type: STRING, Parquet schema: optional int64 amount [i:10 d:1 r:0] But when I query it through hive it returns results correctly. How can I fix it?

drgenious · ‎07-15-2021

I used to have some tables in HIVE but after an incident HIVE went down and some new tables erased and old tables went back to a previous schema state. What I want, is to ask if it is there any option to backup tables' schemas into MySQL which stores HIVE metastore in order to retrieve them if there occurred any incident again.

drgenious · ‎07-14-2021

I had a table that it vanished after an issue. I created again the table and then recover the partitions in order to read again the files that were in hdfs hive direcotry. Although all of a sudden now, it throws me the below error: File 'path/data.0.parq' has an incompatible Parquet schema for column 'db.table.parameter_11'. Column type: STRING, Parquet schema: optional int64 amount [i:10 d:1 r:0]

drgenious · ‎07-08-2021

I have a pyspark job with these configs: self.spark = SparkSession.builder.appName("example") \ .config("hive.exec.dynamic.partition", "true") \ .config("hive.exec.dynamic.partition.mode", "nonstrict") \ .config("hive.exec.max.dynamic.partitions", "5000000") \ .config("hive.exec.max.dynamic.partitions.pernode", "1000000") \ .enableHiveSupport() \ .getOrCreate() I can not find anywhere how to set a configuration to increase the max row size to 150mb. I found the command only in impala. Can you please help me ? Thanks in advance.

drgenious · ‎07-01-2021

Hi, I have noticed that when I restart impala in cloudera manager when I am trying to run queries the execution time is bigger than the usual. Is there any way to warm up impala somehow and after run the queries ?

drgenious · ‎06-17-2021

@ask_bill_brooks Hi, I have a table with a lot of data and I want to make the data be sorted in the level of the table in order to not put order by in my queries. What I want is to sort the main table immediately instead of transfer this data to another sorted table because it is time consuming. The table has been created without sort by. Is there any way to alter the configurations of the table. For example alter table <table_name> order by <column>

Online	Offline
Last Visited	‎06-26-2023 10:46 AM

Member Since	‎01-07-2020 06:44 AM
Last Visited	‎06-26-2023 10:46 AM
Posts	64
Kudos received	1

Cloudera Community

Where can I run tsquery ?

Find the % of the used memory in impala

Re: Impala can not read the files stored in the hi...

Impala can not read the files stored in the hive d...

How to fix Parquet schema: optional int64 amount [...

Backup MySQL tables schemas used in hive

Parquet schema error

Increase max row size in HIVE

Warm up impala

Re: Change the data in an impala table to be sorte...