Member since
11-11-2019
636
Posts
34
Kudos Received
27
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 966 | 10-09-2025 12:29 AM | |
| 8362 | 02-19-2025 09:43 PM | |
| 2439 | 02-28-2023 09:32 PM | |
| 5073 | 02-27-2023 03:33 AM | |
| 26756 | 12-24-2022 05:56 AM |
09-30-2021
10:30 AM
1 Kudo
@cortland I am able to acheive,please find my testcase abc.txt ==== "1","peter","He is Data enginer", "Senior Engineer" "2","Anee","Hadoop Engineer","Lead" "3","James","Data, Architect","Sr Architect" hdfs dfs -put abc.txt /user/hive/ create external table test_csv(num string ,name string ,work string ,designation string ) row format serde 'org.apache.hadoop.hive.serde2.OpenCSVSerde' with serdeproperties ( "separatorChar" = ',' ,"quoteChar" = '"' ) STORED AS TEXTFILE; LOAD DATA INPATH '/user/hive/abc.txt' INTO TABLE test_csv; select * from test_csv +---------------+----------------+---------------------+-----------------------+ | test_csv.num | test_csv.name | test_csv.work | test_csv.designation | +---------------+----------------+---------------------+-----------------------+ | 1 | peter | He is Data enginer | Senior Engineer | | 2 | Anee | Hadoop Engineer | Lead | | 3 | James | Data, Architect | Sr Architect | | 1 | peter | He is Data enginer | Senior Engineer | | 2 | Anee | Hadoop Engineer | Lead | | 3 | James | Data, Architect | Sr Architect | +---------------+----------------+---------------------+-----------------------+ Please accept it as solution,if your queries are answered and the testcase works in your scenario.
... View more
09-30-2021
01:07 AM
1 Kudo
@Mas_Jamie apologies. Unfortunately we dont have the document 😞
... View more
09-30-2021
12:00 AM
@enirys For a memory crash,we need heapdump. Please append the below in JAVA_OPTIONS of hiveserver2 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/disk2/dumps Make sure you provide the path correctly. Whenever there is a crash,an hprof file would be generated. You can use Eclipse MAT or jxray to analyze the leak suspect. You can also take heapdump on demand ,using "jmap" utility when the consumption is 80% jmap -dump:live,format=b,file=/disk2/dumps/dump.hprof <PID of Hiveserver2> Please let us know,if the queries are answered. Please ""Accept This a solution" if your queries are answered.
... View more
09-29-2021
11:52 PM
@Mas_Jamie CDS table is for Column Descriptors. CD stands for column descriptor. It is used to decouple columns from tables/partitions to reduce storage needs. tables/partitions are almost 1-1 map to these descriptors… but a table may have more than one Column Descriptor in the case of schema evolution!
... View more
09-27-2021
10:46 AM
@Anitauzanna could you explain the meaning of below the second column is date in int Could you please provide an example for the scenario??
... View more
09-25-2021
09:59 AM
Lets take an example Hiveserver1 ==> HMS ==> Mysqldatabase with schema(hive) ==> Contains databases as students(student database is in hive not MySQL) Hiveserver2 ==> HMS ==> Mysqldatabase with schema(test) ==> Contains databases as employee(employee database is in hive not MySQL)] When connected to beeline ==> hiveserve1 ==> Show databases would yield students When connected to beeline ==> hiveserve2 ==> Show databases would yield employees This must be performed using config groups in ambari. In CM it must be used using role instances.
... View more
09-25-2021
07:09 AM
@mailsmail This is not tested yet. You can create a POC environment and test. I don't have to create/run new MySQL instance for each metastore. Ans: A separatevschema is required. Avoid using hive/metastore as this would be by-default. use a different database/schema. Is it possible for one hiveserver2 to connect to multiple metastore, so all user will use the same endpoint? or do I have to run another hiveserver2 for each metastore? Ans: You can have one hiveserver2 to connect to multiple HMS by updating "hive.metastore.uris" field. You need to create a separate config group for another HMS with different database and also you need a different hiveserver2 and update "hive.metastore.uris" accordingly Please let me know,if you have any queries or accept it as solution,if your queries are answered.
... View more
09-25-2021
05:33 AM
@mailsmail You can have below approach: 1. create a separate schema in metastore db named user2 2. Add a hivemetastore pointing to same MySQL but different db user2 on hostA 3. Add a separate hiveserver2 on hostA 4. Create a config group and add the hostA 5. Override metastore uri for hiveserver2 This is one way you can achieve multitenancy.Your databases in hive would remain independent. Please let us know if you have additional quries.
... View more
09-22-2021
01:38 AM
You can find number of connections to HMS using lsof -p <HMS PID> | grep -i "ESTABLICHSED" | wc -l For checking number of db/table,you have to rely on hiveserver2 logs . We don't have any STATS to check how many times a particular DB/Table is accessed.
... View more
09-16-2021
10:06 AM
1 Kudo
@Sayed016 This error occurs if the HDFS files are under replicated. Did you see HDFS is under replicated when the issue occurred? Wait for the HDFS to gets replicated and you can query
... View more