About EFasdfSDfaSDFG

EFasdfSDfaSDFG · ‎06-10-2024

So I figured something out: use my_db; CREATE EXTERNAL TABLE IF NOT EXISTS a_iceberg_1 (the_month STRING, the_day INT, the_time INT, host STRING, message STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE LOCATION 'ofs://path/to/file' TBLPROPERTIES('iceberg.catalog'='hadoop.tables') Only now I have to figure out how to delimit the fields properly: Jun 4 03:10:01 host rsyslogd: [origin software="rsyslogd" swVersion="8.24.0-57.el7_9.3" x-pid="3829" x-info="http://www.rsyslog.com"] rsyslogd was HUPed

EFasdfSDfaSDFG · ‎06-10-2024

Hi, From those examples, can I create an external Iceberg table over a file/directory the same way that Hive is able to?

EFasdfSDfaSDFG · ‎06-04-2024

I created the table using the Impala editor

EFasdfSDfaSDFG · ‎06-04-2024

Hello, Complete noob here. I have a file, and have created an iceberg table over it (via Hue). However, I can't select any data. How do I set the schema up correctly? CREATE TABLE iceberg_test_1 (month STRING, day STRING, time STRING, host STRING, source STRING, body STRING) PARTITIONED BY(event_day INT, event_month INT, event_year INT) STORED BY ICEBERG LOCATION 'ofs://ozone/user/logs/test'; The file is: Jun 4 08:26:57 host smartd[2657]: Device: /dev/bus/0 [megaraid_disk_05], SMART Failure: DATA CHANNEL IMPENDING FAILURE GENERAL HARD DRIVE FAILURE

EFasdfSDfaSDFG · ‎03-14-2024

My understanding is that since you have restriction on installing awscli on RHEL, you are seeking alternative solutions to access/manage Ozone. Is my understanding correct? This is correct. Specifically I am asking how to/if I can use curl to upload files to Ozone: @EFasdfSDfaSDFG wrote: Hi, We can access/read/write Ozone etc using aws s3api, but would like to see if we can do the same using curl. We may not have the option of using the awscli package in RHEL. We've tried curl with webhdfs, but that is a bit limited. This almost seems to work: $ curl -vvv -H "Authorization:AWS user@FQDN:aws_key" https://host73.fqdn:9879/s3v/bucket/curl_test But it seems to get stuck here: > GET /s3v/bucket/curl_test HTTP/1.1 > User-Agent: curl/7.29.0 > Host: host73.fqdn:9879 > Accept: */* > Authorization:AWS user@FQDN:aws_key > It makes the connection to the Ozone S3 gateway, but then just hangs. Does anyone know why it's getting stuck? Log messages seems to be inconsistent, and there's nothing in Ranger. Everything we've tried hasn't worked yet. As for https://ozone.apache.org/docs/1.4.0/interface.html, Java API might be a possible better option than cURL.

EFasdfSDfaSDFG · ‎03-11-2024

Hello @vaishaakb , We are running CDP 7.1.8-1.cdh7.1.8.p36.42193080 (and preparing to patch to 7.1.9 soonish), and Ozone 718.2.3-1.ozone718.2.3.p0.46980125.

EFasdfSDfaSDFG · ‎01-29-2024

Log messages show errors like: org.apache.hadoop.ozone.s3.signature.MalformedResourceException But I can;t find details on what the headers or resource should be.

EFasdfSDfaSDFG · ‎01-29-2024

Hi, We can access/read/write Ozone etc using aws s3api, but would like to see if we can do the same using curl. We may not have the option of using the awscli package in RHEL. We've tried curl with webhdfs, but that is a bit limited. This almost seems to work: $ curl -vvv -H "Authorization:AWS user@FQDN:aws_key" https://host73.fqdn:9879/s3v/bucket/curl_test But it seems to get stuck here: > GET /s3v/bucket/curl_test HTTP/1.1 > User-Agent: curl/7.29.0 > Host: host73.fqdn:9879 > Accept: */* > Authorization:AWS user@FQDN:aws_key > It makes the connection to the Ozone S3 gateway, but then just hangs. Does anyone know why it's getting stuck? Log messages seems to be inconsistent, and there's nothing in Ranger. Everything we've tried hasn't worked yet.

Online	Offline
Last Visited	‎12-17-2024 01:27 AM

Member Since	‎01-13-2022 06:43 PM
Last Visited	‎12-17-2024 01:27 AM
Posts	8
Kudos received	1

Cloudera Community

Re: Creating Iceberg table

Re: Creating Iceberg table

Re: Creating Iceberg table

Creating Iceberg table

Re: Ozone S3 with AWS credentials

Re: Ozone S3 with AWS credentials

Re: Ozone S3 with AWS credentials

Ozone S3 with AWS credentials