Member since
02-27-2020
173
Posts
42
Kudos Received
48
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1092 | 11-29-2023 01:16 PM | |
1175 | 10-27-2023 04:29 PM | |
1157 | 07-07-2023 10:20 AM | |
2518 | 03-21-2023 08:35 AM | |
922 | 01-25-2023 08:50 PM |
05-25-2021
10:12 AM
Hello, I haven't used Flume myself, but there is some mention of serializer.delimiter parameter in the Flume documentation. It would be helpful to know what the source of the data is (e.g. file on hdfs) and what the destination is (e.g. Hive). Also you should know that in Cloudera Data Platform, Flume is no longer a supported component. If you are just starting to learn it, I would recommend saving yourself some time and exploring NiFi, Kafka, and Flink (good starter blog post). Regards, Alex
... View more
05-25-2021
10:06 AM
Hi Nuno, I've sent you a private message to get some more information. Please respond there and we'll figure out how to help. Regards, Alex
... View more
01-21-2021
09:11 PM
Hi Igor, You can define what users can and cannot do in Atlas by way of defining authorization policies in Ranger. Details on how to do that can be found here: https://docs.cloudera.com/runtime/7.2.6/atlas-securing/topics/atlas-configure-ranger-authorization.html What you refer to as Bookmarks can potentially be done via Saved Searches (see here), depending on what you want to achieve. As for the popularity score, this could be made a metadata attribute that can be updated by users. There is no automation to derive this score with Atlas out-of-the-box. Hope this helps, Alex
... View more
01-15-2021
04:25 PM
That is an odd behaviour. Two things to try: In Cloudera Manager go hdfs -> Configuration -> Enable Log and Query Redaction and switch that to false. Go to HBase -> Configuration -> Enable Audit Collection and turn it off. Restart the HBase service and run your sqoop job again. See if that helps with performance.
... View more
01-15-2021
10:11 AM
Two things to check: 1. Does your nifi service user account have permissions on the table and hdfs location where it's trying to do the insert; 2. Your Hive SQL statement here looks a bit off to me: insert into transformed_db.tbl_airbnb_listing_transformed
select a.*, 20210113 partition_date_id from
staging_db.etbl_raw_airbnb_listing a Is 20210113 a column name? Are you missing a comma between that and parition_date_id? Is your source staging table partitioned? If you are trying to select only a specific date, than the syntax to do that is different.
... View more
01-14-2021
09:48 AM
There could be a problem with running the SQL in your Hive cluster. I would suggest checking the Hive logs for any relevant errors when NiFi flow is triggered. Another thing to check is the FlowFile itself to make sure it has the data that Hive table would be expected (i.e. schema matches). You can do this by forwarding the failed FlowFiles to a rejected flow.
... View more
12-18-2020
11:56 AM
Settings look fine. _HOST gets replaced by the actual FQDN of the host at runtime. One thing to check is to make sure reverse DNS lookup works on all hosts.
... View more
12-18-2020
10:35 AM
Out-of-the-box Hue can't properly parse this format. There are some potential solutions in this thread: https://stackoverflow.com/questions/13628658/hive-load-csv-with-commas-in-quoted-fields and it depends on what you are comfortable with: pre-processing the file to reformat the input or to use a different SerDe in Hive. Hope that helps, Alex
... View more
12-16-2020
11:03 PM
This could be as simple as a typo somewhere in your configs, but it's hard to tell where. Looks like the username was set to "host" somehow and hdfs can't authenticate that as a user. Without more context it's hard to say any more.
... View more