About ebeb

ebeb · ‎12-21-2021

Impala doesnt support the ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde' even in newer version like v3.4.0. Any other option to remove double quotes in the output from Impala where the input csv file has quotes?

ebeb · ‎12-21-2021

Hello @lalala @ChethanYM did you find any solution to remove double-quote in impala output on a csv external file table. Input external .gz file has row: true,false,"US","Los Angeles","California","Cloudflare" Output of select * from impala external table defined on .gz file above shows: true false "US" "Los Angeles" "California" "Cloudflare" FYI in AWS Athena doc it says: https://docs.aws.amazon.com/athena/latest/ug/lazy-simple-serde.html LazySimpleSerDe for CSV, TSV, and Custom-Delimited Files Use this SerDe if your data does not have values enclosed in quotes. How to remove the double-quotes.

ebeb · ‎12-18-2021

Thanks for the info we will try to remove Activity Monitor. Cloudera should have automatically removed AM during upgrade to 7.x if deprecated as promised.

ebeb · ‎12-18-2021

Hello, After upgrading to CM 7.4.4/CR 7.1.7 the Activity Monitor never starts. It is always down and cannot be started after multiple attempts. It gives error messages below any thoughts? Command (Start this Activity Monitor (17683)) has failed Supervisor returned FATAL. Please check the role log file, stderr, or stdout. The health test result for ACTIVITY_MONITOR_SCM_HEALTH has become bad: This role's process failed to start. STDOUT messages: 13:06:12.204 [main] ERROR org.hibernate.engine.jdbc.batch.internal.BatchingBatch - HHH000315: Exception executing batch [java.sql.BatchUpdateException: Data truncation: Out of range value for column 'METRIC_ID' at row 1], SQL: insert into CMON_METRIC_INFO (NAME, METRIC_ID) values (?, ?) 13:06:12.209 [main] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Data truncation: Out of range value for column 'METRIC_ID' at row 1 Sat Dec 18 13:06:17 EST 2021 JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64 CONF_DIR=/var/run/cloudera-scm-agent/process/7594-cloudera-mgmt-ACTIVITYMONITOR 13:06:30.645 [main] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Data truncation: Out of range value for column 'METRIC_ID' at row 1 at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101) ... 2 more Caused by: java.sql.BatchUpdateException: Data truncation: Out of range value for column 'METRIC_ID' at row 1 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.Util.getInstance(Util.java:408) at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1163) at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1823) at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1307) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970) at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544) at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:121) ... 22 more Caused by: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Out of range value for column 'METRIC_ID' at row 1 at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3976) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2495) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1903) at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2124) at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1801) ... 26 more

ebeb · ‎12-14-2021

The script provided in https://github.com/cloudera/cloudera-scripts-for-log4j to delete the JndiLookup.class from log4j jar may have an issue if command zip or unzip is not installed on the cluster nodes. The script should initially check for yum list zip and unzip modules if available and abort if not found. Else it will give a finished message even though it had errors and didnt run successfully like below: Completed removing JNDI from /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/share/doc/search-1.0.0.7.1.6.0/examples/test-documents/testJPEG_EXIF.jpg.tar.gz Backing up to '/tmp/tmp.FsVTS5Rg9y//opt/cloudera/cm/lib/solr-upgrade-1.0.0.7.1.7.0-547.tar.gz.backup' Patching '/opt/cloudera/cm/lib/solr-upgrade-1.0.0.7.1.7.0-547.tar.gz' Running on '/tmp/tmp.Yxjh6FQYgS' Backing up files to '/tmp/tmp.TsL7gbbmHR' Completed removing JNDI from jar files ./cm_cdp_cdh_log4j_jndi_removal.sh: line 114: unzip: command not found grep: /tmp/unzip_target/**/*.jar: No such file or directory Completed removing JNDI from nar files Recompressing Completed removing JNDI from /opt/cloudera/cm/lib/solr-upgrade-1.0.0.7.1.7.0-547.tar.gz INFO : Finished

ebeb · ‎11-24-2021

Hello @MahendraDevu Did you resolve the error in HUE SAML we are getting this in CDP 7.1.7 after upgrade. SAML was working in CDH5.16 HUE before upgrade: [05/Apr/2019 16:37:03 -0400] views ERROR SAML Identity Provider is not configured correctly: certificate key is missing! UPDATE: Resolved this issue by making the IDP <md:EntityDescriptor entityID same as that on the metadata.xml we specified in HUE Advanced Configuration snippet hue_safety_valve.ini metadata_file . There was a mismatch between IDP value and what was in the metadata file.

ebeb · ‎07-28-2021

After openjdk 1.8.0.292 install we had to fix the mysql scm database ssl connection issue using the property useSSL=false like com.cloudera.cmf.orm.hibernate.connection.url=jdbc:mysql://<mysql-host>/scm?useUnicode=true&characterEncoding=UTF-8&useSSL=false Now same ssl connection issue is happening for nav, navms, amon, rman databases and these components wont start. Can we set the useSSL for these databases also in /etc/cloudera-scm-server/db.properties file like below: com.cloudera.cmf.orm.hibernate.connection.url=jdbc:mysql://localhost/nav?useUnicode=true&characterEncoding=UTF-8&useSSL=false com.cloudera.cmf.orm.hibernate.connection.url=jdbc:mysql://localhost/navms?useUnicode=true&characterEncoding=UTF-8&useSSL=false com.cloudera.cmf.orm.hibernate.connection.url=jdbc:mysql://localhost/rman?useUnicode=true&characterEncoding=UTF-8&useSSL=false

ebeb · ‎06-26-2021

Yes /etc/krb5.conf was the issue in the destination TST cluster. Specifically the [domain_realm] section did not have proper mapping for the replication source prod hosts and the prod realm. Since the PRD and TST hosts were all in the same domain so it was not easy to map the domain and realm using just the .proddomainxyz.com = PRODREALM . Instead we had to specify the individual source prod hosts in the destination TST /etc/krb5.conf file as below: [domain_realm] prodhostxx1 = PRODREALM prodhostxx2 = PRODREALM prodhostxx3 = PRODREALM etc. specify all the source prod hosts here If we don't specify the prodhosts like above replication doesn't know how to map the prodhosts to the PRODREALM and picks up the default TSTREALM which is why the original wrong realm error happened. Thanks!

ebeb · ‎06-23-2021

Hello Experts, We are trying to do hdfs replication from prod(CDH 5.x) to test(CDP 7.x) and getting a kerberos principal error: Server has invalid Kerberos principal: hdfs/prodhostxx.com@TESTREALM, expecting: hdfs/prodhostxx.com@PRODREALM Not sure why it is picking up TEST realm instead of the PROD realm in the job. The Peer test shows connected fine. Thanks!

ebeb · ‎06-17-2021

Darren, This got resolved with the help of Cloudera Support engineer. There were couple of issues. First the IDP and SP entity_id should be different values. Next there is a SAML property that needs to be set by the IDP which is <saml:AudienceRestriction><saml:Audience /></saml:AudienceRestriction></saml:Conditions> After setting the Audience property to the entity_id of the SP the error went away and HUE SAML is working again. See below error before setting Audience property: response DEBUG conditions: <?xml version='1.0' encoding='UTF-8'?> <saml:Conditions xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" NotBefore="2021-06-02T16:02:45.573Z" NotOnOrAfter="2021-06-02T17:02:45.573Z"><saml:AudienceRestriction><saml:Audience /></saml:AudienceRestriction></saml:Conditions> [02/Jun/2021 09:02:45 -0700] client_base ERROR XML parse error: 'NoneType' object has no attribute 'strip' [02/Jun/2021 09:02:45 -0700] middleware INFO Processing exception: 'NoneType' object has no attribute 'strip': Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response This looks like a breaking change from CDH 5.15 HUE saml to CDP 7.1.6 as we never set the Audience value in CDH 5.x in IDP before.

Online	Offline
Last Visited	‎12-20-2023 04:37 PM

Member Since	‎09-14-2017 07:07 AM
Last Visited	‎12-20-2023 04:37 PM
Posts	120
Kudos received	11

Cloudera Community

Re: HUE SAML error after upgrade to CDP 7.1.6

Re: CDP 7.2.4 upgrade - cloudera agents not starti...

Re: How to run Python script in Hue through oozie

Re: Cluster installation failure - src file /opt/c...

Re: spark.yarn.executor.memoryOverhead

Re: How to remove double quote from csv file at ti...

Re: Remove double quotation in impala output

Re: Start this Activity Monitor has failed. Activi...

Start this Activity Monitor has failed. Activity M...

Re: Log4j2 vulnerability - official solution

Re: HUE fails with saml

Re: Cloudera 6.3 MySql 5.7 SSL Warning

Re: CDP 7.x replication kerberos error

CDP 7.x replication kerberos error

Re: HUE SAML error after upgrade to CDP 7.1.6