Hello @Sayed016 , In general the  java.io.IOException: Filesystem closed message happens when the same or a different thread in the same JVM called the "FileSystem.close()" (see JavaDoc) method - and something later tries to access the HDFS filesystem. (in this case the "EventLoggingListener.stop()" tries to access the HDFS to flush the Spark event logs to HDFS) FileSystem.close() should not be called by any custom code, as there is a single shared instance of the FileSystem object in any given JVM instance and it can cause failures for the still running frameworks like Spark. This suggests that the Spark application has the above FileSystem.close() call somewhere in the code. Please review the code and remove those. Hope that helps. Best regards, Miklos ... View more

Hi Rama, yes, you can configure that in the "Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini". OPSAPS-41615 is still open, in the future you can ask the status from any of your account team contacts. If you don't know who are those contacts, please ask/clarify that through the already open support case. Best regards,  Miklos ... View more

Hello @ram76 ,   You can configure Hue to use the XFF header: [desktop] use_x_forwarded_host=true See hue.ini reference: https://github.com/cloudera/hue/blob/master/desktop/conf.dist/hue.ini If not already done, besides using an external load-balancer (like F5 - to let the end users remember only a single Hue login URL) please consider to add "Hue Load Balaner" role in CM > Hue service  (which sets up an Apache httpd) to serve the static contents. See the following for more: https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/hue_use_add_lb.html#hue_use_add_lb  Hope this helps. Best regards, Miklos ... View more

Hello @mhsyed , thanks for reporting this. I see similar description on our Partner page too: https://www.cloudera.com/downloads/partner/intel.html Seems the link is broken because the "Intel-bigdata" Github does not have the "mkl-wrappers-parcel-repo" anymore: https://github.com/orgs/Intel-bigdata/repositories I have involved our respective teams to get in touch with Intel to fix this. Unfortunately I cannot offer any other workaround in the meantime, we ask for your patience.   Best regards  Miklos Szurap Customer Operations Engineer, Cloudera ... View more

Hi @M129 , the error message is not too descriptive. Can you please check the HiveMetaStore logs what is the complete error message - and reason for the failure? Thanks  Miklos  ... View more

Hi @Rajeshhadoop , The Spark DataSource API has the following syntax: val jdbcDF = spark.read.format("jdbc").option("url", "jdbc:...")...load() Please see: https://spark.apache.org/docs/2.4.0/sql-data-sources-jdbc.html  The problem with this approach - using with Hive or Impala is that since the above may run on multiple executors, this could overwhelm and essentially DDOS the Hive / Impala service.  As the documentation states this is not a supported way of connecting from Spark to Hive/Impala.   However you should be able to still connect to Hive and Impala through a simple JDBC connection using "java.sql.DriverManager" or "java.sql.Connection". That in contrast runs only on a single thread, on the Spark driver side - and will create a single connection to a HiveServer2 / Impala daemon instance. The throughput between the Spark driver and Hive/Impala of course is limited with this approach, please use it for simple queries or submitting DDL/DML  queries. Please see  https://www.cloudera.com/downloads/connectors/hive/jdbc.html https://www.cloudera.com/downloads/connectors/impala/jdbc.html  for the JDBC drivers and for examples.   Independently of the above, you can still access Hive tables' data through SparkSQL with val df = spark.sql("select ... from ...") which is the recommended way of accessing and manipulating Hive table data from Spark as it is parallelized through the Spark executors. See docs: https://spark.apache.org/docs/2.4.0/sql-data-sources-hive-tables.html  I hope this clarifies it. Best regards  Miklos ... View more

Hi @Jmeks , Can you clarify which CDH/HDP/CDP version do you have? What is the datatype of that "date" partitioning column? The mentioned syntax works for both string and date datatypes in CDH 6.x. ... View more