This is a known issue and has been fixed in CM 6.2. Here is the relevant item in the release notes.   Cheers, Lars ... View more

This sounds like a result of the drastically increased link latency between your two "racks". While within a single rack you should see latencies less than a millisecond, US-EU latencies will be around 150ms, depending on where in the US and EU your machines are located. Bandwidth between your locations is likely also much lower than between the racks.   Impala currently does not do any rack-aware scheduling of I/O and data exchanges. In addition it is not optimized for high variance in link latencies and throughput. HDFS itself to my knowledge also makes no optimizations for such a case.   Frankly, I don't think you will see good performance in such a scenario. If you want to increase data availability, you could explore replicating the data between your locations while running queries in only one at a time. If you want to increase service availability, you can look into using a load balancer and switching from one cluster to the other in case of failure. ... View more

Thank you Chris for providing more information. It looks like it crashed in the code that writes Parquet files (HdfsParquetTableWriter::ColumnWriter<impala::StringValue>::ProcessValue). However, your query should not write any data: " SELECT a.topLevelField, b.priceFromNestedField FROM db.table a LEFT JOIN a.nestedField b"   I also noticed that the stack looks like it has been overwritten by something.   I don't recall any recent issues in that method and will have a look at the code to see if I can spot anything obvious.   In the meantime, can you double check that this query caused the crash and no other query was running?   Thanks, Lars ... View more

Let's see what the hs_err_pid file contains next.   Additionally, would you be willing to share the Minidump or a core dump with us in private? Please be aware that Minidumps contain process memory of each thread's stack, and core dumps contain all of the process's memory. Let me know if you'd like to do that and I'll share a private upload link with you.   Alternatively you can follow these instructions to resolve the minidump yourself and share the contained stack traces: https://cwiki.apache.org/confluence/display/IMPALA/Debugging+Impala+Minidumps ... View more

Hi,   Can you post the ends of the INFO and ERROR logs? Can you also post the content of the hs_err_pid<pid>.log file?   Thanks, Lars ... View more

For Python it makes a difference whether output gets printed to the terminal (which in this case likely supports unicode) or output is redirected to a file (which means it needs to be encoded in ASCII).   This post on StackOverflow seems to describe the issue well. I linked the post in the JIRA for future reference.   Cheers, Lars ... View more

Hi GeKas,   I'm not sure I understood your question. In general, writing to stdout should respect the local language settings of your shell:   $ echo $LANG en_US.UTF-8   Writing to a file however does not need to respect these, so it's behavior may be different. ... View more

This looks like IMPALA-2717 to me. The Jira has a patch attached to it, but no-one ever seems to have pushed a code review for this. Unfortunately there's no targeted release for this issue. Contributions are always welcome, let me know if you want to give it a shot.   Cheers, Lars ... View more

After more investigation I found that this is already documented as a Known Issue in CM: Known Issues and Workarounds in Cloudera Manager 5   For Impala I opened IMPALA-5631 to explain the problem and possible solutions in the docs. ... View more

@mbigelow - Thank you for keeping the JIRA updated - I'm glad you found the solution through support. It looks like you are hitting a bug in CM and we are working on fixing it. I will reach out to our documentation team to point out this issue in the docs and the release notes of 5.11.1. I'm sorry for the troubles this has caused you. ... View more

Hi Alon,   Have you tried the  inc_stats_size_limit_bytes command line flag as suggested by Tim? It is supported on CDH5.10.0. Here's the full help text from impalad:   -inc_stats_size_limit_bytes (Maximum size of incremental stats the catalog is allowed to serialize per table. This limit is set as a safety check, to prevent the JVM from hitting a maximum array limit of 1GB (or OOM) while building the thrift objects to send to impalads. By default, it's set to 200MB) type: int64 default: 209715200     This should allow you to increase the limit you are hitting.   Cheers, Lars ... View more

Thank you for catching this Tim!   The "SORTBY()" hint was added in IMPALA-4163, which was not included in Impala 2.8.0. It is currently being reworked into a SQL clause (IMPALA-4166), so I cannot make promises as to which release will contain this feature. My apologies for the confusion. I will make sure the documentation gets updated. ... View more

Hi imad87,   What is the purpose of specifying " ROW FORMAT DELIMITED" without a delimiter character?   On first glance it looks like your data file contains the substring "\N" (the \ character followed by the N character) to delimit lines, instead of the "\n" character (ASCII 0xA). Can you double check the file in a hex editor?   Cheers, Lars ... View more