Tutorial Exercise 2 - Analysis Exception

[#1]

I get this msg when create table queries are executed in Impala's query editor..

AnalysisException: Invalid avro.schema.url: hdfs://quickstart/user/examples/sqoop_import_categories.avsc. Path does not exist. CAUSED BY: TableLoadingException: Invalid avro.schema.url: hdfs://quickstart/user/examples/sqoop_import_categories.avsc. Path does not exist.

What should I do?

[#2 · Accepted solution]

In tutorial #1 you copied some *.avsc files into HDFS. My guess is that
step was skipped it failed for some reason. I would suggest trying that
step again.

[#3]

Hi , 

In the same example I got this error message:

Your query has the following error(s):

Could not connect to quickstart.cloudera:21050

and does not allow me to continues.

I follow all the steps from Lesson 1

I'm using the Virtual machine:

cloudera-quickstart-vm-5.4.2-0-vmware 

That I downloaded it from your web site. 

could you help me on this

Regards.

[#4]

I would check if the Impala process is running. It should be by default,
but you can confirm this with 'sudo service impala-server status' on the
command-line. If it is not running, you can try simply restarting it with
'sudo service impala-server restart', or check the logs in /var/log/impala
for any error messages if problems continue.

[#5]

Hi, 

I got the next Error Message at the log file:

Log file created at: 2015/09/08 17:38:18
Running on machine: quickstart.cloudera
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0908 17:38:18.531857 22029 logging.cc:119] stderr will be logged to this file.
E0908 17:39:01.037652 22029 impalad-main.cc:79] Impalad services did not start correctly, exiting.  Error: Couldn't open transport for 127.0.0.1:24000 (connect() faile$
State Store Subscriber did not start up.

and every time that I try to import the data , ( after I restarted the services and the status shows me it OK ).

I run the import script.. and I got the error message:  "Could not connect to quickstart.cloudera:21050"

and the Impala services is failed again.

[cloudera@quickstart ~]$ sudo service impala-server status
Impala Server is dead and pid file exists                  [FAILED]
[cloudera@quickstart ~]$ sudo service impala-server restart
Stopped Impala Server:                                     [  OK  ]
Started Impala Server (impalad):                           [  OK  ]
[cloudera@quickstart ~]$ sudo service impala-server status
Impala Server is running                                   [  OK  ]
[cloudera@quickstart ~]$ sudo service impala-server status
Impala Server is running                                   [  OK  ]
[cloudera@quickstart ~]$ sudo service impala-server status
Impala Server is dead and pid file exists                  [FAILED]
[cloudera@quickstart ~]$ sudo service impala-server restart
Stopped Impala Server:                                     [  OK  ]
Started Impala Server (impalad):                           [  OK  ]
[cloudera@quickstart ~]$ sudo service impala-server status
Impala Server is running                                   [  OK  ]
[cloudera@quickstart ~]$ sudo service impala-server status
Impala Server is dead and pid file exists                  [FAILED]
[cloudera@quickstart ~]$ 

I still had the same error

Regards.

(My reply here is the same from my post "Tutorial Exercise 2  Query Structured Data + impala" )

[#6]

Just Adding the Info Log from Impala:

Log file created at: 2015/09/08 17:38:18
Running on machine: quickstart.cloudera
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0908 17:38:18.530884 22029 logging.cc:118] stdout will be logged to this file.
E0908 17:38:18.531857 22029 logging.cc:119] stderr will be logged to this file.
I0908 17:38:18.533502 22029 authentication.cc:993] Internal communication is not authenticated
I0908 17:38:18.533545 22029 authentication.cc:1014] External communication is not authenticated
I0908 17:38:18.533922 22029 init.cc:157] impalad version 2.2.0-cdh5.4.2 RELEASE (build b7f0e80e29971632ae1c422243d56c9ef65b8c5b)
Built on Tue, 19 May 2015 17:00:53 PST
I0908 17:38:18.533946 22029 init.cc:158] Using hostname: quickstart.cloudera
I0908 17:38:18.544509 22029 logging.cc:154] Flags (see also /varz are on debug webserver):
--catalog_service_port=26000
--load_catalog_in_background=false
--num_metadata_loading_threads=16
--sentry_config=
--disable_optimization_passes=false
--dump_ir=false
--opt_module_dir=
--print_llvm_ir_instruction_count=false
--unopt_module_dir=
--abort_on_config_error=true
--be_port=22000
--be_principal=
--compact_catalog_topic=false
--disable_mem_pools=false
--enable_process_lifetime_heap_profiling=false
--heap_profile_dir=
--hostname=quickstart.cloudera
--keytab_file=
--krb5_conf=
--krb5_debug_file=
--mem_limit=80%
--principal=
--redaction_rules_file=
--max_log_files=10
--log_filename=impalad
--redirect_stdout_stderr=true
--data_source_batch_size=1024
--exchg_node_buffer_size_bytes=10485760
--enable_partitioned_aggregation=true
--enable_partitioned_hash_join=true
--enable_probe_side_filtering=true
--skip_lzo_version_check=false
--convert_legacy_hive_parquet_utc_timestamps=false
--max_row_batches=0
--debug_disable_streaming_gzip=false
--enable_phj_probe_side_filtering=true
--enable_ldap_auth=false
--kerberos_reinit_interval=60
--ldap_allow_anonymous_binds=false
--ldap_baseDN=
--ldap_bind_pattern=
--ldap_ca_certificate=
--ldap_domain=
--ldap_manual_config=false
--ldap_passwords_in_clear_ok=false
--ldap_tls=false
--ldap_uri=
--sasl_path=/usr/lib/sasl2:/usr/lib64/sasl2:/usr/local/lib/sasl2:/usr/lib/x86_64-linux-gnu/sasl2
--rpc_cnxn_attempts=10
--rpc_cnxn_retry_interval_ms=2000
--disk_spill_encryption=false
--insert_inherit_permissions=false
--max_free_io_buffers=128
--min_buffer_size=1024
--num_disks=0
--num_s3_io_threads=16
--num_threads_per_disk=0
--read_size=8388608
--catalog_service_host=127.0.0.1
--cgroup_hierarchy_path=
--enable_rm=false
--enable_webserver=true
--llama_addresses=
--llama_callback_port=28000
--llama_host=
--llama_max_request_attempts=5
--llama_port=15000
--llama_registration_timeout_secs=30
--llama_registration_wait_secs=3
--num_hdfs_worker_threads=16
--resource_broker_cnxn_attempts=1
--resource_broker_cnxn_retry_interval_ms=3000
--resource_broker_recv_timeout=0
--resource_broker_send_timeout=0
--staging_cgroup=impala_staging
--state_store_host=127.0.0.1
--state_store_subscriber_port=23000
--use_statestore=true
--local_library_dir=/tmp
--serialize_batch=false
--status_report_interval=5
--num_threads_per_core=3
--use_local_tz_for_unix_timestamp_conversions=false
--scratch_dirs=/tmp
--queue_wait_timeout_ms=60000
--default_pool_max_queued=200
--default_pool_max_requests=200
--default_pool_mem_limit=
--disable_pool_max_requests=false
--disable_pool_mem_limits=false
--fair_scheduler_allocation_path=
--llama_site_path=
--log_mem_usage_interval=0
--authorization_policy_file=
--authorization_policy_provider_class=org.apache.sentry.provider.common.HadoopGroupResourceAuthorizationProvider
--authorized_proxy_user_config=
--load_catalog_at_startup=false
--server_name=
--abort_on_failed_audit_event=true
--abort_on_failed_lineage_event=true
--audit_event_log_dir=
--be_service_threads=64
--beeswax_port=21000
--cancellation_thread_pool_size=5
--default_query_options=
--fe_service_threads=64
--hs2_port=21050
--idle_query_timeout=0
--idle_session_timeout=0
--lineage_event_log_dir=
--local_nodemanager_url=
--log_query_to_file=true
--max_audit_event_log_file_size=5000
--max_lineage_log_file_size=5000
--max_profile_log_file_size=5000
--max_result_cache_size=100000
--profile_log_dir=
--query_log_size=25
--ssl_client_ca_certificate=
--ssl_private_key=
--ssl_server_certificate=
--max_vcore_oversubscription_ratio=2.5
--rm_always_use_defaults=false
--rm_default_cpu_vcores=2
--rm_default_memory=4G
--disable_admission_control=true
--require_username=false
--statestore_subscriber_cnxn_attempts=10
--statestore_subscriber_cnxn_retry_interval_ms=3000
--statestore_subscriber_timeout_seconds=30
--state_store_port=24000
--statestore_heartbeat_frequency_ms=1000
--statestore_heartbeat_tcp_timeout_seconds=3
--statestore_max_missed_heartbeats=10
--statestore_num_heartbeat_threads=10
--statestore_num_update_threads=10
--statestore_update_frequency_ms=2000
--statestore_update_tcp_timeout_seconds=300
--force_lowercase_usernames=false
--num_cores=0
--web_log_bytes=1048576
--non_impala_java_vlog=0
--periodic_counter_update_period_ms=500
--enable_webserver_doc_root=true
--webserver_authentication_domain=
--webserver_certificate_file=
--webserver_doc_root=/usr/lib/impala
--webserver_interface=
--webserver_password_file=
--webserver_port=25000
--flagfile=
--fromenv=
--tryfromenv=
--undefok=
--tab_completion_columns=80
--tab_completion_word=
--help=false
--helpfull=false
--helpmatch=
--helpon=
--helppackage=false
--helpshort=false
--helpxml=false
--version=false
--alsologtoemail=
--alsologtostderr=false
--drop_log_memory=true
--log_backtrace_at=
--log_dir=/var/log/impala
--log_link=
--log_prefix=true
--logbuflevel=0
--logbufsecs=30
--logbufvlevel=1
--logemaillevel=999
--logmailer=/bin/mail
--logtostderr=false
--max_log_size=1800
--minloglevel=0
--stderrthreshold=4
--stop_logging_if_full_disk=false
--symbolize_stacktrace=true
--v=0
--vmodule=
I0908 17:38:18.544984 22029 init.cc:163] Cpu Info:
  Model: Intel(R) Core(TM) i7-4500U CPU @ 1.80GHz
  Cores: 1
  L1 Cache: 32.00 KB
  L2 Cache: 256.00 KB
  L3 Cache: 4.00 MB
  Hardware Supports:
    ssse3
    sse4_1
    sse4_2
    popcnt
I0908 17:38:18.545011 22029 init.cc:164] Disk Info: 
  Num disks 2: 
    sda (rotational=true)
    dm- (rotational=true)

I0908 17:38:18.545029 22029 init.cc:165] Physical Memory: 3.74 GB
I0908 17:38:18.545040 22029 init.cc:166] OS version: Linux version 2.6.32-358.el6.x86_64 (mockbuild@c6b8.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Fri Feb 22 00:31:26 UTC 2013
I0908 17:38:18.545047 22029 init.cc:167] Process ID: 22029
I0908 17:38:24.873245 22029 hbase-table-scanner.cc:156] Detected HBase version >= 0.95.2
I0908 17:38:25.263391 22029 GlogAppender.java:123] Logging initialized. Impala: INFO, All other: INFO
I0908 17:38:25.265431 22029 JniFrontend.java:129] Authorization is 'DISABLED'.
I0908 17:38:25.265497 22029 JniFrontend.java:131] Java Version Info: Java(TM) SE Runtime Environment (1.7.0_67-b01)
I0908 17:38:30.019829 22029 simple-scheduler.cc:103] Admission control is disabled.
I0908 17:38:30.023651 22029 impala-server.cc:1037] Default query options:TQueryOptions {
  01: abort_on_error (bool) = false,
  02: max_errors (i32) = 0,
  03: disable_codegen (bool) = false,
  04: batch_size (i32) = 0,
  05: num_nodes (i32) = 0,
  06: max_scan_range_length (i64) = 0,
  07: num_scanner_threads (i32) = 0,
  08: max_io_buffers (i32) = 0,
  09: allow_unsupported_formats (bool) = false,
  10: default_order_by_limit (i64) = -1,
  11: debug_action (string) = "",
  12: mem_limit (i64) = 0,
  13: abort_on_default_limit_exceeded (bool) = false,
  15: hbase_caching (i32) = 0,
  16: hbase_cache_blocks (bool) = false,
  17: parquet_file_size (i64) = 0,
  18: explain_level (i32) = 1,
  19: sync_ddl (bool) = false,
  23: disable_cached_reads (bool) = false,
  24: disable_outermost_topn (bool) = false,
  25: rm_initial_mem (i64) = 0,
  26: query_timeout_s (i32) = 0,
  28: appx_count_distinct (bool) = false,
  29: disable_unsafe_spills (bool) = false,
  31: exec_single_node_rows_threshold (i32) = 100,
}
I0908 17:38:33.759960 22029 tmp-file-mgr.cc:77] Created the following scratch dirs:/tmp/impala-scratch
I0908 17:38:33.760257 22029 simple-logger.cc:72] Logging to: /var/log/impala/profiles//impala_profile_log_1.1-1441759113760
I0908 17:38:33.762063 22029 impala-server.cc:456] Event logging is disabled
I0908 17:38:33.762099 22029 impala-server.cc:364] Lineage logging is disabled
I0908 17:38:33.887572 22029 impala-server.cc:1652] Impala Beeswax Service listening on 21000
I0908 17:38:33.890080 22029 impala-server.cc:1674] Impala HiveServer2 Service listening on 21050
I0908 17:38:33.892402 22029 impala-server.cc:1689] ImpalaInternalService listening on 22000
I0908 17:38:33.901568 22029 thrift-server.cc:387] ThriftServer 'backend' started on port: 22000
I0908 17:38:33.901605 22029 exec-env.cc:279] Starting global services
I0908 17:38:33.968703 22029 exec-env.cc:366] Using global memory limit: 2.99 GB
I0908 17:38:33.991452 22029 webserver.cc:196] Starting webserver on 0.0.0.0:25000
I0908 17:38:33.991488 22029 webserver.cc:210] Document root: /usr/lib/impala
I0908 17:38:33.991747 22029 webserver.cc:276] Webserver started
I0908 17:38:33.991772 22029 simple-scheduler.cc:182] Starting simple scheduler
I0908 17:38:33.992027 22029 simple-scheduler.cc:230] Simple-scheduler using 127.0.0.1 as IP address
I0908 17:38:33.992056 22029 statestore-subscriber.cc:192] Starting statestore subscriber
I0908 17:38:34.004215 22029 thrift-server.cc:387] ThriftServer 'StatestoreSubscriber' started on port: 23000
I0908 17:38:34.004248 22029 statestore-subscriber.cc:203] Registering with statestore
I0908 17:38:34.004799 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:38:34.004815 22029 thrift-client.cc:61] (Attempt 1 of 10)
I0908 17:38:37.007431 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:38:37.007480 22029 thrift-client.cc:61] (Attempt 2 of 10)
I0908 17:38:40.008561 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:38:40.008649 22029 thrift-client.cc:61] (Attempt 3 of 10)
I0908 17:38:43.009069 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:38:43.009084 22029 thrift-client.cc:61] (Attempt 4 of 10)
I0908 17:38:46.010131 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:38:46.010174 22029 thrift-client.cc:61] (Attempt 5 of 10)
I0908 17:38:49.011075 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:38:49.011121 22029 thrift-client.cc:61] (Attempt 6 of 10)
I0908 17:38:52.011675 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:38:52.011692 22029 thrift-client.cc:61] (Attempt 7 of 10)
I0908 17:38:55.012751 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:38:55.012789 22029 thrift-client.cc:61] (Attempt 8 of 10)
I0908 17:38:58.015012 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:38:58.015051 22029 thrift-client.cc:61] (Attempt 9 of 10)
I0908 17:39:01.031136 22029 thrift-client.cc:55] Unable to connect to 127.0.0.1:24000
I0908 17:39:01.031179 22029 thrift-client.cc:61] (Attempt 10 of 10)
I0908 17:39:01.031245 22029 statestore-subscriber.cc:209] statestore registration unsuccessful: Couldn't open transport for 127.0.0.1:24000 (connect() failed: Connection refused)
E0908 17:39:01.037652 22029 impalad-main.cc:79] Impalad services did not start correctly, exiting.  Error: Couldn't open transport for 127.0.0.1:24000 (connect() failed: Connection refused)
State Store Subscriber did not start up.

here is the Warning Log:

Log file created at: 2015/09/08 17:38:18
Running on machine: quickstart.cloudera
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0908 17:38:18.531857 22029 logging.cc:119] stderr will be logged to this file.
E0908 17:39:01.037652 22029 impalad-main.cc:79] Impalad services did not start correctly, exiting. Error: Couldn't open transport for 127.0.0.1:24000 (connect() failed: Connection refused)
State Store Subscriber did not start up.

and finaly the Error Log:

Log file created at: 2015/09/08 17:38:18
Running on machine: quickstart.cloudera
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0908 17:38:18.531857 22029 logging.cc:119] stderr will be logged to this file.
E0908 17:39:01.037652 22029 impalad-main.cc:79] Impalad services did not start correctly, exiting.  Error: Couldn't open transport for 127.0.0.1:24000 (connect() failed: Connection refused)
State Store Subscriber did not start up.

Thanks For your help

[#7]

Let's stop using multiple threads. I will reply here: http://community.cloudera.com/t5/Apache-Hadoop-Concepts-and/Tutorial-Exercise-2-Query-Structured-Dat...