[root@sandbox-hdp ~]# yum list available | grep ^druid druid.noarch 0.10.1.2.6.4.0-91HDP-2.6-repo-1 druid_2_6_4_0_91.noarch0.10.1.2.6.4.0-91HDP-2.6-repo-1 [root@sandbox-hdp yum.repos.d]# ls -la /etc/yum.repos.d total 64 drwxr-xr-x 1 root root 4096 2月 1 10:49 . drwxr-xr-x 1 root root 4096 4月 5 07:55 .. -rw-r--r-- 1 root root308 4月 7 08:18 ambari-hdp-1.repo -rw-r--r-- 1 root root306 1月 8 08:33 ambari.repo -rw-r--r-- 1 root root 1991 2月 1 09:51 CentOS-Base.repo -rw-r--r-- 1 root root647 2月 1 09:51 CentOS-Debuginfo.repo -rw-r--r-- 1 root root289 2月 1 09:51 CentOS-fasttrack.repo -rw-r--r-- 1 root root630 2月 1 09:51 CentOS-Media.repo -rw-r--r-- 1 root root892 6月16 2016 CentOS-SCLo-scl-rh.repo -rw-r--r-- 1 root root 7989 2月 1 09:51 CentOS-Vault.repo -rw-r--r-- 1 root root957 11月5 2012 epel.repo -rw-r--r-- 1 root root 1056 11月5 2012 epel-testing.repo -rw-r--r-- 1 root root142 2月 1 10:34 hue.repo -rw-r--r-- 1 root root 1033 2月 1 09:52 mysql.repo -rw-r--r-- 1 root root 1550 9月 9 2016 puppetlabs.repo ... View more

I guess Tu Nguyen want to load external Hive table into Spark, right? If so, think about the following code object SparkHiveJdbc extends App { val spark = SparkSession.builder.master("local[2]").appName("SparkHiveJob").getOrCreate val sc = spark.sparkContext val sqlContext = spark.sqlContext val driverName = "org.apache.hive.jdbc.HiveDriver" Class.forName(driverName) val df = spark.read .format("jdbc") .option("url", "jdbc:hive2://localhost:10000/default") .option("dbtable", "clicks_json") .load() df.printSchema() println(df.count()) df.show() } I run the above code and met error reported as root |-- clicks_json.ip: string (nullable = true) |-- clicks_json.timestamp: long (nullable = true) |-- clicks_json.url: string (nullable = true) |-- clicks_json.referrer: string (nullable = true) |-- clicks_json.useragent: string (nullable = true) |-- clicks_json.sessionid: integer (nullable = true) |-- clicks_json.tenantid: string (nullable = true) |-- clicks_json.datestamp: string (nullable = true) 998 Caused by: java.lang.NumberFormatException: For input string: "clicks_json.timestamp" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at org.apache.hive.jdbc.HiveBaseResultSet.getLong(HiveBaseResultSet.java:368) ... 23 more<br> The reason of error I think is spark load header title row as first row when convert ResultSet into internal Row object. Anything wrong here? ... View more

I will be appreciate if you would like to provide the NiFi template of your tutorial, thanks! ... View more

Hi @Andrew Lim, No, I didn't use Distributed Map Cache Client/Server. Yes, I use MySQL 5.6. I also wondered how to replicate MySQL Data to other None MySQL DB, like Hive or HBase, can we still use PutDatabaseRecord processor? How to handle 'Delete' action? Thanks! ... View more

Why there are NULL database name and table name in delete/update/create output of capture processor of mine but normal in begin/commit event type? Anything wrongly configured? {"type":"insert","timestamp":1507709276000,"binlog_filename":"mysql-bin.000658","binlog_position":93481655,"database":null,"table_name":null,"table_id":null,"columns":[{"id":1,"value":10},{"id":2,"value":"mrs"},{"id":3,"value":"erika"},{"id":4,"value":"king"},{"id":5,"value":"1171 depaul dr"},{"id":6,"value":"addison"},{"id":7,"value":"wisconsin"},{"id":8,"value":"50082"},{"id":9,"value":"F"},{"id":10,"value":"erika.king55@example.com"},{"id":11,"value":"goldenbutterfly498"},{"id":12,"value":"chill"},{"id":13,"value":"(635)-117-5424"},{"id":14,"value":"(662)-110-8448"},{"id":15,"value":"122-71-7145"},{"id":16,"value":null},{"id":17,"value":null},{"id":18,"value":"http://api.randomuser.me/portraits/women/52.jpg"},{"id":19,"value":"http://api.randomuser.me/portraits/med/women/52.jpg"},{"id":20,"value":"http://api.randomuser.me/portraits/thumb/women/52.jpg"},{"id":21,"value":"0.6"},{"id":22,"value":"US"}]} {"type":"commit","timestamp":1507689471000,"binlog_filename":"mysql-bin.000657","binlog_position":21750290,"database":"mercury_dev"} ... View more

What about reading external Hive Data by JDBC from Spark SQL? ... View more

And how to monitor the performance of apache nifi? ... View more

Can anyone provide sample apache nifi template? Thanks! ... View more

I found examples like "Change Data Capture (CDC) with Apache NiFi" but it doesn't provide a generic way. The "JsonPathReader" controller service has to parse data table by table. I am seeking a generic way to leverage schema registry to parse data. Btw, how to use hive streaming with PutHDFS? ... View more