1973
Posts
1225
Kudos Received
124
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1944 | 04-03-2024 06:39 AM | |
| 3054 | 01-12-2024 08:19 AM | |
| 1670 | 12-07-2023 01:49 PM | |
| 2447 | 08-02-2023 07:30 AM | |
| 3406 | 03-29-2023 01:22 PM |
12-15-2016
10:49 PM
Strange error, but fixed due to disconnected network. So connection timeout error.
... View more
12-15-2016
09:52 PM
I have code I have been using for a while and has worked. But today my Java code calling Phoenix, ran into this: : Initiating client connection, connectString=tspann:2181 sessionTimeout=90000 watcher=hconnection-0x4550bb580x0, quorum=tspann:2181, baseZNode=/hbase-unsecure
2016-12-15 16:49:44.898 INFO 11624 --- [42.140.21:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server 24221/242.21:2181. Will not attempt to authenticate using SASL (unknown error)
... View more
Labels:
- Labels:
-
Apache HBase
12-15-2016
04:19 PM
2 Kudos
I have a Phoenix PutSQL that worked in the past, but not today. From the error box, not much help. Thought maybe Hbase or Phoenix was done. Turned out one field was missing. How to investigate this issue?
Digging through the logs I quickly found it:
Caused by: java.sql.SQLException: ERROR 218 (23018): Constraint violation. DRONEDATA1.DATEKEY may not be null
That would be nice in the error listed in NiFi. If something seems like an odd error, dig into the logs for more detailed descriptions. Also the Summary Screen is pretty awesome and will let you find a lot of information for debugging and fixing problems. The Connections tab will let you sort by Queue Size and you can see if things are backed up. If you click the system diagnostics link you will see two tabs of useful information. The first has memory usage and garbage collection. The System tab has your usage for FlowFile and Content Repositories, processors and process load. For more details, check out the following references. Reference: https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#flowfile-repository https://nifi.apache.org/docs.html https://community.hortonworks.com/questions/71001/debugging-nifi-flow.html https://community.hortonworks.com/questions/69004/nifi-monitoring-processor-and-nifi-service.html#answer-69684
... View more
Labels:
12-14-2016
08:48 PM
1 Kudo
see: https://github.com/databricks/spark-xml Github has examples like import org.apache.spark.sql.SQLContext
val sqlContext = new SQLContext(sc)
val df = sqlContext.read
.format("com.databricks.spark.xml")
.option("rowTag", "book")
.load("books.xml")
val selectedData = df.select("author", "_id")
selectedData.write
.format("com.databricks.spark.xml")
.option("rootTag", "books")
.option("rowTag", "book") .save("newbooks.xml") Spark compiled with Scala 2.10 $SPARK_HOME/bin/spark-shell --packages com.databricks:spark-xml_2.10:0.4.1
... View more
12-14-2016
06:20 PM
Is LDAP working? Do you need to restart anything?
... View more
12-14-2016
06:11 PM
1 Kudo
In theory it would be one per client, not sure if you should try 180 or 1800. Both are pretty high though. You should set it to the # of CPU Cores available on all the region servers. Depending on data size, perhaps you need more nodes in the cluster? How big is this data? What version of HBase? What version of Hadoop? What JDK version? How much RAM on the nodes? How big is each region? Did you restart after changing the parameter? Are you looking at the HBase Master, JMX, Logs, Stack Trace and other diagnostics provided by HBase. Also Ambari and other monitoring tools you have may help. Anything in the logs? Do you need the end point coprocessor? Can you scan the data? Read with Spark? Read with NiFi? Or Read through Phoenix as a SQL query? Read: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_hadoop-high-availability/content/config-ha-reads-hbase.html See: https://svn.apache.org/repos/asf/hbase/hbase.apache.org/trunk/0.94/performance.html# http://hbase.apache.org/0.94/book/important_configurations.html#recommended_configurations http://www.slideshare.net/lhofhansl/h-base-tuninghbasecon2015ok http://hbase.apache.org/0.94/book/important_configurations.html https://community.hortonworks.com/articles/46220/phoenix-hbase-tuning-quick-hits.html
... View more
12-14-2016
03:33 AM
So the solution is go to Summary, then Connections. Sort by Queue / Size. Then delete!
... View more
12-14-2016
03:26 AM
"flowFilesQueued":241658 I have a lot of flowFilesqueued up. I am not sure where or how to find them. I want to delete them. I have a hundred of process groups. {"controllerStatus":{"activeThreadCount":0,"queued":"241,658 / 2.99 GB","flowFilesQueued":241658,"bytesQueued":3209622825,"runningCount":119,"stoppedCount":429,"invalidCount":70,"disabledCount":0,"activeRemotePortCount":0,"inactiveRemotePortCount":0}}
Is there something in the UI or API to find and purge?
... View more
Labels:
- Labels:
-
Apache NiFi
12-13-2016
06:09 PM
https://orc.apache.org/docs/core-java.html https://codecheese.wordpress.com/2016/04/20/writing-an-orc-file-using-java/ https://github.com/apache/orc/tree/master/examples
... View more
12-13-2016
06:09 PM
1 Kudo
https://orc.apache.org/docs/core-java.html https://codecheese.wordpress.com/2016/04/20/writing-an-orc-file-using-java/ https://github.com/apache/orc/tree/master/examples
... View more