About khouloud_lander

khouloud_lander · ‎05-28-2018

Thank you @Nishant Bangarwa! "No module psycopg2" solved! Though I am getting another error now: Error: Fatal: no pg_hba.conf entry for host "x.x.x.x", user "x", database "x", SSL off. Plus, I am not sure about the user & password for my Postgres database

khouloud_lander · ‎05-25-2018

I am trying to add a postgres database to superset. This is my connection URI: postgresql://username:password@localhost/mydatabase. I keep getting the following error while trying to test connection: No module named psycopg2 ! I made sure I installed this module using pip install psycopg2 in shell but it's still not working!

khouloud_lander · ‎05-21-2018

I was trying to run this line : val fraud = sc.textFile("hdfs://sandbox-hdp.hortonworks.com:8020/tmp/fraud.csv") but then I kept getting this error (although it worked on spark shell!) java.io.IOException: Failed to create local dir in /tmp/blockmgr-c40d2915-3861-4bbe-8e1c-5eca677c552e/0e. at org.apache.spark.storage.DiskBlockManager.getFile(DiskBlockManager.scala:70) at org.apache.spark.storage.DiskStore.remove(DiskStore.scala:135) at org.apache.spark.storage.BlockManager.removeBlockInternal(BlockManager.scala:1457) at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:991) at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1029) at org.apache.spark.storage.BlockManager.putIterator(BlockManager.scala:792) at org.apache.spark.storage.BlockManager.putSingle(BlockManager.scala:1350) at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:122) at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:88) at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34) at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:56) at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1488) at org.apache.spark.SparkContext$$anonfun$hadoopFile$1.apply(SparkContext.scala:1037) at org.apache.spark.SparkContext$$anonfun$hadoopFile$1.apply(SparkContext.scala:1029) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) at org.apache.spark.SparkContext.withScope(SparkContext.scala:701) at org.apache.spark.SparkContext.hadoopFile(SparkContext.scala:1029) at org.apache.spark.SparkContext$$anonfun$textFile$1.apply(SparkContext.scala:832) at org.apache.spark.SparkContext$$anonfun$textFile$1.apply(SparkContext.scala:830) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) at org.apache.spark.SparkContext.withScope(SparkContext.scala:701) at org.apache.spark.SparkContext.textFile(SparkContext.scala:830) ... 48 elided

khouloud_lander · ‎05-08-2018

3. The output is:

khouloud_lander · ‎05-08-2018

Thank you, @Felix Albani! Actually, besides that error message, anything I try on zeppelin with pyspark, keeps pending and never run. So, I've tried everything you mentioned: 1. Some of the info I got: INFO [2018-05-05 05:59:31,823] ({pool-1-thread-1} PySparkInterpreter.java[interrupt]:421) - Sending SIGINT signal to PID : 7627 WARN [2018-05-05 06:39:08,446] ({ResponseProcessor for block BP-32082187-172.17.0.2-1517480669419:blk_1073742956_2146} DFSOutputStream.java[run]:958) - DFSOutputStream ResponseProcessor exception for block BP- 32082187-172.17.0.2-1517480669419:blk_1073742956_2146 java.io.EOFException: Premature EOF: no length prefix available at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2468) at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:244) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:849) 2. I've tried restarting it but it doesn't work!

khouloud_lander · ‎05-08-2018

khouloud_lander · ‎04-26-2018

I solved the issue! I found out I needed the right delimiter in my csv file (,)! Plus I think Phoenix is case sensitive (when I wrote the create statement using capital letters for the table's name, it worked!

khouloud_lander · ‎04-26-2018

p.s. the cells format is "standard" create table "test" ("nom" varchar, "age" integer, "city" varchar, constraint pk primary key("nom"));

khouloud_lander · ‎04-26-2018

here I am just using a small table to test bulk loading. I keep getting this error message (attached)! Is the warning message "unable to load native-hadoop library" the reason?

Online	Offline
Last Visited	‎11-07-2018 09:03 AM

Date Registered	‎04-17-2018 10:54 AM
Last Visited	‎11-07-2018 09:03 AM
Total Messages Posted	9

Re: [Superset]: No module named psycopg2

[Superset]: No module named psycopg2

java.io.IOException: Failed to create local dir

Re: [Zeppelin]: pyspark is not responding!

Re: [Zeppelin]: pyspark is not responding!

[Zeppelin]: pyspark is not responding!

Re: Unable to upsert data from a csv file into HBa...

Re: Unable to upsert data from a csv file into HBa...

Unable to upsert data from a csv file into HBase t...