Member since
10-14-2016
12
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
601 | 11-28-2016 01:47 PM |
10-11-2019
03:34 AM
Hi All,
I am new to AWS stack.
What would be my data pipeline if i want to move data from any RDBMS to AWS S3 ?
... View more
Labels:
05-15-2018
09:33 AM
Hi All, I am developing an OCR application that get images continuously from one folder and there is one intermediate python code that convert the image into text and it will store those text content in DB. Now how can I integrate the Nifi with my Python code(Execute Process).
... View more
Labels:
10-20-2017
10:15 AM
What is the logic behind this
... View more
10-20-2017
08:41 AM
Could you try running the query after setting property tez.runtime.shuffle.ssl.enable=false and let us know the result?
... View more
10-20-2017
06:52 AM
Try spark-submit --master <master-ip>:<spark-port> to submit the job.
... View more
10-20-2017
06:34 AM
Try this code from pyspark import SparkConf, SparkContext
from pyspark.sql import SQLContext
conf1 = SparkConf().setAppName('sort_desc')
sc1 = SparkContext(conf=conf1)
sql_context = SQLContext(sc1)
csv_file_path = 'emp.csv'
employee_rdd = sc1.textFile(csv_file_path).map(lambda line: line.split(','))
print(type(employee_rdd))
employee_rdd_sorted = employee_rdd.sortByKey(ascending= False)
employee_df = employee_rdd.toDF(['dept','ctc'])
employee_df_sorted = employee_rdd_sorted.toDF(['dept','ctc'])
... View more
03-29-2017
06:21 AM
Hi , I have one query which contain many join. Now I want to create a Dataframe or Dataset from the query (not from a single table) in scala
... View more
Labels:
03-16-2017
01:54 PM
Hi All, In mynifi flow i have two processor one is GetFTP and PutS3Object . consider i have one file in FTP a.txt .after the data get into the S3 the a.txt's timestamp is 12:00:00 in S3 after sometime again one file 'b.txt' is put into ftp now the S3 have two files as below but the timestamp in the S3 is changed for both a.txt and b.txt a.txt 12:01:00 b.txt 12:01:00
... View more
Labels:
11-28-2016
01:50 PM
I have data like this 1, A B C,xxx 2, A B C,yyy i want my output will be look like 1,A,xxx 1,B,xxx 1,C,xxx 2,A,xxx 2,B,xxx 2,C,xxx
... View more
Labels:
11-28-2016
01:47 PM
1 Kudo
Hi Raf Mohammed if you want to do some real-time analysis on twitter do not go with hive or some traditional reporting tools. use flume for pulling data and store data in Elasticsearch and do visualization in Kibana. if you want to do some real-time analytics such as Sentiment Analysis try Flume+Spark Streaming+Elasticsearch+Kibana @Raf Mohammed
... View more
10-05-2016
04:53 PM
Hi All, I have a large amount of data files in Windows now i want to move all files into the HDFS with maintaining same structure what windows have
... View more
Labels: