Welcome to the Cloudera Community
Your Enterprise Data Cloud Community. With over 68,400 members and 18,400 solutions, you've come to the right place!
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- Cloudera Community
- :
- Search
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
- « Previous
- Next »
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
60,000 results
Sorted by:
This article describes various "recipes" on how to accomplish certain tasks with the NiFi processor ExecuteScript, with examples given in Groovy, Jython, Javascript (Nashorn), and JRuby. Recipes in t...
- Find more articles tagged with:
- Data Ingestion & Streaming
- executescript
- How-ToTutorial
- NiFi
Labels:
Show results in comments (8)
-
@mburgess this is just awesome! Thank you so much for this. I am trying to call a rest API vi...
-
You can call a REST API from Nashorn/Javascript in ExecuteScript, but since Nashorn doesn't have ac...
-
Awesome! just tested out... Works like a charm! Thank you so much... just in case if you have any r...
-
Extremely usefull examples. However, I'm not sure how to receive one flowfile but transfer several ...
-
@Matt Burgess I just wanted to clarify your 2 recipes for getting incoming flowfiles. The first men...
-
Thank you! Also just for interest's sake and hope you don't mind answering here, but when will you ...
-
Thanks, Matt. This is very informative.
-
Yes we currently build with JDK 8 so we get Nashorn "for free". This will be the case when building...
Interacting with Hadoop HDFS using Python codes This post will go through the following:
Introducing python “subprocess” module Running HDFS commands with Python Examples of HDFS commands fro...
- Find more articles tagged with:
- code
- coding
- Data Processing
- HDFS
- How-ToTutorial
- Linux
- python
Labels:
Show results in comments (4)
-
Hi All. Here is all steps for doing same!!! Link :- https://www.oreilly.com/library/view/hadoop-...
-
It was an excellent article on interacting of Hadoop HDFS using Python to hear from you which is v...
-
Thanks for the comment Michael. I wrote these commands for hdp environments using standard python 2...
-
In case you want to leverage structured results from HDFS commands or further reduce latency / over...
Objective This tutorial will walk you through the process of using the PyHive Python module from Dropbox to query HiveServer2. You can read more about PyHive here: PyHive Prerequisites
You sho...
- Find more articles tagged with:
- Hive
- How-ToTutorial
- python
- Sandbox
- Sandbox & Learning
- tableau
Labels:
Show results in comments (5)
-
Hi @TimothySpann , Could you please let me know if you was able to resolve issue, since I ...
-
Hive doesn't use thrift anymore. new Python APIs work fine. https://github.com...
-
Can I use pyhive to connect to Hive using Hive JDBC string instead of a single hostname? The follow...
-
Login with username
-
Great article...BUT...this only seems to work for small queries "show databases", "show tables", "S...
Synopsis This tutorial will demonstrate using Spark for data processing operations on a large set of data consisting of pipe delimited text files. Delimited text files are a common format seen in Da...
- Find more articles tagged with:
- Data Science & Advanced Analytics
- DataFrame
- How-ToTutorial
- pyspark
- rdd
- SparkSQL
Labels:
Show results in comments (4)
-
Is the input dataset available somewhere?
-
Is this still valid? I mean there are many improvements on spark-sql & catalyst engine since sp...
-
can we say this difference is only due to the conversion from RDD to dataframe ? because as per ap...
-
Very interesting, but I think it'd have been as more even comparision if you'd have used SparkSQL c...
A temporary table is a convenient way for an application to automatically manage intermediate data generated during a complex query. Rather than manually deleting tables needed only as temporary data...
- Find more articles tagged with:
- hadoop
- HDFS
- Hive
- how-to-tutorial
- Sandbox & Learning
Labels:
Objective
This tutorial walks you through the process of using the CaptureChangeMySQL, EnforceOrder and PutDatabaseRecord processors in Apache NiFi to replicate a database using MySQL event logs. ...
- Find more articles tagged with:
- apache-nifi
- cdc
- Data Ingestion & Streaming
- How-ToTutorial
- MySQL
Labels:
Show results in comments (8)
-
Hi Alim, I just started to repeat our work and found the following error. My testin...
-
Can we do similar with postgres CDC, MS SQL SERVER CDC using CaptureChangeMySQL processor ?
-
@Andrew Lim ,I am getting the below log from the path C:\Users\XXXXX\Desktop\Work_past\nifi-1.9....
-
I'm following this article, everything works fine but I need to fetch data from large database, Eve...
-
NiFi doesn't currently support MS SQL, but it appears there is a NiFi contributor who has it in the...
-
We need to read sql server transnational logs for CDC instead of hitting DB using jdbc . Is that po...
-
@Ramkrishna Utpat, That's great to hear! Glad you were able to get the tutorial working.
-
Hello @Andrew Lim, Thanks for the solution and the documentation you have created. It is really g...
05-21-2016
08:35 AM
Guru
Created on 05-21-2016 08:35 AM
The key thing to remember is that in Spark RDD/DF are immutable. So once created you can not change them. However there are many situation where you want the column type to be different. E.g By def...
- Find more articles tagged with:
- Data Science & Advanced Analytics
- Issue Resolution
- issue-resolution
- Spark
- spark-sql
- SparkSQL
- zeppelin
Labels:
Show results in comments (3)
-
I am also looking for something like this. I need to convert all my date datatypes to varchar in a ...
-
Nice article. I am looking for a solution, Instead of specifying a each column separately. is there...
-
Hi, Its not working for me ..Unable to cast the column data type.Here is my code.I am getting only...
08-16-2016
09:56 PM
19 Kudos
Super Guru
Created on
08-16-2016
09:56 PM
- edited on
02-12-2020
06:33 AM
by
SumitraMenon
NiFi is most effectively used as an "always-on" system, meaning that the data flows are often always operational (running). Doing batch processing is a more difficult task and usually requires some u...
- Find more articles tagged with:
- Data Ingestion & Streaming
- How-ToTutorial
- incremental
- NiFi
- nifi-processor
- querydatabasetable
- rdbms
Labels:
Show results in comments (8)
-
Hi Matt, Is there a suggested way to perform incremental fetching when the columns increas...
-
@mburgess , Could you please suggest how I can achieve the same with Cassandra. I tried to us...
-
Hi @mattburgess i am using the same processor for fetching incremental data from relational tables....
-
Thanks, Matt. Very informative for beginners like me.
-
You may use ExecuteSQL processor for that. You can write custom SQL query with that processor.
-
@Matt Burgess Really helpful!! But, I have a question By using this flow How we will get the upda...
-
Hi, is there any way to use this processor to check for a change in a specific record that may not ...
-
Thanks Matt , one more good article from you. If my nifi went down or processor stopped somehow. A...
Goal: This article provides script to extract the DDLs for all tables and partition in a given hive database. This scripts comes handy when migrating/creating Hive Tables from one cluster to another...
- Find more articles tagged with:
- Hive
- How-ToTutorial
- solutions
Labels:
How to Submit Spark Application through Livy REST API Apache Livy supports using REST APIs to submit Spark applications, it is quite similar to use “spark-submit” in vanilla Spark. In this article w...
- Find more articles tagged with:
- Data Science & Advanced Analytics
- How-ToTutorial
- livy
- Spark
- zeppelin
Labels:
Show results in comments (4)
-
@sshao am submitting spark job through livy , below are my conf parameters data = {'kind': 'pyspar...
-
hi i have exception java.io.FileNotFoundException 2019-06-01 00:43:19,160 WARN util.NativeCodeLoad...
-
I am newer in BigData, i have tried to call spark jobs with apache Livy . With submit command line ...
-
This is great thanks! Also, before too long there should be a NiFi processor and controller service...
