Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

Connecting to DataSift HTTPS API using NiFi GetHTTP

Labels:
avatar
author-rank bigspark
Expert Contributor

Created ‎01-12-2016 10:38 AM

Hi all

Is it possible to use GetHttp processor in Nifi to connect to datasift streaming api and receive live streaming data. I have used Gethttp for http api but for https we need ssl context and username and password. Any ideas how to connect to https url with nifi?

18,105 Views
0
1 ACCEPTED SOLUTION

avatar
author-rank apiri author-rank
Rising Star

Created ‎01-12-2016 07:51 PM

You will need to create and configure an SSLContextService for the processor to use so that it can establish trust with the certificate being presented by the DataSift service. curl works because it is tying into the default system truststore for you.

To provide a similar experience as curl on the command line, you will need to configure the truststore properties for your SSL Context Service instance with:

  • Truststore Filename: the cacerts file from your Java installation
    • If $JAVA_HOME Is set on your system, it should help point you in the right direction. If not, the location of cacerts varies depending on environment, but is approximately the following for their respective OS
      • OS X: /Library/Java/JavaVirtualMachines/jdk<version>.jdk/Contents/Home/jre/lib/security/cacerts
      • Windows: C:\Program Files\Java\jdk<version>\jre\lib\security\cacerts
      • Linux: /usr/lib/jvm/java-<version>/jre/lib/security/cacerts -- You can additionally use $(readlink -f $(which java))
  • Truststore Type: JKS
  • Truststore Password: The default password of "changeit" if you are using the default Java keystore

When this controller service is created and enabled, the associated GetHTTP will need to be updated to reference it.

View solution in original post

11,352 Views
0
10 REPLIES 10

avatar
author-rank rishabh_shetty
Explorer

Created ‎11-16-2017 09:18 AM

Hi @Aldrin Piri, @omer alvi ,I am using GetHTTPS to ingest data from a Facebook page and sending it to PutHDFS. The roadblock is that once the data from that page is ingested and I stop the process, that particular url is not hit again when i restart the process. What are the modifications I can make to my url such that the process of data ingestion is a continuous process?

This is a sample of the url I am currently using :

https://graph.facebook.com/v2.11/542678889114683/?fields=name,likes,posts&access_token="my_access_token"&limit=100

3,189 Views
0 Kudos
Announcements
Community Announcements
February 2025 Community Highlights
What's New @ Cloudera
[RELEASED] Cloudera Streaming Analytics - Kubernetes Operato...
What's New @ Cloudera
[RELEASED] Cloudera Streams Messaging - Kubernetes Operator ...
What's New @ Cloudera
3 Benefits of External IDE Connectivity, Now Available in Cl...
What's New @ Cloudera
Performance comparison of Spark3 on YARN with S3 Standard VS...
View More Announcements