Support Questions
Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Advanced Search

How to identify at cluster side that spark code is using datasets or dataframes

How to identify at cluster side that spark code is using datasets or dataframes

satyap
Explorer

Created on ‎02-10-2020 03:33 AM - last edited on ‎02-10-2020 04:49 AM by Community Manager VidyaSargur Community Manager

Hi Team,

We are using HDP 3.1.0 and spark2.

Is there anyway to identify at cluster side about the spark job that particular job is using datasets or dataframe?

 

Regards,

Satya

Reply
87 Views
0 Kudos
Don't have an account?
Register
Announcements
Product Announcements
[ANNOUNCE] New Cloudera JDBC 2.6.20 Driver for Apache Impala Released
Product Announcements
Transition to private repositories for CDH, HDP and HDF
Product Announcements
[ANNOUNCE] New Applied ML Research from Cloudera Fast Forward: Few-Shot Text Classification
Product Announcements
[ANNOUNCE] New JDBC 2.6.13 Driver for Apache Hive Released
Product Announcements
[ANNOUNCE] Refreshed Research from Cloudera Fast Forward: Semantic Image Search and Federated Learning
View More Announcements