Member since 
    
	
		
		
		06-09-2016
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                529
            
            
                Posts
            
        
                129
            
            
                Kudos Received
            
        
                104
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 1730 | 09-11-2019 10:19 AM | |
| 9317 | 11-26-2018 07:04 PM | |
| 2475 | 11-14-2018 12:10 PM | |
| 5304 | 11-14-2018 12:09 PM | |
| 3139 | 11-12-2018 01:19 PM | 
			
    
	
		
		
		08-13-2018
	
		
		06:48 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Harun
 Zengin please let me know if the above has helped you? Thanks. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-25-2018
	
		
		02:10 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thanks a lot @Felix Albani...you solved my issue. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-20-2018
	
		
		03:03 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thanks for sharing the detailed information and will try to update the spark-env to use default mode as yarn. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-19-2018
	
		
		04:03 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @David Pocivalnik yes, that is correct! Please if you are satifsfied with the answer remember to login and mark as accepted. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		05-21-2019
	
		
		01:46 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I have NIFI API Link https://nifi.apache.org/docs/nifi-docs/rest-api/index.html   But dont understand how to use it? How do you know when to stop NIFI Job.. i mean how do you know all processors have processed the data? Could you please share the documentation if any about it. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-09-2018
	
		
		01:57 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I'm not sure if it matters; but it seems there is a 1.5 hour difference between the 2 ps -ef -s ; it might be possible that somehow one of the hs2 instances have not shut down cleanly earlier;  I would recommend to try shutting both hs2 down; and checking that they are both stopped by hand ; check that the configs are updated - and start both of them... 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-04-2018
	
		
		08:08 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @Zeev Grinberg Here is the link for latest odbc driver for mac:  https://s3.amazonaws.com/public-repo-1.hortonworks.com/HDP/hive-odbc/2.1.12.1017/OSX/hive-odbc-native.dmg  You can find all drivers under HDP Add-Ons in the following URL:  https://hortonworks.com/downloads/  HTH  *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-23-2019
	
		
		06:00 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 for example, in one of my DAG, all that those task do is Sort WithinPartition (so no shuffle) still it spills data on disk because partition size is huge and spark resort to ExternalMergeSort. As a result, I have a high Shuffle Spill (memor) and also some Shuffle Spill(Disk). There is no shuffle here. but on the other hand you can argue that Sorting process moves data in order to sort so it's kind of internal shuffle 🙂 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-03-2018
	
		
		11:39 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Vladislav Shcherbakov   I think using InvokeScriptedProcessor may be a good option on this case. Please take a look here:  http://funnifi.blogspot.com/2016/02/invokescriptedprocessor-hello-world.html  Also there is a similar article that you could review here:  https://community.hortonworks.com/articles/193822/parsing-web-pages-for-images-with-apache-nifi.html  HTH  *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-20-2018
	
		
		03:56 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 This article with cover step by step how to configure HDP Search Solr with Ranger Plugin  Step 1  Download and install the 2.2.9+ mpack - FYI: Previous mpack versions don't support integration of HDP Search Solr with Ranger. Mpack 2.2.9 includes configurable section for the solr-security on ambari which allows to add the authorization information.  wget 'http://public-repo-1.hortonworks.com/HDP-SOLR/hdp-solr-ambari-mp/solr-service-mpack-2.2.9.tar.gz'; -O /tmp/solr-service-mpack-2.2.9.tar.gz
ambari-server install-mpack --mpack=/tmp/solr-service-mpack-2.2.9.tar.gz  Step 2  On HDP solr host run  yum install ranger-solr-plugin.noarch
cd /usr/hdp/2.6.2.0-205/ranger-solr-plugin  Edit install.properties and make sure the following settings at least are properly configured:  POLICY_MGR_URL=http://<ranger-host>:6080 
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar  Edit solr-plugin-install.properties and set correct value for install dir:  COMPONENT_INSTALL_DIR_NAME=/opt/lucidworks-hdpsearch/solr/server   Next source the environment and enable the plugin:  source /etc/hadoop/hadoop-env.sh 
./enable-solr-plugin.sh  Step 3  Update security znode with ranger authorization class  kinit -kt solr.service.keytab solr/<host>@REALM.COM 
/opt/lucidworks-hdpsearch/solr/server/scripts/cloud-scripts/zkcli.sh -z '<zookeeper>:2181' -cmd put /solr/security.json '{"authentication":{"class": "org.apache.solr.security.KerberosPlugin"},"authorization":{"class": "org.apache.ranger.authorization.solr.authorizer.RangerSolrAuthorizer"}}'  Also on Ambari->Solr-Config->Advance solr-security set:  {
"authentication":{"class": "org.apache.solr.security.KerberosPlugin"},
"authorization":{"class": "org.apache.ranger.authorization.solr.authorizer.RangerSolrAuthorizer"}
}   Save and restart, on the operation start output you should see  - call['/opt/lucidworks-hdpsearch/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost <zookeeper1>:2181,<zookeeper2>:2181,<zookeeper3>:2181 -cmd get /solr/security.json'] {'timeout': 60, 'env': {'JAVA_HOME': u'/usr/jdk64/jdk1.8.0_112'}}
- call returned (0, '{\"authentication\":{\"class\": \"org.apache.solr.security.KerberosPlugin\"},\"authorization\":{\"class\": \"org.apache.ranger.authorization.solr.authorizer.RangerSolrAuthorizer\"}}')
- Solr Security Json was found, it will not be overridden  Step 4  Fix clustername for solr plugin  cd /opt/lucidworks-hdpsearch/solr/server/solr-webapp/webapp/WEB-INF/classes/   Edit ranger-solr-audit.xml and add following property    <property>
     <name>ranger.plugin.solr.ambari.cluster.name</name>
      <value>YOUR_CLUSTER_NAME</value>
    </property>  Restart solr for changes to reflect.  Step 5  Open Ranger Admin UI and edit solr repository add New Configurations  tag.download.auth.users = solr
policy.download.auth.users = solr
ambari.service.check.user = ambari-qa 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
	
					
			
		
	
	
	
	
				
		
	
	
 
        













