Member since 
    
	
		
		
		12-06-2016
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                136
            
            
                Posts
            
        
                12
            
            
                Kudos Received
            
        
                1
            
            
                Solution
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 3533 | 01-18-2018 12:56 PM | 
			
    
	
		
		
		10-03-2018
	
		
		02:34 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi!   Have same   problem,   but   Ranger not installed   [root@serv12 ~]# grep -i Ranger  /var/lib/ambari-agent/data/output-869.txt
2018-10-03 17:49:28,216 - Ranger admin not installed     
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		09-25-2018
	
		
		11:32 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 How to respond to messages related to  GC  Failure  notifications  ? :       Logs for container_e152_1537807360391_1057_01_000081 
  
 
   
  
 
  ResourceManager   RM Home   NodeManager  Tools 
 
  2018-09-25 12:22:15 Starting to run new task attempt: attempt_1537807360391_1057_2_04_000182_0
1.148: [GC (Allocation Failure) [PSYoungGen: 134807K->24906K(204288K)] 134807K->24994K(2068480K), 0.0255463 secs] [Times: user=0.21 sys=0.02, real=0.03 secs] 
1.411: [GC (Metadata GC Threshold) [PSYoungGen: 74769K->22331K(379904K)] 74857K->22427K(2244096K), 0.0233332 secs] [Times: user=0.12 sys=0.03, real=0.02 secs] 
1.435: [Full GC (Metadata GC Threshold) [PSYoungGen: 22331K->0K(379904K)] [ParOldGen: 96K->21509K(264192K)] 22427K->21509K(644096K), [Metaspace: 20929K->20929K(1069056K)], 0.0289288 secs] [Times: user=0.32 sys=0.03, real=0.03 secs] 
6.755: [GC (Allocation Failure) [PSYoungGen: 237172K->28660K(379904K)] 258681K->95006K(644096K), 0.0204362 secs] [Times: user=0.08 sys=0.24, real=0.02 secs] 
15.663: [GC (Allocation Failure) [PSYoungGen: 213057K->28659K(547328K)] 279403K->193109K(811520K), 0.0390040 secs] [Times: user=0.14 sys=0.50, real=0.04 secs] 
15.702: [Full GC (Ergonomics) [PSYoungGen: 28659K->0K(547328K)] [ParOldGen: 164449K->172137K(616960K)] 193109K->172137K(1164288K), [Metaspace: 32575K->32575K(1079296K)], 0.0628115 secs] [Times: user=0.96 sys=0.16, real=0.06 secs] 
33.575: [GC (Allocation Failure) [PSYoungGen: 280808K->28661K(569344K)] 5743954K->5559068K(6478848K), 0.0278063 secs] [Times: user=0.21 sys=0.32, real=0.03 secs] 
34.210: [GC (Allocation Failure) [PSYoungGen: 490957K->82416K(791552K)] 6021364K->5612831K(6701056K), 0.0320689 secs] [Times: user=0.57 sys=0.16, real=0.03 secs] 
34.243: [Full GC (Ergonomics) [PSYoungGen: 82416K->0K(791552K)] [ParOldGen: 5530414K->5603149K(6443008K)] 5612831K->5603149K(7234560K), [Metaspace: 33968K->33968K(1079296K)], 0.9965374 secs] [Times: user=26.00 sys=0.29, real=0.99 secs] 
36.154: [GC (Allocation Failure) [PSYoungGen: 459509K->36576K(683520K)] 6062659K->5639733K(7126528K), 0.0232837 secs] [Times: user=0.69 sys=0.02, real=0.02 secs] 
36.746: [GC (Allocation Failure) [PSYoungGen: 466467K->13984K(783360K)] 6069625K->5617149K(7226368K), 0.0202414 secs] [Times: user=0.54 sys=0.01, real=0.02 secs] 
37.123: [GC (Allocation Failure) [PSYoungGen: 321932K->14048K(784896K)] 5925097K->5617213K(7227904K), 0.0258958 secs] [Times: user=0.69 sys=0.01, real=0.02 secs] 
37.723: [GC (Allocation Failure) [PSYoungGen: 566423K->15086K(787968K)] 6169588K->5618252K(7230976K), 0.0238336 secs] [Times: user=0.58 sys=0.00, real=0.02 secs] 
38.036: [GC (Allocation Failure) [PSYoungGen: 259899K->14478K(790016K)] 5863064K->5617644K(7233024K), 0.0251177 secs] [Times: user=0.55 sys=0.01, real=0.03 secs] 
38.293: [GC (Allocation Failure) [PSYoungGen: 222330K->1806K(792576K)] 5825496K->5617860K(7235584K), 0.0262013 secs] [Times: user=0.57 sys=0.02, real=0.02 secs] 
38.624: [GC (Allocation Failure) [PSYoungGen: 270228K->1248K(793088K)] 5886282K->5617856K(7236096K), 0.0277400 secs] [Times: user=0.56 sys=0.00, real=0.03 secs]      
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache Hive
			
    
	
		
		
		07-31-2018
	
		
		10:39 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi!   Try One of   this changes:   
   
   Zeppelin  
   NIFI
  ExecuteSparkInteractive 
      
  
  
   preds =
  spark.sql(‘select * from sandbox.CHURN_PRP_D_PREDS_S ’) 
   preds =
  spark.sql("select
  * from andbox.CHURN_PRP_D_PREDS_S") 
      
  
  
   dir = ‘
  /user/CHURN_PRP_D’ 
   dir =
  "hdfs:/user/CHURN_PRP_D" 
    
  
  
   date_df =
  spark.sql("select to_date('{}') as score_date".format(date_calc)) 
   ttt =
  spark.createDataFrame([(date_calc,)], ["t"])
   date_df =
  ttt.select(to_date(ttt.t))  
    
  
  
   sqlContext.registerDataFrameAsTable(train,
  "train") 
   train.registerTempTable("train") 
    
    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-11-2018
	
		
		01:45 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi!  one more question,   how control  count  "livy-session-""  ? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-11-2018
	
		
		11:46 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Problem was in   "sqlContext.registerDataFrameAsTable"    What investigate similar (HTTP) errors ? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-06-2018
	
		
		03:48 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Do you have example run Zeppelin REST API  ? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-06-2018
	
		
		02:00 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache NiFi
- 
						
							
		
			Apache Zeppelin
			
    
	
		
		
		06-06-2018
	
		
		12:38 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 CODE : executesparkinteractive-code.txt
ERROR :
2018-06-06 15:32:13,566 ERROR [Timer-Driven Process Thread-5] o.a.n.p.livy.ExecuteSparkInteractive ExecuteSparkInteractive[id=aeb74038-5333-13d2-0000-00001ea7e32e] ExecuteSparkInteractive[i
d=aeb74038-5333-13d2-0000-00001ea7e32e] failed to process session due to java.lang.RuntimeException: Failed : HTTP error code : 400 : Bad Request: {}
java.lang.RuntimeException: Failed : HTTP error code : 400 : Bad Request
        at org.apache.nifi.processors.livy.ExecuteSparkInteractive.readJSONObjectFromUrlPOST(ExecuteSparkInteractive.java:282)
        at org.apache.nifi.processors.livy.ExecuteSparkInteractive.submitAndHandleJob(ExecuteSparkInteractive.java:234)
        at org.apache.nifi.processors.livy.ExecuteSparkInteractive.onTrigger(ExecuteSparkInteractive.java:197)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
     
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache NiFi
- 
						
							
		
			Apache Spark
			
    
	
		
		
		05-31-2018
	
		
		08:03 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Thank you!    But what  to do with processor ,   for example FetchFTP ,   after import change (version control)   from DEV to PROD  property:  Password "No value set"  Can add  pasword to variables (on processor group),   but this is not secured. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
        













