About asirna

asirna · ‎10-10-2018

1000 is for spark. You can set common.max_count at a global level. You should not have negative results if you increase the limit. But if your data size if very huge then you may need to tweak the above mentioned params accordingly.

asirna · ‎10-10-2018

@Junfeng Chen, There will be interpreter level properties. For ex: spark has (zeppelin.spark.maxResult) whose default value is 1000. So even if there are more than 1000 rows it will just fetch 1000 rows. If you need more rows, you can increase the limit. You may need to tweak ( zeppelin.interpreter.output.limit, zeppelin.websocket.max.text.message.size, ZEPPELIN_MEM, ZEPPELIN_INTP_MEM ) these properties according to your output size. Refer this link for more info on all the properties https://zeppelin.apache.org/docs/0.7.2/install/configuration.html

asirna · ‎10-10-2018

@Junfeng Chen, Yes. Zeppelin notebook results are stored in JSON format HDFS (from HDP 2.6) and on native filesystem prior to this version. It is stored in HDFS , so it will not be a problem even if the size is huge. You can check the output here Native FS path : /usr/hdp/current/zeppelin-server/notebook/{notebook-id}/note.json HDFS path: /user/zeppelin/notebook/{notebook-id}/note.json You can check for results key in the note.json. . If this helps , please take a moment to login and "Accept" the answer

asirna · ‎10-09-2018

@Lakshmi Prathyusha, I'm not sure of how to do this in Scala. I guess you may have similar date time functions in Scala as well. You can apply this logic in Scala.

asirna · ‎10-09-2018

@Lakshmi Prathyusha, You can write a simple python snippet like below to read the subfolders. I have put a print statement in the code, but you can replace it some subprocess command to run it. from datetime import date, timedelta from dateutil.relativedelta import relativedelta today = date.today() two_months_back = today - relativedelta(months=2) delta = today - two_months_back for i in range(delta.days + 1): dt = str(two_months_back + timedelta(i)).replace("-", "") print "hdfs dfs -ls s3a://bucket/Folder/1005/SoB/%s" % dt . -Aditya

asirna · ‎10-08-2018

@Madhura Mhatre, 1) If you have multiple hiveservers, you will have multiple znodes(of the above mentioned forma) under /hiveserver2. When you stop the hiveserver2, respective znode will be deleted from zookeeper. If you stop all the hiveserver instances, then /hiveserver2 will have no znodes under it. You should not delete /hiveserver2 . hiveserver2 is the parent znode , respective hiveserver znodes will be under the parent znode ie /hiveserver2. 2) By installing do you mean you want to add another hiveserver2 to the cluster. If yes, then follow the below doc link https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_hadoop-high-availability/content/additional_hs2_installed_with_ambari.html.

asirna · ‎10-08-2018

@Madhura Mhatre, When you start HiveServer2 , it will create a znode under /hiveserver2 with the below format serverUri=hiveserver2host:10000;version=3.1.0.3.0.1.0-187;sequence=0000000000 . After hiveserver2 is stopped, it will delete this znode but not the parent /hiveserver znode. If you want to install the hiveserver2 again, you can do it again from Ambari. . Please "Accept" the answer if this helps 🙂

asirna · ‎10-08-2018

@Josh Nicholson, You can grep for the location. I am not able to think of other solution for now.

asirna · ‎10-05-2018

@Josh Nicholson, You can put all your sql commands in a file and run the file using beeline. Ex queries.sql has below statements describe formatted table1; describe formatted table2; You can run the queries.sql like below beeline -u "{url}" -f queries.sql . Please "Accept" the answer if this helps.

asirna · ‎10-05-2018

@Maryem Mary, Did this work for you? Please take a moment to login and "Accept" the answer if this helped. This will be really useful for other community users 🙂

Online	Offline
Last Visited	‎05-28-2019 04:37 PM

Member Since	‎11-07-2016 08:16 AM
Last Visited	‎05-28-2019 04:37 PM
Posts	637
Kudos received	253

Cloudera Community

Re: Ranger "Test Connection" button REST API?

Re: knox PUT request failing

Re: spark 2 jobs with OOZIE on HDP 2.6.5 hangs

Re: Ambari access through private hostname on aws ...

Re: Restart Resource Manager does not work, can no...

Re: How does zeppelin storage query result?

Re: How does zeppelin storage query result?

Re: How does zeppelin storage query result?

Re: Reading files from s3 bucket sub folders

Re: Reading files from s3 bucket sub folders

Re: Will Zookeeper select the stopped hiveserver2?

Re: Will Zookeeper select the stopped hiveserver2?

Re: How can I run multiple beeline commands in a s...

Re: How can I run multiple beeline commands in a s...

Re: How to find the running time for a program in...