About MarcoGarcia

MarcoGarcia · ‎07-08-2018

Hi Guys, My company is working in a proposal to create a TMS ( Transportation Management System ) for Customer. The system will have maybe 1k screens, 1,2k tables, a lot of integration process, 3k users, almost 300 concurrent users. We are planing to set the bus service on Apache Nifi/Kafka, APIs based in python using Spark (maybe Livy). I'm planning to use Phoenix and HBase for Database/OLTP transactions. Any thoughs ? Sugestions ? Somone see any weak point ? Regards,

MarcoGarcia · ‎03-02-2017

Hi Umar, we fixed and works, but know it's stoping in another place, seems me the pid is not been created, maybe it is ? ============================= Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.5/services/ZEPPELIN/package/scripts/master.py", line 235, in <module> Master().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 709, in restart self.status(env) File "/var/lib/ambari-agent/cache/stacks/HDP/2.5/services/ZEPPELIN/package/scripts/master.py", line 185, in status + status_params.zeppelin_user + '*.pid')[0] IndexError: list index out of range

MarcoGarcia · ‎03-02-2017

Guys, we're reinstalling Zeppelin in our HDP 2.5, when we are finishing to install, received this error : ============================== resource_management.core.exceptions.Fail: Execution of '/usr/hdp/current/zeppelin-server/lib/bin/zeppelin-daemon.sh stop >> /var/log/zeppelin/zeppelin-setup.log' returned 127. TERM environment variable not set. -bash: /usr/hdp/current/zeppelin-server/lib/bin/zeppelin-daemon.sh: No such file or directory ============================== Looking at master.py we saw "zeppelin_params.dir" in the place of this problem, in our mind is been valued like : /usr/hdp/current/zeppelin-server/lib/ -- we know that /lib/ is wrong Question, How to change zeppelin_params.dir ?

MarcoGarcia · ‎02-11-2017

Question, Spark Thrift Server should stay on Master or Worker nodes ?

MarcoGarcia · ‎03-23-2016

One of our customers, wants to create a ODS ( Operational Data Store ) for near realtime data and in rest data. All these data comes from Database now and will receive systems logs in future. But their volumes are small, something like 100gb at total, but it will increase in the next phases of project. Our Questions : - Whats the solution to handle with this ? cause data are small for a cluster, we are thinking in use HBASE/Drill - any sugestion for this small volume ? I don't want to say it "Keep in Oracle" cause we want to prepare a architecture for the future. Tks, Regards

MarcoGarcia · ‎03-23-2016

Thanks Neeraj, I'll check with Hwx teams.

MarcoGarcia · ‎03-03-2016

Hi Guys, this is my first post here. I'm talking with some customer, everybody is talking about Data Lakes. But, is there any kind of blueprint ? ou designed architecture for this ? Like in DataWarehouses, we have Kimball and Inmon Regards

Online	Offline
Last Visited	‎05-20-2019 02:21 PM

Date Registered	‎04-10-2019 06:58 PM
Last Visited	‎05-20-2019 02:21 PM
Total Messages Posted	7
Total Kudos Received	6

Phoenix for a huge TMS (OLTP)

Re: Zeppelin Reinstal on HDP 2.5

Zeppelin Reinstal on HDP 2.5

Re: Why do we need to setup Spark Thrift Server?

How to deal with small data ?

Re: Is there a "Data Lake BluePrint" ?

Is there a "Data Lake BluePrint" ?