What is the scope of cloud technology in 2021?

[#1]

I wanted to know about this everchanging technology and its core advantages which can benefit a business

[#2 · Accepted solution]

Hello @william266455 

Thanks for using Cloudera Community. Your Post covers an extremely wide domain yet I would like to share my perspective from working with Customers on CDP Public Cloud:

• From the 3 Cloud Vendors, AWS was the Pioneer to start with & as such, I wish to point to the Link [1], which offers the 6 Major Advantages of Cloud Computing from an AWS WhitePaper. 
• Listing them here, We have "Trading Capital Expenses With Operational Expenses", "Scale", "Stop Guessing Capacity", "Agility", "Go Global", "Avoid Running & Maintaining Data Centre". The PDF does an awesome job of explaining the Benefits & I would skip reiterating the same. 

As you posted the Question in Cloudera Community, I would like to take a few minutes sharing the Current On-Premise Model Short-Coming from Hadoop Ecosystem perspective (Note that On-Premise has it's Advantages & Stating On-Premise is Defunct would be an Over-Statement):

• Associated "Big Data" with "Hadoop" isn't new (Although the Focus should be moving from Data Size/Type to Analytics & Value from Data). Hadoop Ecosystem offers a wide variety of Services & they fit Use-Cases with different needs. As the Services increase, So does the Data being processed & reviewed. As such, any Capacity Planning to hard to begin with. 
• Be it Scaling Up or Scaling Out, Managing Linux or Windows Server isn't Easy. A Simple Task of adding Resources to an Instance isn't performed without Operational Experience & adding New Instances requires getting in touch with Multiple Parties (Admin, Billing, Vendor, Transport, Sponsor etc)
• Other context like Agility, Monitoring, Scaling, Routing are Self Explanatory.  

Now, How does Cloudera helps via Cloudera Data Platform (CDP) is as follows:

• By ensuring Users only bother about their Use-Case (Running a Spark Job, Hive-Impala SQL, NoSQL HBase Analytics etc.) like the Users have been doing on their On-Premise, yet with the Flexibility of Cloud.
• The Best Part is a Customer can have 1 Setup on AWS, another Setup on Azure or GCP as well, yet the UI to perform your Hive/Spark/Kafka/HBase Jobs remains Similar. 
• When no Jobs are running, the Scale-Down happens implicitly & when Jobs are running, the Scale-Up happens without any User's intervention. This ensure User focus on their Job only & Cost is Saved as well. 
• Avoid Noisy Neighbours by ensuring the Jobs runs via Containers with their own resources & avoid any poor Job take away Host level resources, thereby affecting other Jobs. 

As you tag "Cloudera Data Engineering", I wish to share a bit of details on the same. CDE allows you to run Spark Job (Just like you run on On-Premise) yet with the Flexibility of running them on Containers as well. And the Best Part is, End-User don't have to know anything about the Kubernetes Cluster running their Spark Job. 

In short, Cloud offers Flexibility, Agility which is hard to achieve with On-Premise Setup. Yet, with Power comes Responsibility & here, AWS have a Shared-Responsibility Model as shared via [2]. Finally, I wish to share Cloudera Public Cloud Offering via [3] & some Awesome Free Training Videos on CDP via [4]. 

The above Opinion is Completely mine from my Experiences & I have barely scratched the Surface of the benefits from Cloud. I am happy to answer any specific query you may have from the above details. Also, Note that each of the AWS Link or PDF are Public Materials. 

- Smarak

[1] https://docs.aws.amazon.com/whitepapers/latest/aws-overview/aws-overview.pdf#six-advantages-of-cloud...

[2] https://aws.amazon.com/compliance/shared-responsibility-model/?ref=wellarchitected

[3] https://docs.cloudera.com/cdp/latest/index.html

[4] https://www.cloudera.com/about/training.html#?fq=training%3Acomplimentary%2Ffree

[#3]

Hello @william266455 

Hope you are doing well. We wish to follow-up concerning the Post & whether you may have further queries. If there are no further ask, Kindly mark the Post as Solved. 

- Smarak

[#4]

Hello @william266455 

As we haven't heard from your side, We are marking the Post as Solved as the Question wasn't associated with any Issue, rather a Generic discussion on Cloud Technology in 2021. We would like to hear from your side concerning the Cloud Technology Usage & CDP Public Cloud in your daily work & if there is any queries concerning CDP Public Cloud, which you would like us to confirm or answer. 

Thank You for using Cloudera Community. 

- Smarak