It's early 2025 and agentic workflows are the hottest commodities within AI. Giving large language models (LLMs) the ability to use tools to complete actions opens up a huge range of use cases. At the same time, it's important to ensure that the AI agents we build are secure.
Why AI Agents?
AI Agents have the potential to transform society and how we leverage technology to complete routine tasks. Some key use cases for AI Agents can involve:
- Automation Customer Service Tasks
- Orchestrating Business Processes & Workflows
- Taking Actions Based on Unstructured Data
In this article you will get familiar the CrewAI library and how we can build fully private AI agents with CrewAI and Cloudera's AI Inference Service. Powered by Nvidia NeMo Inference Microservices (NIM), Cloudera's AI Inference Service gives you the ability to deploy private LLMs within your infrastructure. This article is designed to walk you through the setup and application of these tools in an agentic worfklow context to create a robust and intelligence system. Next, I hope to explore other frameworks to build AI agents and other applications for LLMs.
Useful Links
- Sample Code on GitHub
- Cloudera AI Inference Setup
Prerequisites:
- LLM Deployed on Cloudera's AI Inference Service and accessible from a Cloudera AI Workspace
- Cloudera AI Workspace in the same environment as the Inference Service
1. Obtain Details from Cloudera's AI Inference Service
First, copy the model endpoint and the model name from your deployed AI Inference. If the URL is in the format "https://<base_url>/v1/chat/completions", you will only need the part up to "https://<base_url>/v1".
2. Install CrewAI Packages
Next, launch a session within your Cloudera AI workspace with the following settings:
- Editor: JupyterLab
- Kernel: Python 3.10
- Edition: Standard
- Version: 2024.10 (or latest)
You are welcome to use a different kernel. However, you may need to resolve any dependency issues separately.
Next, launch Terminal Access and run the following command to install CrewAI.
pip install crewai==0.67.1 crewai-tools==0.12.1
3. Configure CrewAI LLM with AI Inference Endpoint
Next, open up a Jupyter notebook or a Python file to create an example agentic workflow. First, import the following packages:
# Import our key libraries and methods
from crewai import Agent, Task, Crew, Process, LLM
import os
import litellm
import json
import warnings
warnings.filterwarnings('ignore')
litellm.set_verbose=False
# Disable sending metrics to CrewAI
os.environ["OTEL_SDK_DISABLED"] = "true"
Next, let's get the AI Inference Access token in the file accessible from within the session. You can change how long this token stays active using the Knox settings within your Cloudera environment, specifically by modifying the Knox Token Kerberos - TTL (knox_token_kerberos_ttl_ms) duration.
CAII_API_KEY=json.load(open("/tmp/jwt"))["access_token"]
If you want to access the hosted model from outside a Cloudera AI Workbench, you can follow the instructions available here: https://docs.cloudera.com/machine-learning/cloud/ai-inference/topics/ml-caii-authentication.html.
Next, set the model endpoint and model name as environment variables. You are welcome to also set these separately as Workspace or Project level variables. For this example, we set them directly.
# Make sure to update these based on your model and URL
os.getenv("CAII_MODEL_NAME") = "meta/llama-3_1-8b-instruct"
os.getenv("CAII_API_BASE_URL") = "https://<base_url>/v1"
Next, we configure the model endpoint with CrewAI. For the model name, it is important to add the "openai/" prefix, since CrewAI leverages LiteLLM to be able to support various LLMs, and the"openai/" prefix tells LiteLLM that we are working with an OpenAI-compatible endpoint.
llm = LLM(
model="openai/" + os.getenv("CAII_MODEL_NAME"),
api_key=CAII_API_KEY,
base_url=os.getenv("CAII_API_BASE_URL")
)
4. Create a sample Agentic Workflow
Below, we follow an example from CrewAI, and we use the hosted LLama 3.1 8B Instruct model as the LLM backing the 2 agents.
from textwrap import dedent
# First we define the Researcher Agent
researcher_agent = Agent(
role=dedent((
"""
{topic} Senior Data Researcher
""")), # Think of this as the job title
backstory=dedent((
"""
You are a seasoned researcher with a knack for uncovering the latest developments in {topic}.
You are known for your ability to find the most relevant information and present it in a clear and concise manner.
""")), # This is the backstory of the agent, this helps the agent to understand the context of the task
goal=dedent((
"""
Uncover cutting-edge developments in {topic}. Try to keep final answers in markdown format.
""")), # This is the goal that the agent is trying to achieve
allow_delegation=False,
max_iter=2,
max_retry_limit=3,
llm=llm,
verbose=True
)
# Next, we define the Reporting Analyst
reporting_analyst = Agent(
role=dedent((
"""
{topic} Reporting Analyst
""")),
backstory=dedent((
"""
You are a meticulous analyst with a keen eye for detail. You are known for your ability to turn complex data into clear and concise reports,
making it easy for others to understand and act on the information you provide.
""")),
goal=dedent((
"""
Create detailed reports based on {topic} data analysis and research findings.
""")), # This is the goal that the agent is trying to achieve
allow_delegation=False,
max_iter=2,
max_retry_limit=3,
llm=llm,
verbose=True
)
# First, let's create a task for the researcher agent. Note below that we define which agent completes which task.
research_task = Task(
description=dedent((
"""
Conduct thorough research about {topic}
Make sure you find any interesting and relevant information given the current year is 2024.
""")),
expected_output=dedent((
"""
A list with 2 to 4 bullet points of the most relevant information about {topic}
""")),
agent=researcher_agent
)
reporting_task = Task(
description=dedent((
"""
Review the context you got and expand each topic into a full section for a report.
Make sure the report is detailed and contains any and all relevant information.
""")),
expected_output=dedent((
"""
A fully fledge reports with the mains topics, each with a full section of information.
Formatted as markdown without '```'
""")),
agent=reporting_analyst,
output_file='report.md',
context=[research_task]
)
crew = Crew(
agents=[researcher_agent, reporting_analyst],
tasks=[research_task, reporting_task],
verbose=True, # You can set it to True or False - indicates the verbosity level for logging during execution.
process=Process.sequential
)
# Notice here that the object we pass maps to the {topic} field we added to the agents and tasks.
inputs = {
"topic": "Best Cities in the US",
}
For the purposes of the example, we have not configured the agents with tools, and they will prepare this report with their trained knowledge. To kickoff the crew, you can use the following code:
def run_crew():
result = crew.kickoff(inputs=inputs)
run_crew()
You should now see the following output. Ignore any errors related to the SDK being disabled. This is because we disabled OpenTelemetry and sending metrics to CrewAI.
# Agent:
## Task:
Conduct thorough research about Best Cities in the US
Make sure you find any interesting and relevant information given the current year is 2024.
# Agent:
## Final Answer:
* As of 2024, the Best Cities in the US are determined by factors such as quality of life, affordability, job market, education, healthcare, and diversity, among others.
* According to a report by U.S. News & World Report, the top 5 Best Cities in the US in 2024 are:
+ Denver, CO: ranked #1 for its strong economy, high quality of life, and easy access to outdoor recreation.
+ Austin, TX: ranked #2 for its vibrant music scene, diverse economy, and a highly educated population.
+ Raleigh-Durham, NC: ranked #3 for its research triangle park, affordable housing, and a strong job market.
+ Portland, OR: ranked #4 for its eco-friendly lifestyle, cultural attractions, and a thriving food scene.
+ Seattle, WA: ranked #5 for its tech industry, outdoor activities, and a diverse population.
* Additionally, other notable cities in the US include Tampa, FL, Nashville, TN, and Minneapolis-St. Paul, MN, which are recognized for their economic growth, cultural attractions, and high quality of life.
# Agent:
## Task:
Review the context you got and expand each topic into a full section for a report.
Make sure the report is detailed and contains any and all relevant information.
# Agent:
## Final Answer:
# Best Cities in the US Reporting Analyst: 2024 Edition
## Executive Summary
As of 2024, the Best Cities in the US are ranked based on various factors such as quality of life, affordability, job market, education, healthcare, and diversity. This report provides an in-depth analysis of the top 5 Best Cities in the US, as well as other notable cities that are recognized for their economic growth, cultural attractions, and high quality of life.
## Methodology
The ranking of the Best Cities in the US is based on a comprehensive analysis of various data points, including:
* Quality of life: factors such as access to healthcare, education, and recreational activities
* Affordability: costs of living, housing, and transportation
* Job market: employment rates, industry growth, and average salary
* Education: quality of schools, availability of educational resources
* Healthcare: accessibility, quality, and affordability of healthcare services
* Diversity: cultural, racial, and ethnic diversity
## Top 5 Best Cities in the US
### 1. Denver, CO
* Ranked #1 in 2024 for its strong economy, high quality of life, and easy access to outdoor recreation.
* Economic growth: 3.5% annual growth rate
* Unemployment rate: 2.5%
* Average salary: $74,000
* Education: high-performing schools, University of Denver
* Healthcare: access to quality healthcare services, including the University of Colorado Hospital
### 2. Austin, TX
* Ranked #2 in 2024 for its vibrant music scene, diverse economy, and highly educated population.
* Economic growth: 4.2% annual growth rate
* Unemployment rate: 2.2%
* Average salary: $71,000
* Education: highly ranked schools, University of Texas at Austin
* Healthcare: access to quality healthcare services, including Seton Medical Center
### 3. Raleigh-Durham, NC
* Ranked #3 in 2024 for its research triangle park, affordable housing, and strong job market.
* Economic growth: 4.5% annual growth rate
* Unemployment rate: 2.1%
* Average salary: $73,000
* Education: highly ranked schools, University of North Carolina at Chapel Hill
* Healthcare: access to quality healthcare services, including Duke University Medical Center
### 4. Portland, OR
* Ranked #4 in 2024 for its eco-friendly lifestyle, cultural attractions, and thriving food scene.
* Economic growth: 3.8% annual growth rate
* Unemployment rate: 2.6%
* Average salary: $69,000
* Education: highly ranked schools, Portland State University
* Healthcare: access to quality healthcare services, including Oregon Health & Science University
### 5. Seattle, WA
* Ranked #5 in 2024 for its tech industry, outdoor activities, and diverse population.
* Economic growth: 4.1% annual growth rate
* Unemployment rate: 2.4%
* Average salary: $80,000
* Education: highly ranked schools, University of Washington
* Healthcare: access to quality healthcare services, including Swedish Medical Center
## Notable Cities
* Tampa, FL: recognized for its economic growth, cultural attractions, and high quality of life.
* Nashville, TN: recognized for its music industry, thriving food scene, and affordable housing.
* Minneapolis-St. Paul, MN: recognized for its economic growth, cultural attractions, and access to outdoor recreational activities.
## Conclusion
The Best Cities in the US for 2024 have been ranked based on a comprehensive analysis of various data points. These cities offer a high quality of life, strong economies, and access to quality education and healthcare services. Whether you're looking to relocate or simply want to explore new cities, this report provides a valuable resource for understanding the best places to live and work in the US.
Congrats! You just learned how to build your fully-private AI agents using Cloudera's AI Inference Service and CrewAI. The agentic AI applications within enterprises are endless and I wish you the best with your use case. Don't hesitate to get in touch!
