Build an Automated Financial News Aggregation and Analysis System with CrewAI and Ollama on VPS
A guide to building an automated financial news aggregation and analysis system using CrewAI and Ollama on a VPS, leveraging self-hosted large language models (LLMs) for real-time...
Image credit: cegoh · Pixabay Content License
In the digital age, real-time monitoring and analysis of financial news are critical for accurate investment decisions. With the advancement of AI, automated systems for aggregating and analyzing financial news are becoming essential tools. This guide shows how to build such a system using CrewAI and Ollama on a VPS, leveraging self-hosted large language models (LLMs) to efficiently process financial data automatically.
Introduction to CrewAI and Ollama
CrewAI is an open-source AI framework that enables the creation of AI teams (AI Crews) with specialized roles, such as information gathering, data analysis, and report generation. It simulates human workflows by assigning tasks to AI agents, each capable of handling specific parts of the process.
Ollama is a tool that allows running large language models (LLMs) like Llama 3, Mistral, or Gemma on local servers or VPS instances. Ollama simplifies deployment and management of LLMs without relying on public AI platforms, ensuring data security and control.
Requirements and Environment Setup
To deploy the system, you need:
- A High-RAM Cloud VPS (at least 16GB RAM and 4 vCPU).
- Linux operating system (Ubuntu 22.04 or equivalent).
- Python 3.10 or higher.
- Ollama installed and a local LLM model (e.g., Llama 3).
- API access to financial news sources like Alpha Vantage, Finnhub, or Yahoo Finance.
System Implementation Steps
We will implement the system in the following steps:
Step 1: Install and Configure Ollama
On the VPS, install Ollama using the command:
curl -fsSL https://ollama.com/install.sh | sh
After installation, pull the LLM model:
ollama pull llama3
Step 2: Install CrewAI and Dependencies
Set up a Python environment and install required libraries:
pip install crewai langchain ollama requests
Step 3: Design CrewAI Agents
We will create three main agents:
- NewsCollectorAgent: Collects financial news from APIs.
- AnalyzerAgent: Analyzes news content using the LLM.
- ReporterAgent: Generates summary reports and alerts.
Here is a sample code for the news collector agent:
from crewai import Agent, Task, Crew
from langchain_community.llms import Ollama
llm = Ollama(model="llama3")
news_collector = Agent(
role="Financial News",
goal="Collect real-time financial news from various sources",
backstory="You are a financial news specialist gathering data from multiple sources",
llm=llm
)
Step 4: Connect APIs and Process Data
Set up integration with financial news APIs. For example, using Alpha Vantage:
import requests
def fetch_news(api_key, symbol):
url = f"https://www.alphavantage.co/query?function=NEWS_SENTIMENT&symbol={symbol}&apikey={api_key}"
response = requests.get(url)
return response.json()
Run and Test the System
Initialize the Crew and execute the system:
news_task = Task(
description="Collect news about stock {symbol}",
agent=news_collector
)
crew = Crew(
agents=[news_collector],
tasks=[news_task],
verbose=True
)
result = crew.kickoff()
print(result)
Analysis and Optimization
To improve performance, consider:
- Optimizing the LLM by choosing a smaller version if faster processing is needed.
- Using a GPU-enabled VPS for faster inference.
- Implementing caching for repeated API calls.
Troubleshooting
- Model not loading in Ollama: Check network connection and run
ollama pullagain. - API returns no data: Verify API key and access permissions.
- Agent not working: Ensure all required libraries are correctly installed.
References
High-RAM Cloud VPS Plans Optimized for AI, Databases & Cache
Enterprise RAID 10 NVMe SSDs, CMC & FPT District 9 uplinks with Anycast Anti-DDoS and 99.99% SLA.
VPS-4E
10% VAT included
- CPU: 4 vCPU
- RAM: 8 GB
- SSD: 50 GB NVMe
