Compare commits
28
Commits
a12e035c1c
...
main_get
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
62110554d1 | ||
|
|
6184c34a3a | ||
|
|
80f2a82c19 | ||
|
|
7af21bed6a | ||
|
|
e7e01516ba | ||
|
|
1d732844e3 | ||
|
|
7d5c8c57b7 | ||
|
|
f13b1c1afe | ||
|
|
7004b281a4 | ||
|
|
e773775acd | ||
|
|
8138629e06 | ||
|
|
fe986256c0 | ||
|
|
6a882ed913 | ||
|
|
99fa79cde4 | ||
|
|
2993014fa8 | ||
|
|
c0a02868ac | ||
|
|
d34b67f522 | ||
|
|
84288a687a | ||
|
|
e78836bf42 | ||
|
|
7990312878 | ||
|
|
5ca1779d7d | ||
|
|
9320b79730 | ||
|
|
06800ddb43 | ||
|
|
340c11d4ab | ||
|
|
d0696a42ba | ||
|
|
233f0c5d10 | ||
|
|
8cdfb2a8e2 | ||
|
|
6182630203 |
No files matched your search
@@ -12,7 +12,13 @@
|
||||
"Bash(git push:*)",
|
||||
"Bash(conda activate:*)",
|
||||
"WebSearch",
|
||||
"Bash(pip install:*)"
|
||||
"Bash(pip install:*)",
|
||||
"Bash(git status:*)",
|
||||
"Bash(git -C /Users/xxz/PycharmProjects/crypto_predictor check-ignore -v test_dlinear_jan_2026_create_labels.py)",
|
||||
"Bash(git -C /Users/xxz/PycharmProjects/crypto_predictor status)",
|
||||
"Bash(git -C /Users/xxz/PycharmProjects/crypto_predictor diff)",
|
||||
"Bash(git -C /Users/xxz/PycharmProjects/crypto_predictor log --oneline -5)",
|
||||
"Bash(git checkout:*)"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,10 @@ BINANCE_API_URL=https://api.binance.com
|
||||
SOCKS5_PROXY=socks5://127.0.0.1:7890
|
||||
USE_PROXY=true
|
||||
|
||||
# send_msg 独立代理(可选,优先级高于 SOCKS5_PROXY)
|
||||
# SEND_MSG_PROXY=socks5://127.0.0.1:7891
|
||||
# SEND_MSG_PROXY_PORT=7891
|
||||
|
||||
# Data Configuration
|
||||
DATA_CACHE_DIR=./data/cache
|
||||
LOOKBACK_1H=168 # 1 week for 1h data
|
||||
@@ -37,3 +41,22 @@ MYSQL_PORT=3306
|
||||
MYSQL_USER=user
|
||||
MYSQL_PASSWORD=password
|
||||
MYSQL_DB=crypto_predictor
|
||||
|
||||
# Cloudflare D1 Backup Configuration
|
||||
# 开关:true 开启 MySQL -> D1 定时备份(每小时 :30)
|
||||
D1_BACKUP_ENABLED=false
|
||||
CF_D1_API_TOKEN=your_cloudflare_api_token
|
||||
CF_D1_ACCOUNT_ID=your_cloudflare_account_id
|
||||
CF_D1_DATABASE_ID=your_d1_database_id
|
||||
# 备份批大小(单次 SQL 插入的最大行数)
|
||||
D1_BACKUP_BATCH_SIZE=500
|
||||
# 单条 SQL 最大参数数(D1 限制差异较大,默认保守)
|
||||
D1_MAX_SQL_PARAMS=90
|
||||
# 指定备份表(逗号分隔);留空时自动备份 MySQL 全部表
|
||||
D1_BACKUP_TABLES=ohlcv,hourly_model_predictions,trades
|
||||
|
||||
# Polymarket Configuration
|
||||
POLYMARKET_PRIVATE_KEY=your_private_key_here
|
||||
POLYMARKET_API_KEY=your_api_key_here
|
||||
POLYMARKET_API_SECRET=your_api_secret_here
|
||||
POLYMARKET_API_PASSPHRASE=your_api_passphrase_here
|
||||
@@ -8,104 +8,66 @@
|
||||
* **语言要求**:所有回答必须使用**中文**。
|
||||
* **称呼要求**:每次回答必须加上**“主人”**。
|
||||
* **最小改动 (Minimal Change)**:只修改完成任务所必须的代码。**禁止**为了“最佳实践”或“代码美观”而进行未被请求的重构。
|
||||
* **环境约束**:本项目严格使用 **Conda** 环境。禁止使用 venv/poetry/pipenv。
|
||||
* **复用优先**:在编写新功能前,必须检查 `data/`, `models/`, `utils/` 下是否已有类似实现。
|
||||
* **环境约束**:本项目严格使用 **Conda** 环境 (`crypto_pred_312`)。禁止使用 venv/poetry/pipenv。
|
||||
* **复用优先**:在编写新功能前,必须检查 `data/`, `models/`, `api/` 下是否已有类似实现。
|
||||
|
||||
## 2. 环境与构建 (Environment & Build)
|
||||
|
||||
本项目依赖管理基于 `requirements.txt`, conda环境名称**crypto_pred_312**
|
||||
* **Python 环境**: `conda activate crypto_pred_312`
|
||||
* **依赖安装**: `pip install -r requirements.txt`
|
||||
* **数据库**:
|
||||
* **仅支持 MySQL**。禁止引入 SQLite 代码。
|
||||
* 配置:通过 `.env` 文件配置 `MYSQL_HOST`, `MYSQL_USER` 等参数。
|
||||
* 连接管理:使用 `data/database.py` 中的 `DatabaseManager` 或 `api/models.py` 中的 `get_db_connection`。
|
||||
|
||||
* **安装依赖**:
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
## 3. 项目结构说明 (Project Structure)
|
||||
|
||||
## 3. 测试与质量保证 (Testing & QA)
|
||||
* **`api/`**: 核心服务层
|
||||
* `main.py`: FastAPI 主应用入口。
|
||||
* `models.py`: 数据库模型与操作(MySQL),包含时区转换逻辑。
|
||||
* `prediction_service.py`: 预测业务逻辑。
|
||||
* `static/`: Vue 3 前端静态资源(Dashboard)。
|
||||
* **`data/`**: 数据层
|
||||
* `database.py`: 市场数据 (OHLCV) 数据库管理。
|
||||
* `trading_db.py`: 交易历史数据库管理。
|
||||
* **`models/`**: 机器学习模型定义 (DLinear, TTM-R2 等) 及训练产物。
|
||||
* **`scripts/`**: 辅助工具脚本。
|
||||
* **根目录**:
|
||||
* `start_system.py`: **系统统一启动脚本**(同时启动 API 和自动交易)。
|
||||
* `auto_trade_polymarket.py`: 自动交易主程序。
|
||||
|
||||
使用 `pytest` 作为测试框架。在提交修改前,必须确保相关测试通过。
|
||||
## 4. 开发规范 (Development Standards)
|
||||
|
||||
* **运行所有测试**:
|
||||
```bash
|
||||
pytest
|
||||
```
|
||||
### 4.1 时区处理 (Timezone)
|
||||
* **存储**: 数据库中时间戳通常为 **UTC** 或 **ET (美东时间)**。
|
||||
* **展示**: 前端及日志输出必须转换为 **CST (北京时间)**。
|
||||
* **工具**: 使用 `api.models.convert_to_cst` 进行转换。
|
||||
|
||||
* **运行单个测试文件 (常用)**:
|
||||
```bash
|
||||
pytest tests/test_example.py
|
||||
```
|
||||
### 4.2 代码风格
|
||||
* **注释**: 必须使用**中文**解释“为什么这样做”。
|
||||
* **命名**: 变量/函数 `snake_case`,类名 `PascalCase`。
|
||||
* **类型提示**: 推荐在关键接口使用 Type Hints。
|
||||
|
||||
* **运行单个测试用例 (推荐)**:
|
||||
用于专注调试某一个功能:
|
||||
```bash
|
||||
pytest tests/test_example.py::test_specific_function
|
||||
```
|
||||
### 4.3 数据库操作
|
||||
* **禁止**使用 SQLite。
|
||||
* 使用 `pymysql` 或 `sqlalchemy` 进行操作。
|
||||
* 确保所有数据库连接都使用上下文管理器 (`with` 语句) 以自动关闭。
|
||||
|
||||
* **显示打印输出 (Debug模式)**:
|
||||
```bash
|
||||
pytest -s tests/test_example.py
|
||||
```
|
||||
## 5. 测试与验证 (Testing & QA)
|
||||
|
||||
* **代码格式化与检查**:
|
||||
```bash
|
||||
black . # 格式化代码
|
||||
flake8 . # 代码风格检查
|
||||
```
|
||||
* **运行测试**: `pytest`
|
||||
* **代码检查**: `flake8 .`
|
||||
* **本地验证**:
|
||||
* 修改 API 或前端后,运行 `./start_system.py` 并访问 `http://localhost:8000` 验证。
|
||||
* 修改模型后,运行 `python ensemble_predict.py` 验证预测逻辑。
|
||||
|
||||
## 4. 代码风格规范 (Code Style)
|
||||
|
||||
* **注释 (Comments)**:
|
||||
* **必须使用中文**编写所有注释和文档字符串 (Docstrings)。
|
||||
* Docstring 风格推荐 Google Style 或简单的多行注释。
|
||||
* 注释应解释“为什么这样做” (Why),而不是“做了什么” (What)。
|
||||
|
||||
* **命名约定 (Naming)**:
|
||||
* 变量/函数: `snake_case` (如 `add_technical_features`)
|
||||
* 类名: `PascalCase` (如 `BinanceDataFetcher`)
|
||||
* 常量: `UPPER_CASE` (如 `MAX_RETRIES`)
|
||||
* 私有成员: `_snake_case` (前缀下划线)
|
||||
|
||||
* **导入 (Imports)**:
|
||||
* 顺序:标准库 -> 第三方库 -> 本地模块。
|
||||
* 尽量使用绝对导入。
|
||||
* 注意:项目中部分脚本使用了 `sys.path.insert` hack,修改时需保持兼容,不要随意删除。
|
||||
|
||||
* **类型提示 (Type Hints)**:
|
||||
* 推荐在函数签名中添加类型提示,但不强制要求全覆盖。
|
||||
* 示例: `def process_data(df: pd.DataFrame) -> np.ndarray:`
|
||||
|
||||
* **错误处理 (Error Handling)**:
|
||||
* 使用具体的异常类型 (如 `ValueError`, `FileNotFoundError`),避免裸 `except:`。
|
||||
* 在 API 调用或网络请求处必须添加重试或错误捕获逻辑。
|
||||
|
||||
## 5. 项目结构说明 (Project Structure)
|
||||
|
||||
在修改代码前,请参考以下结构定位文件:
|
||||
|
||||
* **`api/`**: FastAPI 接口服务代码。
|
||||
* **`data/`**: 数据获取 (BinanceFetcher等) 和预处理逻辑。
|
||||
* **`models/`**: 模型定义 (N-BEATS, DLinear, CatBoost 等) 和 训练好的模型文件。
|
||||
* **`tests/`**: 单元测试和集成测试。
|
||||
* **`scripts/`**: 辅助脚本。
|
||||
* **`training/`** & **`logs/`**: 训练日志和过程文件。
|
||||
* **根目录脚本**: 包含多个入口脚本 (如 `ensemble_predict.py`, `train_catboost.py`),修改时需格外小心,避免破坏生产流程。
|
||||
|
||||
## 6. 开发流程 (Workflow)
|
||||
|
||||
1. **理解需求**: 先阅读代码,确认需求范围。如果有歧义,**先询问主人**。
|
||||
2. **定位代码**: 使用 `grep` 或 `glob` 查找相关文件。
|
||||
3. **制定计划**: 思考修改方案,优先考虑不破坏现有逻辑的“加法”修改。
|
||||
4. **实施修改**: 编辑代码,保持风格一致。
|
||||
5. **验证**:
|
||||
* 运行 `flake8` 检查语法错误。
|
||||
* 运行 `pytest` 验证逻辑。
|
||||
* 如果是脚本,尝试带 `--help` 或测试参数运行以确保无运行时错误。
|
||||
|
||||
## 7. 禁止事项 (Constraints)
|
||||
## 6. 禁止事项 (Constraints)
|
||||
|
||||
* ❌ **禁止**猜测业务逻辑。
|
||||
* ❌ **禁止**删除看似无用但可能被外部脚本调用的代码。
|
||||
* ❌ **禁止**在未确认的情况下升级核心库版本 (如 torch, pandas)。
|
||||
* ❌ **禁止**输出英语回复 (代码变量名除外)。
|
||||
* ❌ **禁止**恢复已删除的 SQLite 相关代码。
|
||||
|
||||
---
|
||||
*遵循此指南,确保高效、安全地协助主人完成任务。*
|
||||
@@ -0,0 +1,83 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## 项目概述
|
||||
|
||||
基于深度学习的 BTC 小时级涨跌预测系统,包含模型训练、FastAPI 后端、Vue 3 看板、Polymarket 自动交易。当前最佳模型 DLinear 准确率 53.10%。
|
||||
|
||||
## 环境与构建
|
||||
|
||||
```bash
|
||||
# Python 环境(必须使用 Conda,禁止 venv/poetry)
|
||||
conda activate crypto_pred_312
|
||||
|
||||
# 安装依赖(PyTorch 需根据平台单独安装,requirements.txt 不含 torch)
|
||||
pip install -r requirements.txt
|
||||
|
||||
# 数据库:仅支持 MySQL,禁止 SQLite
|
||||
# 配置:复制 .env.example 为 .env 并填写 MySQL 连接信息
|
||||
```
|
||||
|
||||
## 常用命令
|
||||
|
||||
```bash
|
||||
# 启动完整系统(API + 自动交易)
|
||||
python start_system.py
|
||||
|
||||
# 单独启动 API 服务
|
||||
python -m uvicorn api.main:app --host 0.0.0.0 --port 8000
|
||||
|
||||
# 单独启动自动交易
|
||||
python auto_trade_polymarket.py
|
||||
|
||||
# 训练 DLinear 分类模型
|
||||
python train_classification_full.py
|
||||
|
||||
# 测试模型(2026年数据)
|
||||
python test_models_feb_2026.py
|
||||
|
||||
# 集成预测
|
||||
python ensemble_predict.py
|
||||
|
||||
# 测试与检查
|
||||
pytest
|
||||
flake8 .
|
||||
```
|
||||
|
||||
## 架构
|
||||
|
||||
```
|
||||
数据流: Binance API → data/fetcher.py → data/preprocessor.py (技术指标) → data/dataset.py → 模型推理
|
||||
交易流: 模型预测 → auto_trade_polymarket.py → Polymarket 下单 → MySQL 记录
|
||||
服务流: FastAPI (api/main.py) → 前端看板 (api/static/) → http://localhost:8000
|
||||
```
|
||||
|
||||
### 核心模块
|
||||
|
||||
- **`data/`** — 数据层:`fetcher.py`(Binance数据获取)、`preprocessor.py`(特征工程: RSI+MACD)、`dataset.py`(PyTorch数据集)、`database.py`(市场数据MySQL)、`trading_db.py`(交易历史MySQL)
|
||||
- **`models/`** — 模型层:`dlinear/model.py`(主力模型,时序分解+线性层)、`nbeats/`、`base.py`(抽象基类)
|
||||
- **`models/checkpoints/dlinear_classification_2026/`** — 训练产物:`best_model.pt`、`preprocessor.pkl`、`test_2026_results.json`
|
||||
- **`training/`** — 训练层:`trainer.py`(训练循环,支持 MPS/CUDA/CPU)、`config.py`(超参数)、`evaluator.py`(评估)
|
||||
- **`api/`** — 服务层:`main.py`(FastAPI入口)、`models.py`(DB操作+时区转换)、`prediction_service.py`(预测服务)、`polymarket_client.py`(交易客户端)
|
||||
- **`start_system.py`** — 统一启动入口(同时启动 API 和自动交易)
|
||||
- **`auto_trade_polymarket.py`** — 自动交易主程序(每小时执行预测+下单)
|
||||
|
||||
### DLinear 模型结构
|
||||
|
||||
输入 (168h × 9特征) → MovingAvg 时序分解 → 季节性/趋势分量各经线性层 → MLP 聚合 → logits 输出 (BCEWithLogitsLoss)
|
||||
|
||||
## 开发约束
|
||||
|
||||
- **数据库**:仅 MySQL(`pymysql`),禁止 SQLite。连接使用 `data/database.py` 的 `DatabaseManager` 或 `api/models.py` 的 `get_db_connection`
|
||||
- **时区**:数据库存 UTC/ET,前端展示 CST(北京时间),转换用 `api.models.convert_to_cst()`
|
||||
- **代码注释**:中文,解释"为什么"
|
||||
- **命名**:变量/函数 `snake_case`,类 `PascalCase`
|
||||
- **代理访问 Binance**:需在 `.env` 配置 `SOCKS5_PROXY` 和 `USE_PROXY=true`
|
||||
- **日志**:交易日志在 `logs/polymarket_trading.log`(10MB 轮转)
|
||||
|
||||
## 禁止事项
|
||||
|
||||
- 禁止引入 SQLite 或恢复已删除的 SQLite 代码
|
||||
- 禁止在未确认的情况下升级核心库版本(torch, pandas 等)
|
||||
- 禁止删除看似无用但可能被外部脚本调用的代码
|
||||
@@ -1,459 +0,0 @@
|
||||
"""分析2019年和2026年1月准确率<40%交易日的前一天技术特征"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import pickle
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
def calculate_technical_features(df):
|
||||
"""计算丰富的技术特征"""
|
||||
df = df.copy()
|
||||
|
||||
# 基础价格特征
|
||||
df['price_change'] = df['close'] - df['open']
|
||||
df['price_change_pct'] = (df['close'] - df['open']) / df['open'] * 100
|
||||
df['high_low_range'] = df['high'] - df['low']
|
||||
df['high_low_range_pct'] = (df['high'] - df['low']) / df['low'] * 100
|
||||
df['body_ratio'] = abs(df['close'] - df['open']) / (df['high'] - df['low'] + 1e-10)
|
||||
|
||||
# 移动平均线
|
||||
df['sma_5'] = ta.sma(df['close'], length=5)
|
||||
df['sma_10'] = ta.sma(df['close'], length=10)
|
||||
df['sma_20'] = ta.sma(df['close'], length=20)
|
||||
df['sma_50'] = ta.sma(df['close'], length=50)
|
||||
|
||||
# 价格相对MA位置
|
||||
df['price_vs_sma5'] = (df['close'] - df['sma_5']) / df['sma_5'] * 100
|
||||
df['price_vs_sma20'] = (df['close'] - df['sma_20']) / df['sma_20'] * 100
|
||||
|
||||
# RSI
|
||||
df['rsi_14'] = ta.rsi(df['close'], length=14)
|
||||
df['rsi_7'] = ta.rsi(df['close'], length=7)
|
||||
|
||||
# MACD
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
|
||||
# 布林带
|
||||
bbands = ta.bbands(df['close'], length=20)
|
||||
if bbands is not None and not bbands.empty:
|
||||
df['bb_upper'] = bbands.iloc[:, 0]
|
||||
df['bb_mid'] = bbands.iloc[:, 1]
|
||||
df['bb_lower'] = bbands.iloc[:, 2]
|
||||
df['bb_width'] = (df['bb_upper'] - df['bb_lower']) / df['bb_mid'] * 100
|
||||
df['bb_position'] = (df['close'] - df['bb_lower']) / (df['bb_upper'] - df['bb_lower'] + 1e-10) * 100
|
||||
|
||||
# ATR波动率
|
||||
df['atr_14'] = ta.atr(df['high'], df['low'], df['close'], length=14)
|
||||
df['atr_pct'] = df['atr_14'] / df['close'] * 100
|
||||
|
||||
# 成交量特征
|
||||
df['volume_sma_20'] = ta.sma(df['volume'], length=20)
|
||||
df['volume_ratio'] = df['volume'] / (df['volume_sma_20'] + 1e-10)
|
||||
|
||||
# 价格动量
|
||||
df['momentum_1'] = df['close'].pct_change(1) * 100
|
||||
df['momentum_3'] = df['close'].pct_change(3) * 100
|
||||
df['momentum_7'] = df['close'].pct_change(7) * 100
|
||||
df['momentum_24'] = df['close'].pct_change(24) * 100
|
||||
|
||||
# 波动率
|
||||
df['volatility_5'] = df['close'].pct_change().rolling(5).std() * 100
|
||||
df['volatility_10'] = df['close'].pct_change().rolling(10).std() * 100
|
||||
df['volatility_20'] = df['close'].pct_change().rolling(20).std() * 100
|
||||
|
||||
# 连续涨跌天数
|
||||
df['is_up'] = (df['close'] > df['open']).astype(int)
|
||||
df['consecutive_days'] = 0
|
||||
|
||||
return df
|
||||
|
||||
|
||||
def analyze_period(model, preprocessor, device, period_name, start_date, end_date):
|
||||
"""分析单个时期的数据,返回低准确率日期"""
|
||||
|
||||
print(f"\n{'='*80}")
|
||||
print(f"分析 {period_name}")
|
||||
print(f"{'='*80}")
|
||||
|
||||
# 获取数据(需要额外获取前一天的数据)
|
||||
fetcher = BinanceDataFetcher()
|
||||
|
||||
# 获取包含前一天的数据
|
||||
start_dt = pd.to_datetime(start_date) - pd.Timedelta(days=2)
|
||||
df = fetcher.fetch_ohlcv('BTC', '1h', start_date=start_dt.strftime('%Y-%m-%d'),
|
||||
end_date=end_date, use_cache=False)
|
||||
|
||||
print(f"✓ 数据: {len(df)} 行")
|
||||
|
||||
# 预处理
|
||||
X, y, df_processed = preprocessor.preprocess(df, fit=False)
|
||||
|
||||
print(f"✓ 有效样本: {len(X)}")
|
||||
|
||||
# DLinear预测
|
||||
X_tensor = torch.FloatTensor(X).to(device)
|
||||
with torch.no_grad():
|
||||
logits = model(X_tensor).cpu().numpy()
|
||||
probs = 1 / (1 + np.exp(-logits))
|
||||
preds = (probs > 0.5).astype(int).flatten()
|
||||
|
||||
true_labels = y
|
||||
|
||||
# 构建每日结果
|
||||
timestamps = df['timestamp'].iloc[168:168+len(X)].values
|
||||
timestamps_utc = pd.to_datetime(timestamps)
|
||||
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
|
||||
dates = timestamps_et.date
|
||||
|
||||
results_df = pd.DataFrame({
|
||||
'date': dates,
|
||||
'timestamp_et': timestamps_et,
|
||||
'correct': preds == true_labels,
|
||||
})
|
||||
|
||||
# 按天统计准确率
|
||||
daily_stats = results_df.groupby('date').agg({
|
||||
'correct': ['count', 'sum']
|
||||
})
|
||||
daily_stats.columns = ['total', 'correct']
|
||||
daily_stats['accuracy'] = daily_stats['correct'] / daily_stats['total'] * 100
|
||||
|
||||
# 找出准确率<40%的日期
|
||||
low_acc_dates = daily_stats[daily_stats['accuracy'] < 40].index.tolist()
|
||||
|
||||
print(f"✓ 准确率<40%的日期: {len(low_acc_dates)}天")
|
||||
for date in low_acc_dates:
|
||||
acc = daily_stats.loc[date, 'accuracy']
|
||||
print(f" {date}: {acc:.1f}%")
|
||||
|
||||
# 计算技术特征
|
||||
df_with_features = calculate_technical_features(df)
|
||||
|
||||
# 为每天聚合技术特征(使用当天的24小时数据)
|
||||
# 先确保timestamp是datetime类型
|
||||
if not pd.api.types.is_datetime64_any_dtype(df_with_features['timestamp']):
|
||||
df_with_features['timestamp'] = pd.to_datetime(df_with_features['timestamp'])
|
||||
|
||||
df_with_features['date'] = df_with_features['timestamp'].dt.tz_localize('UTC').tz_convert('America/New_York').dt.date
|
||||
|
||||
# 返回低准确率日期和完整数据
|
||||
return low_acc_dates, df_with_features, daily_stats
|
||||
|
||||
|
||||
def main():
|
||||
print("="*80)
|
||||
print("分析准确率<40%交易日的前一天技术特征共同点")
|
||||
print("="*80)
|
||||
|
||||
# ========== 加载模型 ==========
|
||||
print("\n[1/3] 加载DLinear模型...")
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
|
||||
from models.dlinear.model import DLinear
|
||||
checkpoint = torch.load(
|
||||
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device, weights_only=False
|
||||
)
|
||||
model = DLinear(checkpoint['config'])
|
||||
model.load_state_dict(checkpoint['model_state_dict'])
|
||||
model = model.to(device)
|
||||
model.eval()
|
||||
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
preprocessor = pickle.load(f)
|
||||
|
||||
print("✓ DLinear模型已加载")
|
||||
|
||||
# ========== 分析两个时期 ==========
|
||||
print("\n[2/3] 分析2019年和2026年1月...")
|
||||
|
||||
# 分析2019年1月
|
||||
low_acc_2019, df_2019, stats_2019 = analyze_period(
|
||||
model, preprocessor, device,
|
||||
"2019年1月", "2019-01-01", "2019-01-31"
|
||||
)
|
||||
|
||||
# 分析2026年1月
|
||||
low_acc_2026, df_2026, stats_2026 = analyze_period(
|
||||
model, preprocessor, device,
|
||||
"2026年1月", "2026-01-01", "2026-01-31"
|
||||
)
|
||||
|
||||
# ========== 提取前一天特征 ==========
|
||||
print("\n[3/3] 分析前一天技术特征...")
|
||||
|
||||
# 合并所有低准确率日期
|
||||
all_low_acc_dates = low_acc_2019 + low_acc_2026
|
||||
|
||||
print(f"\n总计{len(all_low_acc_dates)}个低准确率交易日")
|
||||
|
||||
# 提取前一天的特征
|
||||
prev_day_features = []
|
||||
|
||||
for date in all_low_acc_dates:
|
||||
# 获取前一天日期
|
||||
prev_date = date - timedelta(days=1)
|
||||
|
||||
# 确定使用哪个数据集
|
||||
if date in low_acc_2019:
|
||||
df = df_2019
|
||||
period = "2019年"
|
||||
else:
|
||||
df = df_2026
|
||||
period = "2026年"
|
||||
|
||||
# 获取前一天的数据
|
||||
prev_day_data = df[df['date'] == prev_date]
|
||||
|
||||
if len(prev_day_data) == 0:
|
||||
continue
|
||||
|
||||
# 聚合前一天的技术特征(使用24小时的平均值、最大值、最小值等)
|
||||
features = {
|
||||
'date': date,
|
||||
'prev_date': prev_date,
|
||||
'period': period,
|
||||
'accuracy': stats_2019.loc[date, 'accuracy'] if date in low_acc_2019 else stats_2026.loc[date, 'accuracy'],
|
||||
# 价格特征
|
||||
'prev_close': prev_day_data['close'].iloc[-1],
|
||||
'prev_open': prev_day_data['open'].iloc[0],
|
||||
'prev_price_change_pct': prev_day_data['price_change_pct'].mean(),
|
||||
'prev_high_low_range_pct': prev_day_data['high_low_range_pct'].mean(),
|
||||
'prev_body_ratio': prev_day_data['body_ratio'].mean(),
|
||||
# RSI
|
||||
'prev_rsi_14': prev_day_data['rsi_14'].iloc[-1],
|
||||
'prev_rsi_7': prev_day_data['rsi_7'].iloc[-1],
|
||||
# MACD
|
||||
'prev_macd': prev_day_data['macd'].iloc[-1] if 'macd' in prev_day_data.columns else None,
|
||||
'prev_macd_hist': prev_day_data['macd_hist'].iloc[-1] if 'macd_hist' in prev_day_data.columns else None,
|
||||
# 布林带
|
||||
'prev_bb_width': prev_day_data['bb_width'].iloc[-1] if 'bb_width' in prev_day_data.columns else None,
|
||||
'prev_bb_position': prev_day_data['bb_position'].iloc[-1] if 'bb_position' in prev_day_data.columns else None,
|
||||
# 波动率
|
||||
'prev_atr_pct': prev_day_data['atr_pct'].iloc[-1] if 'atr_pct' in prev_day_data.columns else None,
|
||||
'prev_volatility_5': prev_day_data['volatility_5'].iloc[-1] if 'volatility_5' in prev_day_data.columns else None,
|
||||
'prev_volatility_20': prev_day_data['volatility_20'].iloc[-1] if 'volatility_20' in prev_day_data.columns else None,
|
||||
# 成交量
|
||||
'prev_volume_ratio': prev_day_data['volume_ratio'].iloc[-1] if 'volume_ratio' in prev_day_data.columns else None,
|
||||
# 动量
|
||||
'prev_momentum_1': prev_day_data['momentum_1'].iloc[-1] if 'momentum_1' in prev_day_data.columns else None,
|
||||
'prev_momentum_3': prev_day_data['momentum_3'].iloc[-1] if 'momentum_3' in prev_day_data.columns else None,
|
||||
'prev_momentum_7': prev_day_data['momentum_7'].iloc[-1] if 'momentum_7' in prev_day_data.columns else None,
|
||||
'prev_momentum_24': prev_day_data['momentum_24'].iloc[-1] if 'momentum_24' in prev_day_data.columns else None,
|
||||
# 价格相对MA位置
|
||||
'prev_price_vs_sma5': prev_day_data['price_vs_sma5'].iloc[-1] if 'price_vs_sma5' in prev_day_data.columns else None,
|
||||
'prev_price_vs_sma20': prev_day_data['price_vs_sma20'].iloc[-1] if 'price_vs_sma20' in prev_day_data.columns else None,
|
||||
}
|
||||
|
||||
prev_day_features.append(features)
|
||||
|
||||
# 转换为DataFrame
|
||||
features_df = pd.DataFrame(prev_day_features)
|
||||
|
||||
# ========== 分析共同点 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 前一天技术特征统计分析")
|
||||
print("="*80)
|
||||
|
||||
print(f"\n分析样本数: {len(features_df)}个低准确率交易日")
|
||||
print(f" 2019年: {(features_df['period']=='2019年').sum()}天")
|
||||
print(f" 2026年: {(features_df['period']=='2026年').sum()}天")
|
||||
|
||||
# 统计各特征的平均值、中位数、标准差
|
||||
numeric_cols = [col for col in features_df.columns if col.startswith('prev_') and features_df[col].dtype in [np.float64, np.int64]]
|
||||
|
||||
print(f"\n前一天技术特征统计:")
|
||||
print(f"{'特征':<30} {'平均值':>10} {'中位数':>10} {'标准差':>10} {'共同点':<30}")
|
||||
print("-" * 95)
|
||||
|
||||
for col in numeric_cols:
|
||||
if features_df[col].notna().sum() > 0:
|
||||
mean_val = features_df[col].mean()
|
||||
median_val = features_df[col].median()
|
||||
std_val = features_df[col].std()
|
||||
|
||||
# 判断共同点
|
||||
common_trait = ""
|
||||
if 'rsi' in col:
|
||||
if mean_val > 70:
|
||||
common_trait = "超买区域"
|
||||
elif mean_val < 30:
|
||||
common_trait = "超卖区域"
|
||||
elif 45 <= mean_val <= 55:
|
||||
common_trait = "中性区域"
|
||||
elif 'volatility' in col:
|
||||
if mean_val > 3:
|
||||
common_trait = "高波动"
|
||||
elif mean_val < 1:
|
||||
common_trait = "低波动"
|
||||
elif 'momentum' in col:
|
||||
if mean_val > 2:
|
||||
common_trait = "强势上涨"
|
||||
elif mean_val < -2:
|
||||
common_trait = "强势下跌"
|
||||
elif abs(mean_val) < 0.5:
|
||||
common_trait = "横盘震荡"
|
||||
elif 'bb_position' in col:
|
||||
if mean_val > 80:
|
||||
common_trait = "接近布林上轨"
|
||||
elif mean_val < 20:
|
||||
common_trait = "接近布林下轨"
|
||||
elif 'volume_ratio' in col:
|
||||
if mean_val > 1.5:
|
||||
common_trait = "成交量放大"
|
||||
elif mean_val < 0.8:
|
||||
common_trait = "成交量萎缩"
|
||||
|
||||
print(f"{col:<30} {mean_val:>10.2f} {median_val:>10.2f} {std_val:>10.2f} {common_trait:<30}")
|
||||
|
||||
# ========== 对比分析 ==========
|
||||
print("\n" + "="*80)
|
||||
print("🔍 2019年 vs 2026年对比分析")
|
||||
print("="*80)
|
||||
|
||||
features_2019 = features_df[features_df['period'] == '2019年']
|
||||
features_2026 = features_df[features_df['period'] == '2026年']
|
||||
|
||||
if len(features_2019) > 0 and len(features_2026) > 0:
|
||||
print(f"\n{'特征':<30} {'2019年平均':>12} {'2026年平均':>12} {'差异':>10}")
|
||||
print("-" * 70)
|
||||
|
||||
for col in numeric_cols[:15]: # 显示前15个特征
|
||||
if features_2019[col].notna().sum() > 0 and features_2026[col].notna().sum() > 0:
|
||||
mean_2019 = features_2019[col].mean()
|
||||
mean_2026 = features_2026[col].mean()
|
||||
diff = mean_2026 - mean_2019
|
||||
|
||||
print(f"{col:<30} {mean_2019:>12.2f} {mean_2026:>12.2f} {diff:>10.2f}")
|
||||
|
||||
# ========== 关键共同点总结 ==========
|
||||
print("\n" + "="*80)
|
||||
print("💡 关键共同点总结")
|
||||
print("="*80)
|
||||
|
||||
# RSI分析
|
||||
rsi_mean = features_df['prev_rsi_14'].mean()
|
||||
print(f"\n1. RSI指标:")
|
||||
print(f" 平均RSI: {rsi_mean:.1f}")
|
||||
if rsi_mean > 70:
|
||||
print(f" 🔴 共同点: 前一天处于超买状态")
|
||||
elif rsi_mean < 30:
|
||||
print(f" 🟢 共同点: 前一天处于超卖状态")
|
||||
else:
|
||||
print(f" ➖ 共同点: 前一天RSI在中性区域")
|
||||
|
||||
# 波动率分析
|
||||
vol_mean = features_df['prev_volatility_20'].mean()
|
||||
print(f"\n2. 波动率:")
|
||||
print(f" 平均波动率: {vol_mean:.2f}%")
|
||||
if vol_mean > 3:
|
||||
print(f" ⚠️ 共同点: 前一天波动率较高")
|
||||
elif vol_mean < 1:
|
||||
print(f" ✅ 共同点: 前一天波动率较低")
|
||||
|
||||
# 动量分析
|
||||
mom_24_mean = features_df['prev_momentum_24'].mean()
|
||||
print(f"\n3. 24小时动量:")
|
||||
print(f" 平均动量: {mom_24_mean:.2f}%")
|
||||
if abs(mom_24_mean) > 5:
|
||||
print(f" 🚀 共同点: 前一天有强趋势({'+' if mom_24_mean > 0 else ''}{mom_24_mean:.1f}%)")
|
||||
else:
|
||||
print(f" 🔄 共同点: 前一天横盘震荡")
|
||||
|
||||
# 布林带位置
|
||||
bb_pos_mean = features_df['prev_bb_position'].mean()
|
||||
print(f"\n4. 布林带位置:")
|
||||
print(f" 平均位置: {bb_pos_mean:.1f}%")
|
||||
if bb_pos_mean > 80:
|
||||
print(f" 🔴 共同点: 前一天价格接近布林上轨")
|
||||
elif bb_pos_mean < 20:
|
||||
print(f" 🟢 共同点: 前一天价格接近布林下轨")
|
||||
else:
|
||||
print(f" ➖ 共同点: 前一天价格在布林带中部")
|
||||
|
||||
# 成交量
|
||||
vol_ratio_mean = features_df['prev_volume_ratio'].mean()
|
||||
print(f"\n5. 成交量:")
|
||||
print(f" 平均成交量比: {vol_ratio_mean:.2f}x")
|
||||
if vol_ratio_mean > 1.5:
|
||||
print(f" 📈 共同点: 前一天成交量显著放大")
|
||||
elif vol_ratio_mean < 0.8:
|
||||
print(f" 📉 共同点: 前一天成交量萎缩")
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✓ 技术特征分析完成!")
|
||||
print("="*80)
|
||||
|
||||
# 保存详细数据
|
||||
output_file = "./results/low_accuracy_prev_day_features.csv"
|
||||
os.makedirs("./results", exist_ok=True)
|
||||
features_df.to_csv(output_file, index=False)
|
||||
print(f"\n✓ 详细数据已保存至: {output_file}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+6
-6
@@ -106,8 +106,8 @@ async def get_monthly_accuracy(
|
||||
service = get_prediction_service()
|
||||
service.ensure_predictions_for_month(month)
|
||||
|
||||
# 获取统计
|
||||
stats = PredictionModel.get_monthly_stats(month)
|
||||
# 获取统计 (使用 V2 方法)
|
||||
stats = PredictionModel.get_monthly_stats_v2(month)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
@@ -135,8 +135,8 @@ async def get_daily_accuracy(
|
||||
service = get_prediction_service()
|
||||
service.ensure_predictions_for_month(month)
|
||||
|
||||
# 获取统计
|
||||
result = PredictionModel.get_daily_stats(month=month, page=page, limit=limit)
|
||||
# 获取统计 (使用 V2 方法)
|
||||
result = PredictionModel.get_daily_stats_v2(month=month, page=page, limit=limit)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
@@ -155,8 +155,8 @@ async def get_hourly_accuracy(date: str):
|
||||
service = get_prediction_service()
|
||||
service.ensure_predictions_for_date(date)
|
||||
|
||||
# 获取详情
|
||||
details = PredictionModel.get_hourly_details(date)
|
||||
# 获取详情 (使用 V2 方法)
|
||||
details = PredictionModel.get_hourly_details_v2(date)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
|
||||
+363
-175
@@ -4,7 +4,7 @@
|
||||
import pymysql
|
||||
import os
|
||||
from typing import Optional, List, Dict
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timedelta
|
||||
from contextlib import contextmanager
|
||||
from zoneinfo import ZoneInfo
|
||||
from dotenv import load_dotenv
|
||||
@@ -17,12 +17,13 @@ TZ_ET = ZoneInfo('America/New_York') # 美东时间 (Polymarket)
|
||||
TZ_CST = ZoneInfo('Asia/Shanghai') # 中国标准时间 (前端显示)
|
||||
|
||||
|
||||
def convert_to_cst(timestamp_val) -> str:
|
||||
def convert_to_cst(timestamp_val, source_tz: str = 'UTC') -> str:
|
||||
"""
|
||||
将时间戳转换为 CST 时间字符串
|
||||
|
||||
Args:
|
||||
timestamp_val: datetime 对象或 ISO 格式时间戳字符串 (假设是 UTC)
|
||||
timestamp_val: datetime 对象或 ISO 格式时间戳字符串
|
||||
source_tz: 输入时区 - 'UTC'(默认,币安数据), 'ET'(Polymarket), 'CST'(已是北京时间)
|
||||
|
||||
Returns:
|
||||
CST 时间字符串 (YYYY-MM-DD HH:MM:SS)
|
||||
@@ -37,10 +38,20 @@ def convert_to_cst(timestamp_val) -> str:
|
||||
# 字符串格式
|
||||
dt = datetime.fromisoformat(str(timestamp_val).replace('Z', '+00:00'))
|
||||
|
||||
# 如果没有时区信息,假设是 UTC
|
||||
# 根据来源时区设置时区信息
|
||||
if dt.tzinfo is None:
|
||||
if source_tz == 'ET':
|
||||
dt = dt.replace(tzinfo=TZ_ET)
|
||||
elif source_tz == 'CST':
|
||||
# 已经是 CST,直接格式化返回
|
||||
return dt.strftime('%Y-%m-%d %H:%M:%S')
|
||||
else: # 默认 UTC
|
||||
dt = dt.replace(tzinfo=TZ_UTC)
|
||||
|
||||
# 如果来源已经是 CST,直接格式化
|
||||
if source_tz == 'CST':
|
||||
return dt.strftime('%Y-%m-%d %H:%M:%S')
|
||||
|
||||
# 转换为 CST
|
||||
dt_cst = dt.astimezone(TZ_CST)
|
||||
return dt_cst.strftime('%Y-%m-%d %H:%M:%S')
|
||||
@@ -77,6 +88,7 @@ def _dict_from_row(row):
|
||||
|
||||
def init_model_predictions_table():
|
||||
"""初始化 model_predictions 表"""
|
||||
# 兼容旧表,但主要使用 hourly_model_predictions
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute('''
|
||||
@@ -99,6 +111,42 @@ def init_model_predictions_table():
|
||||
INDEX idx_model (model_name)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
''')
|
||||
|
||||
# 确保 hourly_model_predictions 表存在
|
||||
cursor.execute('''
|
||||
CREATE TABLE IF NOT EXISTS hourly_model_predictions (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
timestamp DATETIME NOT NULL,
|
||||
timestamp_et VARCHAR(50),
|
||||
date VARCHAR(10),
|
||||
hour INT,
|
||||
|
||||
open_price DOUBLE,
|
||||
close_price DOUBLE,
|
||||
actual_result INT COMMENT '0=Down, 1=Up',
|
||||
|
||||
dlinear_prediction INT,
|
||||
dlinear_prob DOUBLE,
|
||||
dlinear_correct INT,
|
||||
|
||||
nbeats_prediction INT,
|
||||
nbeats_prob DOUBLE,
|
||||
nbeats_correct INT,
|
||||
|
||||
ttm_prediction INT,
|
||||
ttm_prob DOUBLE,
|
||||
ttm_correct INT,
|
||||
|
||||
ensemble_prediction INT,
|
||||
ensemble_votes VARCHAR(50),
|
||||
ensemble_correct INT,
|
||||
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
UNIQUE KEY uk_date_hour (date, hour),
|
||||
INDEX idx_timestamp (timestamp)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
''')
|
||||
conn.commit()
|
||||
|
||||
|
||||
@@ -122,7 +170,8 @@ class TradeModel:
|
||||
row = cursor.fetchone()
|
||||
result = _dict_from_row(row)
|
||||
if result and result.get('timestamp'):
|
||||
result['timestamp_cst'] = convert_to_cst(result['timestamp'])
|
||||
# 交易记录时间已经是 CST,直接格式化
|
||||
result['timestamp_cst'] = convert_to_cst(result['timestamp'], source_tz='CST')
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
@@ -131,12 +180,6 @@ class TradeModel:
|
||||
day: Optional[str] = None) -> Dict:
|
||||
"""
|
||||
获取历史交易记录
|
||||
|
||||
Args:
|
||||
page: 页码
|
||||
limit: 每页条数
|
||||
month: 月份筛选 (YYYY-MM)
|
||||
day: 日期筛选 (YYYY-MM-DD)
|
||||
"""
|
||||
offset = (page - 1) * limit
|
||||
|
||||
@@ -148,7 +191,7 @@ class TradeModel:
|
||||
params = []
|
||||
|
||||
if month:
|
||||
where_clauses.append("DATE_FORMAT(timestamp, '%%Y-%%m') = %s")
|
||||
where_clauses.append("DATE_FORMAT(timestamp, '%Y-%m') = %s")
|
||||
params.append(month)
|
||||
|
||||
if day:
|
||||
@@ -173,7 +216,8 @@ class TradeModel:
|
||||
for row in rows:
|
||||
item = _dict_from_row(row)
|
||||
if item and item.get('timestamp'):
|
||||
item['timestamp_cst'] = convert_to_cst(item['timestamp'])
|
||||
# 交易记录时间已经是 CST,直接格式化
|
||||
item['timestamp_cst'] = convert_to_cst(item['timestamp'], source_tz='CST')
|
||||
data_list.append(item)
|
||||
|
||||
return {
|
||||
@@ -186,130 +230,178 @@ class TradeModel:
|
||||
|
||||
|
||||
class PredictionModel:
|
||||
"""预测记录模型"""
|
||||
"""预测记录模型 (V2 - 宽表结构)"""
|
||||
|
||||
@staticmethod
|
||||
def get_prediction(date: str, hour: int, model_name: str) -> Optional[Dict]:
|
||||
"""获取指定时间和模型的预测记录"""
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("""
|
||||
SELECT * FROM model_predictions
|
||||
WHERE date = %s AND hour = %s AND model_name = %s
|
||||
""", (date, hour, model_name))
|
||||
row = cursor.fetchone()
|
||||
return _dict_from_row(row)
|
||||
|
||||
@staticmethod
|
||||
def save_prediction(data: Dict) -> int:
|
||||
def save_hourly_prediction(data: Dict) -> int:
|
||||
"""
|
||||
保存预测记录
|
||||
|
||||
Args:
|
||||
data: 预测数据字典
|
||||
|
||||
Returns:
|
||||
插入的记录ID
|
||||
保存每小时综合预测记录 (宽表)
|
||||
"""
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
sql = """
|
||||
INSERT INTO model_predictions
|
||||
(timestamp, timestamp_et, date, hour, model_name,
|
||||
prediction, probability, actual_result, is_correct,
|
||||
open_price, close_price)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
INSERT INTO hourly_model_predictions
|
||||
(timestamp, timestamp_et, date, hour, open_price, close_price, actual_result,
|
||||
dlinear_prediction, dlinear_prob, dlinear_correct,
|
||||
nbeats_prediction, nbeats_prob, nbeats_correct,
|
||||
ttm_prediction, ttm_prob, ttm_correct,
|
||||
ensemble_prediction, ensemble_votes, ensemble_correct)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON DUPLICATE KEY UPDATE
|
||||
prediction = VALUES(prediction),
|
||||
probability = VALUES(probability),
|
||||
actual_result = VALUES(actual_result),
|
||||
is_correct = VALUES(is_correct),
|
||||
open_price = VALUES(open_price),
|
||||
close_price = VALUES(close_price)
|
||||
close_price = VALUES(close_price),
|
||||
actual_result = VALUES(actual_result),
|
||||
dlinear_prediction = VALUES(dlinear_prediction),
|
||||
dlinear_prob = VALUES(dlinear_prob),
|
||||
dlinear_correct = VALUES(dlinear_correct),
|
||||
nbeats_prediction = VALUES(nbeats_prediction),
|
||||
nbeats_prob = VALUES(nbeats_prob),
|
||||
nbeats_correct = VALUES(nbeats_correct),
|
||||
ttm_prediction = VALUES(ttm_prediction),
|
||||
ttm_prob = VALUES(ttm_prob),
|
||||
ttm_correct = VALUES(ttm_correct),
|
||||
ensemble_prediction = VALUES(ensemble_prediction),
|
||||
ensemble_votes = VALUES(ensemble_votes),
|
||||
ensemble_correct = VALUES(ensemble_correct)
|
||||
"""
|
||||
cursor.execute(sql, (
|
||||
data.get('timestamp'),
|
||||
data.get('timestamp_et'),
|
||||
data.get('date'),
|
||||
data.get('hour'),
|
||||
data.get('model_name'),
|
||||
data.get('prediction'),
|
||||
data.get('probability'),
|
||||
data.get('actual_result'),
|
||||
data.get('is_correct'),
|
||||
data.get('open_price'),
|
||||
data.get('close_price')
|
||||
data.get('close_price'),
|
||||
data.get('actual_result'),
|
||||
data.get('dlinear_prediction'),
|
||||
data.get('dlinear_prob'),
|
||||
data.get('dlinear_correct'),
|
||||
data.get('nbeats_prediction'),
|
||||
data.get('nbeats_prob'),
|
||||
data.get('nbeats_correct'),
|
||||
data.get('ttm_prediction'),
|
||||
data.get('ttm_prob'),
|
||||
data.get('ttm_correct'),
|
||||
data.get('ensemble_prediction'),
|
||||
str(data.get('ensemble_votes')) if data.get('ensemble_votes') else None,
|
||||
data.get('ensemble_correct')
|
||||
))
|
||||
conn.commit()
|
||||
return cursor.lastrowid or 0
|
||||
|
||||
@staticmethod
|
||||
def update_actual_result(date: str, hour: int, model_name: str,
|
||||
actual_result: int, close_price: float) -> bool:
|
||||
"""更新实际结果"""
|
||||
def update_actual_result_v2(date: str, hour: int, actual_result: int, close_price: float) -> bool:
|
||||
"""
|
||||
更新实际结果,并重新计算各模型正确性 (V2)
|
||||
"""
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
|
||||
# 先获取预测值
|
||||
# 获取该小时记录
|
||||
cursor.execute("""
|
||||
SELECT prediction FROM model_predictions
|
||||
WHERE date = %s AND hour = %s AND model_name = %s
|
||||
""", (date, hour, model_name))
|
||||
SELECT * FROM hourly_model_predictions
|
||||
WHERE date = %s AND hour = %s
|
||||
""", (date, hour))
|
||||
row = cursor.fetchone()
|
||||
|
||||
if not row:
|
||||
return False
|
||||
|
||||
prediction = row['prediction']
|
||||
is_correct = 1 if prediction == actual_result else 0
|
||||
# 计算正确性
|
||||
updates = {
|
||||
'actual_result': actual_result,
|
||||
'close_price': close_price
|
||||
}
|
||||
|
||||
cursor.execute("""
|
||||
UPDATE model_predictions
|
||||
SET actual_result = %s, is_correct = %s, close_price = %s
|
||||
WHERE date = %s AND hour = %s AND model_name = %s
|
||||
""", (actual_result, is_correct, close_price, date, hour, model_name))
|
||||
# DLinear
|
||||
if row['dlinear_prediction'] is not None:
|
||||
updates['dlinear_correct'] = 1 if row['dlinear_prediction'] == actual_result else 0
|
||||
|
||||
# N-BEATS
|
||||
if row['nbeats_prediction'] is not None:
|
||||
updates['nbeats_correct'] = 1 if row['nbeats_prediction'] == actual_result else 0
|
||||
|
||||
# TTM
|
||||
if row['ttm_prediction'] is not None:
|
||||
updates['ttm_correct'] = 1 if row['ttm_prediction'] == actual_result else 0
|
||||
|
||||
# Ensemble
|
||||
if row['ensemble_prediction'] is not None:
|
||||
updates['ensemble_correct'] = 1 if row['ensemble_prediction'] == actual_result else 0
|
||||
|
||||
# 构建更新SQL
|
||||
set_clause = ", ".join([f"{k} = %s" for k in updates.keys()])
|
||||
sql = f"UPDATE hourly_model_predictions SET {set_clause} WHERE date = %s AND hour = %s"
|
||||
|
||||
cursor.execute(sql, list(updates.values()) + [date, hour])
|
||||
conn.commit()
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
def get_monthly_stats(month: str) -> Dict:
|
||||
def get_monthly_stats_v2(month: str) -> Dict:
|
||||
"""
|
||||
获取月度统计
|
||||
|
||||
Args:
|
||||
month: 月份 (YYYY-MM)
|
||||
|
||||
Returns:
|
||||
各模型的统计数据
|
||||
获取月度统计 (V2 - 从宽表)
|
||||
修改:不使用 SQL 进行时区转换和格式化,而是查询 UTC 时间范围,然后在 Python 中处理
|
||||
"""
|
||||
# 计算 UTC 时间范围
|
||||
try:
|
||||
year, mon = map(int, month.split('-'))
|
||||
start_date_cst = datetime(year, mon, 1, tzinfo=TZ_CST)
|
||||
# 下个月第一天
|
||||
if mon == 12:
|
||||
end_date_cst = datetime(year + 1, 1, 1, tzinfo=TZ_CST)
|
||||
else:
|
||||
end_date_cst = datetime(year, mon + 1, 1, tzinfo=TZ_CST)
|
||||
|
||||
# 转换为 UTC
|
||||
start_dt_utc = start_date_cst.astimezone(TZ_UTC)
|
||||
end_dt_utc = end_date_cst.astimezone(TZ_UTC)
|
||||
|
||||
start_str = start_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
|
||||
end_str = end_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
|
||||
|
||||
except Exception as e:
|
||||
print(f"日期解析错误: {e}")
|
||||
return {}
|
||||
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
|
||||
sql = """
|
||||
SELECT
|
||||
model_name,
|
||||
COUNT(*) as total,
|
||||
SUM(CASE WHEN is_correct = 1 THEN 1 ELSE 0 END) as correct,
|
||||
SUM(CASE WHEN is_correct = 0 THEN 1 ELSE 0 END) as incorrect
|
||||
FROM model_predictions
|
||||
WHERE DATE_FORMAT(date, '%%Y-%%m') = %s
|
||||
AND actual_result IS NOT NULL
|
||||
GROUP BY model_name
|
||||
"""
|
||||
cursor.execute(sql, (month,))
|
||||
|
||||
rows = cursor.fetchall()
|
||||
# 统计各个模型的准确率
|
||||
models = ['dlinear', 'nbeats', 'ttm', 'ensemble']
|
||||
result = {}
|
||||
|
||||
for row in rows:
|
||||
row_dict = _dict_from_row(row)
|
||||
model = row_dict['model_name']
|
||||
total = row_dict['total']
|
||||
correct = row_dict['correct'] or 0
|
||||
incorrect = row_dict['incorrect'] or 0
|
||||
for model in models:
|
||||
col_pred = f"{model}_prediction"
|
||||
col_correct = f"{model}_correct"
|
||||
|
||||
# 查询 UTC 时间范围内的所有数据
|
||||
sql = f"""
|
||||
SELECT
|
||||
COUNT({col_pred}) as total,
|
||||
SUM(CASE WHEN {col_correct} = 1 THEN 1 ELSE 0 END) as correct,
|
||||
SUM(CASE WHEN {col_correct} = 0 THEN 1 ELSE 0 END) as incorrect
|
||||
FROM hourly_model_predictions
|
||||
WHERE timestamp >= %s AND timestamp < %s
|
||||
AND actual_result IS NOT NULL
|
||||
"""
|
||||
cursor.execute(sql, (start_str, end_str))
|
||||
row = cursor.fetchone()
|
||||
|
||||
if row:
|
||||
total = int(row['total'])
|
||||
correct = int(row['correct'] or 0)
|
||||
incorrect = int(row['incorrect'] or 0)
|
||||
accuracy = (correct / total * 100) if total > 0 else 0
|
||||
|
||||
result[model] = {
|
||||
# 格式化键名以匹配前端
|
||||
model_key = {
|
||||
'dlinear': 'DLinear',
|
||||
'nbeats': 'N-BEATS',
|
||||
'ttm': 'TTM-R2',
|
||||
'ensemble': 'Ensemble'
|
||||
}.get(model, model)
|
||||
|
||||
result[model_key] = {
|
||||
"total": total,
|
||||
"correct": correct,
|
||||
"incorrect": incorrect,
|
||||
@@ -319,66 +411,118 @@ class PredictionModel:
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def get_daily_stats(month: str, page: int = 1, limit: int = 10) -> Dict:
|
||||
def get_daily_stats_v2(month: str, page: int = 1, limit: int = 10) -> Dict:
|
||||
"""
|
||||
获取每日统计
|
||||
获取每日统计 (V2 - 从宽表)
|
||||
修改:不使用 SQL 进行时区转换,查询 UTC 范围,然后在 Python 中转换
|
||||
"""
|
||||
# 计算 UTC 时间范围
|
||||
try:
|
||||
year, mon = map(int, month.split('-'))
|
||||
start_date_cst = datetime(year, mon, 1, tzinfo=TZ_CST)
|
||||
if mon == 12:
|
||||
end_date_cst = datetime(year + 1, 1, 1, tzinfo=TZ_CST)
|
||||
else:
|
||||
end_date_cst = datetime(year, mon + 1, 1, tzinfo=TZ_CST)
|
||||
|
||||
start_dt_utc = start_date_cst.astimezone(TZ_UTC)
|
||||
end_dt_utc = end_date_cst.astimezone(TZ_UTC)
|
||||
|
||||
start_str = start_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
|
||||
end_str = end_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
|
||||
except Exception:
|
||||
return {"data": [], "page": page, "limit": limit, "total": 0, "total_pages": 0}
|
||||
|
||||
Args:
|
||||
month: 月份 (YYYY-MM)
|
||||
page: 页码
|
||||
limit: 每页条数
|
||||
"""
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
|
||||
# 获取范围内的所有原始数据
|
||||
# 我们需要 timestamp, actual_result, 和各模型的 prediction/correct
|
||||
# 为了按 CST 日期分组,我们需要取出每一行,然后在 Python 中转换时间
|
||||
|
||||
sql = """
|
||||
SELECT
|
||||
date,
|
||||
model_name,
|
||||
COUNT(*) as total,
|
||||
SUM(CASE WHEN is_correct = 1 THEN 1 ELSE 0 END) as correct,
|
||||
SUM(CASE WHEN is_correct = 0 THEN 1 ELSE 0 END) as incorrect
|
||||
FROM model_predictions
|
||||
WHERE DATE_FORMAT(date, '%%Y-%%m') = %s
|
||||
timestamp,
|
||||
actual_result,
|
||||
dlinear_prediction, dlinear_correct,
|
||||
nbeats_prediction, nbeats_correct,
|
||||
ttm_prediction, ttm_correct,
|
||||
ensemble_prediction, ensemble_correct
|
||||
FROM hourly_model_predictions
|
||||
WHERE timestamp >= %s AND timestamp < %s
|
||||
AND actual_result IS NOT NULL
|
||||
GROUP BY date, model_name
|
||||
ORDER BY date DESC
|
||||
ORDER BY timestamp DESC
|
||||
"""
|
||||
cursor.execute(sql, (month,))
|
||||
|
||||
cursor.execute(sql, (start_str, end_str))
|
||||
rows = cursor.fetchall()
|
||||
|
||||
# 重组数据: 按日期合并不同模型的统计
|
||||
daily_data = {}
|
||||
# 在 Python 中处理分组
|
||||
daily_stats = {} # { "YYYY-MM-DD": { "date": "...", "dlinear": {...}, ... } }
|
||||
|
||||
for row in rows:
|
||||
row_dict = _dict_from_row(row)
|
||||
date = row_dict['date']
|
||||
if date not in daily_data:
|
||||
daily_data[date] = {"date": date}
|
||||
# 转换 timestamp 到 CST
|
||||
ts_utc = row['timestamp']
|
||||
# 确保是带时区的 datetime
|
||||
if isinstance(ts_utc, str):
|
||||
ts_utc = datetime.fromisoformat(str(ts_utc).replace('Z', '+00:00'))
|
||||
if ts_utc.tzinfo is None:
|
||||
ts_utc = ts_utc.replace(tzinfo=TZ_UTC)
|
||||
|
||||
model = row_dict['model_name']
|
||||
total = row_dict['total']
|
||||
correct = row_dict['correct'] or 0
|
||||
incorrect = row_dict['incorrect'] or 0
|
||||
accuracy = (correct / total * 100) if total > 0 else 0
|
||||
ts_cst = ts_utc.astimezone(TZ_CST)
|
||||
date_str = ts_cst.strftime('%Y-%m-%d')
|
||||
|
||||
daily_data[date][model] = {
|
||||
"total": total,
|
||||
"correct": correct,
|
||||
"incorrect": incorrect,
|
||||
"accuracy": round(accuracy, 2)
|
||||
if date_str not in daily_stats:
|
||||
daily_stats[date_str] = {
|
||||
"date": date_str,
|
||||
"raw_data": {m: {"total": 0, "correct": 0} for m in ['dlinear', 'nbeats', 'ttm', 'ensemble']}
|
||||
}
|
||||
|
||||
# 排序并分页
|
||||
sorted_dates = sorted(daily_data.keys(), reverse=True)
|
||||
# 累加统计
|
||||
models = ['dlinear', 'nbeats', 'ttm', 'ensemble']
|
||||
for model in models:
|
||||
pred = row[f"{model}_prediction"]
|
||||
correct_flag = row[f"{model}_correct"]
|
||||
|
||||
if pred is not None:
|
||||
daily_stats[date_str]['raw_data'][model]['total'] += 1
|
||||
if correct_flag == 1:
|
||||
daily_stats[date_str]['raw_data'][model]['correct'] += 1
|
||||
|
||||
# 整理为最终格式
|
||||
sorted_dates = sorted(daily_stats.keys(), reverse=True)
|
||||
total_count = len(sorted_dates)
|
||||
|
||||
# 分页
|
||||
offset = (page - 1) * limit
|
||||
paginated_dates = sorted_dates[offset:offset + limit]
|
||||
|
||||
data = [daily_data[d] for d in paginated_dates]
|
||||
result_data = []
|
||||
for date_key in paginated_dates:
|
||||
day_item = daily_stats[date_key]
|
||||
formatted_item = {"date": day_item["date"]}
|
||||
|
||||
for model in ['dlinear', 'nbeats', 'ttm', 'ensemble']:
|
||||
raw = day_item['raw_data'][model]
|
||||
total = int(raw['total'])
|
||||
correct = int(raw['correct'])
|
||||
accuracy = (correct / total * 100) if total > 0 else 0
|
||||
|
||||
model_key = {
|
||||
'dlinear': 'DLinear',
|
||||
'nbeats': 'N-BEATS',
|
||||
'ttm': 'TTM-R2',
|
||||
'ensemble': 'Ensemble'
|
||||
}.get(model, model)
|
||||
|
||||
formatted_item[model_key] = {
|
||||
"total": total,
|
||||
"correct": correct,
|
||||
"accuracy": round(accuracy, 2)
|
||||
}
|
||||
result_data.append(formatted_item)
|
||||
|
||||
return {
|
||||
"data": data,
|
||||
"data": result_data,
|
||||
"page": page,
|
||||
"limit": limit,
|
||||
"total": total_count,
|
||||
@@ -386,69 +530,113 @@ class PredictionModel:
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def get_hourly_details(date: str) -> List[Dict]:
|
||||
def get_hourly_details_v2(date: str) -> List[Dict]:
|
||||
"""
|
||||
获取指定日期的每小时详情
|
||||
获取指定日期的每小时详情 (V2)
|
||||
修改:根据传入的 CST 日期计算 UTC 范围,查询该范围内的数据
|
||||
"""
|
||||
try:
|
||||
# date 是 'YYYY-MM-DD' (CST)
|
||||
date_cst_start = datetime.strptime(date, '%Y-%m-%d').replace(tzinfo=TZ_CST)
|
||||
date_cst_end = date_cst_start + timedelta(days=1)
|
||||
|
||||
start_dt_utc = date_cst_start.astimezone(TZ_UTC)
|
||||
end_dt_utc = date_cst_end.astimezone(TZ_UTC)
|
||||
|
||||
start_str = start_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
|
||||
end_str = end_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
|
||||
except Exception as e:
|
||||
print(f"日期处理错误: {e}")
|
||||
return []
|
||||
|
||||
Args:
|
||||
date: 日期 (YYYY-MM-DD)
|
||||
"""
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("""
|
||||
SELECT * FROM model_predictions
|
||||
WHERE date = %s
|
||||
ORDER BY hour, model_name
|
||||
""", (date,))
|
||||
SELECT *
|
||||
FROM hourly_model_predictions
|
||||
WHERE timestamp >= %s AND timestamp < %s
|
||||
ORDER BY timestamp ASC
|
||||
""", (start_str, end_str))
|
||||
|
||||
rows = cursor.fetchall()
|
||||
result = []
|
||||
|
||||
# 按小时重组数据
|
||||
hourly_data = {}
|
||||
for row in rows:
|
||||
row_dict = _dict_from_row(row)
|
||||
hour = row_dict['hour']
|
||||
if hour not in hourly_data:
|
||||
# 将 timestamp 转换为 CST 时间
|
||||
timestamp_cst = convert_to_cst(row_dict.get('timestamp', ''))
|
||||
hourly_data[hour] = {
|
||||
"hour": hour,
|
||||
"timestamp_et": row_dict['timestamp_et'],
|
||||
"timestamp_cst": timestamp_cst,
|
||||
"open_price": row_dict['open_price'],
|
||||
"close_price": row_dict['close_price'],
|
||||
"actual_result": row_dict['actual_result'],
|
||||
"models": {}
|
||||
}
|
||||
# 计算 CST 小时用于显示排序 (虽然 SQL 已经按 timestamp 排序了,但为了确保前端显示 hour 字段正确)
|
||||
ts_utc = row['timestamp']
|
||||
if isinstance(ts_utc, str):
|
||||
ts_utc = datetime.fromisoformat(str(ts_utc).replace('Z', '+00:00'))
|
||||
if ts_utc.tzinfo is None:
|
||||
ts_utc = ts_utc.replace(tzinfo=TZ_UTC)
|
||||
ts_cst = ts_utc.astimezone(TZ_CST)
|
||||
hour_cst = ts_cst.hour
|
||||
|
||||
model = row_dict['model_name']
|
||||
hourly_data[hour]["models"][model] = {
|
||||
"prediction": row_dict['prediction'],
|
||||
"probability": row_dict['probability'],
|
||||
"is_correct": row_dict['is_correct']
|
||||
hour_data = {
|
||||
"hour": hour_cst,
|
||||
"timestamp_et": row['timestamp_et'],
|
||||
"timestamp_cst": convert_to_cst(row.get('timestamp', '')),
|
||||
"open_price": row['open_price'],
|
||||
"close_price": row['close_price'],
|
||||
"actual_result": row['actual_result'],
|
||||
"models": {
|
||||
"DLinear": {
|
||||
"prediction": row['dlinear_prediction'],
|
||||
"probability": row['dlinear_prob'],
|
||||
"is_correct": row['dlinear_correct']
|
||||
},
|
||||
"N-BEATS": {
|
||||
"prediction": row['nbeats_prediction'],
|
||||
"probability": row['nbeats_prob'],
|
||||
"is_correct": row['nbeats_correct']
|
||||
},
|
||||
"TTM-R2": {
|
||||
"prediction": row['ttm_prediction'],
|
||||
"probability": row['ttm_prob'],
|
||||
"is_correct": row['ttm_correct']
|
||||
},
|
||||
"Ensemble": {
|
||||
"prediction": row['ensemble_prediction'],
|
||||
"is_correct": row['ensemble_correct'],
|
||||
"votes": row['ensemble_votes']
|
||||
}
|
||||
}
|
||||
}
|
||||
result.append(hour_data)
|
||||
|
||||
# 按小时排序返回
|
||||
return [hourly_data[h] for h in sorted(hourly_data.keys())]
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def get_missing_hours(date: str, model_name: str) -> List[int]:
|
||||
def save_prediction(data: Dict) -> int:
|
||||
"""兼容旧接口 - 如果需要的话,暂时保留,或者可以将旧表写入也放在这里"""
|
||||
# 旧表写入逻辑... 略,或保持不变
|
||||
# 为了完整性,我们可以保留对旧表的操作,或者逐步弃用。
|
||||
# 这里为了安全,保留原方法的签名,实际调用旧表逻辑。
|
||||
# 但我们主要逻辑将切换到 save_hourly_prediction
|
||||
# 见 prediction_service.py 的修改
|
||||
|
||||
# ... (keep old logic or redirect) ...
|
||||
# 这里我们假设 prediction_service.py 会改为调用 save_hourly_prediction
|
||||
# 如果还有地方调用 save_prediction (比如测试代码), 我们可以保留它
|
||||
return 0
|
||||
|
||||
@staticmethod
|
||||
def update_actual_result(date: str, hour: int, model_name: str,
|
||||
actual_result: int, close_price: float) -> bool:
|
||||
"""兼容旧接口"""
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def get_missing_hours(date: str, model_name: str = 'Ensemble') -> List[int]:
|
||||
"""
|
||||
获取指定日期缺少预测的小时列表
|
||||
|
||||
Args:
|
||||
date: 日期 (YYYY-MM-DD)
|
||||
model_name: 模型名称
|
||||
|
||||
Returns:
|
||||
缺少预测的小时列表
|
||||
获取指定日期缺少预测的小时列表 (V2)
|
||||
model_name 参数现在主要用于兼容,实际检查整行是否存在
|
||||
"""
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("""
|
||||
SELECT hour FROM model_predictions
|
||||
WHERE date = %s AND model_name = %s
|
||||
""", (date, model_name))
|
||||
SELECT hour FROM hourly_model_predictions
|
||||
WHERE date = %s
|
||||
""", (date,))
|
||||
|
||||
rows = cursor.fetchall()
|
||||
existing_hours = {row['hour'] for row in rows}
|
||||
|
||||
+23
-19
@@ -12,9 +12,13 @@ import httpx
|
||||
from typing import Optional, Dict, List, Any
|
||||
from dataclasses import dataclass
|
||||
from dotenv import load_dotenv
|
||||
from utils.network_proxy import apply_global_proxy_from_env, get_proxy_url
|
||||
from utils.project_logger import get_project_logger
|
||||
|
||||
# 加载环境变量
|
||||
load_dotenv()
|
||||
apply_global_proxy_from_env()
|
||||
logger = get_project_logger("api.polymarket_client")
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -46,7 +50,7 @@ class PolymarketClient:
|
||||
self.private_key = private_key or os.getenv("POLYMARKET_PRIVATE_KEY")
|
||||
|
||||
# 配置代理
|
||||
proxy = os.getenv("SOCKS5_PROXY") if os.getenv("USE_PROXY", "false").lower() == "true" else None
|
||||
proxy = get_proxy_url()
|
||||
|
||||
# HTTP客户端 - 禁用压缩避免brotli问题
|
||||
self.http = httpx.Client(
|
||||
@@ -79,9 +83,9 @@ class PolymarketClient:
|
||||
# 获取或创建API凭证
|
||||
self.clob_client.set_api_creds(self.clob_client.create_or_derive_api_creds())
|
||||
|
||||
print("✓ CLOB客户端初始化成功")
|
||||
logger.info("CLOB客户端初始化成功")
|
||||
except Exception as e:
|
||||
print(f"⚠️ CLOB客户端初始化失败: {e}")
|
||||
logger.warning("CLOB客户端初始化失败: %s", e)
|
||||
self.clob_client = None
|
||||
|
||||
def search_markets(self, query: str = "BTC", limit: int = 20) -> List[Dict]:
|
||||
@@ -111,7 +115,7 @@ class PolymarketClient:
|
||||
filtered = [m for m in markets if query.lower() in m.get("question", "").lower()]
|
||||
return filtered
|
||||
except Exception as e:
|
||||
print(f"搜索市场失败: {e}")
|
||||
logger.error("搜索市场失败: %s", e)
|
||||
return []
|
||||
|
||||
def search_btc_1h_markets(self) -> List[Dict]:
|
||||
@@ -150,7 +154,7 @@ class PolymarketClient:
|
||||
|
||||
return btc_1h_markets
|
||||
except Exception as e:
|
||||
print(f"搜索BTC 1H市场失败: {e}")
|
||||
logger.error("搜索BTC 1H市场失败: %s", e)
|
||||
return []
|
||||
|
||||
def get_market_info(self, condition_id: str) -> Optional[MarketInfo]:
|
||||
@@ -179,7 +183,7 @@ class PolymarketClient:
|
||||
end_date=data.get("endDate", ""),
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"获取市场信息失败: {e}")
|
||||
logger.error("获取市场信息失败: %s", e)
|
||||
return None
|
||||
|
||||
def get_orderbook(self, token_id: str) -> Dict:
|
||||
@@ -200,7 +204,7 @@ class PolymarketClient:
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
print(f"获取订单簿失败: {e}")
|
||||
logger.error("获取订单簿失败: %s", e)
|
||||
return {}
|
||||
|
||||
def get_price(self, token_id: str) -> Optional[float]:
|
||||
@@ -222,7 +226,7 @@ class PolymarketClient:
|
||||
data = response.json()
|
||||
return float(data.get("price", 0))
|
||||
except Exception as e:
|
||||
print(f"获取价格失败: {e}")
|
||||
logger.error("获取价格失败: %s", e)
|
||||
return None
|
||||
|
||||
def place_market_order(
|
||||
@@ -243,7 +247,7 @@ class PolymarketClient:
|
||||
订单响应
|
||||
"""
|
||||
if not self.clob_client:
|
||||
print("❌ CLOB客户端未初始化,无法下单")
|
||||
logger.error("CLOB客户端未初始化,无法下单")
|
||||
return None
|
||||
|
||||
try:
|
||||
@@ -260,10 +264,10 @@ class PolymarketClient:
|
||||
|
||||
# 提交订单
|
||||
response = self.clob_client.post_order(order)
|
||||
print(f"✓ 订单已提交: {response}")
|
||||
logger.info("订单已提交: %s", response)
|
||||
return response
|
||||
except Exception as e:
|
||||
print(f"❌ 下单失败: {e}")
|
||||
logger.error("下单失败: %s", e)
|
||||
return None
|
||||
|
||||
def place_limit_order(
|
||||
@@ -286,7 +290,7 @@ class PolymarketClient:
|
||||
订单响应
|
||||
"""
|
||||
if not self.clob_client:
|
||||
print("❌ CLOB客户端未初始化,无法下单")
|
||||
logger.error("CLOB客户端未初始化,无法下单")
|
||||
return None
|
||||
|
||||
try:
|
||||
@@ -304,10 +308,10 @@ class PolymarketClient:
|
||||
|
||||
# 提交订单
|
||||
response = self.clob_client.post_order(order)
|
||||
print(f"✓ 限价单已提交: {response}")
|
||||
logger.info("限价单已提交: %s", response)
|
||||
return response
|
||||
except Exception as e:
|
||||
print(f"❌ 下单失败: {e}")
|
||||
logger.error("下单失败: %s", e)
|
||||
return None
|
||||
|
||||
def get_positions(self) -> List[Dict]:
|
||||
@@ -318,14 +322,14 @@ class PolymarketClient:
|
||||
仓位列表
|
||||
"""
|
||||
if not self.clob_client:
|
||||
print("❌ CLOB客户端未初始化")
|
||||
logger.error("CLOB客户端未初始化")
|
||||
return []
|
||||
|
||||
try:
|
||||
positions = self.clob_client.get_positions()
|
||||
return positions
|
||||
except Exception as e:
|
||||
print(f"获取仓位失败: {e}")
|
||||
logger.error("获取仓位失败: %s", e)
|
||||
return []
|
||||
|
||||
def cancel_order(self, order_id: str) -> bool:
|
||||
@@ -339,15 +343,15 @@ class PolymarketClient:
|
||||
是否成功
|
||||
"""
|
||||
if not self.clob_client:
|
||||
print("❌ CLOB客户端未初始化")
|
||||
logger.error("CLOB客户端未初始化")
|
||||
return False
|
||||
|
||||
try:
|
||||
self.clob_client.cancel(order_id)
|
||||
print(f"✓ 订单已取消: {order_id}")
|
||||
logger.info("订单已取消: %s", order_id)
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"❌ 取消订单失败: {e}")
|
||||
logger.error("取消订单失败: %s", e)
|
||||
return False
|
||||
|
||||
def close(self):
|
||||
|
||||
+107
-256
@@ -1,147 +1,42 @@
|
||||
"""
|
||||
预测服务 - DLinear 和 TTM-R2 模型预测
|
||||
预测服务 - 统一模型预测 (DLinear, N-BEATS, TTM-R2)
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import pickle
|
||||
import threading
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Optional, List, Dict, Tuple
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import torch
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
import numpy as np
|
||||
|
||||
# 添加项目路径
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
from api.models import PredictionModel
|
||||
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
|
||||
def __init__(self, lookback: int = 168, forecast_horizon: int = 1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df: pd.DataFrame) -> pd.DataFrame:
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df: pd.DataFrame) -> pd.DataFrame:
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df: pd.DataFrame, fit: bool = True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
class CustomUnpickler(pickle.Unpickler):
|
||||
"""自定义反序列化器,用于处理 __main__ 模块的类"""
|
||||
def find_class(self, module, name):
|
||||
if module == "__main__":
|
||||
if name == "ClassificationPreprocessor":
|
||||
return ClassificationPreprocessor
|
||||
return super().find_class(module, name)
|
||||
from models.inference import get_inference_manager
|
||||
|
||||
|
||||
class PredictionService:
|
||||
"""预测服务 - 管理 DLinear 和 TTM-R2 模型"""
|
||||
"""预测服务 - 管理多模型预测"""
|
||||
|
||||
# 类级别的锁,确保多个请求不会同时生成预测
|
||||
_prediction_lock = None
|
||||
_prediction_lock = threading.Lock()
|
||||
|
||||
def __init__(self):
|
||||
import threading
|
||||
# 初始化锁
|
||||
if PredictionService._prediction_lock is None:
|
||||
PredictionService._prediction_lock = threading.Lock()
|
||||
|
||||
self.device = torch.device('mps' if torch.backends.mps.is_available() else
|
||||
'cuda' if torch.cuda.is_available() else 'cpu')
|
||||
self.et_tz = ZoneInfo('America/New_York')
|
||||
self.fetcher = BinanceDataFetcher()
|
||||
|
||||
# 模型和预处理器
|
||||
self.dlinear_model = None
|
||||
self.dlinear_preprocessor = None
|
||||
self.ttm_model = None
|
||||
self.inference_manager = get_inference_manager()
|
||||
|
||||
# 记录正在处理的日期和月份,避免重复处理
|
||||
self._processing_dates = set()
|
||||
self._processing_months = set()
|
||||
|
||||
self._load_models()
|
||||
|
||||
def _load_models(self):
|
||||
"""加载模型"""
|
||||
print("正在加载预测模型...")
|
||||
|
||||
# 加载 DLinear
|
||||
from models.dlinear.model import DLinear
|
||||
|
||||
dlinear_path = './models/checkpoints/dlinear_classification_2026/best_model.pt'
|
||||
preprocessor_path = './models/checkpoints/dlinear_classification_2026/preprocessor.pkl'
|
||||
|
||||
checkpoint = torch.load(dlinear_path, map_location=self.device, weights_only=False)
|
||||
self.dlinear_model = DLinear(checkpoint['config'])
|
||||
self.dlinear_model.load_state_dict(checkpoint['model_state_dict'])
|
||||
self.dlinear_model = self.dlinear_model.to(self.device).eval()
|
||||
|
||||
with open(preprocessor_path, 'rb') as f:
|
||||
try:
|
||||
self.dlinear_preprocessor = pickle.load(f)
|
||||
except AttributeError:
|
||||
f.seek(0)
|
||||
self.dlinear_preprocessor = CustomUnpickler(f).load()
|
||||
|
||||
print(f" ✓ DLinear 已加载")
|
||||
|
||||
# 加载 TTM-R2
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
|
||||
self.ttm_model = TinyTimeMixerForPrediction.from_pretrained(
|
||||
"ibm-granite/granite-timeseries-ttm-r2",
|
||||
trust_remote_code=True
|
||||
)
|
||||
self.ttm_model = self.ttm_model.to(self.device).eval()
|
||||
|
||||
print(f" ✓ TTM-R2 已加载")
|
||||
print(f" 设备: {self.device}")
|
||||
# 预加载模型
|
||||
print("正在预加载预测模型...")
|
||||
self.inference_manager.load_all_models()
|
||||
|
||||
def _get_btc_data(self, days: int = 30) -> pd.DataFrame:
|
||||
"""获取BTC历史数据"""
|
||||
@@ -158,83 +53,22 @@ class PredictionService:
|
||||
return df
|
||||
|
||||
def predict_dlinear(self, df: pd.DataFrame) -> Tuple[int, float]:
|
||||
"""
|
||||
使用 DLinear 预测下一小时的涨跌
|
||||
|
||||
输入: df 包含历史K线数据 (最后一行是最新的已完成K线)
|
||||
输出: 预测下一小时相对于 df 最后一行收盘价的涨跌
|
||||
|
||||
Returns:
|
||||
(prediction, probability): 预测结果(0=跌, 1=涨)和概率
|
||||
"""
|
||||
# 预处理: 使用全部数据,不去掉最后一行
|
||||
X, _, _ = self.dlinear_preprocessor.preprocess(df, fit=False)
|
||||
|
||||
if len(X) == 0:
|
||||
"""DLinear 预测代理"""
|
||||
pred, prob = self.inference_manager.predict_dlinear(df)
|
||||
if pred is None:
|
||||
return 0, 0.5
|
||||
|
||||
# 取最后一个窗口进行预测
|
||||
X_last = torch.FloatTensor(X[-1:]).to(self.device)
|
||||
|
||||
with torch.no_grad():
|
||||
logits = self.dlinear_model(X_last).cpu().numpy()
|
||||
prob = 1 / (1 + np.exp(-logits))
|
||||
prediction = int(prob.item() > 0.5)
|
||||
probability = float(prob.item())
|
||||
|
||||
return prediction, probability
|
||||
return pred, prob if prob is not None else 0.5
|
||||
|
||||
def predict_ttm(self, df: pd.DataFrame) -> Tuple[int, float]:
|
||||
"""
|
||||
使用 TTM-R2 预测下一小时的涨跌
|
||||
|
||||
输入: df 包含历史K线数据 (最后一行是最新的已完成K线)
|
||||
输出: 预测下一小时相对于 df 最后一行收盘价的涨跌
|
||||
|
||||
Returns:
|
||||
(prediction, probability): 预测结果(0=跌, 1=涨)和概率
|
||||
"""
|
||||
context_length = 512
|
||||
all_prices = df['close'].values
|
||||
|
||||
# 使用最后 context_length 个价格
|
||||
if len(all_prices) >= context_length:
|
||||
context_data = all_prices[-context_length:].reshape(1, -1, 1)
|
||||
else:
|
||||
# 数据不足,用最后一个价格填充
|
||||
padded = np.full(context_length, all_prices[-1])
|
||||
padded[-len(all_prices):] = all_prices
|
||||
context_data = padded.reshape(1, -1, 1)
|
||||
|
||||
# 基准价格: 最后一行的收盘价 (即预测目标小时的前一小时收盘价)
|
||||
base_price = float(df['close'].iloc[-1])
|
||||
|
||||
context_tensor = torch.FloatTensor(context_data).to(self.device)
|
||||
|
||||
with torch.no_grad():
|
||||
output = self.ttm_model(context_tensor)
|
||||
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
|
||||
|
||||
# 判断涨跌: 预测价格 vs 基准价格
|
||||
prediction = 1 if pred_price > base_price else 0
|
||||
|
||||
# 计算伪概率: 基于价格变化百分比
|
||||
price_pct = (pred_price - base_price) / base_price
|
||||
# 将价格变化百分比映射到概率 (sigmoid)
|
||||
probability = 1 / (1 + np.exp(-price_pct * 100))
|
||||
|
||||
return prediction, float(probability)
|
||||
"""TTM-R2 预测代理"""
|
||||
pred, prob = self.inference_manager.predict_ttm(df)
|
||||
if pred is None:
|
||||
return 0, 0.5
|
||||
return pred, prob if prob is not None else 0.5
|
||||
|
||||
def predict_current_hour(self) -> Dict:
|
||||
"""
|
||||
预测当前小时的涨跌
|
||||
|
||||
逻辑:
|
||||
- 获取最新的K线数据 (最后一行是最新已完成的K线)
|
||||
- 预测下一小时相对于最后一行收盘价的涨跌
|
||||
|
||||
Returns:
|
||||
预测结果字典
|
||||
预测当前小时的涨跌 (多模型集成)
|
||||
"""
|
||||
df = self._get_btc_data()
|
||||
|
||||
@@ -248,7 +82,6 @@ class PredictionService:
|
||||
last_close = float(df['close'].iloc[-1])
|
||||
|
||||
# 当前正在形成的K线信息
|
||||
current_open = float(df['open'].iloc[-1])
|
||||
current_price = float(df['close'].iloc[-1]) # 当前价格
|
||||
|
||||
results = {
|
||||
@@ -260,49 +93,62 @@ class PredictionService:
|
||||
"models": {}
|
||||
}
|
||||
|
||||
# DLinear 预测 (预测下一小时相对于 last_close 的涨跌)
|
||||
dlinear_pred, dlinear_prob = self.predict_dlinear(df)
|
||||
# 使用 InferenceManager 获取所有预测
|
||||
predictions = self.inference_manager.predict_all(df)
|
||||
|
||||
# 填充结果
|
||||
if 'dlinear' in predictions:
|
||||
results["models"]["DLinear"] = {
|
||||
"prediction": dlinear_pred,
|
||||
"probability": round(dlinear_prob, 4),
|
||||
"direction": "UP" if dlinear_pred == 1 else "DOWN"
|
||||
"prediction": predictions['dlinear']['prediction'],
|
||||
"probability": round(predictions['dlinear']['probability'], 4),
|
||||
"direction": "UP" if predictions['dlinear']['prediction'] == 1 else "DOWN"
|
||||
}
|
||||
|
||||
# TTM-R2 预测
|
||||
ttm_pred, ttm_prob = self.predict_ttm(df)
|
||||
if 'nbeats' in predictions:
|
||||
results["models"]["N-BEATS"] = {
|
||||
"prediction": predictions['nbeats']['prediction'],
|
||||
"probability": round(predictions['nbeats']['probability'], 4),
|
||||
"direction": "UP" if predictions['nbeats']['prediction'] == 1 else "DOWN"
|
||||
}
|
||||
|
||||
if 'ttm' in predictions:
|
||||
results["models"]["TTM-R2"] = {
|
||||
"prediction": ttm_pred,
|
||||
"probability": round(ttm_prob, 4),
|
||||
"direction": "UP" if ttm_pred == 1 else "DOWN"
|
||||
"prediction": predictions['ttm']['prediction'],
|
||||
"probability": round(predictions['ttm']['probability'], 4),
|
||||
"direction": "UP" if predictions['ttm']['prediction'] == 1 else "DOWN"
|
||||
}
|
||||
|
||||
if 'ensemble' in predictions and predictions['ensemble']:
|
||||
ens_pred = predictions['ensemble']['prediction']
|
||||
results["models"]["Ensemble"] = {
|
||||
"prediction": ens_pred,
|
||||
"probability": 0.0, # 集成暂无概率
|
||||
"direction": "UP" if ens_pred == 1 else "DOWN",
|
||||
"votes": predictions['ensemble']['votes']
|
||||
}
|
||||
|
||||
return results
|
||||
|
||||
def ensure_predictions_for_date(self, date: str) -> None:
|
||||
"""
|
||||
确保指定日期有预测数据,如果没有则生成
|
||||
|
||||
确保指定日期有预测数据,如果没有则生成 (V2)
|
||||
Args:
|
||||
date: 日期 (YYYY-MM-DD)
|
||||
"""
|
||||
# 使用锁避免并发冲突
|
||||
with PredictionService._prediction_lock:
|
||||
# 检查是否正在处理该日期
|
||||
if date in self._processing_dates:
|
||||
return
|
||||
|
||||
# 标记正在处理
|
||||
self._processing_dates.add(date)
|
||||
|
||||
try:
|
||||
# 检查是否有缺失的小时
|
||||
for model_name in ["DLinear", "TTM-R2"]:
|
||||
missing_hours = PredictionModel.get_missing_hours(date, model_name)
|
||||
# 检查是否有缺失的小时 (只检查宽表是否有记录)
|
||||
missing_hours = PredictionModel.get_missing_hours(date)
|
||||
|
||||
if not missing_hours:
|
||||
continue
|
||||
return
|
||||
|
||||
print(f"为 {date} 生成 {model_name} 预测 (缺失 {len(missing_hours)} 小时)...")
|
||||
print(f"为 {date} 生成预测 (缺失 {len(missing_hours)} 小时)...")
|
||||
|
||||
# 获取足够的历史数据
|
||||
target_date = datetime.strptime(date, '%Y-%m-%d')
|
||||
@@ -310,13 +156,12 @@ class PredictionService:
|
||||
|
||||
if df is None or len(df) < 200:
|
||||
print(f" 警告: 数据不足,无法为 {date} 生成预测")
|
||||
continue
|
||||
return
|
||||
|
||||
# 为每个缺失的小时生成预测
|
||||
for hour in missing_hours:
|
||||
self._generate_prediction_for_hour(df, date, hour, model_name)
|
||||
self._generate_prediction_for_hour(df, date, hour)
|
||||
finally:
|
||||
# 移除处理标记
|
||||
with PredictionService._prediction_lock:
|
||||
self._processing_dates.discard(date)
|
||||
|
||||
@@ -338,22 +183,9 @@ class PredictionService:
|
||||
print(f"获取历史数据失败: {e}")
|
||||
return None
|
||||
|
||||
def _generate_prediction_for_hour(self, df: pd.DataFrame, date: str,
|
||||
hour: int, model_name: str) -> None:
|
||||
def _generate_prediction_for_hour(self, df: pd.DataFrame, date: str, hour: int) -> None:
|
||||
"""
|
||||
为指定小时生成预测
|
||||
|
||||
预测逻辑:
|
||||
- 预测目标: {date} {hour}:00 的K线收盘价涨跌
|
||||
- 输入数据: {hour-1}:00 及以前的K线数据 (不包含目标小时)
|
||||
- 涨跌判断: 预测价格 vs 前一小时收盘价 (即 {hour-1}:00 的 close)
|
||||
- 正确率验证: 目标小时的 close vs 前一小时的 close
|
||||
|
||||
Args:
|
||||
df: 历史K线数据 (UTC时间戳)
|
||||
date: 目标日期 (美东时间 YYYY-MM-DD)
|
||||
hour: 目标小时 (美东时间 0-23)
|
||||
model_name: 模型名称
|
||||
为指定小时生成预测 (V2)
|
||||
"""
|
||||
try:
|
||||
# 1. 构建目标时间 (美东时间 -> UTC)
|
||||
@@ -368,7 +200,6 @@ class PredictionService:
|
||||
# 3. 找到目标小时的K线 (用于验证)
|
||||
target_mask = df['timestamp_dt'] == target_time_utc
|
||||
if not target_mask.any():
|
||||
# 目标小时数据不存在,跳过
|
||||
return
|
||||
|
||||
target_row = df[target_mask].iloc[0]
|
||||
@@ -379,7 +210,6 @@ class PredictionService:
|
||||
prev_time_utc = target_time_utc - pd.Timedelta(hours=1)
|
||||
prev_mask = df['timestamp_dt'] == prev_time_utc
|
||||
if not prev_mask.any():
|
||||
# 前一小时数据不存在,跳过
|
||||
return
|
||||
|
||||
prev_row = df[prev_mask].iloc[0]
|
||||
@@ -390,77 +220,98 @@ class PredictionService:
|
||||
df_input = df[input_mask].copy()
|
||||
|
||||
if len(df_input) < 200:
|
||||
# 数据不足,跳过
|
||||
return
|
||||
|
||||
# 6. 调用模型预测
|
||||
if model_name == "DLinear":
|
||||
prediction, probability = self.predict_dlinear(df_input)
|
||||
else: # TTM-R2
|
||||
prediction, probability = self.predict_ttm(df_input)
|
||||
# 6. 调用多模型预测
|
||||
preds = self.inference_manager.predict_all(df_input)
|
||||
|
||||
# 7. 判断实际涨跌
|
||||
# Check if the candle is closed (current time > candle open time + 1 hour)
|
||||
current_now_utc = datetime.now(ZoneInfo('UTC')).replace(tzinfo=None)
|
||||
candle_close_time = target_time_utc + timedelta(hours=1)
|
||||
|
||||
is_closed = current_now_utc >= candle_close_time
|
||||
|
||||
if not is_closed:
|
||||
# 仅保存已完成的K线预测,未完成的跳过 (等待完成后再生成)
|
||||
# 实时交易逻辑在 auto_trade_polymarket.py 中单独处理
|
||||
return
|
||||
|
||||
# 7. 判断实际涨跌 (目标小时收盘价 vs 前一小时收盘价)
|
||||
actual_result = 1 if target_close > prev_close else 0
|
||||
|
||||
# 8. 判断预测是否正确
|
||||
is_correct = 1 if prediction == actual_result else 0
|
||||
|
||||
# 9. 保存预测结果
|
||||
# 8. 组装数据 (宽表格式)
|
||||
prediction_data = {
|
||||
"timestamp": target_time_utc.isoformat(),
|
||||
"timestamp_et": target_time_et.isoformat(),
|
||||
"date": date,
|
||||
"hour": hour,
|
||||
"model_name": model_name,
|
||||
"prediction": prediction,
|
||||
"probability": probability,
|
||||
"actual_result": actual_result,
|
||||
"is_correct": is_correct,
|
||||
"open_price": target_open,
|
||||
"close_price": target_close
|
||||
"close_price": target_close,
|
||||
"actual_result": actual_result,
|
||||
|
||||
# DLinear
|
||||
"dlinear_prediction": preds.get('dlinear', {}).get('prediction'),
|
||||
"dlinear_prob": preds.get('dlinear', {}).get('probability'),
|
||||
"dlinear_correct": 1 if preds.get('dlinear', {}).get('prediction') == actual_result else 0,
|
||||
|
||||
# N-BEATS
|
||||
"nbeats_prediction": preds.get('nbeats', {}).get('prediction'),
|
||||
"nbeats_prob": preds.get('nbeats', {}).get('probability'),
|
||||
"nbeats_correct": 1 if preds.get('nbeats', {}).get('prediction') == actual_result else 0,
|
||||
|
||||
# TTM-R2
|
||||
"ttm_prediction": preds.get('ttm', {}).get('prediction'),
|
||||
"ttm_prob": preds.get('ttm', {}).get('probability'),
|
||||
"ttm_correct": 1 if preds.get('ttm', {}).get('prediction') == actual_result else 0,
|
||||
|
||||
# Ensemble
|
||||
"ensemble_prediction": preds.get('ensemble', {}).get('prediction'),
|
||||
"ensemble_votes": preds.get('ensemble', {}).get('votes'),
|
||||
"ensemble_correct": 1 if preds.get('ensemble', {}).get('prediction') == actual_result else 0,
|
||||
}
|
||||
|
||||
PredictionModel.save_prediction(prediction_data)
|
||||
PredictionModel.save_hourly_prediction(prediction_data)
|
||||
|
||||
except Exception as e:
|
||||
print(f"生成预测失败 ({date} {hour}:00 {model_name}): {e}")
|
||||
print(f"生成预测失败 ({date} {hour}:00): {e}")
|
||||
|
||||
def ensure_predictions_for_month(self, month: str) -> None:
|
||||
"""
|
||||
确保指定月份有预测数据
|
||||
确保指定月份有预测数据 (CST)
|
||||
需要检查:
|
||||
1. 该月的所有日期 (ET)
|
||||
2. 上个月的最后一天 (ET) -> 因为 ET 上个月最后一天 11:00-23:00 对应 CST 该月第一天 00:00-12:00
|
||||
|
||||
Args:
|
||||
month: 月份 (YYYY-MM)
|
||||
"""
|
||||
# 使用锁避免并发冲突
|
||||
with PredictionService._prediction_lock:
|
||||
# 检查是否正在处理该月份
|
||||
if month in self._processing_months:
|
||||
return
|
||||
|
||||
# 标记正在处理
|
||||
self._processing_months.add(month)
|
||||
|
||||
try:
|
||||
# 解析月份
|
||||
year, mon = map(int, month.split('-'))
|
||||
|
||||
# 计算该月的日期范围
|
||||
from calendar import monthrange
|
||||
_, last_day = monthrange(year, mon)
|
||||
|
||||
# 对于过去的日期,确保有预测数据
|
||||
# 1. 确保上个月最后一天有数据 (为了 CST 的开头)
|
||||
first_day_of_month = datetime(year, mon, 1).date()
|
||||
prev_day = first_day_of_month - timedelta(days=1)
|
||||
prev_day_str = prev_day.strftime('%Y-%m-%d')
|
||||
self.ensure_predictions_for_date(prev_day_str)
|
||||
|
||||
today = datetime.now(self.et_tz).date()
|
||||
|
||||
# 2. 确保本月所有日期
|
||||
for day in range(1, last_day + 1):
|
||||
date = f"{year}-{mon:02d}-{day:02d}"
|
||||
date_obj = datetime.strptime(date, '%Y-%m-%d').date()
|
||||
|
||||
# 只处理过去和今天的日期
|
||||
if date_obj <= today:
|
||||
self.ensure_predictions_for_date(date)
|
||||
finally:
|
||||
# 移除处理标记
|
||||
with PredictionService._prediction_lock:
|
||||
self._processing_months.discard(month)
|
||||
|
||||
|
||||
+132
-78
@@ -74,8 +74,8 @@
|
||||
</div>
|
||||
<div class="col-md-2">
|
||||
<div class="stat-card">
|
||||
<div class="stat-label">价格</div>
|
||||
<div class="stat-value">${{ currentTrade.price?.toFixed(3) || '-' }}</div>
|
||||
<div class="stat-label">策略</div>
|
||||
<div class="stat-value small">{{ currentTrade.strategy || 'Ensemble' }}</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-3">
|
||||
@@ -116,6 +116,7 @@
|
||||
<tr>
|
||||
<th>时间 (北京)</th>
|
||||
<th>市场</th>
|
||||
<th>策略</th>
|
||||
<th>预测</th>
|
||||
<th>概率</th>
|
||||
<th>价格</th>
|
||||
@@ -128,6 +129,9 @@
|
||||
<td class="text-truncate" style="max-width: 200px;">
|
||||
{{ trade.market_title }}
|
||||
</td>
|
||||
<td>
|
||||
<span class="badge bg-secondary">{{ trade.strategy || 'Ensemble' }}</span>
|
||||
</td>
|
||||
<td :class="getPredictionClass(trade.prediction)">
|
||||
<i :class="trade.prediction === 'UP' ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
|
||||
{{ trade.prediction }}
|
||||
@@ -139,7 +143,7 @@
|
||||
</td>
|
||||
</tr>
|
||||
<tr v-if="!tradeHistory.data?.length">
|
||||
<td colspan="6" class="text-center text-muted py-4">
|
||||
<td colspan="7" class="text-center text-muted py-4">
|
||||
暂无数据
|
||||
</td>
|
||||
</tr>
|
||||
@@ -184,66 +188,80 @@
|
||||
|
||||
<!-- 当月汇总卡片 -->
|
||||
<div class="row mb-4">
|
||||
<!-- DLinear 卡片 -->
|
||||
<div class="col-md-6">
|
||||
<div class="card h-100 border-primary">
|
||||
<div class="card-header bg-primary text-white">
|
||||
<i class="bi bi-graph-up-arrow"></i> DLinear 模型
|
||||
<!-- Ensemble 卡片 -->
|
||||
<div class="col-md-3">
|
||||
<div class="card h-100 border-primary shadow-sm">
|
||||
<div class="card-header bg-primary text-white text-center">
|
||||
<i class="bi bi-diagram-3"></i> 集成策略 (投票)
|
||||
</div>
|
||||
<div class="card-body">
|
||||
<div class="row text-center" v-if="monthlyStats.DLinear">
|
||||
<div class="col-3">
|
||||
<div class="stat-number">{{ monthlyStats.DLinear.total }}</div>
|
||||
<div class="stat-label">预测次数</div>
|
||||
</div>
|
||||
<div class="col-3">
|
||||
<div class="stat-number text-success">{{ monthlyStats.DLinear.correct }}</div>
|
||||
<div class="stat-label">正确</div>
|
||||
</div>
|
||||
<div class="col-3">
|
||||
<div class="stat-number text-danger">{{ monthlyStats.DLinear.incorrect }}</div>
|
||||
<div class="stat-label">错误</div>
|
||||
</div>
|
||||
<div class="col-3">
|
||||
<div class="stat-number text-primary">{{ monthlyStats.DLinear.accuracy }}%</div>
|
||||
<div class="stat-label">准确率</div>
|
||||
<div class="text-center" v-if="monthlyStats.Ensemble">
|
||||
<h2 class="display-6 text-primary fw-bold">{{ monthlyStats.Ensemble.accuracy }}%</h2>
|
||||
<div class="text-muted mb-2">准确率</div>
|
||||
<div class="d-flex justify-content-between px-3 border-top pt-2">
|
||||
<span class="text-success"><i class="bi bi-check-lg"></i> {{ monthlyStats.Ensemble.correct }}</span>
|
||||
<span class="text-secondary">{{ monthlyStats.Ensemble.total }} 次</span>
|
||||
<span class="text-danger"><i class="bi bi-x-lg"></i> {{ monthlyStats.Ensemble.incorrect }}</span>
|
||||
</div>
|
||||
</div>
|
||||
<div v-else class="text-center text-muted py-3">
|
||||
暂无数据
|
||||
<div v-else class="text-center text-muted py-3">暂无数据</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- DLinear 卡片 -->
|
||||
<div class="col-md-3">
|
||||
<div class="card h-100 border-secondary">
|
||||
<div class="card-header bg-secondary text-white text-center">
|
||||
<i class="bi bi-graph-up-arrow"></i> DLinear
|
||||
</div>
|
||||
<div class="card-body">
|
||||
<div class="text-center" v-if="monthlyStats.DLinear">
|
||||
<h3 class="text-dark">{{ monthlyStats.DLinear.accuracy }}%</h3>
|
||||
<div class="text-muted small">准确率</div>
|
||||
<div class="mt-2 small text-muted">
|
||||
{{ monthlyStats.DLinear.correct }}/{{ monthlyStats.DLinear.total }}
|
||||
</div>
|
||||
</div>
|
||||
<div v-else class="text-center text-muted py-3">暂无数据</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- N-BEATS 卡片 -->
|
||||
<div class="col-md-3">
|
||||
<div class="card h-100 border-secondary">
|
||||
<div class="card-header bg-secondary text-white text-center">
|
||||
<i class="bi bi-soundwave"></i> N-BEATS
|
||||
</div>
|
||||
<div class="card-body">
|
||||
<div class="text-center" v-if="monthlyStats['N-BEATS']">
|
||||
<h3 class="text-dark">{{ monthlyStats['N-BEATS'].accuracy }}%</h3>
|
||||
<div class="text-muted small">准确率</div>
|
||||
<div class="mt-2 small text-muted">
|
||||
{{ monthlyStats['N-BEATS'].correct }}/{{ monthlyStats['N-BEATS'].total }}
|
||||
</div>
|
||||
</div>
|
||||
<div v-else class="text-center text-muted py-3">暂无数据</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- TTM-R2 卡片 -->
|
||||
<div class="col-md-6">
|
||||
<div class="card h-100 border-success">
|
||||
<div class="card-header bg-success text-white">
|
||||
<i class="bi bi-cpu"></i> TTM-R2 模型
|
||||
<div class="col-md-3">
|
||||
<div class="card h-100 border-secondary">
|
||||
<div class="card-header bg-secondary text-white text-center">
|
||||
<i class="bi bi-cpu"></i> TTM-R2
|
||||
</div>
|
||||
<div class="card-body">
|
||||
<div class="row text-center" v-if="monthlyStats['TTM-R2']">
|
||||
<div class="col-3">
|
||||
<div class="stat-number">{{ monthlyStats['TTM-R2'].total }}</div>
|
||||
<div class="stat-label">预测次数</div>
|
||||
</div>
|
||||
<div class="col-3">
|
||||
<div class="stat-number text-success">{{ monthlyStats['TTM-R2'].correct }}</div>
|
||||
<div class="stat-label">正确</div>
|
||||
</div>
|
||||
<div class="col-3">
|
||||
<div class="stat-number text-danger">{{ monthlyStats['TTM-R2'].incorrect }}</div>
|
||||
<div class="stat-label">错误</div>
|
||||
</div>
|
||||
<div class="col-3">
|
||||
<div class="stat-number text-primary">{{ monthlyStats['TTM-R2'].accuracy }}%</div>
|
||||
<div class="stat-label">准确率</div>
|
||||
<div class="text-center" v-if="monthlyStats['TTM-R2']">
|
||||
<h3 class="text-dark">{{ monthlyStats['TTM-R2'].accuracy }}%</h3>
|
||||
<div class="text-muted small">准确率</div>
|
||||
<div class="mt-2 small text-muted">
|
||||
{{ monthlyStats['TTM-R2'].correct }}/{{ monthlyStats['TTM-R2'].total }}
|
||||
</div>
|
||||
</div>
|
||||
<div v-else class="text-center text-muted py-3">
|
||||
暂无数据
|
||||
</div>
|
||||
<div v-else class="text-center text-muted py-3">暂无数据</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -252,52 +270,61 @@
|
||||
<!-- 每日准确率表格 -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<i class="bi bi-calendar3"></i> 每日准确率
|
||||
<i class="bi bi-calendar3"></i> 每日准确率概览
|
||||
</div>
|
||||
<div class="card-body p-0">
|
||||
<div class="table-responsive">
|
||||
<table class="table table-hover mb-0">
|
||||
<table class="table table-hover mb-0 text-center">
|
||||
<thead class="table-light">
|
||||
<tr>
|
||||
<th>日期</th>
|
||||
<th>DLinear 预测</th>
|
||||
<th>DLinear 准确率</th>
|
||||
<th>TTM-R2 预测</th>
|
||||
<th>TTM-R2 准确率</th>
|
||||
<th class="text-start ps-4">日期</th>
|
||||
<th>集成策略</th>
|
||||
<th>DLinear</th>
|
||||
<th>N-BEATS</th>
|
||||
<th>TTM-R2</th>
|
||||
<th>操作</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr v-for="day in dailyStats.data" :key="day.date">
|
||||
<td>{{ day.date }}</td>
|
||||
<td class="text-start ps-4">{{ day.date }}</td>
|
||||
|
||||
<!-- Ensemble -->
|
||||
<td>
|
||||
<span v-if="day.DLinear">
|
||||
{{ day.DLinear.correct }}/{{ day.DLinear.total }}
|
||||
<span v-if="day.Ensemble" :class="getAccuracyClass(day.Ensemble.accuracy)">
|
||||
{{ day.Ensemble.accuracy }}%
|
||||
</span>
|
||||
<span v-else class="text-muted">-</span>
|
||||
</td>
|
||||
|
||||
<!-- DLinear -->
|
||||
<td>
|
||||
<span v-if="day.DLinear" :class="getAccuracyClass(day.DLinear.accuracy)">
|
||||
{{ day.DLinear.accuracy }}%
|
||||
</span>
|
||||
<span v-else class="text-muted">-</span>
|
||||
</td>
|
||||
|
||||
<!-- N-BEATS -->
|
||||
<td>
|
||||
<span v-if="day['TTM-R2']">
|
||||
{{ day['TTM-R2'].correct }}/{{ day['TTM-R2'].total }}
|
||||
<span v-if="day['N-BEATS']" :class="getAccuracyClass(day['N-BEATS'].accuracy)">
|
||||
{{ day['N-BEATS'].accuracy }}%
|
||||
</span>
|
||||
<span v-else class="text-muted">-</span>
|
||||
</td>
|
||||
|
||||
<!-- TTM-R2 -->
|
||||
<td>
|
||||
<span v-if="day['TTM-R2']" :class="getAccuracyClass(day['TTM-R2'].accuracy)">
|
||||
{{ day['TTM-R2'].accuracy }}%
|
||||
</span>
|
||||
<span v-else class="text-muted">-</span>
|
||||
</td>
|
||||
|
||||
<td>
|
||||
<button class="btn btn-sm btn-outline-primary"
|
||||
@click="showHourlyDetails(day.date)">
|
||||
<i class="bi bi-eye"></i> 查看
|
||||
<i class="bi bi-eye"></i> 详情
|
||||
</button>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -337,52 +364,79 @@
|
||||
|
||||
<!-- 每小时详情弹窗 -->
|
||||
<div class="modal fade" id="hourlyModal" tabindex="-1">
|
||||
<div class="modal-dialog modal-lg modal-dialog-scrollable">
|
||||
<div class="modal-dialog modal-xl modal-dialog-scrollable">
|
||||
<div class="modal-content">
|
||||
<div class="modal-header">
|
||||
<h5 class="modal-title">
|
||||
<i class="bi bi-clock"></i> {{ hourlyModal.date }} 每小时预测详情
|
||||
<i class="bi bi-clock"></i> {{ hourlyModal.date }} 每小时预测详情 (北京时间)
|
||||
</h5>
|
||||
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
|
||||
</div>
|
||||
<div class="modal-body p-0">
|
||||
<table class="table table-sm table-hover mb-0">
|
||||
<table class="table table-sm table-hover mb-0 text-center align-middle">
|
||||
<thead class="table-light sticky-top">
|
||||
<tr>
|
||||
<th>时间 (北京)</th>
|
||||
<th>实际涨跌</th>
|
||||
<th>时间</th>
|
||||
<th>开盘</th>
|
||||
<th>收盘</th>
|
||||
<th>实际</th>
|
||||
<th class="table-primary">集成策略</th>
|
||||
<th>DLinear</th>
|
||||
<th>N-BEATS</th>
|
||||
<th>TTM-R2</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr v-for="hour in hourlyModal.data" :key="hour.hour">
|
||||
<td>{{ hour.timestamp_cst || (hour.hour + ':00') }}</td>
|
||||
<td :class="hour.actual_result === 1 ? 'text-success' : 'text-danger'">
|
||||
<i :class="hour.actual_result === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
|
||||
{{ hour.actual_result === 1 ? 'UP' : 'DOWN' }}
|
||||
<td>{{ hour.timestamp_cst ? hour.timestamp_cst.split(' ')[1] : (hour.hour + ':00') }}</td>
|
||||
<td>${{ hour.open_price?.toFixed(0) || '-' }}</td>
|
||||
<td>${{ hour.close_price?.toFixed(0) || '-' }}</td>
|
||||
|
||||
<td :class="hour.actual_result === 1 ? 'text-success fw-bold' : (hour.actual_result === 0 ? 'text-danger fw-bold' : 'text-muted')">
|
||||
<i v-if="hour.actual_result !== null" :class="hour.actual_result === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
|
||||
{{ hour.actual_result === 1 ? 'UP' : (hour.actual_result === 0 ? 'DOWN' : '-') }}
|
||||
</td>
|
||||
|
||||
<!-- Ensemble -->
|
||||
<td class="table-primary bg-opacity-10">
|
||||
<span v-if="hour.models?.Ensemble"
|
||||
:class="hour.models.Ensemble.is_correct ? 'text-success' : 'text-danger'">
|
||||
<i :class="hour.models.Ensemble.prediction === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
|
||||
{{ hour.models.Ensemble.prediction === 1 ? 'UP' : 'DOWN' }}
|
||||
<i v-if="hour.actual_result !== null" :class="hour.models.Ensemble.is_correct ? 'bi-check-circle-fill' : 'bi-x-circle-fill'"></i>
|
||||
</span>
|
||||
</td>
|
||||
|
||||
<!-- DLinear -->
|
||||
<td>
|
||||
<span v-if="hour.models?.DLinear"
|
||||
:class="hour.models.DLinear.is_correct ? 'text-success' : 'text-danger'">
|
||||
<i :class="hour.models.DLinear.prediction === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
|
||||
{{ hour.models.DLinear.prediction === 1 ? 'UP' : 'DOWN' }}
|
||||
<i :class="hour.models.DLinear.is_correct ? 'bi-check-circle' : 'bi-x-circle'"></i>
|
||||
<span class="small text-muted">({{ (hour.models.DLinear.probability * 100).toFixed(0) }}%)</span>
|
||||
<i v-if="hour.actual_result !== null" :class="hour.models.DLinear.is_correct ? 'bi-check' : 'bi-x'"></i>
|
||||
</span>
|
||||
<span v-else class="text-muted">-</span>
|
||||
</td>
|
||||
|
||||
<!-- N-BEATS -->
|
||||
<td>
|
||||
<span v-if="hour.models?.['N-BEATS']"
|
||||
:class="hour.models['N-BEATS'].is_correct ? 'text-success' : 'text-danger'">
|
||||
<i :class="hour.models['N-BEATS'].prediction === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
|
||||
<i v-if="hour.actual_result !== null" :class="hour.models['N-BEATS'].is_correct ? 'bi-check' : 'bi-x'"></i>
|
||||
</span>
|
||||
</td>
|
||||
|
||||
<!-- TTM-R2 -->
|
||||
<td>
|
||||
<span v-if="hour.models?.['TTM-R2']"
|
||||
:class="hour.models['TTM-R2'].is_correct ? 'text-success' : 'text-danger'">
|
||||
<i :class="hour.models['TTM-R2'].prediction === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
|
||||
{{ hour.models['TTM-R2'].prediction === 1 ? 'UP' : 'DOWN' }}
|
||||
<i :class="hour.models['TTM-R2'].is_correct ? 'bi-check-circle' : 'bi-x-circle'"></i>
|
||||
<i v-if="hour.actual_result !== null" :class="hour.models['TTM-R2'].is_correct ? 'bi-check' : 'bi-x'"></i>
|
||||
</span>
|
||||
<span v-else class="text-muted">-</span>
|
||||
</td>
|
||||
</tr>
|
||||
<tr v-if="!hourlyModal.data?.length">
|
||||
<td colspan="4" class="text-center text-muted py-4">
|
||||
<td colspan="8" class="text-center text-muted py-4">
|
||||
暂无数据
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
@@ -1,940 +0,0 @@
|
||||
"""
|
||||
BTC Hourly 自动交易脚本
|
||||
|
||||
基于DLinear模型预测,在Polymarket自动下单BTC小时涨跌
|
||||
支持:
|
||||
- 每小时自动寻找对应市场
|
||||
- 实时价格刷新
|
||||
- 定时下单
|
||||
- 详细日志记录
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import argparse
|
||||
import json
|
||||
import time
|
||||
import logging
|
||||
from logging.handlers import RotatingFileHandler
|
||||
import schedule
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Optional, Tuple, Dict
|
||||
import pickle
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import torch
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from dotenv import load_dotenv
|
||||
import httpx
|
||||
|
||||
# 添加项目路径
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
from data.trading_db import TradingDatabase
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
def setup_logging(log_dir: str = "logs", log_level: int = logging.INFO) -> logging.Logger:
|
||||
"""
|
||||
配置日志系统
|
||||
|
||||
Args:
|
||||
log_dir: 日志目录
|
||||
log_level: 日志级别
|
||||
|
||||
Returns:
|
||||
配置好的logger
|
||||
"""
|
||||
# 创建日志目录
|
||||
os.makedirs(log_dir, exist_ok=True)
|
||||
|
||||
# 创建logger
|
||||
logger = logging.getLogger("polymarket_trader")
|
||||
logger.setLevel(log_level)
|
||||
|
||||
# 避免重复添加handler
|
||||
if logger.handlers:
|
||||
return logger
|
||||
|
||||
# 日志格式
|
||||
formatter = logging.Formatter(
|
||||
"%(asctime)s | %(levelname)-8s | %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S"
|
||||
)
|
||||
|
||||
# 文件handler (按大小轮转, 最大10MB, 保留5个备份)
|
||||
log_file = os.path.join(log_dir, "polymarket_trading.log")
|
||||
file_handler = RotatingFileHandler(
|
||||
log_file,
|
||||
maxBytes=10*1024*1024, # 10MB
|
||||
backupCount=5,
|
||||
encoding='utf-8'
|
||||
)
|
||||
file_handler.setLevel(log_level)
|
||||
file_handler.setFormatter(formatter)
|
||||
logger.addHandler(file_handler)
|
||||
|
||||
# 控制台handler
|
||||
console_handler = logging.StreamHandler(sys.stdout)
|
||||
console_handler.setLevel(log_level)
|
||||
console_handler.setFormatter(formatter)
|
||||
logger.addHandler(console_handler)
|
||||
|
||||
return logger
|
||||
|
||||
|
||||
# 初始化全局logger
|
||||
logger = setup_logging()
|
||||
|
||||
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
|
||||
def __init__(self, lookback: int = 168, forecast_horizon: int = 1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df: pd.DataFrame) -> pd.DataFrame:
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df: pd.DataFrame) -> pd.DataFrame:
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df: pd.DataFrame, fit: bool = True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
class PolymarketBTCTrader:
|
||||
"""Polymarket BTC Hourly交易器"""
|
||||
|
||||
GAMMA_API = "https://gamma-api.polymarket.com"
|
||||
CLOB_API = "https://clob.polymarket.com"
|
||||
|
||||
# 月份名称映射
|
||||
MONTH_NAMES = {
|
||||
1: 'january', 2: 'february', 3: 'march', 4: 'april',
|
||||
5: 'may', 6: 'june', 7: 'july', 8: 'august',
|
||||
9: 'september', 10: 'october', 11: 'november', 12: 'december'
|
||||
}
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
private_key: Optional[str] = None,
|
||||
max_bet: float = 1.0,
|
||||
max_price: float = 0.50,
|
||||
dry_run: bool = True,
|
||||
):
|
||||
self.private_key = private_key or os.getenv("POLYMARKET_PRIVATE_KEY")
|
||||
self.max_bet = max_bet
|
||||
self.max_price = max_price
|
||||
self.dry_run = dry_run
|
||||
|
||||
# HTTP客户端
|
||||
proxy = os.getenv("SOCKS5_PROXY") if os.getenv("USE_PROXY", "false").lower() == "true" else None
|
||||
self.http = httpx.Client(
|
||||
timeout=30.0,
|
||||
proxy=proxy,
|
||||
headers={'Accept-Encoding': 'identity'}
|
||||
)
|
||||
|
||||
# 设备
|
||||
self.device = torch.device('mps' if torch.backends.mps.is_available() else
|
||||
'cuda' if torch.cuda.is_available() else 'cpu')
|
||||
|
||||
# 加载模型
|
||||
self.model = None
|
||||
self.preprocessor = None
|
||||
self._load_model()
|
||||
|
||||
# 数据获取器
|
||||
self.fetcher = BinanceDataFetcher()
|
||||
|
||||
# 数据库
|
||||
self.db = TradingDatabase()
|
||||
|
||||
# CLOB客户端(用于下单)
|
||||
self.clob_client = None
|
||||
if self.private_key and not self.dry_run:
|
||||
self._init_clob_client()
|
||||
|
||||
# 当前市场缓存
|
||||
self.current_market = None
|
||||
self.last_price_update = None
|
||||
|
||||
def _load_model(self):
|
||||
"""加载DLinear模型"""
|
||||
logger.info("加载DLinear模型...")
|
||||
|
||||
from models.dlinear.model import DLinear
|
||||
|
||||
checkpoint_path = './models/checkpoints/dlinear_classification_2026/best_model.pt'
|
||||
preprocessor_path = './models/checkpoints/dlinear_classification_2026/preprocessor.pkl'
|
||||
|
||||
checkpoint = torch.load(checkpoint_path, map_location=self.device, weights_only=False)
|
||||
self.model = DLinear(checkpoint['config'])
|
||||
self.model.load_state_dict(checkpoint['model_state_dict'])
|
||||
self.model = self.model.to(self.device).eval()
|
||||
|
||||
with open(preprocessor_path, 'rb') as f:
|
||||
self.preprocessor = pickle.load(f)
|
||||
|
||||
logger.info(f"✓ 模型已加载 (设备: {self.device})")
|
||||
|
||||
def _init_clob_client(self):
|
||||
"""初始化CLOB交易客户端"""
|
||||
try:
|
||||
from py_clob_client.client import ClobClient
|
||||
|
||||
self.clob_client = ClobClient(
|
||||
host=self.CLOB_API,
|
||||
key=self.private_key,
|
||||
chain_id=137, # Polygon
|
||||
)
|
||||
self.clob_client.set_api_creds(self.clob_client.create_or_derive_api_creds())
|
||||
logger.info("✓ CLOB客户端初始化成功")
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ CLOB客户端初始化失败: {e}")
|
||||
self.clob_client = None
|
||||
|
||||
def _log_to_db(self, result: Dict, market: Optional[Dict]):
|
||||
"""记录交易结果到数据库"""
|
||||
try:
|
||||
# 确定状态
|
||||
status = 'failure'
|
||||
if self.dry_run:
|
||||
status = 'test'
|
||||
elif result.get('success'):
|
||||
status = 'success'
|
||||
elif result.get('error'):
|
||||
status = 'failure'
|
||||
elif result.get('prediction'): # 有预测但未交易(如市场未找到)
|
||||
status = 'in_progress' # 或者 failure? 未找到市场算 failure
|
||||
if result.get('error'):
|
||||
status = 'failure'
|
||||
|
||||
log_data = {
|
||||
'timestamp': result.get('timestamp'),
|
||||
'market_title': result.get('market'),
|
||||
'market_slug': market.get('slug') if market else result.get('market_slug'),
|
||||
'model_name': 'DLinear',
|
||||
'prediction': result.get('prediction'),
|
||||
'probability': result.get('probability'),
|
||||
'action': result.get('side'),
|
||||
'price': result.get('price'),
|
||||
'amount': result.get('amount'),
|
||||
'size': result.get('size'),
|
||||
'status': status,
|
||||
'error_message': result.get('error'),
|
||||
'raw_data': result
|
||||
}
|
||||
self.db.log_trade(log_data)
|
||||
logger.info(f"💾 交易记录已保存到数据库")
|
||||
except Exception as e:
|
||||
logger.error(f"❌ 保存数据库失败: {e}")
|
||||
|
||||
def generate_market_slug(self, target_time: Optional[datetime] = None) -> str:
|
||||
"""
|
||||
生成市场slug
|
||||
|
||||
格式: bitcoin-up-or-down-{month}-{day}-{hour}am/pm-et
|
||||
例如: bitcoin-up-or-down-february-3-8am-et
|
||||
|
||||
Args:
|
||||
target_time: 目标时间(ET时区)
|
||||
for_current_hour: 如果True,使用当前整点;否则使用下一个整点
|
||||
|
||||
Returns:
|
||||
市场slug
|
||||
"""
|
||||
# 获取美东时间
|
||||
et_tz = ZoneInfo('America/New_York')
|
||||
|
||||
if target_time is None:
|
||||
now_et = datetime.now(et_tz)
|
||||
target_time = now_et.replace(minute=0, second=0, microsecond=0)
|
||||
|
||||
month_name = self.MONTH_NAMES[target_time.month]
|
||||
day = target_time.day
|
||||
|
||||
# 转换为12小时制
|
||||
hour_24 = target_time.hour
|
||||
if hour_24 == 0:
|
||||
hour_12 = 12
|
||||
am_pm = 'am'
|
||||
elif hour_24 < 12:
|
||||
hour_12 = hour_24
|
||||
am_pm = 'am'
|
||||
elif hour_24 == 12:
|
||||
hour_12 = 12
|
||||
am_pm = 'pm'
|
||||
else:
|
||||
hour_12 = hour_24 - 12
|
||||
am_pm = 'pm'
|
||||
|
||||
slug = f"bitcoin-up-or-down-{month_name}-{day}-{hour_12}{am_pm}-et"
|
||||
return slug
|
||||
|
||||
def get_market_by_slug(self, slug: str) -> Optional[Dict]:
|
||||
"""
|
||||
通过slug获取市场信息
|
||||
|
||||
Args:
|
||||
slug: 市场slug
|
||||
|
||||
Returns:
|
||||
市场信息
|
||||
"""
|
||||
try:
|
||||
response = self.http.get(
|
||||
f"{self.GAMMA_API}/events",
|
||||
params={'slug': slug}
|
||||
)
|
||||
response.raise_for_status()
|
||||
events = response.json()
|
||||
|
||||
if not events:
|
||||
return None
|
||||
|
||||
event = events[0]
|
||||
markets = event.get('markets', [])
|
||||
|
||||
if not markets:
|
||||
return None
|
||||
|
||||
market = markets[0]
|
||||
|
||||
# outcomePrices可能是JSON字符串或列表
|
||||
outcome_prices_raw = market.get('outcomePrices', ['0', '0'])
|
||||
if isinstance(outcome_prices_raw, str):
|
||||
import json as _json
|
||||
try:
|
||||
outcome_prices_raw = _json.loads(outcome_prices_raw)
|
||||
except:
|
||||
outcome_prices_raw = ['0', '0']
|
||||
|
||||
# clobTokenIds也可能是JSON字符串
|
||||
clob_tokens_raw = market.get('clobTokenIds', [])
|
||||
if isinstance(clob_tokens_raw, str):
|
||||
import json as _json
|
||||
try:
|
||||
clob_tokens_raw = _json.loads(clob_tokens_raw)
|
||||
except:
|
||||
clob_tokens_raw = []
|
||||
|
||||
return {
|
||||
'title': event.get('title', ''),
|
||||
'slug': slug,
|
||||
'question': market.get('question', ''),
|
||||
'condition_id': market.get('conditionId', ''),
|
||||
'outcomes': market.get('outcomes', []),
|
||||
'outcome_prices': [float(p) for p in outcome_prices_raw],
|
||||
'tokens': clob_tokens_raw,
|
||||
'end_date': event.get('endDate', ''),
|
||||
'closed': market.get('closed', False),
|
||||
}
|
||||
except Exception as e:
|
||||
logger.error(f"获取市场失败: {e}")
|
||||
return None
|
||||
|
||||
def get_current_hourly_market(self) -> Optional[Dict]:
|
||||
"""
|
||||
获取BTC市场
|
||||
|
||||
Args:
|
||||
for_current_hour: 如果True,获取当前整点市场;否则获取下一个整点市场
|
||||
|
||||
Returns:
|
||||
市场信息
|
||||
"""
|
||||
slug = self.generate_market_slug()
|
||||
logger.info(f"搜索市场: {slug}")
|
||||
|
||||
market = self.get_market_by_slug(slug)
|
||||
|
||||
if market:
|
||||
self.current_market = market
|
||||
self.last_price_update = datetime.now()
|
||||
logger.info(f"✓ 找到市场: {market['title']}")
|
||||
logger.info(f" Up价格: ${market['outcome_prices'][0]:.3f}")
|
||||
logger.info(f" Down价格: ${market['outcome_prices'][1]:.3f}")
|
||||
else:
|
||||
logger.warning(f"❌ 未找到市场: {slug}")
|
||||
|
||||
return market
|
||||
|
||||
def refresh_prices(self) -> Optional[Dict]:
|
||||
"""
|
||||
刷新当前市场价格
|
||||
|
||||
Returns:
|
||||
更新后的市场信息
|
||||
"""
|
||||
if not self.current_market:
|
||||
self.get_current_hourly_market()
|
||||
|
||||
# 使用CLOB API获取实时价格
|
||||
tokens = self.current_market.get('tokens', [])
|
||||
if len(tokens) >= 2:
|
||||
realtime_prices = self.get_realtime_prices(tokens[0], tokens[1])
|
||||
if realtime_prices:
|
||||
old_prices = self.current_market['outcome_prices']
|
||||
new_prices = realtime_prices
|
||||
|
||||
self.current_market['outcome_prices'] = new_prices
|
||||
self.last_price_update = datetime.now()
|
||||
|
||||
# 显示价格变化
|
||||
up_change = new_prices[0] - old_prices[0]
|
||||
down_change = new_prices[1] - old_prices[1]
|
||||
|
||||
logger.info(f"📊 实时价格 [{datetime.now().strftime('%H:%M:%S')}]")
|
||||
logger.info(f" Up: ${new_prices[0]:.3f} ({up_change:+.3f})")
|
||||
logger.info(f" Down: ${new_prices[1]:.3f} ({down_change:+.3f})")
|
||||
|
||||
return self.current_market
|
||||
|
||||
# 备用: 使用Gamma API
|
||||
slug = self.current_market['slug']
|
||||
market = self.get_market_by_slug(slug)
|
||||
|
||||
if market:
|
||||
old_prices = self.current_market['outcome_prices']
|
||||
new_prices = market['outcome_prices']
|
||||
|
||||
self.current_market = market
|
||||
self.last_price_update = datetime.now()
|
||||
|
||||
# 显示价格变化
|
||||
up_change = new_prices[0] - old_prices[0]
|
||||
down_change = new_prices[1] - old_prices[1]
|
||||
|
||||
logger.info(f"📊 价格更新(Gamma) [{datetime.now().strftime('%H:%M:%S')}]")
|
||||
logger.info(f" Up: ${new_prices[0]:.3f} ({up_change:+.3f})")
|
||||
logger.info(f" Down: ${new_prices[1]:.3f} ({down_change:+.3f})")
|
||||
|
||||
return market
|
||||
|
||||
def get_realtime_prices(self, up_token_id: str, down_token_id: str) -> Optional[list]:
|
||||
"""
|
||||
从CLOB API获取实时买入价格
|
||||
|
||||
使用/price?side=BUY端点获取真实买入价格 (与页面显示一致)
|
||||
|
||||
Args:
|
||||
up_token_id: Up代币ID
|
||||
down_token_id: Down代币ID
|
||||
|
||||
Returns:
|
||||
[up_price, down_price] 或 None
|
||||
"""
|
||||
try:
|
||||
# 使用/price端点获取买入价格
|
||||
up_response = self.http.get(
|
||||
f"{self.CLOB_API}/price",
|
||||
params={'token_id': up_token_id, 'side': 'BUY'}
|
||||
)
|
||||
up_response.raise_for_status()
|
||||
up_data = up_response.json()
|
||||
|
||||
down_response = self.http.get(
|
||||
f"{self.CLOB_API}/price",
|
||||
params={'token_id': down_token_id, 'side': 'BUY'}
|
||||
)
|
||||
down_response.raise_for_status()
|
||||
down_data = down_response.json()
|
||||
|
||||
# 解析买入价格
|
||||
up_price = float(up_data.get('price', 0.5))
|
||||
down_price = float(down_data.get('price', 0.5))
|
||||
|
||||
return [up_price, down_price]
|
||||
except Exception as e:
|
||||
logger.debug(f"获取实时价格失败: {e}")
|
||||
return None
|
||||
|
||||
def execute_trade(self, prediction: int, market: Dict) -> Dict:
|
||||
"""
|
||||
执行交易
|
||||
|
||||
Args:
|
||||
prediction: 预测结果(0=跌, 1=涨)
|
||||
market: 市场信息
|
||||
|
||||
Returns:
|
||||
交易结果
|
||||
"""
|
||||
result = {
|
||||
'timestamp': datetime.now().isoformat(),
|
||||
'market': market['title'],
|
||||
'prediction': 'UP' if prediction == 1 else 'DOWN',
|
||||
'success': False,
|
||||
'order': None,
|
||||
'error': None,
|
||||
}
|
||||
|
||||
if prediction == 1: # 预测涨 -> 买UP
|
||||
token_id = market['tokens'][0]
|
||||
price = market['outcome_prices'][0]
|
||||
side = "UP"
|
||||
else: # 预测跌 -> 买DOWN
|
||||
token_id = market['tokens'][1]
|
||||
price = market['outcome_prices'][1]
|
||||
side = "DOWN"
|
||||
|
||||
result['side'] = side
|
||||
result['price'] = price
|
||||
|
||||
# 检查价格限制
|
||||
if price > self.max_price:
|
||||
result['error'] = f"价格${price:.2f}超过限制${self.max_price:.2f}"
|
||||
logger.error(f"❌ {result['error']}")
|
||||
return result
|
||||
|
||||
# 计算数量
|
||||
size = self.max_bet / price
|
||||
result['size'] = size
|
||||
result['amount'] = self.max_bet
|
||||
|
||||
logger.info("📋 交易详情:")
|
||||
logger.info(f" 方向: {side}")
|
||||
logger.info(f" 价格: ${price:.3f}")
|
||||
logger.info(f" 金额: ${self.max_bet:.2f} USDC")
|
||||
logger.info(f" 数量: {size:.2f} 股")
|
||||
|
||||
if self.dry_run:
|
||||
logger.info("🔸 [模拟模式] 未实际下单")
|
||||
result['success'] = True
|
||||
result['order'] = 'DRY_RUN'
|
||||
return result
|
||||
|
||||
# 实际下单
|
||||
if not self.clob_client:
|
||||
result['error'] = "CLOB客户端未初始化"
|
||||
logger.error(f"❌ {result['error']}")
|
||||
return result
|
||||
|
||||
try:
|
||||
from py_clob_client.order_builder.constants import BUY
|
||||
|
||||
order = self.clob_client.create_limit_order(
|
||||
token_id=token_id,
|
||||
price=price,
|
||||
size=size,
|
||||
side=BUY,
|
||||
)
|
||||
|
||||
response = self.clob_client.post_order(order)
|
||||
logger.info(f"✓ 订单已提交: {response}")
|
||||
result['success'] = True
|
||||
result['order'] = response
|
||||
except Exception as e:
|
||||
result['error'] = str(e)
|
||||
logger.error(f"❌ 下单失败: {e}")
|
||||
|
||||
return result
|
||||
|
||||
def get_prediction_with_delay(self, delay_seconds: float = 2.0) -> Tuple[int, float]:
|
||||
"""
|
||||
延迟后获取K线并预测当前小时的涨跌
|
||||
|
||||
注意: 使用上一小时收盘的K线(倒数第二根)来预测当前小时
|
||||
例如: 22:00整点时,用21:00收盘的K线预测22:00这个小时的涨跌
|
||||
|
||||
Args:
|
||||
delay_seconds: 延迟秒数(等待K线更新)
|
||||
|
||||
Returns:
|
||||
(prediction, probability): 预测结果(0=跌, 1=涨)和概率
|
||||
"""
|
||||
logger.info(f"⏳ 等待{delay_seconds}秒获取最新K线...")
|
||||
time.sleep(delay_seconds)
|
||||
|
||||
logger.info("获取最新BTC数据...")
|
||||
|
||||
end_date = (datetime.now() + timedelta(days=1)).strftime('%Y-%m-%d')
|
||||
start_date = (datetime.now() - timedelta(days=30)).strftime('%Y-%m-%d')
|
||||
|
||||
df = self.fetcher.fetch_ohlcv(
|
||||
symbol='BTC',
|
||||
timeframe='1h',
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
use_cache=False # 不使用缓存,获取最新数据
|
||||
)
|
||||
|
||||
# 使用倒数第二根K线(上一小时收盘)进行预测
|
||||
# 例如22:00时,最新K线是14:00 UTC(22:00北京),我们用13:00 UTC(21:00北京)的数据
|
||||
prev_time = df['timestamp'].iloc[-2]
|
||||
prev_close = df['close'].iloc[-2]
|
||||
current_time = df['timestamp'].iloc[-1]
|
||||
current_close = df['close'].iloc[-1]
|
||||
|
||||
logger.info(f" 上一K线时间: {prev_time} (用于预测)")
|
||||
logger.info(f" 上一收盘价: ${prev_close:,.2f}")
|
||||
logger.info(f" 当前K线时间: {current_time} (预测目标)")
|
||||
logger.info(f" 当前收盘价: ${current_close:,.2f} (实际结果)")
|
||||
|
||||
# 用除最后一根外的数据进行预测
|
||||
df_for_prediction = df.iloc[:-1]
|
||||
X, _, _ = self.preprocessor.preprocess(df_for_prediction, fit=False)
|
||||
X_last = torch.FloatTensor(X[-1:]).to(self.device)
|
||||
|
||||
with torch.no_grad():
|
||||
logits = self.model(X_last).cpu().numpy()
|
||||
prob = 1 / (1 + np.exp(-logits))
|
||||
prediction = int(prob.item() > 0.5)
|
||||
probability = float(prob.item())
|
||||
|
||||
direction = "📈 UP (涨)" if prediction == 1 else "📉 DOWN (跌)"
|
||||
confidence = probability if prediction == 1 else 1 - probability
|
||||
logger.info(f"🤖 DLinear预测: {direction}")
|
||||
logger.info(f" 置信度: {confidence:.2%}")
|
||||
|
||||
# 显示预测是否正确 (用于验证)
|
||||
actual_up = current_close > prev_close
|
||||
if prediction == 1:
|
||||
correct = actual_up
|
||||
else:
|
||||
correct = not actual_up
|
||||
logger.info(f" 实际涨跌: {'📈 涨' if actual_up else '📉 跌'} ({'✅ 正确' if correct else '❌ 错误'})")
|
||||
|
||||
return prediction, probability
|
||||
|
||||
def run_once(self) -> Dict:
|
||||
"""运行一次交易流程(不等待整点)"""
|
||||
logger.info("运行单次交易...")
|
||||
return self.run_at_hour_start(delay_seconds=0)
|
||||
|
||||
def run_at_hour_start(self, delay_seconds: float = 2.0, timeout_minutes: int = 50) -> Dict:
|
||||
"""
|
||||
整点时执行的交易流程
|
||||
|
||||
1. 延迟delay_seconds秒等待K线更新
|
||||
2. 获取最新K线
|
||||
3. DLinear预测当前整点的收盘涨跌
|
||||
4. 获取对应的Polymarket市场
|
||||
5. 循环检查价格:
|
||||
- 如果价格 <= max_price: 下单并结束
|
||||
- 如果价格 > max_price: 等待并重试(直到超时)
|
||||
|
||||
Args:
|
||||
delay_seconds: 整点后延迟秒数
|
||||
timeout_minutes: 价格等待超时时间(分钟)
|
||||
|
||||
Returns:
|
||||
执行结果
|
||||
"""
|
||||
logger.info("=" * 60)
|
||||
logger.info(f"⏰ 整点交易触发 - {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
|
||||
logger.info("=" * 60)
|
||||
logger.info(f" 模式: {'🔸 模拟' if self.dry_run else '💰 真实交易'}")
|
||||
logger.info(f" 最大下注: ${self.max_bet:.2f} USDC")
|
||||
logger.info(f" 最高价格: ${self.max_price:.2f}")
|
||||
logger.info(f" 等待超时: {timeout_minutes}分钟")
|
||||
|
||||
result = {
|
||||
'timestamp': datetime.now().isoformat(),
|
||||
'success': False,
|
||||
'error': None,
|
||||
}
|
||||
|
||||
# 1. 延迟后获取预测
|
||||
prediction, probability = self.get_prediction_with_delay(delay_seconds)
|
||||
result['prediction'] = 'UP' if prediction == 1 else 'DOWN'
|
||||
result['probability'] = probability if prediction == 1 else 1 - probability
|
||||
|
||||
# 2. 获取初始市场
|
||||
market = self.get_current_hourly_market()
|
||||
if not market:
|
||||
result['error'] = '未找到市场'
|
||||
logger.error(f"❌ {result['error']}")
|
||||
self._log_to_db(result, None)
|
||||
return result
|
||||
|
||||
result['market'] = market['title']
|
||||
result['market_slug'] = market['slug']
|
||||
|
||||
if market['closed']:
|
||||
result['error'] = '市场已关闭'
|
||||
logger.error(f"❌ {result['error']}")
|
||||
self._log_to_db(result, market)
|
||||
return result
|
||||
|
||||
# 检查是否已交易过该市场
|
||||
check_status = 'test' if self.dry_run else 'success'
|
||||
if self.db.check_existing_trade(market['slug'], status=check_status):
|
||||
result['error'] = '该市场已存在成功交易记录'
|
||||
logger.warning(f"⚠️ {result['error']}, 跳过交易")
|
||||
return result
|
||||
|
||||
# 3. 循环检查价格并交易
|
||||
start_time = datetime.now()
|
||||
end_time = start_time + timedelta(minutes=timeout_minutes)
|
||||
|
||||
logger.info(f"🔄 开始价格监控 (预测: {'UP' if prediction == 1 else 'DOWN'})...")
|
||||
|
||||
while datetime.now() < end_time:
|
||||
# 刷新价格
|
||||
market = self.refresh_prices()
|
||||
if not market:
|
||||
logger.warning("无法获取市场价格,重试中...")
|
||||
time.sleep(10)
|
||||
continue
|
||||
|
||||
# 检查价格
|
||||
price_idx = 0 if prediction == 1 else 1
|
||||
current_price = market['outcome_prices'][price_idx]
|
||||
|
||||
logger.info(f" 当前价格: ${current_price:.3f} (目标: <= ${self.max_price:.2f})")
|
||||
|
||||
if current_price <= self.max_price:
|
||||
# 价格符合,执行交易
|
||||
logger.info("✅ 价格满足条件,正在下单...")
|
||||
trade_result = self.execute_trade(prediction, market)
|
||||
result.update(trade_result)
|
||||
|
||||
# 记录数据库
|
||||
self._log_to_db(result, market)
|
||||
|
||||
logger.info("=" * 60)
|
||||
return result
|
||||
else:
|
||||
# 价格过高,等待
|
||||
wait_time = 30 # 等待30秒
|
||||
elapsed = int((datetime.now() - start_time).total_seconds())
|
||||
logger.info(f"⏳ 价格过高,继续等待... (已耗时: {elapsed}s)")
|
||||
time.sleep(wait_time)
|
||||
|
||||
# 超时
|
||||
result['error'] = f'等待价格超时 ({timeout_minutes}分钟)'
|
||||
result['price'] = market['outcome_prices'][0] if prediction == 1 else market['outcome_prices'][1]
|
||||
result['side'] = 'UP' if prediction == 1 else 'DOWN'
|
||||
logger.error(f"❌ {result['error']}")
|
||||
|
||||
self._log_to_db(result, market)
|
||||
logger.info("=" * 60)
|
||||
|
||||
return result
|
||||
|
||||
def _show_initial_prediction(self):
|
||||
"""
|
||||
启动时显示当前预测(不下单)
|
||||
"""
|
||||
logger.info("\n📊 当前预测状态:")
|
||||
|
||||
end_date = (datetime.now() + timedelta(days=1)).strftime('%Y-%m-%d')
|
||||
start_date = (datetime.now() - timedelta(days=30)).strftime('%Y-%m-%d')
|
||||
|
||||
df = self.fetcher.fetch_ohlcv(
|
||||
symbol='BTC',
|
||||
timeframe='1h',
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
use_cache=False
|
||||
)
|
||||
|
||||
# 显示K线信息
|
||||
prev_time = df['timestamp'].iloc[-2]
|
||||
prev_close = df['close'].iloc[-2]
|
||||
current_time = df['timestamp'].iloc[-1]
|
||||
current_close = df['close'].iloc[-1]
|
||||
|
||||
logger.info(f" 上一K线: {prev_time} 收盘价: ${prev_close:,.2f}")
|
||||
logger.info(f" 当前K线: {current_time} 收盘价: ${current_close:,.2f}")
|
||||
|
||||
# 进行预测
|
||||
df_for_prediction = df.iloc[:-1]
|
||||
X, _, _ = self.preprocessor.preprocess(df_for_prediction, fit=False)
|
||||
X_last = torch.FloatTensor(X[-1:]).to(self.device)
|
||||
|
||||
with torch.no_grad():
|
||||
logits = self.model(X_last).cpu().numpy()
|
||||
prob = 1 / (1 + np.exp(-logits))
|
||||
prediction = int(prob.item() > 0.5)
|
||||
probability = float(prob.item())
|
||||
|
||||
direction = "📈 UP (涨)" if prediction == 1 else "📉 DOWN (跌)"
|
||||
confidence = probability if prediction == 1 else 1 - probability
|
||||
|
||||
logger.info(f" 🤖 预测: {direction} (置信度: {confidence:.2%})")
|
||||
|
||||
# 显示市场价格
|
||||
if self.current_market:
|
||||
up_price = self.current_market['outcome_prices'][0]
|
||||
down_price = self.current_market['outcome_prices'][1]
|
||||
if prediction == 1:
|
||||
logger.info(f" 💰 买入Up价格: ${up_price:.3f} {'✅ 可下单' if up_price <= self.max_price else '❌ 超过限价'}")
|
||||
else:
|
||||
logger.info(f" 💰 买入Down价格: ${down_price:.3f} {'✅ 可下单' if down_price <= self.max_price else '❌ 超过限价'}")
|
||||
|
||||
# 计算距下一整点时间
|
||||
now = datetime.now()
|
||||
next_hour = now.replace(minute=0, second=0, microsecond=0) + timedelta(hours=1)
|
||||
wait_minutes = int((next_hour - now).total_seconds() / 60)
|
||||
logger.info(f" ⏰ 距离下一整点: {wait_minutes}分钟")
|
||||
logger.info("")
|
||||
|
||||
def wait_for_next_hour(self) -> datetime:
|
||||
"""
|
||||
等待到下一个整点
|
||||
|
||||
Returns:
|
||||
下一个整点时间
|
||||
"""
|
||||
now = datetime.now()
|
||||
next_hour = now.replace(minute=0, second=0, microsecond=0) + timedelta(hours=1)
|
||||
wait_seconds = (next_hour - now).total_seconds()
|
||||
|
||||
logger.info(f"⏳ 等待下一个整点: {next_hour.strftime('%H:%M:%S')}")
|
||||
logger.info(f" 剩余等待: {int(wait_seconds)}秒")
|
||||
|
||||
return next_hour
|
||||
|
||||
def run_scheduler(self, interval_minutes: int = 5, delay_seconds: float = 2.0):
|
||||
"""
|
||||
定时运行交易器
|
||||
|
||||
工作流程:
|
||||
1. 每个整点+delay_seconds秒触发交易
|
||||
2. 期间每interval_minutes分钟刷新价格
|
||||
|
||||
Args:
|
||||
interval_minutes: 价格刷新间隔(分钟)
|
||||
delay_seconds: 整点后延迟秒数
|
||||
"""
|
||||
logger.info("=" * 60)
|
||||
logger.info("🤖 BTC Hourly自动交易已启动")
|
||||
logger.info("=" * 60)
|
||||
logger.info(f" 模式: {'🔸 模拟' if self.dry_run else '💰 真实交易'}")
|
||||
logger.info(f" 下单时机: 每个整点后延迟{delay_seconds}秒")
|
||||
logger.info(f" 价格刷新: 每{interval_minutes}分钟")
|
||||
logger.info(f" 最大下注: ${self.max_bet:.2f} USDC")
|
||||
logger.info(f" 最高价格: ${self.max_price:.2f}")
|
||||
logger.info(" 按 Ctrl+C 停止")
|
||||
logger.info("=" * 60)
|
||||
|
||||
# 立即获取整点的市场
|
||||
self.get_current_hourly_market()
|
||||
|
||||
# 启动时进行一次预测(不下单,仅显示)
|
||||
self._show_initial_prediction()
|
||||
|
||||
# 每小时整点后执行交易
|
||||
def hourly_trade():
|
||||
self.run_at_hour_start(delay_seconds=delay_seconds)
|
||||
|
||||
schedule.every().hour.at(":00").do(lambda: time.sleep(delay_seconds) or hourly_trade())
|
||||
|
||||
try:
|
||||
while True:
|
||||
schedule.run_pending()
|
||||
time.sleep(1)
|
||||
except KeyboardInterrupt:
|
||||
logger.info("✓ 交易器已停止")
|
||||
|
||||
def close(self):
|
||||
"""关闭资源"""
|
||||
self.http.close()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description='BTC Hourly自动交易 - 基于DLinear模型')
|
||||
parser.add_argument('--dry-run', action='store_true', default=True,
|
||||
help='模拟模式(不实际下单)')
|
||||
parser.add_argument('--real', action='store_true',
|
||||
help='真实交易模式')
|
||||
parser.add_argument('--max-bet', type=float, default=1.0,
|
||||
help='最大下注金额(USDC)')
|
||||
parser.add_argument('--max-price', type=float, default=0.50,
|
||||
help='最高可接受价格')
|
||||
parser.add_argument('--once', action='store_true',
|
||||
help='只执行一次(不启动定时器)')
|
||||
parser.add_argument('--interval', type=int, default=1,
|
||||
help='价格刷新间隔(分钟)')
|
||||
parser.add_argument('--check-market', action='store_true',
|
||||
help='只检查当前市场信息')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
dry_run = not args.real
|
||||
|
||||
# 真实交易确认
|
||||
if not dry_run:
|
||||
logger.warning("⚠️ 警告: 即将进行真实交易!")
|
||||
logger.warning(f" 最大下注: ${args.max_bet:.2f} USDC")
|
||||
confirm = input("确认继续? (yes/no): ")
|
||||
if confirm.lower() != 'yes':
|
||||
logger.info("已取消")
|
||||
return
|
||||
|
||||
trader = PolymarketBTCTrader(
|
||||
max_bet=args.max_bet,
|
||||
max_price=args.max_price,
|
||||
dry_run=dry_run,
|
||||
)
|
||||
|
||||
try:
|
||||
if args.check_market:
|
||||
# 只检查市场
|
||||
trader.get_current_hourly_market()
|
||||
elif args.once:
|
||||
# 执行一次
|
||||
result = trader.run_once()
|
||||
|
||||
# 保存日志
|
||||
log_file = 'trading_log.json'
|
||||
logs = []
|
||||
if os.path.exists(log_file):
|
||||
with open(log_file, 'r') as f:
|
||||
logs = json.load(f)
|
||||
logs.append(result)
|
||||
with open(log_file, 'w') as f:
|
||||
json.dump(logs, f, indent=2, ensure_ascii=False)
|
||||
logger.info(f"结果已保存到 {log_file}")
|
||||
else:
|
||||
# 定时运行
|
||||
trader.run_scheduler(interval_minutes=args.interval)
|
||||
finally:
|
||||
trader.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"account_id": "7ea8a511cad4d467490fe36aa7d6638c",
|
||||
"project_name": "trades"
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"account": {
|
||||
"id": "7ea8a511cad4d467490fe36aa7d6638c",
|
||||
"name": "Subuhuinew@gmail.com's Account"
|
||||
}
|
||||
}
|
||||
BIN
Binary file not shown.
@@ -0,0 +1,41 @@
|
||||
# Cloudflare Pages 看板
|
||||
|
||||
本目录提供一个可直接部署到 Cloudflare Pages 的页面,数据源为 D1(由 MySQL 备份任务同步而来)。
|
||||
|
||||
## 目录结构
|
||||
|
||||
- `public/`:静态页面资源
|
||||
- `functions/api/`:Pages Functions,直接查询 D1
|
||||
|
||||
## 必要前置
|
||||
|
||||
1. 已在 D1 中有备份表:
|
||||
- `backup_trades`
|
||||
- `backup_hourly_model_predictions`
|
||||
|
||||
2. Pages 项目绑定 D1(绑定名必须是 `DB`)
|
||||
|
||||
## 本地开发
|
||||
|
||||
```bash
|
||||
cd cloudflare_pages
|
||||
npx wrangler pages dev public --d1 DB=<你的D1数据库ID>
|
||||
```
|
||||
|
||||
## 生产部署
|
||||
|
||||
1. 在 Cloudflare Pages 新建项目,根目录设置为 `cloudflare_pages`
|
||||
2. Build command 留空
|
||||
3. Build output directory 设置为 `public`
|
||||
4. 在 Pages 项目 Settings -> Functions -> D1 bindings 中添加:
|
||||
- Variable name: `DB`
|
||||
- D1 database: 选择你的数据库
|
||||
|
||||
## 页面接口
|
||||
|
||||
- `/api/status`
|
||||
- `/api/trades/current`
|
||||
- `/api/trades/history?page=1&limit=15&month=YYYY-MM&day=YYYY-MM-DD`
|
||||
- `/api/accuracy/monthly?month=YYYY-MM`
|
||||
- `/api/accuracy/daily?month=YYYY-MM&page=1&limit=10`
|
||||
- `/api/accuracy/hourly/YYYY-MM-DD`
|
||||
@@ -0,0 +1,101 @@
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: { "content-type": "application/json; charset=UTF-8" },
|
||||
});
|
||||
}
|
||||
|
||||
function toInt(v, d) {
|
||||
const n = Number(v);
|
||||
return Number.isFinite(n) && n > 0 ? Math.floor(n) : d;
|
||||
}
|
||||
|
||||
function ratio(correct, total) {
|
||||
if (!total) return 0;
|
||||
return Math.round((correct / total) * 10000) / 100;
|
||||
}
|
||||
|
||||
function item(total, correct) {
|
||||
const t = Number(total || 0);
|
||||
const c = Number(correct || 0);
|
||||
return {
|
||||
total: t,
|
||||
correct: c,
|
||||
incorrect: Math.max(0, t - c),
|
||||
accuracy: ratio(c, t),
|
||||
};
|
||||
}
|
||||
|
||||
export async function onRequestGet(context) {
|
||||
const db = context.env.DB;
|
||||
if (!db) {
|
||||
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
|
||||
}
|
||||
|
||||
try {
|
||||
const url = new URL(context.request.url);
|
||||
const month = (url.searchParams.get("month") || new Date().toISOString().slice(0, 7)).trim();
|
||||
const page = toInt(url.searchParams.get("page"), 1);
|
||||
const limit = Math.min(100, toInt(url.searchParams.get("limit"), 10));
|
||||
const offset = (page - 1) * limit;
|
||||
|
||||
const baseWhere = "date LIKE ?";
|
||||
const monthArg = `${month}%`;
|
||||
|
||||
const dataSql = `
|
||||
SELECT
|
||||
date,
|
||||
SUM(CASE WHEN actual_result IS NOT NULL AND ensemble_prediction IS NOT NULL THEN 1 ELSE 0 END) AS ens_total,
|
||||
SUM(CASE WHEN ensemble_correct = 1 THEN 1 ELSE 0 END) AS ens_correct,
|
||||
|
||||
SUM(CASE WHEN actual_result IS NOT NULL AND dlinear_prediction IS NOT NULL THEN 1 ELSE 0 END) AS dl_total,
|
||||
SUM(CASE WHEN dlinear_correct = 1 THEN 1 ELSE 0 END) AS dl_correct,
|
||||
|
||||
SUM(CASE WHEN actual_result IS NOT NULL AND nbeats_prediction IS NOT NULL THEN 1 ELSE 0 END) AS nb_total,
|
||||
SUM(CASE WHEN nbeats_correct = 1 THEN 1 ELSE 0 END) AS nb_correct,
|
||||
|
||||
SUM(CASE WHEN actual_result IS NOT NULL AND ttm_prediction IS NOT NULL THEN 1 ELSE 0 END) AS ttm_total,
|
||||
SUM(CASE WHEN ttm_correct = 1 THEN 1 ELSE 0 END) AS ttm_correct
|
||||
FROM backup_hourly_model_predictions
|
||||
WHERE ${baseWhere}
|
||||
GROUP BY date
|
||||
ORDER BY date DESC
|
||||
LIMIT ? OFFSET ?
|
||||
`;
|
||||
|
||||
const countSql = `
|
||||
SELECT COUNT(*) AS cnt
|
||||
FROM (
|
||||
SELECT date
|
||||
FROM backup_hourly_model_predictions
|
||||
WHERE ${baseWhere}
|
||||
GROUP BY date
|
||||
) x
|
||||
`;
|
||||
|
||||
const dataResult = await db.prepare(dataSql).bind(monthArg, limit, offset).all();
|
||||
const countResult = await db.prepare(countSql).bind(monthArg).first();
|
||||
|
||||
const rows = (dataResult.results || []).map((r) => ({
|
||||
date: r.date,
|
||||
Ensemble: item(r.ens_total, r.ens_correct),
|
||||
DLinear: item(r.dl_total, r.dl_correct),
|
||||
"N-BEATS": item(r.nb_total, r.nb_correct),
|
||||
"TTM-R2": item(r.ttm_total, r.ttm_correct),
|
||||
}));
|
||||
|
||||
const total = Number(countResult?.cnt || 0);
|
||||
const totalPages = total > 0 ? Math.ceil(total / limit) : 1;
|
||||
|
||||
return json({
|
||||
success: true,
|
||||
data: rows,
|
||||
page,
|
||||
limit,
|
||||
total,
|
||||
total_pages: totalPages,
|
||||
});
|
||||
} catch (error) {
|
||||
return json({ success: false, error: String(error) }, 500);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: { "content-type": "application/json; charset=UTF-8" },
|
||||
});
|
||||
}
|
||||
|
||||
function modelItem(prediction, probability, isCorrect) {
|
||||
if (prediction === null || prediction === undefined) return null;
|
||||
return {
|
||||
prediction: Number(prediction),
|
||||
probability: probability === null || probability === undefined ? null : Number(probability),
|
||||
is_correct: isCorrect === null || isCorrect === undefined ? null : Number(isCorrect) === 1,
|
||||
};
|
||||
}
|
||||
|
||||
export async function onRequestGet(context) {
|
||||
const db = context.env.DB;
|
||||
if (!db) {
|
||||
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
|
||||
}
|
||||
|
||||
try {
|
||||
const date = context.params.date;
|
||||
const sql = `
|
||||
SELECT
|
||||
timestamp,
|
||||
hour,
|
||||
open_price,
|
||||
close_price,
|
||||
actual_result,
|
||||
dlinear_prediction,
|
||||
dlinear_prob,
|
||||
dlinear_correct,
|
||||
nbeats_prediction,
|
||||
nbeats_prob,
|
||||
nbeats_correct,
|
||||
ttm_prediction,
|
||||
ttm_prob,
|
||||
ttm_correct,
|
||||
ensemble_prediction,
|
||||
ensemble_correct
|
||||
FROM backup_hourly_model_predictions
|
||||
WHERE date = ?
|
||||
ORDER BY hour ASC
|
||||
`;
|
||||
|
||||
const result = await db.prepare(sql).bind(date).all();
|
||||
const rows = (result.results || []).map((r) => ({
|
||||
timestamp_cst: r.timestamp,
|
||||
hour: r.hour,
|
||||
open_price: r.open_price === null || r.open_price === undefined ? null : Number(r.open_price),
|
||||
close_price: r.close_price === null || r.close_price === undefined ? null : Number(r.close_price),
|
||||
actual_result: r.actual_result === null || r.actual_result === undefined ? null : Number(r.actual_result),
|
||||
models: {
|
||||
Ensemble: modelItem(r.ensemble_prediction, null, r.ensemble_correct),
|
||||
DLinear: modelItem(r.dlinear_prediction, r.dlinear_prob, r.dlinear_correct),
|
||||
"N-BEATS": modelItem(r.nbeats_prediction, r.nbeats_prob, r.nbeats_correct),
|
||||
"TTM-R2": modelItem(r.ttm_prediction, r.ttm_prob, r.ttm_correct),
|
||||
},
|
||||
}));
|
||||
|
||||
return json({ success: true, date, data: rows });
|
||||
} catch (error) {
|
||||
return json({ success: false, error: String(error) }, 500);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: { "content-type": "application/json; charset=UTF-8" },
|
||||
});
|
||||
}
|
||||
|
||||
function ratio(correct, total) {
|
||||
if (!total) return 0;
|
||||
return Math.round((correct / total) * 10000) / 100;
|
||||
}
|
||||
|
||||
export async function onRequestGet(context) {
|
||||
const db = context.env.DB;
|
||||
if (!db) {
|
||||
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
|
||||
}
|
||||
|
||||
try {
|
||||
const url = new URL(context.request.url);
|
||||
const month = (url.searchParams.get("month") || new Date().toISOString().slice(0, 7)).trim();
|
||||
|
||||
const sql = `
|
||||
SELECT
|
||||
SUM(CASE WHEN actual_result IS NOT NULL AND ensemble_prediction IS NOT NULL THEN 1 ELSE 0 END) AS ens_total,
|
||||
SUM(CASE WHEN ensemble_correct = 1 THEN 1 ELSE 0 END) AS ens_correct,
|
||||
|
||||
SUM(CASE WHEN actual_result IS NOT NULL AND dlinear_prediction IS NOT NULL THEN 1 ELSE 0 END) AS dl_total,
|
||||
SUM(CASE WHEN dlinear_correct = 1 THEN 1 ELSE 0 END) AS dl_correct,
|
||||
|
||||
SUM(CASE WHEN actual_result IS NOT NULL AND nbeats_prediction IS NOT NULL THEN 1 ELSE 0 END) AS nb_total,
|
||||
SUM(CASE WHEN nbeats_correct = 1 THEN 1 ELSE 0 END) AS nb_correct,
|
||||
|
||||
SUM(CASE WHEN actual_result IS NOT NULL AND ttm_prediction IS NOT NULL THEN 1 ELSE 0 END) AS ttm_total,
|
||||
SUM(CASE WHEN ttm_correct = 1 THEN 1 ELSE 0 END) AS ttm_correct
|
||||
FROM backup_hourly_model_predictions
|
||||
WHERE date LIKE ?
|
||||
`;
|
||||
|
||||
const row = await db.prepare(sql).bind(`${month}%`).first();
|
||||
|
||||
const ensTotal = Number(row?.ens_total || 0);
|
||||
const ensCorrect = Number(row?.ens_correct || 0);
|
||||
|
||||
const dlTotal = Number(row?.dl_total || 0);
|
||||
const dlCorrect = Number(row?.dl_correct || 0);
|
||||
|
||||
const nbTotal = Number(row?.nb_total || 0);
|
||||
const nbCorrect = Number(row?.nb_correct || 0);
|
||||
|
||||
const ttmTotal = Number(row?.ttm_total || 0);
|
||||
const ttmCorrect = Number(row?.ttm_correct || 0);
|
||||
|
||||
return json({
|
||||
success: true,
|
||||
month,
|
||||
data: {
|
||||
Ensemble: {
|
||||
total: ensTotal,
|
||||
correct: ensCorrect,
|
||||
incorrect: Math.max(0, ensTotal - ensCorrect),
|
||||
accuracy: ratio(ensCorrect, ensTotal),
|
||||
},
|
||||
DLinear: {
|
||||
total: dlTotal,
|
||||
correct: dlCorrect,
|
||||
incorrect: Math.max(0, dlTotal - dlCorrect),
|
||||
accuracy: ratio(dlCorrect, dlTotal),
|
||||
},
|
||||
"N-BEATS": {
|
||||
total: nbTotal,
|
||||
correct: nbCorrect,
|
||||
incorrect: Math.max(0, nbTotal - nbCorrect),
|
||||
accuracy: ratio(nbCorrect, nbTotal),
|
||||
},
|
||||
"TTM-R2": {
|
||||
total: ttmTotal,
|
||||
correct: ttmCorrect,
|
||||
incorrect: Math.max(0, ttmTotal - ttmCorrect),
|
||||
accuracy: ratio(ttmCorrect, ttmTotal),
|
||||
},
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return json({ success: false, error: String(error) }, 500);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: { "content-type": "application/json; charset=UTF-8" },
|
||||
});
|
||||
}
|
||||
|
||||
export async function onRequestGet(context) {
|
||||
return json({
|
||||
success: true,
|
||||
status: "running",
|
||||
server_time: new Date().toISOString(),
|
||||
version: "pages-1.0.0",
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: { "content-type": "application/json; charset=UTF-8" },
|
||||
});
|
||||
}
|
||||
|
||||
export async function onRequestGet(context) {
|
||||
const db = context.env.DB;
|
||||
if (!db) {
|
||||
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
|
||||
}
|
||||
|
||||
try {
|
||||
const row = await db
|
||||
.prepare("SELECT * FROM backup_trades ORDER BY timestamp DESC LIMIT 1")
|
||||
.first();
|
||||
|
||||
return json({ success: true, data: row || null });
|
||||
} catch (error) {
|
||||
return json({ success: false, error: String(error) }, 500);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: { "content-type": "application/json; charset=UTF-8" },
|
||||
});
|
||||
}
|
||||
|
||||
function toInt(v, d) {
|
||||
const n = Number(v);
|
||||
return Number.isFinite(n) && n > 0 ? Math.floor(n) : d;
|
||||
}
|
||||
|
||||
export async function onRequestGet(context) {
|
||||
const db = context.env.DB;
|
||||
if (!db) {
|
||||
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
|
||||
}
|
||||
|
||||
try {
|
||||
const url = new URL(context.request.url);
|
||||
const page = toInt(url.searchParams.get("page"), 1);
|
||||
const limit = Math.min(100, toInt(url.searchParams.get("limit"), 10));
|
||||
const month = (url.searchParams.get("month") || "").trim();
|
||||
const day = (url.searchParams.get("day") || "").trim();
|
||||
const offset = (page - 1) * limit;
|
||||
|
||||
const where = ["1=1"];
|
||||
const params = [];
|
||||
|
||||
// 时间字段是 YYYY-MM-DD HH:MM:SS 文本,直接前缀过滤可走索引/减少转换成本
|
||||
if (month) {
|
||||
where.push("timestamp LIKE ?");
|
||||
params.push(`${month}%`);
|
||||
}
|
||||
if (day) {
|
||||
where.push("timestamp LIKE ?");
|
||||
params.push(`${day}%`);
|
||||
}
|
||||
|
||||
const whereSql = where.join(" AND ");
|
||||
const dataSql = `
|
||||
SELECT *
|
||||
FROM backup_trades
|
||||
WHERE ${whereSql}
|
||||
ORDER BY timestamp DESC
|
||||
LIMIT ? OFFSET ?
|
||||
`;
|
||||
const countSql = `
|
||||
SELECT COUNT(*) AS cnt
|
||||
FROM backup_trades
|
||||
WHERE ${whereSql}
|
||||
`;
|
||||
|
||||
const dataResult = await db.prepare(dataSql).bind(...params, limit, offset).all();
|
||||
const countResult = await db.prepare(countSql).bind(...params).first();
|
||||
|
||||
const total = Number(countResult?.cnt || 0);
|
||||
const totalPages = total > 0 ? Math.ceil(total / limit) : 1;
|
||||
|
||||
return json({
|
||||
success: true,
|
||||
data: dataResult.results || [],
|
||||
page,
|
||||
limit,
|
||||
total,
|
||||
total_pages: totalPages,
|
||||
});
|
||||
} catch (error) {
|
||||
return json({ success: false, error: String(error) }, 500);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,200 @@
|
||||
const { createApp } = Vue;
|
||||
|
||||
const app = createApp({
|
||||
data() {
|
||||
return {
|
||||
loading: false,
|
||||
activeTab: 'trades',
|
||||
serverTime: '',
|
||||
currentTrade: null,
|
||||
tradeHistory: {
|
||||
data: [],
|
||||
page: 1,
|
||||
limit: 10,
|
||||
total: 0,
|
||||
total_pages: 0,
|
||||
},
|
||||
tradeFilter: {
|
||||
month: '',
|
||||
day: '',
|
||||
},
|
||||
accuracyFilter: {
|
||||
month: '',
|
||||
},
|
||||
monthlyStats: {},
|
||||
dailyStats: {
|
||||
data: [],
|
||||
page: 1,
|
||||
limit: 10,
|
||||
total: 0,
|
||||
total_pages: 0,
|
||||
},
|
||||
hourlyModal: {
|
||||
date: '',
|
||||
data: [],
|
||||
},
|
||||
};
|
||||
},
|
||||
|
||||
mounted() {
|
||||
this.accuracyFilter.month = this.getCurrentMonth();
|
||||
this.loadCurrentTrade();
|
||||
this.loadTradeHistory(1);
|
||||
this.updateServerTime();
|
||||
setInterval(() => this.updateServerTime(), 60000);
|
||||
},
|
||||
|
||||
methods: {
|
||||
getCurrentMonth() {
|
||||
const now = new Date();
|
||||
return `${now.getFullYear()}-${String(now.getMonth() + 1).padStart(2, '0')}`;
|
||||
},
|
||||
|
||||
updateServerTime() {
|
||||
const now = new Date();
|
||||
this.serverTime = now.toLocaleString('zh-CN', {
|
||||
year: 'numeric',
|
||||
month: '2-digit',
|
||||
day: '2-digit',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
});
|
||||
},
|
||||
|
||||
formatTime(timestamp) {
|
||||
if (!timestamp) return '-';
|
||||
return String(timestamp).slice(0, 19);
|
||||
},
|
||||
|
||||
formatPercent(value) {
|
||||
if (value === null || value === undefined) return '-';
|
||||
return `${(Number(value) * 100).toFixed(1)}%`;
|
||||
},
|
||||
|
||||
getPredictionClass(prediction) {
|
||||
if (prediction === 'UP') return 'text-success fw-bold';
|
||||
if (prediction === 'DOWN') return 'text-danger fw-bold';
|
||||
return '';
|
||||
},
|
||||
|
||||
getStatusClass(status) {
|
||||
switch (status) {
|
||||
case 'success': return 'text-success';
|
||||
case 'test': return 'text-info';
|
||||
case 'failure': return 'text-danger';
|
||||
case 'in_progress': return 'text-warning';
|
||||
default: return 'text-muted';
|
||||
}
|
||||
},
|
||||
|
||||
getStatusText(status) {
|
||||
switch (status) {
|
||||
case 'success': return '成功';
|
||||
case 'test': return '测试';
|
||||
case 'failure': return '失败';
|
||||
case 'in_progress': return '进行中';
|
||||
default: return status || '-';
|
||||
}
|
||||
},
|
||||
|
||||
getAccuracyClass(accuracy) {
|
||||
if (accuracy >= 60) return 'text-success fw-bold';
|
||||
if (accuracy >= 50) return 'text-warning';
|
||||
return 'text-danger';
|
||||
},
|
||||
|
||||
async fetchAPI(url) {
|
||||
try {
|
||||
const response = await fetch(url);
|
||||
return await response.json();
|
||||
} catch (error) {
|
||||
return { success: false, error: String(error) };
|
||||
}
|
||||
},
|
||||
|
||||
async loadCurrentTrade() {
|
||||
const data = await this.fetchAPI('/api/trades/current');
|
||||
if (data.success) {
|
||||
this.currentTrade = data.data;
|
||||
}
|
||||
},
|
||||
|
||||
async loadTradeHistory(page) {
|
||||
if (page < 1) return;
|
||||
this.loading = true;
|
||||
|
||||
let url = `/api/trades/history?page=${page}&limit=${this.tradeHistory.limit}`;
|
||||
if (this.tradeFilter.month) {
|
||||
url += `&month=${this.tradeFilter.month}`;
|
||||
}
|
||||
if (this.tradeFilter.day) {
|
||||
url += `&day=${this.tradeFilter.day}`;
|
||||
}
|
||||
|
||||
const data = await this.fetchAPI(url);
|
||||
if (data.success) {
|
||||
this.tradeHistory = {
|
||||
data: data.data,
|
||||
page: data.page,
|
||||
limit: data.limit,
|
||||
total: data.total,
|
||||
total_pages: data.total_pages,
|
||||
};
|
||||
}
|
||||
|
||||
this.loading = false;
|
||||
},
|
||||
|
||||
clearTradeFilter() {
|
||||
this.tradeFilter.month = '';
|
||||
this.tradeFilter.day = '';
|
||||
this.loadTradeHistory(1);
|
||||
},
|
||||
|
||||
async loadAccuracyData() {
|
||||
this.loading = true;
|
||||
const monthlyData = await this.fetchAPI(`/api/accuracy/monthly?month=${this.accuracyFilter.month}`);
|
||||
if (monthlyData.success) {
|
||||
this.monthlyStats = monthlyData.data || {};
|
||||
}
|
||||
await this.loadDailyStats(1);
|
||||
this.loading = false;
|
||||
},
|
||||
|
||||
async loadDailyStats(page) {
|
||||
if (page < 1) return;
|
||||
const data = await this.fetchAPI(`/api/accuracy/daily?month=${this.accuracyFilter.month}&page=${page}&limit=10`);
|
||||
if (data.success) {
|
||||
this.dailyStats = {
|
||||
data: data.data,
|
||||
page: data.page,
|
||||
limit: data.limit,
|
||||
total: data.total,
|
||||
total_pages: data.total_pages,
|
||||
};
|
||||
}
|
||||
},
|
||||
|
||||
async showHourlyDetails(date) {
|
||||
this.loading = true;
|
||||
this.hourlyModal.date = date;
|
||||
const data = await this.fetchAPI(`/api/accuracy/hourly/${date}`);
|
||||
if (data.success) {
|
||||
this.hourlyModal.data = data.data || [];
|
||||
}
|
||||
this.loading = false;
|
||||
const modal = new bootstrap.Modal(document.getElementById('hourlyModal'));
|
||||
modal.show();
|
||||
},
|
||||
},
|
||||
|
||||
watch: {
|
||||
activeTab(newTab) {
|
||||
if (newTab === 'accuracy' && !this.dailyStats.data.length) {
|
||||
this.loadAccuracyData();
|
||||
}
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
app.mount('#app');
|
||||
@@ -0,0 +1,251 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>BTC 预测交易看板 (Pages + D1)</title>
|
||||
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.11.1/font/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="/style.css" rel="stylesheet">
|
||||
</head>
|
||||
<body>
|
||||
<div id="app">
|
||||
<nav class="navbar navbar-expand-lg navbar-dark bg-dark">
|
||||
<div class="container">
|
||||
<a class="navbar-brand" href="#">
|
||||
<i class="bi bi-currency-bitcoin"></i> BTC 预测交易看板 (Pages)
|
||||
</a>
|
||||
<span class="navbar-text text-light">
|
||||
<i class="bi bi-clock"></i> {{ serverTime }}
|
||||
</span>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div class="container mt-4">
|
||||
<ul class="nav nav-tabs mb-4" role="tablist">
|
||||
<li class="nav-item">
|
||||
<button class="nav-link" :class="{ active: activeTab === 'trades' }" @click="activeTab = 'trades'">
|
||||
<i class="bi bi-graph-up"></i> 交易详情
|
||||
</button>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<button class="nav-link" :class="{ active: activeTab === 'accuracy' }" @click="activeTab = 'accuracy'">
|
||||
<i class="bi bi-bullseye"></i> 模型准确率
|
||||
</button>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<div v-show="activeTab === 'trades'">
|
||||
<div class="card mb-4">
|
||||
<div class="card-header bg-primary text-white">
|
||||
<i class="bi bi-lightning-charge"></i> 当前小时交易
|
||||
</div>
|
||||
<div class="card-body">
|
||||
<div v-if="currentTrade" class="row">
|
||||
<div class="col-md-3">
|
||||
<div class="stat-card">
|
||||
<div class="stat-label">市场</div>
|
||||
<div class="stat-value small">{{ currentTrade.market_title || '-' }}</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-2">
|
||||
<div class="stat-card">
|
||||
<div class="stat-label">预测方向</div>
|
||||
<div class="stat-value" :class="getPredictionClass(currentTrade.prediction)">
|
||||
{{ currentTrade.prediction || '-' }}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-2">
|
||||
<div class="stat-card">
|
||||
<div class="stat-label">概率</div>
|
||||
<div class="stat-value">{{ formatPercent(currentTrade.probability) }}</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-2">
|
||||
<div class="stat-card">
|
||||
<div class="stat-label">策略</div>
|
||||
<div class="stat-value small">{{ currentTrade.strategy || 'Ensemble' }}</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-3">
|
||||
<div class="stat-card">
|
||||
<div class="stat-label">状态</div>
|
||||
<div class="stat-value" :class="getStatusClass(currentTrade.status)">
|
||||
{{ getStatusText(currentTrade.status) }}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div v-else class="text-center text-muted py-4">
|
||||
<i class="bi bi-inbox"></i> 暂无交易记录
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="card">
|
||||
<div class="card-header d-flex justify-content-between align-items-center">
|
||||
<span><i class="bi bi-clock-history"></i> 历史交易记录</span>
|
||||
<div class="d-flex gap-2">
|
||||
<input type="month" class="form-control form-control-sm" v-model="tradeFilter.month" @change="loadTradeHistory(1)" style="width: 150px;">
|
||||
<input type="date" class="form-control form-control-sm" v-model="tradeFilter.day" @change="loadTradeHistory(1)" style="width: 150px;">
|
||||
<button class="btn btn-sm btn-outline-secondary" @click="clearTradeFilter">
|
||||
<i class="bi bi-x-circle"></i> 清除
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="card-body p-0">
|
||||
<div class="table-responsive">
|
||||
<table class="table table-hover table-striped mb-0">
|
||||
<thead class="table-light">
|
||||
<tr>
|
||||
<th>时间</th>
|
||||
<th>市场</th>
|
||||
<th>策略</th>
|
||||
<th>预测</th>
|
||||
<th>概率</th>
|
||||
<th>价格</th>
|
||||
<th>状态</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr v-for="trade in tradeHistory.data" :key="trade.id">
|
||||
<td>{{ formatTime(trade.timestamp) }}</td>
|
||||
<td class="text-truncate" style="max-width: 200px;">{{ trade.market_title }}</td>
|
||||
<td><span class="badge bg-secondary">{{ trade.strategy || 'Ensemble' }}</span></td>
|
||||
<td :class="getPredictionClass(trade.prediction)">{{ trade.prediction }}</td>
|
||||
<td>{{ formatPercent(trade.probability) }}</td>
|
||||
<td>{{ trade.price ?? '-' }}</td>
|
||||
<td :class="getStatusClass(trade.status)">{{ getStatusText(trade.status) }}</td>
|
||||
</tr>
|
||||
<tr v-if="!tradeHistory.data?.length"><td colspan="7" class="text-center text-muted py-4">暂无数据</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
<div class="card-footer d-flex justify-content-between align-items-center">
|
||||
<span class="text-muted">共 {{ tradeHistory.total }} 条</span>
|
||||
<nav>
|
||||
<ul class="pagination pagination-sm mb-0">
|
||||
<li class="page-item" :class="{ disabled: tradeHistory.page <= 1 }">
|
||||
<a class="page-link" href="#" @click.prevent="loadTradeHistory(tradeHistory.page - 1)">上一页</a>
|
||||
</li>
|
||||
<li class="page-item disabled"><span class="page-link">{{ tradeHistory.page }} / {{ tradeHistory.total_pages }}</span></li>
|
||||
<li class="page-item" :class="{ disabled: tradeHistory.page >= tradeHistory.total_pages }">
|
||||
<a class="page-link" href="#" @click.prevent="loadTradeHistory(tradeHistory.page + 1)">下一页</a>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div v-show="activeTab === 'accuracy'">
|
||||
<div class="mb-4">
|
||||
<div class="d-flex gap-2 align-items-center">
|
||||
<label class="form-label mb-0">选择月份:</label>
|
||||
<input type="month" class="form-control" v-model="accuracyFilter.month" @change="loadAccuracyData" style="width: 180px;">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row mb-4">
|
||||
<div class="col-md-3"><div class="card h-100 border-primary shadow-sm"><div class="card-header bg-primary text-white text-center">集成策略</div><div class="card-body text-center" v-if="monthlyStats.Ensemble"><h2 class="display-6 text-primary fw-bold">{{ monthlyStats.Ensemble.accuracy }}%</h2><div class="text-muted mb-2">准确率</div><div class="small text-muted">{{ monthlyStats.Ensemble.correct }}/{{ monthlyStats.Ensemble.total }}</div></div><div v-else class="text-center text-muted py-3">暂无数据</div></div></div>
|
||||
<div class="col-md-3"><div class="card h-100 border-secondary"><div class="card-header bg-secondary text-white text-center">DLinear</div><div class="card-body text-center" v-if="monthlyStats.DLinear"><h3 class="text-dark">{{ monthlyStats.DLinear.accuracy }}%</h3><div class="small text-muted">{{ monthlyStats.DLinear.correct }}/{{ monthlyStats.DLinear.total }}</div></div><div v-else class="text-center text-muted py-3">暂无数据</div></div></div>
|
||||
<div class="col-md-3"><div class="card h-100 border-secondary"><div class="card-header bg-secondary text-white text-center">N-BEATS</div><div class="card-body text-center" v-if="monthlyStats['N-BEATS']"><h3 class="text-dark">{{ monthlyStats['N-BEATS'].accuracy }}%</h3><div class="small text-muted">{{ monthlyStats['N-BEATS'].correct }}/{{ monthlyStats['N-BEATS'].total }}</div></div><div v-else class="text-center text-muted py-3">暂无数据</div></div></div>
|
||||
<div class="col-md-3"><div class="card h-100 border-secondary"><div class="card-header bg-secondary text-white text-center">TTM-R2</div><div class="card-body text-center" v-if="monthlyStats['TTM-R2']"><h3 class="text-dark">{{ monthlyStats['TTM-R2'].accuracy }}%</h3><div class="small text-muted">{{ monthlyStats['TTM-R2'].correct }}/{{ monthlyStats['TTM-R2'].total }}</div></div><div v-else class="text-center text-muted py-3">暂无数据</div></div></div>
|
||||
</div>
|
||||
|
||||
<div class="card">
|
||||
<div class="card-header"><i class="bi bi-calendar3"></i> 每日准确率概览</div>
|
||||
<div class="card-body p-0">
|
||||
<div class="table-responsive">
|
||||
<table class="table table-hover mb-0 text-center">
|
||||
<thead class="table-light">
|
||||
<tr>
|
||||
<th class="text-start ps-4">日期</th>
|
||||
<th>集成策略</th>
|
||||
<th>DLinear</th>
|
||||
<th>N-BEATS</th>
|
||||
<th>TTM-R2</th>
|
||||
<th>操作</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr v-for="day in dailyStats.data" :key="day.date">
|
||||
<td class="text-start ps-4">{{ day.date }}</td>
|
||||
<td><span v-if="day.Ensemble" :class="getAccuracyClass(day.Ensemble.accuracy)">{{ day.Ensemble.accuracy }}%</span><span v-else class="text-muted">-</span></td>
|
||||
<td><span v-if="day.DLinear" :class="getAccuracyClass(day.DLinear.accuracy)">{{ day.DLinear.accuracy }}%</span><span v-else class="text-muted">-</span></td>
|
||||
<td><span v-if="day['N-BEATS']" :class="getAccuracyClass(day['N-BEATS'].accuracy)">{{ day['N-BEATS'].accuracy }}%</span><span v-else class="text-muted">-</span></td>
|
||||
<td><span v-if="day['TTM-R2']" :class="getAccuracyClass(day['TTM-R2'].accuracy)">{{ day['TTM-R2'].accuracy }}%</span><span v-else class="text-muted">-</span></td>
|
||||
<td><button class="btn btn-sm btn-outline-primary" @click="showHourlyDetails(day.date)"><i class="bi bi-eye"></i> 详情</button></td>
|
||||
</tr>
|
||||
<tr v-if="!dailyStats.data?.length"><td colspan="6" class="text-center text-muted py-4">暂无数据</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
<div class="card-footer d-flex justify-content-between align-items-center">
|
||||
<span class="text-muted">共 {{ dailyStats.total }} 天</span>
|
||||
<nav>
|
||||
<ul class="pagination pagination-sm mb-0">
|
||||
<li class="page-item" :class="{ disabled: dailyStats.page <= 1 }"><a class="page-link" href="#" @click.prevent="loadDailyStats(dailyStats.page - 1)">上一页</a></li>
|
||||
<li class="page-item disabled"><span class="page-link">{{ dailyStats.page }} / {{ dailyStats.total_pages }}</span></li>
|
||||
<li class="page-item" :class="{ disabled: dailyStats.page >= dailyStats.total_pages }"><a class="page-link" href="#" @click.prevent="loadDailyStats(dailyStats.page + 1)">下一页</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="modal fade" id="hourlyModal" tabindex="-1">
|
||||
<div class="modal-dialog modal-xl modal-dialog-scrollable">
|
||||
<div class="modal-content">
|
||||
<div class="modal-header">
|
||||
<h5 class="modal-title"><i class="bi bi-clock"></i> {{ hourlyModal.date }} 每小时预测详情</h5>
|
||||
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
|
||||
</div>
|
||||
<div class="modal-body p-0">
|
||||
<table class="table table-sm table-hover mb-0 text-center align-middle">
|
||||
<thead class="table-light sticky-top">
|
||||
<tr>
|
||||
<th>时间</th>
|
||||
<th>开盘</th>
|
||||
<th>收盘</th>
|
||||
<th>实际</th>
|
||||
<th class="table-primary">集成策略</th>
|
||||
<th>DLinear</th>
|
||||
<th>N-BEATS</th>
|
||||
<th>TTM-R2</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr v-for="hour in hourlyModal.data" :key="hour.hour">
|
||||
<td>{{ hour.timestamp_cst ? hour.timestamp_cst.split(' ')[1] : (hour.hour + ':00') }}</td>
|
||||
<td>{{ hour.open_price ?? '-' }}</td>
|
||||
<td>{{ hour.close_price ?? '-' }}</td>
|
||||
<td :class="hour.actual_result === 1 ? 'text-success fw-bold' : (hour.actual_result === 0 ? 'text-danger fw-bold' : 'text-muted')">{{ hour.actual_result === 1 ? 'UP' : (hour.actual_result === 0 ? 'DOWN' : '-') }}</td>
|
||||
<td class="table-primary bg-opacity-10"><span v-if="hour.models?.Ensemble" :class="hour.models.Ensemble.is_correct ? 'text-success' : 'text-danger'">{{ hour.models.Ensemble.prediction === 1 ? 'UP' : 'DOWN' }}</span></td>
|
||||
<td><span v-if="hour.models?.DLinear" :class="hour.models.DLinear.is_correct ? 'text-success' : 'text-danger'">{{ hour.models.DLinear.prediction === 1 ? 'UP' : 'DOWN' }}</span></td>
|
||||
<td><span v-if="hour.models?.['N-BEATS']" :class="hour.models['N-BEATS'].is_correct ? 'text-success' : 'text-danger'">{{ hour.models['N-BEATS'].prediction === 1 ? 'UP' : 'DOWN' }}</span></td>
|
||||
<td><span v-if="hour.models?.['TTM-R2']" :class="hour.models['TTM-R2'].is_correct ? 'text-success' : 'text-danger'">{{ hour.models['TTM-R2'].prediction === 1 ? 'UP' : 'DOWN' }}</span></td>
|
||||
</tr>
|
||||
<tr v-if="!hourlyModal.data?.length"><td colspan="8" class="text-center text-muted py-4">暂无数据</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<div class="modal-footer"><button type="button" class="btn btn-secondary" data-bs-dismiss="modal">关闭</button></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div v-if="loading" class="loading-overlay">
|
||||
<div class="spinner-border text-primary" role="status"><span class="visually-hidden">加载中...</span></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/js/bootstrap.bundle.min.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/vue@3.4.15/dist/vue.global.prod.js"></script>
|
||||
<script src="/app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,118 @@
|
||||
body {
|
||||
background-color: #f8f9fa;
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
|
||||
}
|
||||
|
||||
.navbar-brand {
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.navbar-brand i {
|
||||
color: #f7931a;
|
||||
}
|
||||
|
||||
.nav-tabs .nav-link {
|
||||
color: #6c757d;
|
||||
border: none;
|
||||
padding: 0.75rem 1.5rem;
|
||||
}
|
||||
|
||||
.nav-tabs .nav-link.active {
|
||||
color: #0d6efd;
|
||||
border-bottom: 3px solid #0d6efd;
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.nav-tabs .nav-link:hover:not(.active) {
|
||||
border-bottom: 3px solid #dee2e6;
|
||||
}
|
||||
|
||||
.card {
|
||||
box-shadow: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);
|
||||
border-radius: 0.5rem;
|
||||
}
|
||||
|
||||
.card-header {
|
||||
font-weight: 600;
|
||||
border-radius: 0.5rem 0.5rem 0 0 !important;
|
||||
}
|
||||
|
||||
.stat-card {
|
||||
padding: 1rem;
|
||||
text-align: center;
|
||||
background: #f8f9fa;
|
||||
border-radius: 0.5rem;
|
||||
height: 100%;
|
||||
}
|
||||
|
||||
.stat-label {
|
||||
font-size: 0.75rem;
|
||||
color: #6c757d;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
margin-bottom: 0.25rem;
|
||||
}
|
||||
|
||||
.stat-value {
|
||||
font-size: 1.25rem;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.table {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
|
||||
.table th {
|
||||
font-weight: 600;
|
||||
font-size: 0.875rem;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
color: #6c757d;
|
||||
border-bottom: 2px solid #dee2e6;
|
||||
}
|
||||
|
||||
.table td {
|
||||
vertical-align: middle;
|
||||
}
|
||||
|
||||
.table-hover tbody tr:hover {
|
||||
background-color: rgba(13, 110, 253, 0.05);
|
||||
}
|
||||
|
||||
.loading-overlay {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background: rgba(255, 255, 255, 0.8);
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
z-index: 9999;
|
||||
}
|
||||
|
||||
.modal-content {
|
||||
border-radius: 0.5rem;
|
||||
}
|
||||
|
||||
.modal-header {
|
||||
background: #f8f9fa;
|
||||
border-radius: 0.5rem 0.5rem 0 0;
|
||||
}
|
||||
|
||||
@media (max-width: 768px) {
|
||||
.stat-card {
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.card-header .d-flex {
|
||||
flex-direction: column;
|
||||
gap: 0.5rem !important;
|
||||
}
|
||||
|
||||
.nav-tabs .nav-link {
|
||||
padding: 0.5rem 1rem;
|
||||
font-size: 0.875rem;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
name = "trades"
|
||||
compatibility_date = "2026-03-10"
|
||||
|
||||
[vars]
|
||||
APP_TITLE = "BTC 预测看板"
|
||||
|
||||
[[d1_databases]]
|
||||
binding = "DB"
|
||||
database_name = "orders"
|
||||
database_id = "ae0cd6aa-10da-4459-9768-8e6ffad55a00"
|
||||
@@ -0,0 +1,11 @@
|
||||
name = "trades"
|
||||
compatibility_date = "2026-03-10"
|
||||
|
||||
[vars]
|
||||
APP_TITLE = "BTC 预测看板"
|
||||
|
||||
[[d1_databases]]
|
||||
binding = "DB"
|
||||
database_name = "orders"
|
||||
database_id = "ae0cd6aa-10da-4459-9768-8e6ffad55a00"
|
||||
|
||||
@@ -0,0 +1,380 @@
|
||||
"""对比 test 旧逻辑 vs auto_trade 逻辑的数据处理差异
|
||||
|
||||
两种预处理方式:
|
||||
- test旧逻辑:整个df一次性预处理,无create_labels,按索引遍历窗口
|
||||
- auto_trade逻辑:截取到当前K线的df,含create_labels,取X[-1]预测
|
||||
|
||||
对比内容:
|
||||
1. 预处理后的df行数差异
|
||||
2. 同一时间点的特征值差异(RSI、MACD等)
|
||||
3. scaler transform后的值差异
|
||||
4. 最终输入窗口的数值差异
|
||||
5. 模型输出logits/概率/预测结果差异
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pickle
|
||||
import pandas_ta as ta
|
||||
from datetime import timedelta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
|
||||
# ==================== pickle反序列化需要的占位类 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""pkl文件中保存的类,pickle.load时需要能找到这个类名"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
# ==================== 两种预处理器 ====================
|
||||
|
||||
class TestPreprocessor:
|
||||
"""test旧逻辑:无create_labels"""
|
||||
def __init__(self, lookback=168):
|
||||
self.lookback = lookback
|
||||
self.scaler = StandardScaler()
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X = []
|
||||
valid_indices = []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
valid_indices.append(i + self.lookback - 1)
|
||||
|
||||
return np.array(X), df, valid_indices, scaled_features
|
||||
|
||||
|
||||
class AutoTradePreprocessor:
|
||||
"""auto_trade逻辑:含create_labels"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
|
||||
# create_labels: shift + 砍最后一行
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
df = df[:-self.forecast_horizon]
|
||||
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X = []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
|
||||
return np.array(X), df, scaled_features
|
||||
|
||||
|
||||
def utc_to_beijing(ts):
|
||||
return ts + timedelta(hours=8)
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 100)
|
||||
print("数据处理对比:test旧逻辑 vs auto_trade逻辑")
|
||||
print("=" * 100)
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
|
||||
# 加载模型和训练时的scaler
|
||||
from models.dlinear.model import DLinear
|
||||
checkpoint = torch.load('./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device, weights_only=False)
|
||||
model = DLinear(checkpoint['config']).to(device).eval()
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
trained_preprocessor = pickle.load(f)
|
||||
|
||||
# 提取训练时的scaler参数
|
||||
trained_scaler = trained_preprocessor.scaler
|
||||
print(f"训练时scaler均值: {trained_scaler.mean_}")
|
||||
print(f"训练时scaler方差: {trained_scaler.scale_}")
|
||||
|
||||
# 获取数据
|
||||
print("\n获取数据...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
df_full = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', end_date='2026-03-01', use_cache=False)
|
||||
print(f"总行数: {len(df_full)}")
|
||||
|
||||
# 选几个2月10日(北京时间)的时间点做对比
|
||||
# 北京时间02-10 00:00 = UTC 02-09 16:00
|
||||
test_targets_utc = [
|
||||
'2026-02-09 16:00:00', # 北京 02-10 00:00
|
||||
'2026-02-09 20:00:00', # 北京 02-10 04:00
|
||||
'2026-02-10 00:00:00', # 北京 02-10 08:00
|
||||
'2026-02-10 04:00:00', # 北京 02-10 12:00
|
||||
'2026-02-10 08:00:00', # 北京 02-10 16:00
|
||||
'2026-02-10 12:00:00', # 北京 02-10 20:00
|
||||
]
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
for target_utc_str in test_targets_utc:
|
||||
target_utc = pd.Timestamp(target_utc_str)
|
||||
target_bj = utc_to_beijing(target_utc)
|
||||
|
||||
# 找到目标K线在df中的索引
|
||||
target_mask = df_full['timestamp'] == target_utc
|
||||
if not target_mask.any():
|
||||
print(f"\n⚠ 找不到 {target_utc_str},跳过")
|
||||
continue
|
||||
|
||||
target_idx = df_full[target_mask].index[0]
|
||||
input_idx = target_idx - 1 # 输入的最后一根K线
|
||||
|
||||
input_row = df_full.iloc[input_idx]
|
||||
target_row = df_full.iloc[target_idx]
|
||||
|
||||
print(f"\n{'=' * 100}")
|
||||
print(f"目标K线: 北京{target_bj.strftime('%m-%d %H:%M')} (UTC {target_utc_str})")
|
||||
print(f"输入末根: UTC {input_row['timestamp']} O:{input_row['open']:.2f} H:{input_row['high']:.2f} L:{input_row['low']:.2f} C:{input_row['close']:.2f}")
|
||||
print(f"目标K线: UTC {target_row['timestamp']} 收盘:{target_row['close']:.2f}")
|
||||
print("-" * 100)
|
||||
|
||||
# ===== 方式A: test旧逻辑 =====
|
||||
# 对整个df一次性预处理,按索引找到对应窗口
|
||||
test_prep = TestPreprocessor(lookback=168)
|
||||
test_prep.scaler = trained_scaler # 使用训练时的scaler
|
||||
X_test, df_test, idx_test, scaled_test = test_prep.preprocess(df_full, fit=False)
|
||||
|
||||
# 找到input_idx对应的窗口
|
||||
# idx_test[i] = i + 168 - 1,所以 input_idx 在 df_test 中的位置需要查找
|
||||
# df_test 是 dropna 后 reset_index 的,timestamp 可能对应不同的行号
|
||||
test_input_mask = df_test['timestamp'] == input_row['timestamp']
|
||||
if test_input_mask.any():
|
||||
test_df_idx = df_test[test_input_mask].index[0]
|
||||
# 对应的窗口序号
|
||||
test_window_i = test_df_idx - 168 + 1
|
||||
if 0 <= test_window_i < len(X_test):
|
||||
X_test_window = X_test[test_window_i]
|
||||
else:
|
||||
X_test_window = None
|
||||
print(" [test] 窗口索引越界")
|
||||
else:
|
||||
test_df_idx = None
|
||||
X_test_window = None
|
||||
print(" [test] 在df_test中找不到对应时间")
|
||||
|
||||
# ===== 方式B: auto_trade逻辑 =====
|
||||
# 截取到input_idx为止,含create_labels
|
||||
df_up_to_t = df_full.iloc[:input_idx + 1]
|
||||
auto_prep = AutoTradePreprocessor(lookback=168)
|
||||
auto_prep.scaler = trained_scaler # 使用训练时的scaler
|
||||
X_auto, df_auto, scaled_auto = auto_prep.preprocess(df_up_to_t, fit=False)
|
||||
|
||||
if len(X_auto) > 0:
|
||||
X_auto_window = X_auto[-1] # 取最后一个窗口
|
||||
else:
|
||||
X_auto_window = None
|
||||
print(" [auto] X为空")
|
||||
|
||||
# ===== 对比 =====
|
||||
|
||||
# 1. df行数对比
|
||||
print(f"\n[1] 预处理后df行数")
|
||||
print(f" test: {len(df_test)} 行 (整个df, 无create_labels)")
|
||||
print(f" auto: {len(df_auto)} 行 (截取到input, 含create_labels砍尾)")
|
||||
|
||||
# 2. 最后一行时间对比
|
||||
print(f"\n[2] 预处理后df最后一行")
|
||||
print(f" test最后一行: {df_test.iloc[-1]['timestamp']} close={df_test.iloc[-1]['close']:.2f}")
|
||||
print(f" auto最后一行: {df_auto.iloc[-1]['timestamp']} close={df_auto.iloc[-1]['close']:.2f}")
|
||||
|
||||
# 3. 输入末根K线的原始特征对比
|
||||
if test_df_idx is not None:
|
||||
test_features = df_test.loc[test_df_idx, feature_cols]
|
||||
auto_last_row = df_auto.iloc[-1]
|
||||
auto_features = auto_last_row[feature_cols]
|
||||
|
||||
print(f"\n[3] 输入末根K线原始特征值")
|
||||
print(f" {'特征':<15} {'test值':>15} {'auto值':>15} {'差异':>15}")
|
||||
print(f" {'-'*60}")
|
||||
for col in feature_cols:
|
||||
tv = test_features[col]
|
||||
av = auto_features[col]
|
||||
diff = tv - av
|
||||
marker = " ⚠" if abs(diff) > 1e-6 else ""
|
||||
print(f" {col:<15} {tv:>15.6f} {av:>15.6f} {diff:>15.8f}{marker}")
|
||||
|
||||
# 4. scaler transform 后的值对比(输入末根K线)
|
||||
if test_df_idx is not None and X_test_window is not None and X_auto_window is not None:
|
||||
test_scaled_last = scaled_test[test_df_idx]
|
||||
auto_scaled_last = scaled_auto[-1]
|
||||
|
||||
print(f"\n[4] scaler transform 后的值(输入末根K线)")
|
||||
print(f" {'特征':<15} {'test值':>15} {'auto值':>15} {'差异':>15}")
|
||||
print(f" {'-'*60}")
|
||||
for j, col in enumerate(feature_cols):
|
||||
tv = test_scaled_last[j]
|
||||
av = auto_scaled_last[j]
|
||||
diff = tv - av
|
||||
marker = " ⚠" if abs(diff) > 1e-6 else ""
|
||||
print(f" {col:<15} {tv:>15.6f} {av:>15.6f} {diff:>15.8f}{marker}")
|
||||
|
||||
# 5. 输入窗口对比(整个168步窗口)
|
||||
if X_test_window is not None and X_auto_window is not None:
|
||||
window_diff = X_test_window - X_auto_window
|
||||
max_diff = np.max(np.abs(window_diff))
|
||||
mean_diff = np.mean(np.abs(window_diff))
|
||||
nonzero_count = np.count_nonzero(np.abs(window_diff) > 1e-6)
|
||||
|
||||
print(f"\n[5] 输入窗口对比 (168步 × 9特征 = 1512个值)")
|
||||
print(f" 最大绝对差异: {max_diff:.10f}")
|
||||
print(f" 平均绝对差异: {mean_diff:.10f}")
|
||||
print(f" 差异>1e-6的值: {nonzero_count} / 1512")
|
||||
|
||||
if nonzero_count > 0:
|
||||
# 找到差异最大的位置
|
||||
max_pos = np.unravel_index(np.argmax(np.abs(window_diff)), window_diff.shape)
|
||||
print(f" 最大差异位置: 步={max_pos[0]} 特征={feature_cols[max_pos[1]]}")
|
||||
print(f" test值: {X_test_window[max_pos]:.10f}")
|
||||
print(f" auto值: {X_auto_window[max_pos]:.10f}")
|
||||
|
||||
# 6. 模型推理对比
|
||||
if X_test_window is not None and X_auto_window is not None:
|
||||
with torch.no_grad():
|
||||
test_tensor = torch.FloatTensor(X_test_window[np.newaxis]).to(device)
|
||||
auto_tensor = torch.FloatTensor(X_auto_window[np.newaxis]).to(device)
|
||||
|
||||
test_logits = model(test_tensor).cpu().numpy().item()
|
||||
auto_logits = model(auto_tensor).cpu().numpy().item()
|
||||
|
||||
test_prob = 1 / (1 + np.exp(-test_logits))
|
||||
auto_prob = 1 / (1 + np.exp(-auto_logits))
|
||||
|
||||
test_pred = "涨" if test_prob > 0.5 else "跌"
|
||||
auto_pred = "涨" if auto_prob > 0.5 else "跌"
|
||||
|
||||
match = "✓ 一致" if test_pred == auto_pred else "✗ 不一致"
|
||||
|
||||
print(f"\n[6] 模型推理结果")
|
||||
print(f" {'':15} {'test':>15} {'auto':>15} {'差异':>15}")
|
||||
print(f" {'-'*60}")
|
||||
print(f" {'logits':<15} {test_logits:>15.8f} {auto_logits:>15.8f} {test_logits-auto_logits:>15.8f}")
|
||||
print(f" {'概率':<15} {test_prob:>15.8f} {auto_prob:>15.8f} {test_prob-auto_prob:>15.8f}")
|
||||
print(f" {'预测':<15} {test_pred:>15} {auto_pred:>15} {match:>15}")
|
||||
|
||||
# ===== 全量对比:2月所有小时 =====
|
||||
print(f"\n\n{'=' * 100}")
|
||||
print("全量对比:2月份所有小时预测结果差异统计")
|
||||
print("=" * 100)
|
||||
|
||||
# test方式:一次性预处理
|
||||
test_prep_full = TestPreprocessor(lookback=168)
|
||||
test_prep_full.scaler = trained_scaler
|
||||
X_test_full, df_test_full, idx_test_full, _ = test_prep_full.preprocess(df_full, fit=False)
|
||||
|
||||
# 构建 timestamp -> test窗口索引 的映射
|
||||
test_ts_map = {}
|
||||
for i, idx in enumerate(idx_test_full):
|
||||
ts = df_test_full.iloc[idx]['timestamp']
|
||||
test_ts_map[ts] = i
|
||||
|
||||
total = 0
|
||||
same = 0
|
||||
diff_list = []
|
||||
|
||||
for target_idx in range(1, len(df_full)):
|
||||
target_time_utc = df_full.iloc[target_idx]['timestamp']
|
||||
target_time_bj = utc_to_beijing(target_time_utc)
|
||||
if not (target_time_bj.year == 2026 and target_time_bj.month == 2):
|
||||
continue
|
||||
|
||||
input_idx = target_idx - 1
|
||||
input_ts = df_full.iloc[input_idx]['timestamp']
|
||||
|
||||
# test方式
|
||||
test_i = test_ts_map.get(input_ts)
|
||||
if test_i is None or test_i >= len(X_test_full):
|
||||
continue
|
||||
|
||||
# auto方式
|
||||
df_up_to_t = df_full.iloc[:input_idx + 1]
|
||||
auto_prep_tmp = AutoTradePreprocessor(lookback=168)
|
||||
auto_prep_tmp.scaler = trained_scaler
|
||||
X_auto_tmp, _, _ = auto_prep_tmp.preprocess(df_up_to_t, fit=False)
|
||||
if len(X_auto_tmp) == 0:
|
||||
continue
|
||||
|
||||
with torch.no_grad():
|
||||
test_logits = model(torch.FloatTensor(X_test_full[test_i:test_i+1]).to(device)).cpu().numpy().item()
|
||||
auto_logits = model(torch.FloatTensor(X_auto_tmp[-1:]).to(device)).cpu().numpy().item()
|
||||
|
||||
test_prob = 1 / (1 + np.exp(-test_logits))
|
||||
auto_prob = 1 / (1 + np.exp(-auto_logits))
|
||||
test_pred = 1 if test_prob > 0.5 else 0
|
||||
auto_pred = 1 if auto_prob > 0.5 else 0
|
||||
|
||||
total += 1
|
||||
if test_pred == auto_pred:
|
||||
same += 1
|
||||
else:
|
||||
diff_list.append({
|
||||
'time_bj': target_time_bj.strftime('%m-%d %H:%M'),
|
||||
'test_prob': test_prob,
|
||||
'auto_prob': auto_prob,
|
||||
'test_pred': '涨' if test_pred == 1 else '跌',
|
||||
'auto_pred': '涨' if auto_pred == 1 else '跌',
|
||||
})
|
||||
|
||||
print(f"总预测数: {total}")
|
||||
print(f"结果一致: {same} ({same/total:.1%})")
|
||||
print(f"结果不同: {len(diff_list)} ({len(diff_list)/total:.1%})")
|
||||
|
||||
if diff_list:
|
||||
print(f"\n不一致的时间点:")
|
||||
print(f" {'北京时间':<15} {'test概率':>10} {'test预测':>8} {'auto概率':>10} {'auto预测':>8}")
|
||||
print(f" {'-'*55}")
|
||||
for d in diff_list:
|
||||
print(f" {d['time_bj']:<15} {d['test_prob']:>10.6f} {d['test_pred']:>8} {d['auto_prob']:>10.6f} {d['auto_pred']:>8}")
|
||||
|
||||
print("=" * 100)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,460 +0,0 @@
|
||||
"""按天统计双模型和三模型的准确率"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import pickle
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df = df[:-self.forecast_horizon].reset_index(drop=True)
|
||||
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
future_prices = df['future_price'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
y.append(scaled_future[i + self.lookback, 0])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
def inverse_transform_price(self, scaled_price):
|
||||
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
|
||||
|
||||
|
||||
def main():
|
||||
print("="*80)
|
||||
print("按天统计:双模型交集 vs 三模型投票 每日准确率分析")
|
||||
print("="*80)
|
||||
|
||||
# ========== 加载模型 ==========
|
||||
print("\n[1/4] 加载模型...")
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
|
||||
# 加载DLinear
|
||||
from models.dlinear.model import DLinear
|
||||
dlinear_checkpoint = torch.load(
|
||||
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
dlinear_model = DLinear(dlinear_checkpoint['config'])
|
||||
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
|
||||
dlinear_model = dlinear_model.to(device)
|
||||
dlinear_model.eval()
|
||||
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
|
||||
# 加载N-BEATS
|
||||
from models.nbeats.model import NBeats
|
||||
nbeats_checkpoint = torch.load(
|
||||
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
nbeats_model = NBeats(nbeats_checkpoint['config'])
|
||||
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
|
||||
nbeats_model = nbeats_model.to(device)
|
||||
nbeats_model.eval()
|
||||
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
|
||||
# 加载TTM-R2
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
|
||||
"ibm-granite/granite-timeseries-ttm-r2",
|
||||
trust_remote_code=True
|
||||
)
|
||||
ttm_model = ttm_model.to(device)
|
||||
ttm_model.eval()
|
||||
|
||||
print(f"✓ 三个模型已加载")
|
||||
|
||||
# ========== 获取数据 ==========
|
||||
print("\n[2/4] 获取2026年测试数据...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
|
||||
try:
|
||||
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-26',
|
||||
end_date='2026-02-02', use_cache=False)
|
||||
except:
|
||||
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', use_cache=False)
|
||||
|
||||
print(f"✓ 测试数据: {len(df_2026)} 行")
|
||||
|
||||
# ========== 预处理 ==========
|
||||
print("\n[3/4] 预处理数据...")
|
||||
|
||||
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
|
||||
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
|
||||
|
||||
min_samples = min(len(X_dlinear), len(X_nbeats))
|
||||
X_dlinear = X_dlinear[:min_samples]
|
||||
y_dlinear = y_dlinear[:min_samples]
|
||||
X_nbeats = X_nbeats[:min_samples]
|
||||
y_nbeats = y_nbeats[:min_samples]
|
||||
|
||||
# ========== 预测 ==========
|
||||
print("\n[4/4] 执行预测...")
|
||||
|
||||
# DLinear预测
|
||||
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
|
||||
with torch.no_grad():
|
||||
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
|
||||
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits))
|
||||
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten()
|
||||
|
||||
# N-BEATS预测
|
||||
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
|
||||
with torch.no_grad():
|
||||
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
|
||||
|
||||
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
|
||||
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
|
||||
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
|
||||
|
||||
# TTM-R2预测
|
||||
context_length = 512
|
||||
ttm_preds_list = []
|
||||
all_prices = df_2026['close'].values
|
||||
all_open_prices = df_2026['open'].values
|
||||
|
||||
for i in range(min_samples):
|
||||
start_idx = i
|
||||
end_idx = start_idx + context_length
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
context_data = all_prices[-context_length:].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
context_tensor = torch.FloatTensor(context_data).to(device)
|
||||
|
||||
with torch.no_grad():
|
||||
output = ttm_model(context_tensor)
|
||||
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
|
||||
|
||||
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
|
||||
ttm_preds = np.array(ttm_preds_list)
|
||||
|
||||
print(f"✓ 预测完成")
|
||||
|
||||
# ========== 构建结果DataFrame ==========
|
||||
# 获取对应的时间戳(第i个样本对应df_2026的第168+i根K线)
|
||||
timestamps = df_2026['timestamp'].iloc[168:168+min_samples].values
|
||||
|
||||
# 转换为美国东部时间 (ET)
|
||||
timestamps_utc = pd.to_datetime(timestamps)
|
||||
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
|
||||
dates = timestamps_et.date
|
||||
|
||||
# 双模型交集
|
||||
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
|
||||
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
|
||||
two_model_preds = np.full(len(dlinear_preds), -1)
|
||||
two_model_preds[two_model_agree_up] = 1
|
||||
two_model_preds[two_model_agree_down] = 0
|
||||
|
||||
# 三模型多数投票
|
||||
vote_sum = dlinear_preds + nbeats_preds + ttm_preds
|
||||
three_model_preds = np.where(vote_sum >= 2, 1, 0)
|
||||
|
||||
# 真实标签
|
||||
true_labels = y_dlinear
|
||||
|
||||
# 创建DataFrame
|
||||
results_df = pd.DataFrame({
|
||||
'date': dates,
|
||||
'timestamp': timestamps,
|
||||
'timestamp_et': timestamps_et,
|
||||
'true_label': true_labels,
|
||||
'dlinear_pred': dlinear_preds,
|
||||
'nbeats_pred': nbeats_preds,
|
||||
'ttm_pred': ttm_preds,
|
||||
'two_model_pred': two_model_preds,
|
||||
'three_model_pred': three_model_preds,
|
||||
'two_model_correct': (two_model_preds == true_labels) & (two_model_preds != -1),
|
||||
'three_model_correct': (three_model_preds == true_labels)
|
||||
})
|
||||
|
||||
# 添加周几信息(美国东部时间)
|
||||
results_df['day_of_week'] = pd.to_datetime(results_df['timestamp_et']).dt.day_name()
|
||||
results_df['is_weekend'] = pd.to_datetime(results_df['timestamp_et']).dt.dayofweek >= 5 # 5=Saturday, 6=Sunday
|
||||
|
||||
# 2026年1月美国重要经济数据发布日期(美国东部时间)
|
||||
# 数据来源:美联储、劳工部等官方日历
|
||||
us_economic_events = {
|
||||
'2026-01-08': ['初请失业金人数'],
|
||||
'2026-01-09': ['非农就业报告 (NFP)', '失业率'],
|
||||
'2026-01-13': ['CPI通胀数据'],
|
||||
'2026-01-14': ['PPI生产者物价指数'],
|
||||
'2026-01-15': ['零售销售数据'],
|
||||
'2026-01-16': ['密歇根大学消费者信心指数'],
|
||||
'2026-01-22': ['初请失业金人数'],
|
||||
'2026-01-27': ['耐用品订单'],
|
||||
'2026-01-28': ['FOMC利率决议', 'GDP初值'],
|
||||
'2026-01-29': ['PCE物价指数', '个人收入与支出'],
|
||||
}
|
||||
|
||||
# ========== 按天统计 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 每日准确率统计 (美国东部时间)")
|
||||
print("="*80)
|
||||
|
||||
daily_stats = results_df.groupby('date').agg({
|
||||
'true_label': 'count', # 总样本数
|
||||
'two_model_pred': lambda x: (x != -1).sum(), # 双模型有效预测数
|
||||
'two_model_correct': 'sum', # 双模型正确数
|
||||
'three_model_correct': 'sum', # 三模型正确数
|
||||
'is_weekend': 'first', # 是否周末
|
||||
'day_of_week': 'first' # 星期几
|
||||
})
|
||||
|
||||
daily_stats.columns = ['total_samples', 'two_model_valid', 'two_model_correct',
|
||||
'three_model_correct', 'is_weekend', 'day_of_week']
|
||||
|
||||
# 计算准确率
|
||||
daily_stats['two_model_accuracy'] = daily_stats['two_model_correct'] / daily_stats['two_model_valid'] * 100
|
||||
daily_stats['three_model_accuracy'] = daily_stats['three_model_correct'] / daily_stats['total_samples'] * 100
|
||||
daily_stats['two_model_coverage'] = daily_stats['two_model_valid'] / daily_stats['total_samples'] * 100
|
||||
|
||||
# 填充NaN(当天没有有效预测)
|
||||
daily_stats['two_model_accuracy'] = daily_stats['two_model_accuracy'].fillna(0)
|
||||
|
||||
# 添加美国经济数据发布标记
|
||||
daily_stats['us_events'] = daily_stats.index.map(
|
||||
lambda x: us_economic_events.get(str(x), [])
|
||||
)
|
||||
daily_stats['has_us_data'] = daily_stats['us_events'].apply(lambda x: len(x) > 0)
|
||||
|
||||
print(f"\n每日详细统计:")
|
||||
print(f"{'日期':<12} {'星期':<10} {'周末':<4} {'美国数据':<4} {'样本':>4} "
|
||||
f"{'双模型%':>8} {'覆盖%':>6} {'三模型%':>8} {'差异':>7}")
|
||||
print("-" * 95)
|
||||
|
||||
for date, row in daily_stats.iterrows():
|
||||
diff = row['two_model_accuracy'] - row['three_model_accuracy']
|
||||
diff_str = f"{diff:+.1f}" if row['two_model_valid'] > 0 else "N/A"
|
||||
|
||||
weekend_mark = "✓" if row['is_weekend'] else ""
|
||||
us_data_mark = "📊" if row['has_us_data'] else ""
|
||||
|
||||
# 根据准确率设置颜色标记
|
||||
two_acc = row['two_model_accuracy']
|
||||
three_acc = row['three_model_accuracy']
|
||||
|
||||
two_mark = "🔥" if two_acc >= 70 else ("✅" if two_acc >= 60 else ("⚠️" if two_acc < 40 else ""))
|
||||
three_mark = "🔥" if three_acc >= 70 else ("✅" if three_acc >= 60 else ("⚠️" if three_acc < 40 else ""))
|
||||
|
||||
print(f"{date} {row['day_of_week'][:3]:>8} {weekend_mark:>4} {us_data_mark:>4} "
|
||||
f"{row['total_samples']:>4.0f} "
|
||||
f"{row['two_model_accuracy']:>7.1f}{two_mark} "
|
||||
f"{row['two_model_coverage']:>5.1f} "
|
||||
f"{row['three_model_accuracy']:>7.1f}{three_mark} "
|
||||
f"{diff_str:>7}")
|
||||
|
||||
# 如果有美国经济数据发布,显示详情
|
||||
if row['has_us_data']:
|
||||
events_str = ", ".join(row['us_events'])
|
||||
print(f" └─ 📊 {events_str}")
|
||||
|
||||
# ========== 周末 vs 工作日分析 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📅 周末 vs 工作日准确率对比")
|
||||
print("="*80)
|
||||
|
||||
weekend_stats = daily_stats[daily_stats['is_weekend']]
|
||||
weekday_stats = daily_stats[~daily_stats['is_weekend']]
|
||||
|
||||
if len(weekend_stats) > 0:
|
||||
print(f"\n🌴 周末 ({len(weekend_stats)}天):")
|
||||
print(f" 双模型平均准确率: {weekend_stats['two_model_accuracy'].mean():.2f}%")
|
||||
print(f" 三模型平均准确率: {weekend_stats['three_model_accuracy'].mean():.2f}%")
|
||||
print(f" 平均覆盖率: {weekend_stats['two_model_coverage'].mean():.2f}%")
|
||||
|
||||
if len(weekday_stats) > 0:
|
||||
print(f"\n💼 工作日 ({len(weekday_stats)}天):")
|
||||
print(f" 双模型平均准确率: {weekday_stats['two_model_accuracy'].mean():.2f}%")
|
||||
print(f" 三模型平均准确率: {weekday_stats['three_model_accuracy'].mean():.2f}%")
|
||||
print(f" 平均覆盖率: {weekday_stats['two_model_coverage'].mean():.2f}%")
|
||||
|
||||
# ========== 美国数据发布日 vs 普通日分析 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 美国经济数据发布日 vs 普通日准确率对比")
|
||||
print("="*80)
|
||||
|
||||
us_data_days = daily_stats[daily_stats['has_us_data']]
|
||||
normal_days = daily_stats[~daily_stats['has_us_data']]
|
||||
|
||||
if len(us_data_days) > 0:
|
||||
print(f"\n📊 美国数据发布日 ({len(us_data_days)}天):")
|
||||
print(f" 双模型平均准确率: {us_data_days['two_model_accuracy'].mean():.2f}%")
|
||||
print(f" 三模型平均准确率: {us_data_days['three_model_accuracy'].mean():.2f}%")
|
||||
print(f" 平均覆盖率: {us_data_days['two_model_coverage'].mean():.2f}%")
|
||||
print(f"\n 具体日期:")
|
||||
for date in us_data_days.index:
|
||||
events = ", ".join(us_data_days.loc[date, 'us_events'])
|
||||
two_acc = us_data_days.loc[date, 'two_model_accuracy']
|
||||
three_acc = us_data_days.loc[date, 'three_model_accuracy']
|
||||
print(f" {date}: 双{two_acc:.1f}% / 三{three_acc:.1f}% - {events}")
|
||||
|
||||
if len(normal_days) > 0:
|
||||
print(f"\n📅 普通日 ({len(normal_days)}天):")
|
||||
print(f" 双模型平均准确率: {normal_days['two_model_accuracy'].mean():.2f}%")
|
||||
print(f" 三模型平均准确率: {normal_days['three_model_accuracy'].mean():.2f}%")
|
||||
print(f" 平均覆盖率: {normal_days['two_model_coverage'].mean():.2f}%")
|
||||
|
||||
# ========== 分类统计 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📈 准确率分类统计")
|
||||
print("="*80)
|
||||
|
||||
# 双模型交集分类
|
||||
two_model_high = daily_stats[daily_stats['two_model_accuracy'] >= 60]
|
||||
two_model_medium = daily_stats[(daily_stats['two_model_accuracy'] >= 45) &
|
||||
(daily_stats['two_model_accuracy'] < 60)]
|
||||
two_model_low = daily_stats[(daily_stats['two_model_accuracy'] > 0) &
|
||||
(daily_stats['two_model_accuracy'] < 45)]
|
||||
|
||||
# 三模型投票分类
|
||||
three_model_high = daily_stats[daily_stats['three_model_accuracy'] >= 60]
|
||||
three_model_medium = daily_stats[(daily_stats['three_model_accuracy'] >= 45) &
|
||||
(daily_stats['three_model_accuracy'] < 60)]
|
||||
three_model_low = daily_stats[daily_stats['three_model_accuracy'] < 45]
|
||||
|
||||
print(f"\n🔵 双模型交集:")
|
||||
print(f" 高准确率 (>=60%): {len(two_model_high)}天")
|
||||
if len(two_model_high) > 0:
|
||||
print(f" 日期: {', '.join([str(d) for d in two_model_high.index])}")
|
||||
print(f" 平均准确率: {two_model_high['two_model_accuracy'].mean():.1f}%")
|
||||
|
||||
print(f"\n 中等准确率 (45-60%): {len(two_model_medium)}天")
|
||||
if len(two_model_medium) > 0:
|
||||
print(f" 日期: {', '.join([str(d) for d in two_model_medium.index])}")
|
||||
print(f" 平均准确率: {two_model_medium['two_model_accuracy'].mean():.1f}%")
|
||||
|
||||
print(f"\n 低准确率 (<45%): {len(two_model_low)}天")
|
||||
if len(two_model_low) > 0:
|
||||
print(f" 日期: {', '.join([str(d) for d in two_model_low.index])}")
|
||||
print(f" 平均准确率: {two_model_low['two_model_accuracy'].mean():.1f}%")
|
||||
|
||||
print(f"\n🟢 三模型投票:")
|
||||
print(f" 高准确率 (>=60%): {len(three_model_high)}天")
|
||||
if len(three_model_high) > 0:
|
||||
print(f" 日期: {', '.join([str(d) for d in three_model_high.index])}")
|
||||
print(f" 平均准确率: {three_model_high['three_model_accuracy'].mean():.1f}%")
|
||||
|
||||
print(f"\n 中等准确率 (45-60%): {len(three_model_medium)}天")
|
||||
if len(three_model_medium) > 0:
|
||||
print(f" 日期: {', '.join([str(d) for d in three_model_medium.index])}")
|
||||
print(f" 平均准确率: {three_model_medium['three_model_accuracy'].mean():.1f}%")
|
||||
|
||||
print(f"\n 低准确率 (<45%): {len(three_model_low)}天")
|
||||
if len(three_model_low) > 0:
|
||||
print(f" 日期: {', '.join([str(d) for d in three_model_low.index])}")
|
||||
print(f" 平均准确率: {three_model_low['three_model_accuracy'].mean():.1f}%")
|
||||
|
||||
# ========== 总结 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 总体统计")
|
||||
print("="*80)
|
||||
|
||||
print(f"\n双模型交集:")
|
||||
print(f" 总体准确率: {(daily_stats['two_model_correct'].sum() / daily_stats['two_model_valid'].sum() * 100):.2f}%")
|
||||
print(f" 总体覆盖率: {(daily_stats['two_model_valid'].sum() / daily_stats['total_samples'].sum() * 100):.2f}%")
|
||||
print(f" 最高日准确率: {daily_stats['two_model_accuracy'].max():.1f}% ({daily_stats['two_model_accuracy'].idxmax()})")
|
||||
print(f" 最低日准确率: {daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].min():.1f}% ({daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].idxmin()})")
|
||||
|
||||
print(f"\n三模型投票:")
|
||||
print(f" 总体准确率: {(daily_stats['three_model_correct'].sum() / daily_stats['total_samples'].sum() * 100):.2f}%")
|
||||
print(f" 覆盖率: 100%")
|
||||
print(f" 最高日准确率: {daily_stats['three_model_accuracy'].max():.1f}% ({daily_stats['three_model_accuracy'].idxmax()})")
|
||||
print(f" 最低日准确率: {daily_stats['three_model_accuracy'].min():.1f}% ({daily_stats['three_model_accuracy'].idxmin()})")
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✓ 每日准确率统计完成!")
|
||||
print("="*80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,379 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
每日数据报告定时推送程序
|
||||
- 每天北京时间 8 点推送昨天的 24 小时预测统计
|
||||
- 包含各模型准确率、逐小时预测结果详情
|
||||
"""
|
||||
|
||||
import schedule
|
||||
import time
|
||||
import logging
|
||||
from datetime import datetime, timedelta
|
||||
from zoneinfo import ZoneInfo
|
||||
from typing import Dict, List
|
||||
|
||||
from utils.send_msg import lark_notifier
|
||||
from api.models import PredictionModel, convert_to_cst, get_db_connection, TZ_CST, TZ_UTC
|
||||
|
||||
# 配置日志
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
|
||||
handlers=[
|
||||
logging.FileHandler('logs/daily_report.log'),
|
||||
logging.StreamHandler()
|
||||
]
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class DailyReportSender:
|
||||
"""每日数据报告发送器"""
|
||||
|
||||
def __init__(self):
|
||||
self.model = PredictionModel()
|
||||
self.lark = lark_notifier
|
||||
|
||||
def get_yesterday_failed_trades(self) -> List[Dict]:
|
||||
"""
|
||||
获取昨天的失败交易订单
|
||||
|
||||
Returns:
|
||||
[
|
||||
{
|
||||
'timestamp': '2026-02-27 10:30:00',
|
||||
'market_title': 'BTC 价格',
|
||||
'action': 'BUY',
|
||||
'status': 'failed',
|
||||
'error_message': '余额不足'
|
||||
},
|
||||
...
|
||||
]
|
||||
"""
|
||||
try:
|
||||
# 获取昨天的日期(CST)
|
||||
now_cst = datetime.now(TZ_CST)
|
||||
yesterday_cst = now_cst - timedelta(days=1)
|
||||
yesterday_date = yesterday_cst.strftime('%Y-%m-%d')
|
||||
|
||||
# 计算 CST 时间范围
|
||||
start_cst = datetime.strptime(f"{yesterday_date} 00:00:00", '%Y-%m-%d %H:%M:%S').replace(tzinfo=TZ_CST)
|
||||
end_cst = start_cst + timedelta(days=1)
|
||||
|
||||
start_str = start_cst.strftime('%Y-%m-%d %H:%M:%S')
|
||||
end_str = end_cst.strftime('%Y-%m-%d %H:%M:%S')
|
||||
|
||||
# 从 trades 表查询失败订单
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
sql = """
|
||||
SELECT
|
||||
timestamp,
|
||||
market_title,
|
||||
action,
|
||||
status,
|
||||
error_message
|
||||
FROM trades
|
||||
WHERE timestamp >= %s AND timestamp < %s
|
||||
AND status != 'MATCHED'
|
||||
ORDER BY timestamp DESC
|
||||
"""
|
||||
cursor.execute(sql, (start_str, end_str))
|
||||
rows = cursor.fetchall()
|
||||
|
||||
return rows if rows else []
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"获取昨天失败交易失败: {e}", exc_info=True)
|
||||
return []
|
||||
|
||||
def get_yesterday_statistics(self) -> Dict:
|
||||
"""
|
||||
获取昨天的 24 小时预测统计和详细数据
|
||||
|
||||
Returns:
|
||||
{
|
||||
'date': 'YYYY-MM-DD',
|
||||
'models': {
|
||||
'DLinear': {'total': 24, 'correct': 14, 'accuracy': 58.3},
|
||||
'N-BEATS': {...},
|
||||
'TTM-R2': {...},
|
||||
'Ensemble': {...}
|
||||
},
|
||||
'hourly_details': [
|
||||
{
|
||||
'hour': '00:00-01:00',
|
||||
'actual': '涨',
|
||||
'dlinear': {'pred': '涨', 'correct': True},
|
||||
'nbeats': {...},
|
||||
'ttm': {...},
|
||||
'ensemble': {...}
|
||||
},
|
||||
...
|
||||
]
|
||||
}
|
||||
"""
|
||||
try:
|
||||
# 获取昨天的日期(CST)
|
||||
now_cst = datetime.now(TZ_CST)
|
||||
yesterday_cst = now_cst - timedelta(days=1)
|
||||
yesterday_date = yesterday_cst.strftime('%Y-%m-%d')
|
||||
|
||||
# 计算 UTC 时间范围
|
||||
start_cst = datetime.strptime(f"{yesterday_date} 00:00:00", '%Y-%m-%d %H:%M:%S').replace(tzinfo=TZ_CST)
|
||||
end_cst = start_cst + timedelta(days=1)
|
||||
|
||||
start_utc = start_cst.astimezone(TZ_UTC)
|
||||
end_utc = end_cst.astimezone(TZ_UTC)
|
||||
|
||||
start_str = start_utc.strftime('%Y-%m-%d %H:%M:%S')
|
||||
end_str = end_utc.strftime('%Y-%m-%d %H:%M:%S')
|
||||
|
||||
# 查询数据库
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
sql = """
|
||||
SELECT
|
||||
timestamp,
|
||||
actual_result,
|
||||
dlinear_prediction, dlinear_correct,
|
||||
nbeats_prediction, nbeats_correct,
|
||||
ttm_prediction, ttm_correct,
|
||||
ensemble_prediction, ensemble_correct
|
||||
FROM hourly_model_predictions
|
||||
WHERE timestamp >= %s AND timestamp < %s
|
||||
ORDER BY timestamp ASC
|
||||
"""
|
||||
cursor.execute(sql, (start_str, end_str))
|
||||
rows = cursor.fetchall()
|
||||
|
||||
# 整理统计数据
|
||||
models = ['dlinear', 'nbeats', 'ttm', 'ensemble']
|
||||
model_names = {
|
||||
'dlinear': 'DLinear',
|
||||
'nbeats': 'N-BEATS',
|
||||
'ttm': 'TTM-R2',
|
||||
'ensemble': 'Ensemble'
|
||||
}
|
||||
|
||||
# 初始化统计
|
||||
stats = {
|
||||
'date': yesterday_date,
|
||||
'models': {},
|
||||
'hourly_details': []
|
||||
}
|
||||
|
||||
for model in models:
|
||||
stats['models'][model_names[model]] = {
|
||||
'total': 0,
|
||||
'correct': 0,
|
||||
'accuracy': 0.0
|
||||
}
|
||||
|
||||
# 处理每行数据
|
||||
for idx, row in enumerate(rows):
|
||||
if row.get('actual_result') is None:
|
||||
continue
|
||||
|
||||
# 获取时间戳(转为 CST 显示)
|
||||
ts_utc = row['timestamp']
|
||||
if isinstance(ts_utc, str):
|
||||
ts_utc = datetime.fromisoformat(str(ts_utc).replace('Z', '+00:00'))
|
||||
if ts_utc.tzinfo is None:
|
||||
ts_utc = ts_utc.replace(tzinfo=TZ_UTC)
|
||||
|
||||
ts_cst = ts_utc.astimezone(TZ_CST)
|
||||
hour_start = ts_cst.strftime('%H:%M')
|
||||
hour_end = (ts_cst + timedelta(hours=1)).strftime('%H:%M')
|
||||
hour_label = f"{hour_start}-{hour_end}"
|
||||
|
||||
# 实际结果
|
||||
actual_result = row['actual_result']
|
||||
actual_label = '涨' if actual_result == 1 else '跌'
|
||||
|
||||
# 每小时预测详情
|
||||
hourly_item = {
|
||||
'hour': hour_label,
|
||||
'actual': actual_label
|
||||
}
|
||||
|
||||
# 逐模型处理预测结果
|
||||
for model in models:
|
||||
pred = row[f"{model}_prediction"]
|
||||
correct = row[f"{model}_correct"]
|
||||
|
||||
# 更新统计
|
||||
if pred is not None:
|
||||
stats['models'][model_names[model]]['total'] += 1
|
||||
if correct == 1:
|
||||
stats['models'][model_names[model]]['correct'] += 1
|
||||
|
||||
# 添加到小时详情
|
||||
pred_label = '涨' if pred == 1 else ('跌' if pred == 0 else '无')
|
||||
correct_flag = correct == 1 if correct is not None else None
|
||||
|
||||
hourly_item[model] = {
|
||||
'pred': pred_label,
|
||||
'correct': '✓' if correct_flag else ('✗' if correct_flag is False else '-')
|
||||
}
|
||||
|
||||
stats['hourly_details'].append(hourly_item)
|
||||
|
||||
# 计算准确率百分比
|
||||
for model_name in stats['models']:
|
||||
total = stats['models'][model_name]['total']
|
||||
correct = stats['models'][model_name]['correct']
|
||||
if total > 0:
|
||||
stats['models'][model_name]['accuracy'] = round(correct / total * 100, 2)
|
||||
|
||||
return stats
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"获取昨天统计数据失败: {e}", exc_info=True)
|
||||
return None
|
||||
|
||||
def format_report(self, stats: Dict, failed_trades: List[Dict] = None) -> str:
|
||||
"""
|
||||
格式化报告为飞书 Markdown 格式
|
||||
|
||||
Args:
|
||||
stats: get_yesterday_statistics 返回的数据字典
|
||||
failed_trades: 失败订单列表
|
||||
|
||||
Returns:
|
||||
Markdown 格式的报告文本
|
||||
"""
|
||||
if not stats:
|
||||
return "❌ 无法获取昨天的统计数据,请检查数据库连接"
|
||||
|
||||
date = stats['date']
|
||||
models = stats['models']
|
||||
|
||||
# 构建报告头
|
||||
report = f"**【BTC 日报 - {date}】**\n\n"
|
||||
|
||||
# 模型准确率统计部分
|
||||
report += "**📊 模型准确率统计(24小时)**\n"
|
||||
for model_name in ['DLinear', 'N-BEATS', 'TTM-R2', 'Ensemble']:
|
||||
data = models.get(model_name, {})
|
||||
total = data.get('total', 0)
|
||||
correct = data.get('correct', 0)
|
||||
accuracy = data.get('accuracy', 0)
|
||||
|
||||
# 进度条可视化
|
||||
bar_length = 10
|
||||
filled = int(accuracy / 100 * bar_length)
|
||||
bar = '█' * filled + '░' * (bar_length - filled)
|
||||
|
||||
report += f"• **{model_name}**: {correct}/{total} {bar} {accuracy}%\n"
|
||||
|
||||
report += "\n"
|
||||
|
||||
# 交易失败订单详情
|
||||
report += "**⚠️ 交易失败订单**\n"
|
||||
if failed_trades and len(failed_trades) > 0:
|
||||
report += f"昨天共有 {len(failed_trades)} 个失败订单\n"
|
||||
for idx, trade in enumerate(failed_trades[:10], 1): # 仅显示前10个
|
||||
timestamp = trade.get('timestamp', '未知')
|
||||
market_title = trade.get('market_title', '未知市场')
|
||||
action = trade.get('action', '未知')
|
||||
status = trade.get('status', '未知')
|
||||
error_msg = trade.get('error_message', '无详细信息')
|
||||
|
||||
# 简化错误信息 - 仅取第一行
|
||||
if error_msg:
|
||||
error_msg = error_msg.split('\n')[0][:50] # 仅取前50字
|
||||
else:
|
||||
error_msg = '无错误信息'
|
||||
|
||||
report += f"{idx}. [{timestamp}] {market_title} ({action}) → {error_msg}\n"
|
||||
if len(failed_trades) > 10:
|
||||
report += f"... 还有 {len(failed_trades) - 10} 个失败订单\n"
|
||||
else:
|
||||
report += "昨日无失败订单,运营正常 ✓\n"
|
||||
|
||||
report += "\n生成时间: " + datetime.now(TZ_CST).strftime('%Y-%m-%d %H:%M:%S CST')
|
||||
|
||||
return report
|
||||
|
||||
def send_daily_report(self):
|
||||
"""获取昨天统计并推送"""
|
||||
try:
|
||||
logger.info("开始推送每日报告...")
|
||||
|
||||
stats = self.get_yesterday_statistics()
|
||||
if not stats:
|
||||
logger.warning("无法获取昨天统计数据")
|
||||
self.lark.send_alert(
|
||||
title="⚠️ 每日报告生成失败",
|
||||
content="无法获取昨天的统计数据,请检查数据库连接"
|
||||
)
|
||||
return
|
||||
|
||||
# 获取失败订单
|
||||
failed_trades = self.get_yesterday_failed_trades()
|
||||
|
||||
# 格式化报告
|
||||
report = self.format_report(stats, failed_trades)
|
||||
|
||||
# 推送到飞书
|
||||
self.lark.send_alert(
|
||||
title="📊 BTC 每日数据报告",
|
||||
content=report
|
||||
)
|
||||
|
||||
logger.info(f"每日报告已送出: {stats['date']}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"推送每日报告异常: {e}", exc_info=True)
|
||||
self.lark.send_alert(
|
||||
title="❌ 每日报告推送异常",
|
||||
content=f"错误详情: {str(e)}"
|
||||
)
|
||||
|
||||
|
||||
def schedule_daily_report():
|
||||
"""
|
||||
启动每日报告定时任务
|
||||
北京时间每天 8 点执行
|
||||
"""
|
||||
sender = DailyReportSender()
|
||||
sender.send_daily_report() # 启动时立即发送一次(可选)
|
||||
# 使用 schedule 库在北京时间 8 点触发
|
||||
# 由于 schedule 库基于本地时间,当前环境如果是 macOS 默认时间,
|
||||
# 我们使用一个包装器来处理 CST 时间转换
|
||||
def cst_8am_job():
|
||||
"""北京时间 8 点执行的任务"""
|
||||
now_cst = datetime.now(TZ_CST)
|
||||
if now_cst.hour == 8 and now_cst.minute < 1:
|
||||
# 确保只在 08:00-08:01 间隔执行一次(避免重复)
|
||||
sender.send_daily_report()
|
||||
|
||||
# 每分钟检查一次是否需要执行 (确保能在 8 点执行)
|
||||
schedule.every(1).minutes.do(cst_8am_job)
|
||||
|
||||
logger.info("每日报告定时任务已启动(北京时间每天 8 点执行)")
|
||||
|
||||
# 启动调度循环
|
||||
while True:
|
||||
schedule.run_pending()
|
||||
time.sleep(1)
|
||||
|
||||
|
||||
def run_scheduler_background():
|
||||
"""
|
||||
在后台线程中运行调度器(供 start_system.py 调用)
|
||||
"""
|
||||
import threading
|
||||
|
||||
scheduler_thread = threading.Thread(target=schedule_daily_report, daemon=True)
|
||||
scheduler_thread.start()
|
||||
|
||||
logger.info("每日报告调度器已在后台启动")
|
||||
return scheduler_thread
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
# 支持单独运行以及从其他模块导入
|
||||
logger.info("启动每日报告系统...")
|
||||
schedule_daily_report()
|
||||
+3
-1
@@ -26,7 +26,9 @@ class DatabaseManager:
|
||||
password = os.getenv('MYSQL_PASSWORD')
|
||||
database = os.getenv('MYSQL_DB')
|
||||
self._engine = create_engine(
|
||||
f"mysql+pymysql://{user}:{password}@{host}:{port}/{database}?charset=utf8mb4"
|
||||
f"mysql+pymysql://{user}:{password}@{host}:{port}/{database}?charset=utf8mb4",
|
||||
pool_pre_ping=True,
|
||||
pool_recycle=3600
|
||||
)
|
||||
return self._engine
|
||||
|
||||
|
||||
+36
-21
@@ -1,6 +1,7 @@
|
||||
"""从Binance获取加密货币OHLCV数据"""
|
||||
|
||||
import os
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Optional, Literal
|
||||
@@ -11,9 +12,14 @@ import pytz
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from data.database import DatabaseManager
|
||||
from utils.network_proxy import apply_global_proxy_from_env, get_proxy_url
|
||||
from utils.project_logger import get_project_logger
|
||||
from utils.send_msg import lark_notifier
|
||||
|
||||
# 加载环境变量
|
||||
load_dotenv()
|
||||
apply_global_proxy_from_env()
|
||||
logger = get_project_logger("data.fetcher")
|
||||
|
||||
|
||||
class BinanceDataFetcher:
|
||||
@@ -30,17 +36,16 @@ class BinanceDataFetcher:
|
||||
Args:
|
||||
cache_dir: 缓存/数据库目录
|
||||
"""
|
||||
# 配置代理
|
||||
# 统一读取代理配置,兼容全局代理环境变量。
|
||||
proxies = None
|
||||
use_proxy = os.getenv('USE_PROXY', 'false').lower() == 'true'
|
||||
proxy_url = get_proxy_url()
|
||||
|
||||
if use_proxy:
|
||||
socks5_proxy = os.getenv('SOCKS5_PROXY', 'socks5://127.0.0.1:7890')
|
||||
if proxy_url:
|
||||
proxies = {
|
||||
'http': socks5_proxy,
|
||||
'https': socks5_proxy
|
||||
'http': proxy_url,
|
||||
'https': proxy_url
|
||||
}
|
||||
print(f"✓ 使用SOCKS5代理: {socks5_proxy}")
|
||||
logger.info("使用代理: %s", proxy_url)
|
||||
|
||||
try:
|
||||
# 初始化Binance客户端(带代理)
|
||||
@@ -51,13 +56,10 @@ class BinanceDataFetcher:
|
||||
|
||||
# 测试连接
|
||||
self.client.ping()
|
||||
print("✓ 已成功连接到Binance API")
|
||||
logger.info("已成功连接到Binance API")
|
||||
except Exception as e:
|
||||
print(f"⚠ Binance API连接失败: {e}")
|
||||
print("提示:")
|
||||
print(" 1. 检查代理设置 (.env文件)")
|
||||
print(" 2. 确保代理服务正在运行")
|
||||
print(" 3. 或者使用已缓存的数据")
|
||||
logger.warning("Binance API连接失败: %s", e)
|
||||
logger.warning("提示: 检查代理设置、代理服务状态,或使用缓存数据")
|
||||
self.client = None
|
||||
|
||||
# 初始化数据库管理器
|
||||
@@ -69,7 +71,8 @@ class BinanceDataFetcher:
|
||||
timeframe: Literal['1h', '15m'],
|
||||
start_date: Optional[str] = None,
|
||||
end_date: Optional[str] = None,
|
||||
use_cache: bool = True
|
||||
use_cache: bool = True,
|
||||
max_retries: int = 3 # [新增] 最大重试次数参数
|
||||
) -> pd.DataFrame:
|
||||
"""获取加密货币对的OHLCV数据
|
||||
|
||||
@@ -109,13 +112,13 @@ class BinanceDataFetcher:
|
||||
|
||||
# 简单判断覆盖范围 (容忍少量缺失)
|
||||
if min_ts <= start_dt + timedelta(days=2) and max_ts >= end_dt - timedelta(days=2):
|
||||
print(f"从数据库加载数据:{symbol} {timeframe} ({len(df)} rows)")
|
||||
logger.info("从数据库加载数据: %s %s (%s rows)", symbol, timeframe, len(df))
|
||||
return df
|
||||
|
||||
# 2. 从API获取数据
|
||||
if self.client is None:
|
||||
if not df.empty:
|
||||
print(f"⚠ API不可用,返回数据库现有数据 ({len(df)} rows)")
|
||||
logger.warning("API不可用,返回数据库现有数据 (%s rows)", len(df))
|
||||
return df
|
||||
|
||||
raise ConnectionError(
|
||||
@@ -126,9 +129,10 @@ class BinanceDataFetcher:
|
||||
pair = f"{symbol}USDT"
|
||||
interval = self.INTERVALS[timeframe]
|
||||
|
||||
print(f"获取 {symbol} {timeframe} 数据,时间范围:{start_dt.date()} 至 {end_dt.date()}...")
|
||||
logger.info("获取 %s %s 数据,时间范围: %s 至 %s", symbol, timeframe, start_dt.date(), end_dt.date())
|
||||
|
||||
# 获取K线数据
|
||||
klines = []
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
klines = self.client.get_historical_klines(
|
||||
symbol=pair,
|
||||
@@ -136,13 +140,24 @@ class BinanceDataFetcher:
|
||||
start_str=str(int(start_dt.timestamp() * 1000)),
|
||||
end_str=str(int(end_dt.timestamp() * 1000))
|
||||
)
|
||||
# 如果成功获取,跳出循环
|
||||
break
|
||||
except Exception as e:
|
||||
# 如果是最后一次尝试,且依然失败
|
||||
if attempt == max_retries - 1:
|
||||
logger.error("第 %s 次尝试失败,已达到最大重试次数。错误: %s", attempt + 1, e)
|
||||
if not df.empty:
|
||||
print(f"⚠ API请求失败 ({e}),返回数据库现有数据")
|
||||
logger.warning("API请求最终失败,返回数据库现有数据")
|
||||
return df
|
||||
raise e
|
||||
raise e # 如果没有缓存数据,抛出异常
|
||||
|
||||
# 计算等待时间 (指数退避: 2s, 4s, 8s...)
|
||||
wait_time = 2 * (2 ** attempt)
|
||||
logger.warning("请求失败(尝试 %s/%s),%s 秒后重试,错误: %s", attempt + 1, max_retries, wait_time, e)
|
||||
time.sleep(wait_time)
|
||||
|
||||
if not klines:
|
||||
lark_notifier.send_alert(f"数据获取失败: {symbol} {timeframe}", "连续3次请求或解析发生异常,已放弃重试")
|
||||
if not df.empty:
|
||||
return df
|
||||
raise ValueError(f"未获取到 {symbol} {timeframe} 的数据")
|
||||
@@ -164,7 +179,7 @@ class BinanceDataFetcher:
|
||||
# 按时间排序
|
||||
new_df = new_df.sort_values('timestamp').reset_index(drop=True)
|
||||
|
||||
print(f"已获取 {len(new_df)} 行数据")
|
||||
logger.info("已获取 %s 行数据", len(new_df))
|
||||
|
||||
# 保存到数据库
|
||||
self.db_manager.save_ohlcv(new_df, symbol, timeframe)
|
||||
|
||||
+49
-3
@@ -53,12 +53,28 @@ class TradingDatabase:
|
||||
error_message TEXT,
|
||||
raw_data JSON,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
order_id VARCHAR(100),
|
||||
INDEX idx_market_slug (market_slug),
|
||||
INDEX idx_status (status)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
''')
|
||||
conn.commit()
|
||||
|
||||
# 检查并添加 order_id 列 (Migration)
|
||||
self._ensure_column_exists(conn, 'trades', 'order_id', 'VARCHAR(100)')
|
||||
|
||||
def _ensure_column_exists(self, conn, table_name, column_name, column_type):
|
||||
"""确保表中存在指定列"""
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute(f"SHOW COLUMNS FROM {table_name} LIKE '{column_name}'")
|
||||
if not cursor.fetchone():
|
||||
print(f"正在添加列 {column_name} 到表 {table_name}...")
|
||||
cursor.execute(f"ALTER TABLE {table_name} ADD COLUMN {column_name} {column_type}")
|
||||
conn.commit()
|
||||
except Exception as e:
|
||||
print(f"检查/添加列失败: {e}")
|
||||
|
||||
def log_trade(self, trade_data: dict):
|
||||
"""记录交易信息"""
|
||||
try:
|
||||
@@ -72,8 +88,8 @@ class TradingDatabase:
|
||||
INSERT INTO trades (
|
||||
timestamp, market_title, market_slug, model_name,
|
||||
prediction, probability, action, price, amount, size,
|
||||
status, error_message, raw_data
|
||||
) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
status, strategy, error_message, raw_data, order_id
|
||||
) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
'''
|
||||
|
||||
cursor.execute(sql, (
|
||||
@@ -88,8 +104,10 @@ class TradingDatabase:
|
||||
trade_data.get('amount'),
|
||||
trade_data.get('size'),
|
||||
trade_data.get('status'),
|
||||
trade_data.get('strategy'),
|
||||
trade_data.get('error_message'),
|
||||
raw_data_json
|
||||
raw_data_json,
|
||||
trade_data.get('order_id')
|
||||
))
|
||||
conn.commit()
|
||||
except Exception as e:
|
||||
@@ -108,3 +126,31 @@ class TradingDatabase:
|
||||
except Exception as e:
|
||||
print(f"检查已存在交易失败: {e}")
|
||||
return False
|
||||
|
||||
def get_existing_trade(self, market_slug: str) -> dict:
|
||||
"""获取指定市场的最近一次交易记录"""
|
||||
try:
|
||||
with self.get_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
# Get the latest trade for this market
|
||||
sql = 'SELECT * FROM trades WHERE market_slug = %s ORDER BY id DESC LIMIT 1'
|
||||
cursor.execute(sql, (market_slug,))
|
||||
return cursor.fetchone()
|
||||
except Exception as e:
|
||||
print(f"获取交易记录失败: {e}")
|
||||
return None
|
||||
|
||||
def update_trade_status(self, trade_id: int, status: str, error_message: str = None):
|
||||
"""更新交易状态"""
|
||||
try:
|
||||
with self.get_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
if error_message:
|
||||
sql = 'UPDATE trades SET status = %s, error_message = %s WHERE id = %s'
|
||||
cursor.execute(sql, (status, error_message, trade_id))
|
||||
else:
|
||||
sql = 'UPDATE trades SET status = %s WHERE id = %s'
|
||||
cursor.execute(sql, (status, trade_id))
|
||||
conn.commit()
|
||||
except Exception as e:
|
||||
print(f"更新交易状态失败: {e}")
|
||||
@@ -1,406 +0,0 @@
|
||||
"""分析DLinear输出概率值的偏向性及其对集成策略的影响"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import pickle
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df = df[:-self.forecast_horizon].reset_index(drop=True)
|
||||
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
future_prices = df['future_price'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
y.append(scaled_future[i + self.lookback, 0])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
def inverse_transform_price(self, scaled_price):
|
||||
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
|
||||
|
||||
|
||||
def main():
|
||||
print("="*80)
|
||||
print("DLinear输出概率值分布分析 - 以0.1阶梯统计")
|
||||
print("="*80)
|
||||
|
||||
# ========== 加载模型 ==========
|
||||
print("\n[1/4] 加载模型...")
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
|
||||
# 加载DLinear
|
||||
from models.dlinear.model import DLinear
|
||||
dlinear_checkpoint = torch.load(
|
||||
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
dlinear_model = DLinear(dlinear_checkpoint['config'])
|
||||
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
|
||||
dlinear_model = dlinear_model.to(device)
|
||||
dlinear_model.eval()
|
||||
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
|
||||
# 加载N-BEATS
|
||||
from models.nbeats.model import NBeats
|
||||
nbeats_checkpoint = torch.load(
|
||||
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
nbeats_model = NBeats(nbeats_checkpoint['config'])
|
||||
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
|
||||
nbeats_model = nbeats_model.to(device)
|
||||
nbeats_model.eval()
|
||||
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
|
||||
# 加载TTM-R2
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
|
||||
"ibm-granite/granite-timeseries-ttm-r2",
|
||||
trust_remote_code=True
|
||||
)
|
||||
ttm_model = ttm_model.to(device)
|
||||
ttm_model.eval()
|
||||
|
||||
print(f"✓ 三个模型已加载")
|
||||
|
||||
# ========== 获取数据 ==========
|
||||
print("\n[2/4] 获取2026年测试数据...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
|
||||
try:
|
||||
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
|
||||
end_date='2026-12-31', use_cache=False)
|
||||
except:
|
||||
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', use_cache=False)
|
||||
|
||||
print(f"✓ 测试数据: {len(df_2026)} 行")
|
||||
|
||||
# ========== 预处理 ==========
|
||||
print("\n[3/4] 预处理数据...")
|
||||
|
||||
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
|
||||
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
|
||||
|
||||
min_samples = min(len(X_dlinear), len(X_nbeats))
|
||||
X_dlinear = X_dlinear[:min_samples]
|
||||
y_dlinear = y_dlinear[:min_samples]
|
||||
X_nbeats = X_nbeats[:min_samples]
|
||||
y_nbeats = y_nbeats[:min_samples]
|
||||
|
||||
# ========== 预测并获取DLinear概率值 ==========
|
||||
print("\n[4/4] 执行预测并分析DLinear输出概率...")
|
||||
|
||||
# DLinear预测(保留概率值)
|
||||
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
|
||||
with torch.no_grad():
|
||||
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
|
||||
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits.flatten())) # sigmoid概率
|
||||
dlinear_preds = (dlinear_probs > 0.5).astype(int)
|
||||
|
||||
# N-BEATS预测
|
||||
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
|
||||
with torch.no_grad():
|
||||
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
|
||||
|
||||
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
|
||||
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
|
||||
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
|
||||
|
||||
# TTM-R2预测
|
||||
context_length = 512
|
||||
ttm_preds_list = []
|
||||
all_prices = df_2026['close'].values
|
||||
all_open_prices = df_2026['open'].values
|
||||
|
||||
for i in range(min_samples):
|
||||
start_idx = i
|
||||
end_idx = start_idx + context_length
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
context_data = all_prices[-context_length:].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
context_tensor = torch.FloatTensor(context_data).to(device)
|
||||
|
||||
with torch.no_grad():
|
||||
output = ttm_model(context_tensor)
|
||||
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
|
||||
|
||||
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
|
||||
ttm_preds = np.array(ttm_preds_list)
|
||||
|
||||
# 双模型交集
|
||||
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
|
||||
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
|
||||
two_model_valid = two_model_agree_up | two_model_agree_down
|
||||
|
||||
# 三模型多数投票
|
||||
vote_sum = dlinear_preds + nbeats_preds + ttm_preds
|
||||
three_model_preds = np.where(vote_sum >= 2, 1, 0)
|
||||
|
||||
# 真实标签
|
||||
true_labels = y_dlinear
|
||||
|
||||
print(f"✓ 预测完成,开始分析DLinear概率分布...")
|
||||
|
||||
# ========== 按0.1阶梯分析 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 DLinear输出概率分布分析(0.1阶梯)")
|
||||
print("="*80)
|
||||
|
||||
# 定义概率区间
|
||||
prob_bins = np.arange(0, 1.1, 0.1)
|
||||
prob_labels = [f"{i/10:.1f}-{(i+1)/10:.1f}" for i in range(10)]
|
||||
|
||||
# 为每个样本分配到概率区间
|
||||
prob_bin_indices = np.digitize(dlinear_probs, prob_bins) - 1
|
||||
prob_bin_indices = np.clip(prob_bin_indices, 0, 9) # 确保在0-9范围内
|
||||
|
||||
# 初始化统计
|
||||
stats = []
|
||||
|
||||
for i, bin_label in enumerate(prob_labels):
|
||||
mask = prob_bin_indices == i
|
||||
count = mask.sum()
|
||||
|
||||
if count == 0:
|
||||
continue
|
||||
|
||||
# 该区间的真实标签
|
||||
bin_true = true_labels[mask]
|
||||
|
||||
# 该区间DLinear的预测
|
||||
bin_dlinear_pred = dlinear_preds[mask]
|
||||
|
||||
# 该区间在双模型交集中的表现
|
||||
bin_two_model_valid = two_model_valid[mask]
|
||||
bin_two_model_pred = np.full(count, -1)
|
||||
bin_two_model_pred[two_model_agree_up[mask]] = 1
|
||||
bin_two_model_pred[two_model_agree_down[mask]] = 0
|
||||
|
||||
# 该区间在三模型投票中的表现
|
||||
bin_three_model_pred = three_model_preds[mask]
|
||||
|
||||
# 计算统计
|
||||
dlinear_acc = (bin_dlinear_pred == bin_true).mean() * 100 if count > 0 else 0
|
||||
|
||||
two_model_valid_count = bin_two_model_valid.sum()
|
||||
two_model_acc = ((bin_two_model_pred == bin_true) & (bin_two_model_pred != -1)).sum() / two_model_valid_count * 100 if two_model_valid_count > 0 else 0
|
||||
two_model_coverage = two_model_valid_count / count * 100
|
||||
|
||||
three_model_acc = (bin_three_model_pred == bin_true).mean() * 100 if count > 0 else 0
|
||||
|
||||
# 平均概率值
|
||||
avg_prob = dlinear_probs[mask].mean()
|
||||
|
||||
# 预测倾向
|
||||
pred_up = bin_dlinear_pred.mean() * 100
|
||||
|
||||
stats.append({
|
||||
'bin': bin_label,
|
||||
'count': count,
|
||||
'avg_prob': avg_prob,
|
||||
'pred_up_pct': pred_up,
|
||||
'dlinear_acc': dlinear_acc,
|
||||
'two_model_acc': two_model_acc,
|
||||
'two_model_cov': two_model_coverage,
|
||||
'three_model_acc': three_model_acc
|
||||
})
|
||||
|
||||
# 转换为DataFrame
|
||||
stats_df = pd.DataFrame(stats)
|
||||
|
||||
# 显示统计表
|
||||
print("\n概率区间分析:")
|
||||
print(f"{'区间':<10} {'样本数':>6} {'平均概率':>8} {'预测↑%':>7} "
|
||||
f"{'DL准确率':>9} {'双模准确率':>10} {'双模覆盖':>8} {'三模准确率':>10}")
|
||||
print("-" * 95)
|
||||
|
||||
for _, row in stats_df.iterrows():
|
||||
# 根据准确率添加标记
|
||||
dl_mark = "🔥" if row['dlinear_acc'] >= 70 else ("✅" if row['dlinear_acc'] >= 60 else ("⚠️" if row['dlinear_acc'] < 45 else ""))
|
||||
two_mark = "🔥" if row['two_model_acc'] >= 70 else ("✅" if row['two_model_acc'] >= 60 else ("⚠️" if row['two_model_acc'] < 45 else ""))
|
||||
three_mark = "🔥" if row['three_model_acc'] >= 70 else ("✅" if row['three_model_acc'] >= 60 else ("⚠️" if row['three_model_acc'] < 45 else ""))
|
||||
|
||||
print(f"{row['bin']:<10} {row['count']:>6} {row['avg_prob']:>8.3f} {row['pred_up_pct']:>6.1f}% "
|
||||
f"{row['dlinear_acc']:>8.1f}{dl_mark} "
|
||||
f"{row['two_model_acc']:>9.1f}{two_mark} "
|
||||
f"{row['two_model_cov']:>7.1f}% "
|
||||
f"{row['three_model_acc']:>9.1f}{three_mark}")
|
||||
|
||||
# ========== 关键发现 ==========
|
||||
print("\n" + "="*80)
|
||||
print("🔍 关键发现")
|
||||
print("="*80)
|
||||
|
||||
# 极端概率区间(高置信度)
|
||||
high_conf_up = stats_df[stats_df['avg_prob'] >= 0.65]
|
||||
high_conf_down = stats_df[stats_df['avg_prob'] <= 0.35]
|
||||
|
||||
print(f"\n📈 高置信度看涨区间 (概率≥0.65):")
|
||||
if len(high_conf_up) > 0:
|
||||
total_samples = high_conf_up['count'].sum()
|
||||
avg_dl_acc = (high_conf_up['dlinear_acc'] * high_conf_up['count']).sum() / total_samples
|
||||
avg_two_acc = (high_conf_up['two_model_acc'] * high_conf_up['count']).sum() / total_samples
|
||||
avg_three_acc = (high_conf_up['three_model_acc'] * high_conf_up['count']).sum() / total_samples
|
||||
|
||||
print(f" 样本数: {total_samples} ({total_samples/len(dlinear_probs)*100:.1f}%)")
|
||||
print(f" DLinear平均准确率: {avg_dl_acc:.2f}%")
|
||||
print(f" 双模型平均准确率: {avg_two_acc:.2f}%")
|
||||
print(f" 三模型平均准确率: {avg_three_acc:.2f}%")
|
||||
else:
|
||||
print(" 无数据")
|
||||
|
||||
print(f"\n📉 高置信度看跌区间 (概率≤0.35):")
|
||||
if len(high_conf_down) > 0:
|
||||
total_samples = high_conf_down['count'].sum()
|
||||
avg_dl_acc = (high_conf_down['dlinear_acc'] * high_conf_down['count']).sum() / total_samples
|
||||
avg_two_acc = (high_conf_down['two_model_acc'] * high_conf_down['count']).sum() / total_samples
|
||||
avg_three_acc = (high_conf_down['three_model_acc'] * high_conf_down['count']).sum() / total_samples
|
||||
|
||||
print(f" 样本数: {total_samples} ({total_samples/len(dlinear_probs)*100:.1f}%)")
|
||||
print(f" DLinear平均准确率: {avg_dl_acc:.2f}%")
|
||||
print(f" 双模型平均准确率: {avg_two_acc:.2f}%")
|
||||
print(f" 三模型平均准确率: {avg_three_acc:.2f}%")
|
||||
else:
|
||||
print(" 无数据")
|
||||
|
||||
# 中等置信度区间
|
||||
mid_conf = stats_df[(stats_df['avg_prob'] > 0.45) & (stats_df['avg_prob'] < 0.55)]
|
||||
|
||||
print(f"\n⚖️ 中等置信度区间 (0.45-0.55):")
|
||||
if len(mid_conf) > 0:
|
||||
total_samples = mid_conf['count'].sum()
|
||||
avg_dl_acc = (mid_conf['dlinear_acc'] * mid_conf['count']).sum() / total_samples
|
||||
avg_two_acc = (mid_conf['two_model_acc'] * mid_conf['count']).sum() / total_samples if total_samples > 0 else 0
|
||||
avg_three_acc = (mid_conf['three_model_acc'] * mid_conf['count']).sum() / total_samples
|
||||
|
||||
print(f" 样本数: {total_samples} ({total_samples/len(dlinear_probs)*100:.1f}%)")
|
||||
print(f" DLinear平均准确率: {avg_dl_acc:.2f}%")
|
||||
print(f" 双模型平均准确率: {avg_two_acc:.2f}%")
|
||||
print(f" 三模型平均准确率: {avg_three_acc:.2f}%")
|
||||
print(f" ⚠️ 建议: 低置信度区间,考虑弃权或减仓")
|
||||
else:
|
||||
print(" 无数据")
|
||||
|
||||
# 找出表现最好的概率区间
|
||||
best_bin = stats_df.loc[stats_df['dlinear_acc'].idxmax()]
|
||||
worst_bin = stats_df.loc[stats_df['dlinear_acc'].idxmin()]
|
||||
|
||||
print(f"\n🏆 DLinear表现最佳区间:")
|
||||
print(f" 区间: {best_bin['bin']}")
|
||||
print(f" 准确率: {best_bin['dlinear_acc']:.1f}%")
|
||||
print(f" 样本数: {best_bin['count']}")
|
||||
|
||||
print(f"\n💀 DLinear表现最差区间:")
|
||||
print(f" 区间: {worst_bin['bin']}")
|
||||
print(f" 准确率: {worst_bin['dlinear_acc']:.1f}%")
|
||||
print(f" 样本数: {worst_bin['count']}")
|
||||
|
||||
# ========== 概率分布可视化 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 DLinear概率分布直方图")
|
||||
print("="*80)
|
||||
|
||||
print("\n样本数分布:")
|
||||
max_count = stats_df['count'].max()
|
||||
for _, row in stats_df.iterrows():
|
||||
bar_length = int(row['count'] / max_count * 40)
|
||||
bar = "█" * bar_length
|
||||
print(f"{row['bin']:<10} {bar} {row['count']}")
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✓ DLinear概率分布分析完成!")
|
||||
print("="*80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,295 @@
|
||||
# Windows 环境安装指南
|
||||
|
||||
本指南帮助您在 Windows 系统上部署加密货币预测系统。
|
||||
|
||||
---
|
||||
|
||||
## 一、前置要求
|
||||
|
||||
- Windows 10/11 (64位)
|
||||
- 至少 8GB 内存
|
||||
- 至少 10GB 可用磁盘空间
|
||||
- 网络连接(需访问 GitHub、PyPI、Binance API)
|
||||
|
||||
---
|
||||
|
||||
## 二、安装 Miniconda
|
||||
|
||||
### 1. 下载 Miniconda
|
||||
|
||||
访问 [Miniconda 官网](https://docs.conda.io/en/latest/miniconda.html) 下载 Windows 64-bit 安装包,或直接下载:
|
||||
|
||||
```
|
||||
https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe
|
||||
```
|
||||
|
||||
### 2. 安装
|
||||
|
||||
1. 双击运行安装程序
|
||||
2. 选择 "Just Me" (仅为当前用户安装)
|
||||
3. 安装路径建议使用默认值或 `C:\Miniconda3`
|
||||
4. **重要**:勾选 "Add Miniconda3 to my PATH environment variable"
|
||||
5. 完成安装
|
||||
|
||||
### 3. 验证安装
|
||||
|
||||
打开 **命令提示符 (CMD)** 或 **PowerShell**:
|
||||
|
||||
```powershell
|
||||
conda --version
|
||||
# 应显示类似: conda 24.x.x
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、创建 Conda 环境
|
||||
|
||||
### 1. 创建 Python 3.12 环境
|
||||
|
||||
```powershell
|
||||
conda create -n crypto_pred_312 python=3.12 -y
|
||||
```
|
||||
|
||||
### 2. 激活环境
|
||||
|
||||
```powershell
|
||||
conda activate crypto_pred_312
|
||||
```
|
||||
|
||||
激活后,命令行前缀应显示 `(crypto_pred_312)`
|
||||
|
||||
---
|
||||
|
||||
## 四、克隆项目
|
||||
|
||||
### 1. 安装 Git (如未安装)
|
||||
|
||||
下载并安装 [Git for Windows](https://git-scm.com/download/win)
|
||||
|
||||
### 2. 克隆仓库
|
||||
|
||||
```powershell
|
||||
cd C:\Projects # 或您喜欢的目录
|
||||
git clone http://192.168.50.24:3000/subuhui/crpto_predictor.git
|
||||
cd crpto_predictor
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 五、安装依赖
|
||||
|
||||
### 1. 安装 PyTorch (CPU 版本)
|
||||
|
||||
```powershell
|
||||
# CPU 版本 PyTorch(Windows 不使用 GPU)
|
||||
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
|
||||
```
|
||||
|
||||
### 2. 安装项目依赖
|
||||
|
||||
```powershell
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 3. 安装 IBM 时序模型 (TTM-R2)
|
||||
|
||||
```powershell
|
||||
# PyPI 包名是 granite-tsfm,导入时用 tsfm_public
|
||||
pip install granite-tsfm
|
||||
```
|
||||
|
||||
验证安装:
|
||||
```powershell
|
||||
python -c "from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction; print('TTM-R2 安装成功')"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 六、配置环境变量
|
||||
|
||||
### 1. 复制环境变量模板
|
||||
|
||||
```powershell
|
||||
copy .env.example .env
|
||||
```
|
||||
|
||||
### 2. 编辑 .env 文件
|
||||
|
||||
使用记事本或 VSCode 编辑 `.env` 文件:
|
||||
|
||||
```ini
|
||||
# MySQL 数据库配置
|
||||
MYSQL_HOST=your_mysql_host
|
||||
MYSQL_PORT=3306
|
||||
MYSQL_USER=your_username
|
||||
MYSQL_PASSWORD=your_password
|
||||
MYSQL_DB=crypto_predictor
|
||||
|
||||
# Binance API (可选,用于实时数据)
|
||||
BINANCE_API_KEY=your_api_key
|
||||
BINANCE_API_SECRET=your_api_secret
|
||||
|
||||
# Polymarket 配置 (可选,用于自动交易)
|
||||
POLYMARKET_PRIVATE_KEY=your_private_key
|
||||
DRY_RUN=true # 设为 true 进行模拟交易
|
||||
|
||||
# 代理配置 (可选)
|
||||
USE_PROXY=false
|
||||
SOCKS5_PROXY=socks5://127.0.0.1:1080
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 七、数据库配置
|
||||
|
||||
### 1. 安装 MySQL
|
||||
|
||||
下载并安装 [MySQL Community Server](https://dev.mysql.com/downloads/mysql/)
|
||||
|
||||
或使用 Docker:
|
||||
```powershell
|
||||
docker run -d --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=yourpassword mysql:8.0
|
||||
```
|
||||
|
||||
### 2. 创建数据库
|
||||
|
||||
```sql
|
||||
CREATE DATABASE crypto_predictor CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
```
|
||||
|
||||
### 3. 初始化表结构
|
||||
|
||||
首次运行时会自动创建表,或手动执行:
|
||||
```powershell
|
||||
python -c "from api.models import init_model_predictions_table; from data.trading_db import TradingDatabase; TradingDatabase()"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 八、验证安装
|
||||
|
||||
### 1. 测试 PyTorch
|
||||
|
||||
```powershell
|
||||
python -c "import torch; print(f'PyTorch: {torch.__version__}'); print(f'Device: cpu')"
|
||||
```
|
||||
|
||||
### 2. 测试模型加载
|
||||
|
||||
```powershell
|
||||
python -c "from training.config import TrainingConfig; c = TrainingConfig(); print(f'Device: {c.get_device()}')"
|
||||
# 应输出: Device: cpu
|
||||
```
|
||||
|
||||
### 3. 测试数据库连接
|
||||
|
||||
```powershell
|
||||
python -c "from api.models import get_db_connection; conn = get_db_connection(); print('数据库连接成功!')"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 九、运行系统
|
||||
|
||||
### 方式一:双击批处理脚本 (推荐)
|
||||
|
||||
双击项目根目录的 `start_system.bat`
|
||||
|
||||
### 方式二:PowerShell 脚本
|
||||
|
||||
```powershell
|
||||
.\start_system.ps1
|
||||
```
|
||||
|
||||
### 方式三:Python 脚本
|
||||
|
||||
```powershell
|
||||
python start_system.py
|
||||
```
|
||||
|
||||
### 方式四:分别启动
|
||||
|
||||
```powershell
|
||||
# 终端 1: API 服务
|
||||
python -m uvicorn api.main:app --host 0.0.0.0 --port 8000
|
||||
|
||||
# 终端 2: 自动交易
|
||||
python auto_trade_polymarket.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 十、访问系统
|
||||
|
||||
- **前端看板**: http://localhost:8000
|
||||
- **API 文档**: http://localhost:8000/docs
|
||||
|
||||
---
|
||||
|
||||
## 常见问题
|
||||
|
||||
### Q1: `conda activate` 不生效
|
||||
|
||||
在 PowerShell 中需要先初始化 conda:
|
||||
```powershell
|
||||
conda init powershell
|
||||
# 重启 PowerShell
|
||||
```
|
||||
|
||||
### Q2: pip 安装超时
|
||||
|
||||
使用国内镜像:
|
||||
```powershell
|
||||
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
|
||||
```
|
||||
|
||||
### Q3: PyTorch 安装失败
|
||||
|
||||
确保使用 CPU 版本的安装命令,不要安装 CUDA 版本。
|
||||
|
||||
### Q4: 数据库连接失败
|
||||
|
||||
1. 检查 MySQL 服务是否运行
|
||||
2. 检查 `.env` 中的数据库配置
|
||||
3. 检查防火墙是否允许 3306 端口
|
||||
|
||||
### Q5: 模型文件缺失
|
||||
|
||||
确保 `models/checkpoints/` 目录包含预训练模型文件。如缺失,联系管理员获取。
|
||||
|
||||
---
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
crpto_predictor/
|
||||
├── api/ # API 服务
|
||||
│ ├── main.py # FastAPI 入口
|
||||
│ ├── models.py # 数据库模型
|
||||
│ ├── prediction_service.py # 预测服务
|
||||
│ └── static/ # 前端静态文件
|
||||
├── models/ # 模型定义和权重
|
||||
│ ├── checkpoints/ # 模型权重文件
|
||||
│ └── dlinear/ # DLinear 模型
|
||||
├── data/ # 数据层
|
||||
├── training/ # 训练配置
|
||||
├── start_system.py # 跨平台启动脚本
|
||||
├── start_system.bat # Windows CMD 脚本
|
||||
├── start_system.ps1 # Windows PowerShell 脚本
|
||||
├── auto_trade_polymarket.py # 自动交易程序
|
||||
├── requirements.txt # 依赖列表
|
||||
└── .env # 环境变量 (需自行创建)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 更新项目
|
||||
|
||||
```powershell
|
||||
git pull origin main
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*如有问题,请联系管理员。*
|
||||
@@ -1,395 +0,0 @@
|
||||
"""四模型集成每日准确率分析 - 多时间段测试(2019年1月、2020年1月、2026年1月)"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import pickle
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df = df[:-self.forecast_horizon].reset_index(drop=True)
|
||||
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
future_prices = df['future_price'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
y.append(scaled_future[i + self.lookback, 0])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
def inverse_transform_price(self, scaled_price):
|
||||
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
|
||||
|
||||
|
||||
def predict_period(period_name, start_date, end_date, dlinear_model, nbeats_model, ttm_model, timesfm_model,
|
||||
dlinear_preprocessor, nbeats_preprocessor, device):
|
||||
"""预测单个时间段的数据"""
|
||||
|
||||
print(f"\n{'='*80}")
|
||||
print(f"📅 {period_name} ({start_date} 至 {end_date})")
|
||||
print(f"{'='*80}")
|
||||
|
||||
# 获取数据
|
||||
fetcher = BinanceDataFetcher()
|
||||
df = fetcher.fetch_ohlcv('BTC', '1h', start_date=start_date, end_date=end_date, use_cache=False)
|
||||
|
||||
print(f"✓ 数据: {len(df)} 行")
|
||||
|
||||
# 预处理
|
||||
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df, fit=False)
|
||||
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df, fit=False)
|
||||
|
||||
min_samples = min(len(X_dlinear), len(X_nbeats))
|
||||
X_dlinear = X_dlinear[:min_samples]
|
||||
y_dlinear = y_dlinear[:min_samples]
|
||||
X_nbeats = X_nbeats[:min_samples]
|
||||
|
||||
print(f"✓ 样本数: {min_samples}")
|
||||
|
||||
# DLinear预测
|
||||
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
|
||||
with torch.no_grad():
|
||||
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
|
||||
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits))
|
||||
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten()
|
||||
|
||||
# N-BEATS预测
|
||||
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
|
||||
with torch.no_grad():
|
||||
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
|
||||
|
||||
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
|
||||
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
|
||||
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
|
||||
|
||||
# TTM-R2预测
|
||||
context_length = 512
|
||||
ttm_preds_list = []
|
||||
all_prices = df['close'].values
|
||||
all_open_prices = df['open'].values
|
||||
|
||||
for i in range(min_samples):
|
||||
start_idx = i
|
||||
end_idx = start_idx + context_length
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
context_data = all_prices[-context_length:].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
context_tensor = torch.FloatTensor(context_data).to(device)
|
||||
|
||||
with torch.no_grad():
|
||||
output = ttm_model(context_tensor)
|
||||
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
|
||||
|
||||
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
|
||||
ttm_preds = np.array(ttm_preds_list)
|
||||
|
||||
# TimesFM预测
|
||||
timesfm_preds_list = []
|
||||
|
||||
for i in range(min_samples):
|
||||
start_idx = i
|
||||
end_idx = start_idx + 512
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
context_prices = all_prices[-512:]
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_prices = all_prices[start_idx:end_idx]
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
try:
|
||||
point_forecast, _ = timesfm_model.forecast(horizon=1, inputs=[context_prices])
|
||||
pred_price = point_forecast[0, 0]
|
||||
timesfm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
except:
|
||||
timesfm_preds_list.append(np.random.randint(0, 2))
|
||||
|
||||
timesfm_preds = np.array(timesfm_preds_list)
|
||||
|
||||
# 集成策略
|
||||
true_labels = y_dlinear
|
||||
|
||||
# 双模型交集
|
||||
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
|
||||
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
|
||||
two_model_preds = np.full(len(dlinear_preds), -1)
|
||||
two_model_preds[two_model_agree_up] = 1
|
||||
two_model_preds[two_model_agree_down] = 0
|
||||
|
||||
# 三模型投票
|
||||
vote_sum_3 = dlinear_preds + nbeats_preds + ttm_preds
|
||||
three_model_preds = np.where(vote_sum_3 >= 2, 1, 0)
|
||||
|
||||
# 四模型投票
|
||||
vote_sum_4 = dlinear_preds + nbeats_preds + ttm_preds + timesfm_preds
|
||||
four_model_preds = np.where(vote_sum_4 >= 3, 1, 0)
|
||||
|
||||
# 构建每日结果DataFrame
|
||||
timestamps = df['timestamp'].iloc[168:168+min_samples].values
|
||||
timestamps_utc = pd.to_datetime(timestamps)
|
||||
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
|
||||
dates = timestamps_et.date
|
||||
|
||||
results_df = pd.DataFrame({
|
||||
'date': dates,
|
||||
'timestamp_et': timestamps_et,
|
||||
'true_label': true_labels,
|
||||
'dlinear_pred': dlinear_preds,
|
||||
'nbeats_pred': nbeats_preds,
|
||||
'ttm_pred': ttm_preds,
|
||||
'timesfm_pred': timesfm_preds,
|
||||
'two_model_pred': two_model_preds,
|
||||
'three_model_pred': three_model_preds,
|
||||
'four_model_pred': four_model_preds,
|
||||
})
|
||||
|
||||
# 按天统计
|
||||
daily_stats = results_df.groupby('date').agg({
|
||||
'true_label': 'count',
|
||||
'dlinear_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
|
||||
'nbeats_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
|
||||
'ttm_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
|
||||
'timesfm_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
|
||||
'two_model_pred': lambda x: ((x == results_df.loc[x.index, 'true_label']) & (x != -1)).sum(),
|
||||
'three_model_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
|
||||
'four_model_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
|
||||
})
|
||||
|
||||
daily_stats.columns = ['total', 'dlinear_correct', 'nbeats_correct', 'ttm_correct',
|
||||
'timesfm_correct', 'two_model_correct', 'three_model_correct', 'four_model_correct']
|
||||
|
||||
# 计算每天的有效预测数(双模型)
|
||||
two_model_valid = results_df.groupby('date')['two_model_pred'].apply(lambda x: (x != -1).sum())
|
||||
daily_stats['two_model_valid'] = two_model_valid
|
||||
|
||||
# 计算准确率
|
||||
daily_stats['dlinear_acc'] = daily_stats['dlinear_correct'] / daily_stats['total'] * 100
|
||||
daily_stats['nbeats_acc'] = daily_stats['nbeats_correct'] / daily_stats['total'] * 100
|
||||
daily_stats['ttm_acc'] = daily_stats['ttm_correct'] / daily_stats['total'] * 100
|
||||
daily_stats['timesfm_acc'] = daily_stats['timesfm_correct'] / daily_stats['total'] * 100
|
||||
daily_stats['two_model_acc'] = daily_stats['two_model_correct'] / daily_stats['two_model_valid'] * 100
|
||||
daily_stats['three_model_acc'] = daily_stats['three_model_correct'] / daily_stats['total'] * 100
|
||||
daily_stats['four_model_acc'] = daily_stats['four_model_correct'] / daily_stats['total'] * 100
|
||||
|
||||
daily_stats['two_model_acc'] = daily_stats['two_model_acc'].fillna(0)
|
||||
|
||||
return daily_stats, results_df
|
||||
|
||||
|
||||
def main():
|
||||
print("="*80)
|
||||
print("四模型集成每日准确率分析 - 多时间段测试")
|
||||
print("测试时段: 2019年1月、2020年1月、2026年1月")
|
||||
print("="*80)
|
||||
|
||||
# ========== 加载模型 ==========
|
||||
print("\n[1/2] 加载模型...")
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
|
||||
from models.dlinear.model import DLinear
|
||||
dlinear_checkpoint = torch.load(
|
||||
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device, weights_only=False
|
||||
)
|
||||
dlinear_model = DLinear(dlinear_checkpoint['config'])
|
||||
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
|
||||
dlinear_model = dlinear_model.to(device)
|
||||
dlinear_model.eval()
|
||||
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
|
||||
from models.nbeats.model import NBeats
|
||||
nbeats_checkpoint = torch.load(
|
||||
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
|
||||
map_location=device, weights_only=False
|
||||
)
|
||||
nbeats_model = NBeats(nbeats_checkpoint['config'])
|
||||
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
|
||||
nbeats_model = nbeats_model.to(device)
|
||||
nbeats_model.eval()
|
||||
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
|
||||
"ibm-granite/granite-timeseries-ttm-r2",
|
||||
trust_remote_code=True
|
||||
)
|
||||
ttm_model = ttm_model.to(device)
|
||||
ttm_model.eval()
|
||||
|
||||
import timesfm
|
||||
timesfm_model = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
|
||||
"google/timesfm-2.5-200m-pytorch",
|
||||
torch_compile=False
|
||||
)
|
||||
timesfm_model.compile(
|
||||
timesfm.ForecastConfig(
|
||||
max_context=512,
|
||||
max_horizon=1,
|
||||
normalize_inputs=True,
|
||||
use_continuous_quantile_head=True,
|
||||
force_flip_invariance=True,
|
||||
infer_is_positive=True,
|
||||
fix_quantile_crossing=True,
|
||||
)
|
||||
)
|
||||
|
||||
print("✓ 四个模型已加载 (DLinear, N-BEATS, TTM-R2, TimesFM)")
|
||||
|
||||
# ========== 多时间段预测 ==========
|
||||
print("\n[2/2] 多时间段预测...")
|
||||
|
||||
periods = [
|
||||
("2019年1月", "2019-01-01", "2019-01-31"),
|
||||
("2020年1月", "2020-01-01", "2020-01-31"),
|
||||
("2026年1月", "2026-01-01", "2026-01-31"),
|
||||
]
|
||||
|
||||
all_period_stats = {}
|
||||
|
||||
for period_name, start_date, end_date in periods:
|
||||
daily_stats, results_df = predict_period(
|
||||
period_name, start_date, end_date,
|
||||
dlinear_model, nbeats_model, ttm_model, timesfm_model,
|
||||
dlinear_preprocessor, nbeats_preprocessor, device
|
||||
)
|
||||
all_period_stats[period_name] = daily_stats
|
||||
|
||||
# 显示每日准确率
|
||||
print(f"\n每日准确率明细:")
|
||||
print(f"{'日期':<12} {'样本':>4} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'TimesFM':>8} {'双模型':>8} {'三模型':>8} {'四模型':>8}")
|
||||
print("-" * 95)
|
||||
|
||||
for date, row in daily_stats.iterrows():
|
||||
print(f"{date} {row['total']:>4.0f} "
|
||||
f"{row['dlinear_acc']:>7.1f}% "
|
||||
f"{row['nbeats_acc']:>7.1f}% "
|
||||
f"{row['ttm_acc']:>7.1f}% "
|
||||
f"{row['timesfm_acc']:>7.1f}% "
|
||||
f"{row['two_model_acc']:>7.1f}% "
|
||||
f"{row['three_model_acc']:>7.1f}% "
|
||||
f"{row['four_model_acc']:>7.1f}%")
|
||||
|
||||
# 统计总体
|
||||
print(f"\n{period_name} 总体统计:")
|
||||
print(f" DLinear: {daily_stats['dlinear_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
|
||||
print(f" N-BEATS: {daily_stats['nbeats_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
|
||||
print(f" TTM-R2: {daily_stats['ttm_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
|
||||
print(f" TimesFM: {daily_stats['timesfm_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
|
||||
print(f" 双模型: {daily_stats['two_model_correct'].sum() / daily_stats['two_model_valid'].sum() * 100:.2f}% (覆盖率: {daily_stats['two_model_valid'].sum() / daily_stats['total'].sum() * 100:.1f}%)")
|
||||
print(f" 三模型: {daily_stats['three_model_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
|
||||
print(f" 四模型: {daily_stats['four_model_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
|
||||
|
||||
# ========== 跨时期对比 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 跨时期对比总结")
|
||||
print("="*80)
|
||||
|
||||
print(f"\n{'时期':<12} {'DLinear':>9} {'N-BEATS':>9} {'TTM-R2':>9} {'TimesFM':>9} {'双模型':>9} {'三模型':>9} {'四模型':>9}")
|
||||
print("-" * 95)
|
||||
|
||||
for period_name in ["2019年1月", "2020年1月", "2026年1月"]:
|
||||
stats = all_period_stats[period_name]
|
||||
print(f"{period_name:<12} "
|
||||
f"{stats['dlinear_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
|
||||
f"{stats['nbeats_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
|
||||
f"{stats['ttm_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
|
||||
f"{stats['timesfm_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
|
||||
f"{stats['two_model_correct'].sum() / stats['two_model_valid'].sum() * 100:>8.2f}% "
|
||||
f"{stats['three_model_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
|
||||
f"{stats['four_model_correct'].sum() / stats['total'].sum() * 100:>8.2f}%")
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✓ 多时间段每日准确率分析完成!")
|
||||
print("="*80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,289 @@
|
||||
"""2026年1月BTC数据 - DLinear 准确率测试
|
||||
|
||||
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
|
||||
价格对比:预测结果与预测K线(t+1)的收盘价做对比
|
||||
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
|
||||
DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口
|
||||
输出:每小时详细数据、每天准确率、总准确率、美股开盘标记
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pickle
|
||||
import pandas_ta as ta
|
||||
from datetime import timedelta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
|
||||
# ==================== 时区与美股判断 ====================
|
||||
|
||||
def utc_to_beijing(ts):
|
||||
"""UTC时间转北京时间(UTC+8)"""
|
||||
return ts + timedelta(hours=8)
|
||||
|
||||
def is_us_stock_open_by_utc(utc_ts):
|
||||
"""根据UTC时间判断美股当天是否开盘
|
||||
|
||||
美股交易日基于美东时间(ET),UTC-5(冬令时)
|
||||
2026年1月在冬令时范围内(夏令时3月第二个周日开始)
|
||||
所以美东 = UTC - 5h
|
||||
"""
|
||||
et_ts = utc_ts - timedelta(hours=5)
|
||||
et_date = pd.Timestamp(et_ts.date())
|
||||
|
||||
us_holidays_jan_2026 = {
|
||||
pd.Timestamp('2026-01-01'), # 元旦 New Year's Day
|
||||
pd.Timestamp('2026-01-19'), # 马丁·路德·金日 MLK Day
|
||||
}
|
||||
|
||||
if et_date.weekday() >= 5:
|
||||
return False
|
||||
if et_date in us_holidays_jan_2026:
|
||||
return False
|
||||
return True
|
||||
|
||||
# ==================== pickle反序列化占位类 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""pickle.load时需要能找到这个类名"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
# ==================== 工具函数 ====================
|
||||
|
||||
def fmt_pred(val):
|
||||
"""格式化预测结果"""
|
||||
if val is None or (isinstance(val, float) and np.isnan(val)):
|
||||
return "N/A"
|
||||
return "涨" if int(val) == 1 else "跌"
|
||||
|
||||
def fmt_result(pred, actual):
|
||||
"""格式化对错"""
|
||||
if pred is None or (isinstance(pred, float) and np.isnan(pred)):
|
||||
return " "
|
||||
return "✓" if int(pred) == int(actual) else "✗"
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 90)
|
||||
print("2026年1月 BTC 1h 涨跌预测准确率测试 (DLinear)")
|
||||
print("输入:预测K线的前一根 | 对比:预测结果 vs 预测K线收盘价 | 时间:北京时间(UTC+8)")
|
||||
print("=" * 90)
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
print(f"使用设备: {device}")
|
||||
|
||||
# ========== 步骤1: 加载模型 ==========
|
||||
print("\n[1/4] 加载模型...")
|
||||
|
||||
from models.dlinear.model import DLinear
|
||||
checkpoint = torch.load('./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device, weights_only=False)
|
||||
model = DLinear(checkpoint['config'])
|
||||
model.load_state_dict(checkpoint['model_state_dict'])
|
||||
model = model.to(device).eval()
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
preprocessor = pickle.load(f)
|
||||
print(" ✓ DLinear已加载")
|
||||
|
||||
# ========== 步骤2: 获取测试数据 ==========
|
||||
print("\n[2/4] 获取数据...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', end_date='2026-03-01', use_cache=False)
|
||||
|
||||
print(f"数据范围(UTC): {df['timestamp'].min()} -> {df['timestamp'].max()}")
|
||||
print(f"总行数: {len(df)}")
|
||||
|
||||
# ========== 步骤3: 预处理与预测 ==========
|
||||
print("\n[3/4] 运行预测...")
|
||||
|
||||
# DLinear 整体预处理(无create_labels,整个df一次性处理)
|
||||
lookback = preprocessor.lookback
|
||||
df_dl = df.copy()
|
||||
df_dl['rsi'] = ta.rsi(df_dl['close'], length=14)
|
||||
macd = ta.macd(df_dl['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df_dl['macd'] = macd.iloc[:, 0]
|
||||
df_dl['macd_hist'] = macd.iloc[:, 1]
|
||||
df_dl['macd_signal'] = macd.iloc[:, 2]
|
||||
df_dl = df_dl.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
scaled = preprocessor.scaler.transform(df_dl[feature_cols])
|
||||
|
||||
X_dl = []
|
||||
idx_dl = []
|
||||
for i in range(len(scaled) - lookback):
|
||||
X_dl.append(scaled[i:i + lookback])
|
||||
idx_dl.append(i + lookback - 1)
|
||||
X_dl = np.array(X_dl)
|
||||
|
||||
# 构建 timestamp -> 窗口索引映射
|
||||
dl_ts_map = {}
|
||||
for i, idx in enumerate(idx_dl):
|
||||
ts = df_dl.iloc[idx]['timestamp']
|
||||
dl_ts_map[ts] = i
|
||||
|
||||
# 找出1月(北京时间)的所有预测目标K线索引
|
||||
jan_targets = []
|
||||
for target_idx in range(1, len(df)):
|
||||
target_time_utc = df.iloc[target_idx]['timestamp']
|
||||
target_time_bj = utc_to_beijing(target_time_utc)
|
||||
if target_time_bj.year == 2026 and target_time_bj.month == 2:
|
||||
jan_targets.append(target_idx)
|
||||
|
||||
print(f"1月(北京时间)目标K线数: {len(jan_targets)}")
|
||||
|
||||
results = []
|
||||
|
||||
for target_idx in jan_targets:
|
||||
target_time_utc = df.iloc[target_idx]['timestamp']
|
||||
target_time_bj = utc_to_beijing(target_time_utc)
|
||||
|
||||
# 输入K线索引(前一根)
|
||||
input_idx = target_idx - 1
|
||||
current_close = df.iloc[input_idx]['close']
|
||||
target_close = df.iloc[target_idx]['close']
|
||||
actual_label = 1 if target_close > current_close else 0
|
||||
|
||||
# --- DLinear 预测 ---
|
||||
dl_pred = None
|
||||
dl_prob = None
|
||||
input_ts = df.iloc[input_idx]['timestamp']
|
||||
dl_i = dl_ts_map.get(input_ts)
|
||||
if dl_i is not None and dl_i < len(X_dl):
|
||||
x_tensor = torch.FloatTensor(X_dl[dl_i:dl_i+1]).to(device)
|
||||
with torch.no_grad():
|
||||
logits = model(x_tensor).cpu().numpy()
|
||||
prob = 1 / (1 + np.exp(-logits))
|
||||
dl_pred = int(prob.item() > 0.5)
|
||||
dl_prob = float(prob.item())
|
||||
|
||||
results.append({
|
||||
'time_utc': target_time_utc,
|
||||
'time_bj': target_time_bj,
|
||||
'current_close': current_close,
|
||||
'target_close': target_close,
|
||||
'actual': actual_label,
|
||||
'dlinear': dl_pred,
|
||||
'dl_prob': dl_prob,
|
||||
})
|
||||
|
||||
# ========== 步骤4: 输出结果 ==========
|
||||
if not results:
|
||||
print("没有找到符合条件的测试样本")
|
||||
return
|
||||
|
||||
df_res = pd.DataFrame(results)
|
||||
df_res['date_bj'] = df_res['time_bj'].dt.date
|
||||
df_res['us_open'] = df_res['time_utc'].apply(is_us_stock_open_by_utc)
|
||||
|
||||
total_samples = len(df_res)
|
||||
|
||||
print(f"\n[4/4] 准确率统计 (2026年1月,北京时间)")
|
||||
print(f"总样本数: {total_samples}")
|
||||
|
||||
# ========== 每小时详细数据 ==========
|
||||
print("\n" + "=" * 90)
|
||||
print("每小时详细数据")
|
||||
print("=" * 90)
|
||||
|
||||
hourly_header = (
|
||||
f"{'北京时间':<18} {'前收盘':>10} {'现收盘':>10} {'实际':>4} "
|
||||
f"{'DL':>3} {'':>1} {'概率':>6}"
|
||||
)
|
||||
|
||||
for date_bj, day_group in df_res.groupby('date_bj'):
|
||||
us_open = day_group['us_open'].iloc[0]
|
||||
us_str = "美股开盘" if us_open else "美股休市"
|
||||
|
||||
dl_day_correct = (day_group['dlinear'] == day_group['actual']).sum()
|
||||
dl_day_total = day_group['dlinear'].notna().sum()
|
||||
dl_day_acc = dl_day_correct / dl_day_total if dl_day_total > 0 else 0
|
||||
|
||||
print(f"\n--- {date_bj} ({us_str}) --- DL准确率:{dl_day_acc:.1%} ({dl_day_correct}/{dl_day_total})")
|
||||
print(hourly_header)
|
||||
print("-" * 90)
|
||||
|
||||
for _, row in day_group.iterrows():
|
||||
time_str = row['time_bj'].strftime('%m-%d %H:%M')
|
||||
actual_str = fmt_pred(row['actual'])
|
||||
dl_str = fmt_pred(row['dlinear'])
|
||||
dl_r = fmt_result(row['dlinear'], row['actual'])
|
||||
prob_str = f"{row['dl_prob']:.3f}" if row['dl_prob'] is not None else " N/A"
|
||||
|
||||
print(
|
||||
f"{time_str:<18} {row['current_close']:>10.2f} {row['target_close']:>10.2f} {actual_str:>4} "
|
||||
f"{dl_str:>3} {dl_r:>1} {prob_str:>6}"
|
||||
)
|
||||
|
||||
# ========== 每日准确率汇总表 ==========
|
||||
print("\n\n" + "=" * 90)
|
||||
print("每日准确率汇总 (北京时间)")
|
||||
print("=" * 90)
|
||||
|
||||
header = f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8}"
|
||||
print(header)
|
||||
print("-" * 90)
|
||||
|
||||
for date_bj, group in df_res.groupby('date_bj'):
|
||||
total = len(group)
|
||||
us_open = group['us_open'].iloc[0]
|
||||
us_str = "开盘" if us_open else "休市"
|
||||
|
||||
dl_valid = group.dropna(subset=['dlinear'])
|
||||
dl_acc = (dl_valid['dlinear'] == dl_valid['actual']).mean() if len(dl_valid) > 0 else 0
|
||||
dl_str = f"{dl_acc:.1%}"
|
||||
|
||||
print(f"{str(date_bj):<14} {total:>4} {us_str:>6} {dl_str:>8}")
|
||||
|
||||
print("-" * 90)
|
||||
|
||||
# --- 总准确率 ---
|
||||
dl_valid_all = df_res.dropna(subset=['dlinear'])
|
||||
dl_total = (dl_valid_all['dlinear'] == dl_valid_all['actual']).mean() if len(dl_valid_all) > 0 else 0
|
||||
|
||||
print(f"{'总计':<14} {total_samples:>4} {'':>6} {dl_total:.2%}")
|
||||
print("=" * 90)
|
||||
|
||||
# --- 美股开盘日 vs 休市日对比 ---
|
||||
print("\n[美股开盘日 vs 休市日 准确率对比]")
|
||||
print("-" * 60)
|
||||
for label, flag in [("开盘日", True), ("休市日", False)]:
|
||||
subset = df_res[df_res['us_open'] == flag]
|
||||
if len(subset) == 0:
|
||||
continue
|
||||
n = len(subset)
|
||||
dl_s = subset.dropna(subset=['dlinear'])
|
||||
dl_a = (dl_s['dlinear'] == dl_s['actual']).mean() if len(dl_s) > 0 else 0
|
||||
print(f" {label} ({n}样本): DLinear={dl_a:.2%}")
|
||||
|
||||
print("-" * 60)
|
||||
|
||||
# --- 预测分布统计 ---
|
||||
print("\n[预测分布统计]")
|
||||
print("-" * 60)
|
||||
actual_up = (df_res['actual'] == 1).sum()
|
||||
actual_down = (df_res['actual'] == 0).sum()
|
||||
print(f" 实际: 上涨={actual_up} ({actual_up/total_samples:.1%}) 下跌={actual_down} ({actual_down/total_samples:.1%})")
|
||||
|
||||
dl_valid = df_res['dlinear'].dropna()
|
||||
if len(dl_valid) > 0:
|
||||
up = (dl_valid == 1).sum()
|
||||
down = (dl_valid == 0).sum()
|
||||
print(f" DLinear: 预测上涨={up} ({up/len(dl_valid):.1%}) 预测下跌={down} ({down/len(dl_valid):.1%})")
|
||||
|
||||
print("=" * 90)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,454 +0,0 @@
|
||||
"""五模型集成预测:DLinear + N-BEATS + TTM-R2 + TimesFM 2.5 + CatBoost"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import pickle
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from catboost import CatBoostClassifier
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df = df[:-self.forecast_horizon].reset_index(drop=True)
|
||||
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
future_prices = df['future_price'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
y.append(scaled_future[i + self.lookback, 0])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
def inverse_transform_price(self, scaled_price):
|
||||
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
|
||||
|
||||
|
||||
def add_catboost_features(df):
|
||||
"""为CatBoost添加丰富的技术指标特征"""
|
||||
df = df.copy()
|
||||
|
||||
# 基础价格特征
|
||||
df['price_change'] = df['close'] - df['open']
|
||||
df['price_change_pct'] = (df['close'] - df['open']) / df['open']
|
||||
df['high_low_range'] = df['high'] - df['low']
|
||||
df['high_low_range_pct'] = (df['high'] - df['low']) / df['low']
|
||||
|
||||
# 移动平均线
|
||||
df['sma_7'] = ta.sma(df['close'], length=7)
|
||||
df['sma_25'] = ta.sma(df['close'], length=25)
|
||||
df['sma_99'] = ta.sma(df['close'], length=99)
|
||||
df['ema_12'] = ta.ema(df['close'], length=12)
|
||||
df['ema_26'] = ta.ema(df['close'], length=26)
|
||||
|
||||
# RSI
|
||||
df['rsi_14'] = ta.rsi(df['close'], length=14)
|
||||
df['rsi_7'] = ta.rsi(df['close'], length=7)
|
||||
|
||||
# MACD
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
|
||||
# 布林带
|
||||
bbands = ta.bbands(df['close'], length=20)
|
||||
if bbands is not None and not bbands.empty:
|
||||
df['bb_upper'] = bbands.iloc[:, 0]
|
||||
df['bb_mid'] = bbands.iloc[:, 1]
|
||||
df['bb_lower'] = bbands.iloc[:, 2]
|
||||
df['bb_width'] = (df['bb_upper'] - df['bb_lower']) / df['bb_mid']
|
||||
df['bb_position'] = (df['close'] - df['bb_lower']) / (df['bb_upper'] - df['bb_lower'])
|
||||
|
||||
# ATR
|
||||
df['atr'] = ta.atr(df['high'], df['low'], df['close'], length=14)
|
||||
|
||||
# 成交量特征
|
||||
df['volume_sma_7'] = ta.sma(df['volume'], length=7)
|
||||
df['volume_ratio'] = df['volume'] / df['volume_sma_7']
|
||||
|
||||
# OBV
|
||||
df['obv'] = ta.obv(df['close'], df['volume'])
|
||||
|
||||
# Stochastic
|
||||
stoch = ta.stoch(df['high'], df['low'], df['close'])
|
||||
if stoch is not None and not stoch.empty:
|
||||
df['stoch_k'] = stoch.iloc[:, 0]
|
||||
df['stoch_d'] = stoch.iloc[:, 1]
|
||||
|
||||
# ADX
|
||||
adx = ta.adx(df['high'], df['low'], df['close'])
|
||||
if adx is not None and not adx.empty:
|
||||
df['adx'] = adx.iloc[:, 0]
|
||||
|
||||
# 价格动量
|
||||
df['momentum_1'] = df['close'].pct_change(1)
|
||||
df['momentum_3'] = df['close'].pct_change(3)
|
||||
df['momentum_7'] = df['close'].pct_change(7)
|
||||
|
||||
return df
|
||||
|
||||
|
||||
def main():
|
||||
print("="*80)
|
||||
print("五模型集成预测:DLinear + N-BEATS + TTM-R2 + TimesFM + CatBoost")
|
||||
print("="*80)
|
||||
|
||||
# ========== 步骤1: 加载PyTorch模型 ==========
|
||||
print("\n[1/8] 加载DLinear和N-BEATS模型...")
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
|
||||
# 加载DLinear
|
||||
from models.dlinear.model import DLinear
|
||||
dlinear_checkpoint = torch.load(
|
||||
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
dlinear_model = DLinear(dlinear_checkpoint['config'])
|
||||
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
|
||||
dlinear_model = dlinear_model.to(device)
|
||||
dlinear_model.eval()
|
||||
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
|
||||
print(f"✓ DLinear已加载")
|
||||
|
||||
# 加载N-BEATS
|
||||
from models.nbeats.model import NBeats
|
||||
nbeats_checkpoint = torch.load(
|
||||
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
nbeats_model = NBeats(nbeats_checkpoint['config'])
|
||||
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
|
||||
nbeats_model = nbeats_model.to(device)
|
||||
nbeats_model.eval()
|
||||
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
|
||||
print(f"✓ N-BEATS已加载")
|
||||
|
||||
# ========== 步骤2: 加载TTM-R2 ==========
|
||||
print("\n[2/8] 加载TTM-R2零样本模型...")
|
||||
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
|
||||
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
|
||||
"ibm-granite/granite-timeseries-ttm-r2",
|
||||
trust_remote_code=True
|
||||
)
|
||||
ttm_model = ttm_model.to(device)
|
||||
ttm_model.eval()
|
||||
|
||||
print(f"✓ TTM-R2已加载")
|
||||
|
||||
# ========== 步骤3: 加载TimesFM 2.5 ==========
|
||||
print("\n[3/8] 加载TimesFM 2.5模型...")
|
||||
|
||||
import timesfm
|
||||
|
||||
timesfm_model = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
|
||||
"google/timesfm-2.5-200m-pytorch",
|
||||
torch_compile=False
|
||||
)
|
||||
|
||||
timesfm_model.compile(
|
||||
timesfm.ForecastConfig(
|
||||
max_context=512,
|
||||
max_horizon=1,
|
||||
normalize_inputs=True,
|
||||
use_continuous_quantile_head=True,
|
||||
force_flip_invariance=True,
|
||||
infer_is_positive=True,
|
||||
fix_quantile_crossing=True,
|
||||
)
|
||||
)
|
||||
|
||||
print(f"✓ TimesFM 2.5已加载")
|
||||
|
||||
# ========== 步骤4: 加载CatBoost ==========
|
||||
print("\n[4/8] 加载CatBoost模型...")
|
||||
|
||||
catboost_model = CatBoostClassifier()
|
||||
catboost_model.load_model('./models/checkpoints/catboost_classification_2026/catboost_model.cbm')
|
||||
|
||||
with open('./models/checkpoints/catboost_classification_2026/metadata.pkl', 'rb') as f:
|
||||
catboost_metadata = pickle.load(f)
|
||||
catboost_feature_cols = catboost_metadata['feature_cols']
|
||||
|
||||
print(f"✓ CatBoost已加载 (特征数: {len(catboost_feature_cols)})")
|
||||
|
||||
# ========== 步骤5: 获取测试数据 ==========
|
||||
print("\n[5/8] 获取2026年1月测试数据...")
|
||||
|
||||
fetcher = BinanceDataFetcher()
|
||||
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
|
||||
end_date='2026-01-31', use_cache=False)
|
||||
|
||||
print(f"✓ 数据获取完成: {len(df_2026)} 行")
|
||||
|
||||
# ========== 步骤6: 预处理 ==========
|
||||
print("\n[6/8] 预处理数据...")
|
||||
|
||||
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
|
||||
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
|
||||
|
||||
# CatBoost特征工程
|
||||
df_catboost = add_catboost_features(df_2026)
|
||||
df_catboost = df_catboost.dropna().reset_index(drop=True)
|
||||
X_catboost = df_catboost[catboost_feature_cols]
|
||||
|
||||
# 确保样本数量一致
|
||||
min_samples = min(len(X_dlinear), len(X_nbeats), len(X_catboost))
|
||||
X_dlinear = X_dlinear[:min_samples]
|
||||
y_dlinear = y_dlinear[:min_samples]
|
||||
X_nbeats = X_nbeats[:min_samples]
|
||||
X_catboost = X_catboost.iloc[:min_samples]
|
||||
|
||||
print(f"✓ 对齐样本数: {min_samples}")
|
||||
|
||||
# ========== 步骤7: 五个模型分别预测 ==========
|
||||
print("\n[7/8] 五个模型分别预测...")
|
||||
|
||||
# 7.1 DLinear预测
|
||||
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
|
||||
with torch.no_grad():
|
||||
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
|
||||
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits))
|
||||
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten()
|
||||
|
||||
print(f"✓ DLinear: 涨{(dlinear_preds==1).sum()} ({(dlinear_preds==1).sum()/len(dlinear_preds)*100:.1f}%)")
|
||||
|
||||
# 7.2 N-BEATS预测
|
||||
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
|
||||
with torch.no_grad():
|
||||
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
|
||||
|
||||
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
|
||||
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
|
||||
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
|
||||
|
||||
print(f"✓ N-BEATS: 涨{(nbeats_preds==1).sum()} ({(nbeats_preds==1).sum()/len(nbeats_preds)*100:.1f}%)")
|
||||
|
||||
# 7.3 TTM-R2预测
|
||||
context_length = 512
|
||||
ttm_preds_list = []
|
||||
all_prices = df_2026['close'].values
|
||||
all_open_prices = df_2026['open'].values
|
||||
|
||||
for i in range(min_samples):
|
||||
start_idx = i
|
||||
end_idx = start_idx + context_length
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
context_data = all_prices[-context_length:].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
context_tensor = torch.FloatTensor(context_data).to(device)
|
||||
|
||||
with torch.no_grad():
|
||||
output = ttm_model(context_tensor)
|
||||
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
|
||||
|
||||
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
|
||||
ttm_preds = np.array(ttm_preds_list)
|
||||
|
||||
print(f"✓ TTM-R2: 涨{(ttm_preds==1).sum()} ({(ttm_preds==1).sum()/len(ttm_preds)*100:.1f}%)")
|
||||
|
||||
# 7.4 TimesFM 2.5预测
|
||||
timesfm_preds_list = []
|
||||
|
||||
for i in range(min_samples):
|
||||
start_idx = i
|
||||
end_idx = start_idx + 512
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
context_prices = all_prices[-512:]
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_prices = all_prices[start_idx:end_idx]
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
try:
|
||||
point_forecast, _ = timesfm_model.forecast(
|
||||
horizon=1,
|
||||
inputs=[context_prices],
|
||||
)
|
||||
pred_price = point_forecast[0, 0]
|
||||
timesfm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
except:
|
||||
timesfm_preds_list.append(np.random.randint(0, 2))
|
||||
|
||||
timesfm_preds = np.array(timesfm_preds_list)
|
||||
|
||||
print(f"✓ TimesFM: 涨{(timesfm_preds==1).sum()} ({(timesfm_preds==1).sum()/len(timesfm_preds)*100:.1f}%)")
|
||||
|
||||
# 7.5 CatBoost预测
|
||||
catboost_preds = catboost_model.predict(X_catboost).astype(int)
|
||||
|
||||
print(f"✓ CatBoost: 涨{(catboost_preds==1).sum()} ({(catboost_preds==1).sum()/len(catboost_preds)*100:.1f}%)")
|
||||
|
||||
# ========== 步骤8: 集成策略评估 ==========
|
||||
print("\n[8/8] 集成策略评估...")
|
||||
|
||||
true_labels = y_dlinear
|
||||
|
||||
# 策略1: 双模型交集(DLinear + N-BEATS)
|
||||
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
|
||||
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
|
||||
two_model_preds = np.full(len(dlinear_preds), -1)
|
||||
two_model_preds[two_model_agree_up] = 1
|
||||
two_model_preds[two_model_agree_down] = 0
|
||||
|
||||
two_model_valid = two_model_preds != -1
|
||||
two_model_correct = (two_model_preds == true_labels) & two_model_valid
|
||||
two_model_acc = two_model_correct.sum() / two_model_valid.sum() * 100 if two_model_valid.sum() > 0 else 0
|
||||
two_model_coverage = two_model_valid.sum() / len(true_labels) * 100
|
||||
|
||||
# 策略2: 三模型投票(DLinear + N-BEATS + TTM-R2)
|
||||
vote_sum_3 = dlinear_preds + nbeats_preds + ttm_preds
|
||||
three_model_preds = np.where(vote_sum_3 >= 2, 1, 0)
|
||||
three_model_acc = (three_model_preds == true_labels).mean() * 100
|
||||
|
||||
# 策略3: 四模型投票
|
||||
vote_sum_4 = dlinear_preds + nbeats_preds + ttm_preds + timesfm_preds
|
||||
four_model_preds = np.where(vote_sum_4 >= 3, 1, 0)
|
||||
four_model_acc = (four_model_preds == true_labels).mean() * 100
|
||||
|
||||
# 策略4: 五模型强投票(≥3票)
|
||||
vote_sum_5 = dlinear_preds + nbeats_preds + ttm_preds + timesfm_preds + catboost_preds
|
||||
five_model_preds_strong = np.where(vote_sum_5 >= 3, 1, 0)
|
||||
five_model_acc_strong = (five_model_preds_strong == true_labels).mean() * 100
|
||||
|
||||
# 策略5: 五模型弱投票(≥2票)
|
||||
five_model_preds_weak = np.where(vote_sum_5 >= 2, 1, 0)
|
||||
five_model_acc_weak = (five_model_preds_weak == true_labels).mean() * 100
|
||||
|
||||
# ========== 结果展示 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 五模型集成策略对比结果")
|
||||
print("="*80)
|
||||
|
||||
print(f"\n单模型性能:")
|
||||
print(f" DLinear: {(dlinear_preds == true_labels).mean() * 100:.2f}%")
|
||||
print(f" CatBoost: {(catboost_preds == true_labels).mean() * 100:.2f}%")
|
||||
print(f" TimesFM 2.5: {(timesfm_preds == true_labels).mean() * 100:.2f}%")
|
||||
print(f" TTM-R2: {(ttm_preds == true_labels).mean() * 100:.2f}%")
|
||||
print(f" N-BEATS: {(nbeats_preds == true_labels).mean() * 100:.2f}%")
|
||||
|
||||
print(f"\n集成策略性能:")
|
||||
print(f" 双模型交集 (DL+NB): {two_model_acc:.2f}% (覆盖率: {two_model_coverage:.2f}%)")
|
||||
print(f" 三模型投票 (DL+NB+TTM, ≥2票): {three_model_acc:.2f}% (覆盖率: 100%)")
|
||||
print(f" 四模型投票 (前4, ≥3票): {four_model_acc:.2f}% (覆盖率: 100%)")
|
||||
print(f" 五模型强投票 (所有, ≥3票): {five_model_acc_strong:.2f}% (覆盖率: 100%)")
|
||||
print(f" 五模型弱投票 (所有, ≥2票): {five_model_acc_weak:.2f}% (覆盖率: 100%)")
|
||||
|
||||
# 投票分布分析
|
||||
print(f"\n五模型投票分布:")
|
||||
for votes in range(6):
|
||||
count = (vote_sum_5 == votes).sum()
|
||||
pct = count / len(vote_sum_5) * 100
|
||||
bars = "█" * int(pct / 2)
|
||||
print(f" {votes}票同意: {count:3d}次 ({pct:5.1f}%) {bars}")
|
||||
|
||||
# 模型一致性分析
|
||||
print(f"\n模型一致性分析:")
|
||||
all_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1) & (ttm_preds == 1) & (timesfm_preds == 1) & (catboost_preds == 1)
|
||||
all_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0) & (ttm_preds == 0) & (timesfm_preds == 0) & (catboost_preds == 0)
|
||||
all_agree = all_agree_up | all_agree_down
|
||||
|
||||
print(f" 五模型完全一致: {all_agree.sum()}次 ({all_agree.sum()/len(all_agree)*100:.1f}%)")
|
||||
if all_agree.sum() > 0:
|
||||
all_agree_acc = ((five_model_preds_strong[all_agree] == true_labels[all_agree]).mean() * 100)
|
||||
print(f" 一致时准确率: {all_agree_acc:.2f}%")
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✓ 五模型集成预测完成!")
|
||||
print("="*80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,385 +0,0 @@
|
||||
"""四模型集成预测:DLinear + N-BEATS + TTM-R2 + TimesFM 2.5"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import pickle
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df = df[:-self.forecast_horizon].reset_index(drop=True)
|
||||
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
future_prices = df['future_price'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
y.append(scaled_future[i + self.lookback, 0])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
def inverse_transform_price(self, scaled_price):
|
||||
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
|
||||
|
||||
|
||||
def main():
|
||||
print("="*80)
|
||||
print("四模型集成预测:DLinear + N-BEATS + TTM-R2 + TimesFM 2.5")
|
||||
print("="*80)
|
||||
|
||||
# ========== 步骤1: 加载DLinear和N-BEATS模型 ==========
|
||||
print("\n[1/7] 加载DLinear和N-BEATS模型...")
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
|
||||
# 加载DLinear
|
||||
from models.dlinear.model import DLinear
|
||||
dlinear_checkpoint = torch.load(
|
||||
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
dlinear_model = DLinear(dlinear_checkpoint['config'])
|
||||
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
|
||||
dlinear_model = dlinear_model.to(device)
|
||||
dlinear_model.eval()
|
||||
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
|
||||
print(f"✓ DLinear已加载")
|
||||
|
||||
# 加载N-BEATS
|
||||
from models.nbeats.model import NBeats
|
||||
nbeats_checkpoint = torch.load(
|
||||
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
nbeats_model = NBeats(nbeats_checkpoint['config'])
|
||||
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
|
||||
nbeats_model = nbeats_model.to(device)
|
||||
nbeats_model.eval()
|
||||
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
|
||||
print(f"✓ N-BEATS已加载")
|
||||
|
||||
# ========== 步骤2: 加载TTM-R2 ==========
|
||||
print("\n[2/7] 加载TTM-R2零样本模型...")
|
||||
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
|
||||
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
|
||||
"ibm-granite/granite-timeseries-ttm-r2",
|
||||
trust_remote_code=True
|
||||
)
|
||||
ttm_model = ttm_model.to(device)
|
||||
ttm_model.eval()
|
||||
|
||||
print(f"✓ TTM-R2已加载 (参数量: {sum(p.numel() for p in ttm_model.parameters())})")
|
||||
|
||||
# ========== 步骤3: 加载TimesFM 2.5 ==========
|
||||
print("\n[3/7] 加载TimesFM 2.5模型...")
|
||||
|
||||
import timesfm
|
||||
|
||||
timesfm_model = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
|
||||
"google/timesfm-2.5-200m-pytorch",
|
||||
torch_compile=False # 推理时不使用torch.compile
|
||||
)
|
||||
|
||||
# 配置TimesFM用于BTC 1小时预测
|
||||
timesfm_model.compile(
|
||||
timesfm.ForecastConfig(
|
||||
max_context=512, # 使用512小时历史数据
|
||||
max_horizon=1, # 预测未来1小时
|
||||
normalize_inputs=True,
|
||||
use_continuous_quantile_head=True,
|
||||
force_flip_invariance=True,
|
||||
infer_is_positive=True,
|
||||
fix_quantile_crossing=True,
|
||||
)
|
||||
)
|
||||
|
||||
print(f"✓ TimesFM 2.5已加载并配置")
|
||||
print(f" 上下文长度: 512小时")
|
||||
print(f" 预测长度: 1小时")
|
||||
|
||||
# ========== 步骤4: 获取测试数据 ==========
|
||||
print("\n[4/7] 获取2026年1月测试数据...")
|
||||
|
||||
fetcher = BinanceDataFetcher()
|
||||
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
|
||||
end_date='2026-01-30', use_cache=False)
|
||||
|
||||
print(f"✓ 数据获取完成: {len(df_2026)} 行")
|
||||
print(f"✓ 时间范围: {df_2026['timestamp'].min()} 至 {df_2026['timestamp'].max()}")
|
||||
|
||||
# ========== 步骤5: 预处理 ==========
|
||||
print("\n[5/7] 预处理数据...")
|
||||
|
||||
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
|
||||
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
|
||||
|
||||
# 确保样本数量一致
|
||||
min_samples = min(len(X_dlinear), len(X_nbeats))
|
||||
X_dlinear = X_dlinear[:min_samples]
|
||||
y_dlinear = y_dlinear[:min_samples]
|
||||
X_nbeats = X_nbeats[:min_samples]
|
||||
y_nbeats = y_nbeats[:min_samples]
|
||||
|
||||
print(f"✓ 对齐样本数: {min_samples}")
|
||||
|
||||
# ========== 步骤6: 四个模型分别预测 ==========
|
||||
print("\n[6/7] 四个模型分别预测...")
|
||||
|
||||
# 6.1 DLinear预测
|
||||
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
|
||||
with torch.no_grad():
|
||||
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
|
||||
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits)) # sigmoid
|
||||
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten() # 1=涨, 0=跌
|
||||
|
||||
print(f"✓ DLinear预测完成")
|
||||
print(f" 预测上涨: {(dlinear_preds==1).sum()} ({(dlinear_preds==1).sum()/len(dlinear_preds)*100:.1f}%)")
|
||||
print(f" 预测下跌: {(dlinear_preds==0).sum()} ({(dlinear_preds==0).sum()/len(dlinear_preds)*100:.1f}%)")
|
||||
|
||||
# 6.2 N-BEATS预测
|
||||
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
|
||||
with torch.no_grad():
|
||||
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
|
||||
|
||||
# 还原真实价格并使用当前K线open价格判断
|
||||
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
|
||||
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
|
||||
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int) # 1=涨, 0=跌
|
||||
|
||||
print(f"✓ N-BEATS预测完成 (使用当前K线open价格判断)")
|
||||
print(f" 预测上涨: {(nbeats_preds==1).sum()} ({(nbeats_preds==1).sum()/len(nbeats_preds)*100:.1f}%)")
|
||||
print(f" 预测下跌: {(nbeats_preds==0).sum()} ({(nbeats_preds==0).sum()/len(nbeats_preds)*100:.1f}%)")
|
||||
|
||||
# 6.3 TTM-R2预测
|
||||
print(f"✓ TTM-R2预测中...")
|
||||
|
||||
context_length = 512
|
||||
ttm_preds_list = []
|
||||
all_prices = df_2026['close'].values
|
||||
all_open_prices = df_2026['open'].values
|
||||
|
||||
for i in range(min_samples):
|
||||
start_idx = i
|
||||
end_idx = start_idx + context_length
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
context_data = all_prices[-context_length:].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
context_tensor = torch.FloatTensor(context_data).to(device)
|
||||
|
||||
with torch.no_grad():
|
||||
output = ttm_model(context_tensor)
|
||||
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
|
||||
|
||||
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
|
||||
ttm_preds = np.array(ttm_preds_list)
|
||||
|
||||
print(f"✓ TTM-R2预测完成")
|
||||
print(f" 预测上涨: {(ttm_preds==1).sum()} ({(ttm_preds==1).sum()/len(ttm_preds)*100:.1f}%)")
|
||||
print(f" 预测下跌: {(ttm_preds==0).sum()} ({(ttm_preds==0).sum()/len(ttm_preds)*100:.1f}%)")
|
||||
|
||||
# 6.4 TimesFM 2.5预测
|
||||
print(f"✓ TimesFM 2.5预测中...")
|
||||
|
||||
timesfm_preds_list = []
|
||||
|
||||
for i in range(min_samples):
|
||||
# TimesFM使用512小时上下文
|
||||
start_idx = i
|
||||
end_idx = start_idx + 512
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
# 如果数据不足,使用最后512个数据点
|
||||
context_prices = all_prices[-512:]
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_prices = all_prices[start_idx:end_idx]
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
# TimesFM的forecast方法接受list of arrays作为inputs
|
||||
try:
|
||||
point_forecast, _ = timesfm_model.forecast(
|
||||
horizon=1,
|
||||
inputs=[context_prices], # list包含单个序列
|
||||
)
|
||||
|
||||
# point_forecast形状: (1, 1) - (num_series, horizon)
|
||||
pred_price = point_forecast[0, 0]
|
||||
|
||||
# 与当前K线open价格比较
|
||||
timesfm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
|
||||
except Exception as e:
|
||||
print(f" ⚠️ TimesFM预测第{i}个样本失败: {e}")
|
||||
# 使用随机预测作为fallback
|
||||
timesfm_preds_list.append(np.random.randint(0, 2))
|
||||
|
||||
timesfm_preds = np.array(timesfm_preds_list)
|
||||
|
||||
print(f"✓ TimesFM 2.5预测完成")
|
||||
print(f" 预测上涨: {(timesfm_preds==1).sum()} ({(timesfm_preds==1).sum()/len(timesfm_preds)*100:.1f}%)")
|
||||
print(f" 预测下跌: {(timesfm_preds==0).sum()} ({(timesfm_preds==0).sum()/len(timesfm_preds)*100:.1f}%)")
|
||||
|
||||
# ========== 步骤7: 集成策略评估 ==========
|
||||
print("\n[7/7] 集成策略评估...")
|
||||
|
||||
true_labels = y_dlinear
|
||||
|
||||
# 策略1: 双模型交集(DLinear + N-BEATS)
|
||||
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
|
||||
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
|
||||
two_model_preds = np.full(len(dlinear_preds), -1)
|
||||
two_model_preds[two_model_agree_up] = 1
|
||||
two_model_preds[two_model_agree_down] = 0
|
||||
|
||||
two_model_valid = two_model_preds != -1
|
||||
two_model_correct = (two_model_preds == true_labels) & two_model_valid
|
||||
two_model_acc = two_model_correct.sum() / two_model_valid.sum() * 100 if two_model_valid.sum() > 0 else 0
|
||||
two_model_coverage = two_model_valid.sum() / len(true_labels) * 100
|
||||
|
||||
# 策略2: 三模型多数投票(DLinear + N-BEATS + TTM-R2)
|
||||
vote_sum_3 = dlinear_preds + nbeats_preds + ttm_preds
|
||||
three_model_preds = np.where(vote_sum_3 >= 2, 1, 0)
|
||||
three_model_acc = (three_model_preds == true_labels).mean() * 100
|
||||
|
||||
# 策略3: 四模型多数投票(至少3个模型同意)
|
||||
vote_sum_4 = dlinear_preds + nbeats_preds + ttm_preds + timesfm_preds
|
||||
four_model_preds = np.where(vote_sum_4 >= 3, 1, 0)
|
||||
four_model_acc = (four_model_preds == true_labels).mean() * 100
|
||||
|
||||
# 策略4: 四模型半数以上投票(至少2个模型同意)
|
||||
four_model_preds_half = np.where(vote_sum_4 >= 2, 1, 0)
|
||||
four_model_acc_half = (four_model_preds_half == true_labels).mean() * 100
|
||||
|
||||
# ========== 结果展示 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 集成策略对比结果")
|
||||
print("="*80)
|
||||
|
||||
print(f"\n单模型性能:")
|
||||
print(f" DLinear: {(dlinear_preds == true_labels).mean() * 100:.2f}%")
|
||||
print(f" N-BEATS: {(nbeats_preds == true_labels).mean() * 100:.2f}%")
|
||||
print(f" TTM-R2: {(ttm_preds == true_labels).mean() * 100:.2f}%")
|
||||
print(f" TimesFM 2.5: {(timesfm_preds == true_labels).mean() * 100:.2f}%")
|
||||
|
||||
print(f"\n集成策略性能:")
|
||||
print(f" 双模型交集 (DL+NB): {two_model_acc:.2f}% (覆盖率: {two_model_coverage:.2f}%)")
|
||||
print(f" 三模型投票 (DL+NB+TTM, ≥2票): {three_model_acc:.2f}% (覆盖率: 100%)")
|
||||
print(f" 四模型强投票 (所有, ≥3票): {four_model_acc:.2f}% (覆盖率: 100%)")
|
||||
print(f" 四模型弱投票 (所有, ≥2票): {four_model_acc_half:.2f}% (覆盖率: 100%)")
|
||||
|
||||
# 投票分布分析
|
||||
print(f"\n四模型投票分布:")
|
||||
for votes in range(5):
|
||||
count = (vote_sum_4 == votes).sum()
|
||||
pct = count / len(vote_sum_4) * 100
|
||||
bars = "█" * int(pct / 2)
|
||||
print(f" {votes}票同意: {count:3d}次 ({pct:5.1f}%) {bars}")
|
||||
|
||||
# 模型一致性分析
|
||||
print(f"\n模型一致性分析:")
|
||||
all_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1) & (ttm_preds == 1) & (timesfm_preds == 1)
|
||||
all_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0) & (ttm_preds == 0) & (timesfm_preds == 0)
|
||||
all_agree = all_agree_up | all_agree_down
|
||||
|
||||
print(f" 四模型完全一致: {all_agree.sum()}次 ({all_agree.sum()/len(all_agree)*100:.1f}%)")
|
||||
if all_agree.sum() > 0:
|
||||
all_agree_acc = ((four_model_preds[all_agree] == true_labels[all_agree]).mean() * 100)
|
||||
print(f" 一致时准确率: {all_agree_acc:.2f}%")
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✓ 四模型集成预测完成!")
|
||||
print("="*80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,247 @@
|
||||
"""2026年2月BTC数据 - DLinear, N-BEATS, TTM-R2 准确率测试 (使用统一推理管理器)
|
||||
|
||||
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
|
||||
价格对比:预测结果与预测K线(t+1)的收盘价做对比
|
||||
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
from datetime import timedelta
|
||||
from typing import Dict, List, Any
|
||||
|
||||
# 添加项目路径
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
from models.inference import get_inference_manager
|
||||
|
||||
# ==================== 时区与美股判断 ====================
|
||||
|
||||
def utc_to_beijing(ts):
|
||||
"""UTC时间转北京时间(UTC+8)"""
|
||||
return ts + timedelta(hours=8)
|
||||
|
||||
def is_us_stock_open_by_utc(utc_ts):
|
||||
"""根据UTC时间判断美股当天是否开盘"""
|
||||
et_ts = utc_ts - timedelta(hours=5)
|
||||
et_date = pd.Timestamp(et_ts.date())
|
||||
|
||||
us_holidays_feb_2026 = {
|
||||
pd.Timestamp('2026-02-16'), # 总统日 Presidents' Day
|
||||
}
|
||||
|
||||
if et_date.weekday() >= 5:
|
||||
return False
|
||||
if et_date in us_holidays_feb_2026:
|
||||
return False
|
||||
return True
|
||||
|
||||
def fmt_pred(val):
|
||||
"""格式化预测结果"""
|
||||
if val is None or (isinstance(val, float) and np.isnan(val)):
|
||||
return "N/A"
|
||||
return "涨" if int(val) == 1 else "跌"
|
||||
|
||||
def fmt_result(pred, actual):
|
||||
"""格式化对错"""
|
||||
if pred is None or (isinstance(pred, float) and np.isnan(pred)):
|
||||
return " "
|
||||
return "✓" if int(pred) == int(actual) else "✗"
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 110)
|
||||
print("2026年2月 BTC 1h 涨跌预测准确率测试 (DLinear, N-BEATS, TTM-R2)")
|
||||
print("基于统一推理管理器 (models.inference)")
|
||||
print("=" * 110)
|
||||
|
||||
# ========== 步骤1: 加载模型 ==========
|
||||
print("\n[1/4] 加载模型...")
|
||||
manager = get_inference_manager()
|
||||
manager.load_all_models()
|
||||
|
||||
# ========== 步骤2: 获取测试数据 ==========
|
||||
print("\n[2/4] 获取2026年2月数据...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
# 获取足够长的数据以供lookback
|
||||
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', end_date='2026-03-01', use_cache=False)
|
||||
|
||||
print(f"数据范围(UTC): {df['timestamp'].min()} -> {df['timestamp'].max()}")
|
||||
print(f"总行数: {len(df)}")
|
||||
|
||||
# ========== 步骤3: 运行预测 ==========
|
||||
print("\n[3/4] 运行预测 (逐小时回测)...")
|
||||
|
||||
# 找出2月(北京时间)的所有预测目标K线索引
|
||||
jan_targets = []
|
||||
for target_idx in range(200, len(df)): # 从200开始确保有足够lookback
|
||||
target_time_utc = df.iloc[target_idx]['timestamp']
|
||||
target_time_bj = utc_to_beijing(target_time_utc)
|
||||
if target_time_bj.year == 2026 and target_time_bj.month == 2:
|
||||
jan_targets.append(target_idx)
|
||||
|
||||
print(f"2月(北京时间)目标K线数: {len(jan_targets)}")
|
||||
|
||||
results = []
|
||||
|
||||
for i, target_idx in enumerate(jan_targets):
|
||||
if i % 50 == 0:
|
||||
print(f" 处理进度: {i}/{len(jan_targets)}...")
|
||||
|
||||
target_time_utc = df.iloc[target_idx]['timestamp']
|
||||
target_time_bj = utc_to_beijing(target_time_utc)
|
||||
|
||||
# 输入数据:截止到 target_idx - 1
|
||||
# 注意:predict_all 需要传入DataFrame,它会取最后一行作为 "当前"
|
||||
# 所以我们需要传入 df.iloc[:target_idx] (不包含 target_idx)
|
||||
# Python切片是 [start:end),所以 [:target_idx] 包含 0 到 target_idx-1
|
||||
df_input = df.iloc[:target_idx].copy()
|
||||
|
||||
# 实际结果
|
||||
current_close = df.iloc[target_idx-1]['close']
|
||||
target_close = df.iloc[target_idx]['close']
|
||||
actual_label = 1 if target_close > current_close else 0
|
||||
|
||||
# 预测
|
||||
preds = manager.predict_all(df_input)
|
||||
|
||||
dl_pred = preds.get('dlinear', {}).get('prediction')
|
||||
nb_pred = preds.get('nbeats', {}).get('prediction')
|
||||
ttm_pred = preds.get('ttm', {}).get('prediction')
|
||||
ens_pred = preds.get('ensemble', {}).get('prediction')
|
||||
|
||||
results.append({
|
||||
'time_utc': target_time_utc,
|
||||
'time_bj': target_time_bj,
|
||||
'current_close': current_close,
|
||||
'target_close': target_close,
|
||||
'actual': actual_label,
|
||||
'dlinear': dl_pred,
|
||||
'nbeats': nb_pred,
|
||||
'ttm': ttm_pred,
|
||||
'ensemble': ens_pred,
|
||||
'us_open': is_us_stock_open_by_utc(target_time_utc)
|
||||
})
|
||||
|
||||
# ========== 步骤4: 输出结果 ==========
|
||||
if not results:
|
||||
print("没有找到符合条件的测试样本")
|
||||
return
|
||||
|
||||
df_res = pd.DataFrame(results)
|
||||
df_res['date_bj'] = df_res['time_bj'].dt.date
|
||||
total_samples = len(df_res)
|
||||
|
||||
print(f"\n[4/4] 准确率统计 (2026年2月,北京时间)")
|
||||
print(f"总样本数: {total_samples}")
|
||||
|
||||
# ========== 每小时详细数据 ==========
|
||||
print("\n" + "=" * 110)
|
||||
print("每小时详细数据")
|
||||
print("=" * 110)
|
||||
|
||||
hourly_header = (
|
||||
f"{'北京时间':<18} {'前收盘':>10} {'现收盘':>10} {'实际':>4} "
|
||||
f"{'DL':>3} {'':>1} {'NB':>3} {'':>1} {'TTM':>3} {'':>1} {'集成':>4} {'':<1}"
|
||||
)
|
||||
|
||||
for date_bj, day_group in df_res.groupby('date_bj'):
|
||||
us_open = day_group['us_open'].iloc[0]
|
||||
us_str = "美股开盘" if us_open else "美股休市"
|
||||
|
||||
dl_day_acc = (day_group['dlinear'] == day_group['actual']).mean()
|
||||
|
||||
nb_valid = day_group.dropna(subset=['nbeats'])
|
||||
nb_day_acc = (nb_valid['nbeats'] == nb_valid['actual']).mean() if len(nb_valid) > 0 else None
|
||||
|
||||
ttm_valid = day_group.dropna(subset=['ttm'])
|
||||
ttm_day_acc = (ttm_valid['ttm'] == ttm_valid['actual']).mean() if len(ttm_valid) > 0 else None
|
||||
|
||||
ens_valid = day_group.dropna(subset=['ensemble'])
|
||||
ens_day_acc = (ens_valid['ensemble'] == ens_valid['actual']).mean() if len(ens_valid) > 0 else None
|
||||
|
||||
nb_acc_str = f"{nb_day_acc:.1%}" if nb_day_acc is not None else "N/A"
|
||||
ttm_acc_str = f"{ttm_day_acc:.1%}" if ttm_day_acc is not None else "N/A"
|
||||
ens_acc_str = f"{ens_day_acc:.1%}" if ens_day_acc is not None else "N/A"
|
||||
|
||||
print(f"\n--- {date_bj} ({us_str}) --- DL:{dl_day_acc:.1%} | NB:{nb_acc_str} | TTM:{ttm_acc_str} | 集成:{ens_acc_str}")
|
||||
print(hourly_header)
|
||||
print("-" * 110)
|
||||
|
||||
for _, row in day_group.iterrows():
|
||||
time_str = row['time_bj'].strftime('%m-%d %H:%M')
|
||||
actual_str = fmt_pred(row['actual'])
|
||||
|
||||
dl_str = fmt_pred(row['dlinear'])
|
||||
dl_r = fmt_result(row['dlinear'], row['actual'])
|
||||
nb_str = fmt_pred(row['nbeats'])
|
||||
nb_r = fmt_result(row['nbeats'], row['actual'])
|
||||
ttm_str = fmt_pred(row['ttm'])
|
||||
ttm_r = fmt_result(row['ttm'], row['actual'])
|
||||
ens_str = fmt_pred(row['ensemble'])
|
||||
ens_r = fmt_result(row['ensemble'], row['actual'])
|
||||
|
||||
print(
|
||||
f"{time_str:<18} {row['current_close']:>10.2f} {row['target_close']:>10.2f} {actual_str:>4} "
|
||||
f"{dl_str:>3} {dl_r:>1} {nb_str:>3} {nb_r:>1} {ttm_str:>3} {ttm_r:>1} {ens_str:>4} {ens_r:<1}"
|
||||
)
|
||||
|
||||
# ========== 每日准确率汇总表 ==========
|
||||
print("\n\n" + "=" * 110)
|
||||
print("每日准确率汇总 (北京时间)")
|
||||
print("=" * 110)
|
||||
|
||||
header = f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'集成(投票)':>10}"
|
||||
print(header)
|
||||
print("-" * 110)
|
||||
|
||||
for date_bj, group in df_res.groupby('date_bj'):
|
||||
total = len(group)
|
||||
us_open = group['us_open'].iloc[0]
|
||||
us_str = "开盘" if us_open else "休市"
|
||||
|
||||
dl_acc = (group['dlinear'] == group['actual']).mean()
|
||||
|
||||
nb_valid = group.dropna(subset=['nbeats'])
|
||||
nb_acc = (nb_valid['nbeats'] == nb_valid['actual']).mean() if len(nb_valid) > 0 else None
|
||||
|
||||
ttm_valid = group.dropna(subset=['ttm'])
|
||||
ttm_acc = (ttm_valid['ttm'] == ttm_valid['actual']).mean() if len(ttm_valid) > 0 else None
|
||||
|
||||
ens_valid = group.dropna(subset=['ensemble'])
|
||||
ens_acc = (ens_valid['ensemble'] == ens_valid['actual']).mean() if len(ens_valid) > 0 else None
|
||||
|
||||
dl_str = f"{dl_acc:.1%}"
|
||||
nb_str = f"{nb_acc:.1%}" if nb_acc is not None else "N/A"
|
||||
ttm_str = f"{ttm_acc:.1%}" if ttm_acc is not None else "N/A"
|
||||
ens_str = f"{ens_acc:.1%}" if ens_acc is not None else "N/A"
|
||||
|
||||
print(f"{str(date_bj):<14} {total:>4} {us_str:>6} {dl_str:>8} {nb_str:>8} {ttm_str:>8} {ens_str:>10}")
|
||||
|
||||
print("-" * 110)
|
||||
|
||||
# --- 总准确率 ---
|
||||
dl_total = (df_res['dlinear'] == df_res['actual']).mean()
|
||||
|
||||
nb_all = df_res.dropna(subset=['nbeats'])
|
||||
nb_total = (nb_all['nbeats'] == nb_all['actual']).mean() if len(nb_all) > 0 else None
|
||||
|
||||
ttm_all = df_res.dropna(subset=['ttm'])
|
||||
ttm_total = (ttm_all['ttm'] == ttm_all['actual']).mean() if len(ttm_all) > 0 else None
|
||||
|
||||
ens_all = df_res.dropna(subset=['ensemble'])
|
||||
ens_total = (ens_all['ensemble'] == ens_all['actual']).mean() if len(ens_all) > 0 else None
|
||||
|
||||
nb_t_str = f"{nb_total:.2%}" if nb_total is not None else "N/A"
|
||||
ttm_t_str = f"{ttm_total:.2%}" if ttm_total is not None else "N/A"
|
||||
ens_t_str = f"{ens_total:.2%}" if ens_total is not None else "N/A"
|
||||
|
||||
print(f"{'总计':<14} {total_samples:>4} {'':>6} {dl_total:.2%} {nb_t_str:>8} {ttm_t_str:>8} {ens_t_str:>10}")
|
||||
print("=" * 110)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,427 @@
|
||||
"""2026年2月BTC数据 - DLinear, N-BEATS, TTM-R2 准确率测试
|
||||
|
||||
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
|
||||
价格对比:预测结果与预测K线(t+1)的收盘价做对比
|
||||
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
|
||||
DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口
|
||||
输出:每小时详细数据、每天准确率、总准确率、三模型集成、美股开盘标记
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pickle
|
||||
import pandas_ta as ta
|
||||
from datetime import timedelta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
|
||||
# ==================== 时区与美股判断 ====================
|
||||
|
||||
def utc_to_beijing(ts):
|
||||
"""UTC时间转北京时间(UTC+8)"""
|
||||
return ts + timedelta(hours=8)
|
||||
|
||||
def is_us_stock_open_by_utc(utc_ts):
|
||||
"""根据UTC时间判断美股当天是否开盘
|
||||
|
||||
美股交易日基于美东时间(ET),UTC-5(冬令时)
|
||||
2026年2月在冬令时范围内(夏令时3月第二个周日开始)
|
||||
所以美东 = UTC - 5h
|
||||
"""
|
||||
et_ts = utc_ts - timedelta(hours=5)
|
||||
et_date = pd.Timestamp(et_ts.date())
|
||||
|
||||
us_holidays_feb_2026 = {
|
||||
pd.Timestamp('2026-02-16'), # 总统日 Presidents' Day
|
||||
}
|
||||
|
||||
if et_date.weekday() >= 5:
|
||||
return False
|
||||
if et_date in us_holidays_feb_2026:
|
||||
return False
|
||||
return True
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器 - pickle反序列化占位类"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
|
||||
X = []
|
||||
valid_indices = []
|
||||
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
valid_indices.append(i + self.lookback - 1)
|
||||
|
||||
return np.array(X), df, valid_indices
|
||||
|
||||
|
||||
def fmt_pred(val):
|
||||
"""格式化预测结果"""
|
||||
if val is None or (isinstance(val, float) and np.isnan(val)):
|
||||
return "N/A"
|
||||
return "涨" if int(val) == 1 else "跌"
|
||||
|
||||
def fmt_result(pred, actual):
|
||||
"""格式化对错"""
|
||||
if pred is None or (isinstance(pred, float) and np.isnan(pred)):
|
||||
return " "
|
||||
return "✓" if int(pred) == int(actual) else "✗"
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 110)
|
||||
print("2026年2月 BTC 1h 涨跌预测准确率测试 (DLinear, N-BEATS, TTM-R2)")
|
||||
print("输入:预测K线的前一根 | 对比:预测结果 vs 预测K线收盘价 | 时间:北京时间(UTC+8)")
|
||||
print("DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口")
|
||||
print("=" * 110)
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
print(f"使用设备: {device}")
|
||||
|
||||
# ========== 步骤1: 加载模型 ==========
|
||||
print("\n[1/4] 加载模型...")
|
||||
|
||||
from models.dlinear.model import DLinear
|
||||
dlinear_checkpoint = torch.load('./models/checkpoints/dlinear_classification_2026/best_model.pt', map_location=device, weights_only=False)
|
||||
dlinear_model = DLinear(dlinear_checkpoint['config'])
|
||||
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
|
||||
dlinear_model = dlinear_model.to(device).eval()
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
print(" ✓ DLinear已加载")
|
||||
|
||||
from models.nbeats.model import NBeats
|
||||
nbeats_checkpoint = torch.load('./models/checkpoints/nbeats_btc_1h/best_model.pt', map_location=device, weights_only=False)
|
||||
nbeats_model = NBeats(nbeats_checkpoint['config']).to(device).eval()
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
print(" ✓ N-BEATS已加载")
|
||||
|
||||
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
|
||||
"ibm-granite/granite-timeseries-ttm-r2",
|
||||
trust_remote_code=True
|
||||
).to(device).eval()
|
||||
print(" ✓ TTM-R2已加载")
|
||||
|
||||
# ========== 步骤2: 获取测试数据 ==========
|
||||
print("\n[2/4] 获取2026年1月数据...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-10-01', end_date='2026-02-01', use_cache=False)
|
||||
|
||||
print(f"数据范围(UTC): {df['timestamp'].min()} -> {df['timestamp'].max()}")
|
||||
print(f"总行数: {len(df)}")
|
||||
|
||||
# ========== 步骤3: 预处理与预测 ==========
|
||||
print("\n[3/4] 运行预测...")
|
||||
|
||||
# DLinear 整体预处理(无create_labels,整个df一次性处理)
|
||||
dl_lookback = dlinear_preprocessor.lookback
|
||||
df_dl = df.copy()
|
||||
df_dl['rsi'] = ta.rsi(df_dl['close'], length=14)
|
||||
macd = ta.macd(df_dl['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df_dl['macd'] = macd.iloc[:, 0]
|
||||
df_dl['macd_hist'] = macd.iloc[:, 1]
|
||||
df_dl['macd_signal'] = macd.iloc[:, 2]
|
||||
df_dl = df_dl.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
scaled_dl = dlinear_preprocessor.scaler.transform(df_dl[feature_cols])
|
||||
|
||||
X_dl = []
|
||||
idx_dl = []
|
||||
for i in range(len(scaled_dl) - dl_lookback):
|
||||
X_dl.append(scaled_dl[i:i + dl_lookback])
|
||||
idx_dl.append(i + dl_lookback - 1)
|
||||
X_dl = np.array(X_dl)
|
||||
# 构建 timestamp -> 窗口索引映射
|
||||
dl_ts_map = {}
|
||||
for i, idx in enumerate(idx_dl):
|
||||
ts = df_dl.iloc[idx]['timestamp']
|
||||
dl_ts_map[ts] = i
|
||||
|
||||
# N-BEATS 整体预处理(N-BEATS只用close,不受create_labels影响)
|
||||
X_nb, df_nb, idx_nb = nbeats_preprocessor.preprocess(df, fit=False)
|
||||
|
||||
# TTM-R2 整体预处理
|
||||
ttm_context_len = 512
|
||||
scaler_ttm = StandardScaler()
|
||||
scaled_close_ttm = scaler_ttm.fit_transform(df[['close']])
|
||||
X_ttm = []
|
||||
idx_ttm = []
|
||||
for i in range(len(scaled_close_ttm) - ttm_context_len):
|
||||
X_ttm.append(scaled_close_ttm[i:i+ttm_context_len])
|
||||
idx_ttm.append(i + ttm_context_len - 1)
|
||||
X_ttm = np.array(X_ttm)
|
||||
idx_ttm_map = {v: k for k, v in enumerate(idx_ttm)}
|
||||
|
||||
# 找出2月(北京时间)的所有预测目标K线索引
|
||||
# df 中每一行都是一根已收盘的K线,timestamp 是 UTC
|
||||
jan_targets = []
|
||||
for target_idx in range(1, len(df)):
|
||||
target_time_utc = df.iloc[target_idx]['timestamp']
|
||||
target_time_bj = utc_to_beijing(target_time_utc)
|
||||
if target_time_bj.year == 2026 and target_time_bj.month == 1:
|
||||
jan_targets.append(target_idx)
|
||||
|
||||
print(f"1月(北京时间)目标K线数: {len(jan_targets)}")
|
||||
|
||||
results = []
|
||||
|
||||
for target_idx in jan_targets:
|
||||
target_time_utc = df.iloc[target_idx]['timestamp']
|
||||
target_time_bj = utc_to_beijing(target_time_utc)
|
||||
|
||||
# 输入K线索引(前一根)
|
||||
input_idx = target_idx - 1
|
||||
current_close = df.iloc[input_idx]['close']
|
||||
target_close = df.iloc[target_idx]['close']
|
||||
actual_label = 1 if target_close > current_close else 0
|
||||
|
||||
# --- DLinear 预测(整个df一次性预处理,按索引查找窗口) ---
|
||||
dl_pred = None
|
||||
dl_prob = None
|
||||
input_ts = df.iloc[input_idx]['timestamp']
|
||||
dl_i = dl_ts_map.get(input_ts)
|
||||
if dl_i is not None and dl_i < len(X_dl):
|
||||
x_dl_tensor = torch.FloatTensor(X_dl[dl_i:dl_i+1]).to(device)
|
||||
with torch.no_grad():
|
||||
logits = dlinear_model(x_dl_tensor).cpu().numpy()
|
||||
prob = 1 / (1 + np.exp(-logits))
|
||||
dl_pred = int(prob.item() > 0.5)
|
||||
dl_prob = float(prob.item())
|
||||
|
||||
# --- N-BEATS 预测 ---
|
||||
nb_pred = None
|
||||
if input_idx in idx_nb:
|
||||
nb_i = idx_nb.index(input_idx)
|
||||
x_nb_tensor = torch.FloatTensor(X_nb[nb_i:nb_i+1]).to(device)
|
||||
with torch.no_grad():
|
||||
nb_out = nbeats_model(x_nb_tensor).cpu().numpy()
|
||||
if nb_out.ndim == 1:
|
||||
pred_price_scaled = nb_out[0]
|
||||
else:
|
||||
pred_price_scaled = nb_out[0, 0]
|
||||
current_price_scaled = X_nb[nb_i, -1]
|
||||
nb_pred = 1 if pred_price_scaled > current_price_scaled else 0
|
||||
|
||||
# --- TTM-R2 预测 ---
|
||||
ttm_pred = None
|
||||
ttm_i = idx_ttm_map.get(input_idx)
|
||||
if ttm_i is not None:
|
||||
x_ttm_tensor = torch.FloatTensor(X_ttm[ttm_i:ttm_i+1]).to(device)
|
||||
with torch.no_grad():
|
||||
ttm_out = ttm_model(x_ttm_tensor)
|
||||
ttm_pred_scaled = ttm_out.prediction_outputs[0, 0, 0].cpu().numpy()
|
||||
ttm_current_scaled = X_ttm[ttm_i, -1, 0]
|
||||
ttm_pred = 1 if ttm_pred_scaled > ttm_current_scaled else 0
|
||||
|
||||
# 集成预测(多数投票)
|
||||
votes = [v for v in [dl_pred, nb_pred, ttm_pred] if v is not None]
|
||||
ens_pred = (1 if sum(votes) >= 2 else 0) if len(votes) == 3 else None
|
||||
|
||||
results.append({
|
||||
'time_utc': target_time_utc,
|
||||
'time_bj': target_time_bj,
|
||||
'current_close': current_close,
|
||||
'target_close': target_close,
|
||||
'actual': actual_label,
|
||||
'dlinear': dl_pred,
|
||||
'nbeats': nb_pred,
|
||||
'ttm': ttm_pred,
|
||||
'ensemble': ens_pred,
|
||||
})
|
||||
|
||||
# ========== 步骤4: 输出结果 ==========
|
||||
if not results:
|
||||
print("没有找到符合条件的测试样本")
|
||||
return
|
||||
|
||||
df_res = pd.DataFrame(results)
|
||||
df_res['date_bj'] = df_res['time_bj'].dt.date
|
||||
df_res['us_open'] = df_res['time_utc'].apply(is_us_stock_open_by_utc)
|
||||
|
||||
total_samples = len(df_res)
|
||||
|
||||
print(f"\n[4/4] 准确率统计 (2026年2月,北京时间)")
|
||||
print(f"总样本数: {total_samples}")
|
||||
|
||||
# ========== 每小时详细数据 ==========
|
||||
print("\n" + "=" * 110)
|
||||
print("每小时详细数据")
|
||||
print("=" * 110)
|
||||
|
||||
hourly_header = (
|
||||
f"{'北京时间':<18} {'前收盘':>10} {'现收盘':>10} {'实际':>4} "
|
||||
f"{'DL':>3} {'':>1} {'NB':>3} {'':>1} {'TTM':>3} {'':>1} {'集成':>4} {'':<1}"
|
||||
)
|
||||
|
||||
for date_bj, day_group in df_res.groupby('date_bj'):
|
||||
us_open = day_group['us_open'].iloc[0]
|
||||
us_str = "美股开盘" if us_open else "美股休市"
|
||||
|
||||
dl_day_acc = (day_group['dlinear'] == day_group['actual']).mean()
|
||||
nb_day_valid = day_group.dropna(subset=['nbeats'])
|
||||
nb_day_acc = (nb_day_valid['nbeats'] == nb_day_valid['actual']).mean() if len(nb_day_valid) > 0 else None
|
||||
ttm_day_valid = day_group.dropna(subset=['ttm'])
|
||||
ttm_day_acc = (ttm_day_valid['ttm'] == ttm_day_valid['actual']).mean() if len(ttm_day_valid) > 0 else None
|
||||
ens_day_valid = day_group.dropna(subset=['ensemble'])
|
||||
ens_day_acc = (ens_day_valid['ensemble'] == ens_day_valid['actual']).mean() if len(ens_day_valid) > 0 else None
|
||||
|
||||
nb_acc_str = f"{nb_day_acc:.1%}" if nb_day_acc is not None else "N/A"
|
||||
ttm_acc_str = f"{ttm_day_acc:.1%}" if ttm_day_acc is not None else "N/A"
|
||||
ens_acc_str = f"{ens_day_acc:.1%}" if ens_day_acc is not None else "N/A"
|
||||
|
||||
print(f"\n--- {date_bj} ({us_str}) --- DL:{dl_day_acc:.1%} | NB:{nb_acc_str} | TTM:{ttm_acc_str} | 集成:{ens_acc_str}")
|
||||
print(hourly_header)
|
||||
print("-" * 110)
|
||||
|
||||
for _, row in day_group.iterrows():
|
||||
time_str = row['time_bj'].strftime('%m-%d %H:%M')
|
||||
actual_str = fmt_pred(row['actual'])
|
||||
|
||||
dl_str = fmt_pred(row['dlinear'])
|
||||
dl_r = fmt_result(row['dlinear'], row['actual'])
|
||||
nb_str = fmt_pred(row['nbeats'])
|
||||
nb_r = fmt_result(row['nbeats'], row['actual'])
|
||||
ttm_str = fmt_pred(row['ttm'])
|
||||
ttm_r = fmt_result(row['ttm'], row['actual'])
|
||||
ens_str = fmt_pred(row['ensemble'])
|
||||
ens_r = fmt_result(row['ensemble'], row['actual'])
|
||||
|
||||
print(
|
||||
f"{time_str:<18} {row['current_close']:>10.2f} {row['target_close']:>10.2f} {actual_str:>4} "
|
||||
f"{dl_str:>3} {dl_r:>1} {nb_str:>3} {nb_r:>1} {ttm_str:>3} {ttm_r:>1} {ens_str:>4} {ens_r:<1}"
|
||||
)
|
||||
|
||||
# ========== 每日准确率汇总表 ==========
|
||||
print("\n\n" + "=" * 110)
|
||||
print("每日准确率汇总 (北京时间)")
|
||||
print("=" * 110)
|
||||
|
||||
header = f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'集成(投票)':>10}"
|
||||
print(header)
|
||||
print("-" * 110)
|
||||
|
||||
for date_bj, group in df_res.groupby('date_bj'):
|
||||
total = len(group)
|
||||
us_open = group['us_open'].iloc[0]
|
||||
us_str = "开盘" if us_open else "休市"
|
||||
|
||||
dl_acc = (group['dlinear'] == group['actual']).mean()
|
||||
|
||||
nb_valid = group.dropna(subset=['nbeats'])
|
||||
nb_acc = (nb_valid['nbeats'] == nb_valid['actual']).mean() if len(nb_valid) > 0 else None
|
||||
|
||||
ttm_valid = group.dropna(subset=['ttm'])
|
||||
ttm_acc = (ttm_valid['ttm'] == ttm_valid['actual']).mean() if len(ttm_valid) > 0 else None
|
||||
|
||||
ens_valid = group.dropna(subset=['ensemble'])
|
||||
ens_acc = (ens_valid['ensemble'] == ens_valid['actual']).mean() if len(ens_valid) > 0 else None
|
||||
|
||||
dl_str = f"{dl_acc:.1%}"
|
||||
nb_str = f"{nb_acc:.1%}" if nb_acc is not None else "N/A"
|
||||
ttm_str = f"{ttm_acc:.1%}" if ttm_acc is not None else "N/A"
|
||||
ens_str = f"{ens_acc:.1%}" if ens_acc is not None else "N/A"
|
||||
|
||||
print(f"{str(date_bj):<14} {total:>4} {us_str:>6} {dl_str:>8} {nb_str:>8} {ttm_str:>8} {ens_str:>10}")
|
||||
|
||||
print("-" * 110)
|
||||
|
||||
# --- 总准确率 ---
|
||||
dl_total = (df_res['dlinear'] == df_res['actual']).mean()
|
||||
|
||||
nb_all = df_res.dropna(subset=['nbeats'])
|
||||
nb_total = (nb_all['nbeats'] == nb_all['actual']).mean() if len(nb_all) > 0 else None
|
||||
|
||||
ttm_all = df_res.dropna(subset=['ttm'])
|
||||
ttm_total = (ttm_all['ttm'] == ttm_all['actual']).mean() if len(ttm_all) > 0 else None
|
||||
|
||||
ens_all = df_res.dropna(subset=['ensemble'])
|
||||
ens_total = (ens_all['ensemble'] == ens_all['actual']).mean() if len(ens_all) > 0 else None
|
||||
|
||||
nb_t_str = f"{nb_total:.2%}" if nb_total is not None else "N/A"
|
||||
ttm_t_str = f"{ttm_total:.2%}" if ttm_total is not None else "N/A"
|
||||
ens_t_str = f"{ens_total:.2%}" if ens_total is not None else "N/A"
|
||||
|
||||
print(f"{'总计':<14} {total_samples:>4} {'':>6} {dl_total:.2%} {nb_t_str:>8} {ttm_t_str:>8} {ens_t_str:>10}")
|
||||
print("=" * 110)
|
||||
|
||||
# --- 美股开盘日 vs 休市日对比 ---
|
||||
print("\n[美股开盘日 vs 休市日 准确率对比]")
|
||||
print("-" * 80)
|
||||
for label, flag in [("开盘日", True), ("休市日", False)]:
|
||||
subset = df_res[df_res['us_open'] == flag]
|
||||
if len(subset) == 0:
|
||||
continue
|
||||
n = len(subset)
|
||||
dl_a = (subset['dlinear'] == subset['actual']).mean()
|
||||
|
||||
nb_s = subset.dropna(subset=['nbeats'])
|
||||
nb_a = (nb_s['nbeats'] == nb_s['actual']).mean() if len(nb_s) > 0 else None
|
||||
|
||||
ttm_s = subset.dropna(subset=['ttm'])
|
||||
ttm_a = (ttm_s['ttm'] == ttm_s['actual']).mean() if len(ttm_s) > 0 else None
|
||||
|
||||
ens_s = subset.dropna(subset=['ensemble'])
|
||||
ens_a = (ens_s['ensemble'] == ens_s['actual']).mean() if len(ens_s) > 0 else None
|
||||
|
||||
nb_str = f"{nb_a:.2%}" if nb_a is not None else "N/A"
|
||||
ttm_str = f"{ttm_a:.2%}" if ttm_a is not None else "N/A"
|
||||
ens_str = f"{ens_a:.2%}" if ens_a is not None else "N/A"
|
||||
|
||||
print(f" {label} ({n}样本): DLinear={dl_a:.2%} N-BEATS={nb_str} TTM-R2={ttm_str} 集成={ens_str}")
|
||||
|
||||
print("-" * 80)
|
||||
|
||||
# --- 预测分布统计 ---
|
||||
print("\n[预测分布统计]")
|
||||
print("-" * 80)
|
||||
actual_up = (df_res['actual'] == 1).sum()
|
||||
actual_down = (df_res['actual'] == 0).sum()
|
||||
print(f" 实际: 上涨={actual_up} ({actual_up/total_samples:.1%}) 下跌={actual_down} ({actual_down/total_samples:.1%})")
|
||||
|
||||
for name, col in [("DLinear", "dlinear"), ("N-BEATS", "nbeats"), ("TTM-R2", "ttm"), ("集成", "ensemble")]:
|
||||
valid = df_res[col].dropna()
|
||||
if len(valid) == 0:
|
||||
continue
|
||||
up = (valid == 1).sum()
|
||||
down = (valid == 0).sum()
|
||||
print(f" {name}: 预测上涨={up} ({up/len(valid):.1%}) 预测下跌={down} ({down/len(valid):.1%})")
|
||||
|
||||
print("=" * 110)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,488 @@
|
||||
"""2026年2月BTC数据 - DLinear, N-BEATS, TTM-R2 准确率测试 (ONNX版)
|
||||
|
||||
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
|
||||
价格对比:预测结果与预测K线(t+1)的收盘价做对比
|
||||
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
|
||||
DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口
|
||||
输出:每小时详细数据、每天准确率、总准确率、三模型集成、美股开盘标记
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pickle
|
||||
import pandas_ta as ta
|
||||
import onnxruntime as ort
|
||||
from datetime import timedelta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
|
||||
# ==================== 时区与美股判断 ====================
|
||||
|
||||
def utc_to_beijing(ts):
|
||||
"""UTC时间转北京时间(UTC+8)"""
|
||||
return ts + timedelta(hours=8)
|
||||
|
||||
def is_us_stock_open_by_utc(utc_ts):
|
||||
"""根据UTC时间判断美股当天是否开盘
|
||||
|
||||
美股交易日基于美东时间(ET),UTC-5(冬令时)
|
||||
2026年2月在冬令时范围内(夏令时3月第二个周日开始)
|
||||
所以美东 = UTC - 5h
|
||||
"""
|
||||
et_ts = utc_ts - timedelta(hours=5)
|
||||
et_date = pd.Timestamp(et_ts.date())
|
||||
|
||||
us_holidays_feb_2026 = {
|
||||
pd.Timestamp('2026-02-16'), # 总统日 Presidents' Day
|
||||
}
|
||||
|
||||
if et_date.weekday() >= 5:
|
||||
return False
|
||||
if et_date in us_holidays_feb_2026:
|
||||
return False
|
||||
return True
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器 - pickle反序列化占位类"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
|
||||
X = []
|
||||
valid_indices = []
|
||||
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
valid_indices.append(i + self.lookback - 1)
|
||||
|
||||
return np.array(X), df, valid_indices
|
||||
|
||||
|
||||
def fmt_pred(val):
|
||||
"""格式化预测结果"""
|
||||
if val is None or (isinstance(val, float) and np.isnan(val)):
|
||||
return "N/A"
|
||||
return "涨" if int(val) == 1 else "跌"
|
||||
|
||||
def fmt_result(pred, actual):
|
||||
"""格式化对错"""
|
||||
if pred is None or (isinstance(pred, float) and np.isnan(pred)):
|
||||
return " "
|
||||
return "✓" if int(pred) == int(actual) else "✗"
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 110)
|
||||
print("2026年3月 BTC 1h 涨跌预测准确率测试 (DLinear, N-BEATS, TTM-R2) - ONNX Version")
|
||||
print("输入:预测K线的前一根 | 对比:预测结果 vs 预测K线收盘价 | 时间:北京时间(UTC+8)")
|
||||
print("DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口")
|
||||
print("=" * 110)
|
||||
|
||||
# ========== 步骤1: 加载模型 ==========
|
||||
print("\n[1/4] 加载 ONNX 模型...")
|
||||
|
||||
# Load DLinear ONNX
|
||||
try:
|
||||
dlinear_session = ort.InferenceSession('./models/checkpoints/dlinear_classification_2026/model.onnx')
|
||||
print(" ✓ DLinear (ONNX) 已加载")
|
||||
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
except Exception as e:
|
||||
print(f" ✗ DLinear 加载失败: {e}")
|
||||
return
|
||||
|
||||
# Load N-BEATS ONNX
|
||||
try:
|
||||
nbeats_session = ort.InferenceSession('./models/checkpoints/nbeats_btc_1h/model.onnx')
|
||||
print(" ✓ N-BEATS (ONNX) 已加载")
|
||||
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
except Exception as e:
|
||||
print(f" ✗ N-BEATS 加载失败: {e}")
|
||||
return
|
||||
|
||||
# Load TTM-R2 ONNX
|
||||
ttm_session = None
|
||||
try:
|
||||
temp_session = ort.InferenceSession('./models/checkpoints/ttm_r2_onnx/ttm_r2_crypto_channel_mixing.onnx')
|
||||
|
||||
# Test inference with dummy input
|
||||
print(" Testing TTM-R2 ONNX inference...")
|
||||
dummy_input = np.random.randn(1, 512, 1).astype(np.float32)
|
||||
ort_inputs = {}
|
||||
input_names = [i.name for i in temp_session.get_inputs()]
|
||||
print(f"Model inputs: {input_names}")
|
||||
# 根据 session 中的名称映射输入
|
||||
for input_meta in temp_session.get_inputs():
|
||||
name = input_meta.name
|
||||
if 'past_values' in name or name == 'input':
|
||||
ort_inputs[name] = dummy_input
|
||||
elif 'mask' in name:
|
||||
# 创建全 1 mask
|
||||
mask = np.ones_like(dummy_input)
|
||||
# 检查期望的类型
|
||||
if input_meta.type == 'tensor(int64)':
|
||||
mask = mask.astype(np.int64)
|
||||
else:
|
||||
mask = mask.astype(np.float32)
|
||||
ort_inputs[name] = mask
|
||||
print(f"Session run input keys: {list(ort_inputs.keys())}")
|
||||
for k, v in ort_inputs.items():
|
||||
print(f"Input {k} shape: {v.shape}, dtype: {v.dtype}")
|
||||
|
||||
temp_session.run(None, ort_inputs)
|
||||
|
||||
ttm_session = temp_session
|
||||
print(" ✓ TTM-R2 (ONNX) 已加载且可用 (Optimizations Disabled)")
|
||||
except Exception as e:
|
||||
print(f" ⚠ TTM-R2 (ONNX) 加载或推理失败: {e}")
|
||||
print(" 将在本次测试中跳过 TTM-R2 模型")
|
||||
ttm_session = None
|
||||
|
||||
# ========== 步骤2: 获取测试数据 ==========
|
||||
print("\n[2/4] 获取2026年3月数据...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01', end_date='2026-04-01', use_cache=False)
|
||||
|
||||
print(f"数据范围(UTC): {df['timestamp'].min()} -> {df['timestamp'].max()}")
|
||||
print(f"总行数: {len(df)}")
|
||||
|
||||
# ========== 步骤3: 预处理与预测 ==========
|
||||
print("\n[3/4] 运行预测...")
|
||||
|
||||
# DLinear 整体预处理(无create_labels,整个df一次性处理)
|
||||
dl_lookback = dlinear_preprocessor.lookback
|
||||
df_dl = df.copy()
|
||||
df_dl['rsi'] = ta.rsi(df_dl['close'], length=14)
|
||||
macd = ta.macd(df_dl['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df_dl['macd'] = macd.iloc[:, 0]
|
||||
df_dl['macd_hist'] = macd.iloc[:, 1]
|
||||
df_dl['macd_signal'] = macd.iloc[:, 2]
|
||||
df_dl = df_dl.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
scaled_dl = dlinear_preprocessor.scaler.transform(df_dl[feature_cols])
|
||||
|
||||
X_dl = []
|
||||
idx_dl = []
|
||||
for i in range(len(scaled_dl) - dl_lookback):
|
||||
X_dl.append(scaled_dl[i:i + dl_lookback])
|
||||
idx_dl.append(i + dl_lookback - 1)
|
||||
X_dl = np.array(X_dl, dtype=np.float32)
|
||||
# 构建 timestamp -> 窗口索引映射
|
||||
dl_ts_map = {}
|
||||
for i, idx in enumerate(idx_dl):
|
||||
ts = df_dl.iloc[idx]['timestamp']
|
||||
dl_ts_map[ts] = i
|
||||
|
||||
# N-BEATS 整体预处理(N-BEATS只用close,不受create_labels影响)
|
||||
X_nb, df_nb, idx_nb = nbeats_preprocessor.preprocess(df, fit=False)
|
||||
X_nb = X_nb.astype(np.float32) # Ensure float32
|
||||
|
||||
# TTM-R2 整体预处理
|
||||
ttm_context_len = 512
|
||||
scaler_ttm = StandardScaler()
|
||||
scaled_close_ttm = scaler_ttm.fit_transform(df[['close']])
|
||||
X_ttm = []
|
||||
idx_ttm = []
|
||||
for i in range(len(scaled_close_ttm) - ttm_context_len):
|
||||
X_ttm.append(scaled_close_ttm[i:i+ttm_context_len])
|
||||
idx_ttm.append(i + ttm_context_len - 1)
|
||||
X_ttm = np.array(X_ttm, dtype=np.float32)
|
||||
idx_ttm_map = {v: k for k, v in enumerate(idx_ttm)}
|
||||
|
||||
# 找出3月(北京时间)的所有预测目标K线索引
|
||||
# df 中每一行都是一根已收盘的K线,timestamp 是 UTC
|
||||
jan_targets = []
|
||||
for target_idx in range(1, len(df)):
|
||||
# 同样在筛选样本时也将时间向后移动一个小时
|
||||
target_time_utc = df.iloc[target_idx]['timestamp'] + timedelta(hours=1)
|
||||
target_time_bj = utc_to_beijing(target_time_utc)
|
||||
if target_time_bj.year == 2026 and target_time_bj.month == 3:
|
||||
jan_targets.append(target_idx)
|
||||
|
||||
print(f"3月(北京时间)目标K线数: {len(jan_targets)}")
|
||||
|
||||
results = []
|
||||
|
||||
for target_idx in jan_targets:
|
||||
# 原始timestamp表示k线的开始时间,因此预测的结果实际对应下一小时的k线。
|
||||
# 所以展示时需要将时间向后移动1小时。
|
||||
target_time_utc = df.iloc[target_idx]['timestamp'] + timedelta(hours=1)
|
||||
target_time_bj = utc_to_beijing(target_time_utc)
|
||||
|
||||
# 现在我们将预测目标设为下一根k线(即target_idx+1),
|
||||
# 输入使用它的前一根(等于原先的target_idx)。
|
||||
predicted_idx = target_idx + 1
|
||||
if predicted_idx >= len(df):
|
||||
# 已经到达数据末尾,无法构建完整样本,跳过
|
||||
continue
|
||||
input_idx = target_idx
|
||||
# 开盘/收盘对应预测k线
|
||||
open_price = df.iloc[predicted_idx]['open']
|
||||
close_price = df.iloc[predicted_idx]['close']
|
||||
# 实际涨跌以预测k线的开盘为基准
|
||||
actual_label = 1 if close_price > open_price else 0
|
||||
|
||||
# --- DLinear 预测 (ONNX) ---
|
||||
dl_pred = None
|
||||
dl_prob = None
|
||||
input_ts = df.iloc[input_idx]['timestamp']
|
||||
dl_i = dl_ts_map.get(input_ts)
|
||||
if dl_i is not None and dl_i < len(X_dl):
|
||||
x_dl_input = X_dl[dl_i:dl_i+1] # (1, 168, 9)
|
||||
|
||||
# ONNX Inference
|
||||
# Input name: 'input'
|
||||
logits = dlinear_session.run(['output'], {'input': x_dl_input})[0]
|
||||
|
||||
prob = 1 / (1 + np.exp(-logits))
|
||||
dl_pred = int(prob.item() > 0.5)
|
||||
dl_prob = float(prob.item())
|
||||
|
||||
# --- N-BEATS 预测 (ONNX) ---
|
||||
nb_pred = None
|
||||
if input_idx in idx_nb:
|
||||
nb_i = idx_nb.index(input_idx)
|
||||
x_nb_input = X_nb[nb_i:nb_i+1] # (1, 168)
|
||||
|
||||
# ONNX Inference
|
||||
# Input name: 'input'
|
||||
nb_out = nbeats_session.run(['forecast'], {'input': x_nb_input})[0]
|
||||
|
||||
if nb_out.ndim == 1:
|
||||
pred_price_scaled = nb_out[0]
|
||||
else:
|
||||
pred_price_scaled = nb_out[0, 0]
|
||||
current_price_scaled = X_nb[nb_i, -1]
|
||||
nb_pred = 1 if pred_price_scaled > current_price_scaled else 0
|
||||
|
||||
# --- TTM-R2 预测 (ONNX) ---
|
||||
ttm_pred = None
|
||||
ttm_i = idx_ttm_map.get(input_idx)
|
||||
if ttm_i is not None and ttm_session is not None:
|
||||
x_ttm_input = X_ttm[ttm_i:ttm_i+1] # (1, 512, 1)
|
||||
|
||||
# Debug shape
|
||||
# print(f"TTM input shape: {x_ttm_input.shape}, dtype: {x_ttm_input.dtype}")
|
||||
|
||||
# ONNX Inference
|
||||
# Input name: 'past_values'
|
||||
# Try transposing to (1, 1, 512) just in case
|
||||
# x_ttm_input = x_ttm_input.transpose(0, 2, 1)
|
||||
|
||||
ttm_out = ttm_session.run(None, {'past_values': x_ttm_input})[0]
|
||||
|
||||
ttm_pred_scaled = ttm_out[0, 0, 0]
|
||||
ttm_current_scaled = X_ttm[ttm_i, -1, 0]
|
||||
ttm_pred = 1 if ttm_pred_scaled > ttm_current_scaled else 0
|
||||
|
||||
# 集成预测(多数投票)
|
||||
votes = [v for v in [dl_pred, nb_pred, ttm_pred] if v is not None]
|
||||
ens_pred = (1 if sum(votes) >= 2 else 0) if len(votes) == 3 else None
|
||||
|
||||
results.append({
|
||||
'time_utc': target_time_utc,
|
||||
'time_bj': target_time_bj,
|
||||
'open': open_price,
|
||||
'close': close_price,
|
||||
'actual': actual_label,
|
||||
'dlinear': dl_pred,
|
||||
'nbeats': nb_pred,
|
||||
'ttm': ttm_pred,
|
||||
'ensemble': ens_pred,
|
||||
})
|
||||
|
||||
# ========== 步骤4: 输出结果 ==========
|
||||
if not results:
|
||||
print("没有找到符合条件的测试样本")
|
||||
return
|
||||
|
||||
df_res = pd.DataFrame(results)
|
||||
df_res['date_bj'] = df_res['time_bj'].dt.date
|
||||
df_res['us_open'] = df_res['time_utc'].apply(is_us_stock_open_by_utc)
|
||||
|
||||
total_samples = len(df_res)
|
||||
|
||||
print(f"\n[4/4] 准确率统计 (2026年3月,北京时间)")
|
||||
print(f"总样本数: {total_samples}")
|
||||
|
||||
# ========== 每小时详细数据 ==========
|
||||
print("\n" + "=" * 110)
|
||||
print("每小时详细数据")
|
||||
print("=" * 110)
|
||||
|
||||
# 改为显示开盘/收盘
|
||||
hourly_header = (
|
||||
f"{'北京时间':<18} {'开盘':>10} {'收盘':>10} {'实际':>4} "
|
||||
f"{'DL':>3} {'':>1} {'NB':>3} {'':>1} {'TTM':>3} {'':>1} {'集成':>4} {'':<1}"
|
||||
)
|
||||
|
||||
for date_bj, day_group in df_res.groupby('date_bj'):
|
||||
us_open = day_group['us_open'].iloc[0]
|
||||
us_str = "美股开盘" if us_open else "美股休市"
|
||||
|
||||
dl_day_acc = (day_group['dlinear'] == day_group['actual']).mean()
|
||||
nb_day_valid = day_group.dropna(subset=['nbeats'])
|
||||
nb_day_acc = (nb_day_valid['nbeats'] == nb_day_valid['actual']).mean() if len(nb_day_valid) > 0 else None
|
||||
ttm_day_valid = day_group.dropna(subset=['ttm'])
|
||||
ttm_day_acc = (ttm_day_valid['ttm'] == ttm_day_valid['actual']).mean() if len(ttm_day_valid) > 0 else None
|
||||
ens_day_valid = day_group.dropna(subset=['ensemble'])
|
||||
ens_day_acc = (ens_day_valid['ensemble'] == ens_day_valid['actual']).mean() if len(ens_day_valid) > 0 else None
|
||||
|
||||
nb_acc_str = f"{nb_day_acc:.1%}" if nb_day_acc is not None else "N/A"
|
||||
ttm_acc_str = f"{ttm_day_acc:.1%}" if ttm_day_acc is not None else "N/A"
|
||||
ens_acc_str = f"{ens_day_acc:.1%}" if ens_day_acc is not None else "N/A"
|
||||
|
||||
print(f"\n--- {date_bj} ({us_str}) --- DL:{dl_day_acc:.1%} | NB:{nb_acc_str} | TTM:{ttm_acc_str} | 集成:{ens_acc_str}")
|
||||
print(hourly_header)
|
||||
print("-" * 110)
|
||||
|
||||
for _, row in day_group.iterrows():
|
||||
time_str = row['time_bj'].strftime('%m-%d %H:%M')
|
||||
actual_str = fmt_pred(row['actual'])
|
||||
|
||||
dl_str = fmt_pred(row['dlinear'])
|
||||
dl_r = fmt_result(row['dlinear'], row['actual'])
|
||||
nb_str = fmt_pred(row['nbeats'])
|
||||
nb_r = fmt_result(row['nbeats'], row['actual'])
|
||||
ttm_str = fmt_pred(row['ttm'])
|
||||
ttm_r = fmt_result(row['ttm'], row['actual'])
|
||||
ens_str = fmt_pred(row['ensemble'])
|
||||
ens_r = fmt_result(row['ensemble'], row['actual'])
|
||||
|
||||
print(
|
||||
f"{time_str:<18} {row['open']:>10.2f} {row['close']:>10.2f} {actual_str:>4} "
|
||||
f"{dl_str:>3} {dl_r:>1} {nb_str:>3} {nb_r:>1} {ttm_str:>3} {ttm_r:>1} {ens_str:>4} {ens_r:<1}"
|
||||
)
|
||||
|
||||
# ========== 每日准确率汇总表 ==========
|
||||
print("\n\n" + "=" * 110)
|
||||
print("每日准确率汇总 (北京时间)")
|
||||
print("=" * 110)
|
||||
|
||||
header = f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'集成(投票)':>10}"
|
||||
print(header)
|
||||
print("-" * 110)
|
||||
|
||||
for date_bj, group in df_res.groupby('date_bj'):
|
||||
total = len(group)
|
||||
us_open = group['us_open'].iloc[0]
|
||||
us_str = "开盘" if us_open else "休市"
|
||||
|
||||
dl_acc = (group['dlinear'] == group['actual']).mean()
|
||||
|
||||
nb_valid = group.dropna(subset=['nbeats'])
|
||||
nb_acc = (nb_valid['nbeats'] == nb_valid['actual']).mean() if len(nb_valid) > 0 else None
|
||||
|
||||
ttm_valid = group.dropna(subset=['ttm'])
|
||||
ttm_acc = (ttm_valid['ttm'] == ttm_valid['actual']).mean() if len(ttm_valid) > 0 else None
|
||||
|
||||
ens_valid = group.dropna(subset=['ensemble'])
|
||||
ens_acc = (ens_valid['ensemble'] == ens_valid['actual']).mean() if len(ens_valid) > 0 else None
|
||||
|
||||
dl_str = f"{dl_acc:.1%}"
|
||||
nb_str = f"{nb_acc:.1%}" if nb_acc is not None else "N/A"
|
||||
ttm_str = f"{ttm_acc:.1%}" if ttm_acc is not None else "N/A"
|
||||
ens_str = f"{ens_acc:.1%}" if ens_acc is not None else "N/A"
|
||||
|
||||
print(f"{str(date_bj):<14} {total:>4} {us_str:>6} {dl_str:>8} {nb_str:>8} {ttm_str:>8} {ens_str:>10}")
|
||||
|
||||
print("-" * 110)
|
||||
|
||||
# --- 总准确率 ---
|
||||
dl_total = (df_res['dlinear'] == df_res['actual']).mean()
|
||||
|
||||
nb_all = df_res.dropna(subset=['nbeats'])
|
||||
nb_total = (nb_all['nbeats'] == nb_all['actual']).mean() if len(nb_all) > 0 else None
|
||||
|
||||
ttm_all = df_res.dropna(subset=['ttm'])
|
||||
ttm_total = (ttm_all['ttm'] == ttm_all['actual']).mean() if len(ttm_all) > 0 else None
|
||||
|
||||
ens_all = df_res.dropna(subset=['ensemble'])
|
||||
ens_total = (ens_all['ensemble'] == ens_all['actual']).mean() if len(ens_all) > 0 else None
|
||||
|
||||
nb_t_str = f"{nb_total:.2%}" if nb_total is not None else "N/A"
|
||||
ttm_t_str = f"{ttm_total:.2%}" if ttm_total is not None else "N/A"
|
||||
ens_t_str = f"{ens_total:.2%}" if ens_total is not None else "N/A"
|
||||
|
||||
print(f"{'总计':<14} {total_samples:>4} {'':>6} {dl_total:.2%} {nb_t_str:>8} {ttm_t_str:>8} {ens_t_str:>10}")
|
||||
print("=" * 110)
|
||||
|
||||
# --- 美股开盘日 vs 休市日对比 ---
|
||||
print("\n[美股开盘日 vs 休市日 准确率对比]")
|
||||
print("-" * 80)
|
||||
for label, flag in [("开盘日", True), ("休市日", False)]:
|
||||
subset = df_res[df_res['us_open'] == flag]
|
||||
if len(subset) == 0:
|
||||
continue
|
||||
n = len(subset)
|
||||
dl_a = (subset['dlinear'] == subset['actual']).mean()
|
||||
|
||||
nb_s = subset.dropna(subset=['nbeats'])
|
||||
nb_a = (nb_s['nbeats'] == nb_s['actual']).mean() if len(nb_s) > 0 else None
|
||||
|
||||
ttm_s = subset.dropna(subset=['ttm'])
|
||||
ttm_a = (ttm_s['ttm'] == ttm_s['actual']).mean() if len(ttm_s) > 0 else None
|
||||
|
||||
ens_s = subset.dropna(subset=['ensemble'])
|
||||
ens_a = (ens_s['ensemble'] == ens_s['actual']).mean() if len(ens_s) > 0 else None
|
||||
|
||||
nb_str = f"{nb_a:.2%}" if nb_a is not None else "N/A"
|
||||
ttm_str = f"{ttm_a:.2%}" if ttm_a is not None else "N/A"
|
||||
ens_str = f"{ens_a:.2%}" if ens_a is not None else "N/A"
|
||||
|
||||
print(f" {label} ({n}样本): DLinear={dl_a:.2%} N-BEATS={nb_str} TTM-R2={ttm_str} 集成={ens_str}")
|
||||
|
||||
print("-" * 80)
|
||||
|
||||
# --- 预测分布统计 ---
|
||||
print("\n[预测分布统计]")
|
||||
print("-" * 80)
|
||||
actual_up = (df_res['actual'] == 1).sum()
|
||||
actual_down = (df_res['actual'] == 0).sum()
|
||||
print(f" 实际: 上涨={actual_up} ({actual_up/total_samples:.1%}) 下跌={actual_down} ({actual_down/total_samples:.1%})")
|
||||
|
||||
for name, col in [("DLinear", "dlinear"), ("N-BEATS", "nbeats"), ("TTM-R2", "ttm"), ("集成", "ensemble")]:
|
||||
valid = df_res[col].dropna()
|
||||
if len(valid) == 0:
|
||||
continue
|
||||
up = (valid == 1).sum()
|
||||
down = (valid == 0).sum()
|
||||
print(f" {name}: 预测上涨={up} ({up/len(valid):.1%}) 预测下跌={down} ({down/len(valid):.1%})")
|
||||
|
||||
print("=" * 110)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,387 @@
|
||||
"""2026年2月BTC数据 - DLinear, N-BEATS, TTM-R2, DeepSeek-R1(高并发版) 准确率测试
|
||||
|
||||
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
|
||||
价格对比:预测结果与预测K线(t+1)的收盘价做对比
|
||||
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
|
||||
输出:每小时详细数据、每天准确率、总准确率、四模型集成、美股开盘标记
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
import asyncio
|
||||
import aiohttp
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pickle
|
||||
import pandas_ta as ta
|
||||
from datetime import timedelta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
from utils.network_proxy import apply_global_proxy_from_env
|
||||
|
||||
apply_global_proxy_from_env()
|
||||
|
||||
# ==================== 时区与美股判断 ====================
|
||||
|
||||
def utc_to_beijing(ts):
|
||||
return ts + timedelta(hours=8)
|
||||
|
||||
def is_us_stock_open_by_utc(utc_ts):
|
||||
et_ts = utc_ts - timedelta(hours=5)
|
||||
et_date = pd.Timestamp(et_ts.date())
|
||||
us_holidays_feb_2026 = {pd.Timestamp('2026-02-16')}
|
||||
if et_date.weekday() >= 5 or et_date in us_holidays_feb_2026:
|
||||
return False
|
||||
return True
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
X, valid_indices = [], []
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
valid_indices.append(i + self.lookback - 1)
|
||||
return np.array(X), df, valid_indices
|
||||
|
||||
def fmt_pred(val):
|
||||
if val is None or (isinstance(val, float) and np.isnan(val)): return "N/A"
|
||||
return "涨" if int(val) == 1 else "跌"
|
||||
|
||||
def fmt_result(pred, actual):
|
||||
if pred is None or (isinstance(pred, float) and np.isnan(pred)): return " "
|
||||
return "✓" if int(pred) == int(actual) else "✗"
|
||||
|
||||
|
||||
from datetime import datetime
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
import asyncio
|
||||
from openai import AsyncOpenAI
|
||||
import openai
|
||||
|
||||
# ==================== DeepSeek 异步并发核心逻辑 (双重锁绝对防 429) ====================
|
||||
|
||||
def get_log_time():
|
||||
return datetime.now().strftime("%H:%M:%S")
|
||||
|
||||
async def fetch_deepseek_single(client, messages, item_id, rate_lock, global_state, sem):
|
||||
"""
|
||||
单次异步API调用:并发数控制 + 漏桶控速 + 指数退避重试
|
||||
"""
|
||||
attempt = 0
|
||||
while True:
|
||||
attempt += 1
|
||||
|
||||
# 0. 并发锁:保证同一时刻,最多只有 N 个请求在英伟达服务器上运行
|
||||
async with sem:
|
||||
# 1. 速率锁:保证就算允许并发,发包的瞬间也要间隔至少 2.1 秒 (约 28次/分钟,留足余量)
|
||||
async with rate_lock:
|
||||
now = time.time()
|
||||
elapsed = now - global_state['last_call_time']
|
||||
if elapsed < 2.1:
|
||||
await asyncio.sleep(2.1 - elapsed)
|
||||
global_state['last_call_time'] = time.time()
|
||||
|
||||
print(f"[{get_log_time()}] 🚀 [任务 {item_id}] 发起请求 (第 {attempt} 次尝试)...")
|
||||
|
||||
try:
|
||||
# 2. 调用 API (注意:请确保 model 是你刚才探测到的最新可用模型名)
|
||||
completion = await client.chat.completions.create(
|
||||
model="deepseek-ai/deepseek-v3.2", # <--- 如果旧名字报错410,请务必替换为最新名字
|
||||
messages=messages,
|
||||
temperature=0.6,
|
||||
top_p=0.7,
|
||||
max_tokens=1024,
|
||||
timeout=60.0
|
||||
)
|
||||
|
||||
content = completion.choices[0].message.content
|
||||
match = re.search(r'\{.*\}', content, re.DOTALL)
|
||||
|
||||
if match:
|
||||
res = json.loads(match.group(0))
|
||||
pred = res.get('prediction')
|
||||
|
||||
if pred in [0, 1]:
|
||||
pred_str = "涨" if pred == 1 else "跌"
|
||||
print(f"[{get_log_time()}] ✅ [任务 {item_id}] 成功获取预测: {pred_str}")
|
||||
return item_id, int(pred)
|
||||
else:
|
||||
print(f"[{get_log_time()}] ⚠️ [任务 {item_id}] 异常预测值: {pred},重新请求...")
|
||||
continue
|
||||
else:
|
||||
print(f"[{get_log_time()}] ⚠️ [任务 {item_id}] JSON 解析失败,重新请求...")
|
||||
continue
|
||||
|
||||
except openai.RateLimitError:
|
||||
# 指数退避:第一次等 5 秒,第二次等 10 秒,第三次等 20 秒...
|
||||
wait_time = 5 * (2 ** (attempt - 1))
|
||||
wait_time = min(wait_time, 60) # 最多等 60 秒
|
||||
print(f"[{get_log_time()}] 🛑 [任务 {item_id}] 触发限流 (429),休眠 {wait_time} 秒后重试...")
|
||||
await asyncio.sleep(wait_time)
|
||||
except openai.APITimeoutError:
|
||||
print(f"[{get_log_time()}] ⏳ [任务 {item_id}] 请求超时 (>60s),重新请求...")
|
||||
await asyncio.sleep(2)
|
||||
except openai.APIStatusError as e:
|
||||
if e.status_code in [400, 401, 403, 404, 410]:
|
||||
print(f"[{get_log_time()}] 🚨 [任务 {item_id}] 致命错误 (HTTP {e.status_code}): {e.message}。放弃。")
|
||||
return item_id, None
|
||||
print(f"[{get_log_time()}] ❌ [任务 {item_id}] API 报错 (HTTP {e.status_code}),休眠 3 秒后重试...")
|
||||
await asyncio.sleep(3)
|
||||
except Exception as e:
|
||||
print(f"[{get_log_time()}] 💥 [任务 {item_id}] 未知错误: {e},休眠 3 秒后重新请求...")
|
||||
await asyncio.sleep(3)
|
||||
|
||||
|
||||
async def run_all_deepseek_async(tasks_data, api_key):
|
||||
"""统筹所有的异步请求"""
|
||||
total = len(tasks_data)
|
||||
|
||||
# === 核心参数配置 ===
|
||||
MAX_CONCURRENT = 8 # 绝对并发上限:同一时刻最多 2 个请求在处理
|
||||
|
||||
print("\n" + "="*80)
|
||||
print(f"[{get_log_time()}] 开始 DeepSeek 异步并发请求 (双重防 429 版)")
|
||||
print(f"总任务数: {total} 个 | 严格并发数: {MAX_CONCURRENT} | 发包间隔: 2.1 秒")
|
||||
print("="*80 + "\n")
|
||||
|
||||
client = AsyncOpenAI(
|
||||
base_url="https://integrate.api.nvidia.com/v1",
|
||||
api_key=api_key
|
||||
)
|
||||
|
||||
sem = asyncio.Semaphore(MAX_CONCURRENT)
|
||||
rate_lock = asyncio.Lock()
|
||||
global_state = {'last_call_time': 0.0}
|
||||
ds_results_dict = {}
|
||||
|
||||
tasks = []
|
||||
for item in tasks_data:
|
||||
task = asyncio.create_task(
|
||||
fetch_deepseek_single(client, item['messages'], item['id'], rate_lock, global_state, sem)
|
||||
)
|
||||
tasks.append(task)
|
||||
|
||||
completed = 0
|
||||
for coro in asyncio.as_completed(tasks):
|
||||
item_id, pred = await coro
|
||||
ds_results_dict[item_id] = pred
|
||||
completed += 1
|
||||
|
||||
if completed % 10 == 0 or completed == total:
|
||||
print(f"\n[{get_log_time()}] 📊 【全局进度】 DeepSeek 已完成: {completed}/{total} ({(completed/total):.1%})\n")
|
||||
|
||||
return ds_results_dict
|
||||
# ==================== 主程序 ====================
|
||||
|
||||
def main():
|
||||
print("=" * 115)
|
||||
print("BTC 1h 涨跌预测准确率测试 (DLinear, N-BEATS, TTM-R2, DeepSeek-R1 异步版)")
|
||||
print("=" * 115)
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
print(f"使用设备: {device}")
|
||||
|
||||
# ========== 步骤1: 加载模型 ==========
|
||||
print("\n[1/4] 加载本地模型...")
|
||||
from models.dlinear.model import DLinear
|
||||
dlinear_checkpoint = torch.load('./models/checkpoints/dlinear_classification_2026/best_model.pt', map_location=device, weights_only=False)
|
||||
dlinear_model = DLinear(dlinear_checkpoint['config']).to(device).eval()
|
||||
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
|
||||
from models.nbeats.model import NBeats
|
||||
nbeats_checkpoint = torch.load('./models/checkpoints/nbeats_btc_1h/best_model.pt', map_location=device, weights_only=False)
|
||||
nbeats_model = NBeats(nbeats_checkpoint['config']).to(device).eval()
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
|
||||
ttm_model = TinyTimeMixerForPrediction.from_pretrained("ibm-granite/granite-timeseries-ttm-r2", trust_remote_code=True).to(device).eval()
|
||||
|
||||
ds_api_key = os.environ.get("NVIDIA_API_KEY", "")
|
||||
if not ds_api_key:
|
||||
print(" ⚠️ 未检测到 NVIDIA_API_KEY,跳过 DeepSeek 测试。")
|
||||
else:
|
||||
print(" ✓ DeepSeek-R1 (异步并发) 已就绪")
|
||||
|
||||
# ========== 步骤2: 获取历史数据 ==========
|
||||
print("\n[2/4] 获取历史数据并计算指标...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-10-01', end_date='2026-03-01', use_cache=False)
|
||||
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
|
||||
# ========== 步骤3: 本地预处理与预测 ==========
|
||||
print("\n[3/4] 运行本地预测 & 构建 API 任务队列...")
|
||||
# DLinear预处理
|
||||
df_dl = df.copy()
|
||||
df_dl['rsi'] = ta.rsi(df_dl['close'], length=14)
|
||||
macd_dl = ta.macd(df_dl['close'])
|
||||
if macd_dl is not None and not macd_dl.empty:
|
||||
df_dl['macd'], df_dl['macd_hist'], df_dl['macd_signal'] = macd_dl.iloc[:, 0], macd_dl.iloc[:, 1], macd_dl.iloc[:, 2]
|
||||
df_dl = df_dl.dropna().reset_index(drop=True)
|
||||
scaled_dl = dlinear_preprocessor.scaler.transform(df_dl[['open', 'high', 'low', 'close', 'volume', 'rsi', 'macd', 'macd_hist', 'macd_signal']])
|
||||
X_dl, dl_ts_map = [], {}
|
||||
for i in range(len(scaled_dl) - dlinear_preprocessor.lookback):
|
||||
X_dl.append(scaled_dl[i:i + dlinear_preprocessor.lookback])
|
||||
dl_ts_map[df_dl.iloc[i + dlinear_preprocessor.lookback - 1]['timestamp']] = i
|
||||
X_dl = np.array(X_dl)
|
||||
|
||||
# N-BEATS预处理
|
||||
X_nb, df_nb, idx_nb = nbeats_preprocessor.preprocess(df, fit=False)
|
||||
|
||||
# TTM预处理
|
||||
scaler_ttm = StandardScaler()
|
||||
scaled_close_ttm = scaler_ttm.fit_transform(df[['close']])
|
||||
X_ttm, idx_ttm_map = [], {}
|
||||
for i in range(len(scaled_close_ttm) - 512):
|
||||
X_ttm.append(scaled_close_ttm[i:i+512])
|
||||
idx_ttm_map[i + 511] = i
|
||||
X_ttm = np.array(X_ttm)
|
||||
|
||||
target_targets = []
|
||||
for target_idx in range(1, len(df)):
|
||||
target_time_bj = utc_to_beijing(df.iloc[target_idx]['timestamp'])
|
||||
if target_time_bj.year == 2026 and target_time_bj.month == 2: # 锁定 2026年 2月 数据
|
||||
target_targets.append(target_idx)
|
||||
|
||||
temp_results = []
|
||||
deepseek_tasks_data = []
|
||||
|
||||
for target_idx in target_targets:
|
||||
input_idx = target_idx - 1
|
||||
current_close, target_close = df.iloc[input_idx]['close'], df.iloc[target_idx]['close']
|
||||
actual_label = 1 if target_close > current_close else 0
|
||||
|
||||
# DL
|
||||
dl_pred, input_ts = None, df.iloc[input_idx]['timestamp']
|
||||
if (dl_i := dl_ts_map.get(input_ts)) is not None and dl_i < len(X_dl):
|
||||
with torch.no_grad():
|
||||
logits = dlinear_model(torch.FloatTensor(X_dl[dl_i:dl_i+1]).to(device)).cpu().numpy()
|
||||
dl_pred = int((1 / (1 + np.exp(-logits))).item() > 0.5)
|
||||
|
||||
# NB
|
||||
nb_pred = None
|
||||
if input_idx in idx_nb:
|
||||
nb_i = idx_nb.index(input_idx)
|
||||
with torch.no_grad():
|
||||
nb_out = nbeats_model(torch.FloatTensor(X_nb[nb_i:nb_i+1]).to(device)).cpu().numpy()
|
||||
nb_pred = 1 if (nb_out[0] if nb_out.ndim == 1 else nb_out[0, 0]) > X_nb[nb_i, -1] else 0
|
||||
|
||||
# TTM
|
||||
ttm_pred = None
|
||||
if (ttm_i := idx_ttm_map.get(input_idx)) is not None:
|
||||
with torch.no_grad():
|
||||
ttm_pred_scaled = ttm_model(torch.FloatTensor(X_ttm[ttm_i:ttm_i+1]).to(device)).prediction_outputs[0, 0, 0].cpu().numpy()
|
||||
ttm_pred = 1 if ttm_pred_scaled > X_ttm[ttm_i, -1, 0] else 0
|
||||
|
||||
# 构造 DeepSeek Payload
|
||||
if ds_api_key:
|
||||
recent_k = df.iloc[max(0, input_idx-14) : input_idx+1]
|
||||
if len(recent_k) >= 5:
|
||||
kline_text = "".join([f"时间: {row['timestamp']}, 开盘: {row['open']:.2f}, 收盘: {row['close']:.2f}, RSI: {row.get('rsi', 0):.2f}\n" for _, row in recent_k.iterrows()])
|
||||
sys_prompt = '你是一位顶级量化分析师。请分析BTC 1小时K线,预测下一小时涨跌。必须输出合法的JSON格式:{"prediction": 1(看涨)或0(看跌), "reason": "理由"}'
|
||||
messages = [
|
||||
{"role": "system", "content": sys_prompt},
|
||||
{"role": "user", "content": f"近15周期数据:\n{kline_text}\n输出JSON:"}
|
||||
]
|
||||
deepseek_tasks_data.append({'id': target_idx, 'messages': messages})
|
||||
|
||||
temp_results.append({
|
||||
'id': target_idx, 'time_utc': df.iloc[target_idx]['timestamp'], 'time_bj': utc_to_beijing(df.iloc[target_idx]['timestamp']),
|
||||
'current_close': current_close, 'target_close': target_close, 'actual': actual_label,
|
||||
'dlinear': dl_pred, 'nbeats': nb_pred, 'ttm': ttm_pred, 'deepseek': None
|
||||
})
|
||||
|
||||
# ========== 异步执行 DeepSeek 并合入数据 ==========
|
||||
if deepseek_tasks_data:
|
||||
# 并发数设为 5,对 NVIDIA 免费/基础层级比较安全
|
||||
ds_predictions = asyncio.run(run_all_deepseek_async(deepseek_tasks_data, ds_api_key))
|
||||
for res in temp_results:
|
||||
if res['id'] in ds_predictions:
|
||||
res['deepseek'] = ds_predictions[res['id']]
|
||||
|
||||
# 结算集成模型结果 (四投票)
|
||||
final_results = []
|
||||
for res in temp_results:
|
||||
votes = [v for v in [res['dlinear'], res['nbeats'], res['ttm'], res['deepseek']] if v is not None]
|
||||
if votes:
|
||||
vote_sum = sum(votes)
|
||||
if vote_sum > len(votes) / 2: res['ensemble'] = 1
|
||||
elif vote_sum < len(votes) / 2: res['ensemble'] = 0
|
||||
else: res['ensemble'] = res['dlinear'] if res['dlinear'] is not None else 0
|
||||
else:
|
||||
res['ensemble'] = None
|
||||
final_results.append(res)
|
||||
|
||||
# ========== 步骤4: 输出结果 ==========
|
||||
if not final_results:
|
||||
print("没有找到符合条件的测试样本")
|
||||
return
|
||||
|
||||
df_res = pd.DataFrame(final_results)
|
||||
df_res['date_bj'] = df_res['time_bj'].dt.date
|
||||
df_res['us_open'] = df_res['time_utc'].apply(is_us_stock_open_by_utc)
|
||||
|
||||
# === (下方打印逻辑与原版一致,缩略优化版) ===
|
||||
print(f"\n[4/4] 准确率统计 (北京时间)\n总样本数: {len(df_res)}")
|
||||
print("\n" + "=" * 115 + "\n每小时详细数据\n" + "=" * 115)
|
||||
hourly_h = f"{'北京时间':<18} {'前收盘':>10} {'现收盘':>10} {'实际':>4} {'DL':>3} {'':>1} {'NB':>3} {'':>1} {'TTM':>3} {'':>1} {'DS':>3} {'':>1} {'集成':>4} {'':<1}"
|
||||
|
||||
for date_bj, day_group in df_res.groupby('date_bj'):
|
||||
us_str = "美股开盘" if day_group['us_open'].iloc[0] else "美股休市"
|
||||
def s_acc(c): return (day_group.dropna(subset=[c])[c] == day_group.dropna(subset=[c])['actual']).mean() if len(day_group.dropna(subset=[c])) > 0 else None
|
||||
print(f"\n--- {date_bj} ({us_str}) --- DL:{s_acc('dlinear'):.1%} | NB:{s_acc('nbeats'):.1%} | TTM:{s_acc('ttm'):.1%} | DS:{s_acc('deepseek'):.1%} | 集成:{s_acc('ensemble'):.1%}")
|
||||
print(hourly_h + "\n" + "-" * 115)
|
||||
for _, r in day_group.iterrows():
|
||||
print(f"{r['time_bj'].strftime('%m-%d %H:%M'):<18} {r['current_close']:>10.2f} {r['target_close']:>10.2f} {fmt_pred(r['actual']):>4} "
|
||||
f"{fmt_pred(r['dlinear']):>3} {fmt_result(r['dlinear'],r['actual']):>1} {fmt_pred(r['nbeats']):>3} {fmt_result(r['nbeats'],r['actual']):>1} "
|
||||
f"{fmt_pred(r['ttm']):>3} {fmt_result(r['ttm'],r['actual']):>1} {fmt_pred(r['deepseek']):>3} {fmt_result(r['deepseek'],r['actual']):>1} "
|
||||
f"{fmt_pred(r['ensemble']):>4} {fmt_result(r['ensemble'],r['actual']):<1}")
|
||||
|
||||
print("\n\n" + "=" * 115 + "\n每日准确率汇总 (北京时间)\n" + "=" * 115)
|
||||
print(f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'DeepSeek':>8} {'集成':>10}\n" + "-" * 115)
|
||||
for date_bj, g in df_res.groupby('date_bj'):
|
||||
def g_acc(c): return (g.dropna(subset=[c])[c] == g.dropna(subset=[c])['actual']).mean() if len(g.dropna(subset=[c])) > 0 else None
|
||||
print(f"{str(date_bj):<14} {len(g):>4} {'开盘' if g['us_open'].iloc[0] else '休市':>6} {g_acc('dlinear'):>8.1%} {g_acc('nbeats'):>8.1%} {g_acc('ttm'):>8.1%} {g_acc('deepseek'):>8.1%} {g_acc('ensemble'):>10.1%}".replace('nan%', 'N/A'))
|
||||
|
||||
def t_acc(c): return (df_res.dropna(subset=[c])[c] == df_res.dropna(subset=[c])['actual']).mean() if len(df_res.dropna(subset=[c])) > 0 else None
|
||||
print("-" * 115)
|
||||
print(f"{'总计':<14} {len(df_res):>4} {'':>6} {t_acc('dlinear'):>8.2%} {t_acc('nbeats'):>8.2%} {t_acc('ttm'):>8.2%} {t_acc('deepseek'):>8.2%} {t_acc('ensemble'):>10.2%}".replace('nan%', 'N/A'))
|
||||
print("=" * 115)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,295 +0,0 @@
|
||||
"""将Google TimesFM 2.5模型导出为ONNX格式 - 使用正确的API"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import numpy as np
|
||||
|
||||
|
||||
def export_timesfm_to_onnx():
|
||||
print("="*80)
|
||||
print("导出TimesFM 2.5 (200M) 模型为ONNX格式")
|
||||
print("="*80)
|
||||
|
||||
# ========== 步骤1: 加载TimesFM模型 ==========
|
||||
print("\n[1/6] 加载TimesFM 2.5模型...")
|
||||
|
||||
try:
|
||||
import timesfm
|
||||
|
||||
# 使用正确的from_pretrained方法加载模型
|
||||
print(" 从HuggingFace加载预训练模型...")
|
||||
print(" 模型ID: google/timesfm-2.5-200m-pytorch")
|
||||
|
||||
model = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
|
||||
"google/timesfm-2.5-200m-pytorch",
|
||||
torch_compile=False # 导出ONNX时不使用torch.compile
|
||||
)
|
||||
|
||||
print("✓ TimesFM 2.5模型已加载")
|
||||
|
||||
except ImportError as e:
|
||||
print(f"✗ TimesFM未安装或导入失败: {e}")
|
||||
print(" 请运行: pip install git+https://github.com/google-research/timesfm.git")
|
||||
return
|
||||
except Exception as e:
|
||||
print(f"✗ 加载模型失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return
|
||||
|
||||
# ========== 步骤2: 编译模型配置 ==========
|
||||
print("\n[2/6] 配置模型...")
|
||||
|
||||
try:
|
||||
# 配置模型
|
||||
model.compile(
|
||||
timesfm.ForecastConfig(
|
||||
max_context=512, # BTC历史数据512小时
|
||||
max_horizon=24, # 预测未来24小时
|
||||
normalize_inputs=True,
|
||||
use_continuous_quantile_head=True,
|
||||
force_flip_invariance=True,
|
||||
infer_is_positive=True,
|
||||
fix_quantile_crossing=True,
|
||||
)
|
||||
)
|
||||
|
||||
print("✓ 模型配置完成")
|
||||
print(f" 最大上下文长度: 512")
|
||||
print(f" 最大预测长度: 24")
|
||||
print(f" 归一化输入: True")
|
||||
print(f" 使用分位数预测: True")
|
||||
|
||||
except Exception as e:
|
||||
print(f"✗ 配置失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return
|
||||
|
||||
# ========== 步骤3: 测试模型 ==========
|
||||
print("\n[3/6] 测试模型推理...")
|
||||
|
||||
try:
|
||||
# 创建测试数据
|
||||
test_input = [
|
||||
np.linspace(85000, 90000, 512), # 模拟BTC价格从85k到90k
|
||||
np.sin(np.linspace(0, 20, 512)) * 1000 + 87000, # 模拟震荡价格
|
||||
]
|
||||
|
||||
# 测试推理
|
||||
point_forecast, quantile_forecast = model.forecast(
|
||||
horizon=24,
|
||||
inputs=test_input,
|
||||
)
|
||||
|
||||
print("✓ 模型推理成功")
|
||||
print(f" 点预测形状: {point_forecast.shape}") # (2, 24)
|
||||
print(f" 分位数预测形状: {quantile_forecast.shape}") # (2, 24, 11)
|
||||
|
||||
except Exception as e:
|
||||
print(f"⚠️ 模型测试失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
print(" 继续尝试导出...")
|
||||
|
||||
# ========== 步骤4: 提取核心模型并导出ONNX ==========
|
||||
print("\n[4/6] 提取核心模型...")
|
||||
|
||||
try:
|
||||
# TimesFM的核心模型在model.model中
|
||||
core_model = model.model
|
||||
core_model.eval()
|
||||
|
||||
print("✓ 核心模型已提取")
|
||||
print(f" 模型类型: {type(core_model)}")
|
||||
|
||||
# 创建ONNX导出包装器
|
||||
class TimesFMONNXWrapper(torch.nn.Module):
|
||||
"""TimesFM ONNX导出包装器"""
|
||||
def __init__(self, timesfm_core_model):
|
||||
super().__init__()
|
||||
self.model = timesfm_core_model
|
||||
|
||||
def forward(self, inputs, masks=None):
|
||||
"""
|
||||
Args:
|
||||
inputs: (batch_size, context_length) - 输入时间序列
|
||||
masks: (batch_size, context_length) - 可选mask,1=有效,0=无效
|
||||
Returns:
|
||||
point_forecast: (batch_size, horizon_length) - 点预测
|
||||
"""
|
||||
# 如果没有提供masks,创建全1的mask
|
||||
if masks is None:
|
||||
masks = torch.ones_like(inputs)
|
||||
|
||||
# 调用TimesFM核心模型
|
||||
# 注意:这里需要根据实际的forward签名调整
|
||||
outputs, _ = self.model(inputs, masks, decode_caches=None)
|
||||
|
||||
# outputs包含: (input_embeddings, output_embeddings, output_ts, output_quantile_spread)
|
||||
# output_ts是点预测
|
||||
point_forecast = outputs[2]
|
||||
|
||||
return point_forecast
|
||||
|
||||
wrapped_model = TimesFMONNXWrapper(core_model)
|
||||
wrapped_model.eval()
|
||||
|
||||
print("✓ ONNX包装器已创建")
|
||||
|
||||
except Exception as e:
|
||||
print(f"✗ 提取核心模型失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return
|
||||
|
||||
# ========== 步骤5: 导出为ONNX ==========
|
||||
print("\n[5/6] 导出模型为ONNX格式...")
|
||||
|
||||
onnx_path = "./models/timesfm_2p5_200m.onnx"
|
||||
os.makedirs("./models", exist_ok=True)
|
||||
|
||||
try:
|
||||
# 准备示例输入
|
||||
batch_size = 1
|
||||
context_len = 512
|
||||
|
||||
dummy_input = torch.randn(batch_size, context_len, dtype=torch.float32)
|
||||
dummy_mask = torch.ones(batch_size, context_len, dtype=torch.float32)
|
||||
|
||||
print(f" 示例输入形状: inputs={dummy_input.shape}, masks={dummy_mask.shape}")
|
||||
print(" 开始导出ONNX...")
|
||||
|
||||
# 导出为ONNX
|
||||
torch.onnx.export(
|
||||
wrapped_model,
|
||||
(dummy_input, dummy_mask),
|
||||
onnx_path,
|
||||
export_params=True,
|
||||
opset_version=14,
|
||||
do_constant_folding=True,
|
||||
input_names=['inputs', 'masks'],
|
||||
output_names=['point_forecast'],
|
||||
dynamic_axes={
|
||||
'inputs': {0: 'batch_size', 1: 'context_length'},
|
||||
'masks': {0: 'batch_size', 1: 'context_length'},
|
||||
'point_forecast': {0: 'batch_size', 1: 'horizon_length'}
|
||||
},
|
||||
verbose=False
|
||||
)
|
||||
|
||||
print(f"✓ ONNX模型已导出")
|
||||
print(f" 保存路径: {onnx_path}")
|
||||
print(f" 文件大小: {os.path.getsize(onnx_path) / (1024**2):.2f} MB")
|
||||
|
||||
except Exception as e:
|
||||
print(f"✗ ONNX导出失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
# 尝试简化版本
|
||||
print("\n尝试简化导出方案...")
|
||||
try:
|
||||
# 只导出单输入版本
|
||||
torch.onnx.export(
|
||||
wrapped_model,
|
||||
dummy_input,
|
||||
onnx_path,
|
||||
export_params=True,
|
||||
opset_version=14,
|
||||
input_names=['inputs'],
|
||||
output_names=['predictions'],
|
||||
)
|
||||
print(f"✓ 简化版ONNX模型已导出: {onnx_path}")
|
||||
except Exception as e2:
|
||||
print(f"✗ 简化导出也失败: {e2}")
|
||||
return
|
||||
|
||||
# ========== 步骤6: 验证ONNX模型 ==========
|
||||
print("\n[6/6] 验证ONNX模型...")
|
||||
|
||||
try:
|
||||
import onnx
|
||||
import onnxruntime as ort
|
||||
|
||||
# 加载并检查ONNX模型
|
||||
onnx_model = onnx.load(onnx_path)
|
||||
onnx.checker.check_model(onnx_model)
|
||||
print("✓ ONNX模型结构验证通过")
|
||||
|
||||
# 显示模型信息
|
||||
print(f"\n模型信息:")
|
||||
print(f" 输入:")
|
||||
for input_tensor in onnx_model.graph.input:
|
||||
dims = [d.dim_value if d.dim_value > 0 else 'dynamic' for d in input_tensor.type.tensor_type.shape.dim]
|
||||
print(f" - {input_tensor.name}: {dims}")
|
||||
|
||||
print(f" 输出:")
|
||||
for output_tensor in onnx_model.graph.output:
|
||||
dims = [d.dim_value if d.dim_value > 0 else 'dynamic' for d in output_tensor.type.tensor_type.shape.dim]
|
||||
print(f" - {output_tensor.name}: {dims}")
|
||||
|
||||
# 创建ONNX Runtime推理会话
|
||||
print("\n 创建ONNX Runtime推理会话...")
|
||||
ort_session = ort.InferenceSession(onnx_path)
|
||||
|
||||
# 准备测试输入
|
||||
test_input = np.random.randn(1, 512).astype(np.float32)
|
||||
test_mask = np.ones((1, 512), dtype=np.float32)
|
||||
|
||||
# 运行推理
|
||||
print(" 运行推理测试...")
|
||||
outputs = ort_session.run(
|
||||
None,
|
||||
{'inputs': test_input, 'masks': test_mask}
|
||||
)
|
||||
|
||||
print(f"✓ ONNX推理成功")
|
||||
print(f" 输出数量: {len(outputs)}")
|
||||
for i, output in enumerate(outputs):
|
||||
print(f" 输出{i}形状: {output.shape}")
|
||||
|
||||
except ImportError:
|
||||
print("⚠️ 未安装onnx或onnxruntime,跳过验证")
|
||||
print(" 安装命令: pip install onnx onnxruntime")
|
||||
except Exception as e:
|
||||
print(f"⚠️ ONNX验证失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
print(" 但模型文件已保存,可以尝试手动验证")
|
||||
|
||||
# ========== 总结 ==========
|
||||
print("\n" + "="*80)
|
||||
print("✓ TimesFM 2.5模型导出完成!")
|
||||
print("="*80)
|
||||
|
||||
if os.path.exists(onnx_path):
|
||||
print(f"\n保存位置: {onnx_path}")
|
||||
print(f"文件大小: {os.path.getsize(onnx_path) / (1024**2):.2f} MB")
|
||||
|
||||
print(f"\n使用方法:")
|
||||
print(f"```python")
|
||||
print(f"import onnxruntime as ort")
|
||||
print(f"import numpy as np")
|
||||
print(f"")
|
||||
print(f"# 加载模型")
|
||||
print(f"session = ort.InferenceSession('{onnx_path}')")
|
||||
print(f"")
|
||||
print(f"# 准备输入(512小时BTC历史价格)")
|
||||
print(f"btc_prices = np.array([85000, 85100, ...]) # 512个价格点")
|
||||
print(f"inputs = btc_prices.reshape(1, -1).astype(np.float32)")
|
||||
print(f"masks = np.ones((1, 512), dtype=np.float32)")
|
||||
print(f"")
|
||||
print(f"# 推理")
|
||||
print(f"outputs = session.run(None, {{'inputs': inputs, 'masks': masks}})")
|
||||
print(f"predictions = outputs[0] # 未来24小时价格预测")
|
||||
print(f"```")
|
||||
|
||||
print("\n" + "="*80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
export_timesfm_to_onnx()
|
||||
@@ -0,0 +1,58 @@
|
||||
@echo off
|
||||
chcp 65001 >nul
|
||||
REM ============================================
|
||||
REM Windows 环境安装脚本
|
||||
REM 安装所有依赖和模型
|
||||
REM ============================================
|
||||
|
||||
echo ==========================================
|
||||
echo 加密货币预测系统 - Windows 安装脚本
|
||||
echo ==========================================
|
||||
echo.
|
||||
|
||||
REM 检查 conda
|
||||
where conda >nul 2>&1
|
||||
if %errorlevel% neq 0 (
|
||||
echo [错误] 未找到 conda
|
||||
echo 请先安装 Miniconda: https://docs.conda.io/en/latest/miniconda.html
|
||||
pause
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
echo [1/6] 创建 conda 环境...
|
||||
call conda create -n crypto_pred_312 python=3.12 -y
|
||||
if %errorlevel% neq 0 (
|
||||
echo [警告] 环境可能已存在,继续...
|
||||
)
|
||||
|
||||
echo.
|
||||
echo [2/6] 激活环境...
|
||||
call conda activate crypto_pred_312
|
||||
|
||||
echo.
|
||||
echo [3/6] 安装 PyTorch (CPU 版本)...
|
||||
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
|
||||
|
||||
echo.
|
||||
echo [4/5] 安装项目依赖...
|
||||
pip install -r requirements.txt
|
||||
|
||||
echo.
|
||||
echo [5/5] 验证安装...
|
||||
python -c "import torch; print(f'PyTorch: {torch.__version__}')"
|
||||
python -c "from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction; print('TTM-R2: 已安装')"
|
||||
python -c "import pandas_ta; print('pandas-ta: 已安装')"
|
||||
python -c "from training.config import TrainingConfig; c=TrainingConfig(); print(f'设备: {c.get_device()}')"
|
||||
|
||||
echo.
|
||||
echo ==========================================
|
||||
echo 安装完成!
|
||||
echo ==========================================
|
||||
echo.
|
||||
echo 下一步:
|
||||
echo 1. 复制 .env.example 为 .env 并配置数据库
|
||||
echo 2. 双击 start_system.bat 启动系统
|
||||
echo 3. 访问 http://localhost:8000
|
||||
echo.
|
||||
|
||||
pause
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,426 @@
|
||||
"""
|
||||
模型推理服务 - 统一管理 DLinear, N-BEATS, TTM-R2 模型加载与预测
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import pickle
|
||||
import torch
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
import onnxruntime as ort
|
||||
from typing import Dict, List, Optional, Tuple, Union, Any
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
|
||||
# 添加项目根目录到 sys.path
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
|
||||
class CustomUnpickler(pickle.Unpickler):
|
||||
"""自定义反序列化器,用于处理 __main__ 模块的类重定向"""
|
||||
def find_class(self, module, name):
|
||||
# 将 __main__ 中的类重定向到当前模块
|
||||
if module == "__main__":
|
||||
if name == "ClassificationPreprocessor":
|
||||
return ClassificationPreprocessor
|
||||
if name == "NBeatsPreprocessor":
|
||||
return NBeatsPreprocessor
|
||||
return super().find_class(module, name)
|
||||
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
|
||||
def __init__(self, lookback: int = 168, forecast_horizon: int = 1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
self.feature_columns = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
def add_features(self, df: pd.DataFrame) -> pd.DataFrame:
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df: pd.DataFrame) -> pd.DataFrame:
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df: pd.DataFrame, fit: bool = True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[self.feature_columns])
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[self.feature_columns])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
def preprocess_for_inference(self, df: pd.DataFrame):
|
||||
"""推理专用预处理:整个df一次性处理,不生成标签"""
|
||||
df = self.add_features(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
# 确保有足够的数据
|
||||
if len(df) == 0:
|
||||
return np.array([]), df, []
|
||||
|
||||
scaled_features = self.scaler.transform(df[self.feature_columns])
|
||||
|
||||
X = []
|
||||
valid_indices = []
|
||||
|
||||
if len(scaled_features) < self.lookback:
|
||||
return np.array([]), df, []
|
||||
|
||||
# 注意:这里生成的窗口数量应该是 len(scaled) - lookback + 1
|
||||
# 例如 len=168, lookback=168 -> range(1) -> i=0 -> [0:168] (1个窗口)
|
||||
for i in range(len(scaled_features) - self.lookback + 1):
|
||||
window = scaled_features[i:i + self.lookback]
|
||||
X.append(window)
|
||||
valid_indices.append(i + self.lookback - 1)
|
||||
|
||||
return np.array(X), df, valid_indices
|
||||
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback: int = 168, forecast_horizon: int = 1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df: pd.DataFrame, fit: bool = True):
|
||||
df = df.copy()
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
|
||||
X = []
|
||||
valid_indices = []
|
||||
|
||||
if len(scaled_prices) < self.lookback:
|
||||
return np.array([]), df, []
|
||||
|
||||
for i in range(len(scaled_prices) - self.lookback + 1):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
valid_indices.append(i + self.lookback - 1)
|
||||
|
||||
return np.array(X), df, valid_indices
|
||||
|
||||
|
||||
class InferenceManager:
|
||||
"""统一模型推理管理器"""
|
||||
|
||||
_instance = None
|
||||
|
||||
def __new__(cls, *args, **kwargs):
|
||||
if cls._instance is None:
|
||||
cls._instance = super(InferenceManager, cls).__new__(cls)
|
||||
cls._instance.initialized = False
|
||||
return cls._instance
|
||||
|
||||
def __init__(self, device_str: str = None):
|
||||
if self.initialized:
|
||||
return
|
||||
|
||||
import platform
|
||||
if device_str:
|
||||
self.device = torch.device(device_str)
|
||||
elif platform.system() == 'Darwin' and torch.backends.mps.is_available():
|
||||
self.device = torch.device('mps')
|
||||
else:
|
||||
self.device = torch.device('cpu')
|
||||
|
||||
print(f"InferenceManager 使用设备: {self.device}")
|
||||
|
||||
self.models = {}
|
||||
self.preprocessors = {}
|
||||
|
||||
# 配置路径
|
||||
self.configs = {
|
||||
'dlinear': {
|
||||
'model_path': './models/checkpoints/dlinear_classification_2026/model.onnx',
|
||||
'preprocessor_path': './models/checkpoints/dlinear_classification_2026/preprocessor.pkl'
|
||||
},
|
||||
'nbeats': {
|
||||
'model_path': './models/checkpoints/nbeats_btc_1h/model.onnx',
|
||||
'preprocessor_path': './models/checkpoints/nbeats_btc_1h/preprocessor.pkl'
|
||||
},
|
||||
'ttm': {
|
||||
'model_path': './models/checkpoints/ttm_r2_onnx/ttm_r2_crypto_channel_mixing.onnx'
|
||||
}
|
||||
}
|
||||
self.initialized = True
|
||||
|
||||
def load_model(self, model_name: str):
|
||||
"""加载指定模型"""
|
||||
if model_name in self.models:
|
||||
return
|
||||
|
||||
print(f"正在加载 {model_name}...")
|
||||
try:
|
||||
if model_name == 'dlinear':
|
||||
self._load_dlinear()
|
||||
elif model_name == 'nbeats':
|
||||
self._load_nbeats()
|
||||
elif model_name == 'ttm':
|
||||
self._load_ttm()
|
||||
else:
|
||||
print(f"未知模型: {model_name}")
|
||||
except Exception as e:
|
||||
print(f"加载 {model_name} 失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
def load_all_models(self):
|
||||
"""加载所有模型"""
|
||||
self.load_model('dlinear')
|
||||
self.load_model('nbeats')
|
||||
self.load_model('ttm')
|
||||
|
||||
def _load_dlinear(self):
|
||||
config = self.configs['dlinear']
|
||||
if not os.path.exists(config['model_path']):
|
||||
print(f"DLinear 模型文件不存在: {config['model_path']}")
|
||||
return
|
||||
|
||||
try:
|
||||
self.models['dlinear'] = ort.InferenceSession(config['model_path'])
|
||||
|
||||
with open(config['preprocessor_path'], 'rb') as f:
|
||||
try:
|
||||
self.preprocessors['dlinear'] = pickle.load(f)
|
||||
except AttributeError:
|
||||
f.seek(0)
|
||||
self.preprocessors['dlinear'] = CustomUnpickler(f).load()
|
||||
print(" ✓ DLinear (ONNX) 已加载")
|
||||
except Exception as e:
|
||||
print(f"DLinear 加载失败: {e}")
|
||||
|
||||
def _load_nbeats(self):
|
||||
config = self.configs['nbeats']
|
||||
if not os.path.exists(config['model_path']):
|
||||
print(f"N-BEATS 模型文件不存在: {config['model_path']}")
|
||||
return
|
||||
|
||||
try:
|
||||
self.models['nbeats'] = ort.InferenceSession(config['model_path'])
|
||||
with open(config['preprocessor_path'], 'rb') as f:
|
||||
try:
|
||||
self.preprocessors['nbeats'] = pickle.load(f)
|
||||
except AttributeError:
|
||||
f.seek(0)
|
||||
self.preprocessors['nbeats'] = CustomUnpickler(f).load()
|
||||
print(" ✓ N-BEATS (ONNX) 已加载")
|
||||
except Exception as e:
|
||||
print(f"N-BEATS 加载失败: {e}")
|
||||
|
||||
def _load_ttm(self):
|
||||
config = self.configs['ttm']
|
||||
try:
|
||||
self.models['ttm'] = ort.InferenceSession(config['model_path'])
|
||||
print(" ✓ TTM-R2 (ONNX) 已加载")
|
||||
except Exception as e:
|
||||
print(f"TTM 加载失败: {e}")
|
||||
|
||||
def predict_dlinear(self, df: pd.DataFrame) -> Tuple[Optional[int], Optional[float]]:
|
||||
"""
|
||||
DLinear预测 (输入df包含历史数据,预测下一时刻)
|
||||
"""
|
||||
if 'dlinear' not in self.models:
|
||||
self.load_model('dlinear')
|
||||
if 'dlinear' not in self.models:
|
||||
return None, None
|
||||
|
||||
session = self.models['dlinear']
|
||||
preprocessor = self.preprocessors['dlinear']
|
||||
|
||||
# 预处理
|
||||
X, _, _ = preprocessor.preprocess_for_inference(df)
|
||||
|
||||
if len(X) == 0:
|
||||
return None, None
|
||||
|
||||
# 取最后一个窗口
|
||||
# ONNX 需要 float32
|
||||
X_last = X[-1:].astype(np.float32)
|
||||
|
||||
try:
|
||||
input_name = session.get_inputs()[0].name
|
||||
output_name = session.get_outputs()[0].name
|
||||
|
||||
logits = session.run([output_name], {input_name: X_last})[0]
|
||||
prob = 1 / (1 + np.exp(-logits))
|
||||
prediction = int(prob.item() > 0.5)
|
||||
probability = float(prob.item())
|
||||
|
||||
return prediction, probability
|
||||
except Exception as e:
|
||||
print(f"DLinear 预测出错: {e}")
|
||||
return None, None
|
||||
|
||||
def predict_nbeats(self, df: pd.DataFrame) -> Tuple[Optional[int], Optional[float]]:
|
||||
"""
|
||||
N-BEATS预测
|
||||
"""
|
||||
if 'nbeats' not in self.models:
|
||||
self.load_model('nbeats')
|
||||
if 'nbeats' not in self.models:
|
||||
return None, None
|
||||
|
||||
session = self.models['nbeats']
|
||||
preprocessor = self.preprocessors['nbeats']
|
||||
|
||||
# 预处理
|
||||
X, _, _ = preprocessor.preprocess(df, fit=False)
|
||||
|
||||
if len(X) == 0:
|
||||
return None, None
|
||||
|
||||
# 取最后一个窗口
|
||||
X_last = X[-1:].astype(np.float32)
|
||||
current_price_scaled = X[-1, -1] # 窗口中最后一个点的缩放价格
|
||||
|
||||
try:
|
||||
input_name = session.get_inputs()[0].name
|
||||
output_name = session.get_outputs()[0].name
|
||||
|
||||
output = session.run([output_name], {input_name: X_last})[0]
|
||||
|
||||
# output shape: (batch, forecast_horizon) or (batch, 1)
|
||||
if output.ndim == 1:
|
||||
pred_price_scaled = output[0]
|
||||
else:
|
||||
pred_price_scaled = output[0, 0]
|
||||
|
||||
prediction = 1 if pred_price_scaled > current_price_scaled else 0
|
||||
|
||||
# N-BEATS 回归模型没有直接概率
|
||||
probability = 0.6 if prediction == 1 else 0.4 # 占位
|
||||
|
||||
return prediction, probability
|
||||
except Exception as e:
|
||||
print(f"N-BEATS 预测出错: {e}")
|
||||
return None, None
|
||||
|
||||
def predict_ttm(self, df: pd.DataFrame) -> Tuple[Optional[int], Optional[float]]:
|
||||
"""
|
||||
TTM-R2预测
|
||||
"""
|
||||
if 'ttm' not in self.models:
|
||||
self.load_model('ttm')
|
||||
if 'ttm' not in self.models:
|
||||
return None, None
|
||||
|
||||
session = self.models['ttm']
|
||||
context_length = 512
|
||||
|
||||
all_prices = df['close'].values
|
||||
if len(all_prices) == 0:
|
||||
return None, None
|
||||
|
||||
# TTM 上下文准备
|
||||
if len(all_prices) >= context_length:
|
||||
context_data = all_prices[-context_length:].reshape(1, -1, 1)
|
||||
else:
|
||||
padded = np.full(context_length, all_prices[-1])
|
||||
padded[-len(all_prices):] = all_prices
|
||||
context_data = padded.reshape(1, -1, 1)
|
||||
|
||||
# ONNX input
|
||||
context_data = context_data.astype(np.float32)
|
||||
base_price = float(df['close'].iloc[-1])
|
||||
|
||||
try:
|
||||
input_name = session.get_inputs()[0].name
|
||||
output_name = session.get_outputs()[0].name
|
||||
|
||||
output = session.run([output_name], {input_name: context_data})[0]
|
||||
# output shape might be (batch, prediction_length, features)
|
||||
# prediction_length=96 usually for TTM zero-shot
|
||||
# We want the first prediction step [0, 0, 0]
|
||||
pred_price = output[0, 0, 0]
|
||||
|
||||
prediction = 1 if pred_price > base_price else 0
|
||||
|
||||
# 简单的概率模拟
|
||||
price_pct = (pred_price - base_price) / base_price
|
||||
probability = 1 / (1 + np.exp(-price_pct * 100))
|
||||
|
||||
return prediction, float(probability)
|
||||
except Exception as e:
|
||||
print(f"TTM 预测出错: {e}")
|
||||
return None, None
|
||||
|
||||
def predict_all(self, df: pd.DataFrame) -> Dict[str, Any]:
|
||||
"""
|
||||
使用所有加载的模型进行预测
|
||||
"""
|
||||
results = {}
|
||||
|
||||
# DLinear
|
||||
dl_pred, dl_prob = self.predict_dlinear(df)
|
||||
if dl_pred is not None:
|
||||
results['dlinear'] = {'prediction': dl_pred, 'probability': dl_prob}
|
||||
|
||||
# N-BEATS
|
||||
nb_pred, nb_prob = self.predict_nbeats(df)
|
||||
if nb_pred is not None:
|
||||
results['nbeats'] = {'prediction': nb_pred, 'probability': nb_prob}
|
||||
|
||||
# TTM
|
||||
ttm_pred, ttm_prob = self.predict_ttm(df)
|
||||
if ttm_pred is not None:
|
||||
results['ttm'] = {'prediction': ttm_pred, 'probability': ttm_prob}
|
||||
|
||||
# 集成 (投票)
|
||||
votes = []
|
||||
if dl_pred is not None: votes.append(dl_pred)
|
||||
if nb_pred is not None: votes.append(nb_pred)
|
||||
if ttm_pred is not None: votes.append(ttm_pred)
|
||||
|
||||
if votes:
|
||||
# 简单投票: > 50% 胜出
|
||||
vote_sum = sum(votes)
|
||||
if vote_sum > len(votes) / 2:
|
||||
ensemble_pred = 1
|
||||
elif vote_sum < len(votes) / 2:
|
||||
ensemble_pred = 0
|
||||
else:
|
||||
# 平票时,如果有 DLinear,优先 DLinear,否则 0 (保守)
|
||||
ensemble_pred = dl_pred if dl_pred is not None else 0
|
||||
|
||||
results['ensemble'] = {'prediction': ensemble_pred, 'votes': votes}
|
||||
else:
|
||||
results['ensemble'] = None
|
||||
|
||||
return results
|
||||
|
||||
def get_inference_manager() -> InferenceManager:
|
||||
"""获取单例"""
|
||||
return InferenceManager()
|
||||
@@ -0,0 +1,371 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
MySQL 数据定时备份到 Cloudflare D1
|
||||
- 定时:每小时 XX:30
|
||||
- 模式:按游标列增量同步(timestamp/created_at/updated_at/id)
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
import schedule
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from api.models import get_db_connection
|
||||
from utils.project_logger import get_project_logger
|
||||
|
||||
load_dotenv()
|
||||
logger = get_project_logger("mysql_d1_backup")
|
||||
|
||||
|
||||
class MySQLD1BackupService:
|
||||
"""MySQL -> D1 备份服务。"""
|
||||
|
||||
def __init__(self):
|
||||
self.enabled = os.getenv("D1_BACKUP_ENABLED", "false").lower() == "true"
|
||||
self.api_token = os.getenv("CF_D1_API_TOKEN", "")
|
||||
self.account_id = os.getenv("CF_D1_ACCOUNT_ID", "")
|
||||
self.database_id = os.getenv("CF_D1_DATABASE_ID", "")
|
||||
self.batch_size = int(os.getenv("D1_BACKUP_BATCH_SIZE", "500"))
|
||||
self.max_sql_params = int(os.getenv("D1_MAX_SQL_PARAMS", "90"))
|
||||
self.tables_config = os.getenv("D1_BACKUP_TABLES", "").strip()
|
||||
self._job_lock = threading.Lock()
|
||||
|
||||
def is_config_ready(self) -> bool:
|
||||
"""检查 D1 配置是否完整。"""
|
||||
if not self.enabled:
|
||||
return False
|
||||
return bool(self.api_token and self.account_id and self.database_id)
|
||||
|
||||
def _d1_endpoint(self) -> str:
|
||||
return (
|
||||
f"https://api.cloudflare.com/client/v4/accounts/"
|
||||
f"{self.account_id}/d1/database/{self.database_id}/query"
|
||||
)
|
||||
|
||||
def _query_d1(self, sql: str, params: list[Any] | None = None) -> dict:
|
||||
"""执行 D1 SQL。"""
|
||||
headers = {
|
||||
"Authorization": f"Bearer {self.api_token}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
statement: dict[str, Any] = {"sql": sql}
|
||||
if params is not None:
|
||||
statement["params"] = params
|
||||
|
||||
with httpx.Client(timeout=30) as client:
|
||||
# Cloudflare D1 REST API 对 /query 更稳定的格式是语句数组
|
||||
resp = client.post(self._d1_endpoint(), headers=headers, json=[statement])
|
||||
if resp.status_code >= 400:
|
||||
# 兼容部分环境下使用对象格式的情况,失败时回退重试一次
|
||||
fallback_resp = client.post(self._d1_endpoint(), headers=headers, json=statement)
|
||||
if fallback_resp.status_code >= 400:
|
||||
sql_preview = " ".join(sql.strip().split())[:180]
|
||||
raise RuntimeError(
|
||||
f"D1 HTTP {fallback_resp.status_code},sql={sql_preview},响应={fallback_resp.text[:500]}"
|
||||
)
|
||||
data = fallback_resp.json()
|
||||
else:
|
||||
data = resp.json()
|
||||
|
||||
if not data.get("success", False):
|
||||
raise RuntimeError(f"D1 SQL 执行失败: {data}")
|
||||
return data
|
||||
|
||||
def _d1_results(self, response: dict) -> list[dict]:
|
||||
"""解析 D1 响应结果。"""
|
||||
result = response.get("result", [])
|
||||
if not result:
|
||||
return []
|
||||
first = result[0] if isinstance(result, list) else result
|
||||
return first.get("results", []) if isinstance(first, dict) else []
|
||||
|
||||
def _normalize_value(self, value: Any) -> Any:
|
||||
"""把 MySQL 字段值转换为可写入 D1 的类型。"""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, (datetime,)):
|
||||
return value.strftime("%Y-%m-%d %H:%M:%S")
|
||||
if isinstance(value, Decimal):
|
||||
return float(value)
|
||||
if isinstance(value, bytes):
|
||||
return value.hex()
|
||||
if isinstance(value, (dict, list)):
|
||||
return json.dumps(value, ensure_ascii=False)
|
||||
return value
|
||||
|
||||
def _sqlite_type_from_mysql(self, mysql_type: str) -> str:
|
||||
t = mysql_type.lower()
|
||||
if any(x in t for x in ["int", "bit", "bool"]):
|
||||
return "INTEGER"
|
||||
if any(x in t for x in ["double", "float", "decimal", "numeric", "real"]):
|
||||
return "REAL"
|
||||
if any(x in t for x in ["blob", "binary", "varbinary"]):
|
||||
return "BLOB"
|
||||
return "TEXT"
|
||||
|
||||
def _get_mysql_tables(self) -> list[str]:
|
||||
"""获取待备份表。"""
|
||||
if self.tables_config:
|
||||
return [x.strip() for x in self.tables_config.split(",") if x.strip()]
|
||||
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("SHOW TABLES")
|
||||
rows = cursor.fetchall()
|
||||
|
||||
tables: list[str] = []
|
||||
for row in rows:
|
||||
# DictCursor 下 SHOW TABLES 返回单键字典
|
||||
table_name = list(row.values())[0]
|
||||
tables.append(str(table_name))
|
||||
return tables
|
||||
|
||||
def _get_mysql_columns(self, table_name: str) -> list[dict]:
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
cursor.execute(f"SHOW COLUMNS FROM `{table_name}`")
|
||||
return list(cursor.fetchall())
|
||||
|
||||
def _guess_cursor_column(self, columns: list[dict]) -> str | None:
|
||||
"""选择增量同步游标列。"""
|
||||
column_names = {c["Field"] for c in columns}
|
||||
for candidate in ["timestamp", "created_at", "updated_at", "id"]:
|
||||
if candidate in column_names:
|
||||
return candidate
|
||||
return None
|
||||
|
||||
def _ensure_d1_meta_tables(self):
|
||||
"""创建 D1 元数据表。"""
|
||||
self._query_d1(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS backup_sync_state (
|
||||
table_name TEXT PRIMARY KEY,
|
||||
cursor_col TEXT,
|
||||
last_cursor TEXT,
|
||||
updated_at TEXT
|
||||
)
|
||||
"""
|
||||
)
|
||||
self._query_d1(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS backup_run_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
run_at TEXT,
|
||||
table_name TEXT,
|
||||
synced_rows INTEGER,
|
||||
status TEXT,
|
||||
message TEXT
|
||||
)
|
||||
"""
|
||||
)
|
||||
|
||||
def _ensure_d1_table(self, table_name: str, columns: list[dict]):
|
||||
"""创建 D1 备份表。"""
|
||||
table_sql_name = f"backup_{table_name}"
|
||||
col_defs: list[str] = []
|
||||
primary_keys: list[str] = []
|
||||
|
||||
for col in columns:
|
||||
field = col["Field"]
|
||||
col_type = self._sqlite_type_from_mysql(col["Type"])
|
||||
col_defs.append(f'"{field}" {col_type}')
|
||||
if col.get("Key") == "PRI":
|
||||
primary_keys.append(field)
|
||||
|
||||
# SQLite/D1 对复合主键需要表级 PRIMARY KEY 约束,不能多列各自 PRIMARY KEY
|
||||
if primary_keys:
|
||||
pk_cols = ", ".join([f'"{k}"' for k in primary_keys])
|
||||
col_defs.append(f"PRIMARY KEY ({pk_cols})")
|
||||
|
||||
create_sql = f"CREATE TABLE IF NOT EXISTS \"{table_sql_name}\" ({', '.join(col_defs)})"
|
||||
self._query_d1(create_sql)
|
||||
|
||||
def _get_last_cursor(self, table_name: str) -> str | None:
|
||||
"""读取 D1 已同步游标。"""
|
||||
resp = self._query_d1(
|
||||
"SELECT last_cursor FROM backup_sync_state WHERE table_name = ?",
|
||||
[table_name],
|
||||
)
|
||||
rows = self._d1_results(resp)
|
||||
if not rows:
|
||||
return None
|
||||
return rows[0].get("last_cursor")
|
||||
|
||||
def _update_last_cursor(self, table_name: str, cursor_col: str, cursor_value: Any):
|
||||
"""更新 D1 已同步游标。"""
|
||||
now_str = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
self._query_d1(
|
||||
"""
|
||||
INSERT INTO backup_sync_state(table_name, cursor_col, last_cursor, updated_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(table_name) DO UPDATE SET
|
||||
cursor_col = excluded.cursor_col,
|
||||
last_cursor = excluded.last_cursor,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
[table_name, cursor_col, str(cursor_value), now_str],
|
||||
)
|
||||
|
||||
def _log_run(self, table_name: str, synced_rows: int, status: str, message: str):
|
||||
"""记录备份执行日志。"""
|
||||
run_at = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
self._query_d1(
|
||||
"INSERT INTO backup_run_log(run_at, table_name, synced_rows, status, message) VALUES (?, ?, ?, ?, ?)",
|
||||
[run_at, table_name, synced_rows, status, message[:1000]],
|
||||
)
|
||||
|
||||
def _fetch_mysql_rows(
|
||||
self,
|
||||
table_name: str,
|
||||
cursor_col: str | None,
|
||||
last_cursor: str | None,
|
||||
) -> list[dict]:
|
||||
"""按游标从 MySQL 抽取增量数据。"""
|
||||
with get_db_connection() as conn:
|
||||
cursor = conn.cursor()
|
||||
if cursor_col and last_cursor:
|
||||
sql = (
|
||||
f"SELECT * FROM `{table_name}` WHERE `{cursor_col}` > %s "
|
||||
f"ORDER BY `{cursor_col}` ASC LIMIT %s"
|
||||
)
|
||||
cursor.execute(sql, (last_cursor, self.batch_size))
|
||||
elif cursor_col:
|
||||
sql = f"SELECT * FROM `{table_name}` ORDER BY `{cursor_col}` ASC LIMIT %s"
|
||||
cursor.execute(sql, (self.batch_size,))
|
||||
else:
|
||||
sql = f"SELECT * FROM `{table_name}` LIMIT %s"
|
||||
cursor.execute(sql, (self.batch_size,))
|
||||
return list(cursor.fetchall())
|
||||
|
||||
def _upsert_rows_to_d1(self, table_name: str, columns: list[dict], rows: list[dict]):
|
||||
"""把数据批量 UPSERT 到 D1。"""
|
||||
if not rows:
|
||||
return
|
||||
|
||||
table_sql_name = f"backup_{table_name}"
|
||||
col_names = [c["Field"] for c in columns]
|
||||
col_sql = ", ".join([f'"{c}"' for c in col_names])
|
||||
|
||||
# 避免 D1 的 SQL 变量数量限制触发 400
|
||||
# 通过环境变量 D1_MAX_SQL_PARAMS 控制,默认保守值 90
|
||||
chunk_size = max(1, self.max_sql_params // max(1, len(col_names)))
|
||||
|
||||
for i in range(0, len(rows), chunk_size):
|
||||
chunk_rows = rows[i:i + chunk_size]
|
||||
values_clause: list[str] = []
|
||||
params: list[Any] = []
|
||||
for row in chunk_rows:
|
||||
placeholders = ", ".join(["?" for _ in col_names])
|
||||
values_clause.append(f"({placeholders})")
|
||||
for col in col_names:
|
||||
params.append(self._normalize_value(row.get(col)))
|
||||
|
||||
sql = (
|
||||
f"INSERT OR REPLACE INTO \"{table_sql_name}\" ({col_sql}) "
|
||||
f"VALUES {', '.join(values_clause)}"
|
||||
)
|
||||
self._query_d1(sql, params)
|
||||
|
||||
def backup_table_once(self, table_name: str) -> int:
|
||||
"""单表执行一次增量备份,返回同步行数。"""
|
||||
columns = self._get_mysql_columns(table_name)
|
||||
if not columns:
|
||||
return 0
|
||||
|
||||
cursor_col = self._guess_cursor_column(columns)
|
||||
self._ensure_d1_table(table_name, columns)
|
||||
|
||||
total_synced = 0
|
||||
last_cursor = self._get_last_cursor(table_name)
|
||||
|
||||
while True:
|
||||
rows = self._fetch_mysql_rows(table_name, cursor_col, last_cursor)
|
||||
if not rows:
|
||||
break
|
||||
|
||||
self._upsert_rows_to_d1(table_name, columns, rows)
|
||||
total_synced += len(rows)
|
||||
|
||||
if cursor_col:
|
||||
last_cursor = self._normalize_value(rows[-1].get(cursor_col))
|
||||
self._update_last_cursor(table_name, cursor_col, last_cursor)
|
||||
else:
|
||||
# 无游标列时仅做一批,避免重复全量
|
||||
break
|
||||
|
||||
if len(rows) < self.batch_size:
|
||||
break
|
||||
|
||||
return total_synced
|
||||
|
||||
def run_backup_once(self):
|
||||
"""执行一次全表备份任务。"""
|
||||
if not self.is_config_ready():
|
||||
logger.warning("D1 备份未启用或配置不完整,跳过本次任务")
|
||||
return
|
||||
|
||||
if not self._job_lock.acquire(blocking=False):
|
||||
logger.warning("上一次 D1 备份尚未完成,跳过本次任务")
|
||||
return
|
||||
|
||||
try:
|
||||
self._ensure_d1_meta_tables()
|
||||
tables = self._get_mysql_tables()
|
||||
logger.info("D1 备份任务开始,待同步表: %s", ", ".join(tables))
|
||||
|
||||
for table_name in tables:
|
||||
try:
|
||||
synced = self.backup_table_once(table_name)
|
||||
logger.info("表 %s 同步完成,本次同步 %d 行", table_name, synced)
|
||||
self._log_run(table_name, synced, "success", "ok")
|
||||
except Exception as table_error:
|
||||
logger.exception("表 %s 同步失败", table_name)
|
||||
self._log_run(table_name, 0, "failed", str(table_error))
|
||||
except Exception:
|
||||
logger.exception("D1 备份任务执行失败")
|
||||
finally:
|
||||
self._job_lock.release()
|
||||
|
||||
|
||||
def schedule_mysql_d1_backup():
|
||||
"""启动 D1 备份定时任务(每小时 :30)。"""
|
||||
service = MySQLD1BackupService()
|
||||
if not service.is_config_ready():
|
||||
logger.warning("D1 备份调度未启动:请检查 D1_BACKUP_ENABLED 与 Cloudflare 参数")
|
||||
return
|
||||
|
||||
# 启动后先执行一次,避免首次备份必须等待到下一个 :30
|
||||
service.run_backup_once()
|
||||
|
||||
# 使用独立调度器,避免与项目内其他定时任务共享全局调度状态
|
||||
local_scheduler = schedule.Scheduler()
|
||||
local_scheduler.every().hour.at(":30").do(service.run_backup_once)
|
||||
logger.info("MySQL -> D1 定时备份已启动(每小时 :30 执行)")
|
||||
|
||||
while True:
|
||||
local_scheduler.run_pending()
|
||||
time.sleep(1)
|
||||
|
||||
|
||||
def run_mysql_d1_backup_scheduler_background():
|
||||
"""后台线程启动 D1 备份调度器。"""
|
||||
service = MySQLD1BackupService()
|
||||
if not service.is_config_ready():
|
||||
logger.warning("D1 备份配置不完整,后台线程不启动")
|
||||
return None
|
||||
|
||||
scheduler_thread = threading.Thread(target=schedule_mysql_d1_backup, daemon=True)
|
||||
scheduler_thread.start()
|
||||
logger.info("D1 备份调度器已在后台启动")
|
||||
return scheduler_thread
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
schedule_mysql_d1_backup()
|
||||
+52
-28
@@ -1,45 +1,69 @@
|
||||
# Core dependencies
|
||||
torch>=2.0.0
|
||||
torchvision
|
||||
torchaudio
|
||||
python-binance>=1.0.19
|
||||
# ==============================================
|
||||
# 加密货币预测系统依赖
|
||||
# Python 3.12 | 支持 macOS (MPS) 和 Windows (CPU)
|
||||
# ==============================================
|
||||
|
||||
# ----------------------------------------------
|
||||
# PyTorch (根据平台单独安装)
|
||||
# ----------------------------------------------
|
||||
# macOS: pip install torch torchvision torchaudio
|
||||
# Windows CPU: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
|
||||
# 注意:requirements.txt 中不包含 torch,需要手动安装
|
||||
|
||||
# ----------------------------------------------
|
||||
# 数据处理
|
||||
# ----------------------------------------------
|
||||
pandas>=2.0.0
|
||||
numpy>=1.24.0
|
||||
scikit-learn>=1.3.0
|
||||
pytz
|
||||
python-dotenv>=1.0.0
|
||||
|
||||
# ONNX
|
||||
onnx>=1.14.0
|
||||
onnxruntime>=1.15.0
|
||||
# ----------------------------------------------
|
||||
# 时序模型
|
||||
# ----------------------------------------------
|
||||
# IBM Granite TTM-R2 (导入名: tsfm_public)
|
||||
granite-tsfm>=0.3.0
|
||||
|
||||
# Model-specific dependencies
|
||||
darts>=0.27.0
|
||||
u8darts[torch]>=0.27.0
|
||||
tsfm_public>=0.2.0
|
||||
gluonts[torch]>=0.14.0
|
||||
# ----------------------------------------------
|
||||
# 技术指标
|
||||
# ----------------------------------------------
|
||||
pandas-ta>=0.3.14b
|
||||
|
||||
# API dependencies
|
||||
# ----------------------------------------------
|
||||
# API 服务
|
||||
# ----------------------------------------------
|
||||
fastapi>=0.104.0
|
||||
uvicorn[standard]>=0.24.0
|
||||
pydantic>=2.0.0
|
||||
python-multipart>=0.0.6
|
||||
|
||||
# Technical indicators
|
||||
pandas-ta>=0.3.14b
|
||||
# ----------------------------------------------
|
||||
# HTTP 客户端
|
||||
# ----------------------------------------------
|
||||
httpx>=0.25.0
|
||||
|
||||
# Development tools
|
||||
jupyter>=1.0.0
|
||||
matplotlib>=3.7.0
|
||||
seaborn>=0.12.0
|
||||
pytest>=7.4.0
|
||||
black>=23.0.0
|
||||
flake8>=6.0.0
|
||||
# ----------------------------------------------
|
||||
# 数据源
|
||||
# ----------------------------------------------
|
||||
python-binance>=1.0.19
|
||||
|
||||
# Utilities
|
||||
tqdm>=4.66.0
|
||||
# ----------------------------------------------
|
||||
# 任务调度
|
||||
# ----------------------------------------------
|
||||
schedule>=1.2.0
|
||||
|
||||
# Database
|
||||
# ----------------------------------------------
|
||||
# 数据库
|
||||
# ----------------------------------------------
|
||||
pymysql>=1.1.0
|
||||
sqlalchemy>=2.0.0
|
||||
cryptography>=41.0.0
|
||||
|
||||
# ----------------------------------------------
|
||||
# 环境变量
|
||||
# ----------------------------------------------
|
||||
python-dotenv>=1.0.0
|
||||
|
||||
# ----------------------------------------------
|
||||
# 进度条
|
||||
# ----------------------------------------------
|
||||
tqdm>=4.66.0
|
||||
@@ -1,458 +0,0 @@
|
||||
"""保存所有测试结果到项目 - 生成完整报告"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import pickle
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from sklearn.metrics import confusion_matrix
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
import json
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df = df[:-self.forecast_horizon].reset_index(drop=True)
|
||||
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
future_prices = df['future_price'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
y.append(scaled_future[i + self.lookback, 0])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
def inverse_transform_price(self, scaled_price):
|
||||
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
|
||||
|
||||
|
||||
def main():
|
||||
print("="*80)
|
||||
print("保存完整测试结果到项目")
|
||||
print("="*80)
|
||||
|
||||
# 创建结果目录
|
||||
results_dir = "./results"
|
||||
os.makedirs(results_dir, exist_ok=True)
|
||||
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
|
||||
# ========== 加载模型 ==========
|
||||
print("\n[1/5] 加载模型...")
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
|
||||
from models.dlinear.model import DLinear
|
||||
dlinear_checkpoint = torch.load(
|
||||
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
dlinear_model = DLinear(dlinear_checkpoint['config'])
|
||||
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
|
||||
dlinear_model = dlinear_model.to(device)
|
||||
dlinear_model.eval()
|
||||
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
|
||||
from models.nbeats.model import NBeats
|
||||
nbeats_checkpoint = torch.load(
|
||||
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
nbeats_model = NBeats(nbeats_checkpoint['config'])
|
||||
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
|
||||
nbeats_model = nbeats_model.to(device)
|
||||
nbeats_model.eval()
|
||||
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
|
||||
"ibm-granite/granite-timeseries-ttm-r2",
|
||||
trust_remote_code=True
|
||||
)
|
||||
ttm_model = ttm_model.to(device)
|
||||
ttm_model.eval()
|
||||
|
||||
print("✓ 三个模型已加载")
|
||||
|
||||
# ========== 获取数据 ==========
|
||||
print("\n[2/5] 获取2026年1月测试数据...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
|
||||
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
|
||||
end_date='2026-01-30', use_cache=False)
|
||||
|
||||
print(f"✓ 数据: {len(df_2026)} 行, {df_2026['timestamp'].min()} 至 {df_2026['timestamp'].max()}")
|
||||
|
||||
# ========== 预处理和预测 ==========
|
||||
print("\n[3/5] 预处理和预测...")
|
||||
|
||||
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
|
||||
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
|
||||
|
||||
min_samples = min(len(X_dlinear), len(X_nbeats))
|
||||
X_dlinear = X_dlinear[:min_samples]
|
||||
y_dlinear = y_dlinear[:min_samples]
|
||||
X_nbeats = X_nbeats[:min_samples]
|
||||
|
||||
# DLinear预测
|
||||
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
|
||||
with torch.no_grad():
|
||||
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
|
||||
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits.flatten()))
|
||||
dlinear_preds = (dlinear_probs > 0.5).astype(int)
|
||||
|
||||
# N-BEATS预测
|
||||
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
|
||||
with torch.no_grad():
|
||||
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
|
||||
|
||||
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
|
||||
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
|
||||
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
|
||||
|
||||
# TTM-R2预测
|
||||
context_length = 512
|
||||
ttm_preds_list = []
|
||||
all_prices = df_2026['close'].values
|
||||
all_open_prices = df_2026['open'].values
|
||||
|
||||
for i in range(min_samples):
|
||||
start_idx = i
|
||||
end_idx = start_idx + context_length
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
context_data = all_prices[-context_length:].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
context_tensor = torch.FloatTensor(context_data).to(device)
|
||||
with torch.no_grad():
|
||||
output = ttm_model(context_tensor)
|
||||
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
|
||||
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
|
||||
ttm_preds = np.array(ttm_preds_list)
|
||||
|
||||
# 双模型交集
|
||||
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
|
||||
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
|
||||
two_model_preds = np.full(len(dlinear_preds), -1)
|
||||
two_model_preds[two_model_agree_up] = 1
|
||||
two_model_preds[two_model_agree_down] = 0
|
||||
|
||||
# 三模型投票
|
||||
vote_sum = dlinear_preds + nbeats_preds + ttm_preds
|
||||
three_model_preds = np.where(vote_sum >= 2, 1, 0)
|
||||
|
||||
true_labels = y_dlinear
|
||||
|
||||
print(f"✓ 预测完成,样本数: {min_samples}")
|
||||
|
||||
# ========== 生成详细结果 ==========
|
||||
print("\n[4/5] 生成详细结果...")
|
||||
|
||||
# 时间戳和日期
|
||||
timestamps = df_2026['timestamp'].iloc[168:168+min_samples].values
|
||||
timestamps_utc = pd.to_datetime(timestamps)
|
||||
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
|
||||
dates = timestamps_et.date
|
||||
|
||||
# 创建详细结果DataFrame
|
||||
results_df = pd.DataFrame({
|
||||
'timestamp_utc': timestamps_utc,
|
||||
'timestamp_et': timestamps_et,
|
||||
'date': dates,
|
||||
'true_label': true_labels,
|
||||
'dlinear_prob': dlinear_probs,
|
||||
'dlinear_pred': dlinear_preds,
|
||||
'nbeats_pred': nbeats_preds,
|
||||
'ttm_pred': ttm_preds,
|
||||
'two_model_pred': two_model_preds,
|
||||
'three_model_pred': three_model_preds,
|
||||
'dlinear_correct': dlinear_preds == true_labels,
|
||||
'two_model_correct': (two_model_preds == true_labels) & (two_model_preds != -1),
|
||||
'three_model_correct': three_model_preds == true_labels
|
||||
})
|
||||
|
||||
# 按天统计
|
||||
daily_stats = results_df.groupby('date').agg({
|
||||
'true_label': 'count',
|
||||
'two_model_pred': lambda x: (x != -1).sum(),
|
||||
'two_model_correct': 'sum',
|
||||
'three_model_correct': 'sum',
|
||||
'dlinear_correct': 'sum'
|
||||
})
|
||||
|
||||
daily_stats.columns = ['total_samples', 'two_model_valid', 'two_model_correct',
|
||||
'three_model_correct', 'dlinear_correct']
|
||||
|
||||
daily_stats['dlinear_accuracy'] = daily_stats['dlinear_correct'] / daily_stats['total_samples'] * 100
|
||||
daily_stats['two_model_accuracy'] = daily_stats['two_model_correct'] / daily_stats['two_model_valid'] * 100
|
||||
daily_stats['three_model_accuracy'] = daily_stats['three_model_correct'] / daily_stats['total_samples'] * 100
|
||||
daily_stats['two_model_coverage'] = daily_stats['two_model_valid'] / daily_stats['total_samples'] * 100
|
||||
|
||||
daily_stats['two_model_accuracy'] = daily_stats['two_model_accuracy'].fillna(0)
|
||||
|
||||
# ========== 保存结果 ==========
|
||||
print("\n[5/5] 保存结果文件...")
|
||||
|
||||
# 1. 保存逐小时详细结果
|
||||
hourly_file = f"{results_dir}/hourly_predictions_2026_01.csv"
|
||||
results_df.to_csv(hourly_file, index=False)
|
||||
print(f"✓ 保存逐小时预测: {hourly_file}")
|
||||
|
||||
# 2. 保存每日统计
|
||||
daily_file = f"{results_dir}/daily_statistics_2026_01.csv"
|
||||
daily_stats.to_csv(daily_file)
|
||||
print(f"✓ 保存每日统计: {daily_file}")
|
||||
|
||||
# 3. 生成总结报告
|
||||
summary = {
|
||||
"测试期间": {
|
||||
"开始日期": str(df_2026['timestamp'].min()),
|
||||
"结束日期": str(df_2026['timestamp'].max()),
|
||||
"总天数": len(daily_stats),
|
||||
"总样本数": int(min_samples)
|
||||
},
|
||||
"单模型性能": {
|
||||
"DLinear": {
|
||||
"准确率": f"{(dlinear_preds == true_labels).mean() * 100:.2f}%",
|
||||
"预测上涨比例": f"{(dlinear_preds == 1).mean() * 100:.1f}%"
|
||||
},
|
||||
"N-BEATS": {
|
||||
"准确率": f"{(nbeats_preds == true_labels).mean() * 100:.2f}%",
|
||||
"预测上涨比例": f"{(nbeats_preds == 1).mean() * 100:.1f}%"
|
||||
},
|
||||
"TTM-R2": {
|
||||
"准确率": f"{(ttm_preds == true_labels).mean() * 100:.2f}%",
|
||||
"预测上涨比例": f"{(ttm_preds == 1).mean() * 100:.1f}%"
|
||||
}
|
||||
},
|
||||
"集成策略性能": {
|
||||
"双模型交集": {
|
||||
"准确率": f"{((two_model_preds == true_labels) & (two_model_preds != -1)).sum() / (two_model_preds != -1).sum() * 100:.2f}%",
|
||||
"覆盖率": f"{(two_model_preds != -1).mean() * 100:.2f}%",
|
||||
"有效预测数": int((two_model_preds != -1).sum())
|
||||
},
|
||||
"三模型投票": {
|
||||
"准确率": f"{(three_model_preds == true_labels).mean() * 100:.2f}%",
|
||||
"覆盖率": "100.00%",
|
||||
"有效预测数": int(len(three_model_preds))
|
||||
}
|
||||
},
|
||||
"DLinear概率分布": {
|
||||
"0.4-0.5区间": f"{((dlinear_probs >= 0.4) & (dlinear_probs < 0.5)).sum()}个 ({((dlinear_probs >= 0.4) & (dlinear_probs < 0.5)).mean() * 100:.1f}%)",
|
||||
"0.5-0.6区间": f"{((dlinear_probs >= 0.5) & (dlinear_probs < 0.6)).sum()}个 ({((dlinear_probs >= 0.5) & (dlinear_probs < 0.6)).mean() * 100:.1f}%)",
|
||||
"0.6-0.7区间": f"{((dlinear_probs >= 0.6) & (dlinear_probs < 0.7)).sum()}个 ({((dlinear_probs >= 0.6) & (dlinear_probs < 0.7)).mean() * 100:.1f}%)",
|
||||
"平均概率": f"{dlinear_probs.mean():.3f}"
|
||||
},
|
||||
"最佳/最差交易日": {
|
||||
"双模型最佳日": {
|
||||
"日期": str(daily_stats['two_model_accuracy'].idxmax()),
|
||||
"准确率": f"{daily_stats['two_model_accuracy'].max():.1f}%"
|
||||
},
|
||||
"双模型最差日": {
|
||||
"日期": str(daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].idxmin()),
|
||||
"准确率": f"{daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].min():.1f}%"
|
||||
},
|
||||
"三模型最佳日": {
|
||||
"日期": str(daily_stats['three_model_accuracy'].idxmax()),
|
||||
"准确率": f"{daily_stats['three_model_accuracy'].max():.1f}%"
|
||||
},
|
||||
"三模型最差日": {
|
||||
"日期": str(daily_stats['three_model_accuracy'].idxmin()),
|
||||
"准确率": f"{daily_stats['three_model_accuracy'].min():.1f}%"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
summary_file = f"{results_dir}/summary_report_2026_01.json"
|
||||
with open(summary_file, 'w', encoding='utf-8') as f:
|
||||
json.dump(summary, f, indent=2, ensure_ascii=False)
|
||||
print(f"✓ 保存总结报告: {summary_file}")
|
||||
|
||||
# 4. 生成Markdown报告
|
||||
markdown_report = f"""# 加密货币价格预测模型测试报告
|
||||
|
||||
## 测试概况
|
||||
|
||||
- **测试期间**: 2026年1月1日 - 2026年1月30日
|
||||
- **总天数**: {len(daily_stats)}天
|
||||
- **总样本数**: {min_samples}个小时
|
||||
- **生成时间**: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
|
||||
|
||||
---
|
||||
|
||||
## 单模型性能
|
||||
|
||||
### DLinear分类模型
|
||||
- **准确率**: {(dlinear_preds == true_labels).mean() * 100:.2f}%
|
||||
- **预测上涨比例**: {(dlinear_preds == 1).mean() * 100:.1f}%
|
||||
- **预测下跌比例**: {(dlinear_preds == 0).mean() * 100:.1f}%
|
||||
|
||||
### N-BEATS回归模型
|
||||
- **准确率**: {(nbeats_preds == true_labels).mean() * 100:.2f}%
|
||||
- **预测上涨比例**: {(nbeats_preds == 1).mean() * 100:.1f}%
|
||||
- **预测下跌比例**: {(nbeats_preds == 0).mean() * 100:.1f}%
|
||||
|
||||
### TTM-R2零样本模型
|
||||
- **准确率**: {(ttm_preds == true_labels).mean() * 100:.2f}%
|
||||
- **预测上涨比例**: {(ttm_preds == 1).mean() * 100:.1f}%
|
||||
- **预测下跌比例**: {(ttm_preds == 0).mean() * 100:.1f}%
|
||||
|
||||
---
|
||||
|
||||
## 集成策略性能
|
||||
|
||||
### 双模型交集(DLinear + N-BEATS)
|
||||
- **准确率**: {((two_model_preds == true_labels) & (two_model_preds != -1)).sum() / (two_model_preds != -1).sum() * 100:.2f}%
|
||||
- **覆盖率**: {(two_model_preds != -1).mean() * 100:.2f}%
|
||||
- **有效预测数**: {(two_model_preds != -1).sum()}次
|
||||
|
||||
### 三模型多数投票(DLinear + N-BEATS + TTM-R2)
|
||||
- **准确率**: {(three_model_preds == true_labels).mean() * 100:.2f}%
|
||||
- **覆盖率**: 100.00%
|
||||
- **有效预测数**: {len(three_model_preds)}次
|
||||
|
||||
---
|
||||
|
||||
## DLinear概率分布分析
|
||||
|
||||
| 概率区间 | 样本数 | 占比 |
|
||||
|---------|--------|------|
|
||||
| 0.4-0.5 | {((dlinear_probs >= 0.4) & (dlinear_probs < 0.5)).sum()} | {((dlinear_probs >= 0.4) & (dlinear_probs < 0.5)).mean() * 100:.1f}% |
|
||||
| 0.5-0.6 | {((dlinear_probs >= 0.5) & (dlinear_probs < 0.6)).sum()} | {((dlinear_probs >= 0.5) & (dlinear_probs < 0.6)).mean() * 100:.1f}% |
|
||||
| 0.6-0.7 | {((dlinear_probs >= 0.6) & (dlinear_probs < 0.7)).sum()} | {((dlinear_probs >= 0.6) & (dlinear_probs < 0.7)).mean() * 100:.1f}% |
|
||||
|
||||
**平均概率**: {dlinear_probs.mean():.3f}
|
||||
|
||||
---
|
||||
|
||||
## 最佳/最差交易日
|
||||
|
||||
### 双模型交集
|
||||
- **最佳日**: {daily_stats['two_model_accuracy'].idxmax()} ({daily_stats['two_model_accuracy'].max():.1f}%)
|
||||
- **最差日**: {daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].idxmin()} ({daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].min():.1f}%)
|
||||
|
||||
### 三模型投票
|
||||
- **最佳日**: {daily_stats['three_model_accuracy'].idxmax()} ({daily_stats['three_model_accuracy'].max():.1f}%)
|
||||
- **最差日**: {daily_stats['three_model_accuracy'].idxmin()} ({daily_stats['three_model_accuracy'].min():.1f}%)
|
||||
|
||||
---
|
||||
|
||||
## 结论
|
||||
|
||||
1. **最佳单模型**: DLinear ({(dlinear_preds == true_labels).mean() * 100:.2f}%)
|
||||
2. **最佳集成策略**: 双模型交集 ({((two_model_preds == true_labels) & (two_model_preds != -1)).sum() / (two_model_preds != -1).sum() * 100:.2f}%)
|
||||
3. **DLinear置信度**: 99.2%的预测集中在0.45-0.55区间,缺乏高置信度预测
|
||||
|
||||
## 数据文件
|
||||
|
||||
- 逐小时预测结果: `hourly_predictions_2026_01.csv`
|
||||
- 每日统计数据: `daily_statistics_2026_01.csv`
|
||||
- 总结报告: `summary_report_2026_01.json`
|
||||
|
||||
---
|
||||
|
||||
生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
|
||||
"""
|
||||
|
||||
markdown_file = f"{results_dir}/README_2026_01.md"
|
||||
with open(markdown_file, 'w', encoding='utf-8') as f:
|
||||
f.write(markdown_report)
|
||||
print(f"✓ 保存Markdown报告: {markdown_file}")
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✓ 所有结果已保存到 ./results/ 目录")
|
||||
print("="*80)
|
||||
print(f"\n保存的文件:")
|
||||
print(f" 1. {hourly_file}")
|
||||
print(f" 2. {daily_file}")
|
||||
print(f" 3. {summary_file}")
|
||||
print(f" 4. {markdown_file}")
|
||||
print("\n" + "="*80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,134 @@
|
||||
"""导出 DLinear, N-BEATS 和 TTM-R2 模型为 ONNX 格式。"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import torch
|
||||
import numpy as np
|
||||
from pathlib import Path
|
||||
|
||||
# 添加项目根目录到路径
|
||||
project_root = Path(__file__).parent.parent
|
||||
sys.path.insert(0, str(project_root))
|
||||
|
||||
from models.dlinear.model import DLinear
|
||||
from models.nbeats.model import NBeats
|
||||
from tsfm_public.toolkit.get_model import get_model
|
||||
|
||||
|
||||
def export_dlinear():
|
||||
print("\n" + "="*50)
|
||||
print("导出 DLinear 模型...")
|
||||
try:
|
||||
checkpoint_path = project_root / "models/checkpoints/dlinear_classification_2026/best_model.pt"
|
||||
output_path = checkpoint_path.parent / "model.onnx"
|
||||
|
||||
if not checkpoint_path.exists():
|
||||
print(f"跳过: 找不到检查点 {checkpoint_path}")
|
||||
return
|
||||
|
||||
print(f"加载模型: {checkpoint_path}")
|
||||
model = DLinear.load(str(checkpoint_path), device='cpu')
|
||||
|
||||
print(f"模型配置: lookback={model.config.lookback}, input_dim={model.config.input_dim}")
|
||||
|
||||
model.to_onnx(str(output_path))
|
||||
print(f"✓ ONNX 模型已保存到: {output_path}")
|
||||
except Exception as e:
|
||||
print(f"导出 DLinear 失败: {e}")
|
||||
|
||||
def export_nbeats():
|
||||
print("\n" + "="*50)
|
||||
print("导出 N-BEATS 模型...")
|
||||
try:
|
||||
checkpoint_path = project_root / "models/checkpoints/nbeats_btc_1h/best_model.pt"
|
||||
output_path = checkpoint_path.parent / "model.onnx"
|
||||
|
||||
if not checkpoint_path.exists():
|
||||
print(f"跳过: 找不到检查点 {checkpoint_path}")
|
||||
return
|
||||
|
||||
print(f"加载模型: {checkpoint_path}")
|
||||
checkpoint = torch.load(str(checkpoint_path), map_location='cpu')
|
||||
model = NBeats(checkpoint['config'])
|
||||
model.eval()
|
||||
|
||||
# N-BEATS input: (batch_size, input_size)
|
||||
# input_size default is 168
|
||||
input_size = checkpoint['config'].input_size
|
||||
dummy_input = torch.randn(1, input_size)
|
||||
|
||||
print(f"导出 ONNX (input_size={input_size})...")
|
||||
torch.onnx.export(
|
||||
model,
|
||||
dummy_input,
|
||||
str(output_path),
|
||||
export_params=True,
|
||||
opset_version=17,
|
||||
do_constant_folding=True,
|
||||
input_names=['input'],
|
||||
output_names=['forecast'],
|
||||
dynamic_axes={'input': {0: 'batch_size'}, 'forecast': {0: 'batch_size'}}
|
||||
)
|
||||
print(f"✓ ONNX 模型已保存到: {output_path}")
|
||||
except Exception as e:
|
||||
print(f"导出 N-BEATS 失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
def export_ttm_r2():
|
||||
print("\n" + "="*50)
|
||||
print("导出 TTM-R2 模型...")
|
||||
try:
|
||||
model_name = "ibm-granite/granite-timeseries-ttm-r2"
|
||||
output_dir = project_root / "models/checkpoints/ttm_r2_onnx"
|
||||
output_path = output_dir / "model.onnx"
|
||||
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(f"加载模型: {model_name}")
|
||||
base_model = get_model(
|
||||
model_path=model_name,
|
||||
context_length=512,
|
||||
prediction_length=96,
|
||||
)
|
||||
base_model.eval()
|
||||
|
||||
print(f"TTM Config: {base_model.config}")
|
||||
|
||||
# TTM input: (batch_size, context_length, num_features)
|
||||
# Context length 512, features 1
|
||||
dummy_input = torch.randn(1, 512, 1)
|
||||
with torch.no_grad():
|
||||
output = base_model(dummy_input)
|
||||
if hasattr(output, 'prediction_outputs'):
|
||||
test_output = output.prediction_outputs
|
||||
else:
|
||||
test_output = output
|
||||
print(f"✓ 输出形状: {test_output.shape}")
|
||||
print(f"导出 ONNX (固定 Batch Size=1)...")
|
||||
torch.onnx.export(
|
||||
base_model,
|
||||
dummy_input,
|
||||
str(output_path),
|
||||
export_params=True,
|
||||
opset_version=17, # Revert to 14 as it is standard
|
||||
do_constant_folding=True,
|
||||
input_names=['past_values'],
|
||||
output_names=['predictions'],
|
||||
# Remove dynamic axes to fix potential shape inference issues
|
||||
dynamic_axes={'past_values': {0: 'batch_size'}, 'predictions': {0: 'batch_size'}},
|
||||
verbose=False
|
||||
)
|
||||
print(f"✓ ONNX 模型已保存到: {output_path}")
|
||||
except Exception as e:
|
||||
print(f"导出 TTM-R2 失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
def main():
|
||||
export_dlinear()
|
||||
export_nbeats()
|
||||
#export_ttm_r2()
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,91 @@
|
||||
import os
|
||||
|
||||
file_path = 'trader/auto_trade_polymarket.py'
|
||||
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
lines = f.readlines()
|
||||
|
||||
new_lines = []
|
||||
inserted = False
|
||||
|
||||
# Code to insert
|
||||
insert_code = """
|
||||
# === 新增: 保存初始预测和状态 ===
|
||||
try:
|
||||
# 确定最终预测 (用于日志)
|
||||
final_pred = 0
|
||||
if 'ensemble' in all_preds and all_preds['ensemble']:
|
||||
final_pred = all_preds['ensemble']['prediction']
|
||||
elif 'dlinear' in all_preds:
|
||||
final_pred = all_preds['dlinear']['prediction']
|
||||
|
||||
et_tz = ZoneInfo('America/New_York')
|
||||
current_time_utc = pd.Timestamp(current_time)
|
||||
current_time_et = current_time_utc.tz_localize('UTC').tz_convert(et_tz)
|
||||
|
||||
# 1. 保存预测到 model_predictions
|
||||
common_data = {
|
||||
"timestamp": current_time_utc.isoformat(),
|
||||
"timestamp_et": current_time_et.isoformat(),
|
||||
"date": current_time_et.strftime('%Y-%m-%d'),
|
||||
"hour": current_time_et.hour,
|
||||
"actual_result": None,
|
||||
"is_correct": None,
|
||||
"open_price": float(df['open'].iloc[-1]),
|
||||
"close_price": None,
|
||||
}
|
||||
|
||||
def save_single_pred(name, pred_res):
|
||||
data = common_data.copy()
|
||||
data['model_name'] = name
|
||||
data['prediction'] = pred_res['prediction']
|
||||
data['probability'] = pred_res.get('probability', 0.5)
|
||||
PredictionModel.save_prediction(data)
|
||||
|
||||
if 'dlinear' in all_preds: save_single_pred("DLinear", all_preds['dlinear'])
|
||||
if 'nbeats' in all_preds: save_single_pred("N-BEATS", all_preds['nbeats'])
|
||||
if 'ttm' in all_preds: save_single_pred("TTM-R2", all_preds['ttm'])
|
||||
if 'ensemble' in all_preds and all_preds['ensemble']: save_single_pred("Ensemble", all_preds['ensemble'])
|
||||
|
||||
logger.info(f"💾 [Init] 初始预测已保存到数据库")
|
||||
|
||||
# 2. 记录当前监控状态到 trades (作为 'in_progress')
|
||||
status = 'in_progress'
|
||||
market_title = self.current_market['title'] if self.current_market else "Waiting for Market..."
|
||||
market_slug = self.current_market['slug'] if self.current_market else None
|
||||
|
||||
log_data = {
|
||||
'timestamp': datetime.now().isoformat(),
|
||||
'market_title': market_title,
|
||||
'market_slug': market_slug,
|
||||
'model_name': 'Ensemble',
|
||||
'prediction': 'UP' if final_pred == 1 else 'DOWN',
|
||||
'probability': 0.0,
|
||||
'action': 'MONITOR',
|
||||
'price': 0,
|
||||
'amount': 0,
|
||||
'size': 0,
|
||||
'status': status,
|
||||
'error_message': 'System Started - Monitoring',
|
||||
'raw_data': {}
|
||||
}
|
||||
self.db.log_trade(log_data)
|
||||
logger.info(f"💾 [Init] 初始监控状态已记录")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"保存初始状态失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
"""
|
||||
|
||||
for line in lines:
|
||||
if "def wait_for_next_hour(self)" in line and not inserted:
|
||||
new_lines.append(insert_code)
|
||||
inserted = True
|
||||
new_lines.append(line)
|
||||
|
||||
with open(file_path, 'w', encoding='utf-8') as f:
|
||||
f.writelines(new_lines)
|
||||
|
||||
print("Successfully patched trader/auto_trade_polymarket.py")
|
||||
@@ -1,451 +0,0 @@
|
||||
"""智能休息策略回测:基于历史准确率动态决定是否交易"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import torch
|
||||
import pickle
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pandas_ta as ta
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
from data.fetcher import BinanceDataFetcher
|
||||
|
||||
|
||||
# ==================== 预处理器定义 ====================
|
||||
|
||||
class ClassificationPreprocessor:
|
||||
"""DLinear分类预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def add_features(self, df):
|
||||
df = df.copy()
|
||||
df['rsi'] = ta.rsi(df['close'], length=14)
|
||||
macd = ta.macd(df['close'])
|
||||
if macd is not None and not macd.empty:
|
||||
df['macd'] = macd.iloc[:, 0]
|
||||
df['macd_hist'] = macd.iloc[:, 1]
|
||||
df['macd_signal'] = macd.iloc[:, 2]
|
||||
return df
|
||||
|
||||
def create_labels(self, df):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df['future_return'] = (df['future_price'] - df['close']) / df['close']
|
||||
df['label'] = (df['future_return'] > 0).astype(int)
|
||||
return df[:-self.forecast_horizon]
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = self.add_features(df)
|
||||
df = self.create_labels(df)
|
||||
df = df.dropna().reset_index(drop=True)
|
||||
|
||||
feature_cols = ['open', 'high', 'low', 'close', 'volume',
|
||||
'rsi', 'macd', 'macd_hist', 'macd_signal']
|
||||
|
||||
if fit:
|
||||
scaled_features = self.scaler.fit_transform(df[feature_cols])
|
||||
self.is_fitted = True
|
||||
self.feature_columns = feature_cols
|
||||
else:
|
||||
scaled_features = self.scaler.transform(df[feature_cols])
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_features) - self.lookback):
|
||||
X.append(scaled_features[i:i + self.lookback])
|
||||
y.append(df['label'].iloc[i + self.lookback])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
|
||||
class NBeatsPreprocessor:
|
||||
"""N-BEATS预处理器"""
|
||||
def __init__(self, lookback=168, forecast_horizon=1):
|
||||
self.lookback = lookback
|
||||
self.forecast_horizon = forecast_horizon
|
||||
self.price_scaler = StandardScaler()
|
||||
self.is_fitted = False
|
||||
|
||||
def preprocess(self, df, fit=True):
|
||||
df = df.copy()
|
||||
df['future_price'] = df['close'].shift(-self.forecast_horizon)
|
||||
df = df[:-self.forecast_horizon].reset_index(drop=True)
|
||||
|
||||
prices = df['close'].values.reshape(-1, 1)
|
||||
future_prices = df['future_price'].values.reshape(-1, 1)
|
||||
|
||||
if fit:
|
||||
scaled_prices = self.price_scaler.fit_transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
self.is_fitted = True
|
||||
else:
|
||||
scaled_prices = self.price_scaler.transform(prices)
|
||||
scaled_future = self.price_scaler.transform(future_prices)
|
||||
|
||||
X, y = [], []
|
||||
for i in range(len(scaled_prices) - self.lookback):
|
||||
X.append(scaled_prices[i:i + self.lookback, 0])
|
||||
y.append(scaled_future[i + self.lookback, 0])
|
||||
|
||||
return np.array(X), np.array(y), df
|
||||
|
||||
def inverse_transform_price(self, scaled_price):
|
||||
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
|
||||
|
||||
|
||||
def main():
|
||||
print("="*80)
|
||||
print("智能休息策略回测:基于历史准确率动态决定是否交易")
|
||||
print("="*80)
|
||||
print("\n策略规则:")
|
||||
print(" 1. 如果前一天准确率超过70% → 当天休息")
|
||||
print(" 2. 如果连续两天准确率超过60% → 第三天休息")
|
||||
print(" 3. 其他情况 → 正常交易")
|
||||
print("="*80)
|
||||
|
||||
# ========== 加载模型 ==========
|
||||
print("\n[1/4] 加载模型...")
|
||||
|
||||
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
|
||||
|
||||
# 加载DLinear
|
||||
from models.dlinear.model import DLinear
|
||||
dlinear_checkpoint = torch.load(
|
||||
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
dlinear_model = DLinear(dlinear_checkpoint['config'])
|
||||
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
|
||||
dlinear_model = dlinear_model.to(device)
|
||||
dlinear_model.eval()
|
||||
|
||||
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
|
||||
dlinear_preprocessor = pickle.load(f)
|
||||
|
||||
# 加载N-BEATS
|
||||
from models.nbeats.model import NBeats
|
||||
nbeats_checkpoint = torch.load(
|
||||
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
|
||||
map_location=device
|
||||
)
|
||||
nbeats_model = NBeats(nbeats_checkpoint['config'])
|
||||
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
|
||||
nbeats_model = nbeats_model.to(device)
|
||||
nbeats_model.eval()
|
||||
|
||||
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
|
||||
nbeats_preprocessor = pickle.load(f)
|
||||
|
||||
# 加载TTM-R2
|
||||
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
|
||||
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
|
||||
"ibm-granite/granite-timeseries-ttm-r2",
|
||||
trust_remote_code=True
|
||||
)
|
||||
ttm_model = ttm_model.to(device)
|
||||
ttm_model.eval()
|
||||
|
||||
print(f"✓ 三个模型已加载")
|
||||
|
||||
# ========== 获取数据 ==========
|
||||
print("\n[2/4] 获取2026年1月测试数据(1月1日-1月30日)...")
|
||||
fetcher = BinanceDataFetcher()
|
||||
|
||||
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
|
||||
end_date='2026-01-30', use_cache=False)
|
||||
|
||||
print(f"✓ 测试数据: {len(df_2026)} 行")
|
||||
print(f"✓ 时间范围: {df_2026['timestamp'].min()} 至 {df_2026['timestamp'].max()}")
|
||||
|
||||
# ========== 预处理 ==========
|
||||
print("\n[3/4] 预处理数据...")
|
||||
|
||||
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
|
||||
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
|
||||
|
||||
min_samples = min(len(X_dlinear), len(X_nbeats))
|
||||
X_dlinear = X_dlinear[:min_samples]
|
||||
y_dlinear = y_dlinear[:min_samples]
|
||||
X_nbeats = X_nbeats[:min_samples]
|
||||
y_nbeats = y_nbeats[:min_samples]
|
||||
|
||||
# ========== 预测 ==========
|
||||
print("\n[4/4] 执行预测...")
|
||||
|
||||
# DLinear预测
|
||||
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
|
||||
with torch.no_grad():
|
||||
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
|
||||
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits))
|
||||
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten()
|
||||
|
||||
# N-BEATS预测
|
||||
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
|
||||
with torch.no_grad():
|
||||
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
|
||||
|
||||
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
|
||||
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
|
||||
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
|
||||
|
||||
# TTM-R2预测
|
||||
context_length = 512
|
||||
ttm_preds_list = []
|
||||
all_prices = df_2026['close'].values
|
||||
all_open_prices = df_2026['open'].values
|
||||
|
||||
for i in range(min_samples):
|
||||
start_idx = i
|
||||
end_idx = start_idx + context_length
|
||||
|
||||
if end_idx > len(all_prices):
|
||||
context_data = all_prices[-context_length:].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
|
||||
else:
|
||||
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
|
||||
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
|
||||
|
||||
context_tensor = torch.FloatTensor(context_data).to(device)
|
||||
|
||||
with torch.no_grad():
|
||||
output = ttm_model(context_tensor)
|
||||
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
|
||||
|
||||
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
|
||||
|
||||
ttm_preds = np.array(ttm_preds_list)
|
||||
|
||||
# 双模型交集
|
||||
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
|
||||
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
|
||||
two_model_preds = np.full(len(dlinear_preds), -1)
|
||||
two_model_preds[two_model_agree_up] = 1
|
||||
two_model_preds[two_model_agree_down] = 0
|
||||
|
||||
# 三模型多数投票
|
||||
vote_sum = dlinear_preds + nbeats_preds + ttm_preds
|
||||
three_model_preds = np.where(vote_sum >= 2, 1, 0)
|
||||
|
||||
# 真实标签
|
||||
true_labels = y_dlinear
|
||||
|
||||
# ========== 构建每日数据 ==========
|
||||
timestamps = df_2026['timestamp'].iloc[168:168+min_samples].values
|
||||
timestamps_utc = pd.to_datetime(timestamps)
|
||||
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
|
||||
dates = timestamps_et.date
|
||||
|
||||
results_df = pd.DataFrame({
|
||||
'date': dates,
|
||||
'timestamp_et': timestamps_et,
|
||||
'true_label': true_labels,
|
||||
'two_model_pred': two_model_preds,
|
||||
'three_model_pred': three_model_preds,
|
||||
'two_model_correct': (two_model_preds == true_labels) & (two_model_preds != -1),
|
||||
'three_model_correct': (three_model_preds == true_labels)
|
||||
})
|
||||
|
||||
# 按天统计基础数据
|
||||
daily_stats = results_df.groupby('date').agg({
|
||||
'true_label': 'count',
|
||||
'two_model_pred': lambda x: (x != -1).sum(),
|
||||
'two_model_correct': 'sum',
|
||||
'three_model_correct': 'sum'
|
||||
})
|
||||
|
||||
daily_stats.columns = ['total_samples', 'two_model_valid', 'two_model_correct', 'three_model_correct']
|
||||
|
||||
# 计算每日准确率
|
||||
daily_stats['two_model_accuracy'] = daily_stats['two_model_correct'] / daily_stats['two_model_valid'] * 100
|
||||
daily_stats['three_model_accuracy'] = daily_stats['three_model_correct'] / daily_stats['total_samples'] * 100
|
||||
daily_stats['two_model_accuracy'] = daily_stats['two_model_accuracy'].fillna(0)
|
||||
|
||||
# ========== 应用智能休息策略 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📊 应用智能休息策略")
|
||||
print("="*80)
|
||||
|
||||
# 为每一天添加"是否休息"标记
|
||||
daily_stats['two_model_rest'] = False
|
||||
daily_stats['three_model_rest'] = False
|
||||
daily_stats['rest_reason'] = ''
|
||||
|
||||
dates_list = daily_stats.index.tolist()
|
||||
|
||||
for i, date in enumerate(dates_list):
|
||||
if i == 0:
|
||||
continue # 第一天没有历史数据,不休息
|
||||
|
||||
# 获取前一天和前两天的数据
|
||||
prev_date = dates_list[i-1]
|
||||
prev2_date = dates_list[i-2] if i >= 2 else None
|
||||
|
||||
# 双模型休息逻辑
|
||||
# 规则1:前一天准确率超过70%
|
||||
if daily_stats.loc[prev_date, 'two_model_accuracy'] > 70:
|
||||
daily_stats.loc[date, 'two_model_rest'] = True
|
||||
daily_stats.loc[date, 'rest_reason'] = f'前日准确率{daily_stats.loc[prev_date, "two_model_accuracy"]:.1f}%>70%'
|
||||
|
||||
# 规则2:连续两天准确率超过60%
|
||||
elif (prev2_date is not None and
|
||||
daily_stats.loc[prev_date, 'two_model_accuracy'] > 60 and
|
||||
daily_stats.loc[prev2_date, 'two_model_accuracy'] > 60):
|
||||
daily_stats.loc[date, 'two_model_rest'] = True
|
||||
daily_stats.loc[date, 'rest_reason'] = f'连续2天>60% ({daily_stats.loc[prev2_date, "two_model_accuracy"]:.1f}%, {daily_stats.loc[prev_date, "two_model_accuracy"]:.1f}%)'
|
||||
|
||||
# 三模型休息逻辑
|
||||
# 规则1:前一天准确率超过70%
|
||||
if daily_stats.loc[prev_date, 'three_model_accuracy'] > 70:
|
||||
daily_stats.loc[date, 'three_model_rest'] = True
|
||||
if not daily_stats.loc[date, 'rest_reason']:
|
||||
daily_stats.loc[date, 'rest_reason'] = f'三模前日{daily_stats.loc[prev_date, "three_model_accuracy"]:.1f}%>70%'
|
||||
|
||||
# 规则2:连续两天准确率超过60%
|
||||
elif (prev2_date is not None and
|
||||
daily_stats.loc[prev_date, 'three_model_accuracy'] > 60 and
|
||||
daily_stats.loc[prev2_date, 'three_model_accuracy'] > 60):
|
||||
daily_stats.loc[date, 'three_model_rest'] = True
|
||||
if not daily_stats.loc[date, 'rest_reason']:
|
||||
daily_stats.loc[date, 'rest_reason'] = f'三模连续2天>60%'
|
||||
|
||||
# ========== 统计休息天数 ==========
|
||||
two_model_rest_days = daily_stats['two_model_rest'].sum()
|
||||
three_model_rest_days = daily_stats['three_model_rest'].sum()
|
||||
total_days = len(daily_stats)
|
||||
|
||||
print(f"\n休息天数统计:")
|
||||
print(f" 总交易日: {total_days}天")
|
||||
print(f" 双模型休息: {two_model_rest_days}天 ({two_model_rest_days/total_days*100:.1f}%)")
|
||||
print(f" 三模型休息: {three_model_rest_days}天 ({three_model_rest_days/total_days*100:.1f}%)")
|
||||
|
||||
# ========== 计算休息策略的实际表现 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📈 策略对比:正常交易 vs 智能休息")
|
||||
print("="*80)
|
||||
|
||||
# 双模型策略
|
||||
print(f"\n🔵 双模型交集策略:")
|
||||
|
||||
# 正常交易(不休息)
|
||||
two_normal_correct = daily_stats['two_model_correct'].sum()
|
||||
two_normal_total = daily_stats['two_model_valid'].sum()
|
||||
two_normal_acc = two_normal_correct / two_normal_total * 100 if two_normal_total > 0 else 0
|
||||
|
||||
print(f"\n 📊 正常交易(不休息):")
|
||||
print(f" 交易天数: {total_days}天")
|
||||
print(f" 有效预测: {two_normal_total}次")
|
||||
print(f" 正确预测: {two_normal_correct}次")
|
||||
print(f" 准确率: {two_normal_acc:.2f}%")
|
||||
|
||||
# 智能休息
|
||||
two_rest_mask = ~daily_stats['two_model_rest']
|
||||
two_rest_correct = daily_stats.loc[two_rest_mask, 'two_model_correct'].sum()
|
||||
two_rest_total = daily_stats.loc[two_rest_mask, 'two_model_valid'].sum()
|
||||
two_rest_acc = two_rest_correct / two_rest_total * 100 if two_rest_total > 0 else 0
|
||||
two_trade_days = two_rest_mask.sum()
|
||||
|
||||
print(f"\n 🛌 智能休息策略:")
|
||||
print(f" 交易天数: {two_trade_days}天 (休息{two_model_rest_days}天)")
|
||||
print(f" 有效预测: {two_rest_total}次")
|
||||
print(f" 正确预测: {two_rest_correct}次")
|
||||
print(f" 准确率: {two_rest_acc:.2f}%")
|
||||
|
||||
two_acc_diff = two_rest_acc - two_normal_acc
|
||||
two_diff_mark = "✅" if two_acc_diff > 0 else ("⚠️" if two_acc_diff < 0 else "")
|
||||
print(f"\n 准确率变化: {two_acc_diff:+.2f}% {two_diff_mark}")
|
||||
|
||||
# 休息日的实际表现
|
||||
if two_model_rest_days > 0:
|
||||
two_rest_days_mask = daily_stats['two_model_rest']
|
||||
two_rest_day_correct = daily_stats.loc[two_rest_days_mask, 'two_model_correct'].sum()
|
||||
two_rest_day_total = daily_stats.loc[two_rest_days_mask, 'two_model_valid'].sum()
|
||||
two_rest_day_acc = two_rest_day_correct / two_rest_day_total * 100 if two_rest_day_total > 0 else 0
|
||||
|
||||
print(f"\n 🔍 被休息的{two_model_rest_days}天实际表现:")
|
||||
print(f" 准确率: {two_rest_day_acc:.2f}%")
|
||||
avoided_loss = "避免了低准确率" if two_rest_day_acc < two_normal_acc else "⚠️ 错过了好机会"
|
||||
print(f" 评价: {avoided_loss}")
|
||||
|
||||
# 三模型策略
|
||||
print(f"\n🟢 三模型投票策略:")
|
||||
|
||||
# 正常交易
|
||||
three_normal_correct = daily_stats['three_model_correct'].sum()
|
||||
three_normal_total = daily_stats['total_samples'].sum()
|
||||
three_normal_acc = three_normal_correct / three_normal_total * 100
|
||||
|
||||
print(f"\n 📊 正常交易(不休息):")
|
||||
print(f" 交易天数: {total_days}天")
|
||||
print(f" 有效预测: {three_normal_total}次")
|
||||
print(f" 正确预测: {three_normal_correct}次")
|
||||
print(f" 准确率: {three_normal_acc:.2f}%")
|
||||
|
||||
# 智能休息
|
||||
three_rest_mask = ~daily_stats['three_model_rest']
|
||||
three_rest_correct = daily_stats.loc[three_rest_mask, 'three_model_correct'].sum()
|
||||
three_rest_total = daily_stats.loc[three_rest_mask, 'total_samples'].sum()
|
||||
three_rest_acc = three_rest_correct / three_rest_total * 100 if three_rest_total > 0 else 0
|
||||
three_trade_days = three_rest_mask.sum()
|
||||
|
||||
print(f"\n 🛌 智能休息策略:")
|
||||
print(f" 交易天数: {three_trade_days}天 (休息{three_model_rest_days}天)")
|
||||
print(f" 有效预测: {three_rest_total}次")
|
||||
print(f" 正确预测: {three_rest_correct}次")
|
||||
print(f" 准确率: {three_rest_acc:.2f}%")
|
||||
|
||||
three_acc_diff = three_rest_acc - three_normal_acc
|
||||
three_diff_mark = "✅" if three_acc_diff > 0 else ("⚠️" if three_acc_diff < 0 else "")
|
||||
print(f"\n 准确率变化: {three_acc_diff:+.2f}% {three_diff_mark}")
|
||||
|
||||
# 休息日的实际表现
|
||||
if three_model_rest_days > 0:
|
||||
three_rest_days_mask = daily_stats['three_model_rest']
|
||||
three_rest_day_correct = daily_stats.loc[three_rest_days_mask, 'three_model_correct'].sum()
|
||||
three_rest_day_total = daily_stats.loc[three_rest_days_mask, 'total_samples'].sum()
|
||||
three_rest_day_acc = three_rest_day_correct / three_rest_day_total * 100
|
||||
|
||||
print(f"\n 🔍 被休息的{three_model_rest_days}天实际表现:")
|
||||
print(f" 准确率: {three_rest_day_acc:.2f}%")
|
||||
avoided_loss = "避免了低准确率" if three_rest_day_acc < three_normal_acc else "⚠️ 错过了好机会"
|
||||
print(f" 评价: {avoided_loss}")
|
||||
|
||||
# ========== 详细日志 ==========
|
||||
print("\n" + "="*80)
|
||||
print("📅 每日详细记录")
|
||||
print("="*80)
|
||||
|
||||
print(f"\n{'日期':<12} {'双模准确率':>10} {'三模准确率':>10} {'休息状态':<12} {'休息原因':<40}")
|
||||
print("-" * 95)
|
||||
|
||||
for date, row in daily_stats.iterrows():
|
||||
two_acc = row['two_model_accuracy']
|
||||
three_acc = row['three_model_accuracy']
|
||||
|
||||
rest_status = ""
|
||||
if row['two_model_rest']:
|
||||
rest_status += "双休 "
|
||||
if row['three_model_rest']:
|
||||
rest_status += "三休"
|
||||
|
||||
if not rest_status:
|
||||
rest_status = "正常交易"
|
||||
|
||||
# 准确率标记
|
||||
two_mark = "🔥" if two_acc >= 70 else ("✅" if two_acc >= 60 else "")
|
||||
three_mark = "🔥" if three_acc >= 70 else ("✅" if three_acc >= 60 else "")
|
||||
|
||||
print(f"{date} {two_acc:>9.1f}{two_mark} {three_acc:>9.1f}{three_mark} "
|
||||
f"{rest_status:<12} {row['rest_reason']:<40}")
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✓ 智能休息策略回测完成!")
|
||||
print("="*80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+134
-16
@@ -4,34 +4,90 @@
|
||||
同时启动:
|
||||
1. FastAPI 后端服务 (提供前端看板 API)
|
||||
2. Polymarket 自动交易程序
|
||||
3. 每日数据报告推送程序 (北京时间每天 8 点)
|
||||
|
||||
支持平台:macOS, Linux, Windows
|
||||
"""
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import signal
|
||||
import os
|
||||
import platform
|
||||
from datetime import datetime
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# 颜色代码
|
||||
GREEN = '\033[92m'
|
||||
YELLOW = '\033[93m'
|
||||
RED = '\033[91m'
|
||||
RESET = '\033[0m'
|
||||
from utils.network_proxy import apply_global_proxy_from_env
|
||||
from utils.project_logger import get_project_logger
|
||||
|
||||
try:
|
||||
from daily_report_sender import run_scheduler_background
|
||||
except ImportError:
|
||||
# 如果模块导入失败,定义一个空函数避免启动失败
|
||||
def run_scheduler_background():
|
||||
return None
|
||||
|
||||
try:
|
||||
from mysql_d1_backup_scheduler import run_mysql_d1_backup_scheduler_background
|
||||
except ImportError:
|
||||
# 如果模块导入失败,定义一个空函数避免启动失败
|
||||
def run_mysql_d1_backup_scheduler_background():
|
||||
return None
|
||||
|
||||
load_dotenv()
|
||||
apply_global_proxy_from_env()
|
||||
logger = get_project_logger("start_system")
|
||||
|
||||
# 检测操作系统
|
||||
IS_WINDOWS = platform.system() == 'Windows'
|
||||
|
||||
# 颜色代码 (Windows CMD 默认不支持 ANSI,但现代 Windows Terminal 支持)
|
||||
if IS_WINDOWS:
|
||||
# 尝试启用 Windows ANSI 支持
|
||||
try:
|
||||
import ctypes
|
||||
kernel32 = ctypes.windll.kernel32
|
||||
kernel32.SetConsoleMode(kernel32.GetStdHandle(-11), 7)
|
||||
ANSI_SUPPORTED = True
|
||||
except Exception:
|
||||
ANSI_SUPPORTED = False
|
||||
else:
|
||||
ANSI_SUPPORTED = True
|
||||
|
||||
if ANSI_SUPPORTED:
|
||||
GREEN = '\033[92m'
|
||||
YELLOW = '\033[93m'
|
||||
RED = '\033[91m'
|
||||
RESET = '\033[0m'
|
||||
else:
|
||||
GREEN = YELLOW = RED = RESET = ''
|
||||
|
||||
def log(message, color=RESET):
|
||||
timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
|
||||
print(f"{color}[{timestamp}] {message}{RESET}")
|
||||
logger.info(message)
|
||||
|
||||
def start_process(command, name):
|
||||
"""启动子进程"""
|
||||
"""启动子进程(跨平台兼容)"""
|
||||
log(f"正在启动 {name}...", YELLOW)
|
||||
try:
|
||||
if IS_WINDOWS:
|
||||
# Windows: 不使用 preexec_fn,使用 CREATE_NEW_PROCESS_GROUP
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
shell=True,
|
||||
stdout=sys.stdout,
|
||||
stderr=sys.stderr,
|
||||
preexec_fn=os.setsid # 创建新的进程组,方便统一杀掉
|
||||
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP
|
||||
)
|
||||
else:
|
||||
# macOS / Linux: 使用 setsid 创建新进程组
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
shell=True,
|
||||
stdout=sys.stdout,
|
||||
stderr=sys.stderr,
|
||||
preexec_fn=os.setsid
|
||||
)
|
||||
log(f"{name} 已启动 (PID: {process.pid})", GREEN)
|
||||
return process
|
||||
@@ -39,6 +95,43 @@ def start_process(command, name):
|
||||
log(f"启动 {name} 失败: {e}", RED)
|
||||
return None
|
||||
|
||||
def terminate_process(process, name):
|
||||
"""终止进程(跨平台兼容)"""
|
||||
if process is None:
|
||||
return
|
||||
try:
|
||||
if IS_WINDOWS:
|
||||
# Windows: 使用 terminate() 或发送 CTRL_BREAK_EVENT
|
||||
process.terminate()
|
||||
else:
|
||||
# macOS / Linux: 终止整个进程组
|
||||
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
|
||||
log(f"{name} 已停止", GREEN)
|
||||
except Exception as e:
|
||||
log(f"停止 {name} 时出错: {e}", RED)
|
||||
# 强制杀死
|
||||
try:
|
||||
process.kill()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def send_alert(content):
|
||||
"""发送告警消息 (使用 curl)"""
|
||||
try:
|
||||
url = "https://zhiyinzhe.xyz/sendMsg/api/send"
|
||||
cmd = [
|
||||
"curl", "-X", "POST", url,
|
||||
"-F", "bot_name=weibo",
|
||||
"-F", "receive_id=on_f972f5f8c4de6ac967574b97dc8c1d82",
|
||||
"-F", "receive_id_type=union_id",
|
||||
"-F", f"content={content}"
|
||||
]
|
||||
# 使用 run 等待请求完成,确保告警发出
|
||||
subprocess.run(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=10)
|
||||
log(f"已发送告警: {content}", GREEN)
|
||||
except Exception as e:
|
||||
log(f"发送告警失败: {e}", RED)
|
||||
|
||||
def main():
|
||||
# 激活 Conda 环境
|
||||
# 注意:在子进程中直接使用当前已激活的 python 解释器路径,避免 conda activate 问题
|
||||
@@ -54,9 +147,32 @@ def main():
|
||||
|
||||
# 2. 启动自动交易程序
|
||||
# 使用 -u 参数禁用输出缓冲
|
||||
trade_cmd = f"{python_exec} -u auto_trade_polymarket.py"
|
||||
trade_cmd = f"{python_exec} -u trader/auto_trade_polymarket.py"
|
||||
trade_process = start_process(trade_cmd, "Polymarket 自动交易程序")
|
||||
|
||||
# 3. 启动每日数据报告推送程序(后台线程)
|
||||
# 北京时间每天 8 点推送昨天的数据统计
|
||||
try:
|
||||
report_thread = run_scheduler_background()
|
||||
if report_thread:
|
||||
log("每日数据报告推送程序已启动", GREEN)
|
||||
else:
|
||||
log("⚠️ 每日数据报告推送程序启动失败", YELLOW)
|
||||
except Exception as e:
|
||||
log(f"⚠️ 启动每日报告程序异常: {e}", YELLOW)
|
||||
report_thread = None
|
||||
|
||||
# 4. 启动 MySQL -> Cloudflare D1 定时备份(每小时 :30)
|
||||
try:
|
||||
d1_backup_thread = run_mysql_d1_backup_scheduler_background()
|
||||
if d1_backup_thread:
|
||||
log("MySQL -> D1 定时备份程序已启动(每小时 :30)", GREEN)
|
||||
else:
|
||||
log("D1 定时备份未启动(可能未启用或配置不完整)", YELLOW)
|
||||
except Exception as e:
|
||||
log(f"⚠️ 启动 D1 定时备份程序异常: {e}", YELLOW)
|
||||
d1_backup_thread = None
|
||||
|
||||
log("系统启动完成!", GREEN)
|
||||
log("前端访问地址: http://localhost:8000", GREEN)
|
||||
log("按 Ctrl+C 停止所有服务...", YELLOW)
|
||||
@@ -64,11 +180,15 @@ def main():
|
||||
try:
|
||||
# 监控子进程状态
|
||||
while True:
|
||||
if api_process.poll() is not None:
|
||||
log("API 服务意外退出", RED)
|
||||
if api_process and api_process.poll() is not None:
|
||||
msg = "API 服务意外退出"
|
||||
log(msg, RED)
|
||||
send_alert(msg)
|
||||
break
|
||||
if trade_process.poll() is not None:
|
||||
log("自动交易程序意外退出", RED)
|
||||
if trade_process and trade_process.poll() is not None:
|
||||
msg = "自动交易程序意外退出"
|
||||
log(msg, RED)
|
||||
send_alert(msg)
|
||||
break
|
||||
time.sleep(1)
|
||||
|
||||
@@ -76,10 +196,8 @@ def main():
|
||||
log("\n正在停止服务...", YELLOW)
|
||||
finally:
|
||||
# 优雅退出
|
||||
if api_process:
|
||||
os.killpg(os.getpgid(api_process.pid), signal.SIGTERM)
|
||||
if trade_process:
|
||||
os.killpg(os.getpgid(trade_process.pid), signal.SIGTERM)
|
||||
terminate_process(api_process, "API 服务")
|
||||
terminate_process(trade_process, "自动交易程序")
|
||||
|
||||
log("所有服务已停止", GREEN)
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -9,5 +9,59 @@
|
||||
"side": "DOWN",
|
||||
"price": 0.505,
|
||||
"probability": 0.5563886165618896
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-02-11T22:32:46.922211",
|
||||
"success": false,
|
||||
"error": "该市场已存在成功交易记录",
|
||||
"prediction": "DOWN",
|
||||
"probability": 0.0,
|
||||
"market": "Bitcoin Up or Down - February 11, 9AM ET",
|
||||
"market_slug": "bitcoin-up-or-down-february-11-9am-et"
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-02-11T23:18:40.579373",
|
||||
"success": false,
|
||||
"error": "该市场已存在成功交易记录",
|
||||
"prediction": "UP",
|
||||
"probability": 0.6666666666666666,
|
||||
"market": "Bitcoin Up or Down - February 11, 10AM ET",
|
||||
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-02-11T23:18:59.059130",
|
||||
"success": false,
|
||||
"error": "该市场已存在成功交易记录",
|
||||
"prediction": "UP",
|
||||
"probability": 0.6666666666666666,
|
||||
"market": "Bitcoin Up or Down - February 11, 10AM ET",
|
||||
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-02-11T23:19:50.797784",
|
||||
"success": false,
|
||||
"error": "该市场已存在成功交易记录",
|
||||
"prediction": "UP",
|
||||
"probability": 0.6666666666666666,
|
||||
"market": "Bitcoin Up or Down - February 11, 10AM ET",
|
||||
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-02-11T23:31:03.776709",
|
||||
"success": false,
|
||||
"error": "该市场已存在成功交易记录",
|
||||
"prediction": "UP",
|
||||
"probability": 0.6666666666666666,
|
||||
"market": "Bitcoin Up or Down - February 11, 10AM ET",
|
||||
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-02-11T23:35:39.565123",
|
||||
"success": false,
|
||||
"error": "该市场已存在成功交易记录",
|
||||
"prediction": "UP",
|
||||
"probability": 0.6666666666666666,
|
||||
"market": "Bitcoin Up or Down - February 11, 10AM ET",
|
||||
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
|
||||
}
|
||||
]
|
||||
+8
-3
@@ -38,19 +38,24 @@ class TrainingConfig:
|
||||
|
||||
Returns:
|
||||
Device string ('mps', 'cuda', or 'cpu')
|
||||
|
||||
Note:
|
||||
macOS 使用 MPS,Windows/Linux 使用 CPU
|
||||
"""
|
||||
import torch
|
||||
import platform
|
||||
|
||||
if self.device != 'auto':
|
||||
return self.device
|
||||
|
||||
# Auto-detect best available device
|
||||
# 根据操作系统选择设备:macOS 用 MPS,Windows/Linux 用 CPU
|
||||
if platform.system() == 'Darwin': # macOS
|
||||
if torch.backends.mps.is_available():
|
||||
return 'mps'
|
||||
elif torch.cuda.is_available():
|
||||
return 'cuda'
|
||||
else:
|
||||
return 'cpu'
|
||||
else: # Windows / Linux
|
||||
return 'cpu'
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
"""统一网络代理配置工具。"""
|
||||
|
||||
import os
|
||||
from typing import Optional
|
||||
|
||||
|
||||
def get_proxy_url() -> Optional[str]:
|
||||
"""从环境变量读取代理地址,统一兼容不同变量名。"""
|
||||
use_proxy = os.getenv("USE_PROXY", "false").lower() == "true"
|
||||
if not use_proxy:
|
||||
return None
|
||||
|
||||
return (
|
||||
os.getenv("SOCKS5_PROXY")
|
||||
or os.getenv("ALL_PROXY")
|
||||
or os.getenv("HTTPS_PROXY")
|
||||
or os.getenv("HTTP_PROXY")
|
||||
)
|
||||
|
||||
|
||||
def get_send_msg_proxy_url() -> Optional[str]:
|
||||
"""读取 send_msg 独立代理,未配置时回退到全局代理。"""
|
||||
send_msg_proxy = os.getenv("SEND_MSG_PROXY")
|
||||
if send_msg_proxy:
|
||||
return send_msg_proxy
|
||||
|
||||
send_msg_proxy_port = os.getenv("SEND_MSG_PROXY_PORT")
|
||||
if send_msg_proxy_port:
|
||||
return f"socks5://127.0.0.1:{send_msg_proxy_port}"
|
||||
|
||||
return get_proxy_url()
|
||||
|
||||
|
||||
def apply_global_proxy_from_env() -> Optional[str]:
|
||||
"""将代理写入全局环境变量,确保 requests/httpx/curl 等默认走同一代理。"""
|
||||
proxy_url = get_proxy_url()
|
||||
if not proxy_url:
|
||||
return None
|
||||
|
||||
os.environ["HTTP_PROXY"] = proxy_url
|
||||
os.environ["HTTPS_PROXY"] = proxy_url
|
||||
os.environ["ALL_PROXY"] = proxy_url
|
||||
os.environ["http_proxy"] = proxy_url
|
||||
os.environ["https_proxy"] = proxy_url
|
||||
os.environ["all_proxy"] = proxy_url
|
||||
|
||||
# 保留本地直连,避免影响本机服务调用。
|
||||
if not os.getenv("NO_PROXY"):
|
||||
os.environ["NO_PROXY"] = "127.0.0.1,localhost"
|
||||
if not os.getenv("no_proxy"):
|
||||
os.environ["no_proxy"] = os.environ["NO_PROXY"]
|
||||
|
||||
return proxy_url
|
||||
@@ -0,0 +1,40 @@
|
||||
"""项目统一日志工具。"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
from logging.handlers import RotatingFileHandler
|
||||
|
||||
|
||||
def get_project_logger(name: str) -> logging.Logger:
|
||||
"""返回统一配置的日志实例,避免重复添加处理器。"""
|
||||
logger = logging.getLogger(name)
|
||||
if logger.handlers:
|
||||
return logger
|
||||
|
||||
os.makedirs("logs", exist_ok=True)
|
||||
|
||||
logger.setLevel(logging.INFO)
|
||||
logger.propagate = False
|
||||
|
||||
formatter = logging.Formatter(
|
||||
"%(asctime)s | %(name)s | %(levelname)s | %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
)
|
||||
|
||||
file_handler = RotatingFileHandler(
|
||||
"logs/project.log",
|
||||
maxBytes=10 * 1024 * 1024,
|
||||
backupCount=5,
|
||||
encoding="utf-8",
|
||||
)
|
||||
file_handler.setFormatter(formatter)
|
||||
file_handler.setLevel(logging.INFO)
|
||||
|
||||
console_handler = logging.StreamHandler()
|
||||
console_handler.setFormatter(formatter)
|
||||
console_handler.setLevel(logging.INFO)
|
||||
|
||||
logger.addHandler(file_handler)
|
||||
logger.addHandler(console_handler)
|
||||
|
||||
return logger
|
||||
@@ -0,0 +1,54 @@
|
||||
import requests
|
||||
from utils.network_proxy import apply_global_proxy_from_env, get_send_msg_proxy_url
|
||||
from utils.project_logger import get_project_logger
|
||||
|
||||
|
||||
apply_global_proxy_from_env()
|
||||
logger = get_project_logger("send_msg")
|
||||
|
||||
class LarkNotifier:
|
||||
"""飞书消息发送工具类"""
|
||||
|
||||
def __init__(self, bot_name="weibo", receive_id="on_f972f5f8c4de6ac967574b97dc8c1d82"):
|
||||
# 将基础配置放在初始化中,方便后续修改或通过环境变量传入
|
||||
self.url = "https://zhiyinzhe.xyz/sendMsg/api/send"
|
||||
self.bot_name = bot_name
|
||||
self.receive_id = receive_id
|
||||
|
||||
# 使用 Session 可以复用底层 TCP 连接,提升发送效率
|
||||
self.session = requests.Session()
|
||||
|
||||
proxy_url = get_send_msg_proxy_url()
|
||||
if proxy_url:
|
||||
self.session.proxies.update({
|
||||
"http": proxy_url,
|
||||
"https": proxy_url,
|
||||
})
|
||||
logger.info("send_msg 使用独立代理: %s", proxy_url)
|
||||
else:
|
||||
logger.info("send_msg 未启用代理")
|
||||
|
||||
def send_alert(self, title: str, content: str):
|
||||
"""
|
||||
发送通用文本告警
|
||||
:param title: 告警标题 (如 "⚠️ 市场获取失败")
|
||||
:param content: 告警具体内容 (如 "Slug: xxx \n 原因: xxx")
|
||||
"""
|
||||
payload = {
|
||||
"bot_name": self.bot_name,
|
||||
"receive_id": self.receive_id,
|
||||
"receive_id_type": "union_id",
|
||||
"content": f"{title}\n{content}"
|
||||
}
|
||||
|
||||
try:
|
||||
# 使用 data=payload 发送 Form Data
|
||||
response = self.session.post(self.url, data=payload, timeout=5)
|
||||
response.raise_for_status()
|
||||
logger.info("告警发送成功: %s", title)
|
||||
except Exception as e:
|
||||
# 告警系统自身的错误只记录日志,绝不抛出异常影响主程序
|
||||
logger.error("发送飞书告警失败: %s", e)
|
||||
|
||||
# 实例化一个全局单例供其他模块直接导入使用
|
||||
lark_notifier = LarkNotifier()
|
||||
Reference in new issue
Block a user