28 Commits
Author SHA1 Message Date
crypto_predictor 62110554d1 feat: 添加 cloudflare_pages、mysql_d1_backup_scheduler,更新 start_system、ensemble_models 及配置文件 2026-03-15 14:22:46 +08:00
crypto_predictor 6184c34a3a fix: 更新 generate_market_slug 格式,新增年份字段 2026-03-15 14:21:49 +08:00
crypto_predictor 80f2a82c19 feat: unify proxy config and centralize runtime logging 2026-03-07 14:45:34 +08:00
crypto_predictor 7af21bed6a feat: 添加每日数据报告定时推送功能
- 新增 daily_report_sender.py 定时任务程序
- 北京时间每天 8 点推送昨天 24 小时预测统计
- 包含各模型准确率、逐小时预测详情
- 失败交易订单汇总
- start_system.py 集成启动报告推送服务
2026-02-28 10:55:34 +08:00
crypto_predictor e7e01516ba chore: staged changes 2026-02-27 13:36:37 +08:00
subuhui 1d732844e3 retry 2026-02-24 17:42:00 +08:00
crypto_predictor 7d5c8c57b7 ensemble file 2026-02-22 19:46:21 +08:00
crypto_predictor f13b1c1afe update polymarket trading: fix auth, add order monitoring, optimize status handling, remove unused scripts 2026-02-12 21:47:09 +08:00
crypto_predictor 7004b281a4 refactor: Migrate inference service to use ONNX Runtime 2026-02-12 12:43:58 +08:00
crypto_predictor e773775acd feat: Update ONNX export script and add TTM-R2 ONNX model 2026-02-12 12:02:46 +08:00
crypto_predictor 8138629e06 Merge branch 'feature/code_optimization' 2026-02-12 10:10:29 +08:00
crypto_predictor fe986256c0 feat: Export N-BEATS and TTM-R2 models to ONNX format 2026-02-12 10:02:26 +08:00
crypto_predictor 6a882ed913 fix: update initial trade logic
- Enable immediate trading attempt on startup
- Dynamic timeout calculation (remaining minutes - 10) for initial trade
- Added duplicate check for initial prediction saving to prevent DB errors
2026-02-11 23:36:07 +08:00
crypto_predictor 99fa79cde4 fix: timezone handling and prediction logic
- api/models.py: Fix timezone conversion in stats queries by calculating UTC ranges in Python instead of using SQL CONVERT_TZ. Ensure JSON serializable types.
- api/prediction_service.py:
    - Update ensure_predictions_for_month to check previous day (ET) to cover CST month start.
    - Update _generate_prediction_for_hour to skip saving unclosed candles (current hour).
- trader/auto_trade_polymarket.py: Update to use hourly_model_predictions (V2) table for storing live trading predictions.
2026-02-11 23:14:50 +08:00
crypto_predictor 2993014fa8 feat: upgrade to multi-model ensemble system
- Updated DB schemas: added 'strategy' to trades, new 'hourly_model_predictions' wide table
- Refactored backend: api/models.py, api/prediction_service.py to support wide format
- Updated trader: auto_trade_polymarket.py now logs strategy and saves to new table
- Updated frontend: index.html and app.js to display multi-model accuracy and trade strategy
2026-02-11 22:13:35 +08:00
crypto_predictor c0a02868ac feat: refactor model inference architecture
- Unified inference logic into models/inference.py
- Added InferenceManager for handling DLinear, N-BEATS, and TTM-R2 models
- Updated prediction_service.py and auto_trade_polymarket.py to use the new manager
- Fixed dependency issues (torch, transformers)
- Resolved logger duplication issue
- Verified with test_models_feb_2026.py
2026-02-11 22:07:32 +08:00
crypto_predictor d34b67f522 Add CLAUDE.md guide and preprocessing comparison script 2026-02-10 23:26:44 +08:00
crypto_predictorandClaude Opus 4.6 84288a687a feat: auto_trade 每小时预测结果写入预测表并回填上一小时实际结果
- 导入 PredictionModel,预测完成后保存到 model_predictions 表
- 当前小时预测 actual_result/is_correct/close_price 留空(未收盘)
- 新增 _update_previous_prediction:每次整点回填上一小时实际涨跌(收盘价 vs 开盘价)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 23:19:57 +08:00
crypto_predictorandClaude Opus 4.6 e78836bf42 fix: 修复 auto_trade 推理预处理逻辑,新增 create_labels 版对比测试脚本
auto_trade_polymarket.py: 新增 preprocess_for_inference 方法,推理时不调用
create_labels(避免 df[:-1] 砍掉输入K线),与 test 脚本逻辑一致。
test_dlinear_jan_2026_create_labels.py: 使用 create_labels 方式的 DLinear 测试
脚本,用于对比两种预处理方式的准确率差异。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 23:05:17 +08:00
crypto_predictor 7990312878 feat: 添加 ONNX 导出功能及 DLinear 模型导出 2026-02-08 15:05:09 +08:00
crypto_predictor 5ca1779d7d fix: 精简 requirements.txt,移除未使用的依赖
移除的依赖:
- gluonts: 项目中未使用
- pytorch-lightning: 非必需
- torchmetrics: 非必需
- onnx/onnxruntime: 非核心功能
- sqlalchemy: 使用 pymysql 直连
- jupyter/matplotlib/seaborn/pytest: 开发工具,非运行必需

保留的核心依赖:
- pandas, numpy, scikit-learn: 数据处理
- granite-tsfm: TTM-R2 模型
- pandas-ta: 技术指标
- fastapi, uvicorn: API 服务
- httpx, python-binance: 数据获取
- schedule: 任务调度
- pymysql, cryptography: 数据库

注意:PyTorch 需要单独安装(Windows 用 CPU 版本)
2026-02-07 22:28:46 +08:00
crypto_predictor 9320b79730 refactor: 重新整理 requirements.txt
主要更改:
- 修正 TTM 模型包名: tsfm_public -> granite-tsfm
- 添加缺失依赖: httpx, schedule, pytorch-lightning, torchmetrics
- 移除过时依赖: darts, u8darts
- 按功能分类组织依赖
- 添加 Windows/macOS 安装说明注释
- 更新安装脚本和文档中的包名
2026-02-07 22:26:07 +08:00
crypto_predictor 06800ddb43 fix: 修正 tsfm-public 包名,添加一键安装脚本
- 添加 install_windows.bat 一键安装脚本
- 修正文档中 tsfm_public -> tsfm-public (PyPI 包名用连字符)
- 添加 GitHub 备用安装方式
2026-02-07 22:02:56 +08:00
crypto_predictor 340c11d4ab docs: 添加 Windows 环境安装指南
包含完整的安装流程:
- Miniconda 安装
- Conda 环境创建
- PyTorch CPU 版本安装
- 项目依赖安装
- 数据库配置
- 环境变量配置
- 运行和验证步骤
- 常见问题解答
2026-02-07 21:55:56 +08:00
crypto_predictor d0696a42ba feat: 添加 Windows 启动脚本
- start_system.bat: CMD 批处理脚本,双击即可运行
- start_system.ps1: PowerShell 脚本,支持进程监控和优雅退出

两个脚本都会:
1. 激活 crypto_pred_312 conda 环境
2. 启动 API 服务 (端口 8000)
3. 启动自动交易程序
2026-02-07 21:54:46 +08:00
crypto_predictor 233f0c5d10 feat: 添加 Windows 兼容性支持,优化设备选择逻辑
- start_system.py: 跨平台进程管理
  - Windows 使用 CREATE_NEW_PROCESS_GROUP 和 terminate()
  - Unix 使用 setsid 和 killpg
  - 自动检测并启用 Windows ANSI 颜色支持

- 设备选择逻辑优化:
  - macOS: 优先使用 MPS,回退到 CPU
  - Windows/Linux: 强制使用 CPU(无 CUDA 依赖)

- 影响文件:
  - api/prediction_service.py
  - auto_trade_polymarket.py
  - training/config.py
2026-02-07 21:48:06 +08:00
crypto_predictor 8cdfb2a8e2 fix: 统一时区处理,准确率统计使用CST时间分组
- 增强 convert_to_cst() 函数,新增 source_tz 参数支持 UTC/ET/CST 输入
- 修改 get_monthly_stats() 使用 CONVERT_TZ 按 CST 月份统计
- 修改 get_daily_stats() 使用 CONVERT_TZ 按 CST 日期分组
- 修改 get_hourly_details() 按 CST 日期筛选和小时排序
- 交易记录使用 source_tz='CST' 避免重复转换
- 前端表头添加'(北京)'时区标注
2026-02-07 18:14:37 +08:00
crypto_predictor 6182630203 docs: 更新 AGENTS.md 开发指南,反映最新的架构变更 2026-02-07 16:58:44 +08:00
63 changed files with 7338 additions and 5387 deletions

No files matched your search

+7 -1
View File
@@ -12,7 +12,13 @@
"Bash(git push:*)",
"Bash(conda activate:*)",
"WebSearch",
"Bash(pip install:*)"
"Bash(pip install:*)",
"Bash(git status:*)",
"Bash(git -C /Users/xxz/PycharmProjects/crypto_predictor check-ignore -v test_dlinear_jan_2026_create_labels.py)",
"Bash(git -C /Users/xxz/PycharmProjects/crypto_predictor status)",
"Bash(git -C /Users/xxz/PycharmProjects/crypto_predictor diff)",
"Bash(git -C /Users/xxz/PycharmProjects/crypto_predictor log --oneline -5)",
"Bash(git checkout:*)"
]
}
}
+23
View File
@@ -5,6 +5,10 @@ BINANCE_API_URL=https://api.binance.com
SOCKS5_PROXY=socks5://127.0.0.1:7890
USE_PROXY=true
# send_msg 独立代理(可选,优先级高于 SOCKS5_PROXY)
# SEND_MSG_PROXY=socks5://127.0.0.1:7891
# SEND_MSG_PROXY_PORT=7891
# Data Configuration
DATA_CACHE_DIR=./data/cache
LOOKBACK_1H=168 # 1 week for 1h data
@@ -37,3 +41,22 @@ MYSQL_PORT=3306
MYSQL_USER=user
MYSQL_PASSWORD=password
MYSQL_DB=crypto_predictor
# Cloudflare D1 Backup Configuration
# 开关:true 开启 MySQL -> D1 定时备份(每小时 :30)
D1_BACKUP_ENABLED=false
CF_D1_API_TOKEN=your_cloudflare_api_token
CF_D1_ACCOUNT_ID=your_cloudflare_account_id
CF_D1_DATABASE_ID=your_d1_database_id
# 备份批大小(单次 SQL 插入的最大行数)
D1_BACKUP_BATCH_SIZE=500
# 单条 SQL 最大参数数(D1 限制差异较大,默认保守)
D1_MAX_SQL_PARAMS=90
# 指定备份表(逗号分隔);留空时自动备份 MySQL 全部表
D1_BACKUP_TABLES=ohlcv,hourly_model_predictions,trades
# Polymarket Configuration
POLYMARKET_PRIVATE_KEY=your_private_key_here
POLYMARKET_API_KEY=your_api_key_here
POLYMARKET_API_SECRET=your_api_secret_here
POLYMARKET_API_PASSPHRASE=your_api_passphrase_here
+43 -81
View File
@@ -8,104 +8,66 @@
* **语言要求**:所有回答必须使用**中文**。
* **称呼要求**:每次回答必须加上**“主人”**。
* **最小改动 (Minimal Change)**:只修改完成任务所必须的代码。**禁止**为了“最佳实践”或“代码美观”而进行未被请求的重构。
* **环境约束**:本项目严格使用 **Conda** 环境。禁止使用 venv/poetry/pipenv。
* **复用优先**:在编写新功能前,必须检查 `data/`, `models/`, `utils/` 下是否已有类似实现。
* **环境约束**:本项目严格使用 **Conda** 环境 (`crypto_pred_312`)。禁止使用 venv/poetry/pipenv。
* **复用优先**:在编写新功能前,必须检查 `data/`, `models/`, `api/` 下是否已有类似实现。
## 2. 环境与构建 (Environment & Build)
本项目依赖管理基于 `requirements.txt`, conda环境名称**crypto_pred_312**
* **Python 环境**: `conda activate crypto_pred_312`
* **依赖安装**: `pip install -r requirements.txt`
* **数据库**:
* **仅支持 MySQL**。禁止引入 SQLite 代码。
* 配置:通过 `.env` 文件配置 `MYSQL_HOST`, `MYSQL_USER` 等参数。
* 连接管理:使用 `data/database.py` 中的 `DatabaseManager` 或 `api/models.py` 中的 `get_db_connection`。
* **安装依赖**:
```bash
pip install -r requirements.txt
```
## 3. 项目结构说明 (Project Structure)
## 3. 测试与质量保证 (Testing & QA)
* **`api/`**: 核心服务层
* `main.py`: FastAPI 主应用入口。
* `models.py`: 数据库模型与操作(MySQL),包含时区转换逻辑。
* `prediction_service.py`: 预测业务逻辑。
* `static/`: Vue 3 前端静态资源(Dashboard)。
* **`data/`**: 数据层
* `database.py`: 市场数据 (OHLCV) 数据库管理。
* `trading_db.py`: 交易历史数据库管理。
* **`models/`**: 机器学习模型定义 (DLinear, TTM-R2 等) 及训练产物。
* **`scripts/`**: 辅助工具脚本。
* **根目录**:
* `start_system.py`: **系统统一启动脚本**(同时启动 API 和自动交易)。
* `auto_trade_polymarket.py`: 自动交易主程序。
使用 `pytest` 作为测试框架。在提交修改前,必须确保相关测试通过。
## 4. 开发规范 (Development Standards)
* **运行所有测试**:
```bash
pytest
```
### 4.1 时区处理 (Timezone)
* **存储**: 数据库中时间戳通常为 **UTC** 或 **ET (美东时间)**。
* **展示**: 前端及日志输出必须转换为 **CST (北京时间)**。
* **工具**: 使用 `api.models.convert_to_cst` 进行转换。
* **运行单个测试文件 (常用)**:
```bash
pytest tests/test_example.py
```
### 4.2 代码风格
* **注释**: 必须使用**中文**解释“为什么这样做”。
* **命名**: 变量/函数 `snake_case`,类名 `PascalCase`。
* **类型提示**: 推荐在关键接口使用 Type Hints。
* **运行单个测试用例 (推荐)**:
用于专注调试某一个功能:
```bash
pytest tests/test_example.py::test_specific_function
```
### 4.3 数据库操作
* **禁止**使用 SQLite。
* 使用 `pymysql` 或 `sqlalchemy` 进行操作。
* 确保所有数据库连接都使用上下文管理器 (`with` 语句) 以自动关闭。
* **显示打印输出 (Debug模式)**:
```bash
pytest -s tests/test_example.py
```
## 5. 测试与验证 (Testing & QA)
* **代码格式化与检查**:
```bash
black . # 格式化代码
flake8 . # 代码风格检查
```
* **运行测试**: `pytest`
* **代码检查**: `flake8 .`
* **本地验证**:
* 修改 API 或前端后,运行 `./start_system.py` 并访问 `http://localhost:8000` 验证。
* 修改模型后,运行 `python ensemble_predict.py` 验证预测逻辑。
## 4. 代码风格规范 (Code Style)
* **注释 (Comments)**:
* **必须使用中文**编写所有注释和文档字符串 (Docstrings)。
* Docstring 风格推荐 Google Style 或简单的多行注释。
* 注释应解释“为什么这样做” (Why),而不是“做了什么” (What)。
* **命名约定 (Naming)**:
* 变量/函数: `snake_case` (如 `add_technical_features`)
* 类名: `PascalCase` (如 `BinanceDataFetcher`)
* 常量: `UPPER_CASE` (如 `MAX_RETRIES`)
* 私有成员: `_snake_case` (前缀下划线)
* **导入 (Imports)**:
* 顺序:标准库 -> 第三方库 -> 本地模块。
* 尽量使用绝对导入。
* 注意:项目中部分脚本使用了 `sys.path.insert` hack,修改时需保持兼容,不要随意删除。
* **类型提示 (Type Hints)**:
* 推荐在函数签名中添加类型提示,但不强制要求全覆盖。
* 示例: `def process_data(df: pd.DataFrame) -> np.ndarray:`
* **错误处理 (Error Handling)**:
* 使用具体的异常类型 (如 `ValueError`, `FileNotFoundError`),避免裸 `except:`。
* 在 API 调用或网络请求处必须添加重试或错误捕获逻辑。
## 5. 项目结构说明 (Project Structure)
在修改代码前,请参考以下结构定位文件:
* **`api/`**: FastAPI 接口服务代码。
* **`data/`**: 数据获取 (BinanceFetcher等) 和预处理逻辑。
* **`models/`**: 模型定义 (N-BEATS, DLinear, CatBoost 等) 和 训练好的模型文件。
* **`tests/`**: 单元测试和集成测试。
* **`scripts/`**: 辅助脚本。
* **`training/`** & **`logs/`**: 训练日志和过程文件。
* **根目录脚本**: 包含多个入口脚本 (如 `ensemble_predict.py`, `train_catboost.py`),修改时需格外小心,避免破坏生产流程。
## 6. 开发流程 (Workflow)
1. **理解需求**: 先阅读代码,确认需求范围。如果有歧义,**先询问主人**。
2. **定位代码**: 使用 `grep` 或 `glob` 查找相关文件。
3. **制定计划**: 思考修改方案,优先考虑不破坏现有逻辑的“加法”修改。
4. **实施修改**: 编辑代码,保持风格一致。
5. **验证**:
* 运行 `flake8` 检查语法错误。
* 运行 `pytest` 验证逻辑。
* 如果是脚本,尝试带 `--help` 或测试参数运行以确保无运行时错误。
## 7. 禁止事项 (Constraints)
## 6. 禁止事项 (Constraints)
* ❌ **禁止**猜测业务逻辑。
* ❌ **禁止**删除看似无用但可能被外部脚本调用的代码。
* ❌ **禁止**在未确认的情况下升级核心库版本 (如 torch, pandas)。
* ❌ **禁止**输出英语回复 (代码变量名除外)。
* ❌ **禁止**恢复已删除的 SQLite 相关代码。
---
*遵循此指南,确保高效、安全地协助主人完成任务。*
+83
View File
@@ -0,0 +1,83 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## 项目概述
基于深度学习的 BTC 小时级涨跌预测系统,包含模型训练、FastAPI 后端、Vue 3 看板、Polymarket 自动交易。当前最佳模型 DLinear 准确率 53.10%。
## 环境与构建
```bash
# Python 环境(必须使用 Conda,禁止 venv/poetry)
conda activate crypto_pred_312
# 安装依赖(PyTorch 需根据平台单独安装,requirements.txt 不含 torch)
pip install -r requirements.txt
# 数据库:仅支持 MySQL,禁止 SQLite
# 配置:复制 .env.example 为 .env 并填写 MySQL 连接信息
```
## 常用命令
```bash
# 启动完整系统(API + 自动交易)
python start_system.py
# 单独启动 API 服务
python -m uvicorn api.main:app --host 0.0.0.0 --port 8000
# 单独启动自动交易
python auto_trade_polymarket.py
# 训练 DLinear 分类模型
python train_classification_full.py
# 测试模型(2026年数据)
python test_models_feb_2026.py
# 集成预测
python ensemble_predict.py
# 测试与检查
pytest
flake8 .
```
## 架构
```
数据流: Binance API → data/fetcher.py → data/preprocessor.py (技术指标) → data/dataset.py → 模型推理
交易流: 模型预测 → auto_trade_polymarket.py → Polymarket 下单 → MySQL 记录
服务流: FastAPI (api/main.py) → 前端看板 (api/static/) → http://localhost:8000
```
### 核心模块
- **`data/`** — 数据层:`fetcher.py`(Binance数据获取)、`preprocessor.py`(特征工程: RSI+MACD)、`dataset.py`(PyTorch数据集)、`database.py`(市场数据MySQL)、`trading_db.py`(交易历史MySQL)
- **`models/`** — 模型层:`dlinear/model.py`(主力模型,时序分解+线性层)、`nbeats/`、`base.py`(抽象基类)
- **`models/checkpoints/dlinear_classification_2026/`** — 训练产物:`best_model.pt`、`preprocessor.pkl`、`test_2026_results.json`
- **`training/`** — 训练层:`trainer.py`(训练循环,支持 MPS/CUDA/CPU)、`config.py`(超参数)、`evaluator.py`(评估)
- **`api/`** — 服务层:`main.py`(FastAPI入口)、`models.py`(DB操作+时区转换)、`prediction_service.py`(预测服务)、`polymarket_client.py`(交易客户端)
- **`start_system.py`** — 统一启动入口(同时启动 API 和自动交易)
- **`auto_trade_polymarket.py`** — 自动交易主程序(每小时执行预测+下单)
### DLinear 模型结构
输入 (168h × 9特征) → MovingAvg 时序分解 → 季节性/趋势分量各经线性层 → MLP 聚合 → logits 输出 (BCEWithLogitsLoss)
## 开发约束
- **数据库**:仅 MySQL(`pymysql`),禁止 SQLite。连接使用 `data/database.py` 的 `DatabaseManager` 或 `api/models.py` 的 `get_db_connection`
- **时区**:数据库存 UTC/ET,前端展示 CST(北京时间),转换用 `api.models.convert_to_cst()`
- **代码注释**:中文,解释"为什么"
- **命名**:变量/函数 `snake_case`,类 `PascalCase`
- **代理访问 Binance**:需在 `.env` 配置 `SOCKS5_PROXY` 和 `USE_PROXY=true`
- **日志**:交易日志在 `logs/polymarket_trading.log`(10MB 轮转)
## 禁止事项
- 禁止引入 SQLite 或恢复已删除的 SQLite 代码
- 禁止在未确认的情况下升级核心库版本(torch, pandas 等)
- 禁止删除看似无用但可能被外部脚本调用的代码
-459
View File
@@ -1,459 +0,0 @@
"""分析2019年和2026年1月准确率<40%交易日的前一天技术特征"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import pickle
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
from datetime import datetime, timedelta
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df, fit=True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
def calculate_technical_features(df):
"""计算丰富的技术特征"""
df = df.copy()
# 基础价格特征
df['price_change'] = df['close'] - df['open']
df['price_change_pct'] = (df['close'] - df['open']) / df['open'] * 100
df['high_low_range'] = df['high'] - df['low']
df['high_low_range_pct'] = (df['high'] - df['low']) / df['low'] * 100
df['body_ratio'] = abs(df['close'] - df['open']) / (df['high'] - df['low'] + 1e-10)
# 移动平均线
df['sma_5'] = ta.sma(df['close'], length=5)
df['sma_10'] = ta.sma(df['close'], length=10)
df['sma_20'] = ta.sma(df['close'], length=20)
df['sma_50'] = ta.sma(df['close'], length=50)
# 价格相对MA位置
df['price_vs_sma5'] = (df['close'] - df['sma_5']) / df['sma_5'] * 100
df['price_vs_sma20'] = (df['close'] - df['sma_20']) / df['sma_20'] * 100
# RSI
df['rsi_14'] = ta.rsi(df['close'], length=14)
df['rsi_7'] = ta.rsi(df['close'], length=7)
# MACD
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
# 布林带
bbands = ta.bbands(df['close'], length=20)
if bbands is not None and not bbands.empty:
df['bb_upper'] = bbands.iloc[:, 0]
df['bb_mid'] = bbands.iloc[:, 1]
df['bb_lower'] = bbands.iloc[:, 2]
df['bb_width'] = (df['bb_upper'] - df['bb_lower']) / df['bb_mid'] * 100
df['bb_position'] = (df['close'] - df['bb_lower']) / (df['bb_upper'] - df['bb_lower'] + 1e-10) * 100
# ATR波动率
df['atr_14'] = ta.atr(df['high'], df['low'], df['close'], length=14)
df['atr_pct'] = df['atr_14'] / df['close'] * 100
# 成交量特征
df['volume_sma_20'] = ta.sma(df['volume'], length=20)
df['volume_ratio'] = df['volume'] / (df['volume_sma_20'] + 1e-10)
# 价格动量
df['momentum_1'] = df['close'].pct_change(1) * 100
df['momentum_3'] = df['close'].pct_change(3) * 100
df['momentum_7'] = df['close'].pct_change(7) * 100
df['momentum_24'] = df['close'].pct_change(24) * 100
# 波动率
df['volatility_5'] = df['close'].pct_change().rolling(5).std() * 100
df['volatility_10'] = df['close'].pct_change().rolling(10).std() * 100
df['volatility_20'] = df['close'].pct_change().rolling(20).std() * 100
# 连续涨跌天数
df['is_up'] = (df['close'] > df['open']).astype(int)
df['consecutive_days'] = 0
return df
def analyze_period(model, preprocessor, device, period_name, start_date, end_date):
"""分析单个时期的数据,返回低准确率日期"""
print(f"\n{'='*80}")
print(f"分析 {period_name}")
print(f"{'='*80}")
# 获取数据(需要额外获取前一天的数据)
fetcher = BinanceDataFetcher()
# 获取包含前一天的数据
start_dt = pd.to_datetime(start_date) - pd.Timedelta(days=2)
df = fetcher.fetch_ohlcv('BTC', '1h', start_date=start_dt.strftime('%Y-%m-%d'),
end_date=end_date, use_cache=False)
print(f"✓ 数据: {len(df)} 行")
# 预处理
X, y, df_processed = preprocessor.preprocess(df, fit=False)
print(f"✓ 有效样本: {len(X)}")
# DLinear预测
X_tensor = torch.FloatTensor(X).to(device)
with torch.no_grad():
logits = model(X_tensor).cpu().numpy()
probs = 1 / (1 + np.exp(-logits))
preds = (probs > 0.5).astype(int).flatten()
true_labels = y
# 构建每日结果
timestamps = df['timestamp'].iloc[168:168+len(X)].values
timestamps_utc = pd.to_datetime(timestamps)
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
dates = timestamps_et.date
results_df = pd.DataFrame({
'date': dates,
'timestamp_et': timestamps_et,
'correct': preds == true_labels,
})
# 按天统计准确率
daily_stats = results_df.groupby('date').agg({
'correct': ['count', 'sum']
})
daily_stats.columns = ['total', 'correct']
daily_stats['accuracy'] = daily_stats['correct'] / daily_stats['total'] * 100
# 找出准确率<40%的日期
low_acc_dates = daily_stats[daily_stats['accuracy'] < 40].index.tolist()
print(f"✓ 准确率<40%的日期: {len(low_acc_dates)}天")
for date in low_acc_dates:
acc = daily_stats.loc[date, 'accuracy']
print(f" {date}: {acc:.1f}%")
# 计算技术特征
df_with_features = calculate_technical_features(df)
# 为每天聚合技术特征(使用当天的24小时数据)
# 先确保timestamp是datetime类型
if not pd.api.types.is_datetime64_any_dtype(df_with_features['timestamp']):
df_with_features['timestamp'] = pd.to_datetime(df_with_features['timestamp'])
df_with_features['date'] = df_with_features['timestamp'].dt.tz_localize('UTC').tz_convert('America/New_York').dt.date
# 返回低准确率日期和完整数据
return low_acc_dates, df_with_features, daily_stats
def main():
print("="*80)
print("分析准确率<40%交易日的前一天技术特征共同点")
print("="*80)
# ========== 加载模型 ==========
print("\n[1/3] 加载DLinear模型...")
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
from models.dlinear.model import DLinear
checkpoint = torch.load(
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device, weights_only=False
)
model = DLinear(checkpoint['config'])
model.load_state_dict(checkpoint['model_state_dict'])
model = model.to(device)
model.eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
preprocessor = pickle.load(f)
print("✓ DLinear模型已加载")
# ========== 分析两个时期 ==========
print("\n[2/3] 分析2019年和2026年1月...")
# 分析2019年1月
low_acc_2019, df_2019, stats_2019 = analyze_period(
model, preprocessor, device,
"2019年1月", "2019-01-01", "2019-01-31"
)
# 分析2026年1月
low_acc_2026, df_2026, stats_2026 = analyze_period(
model, preprocessor, device,
"2026年1月", "2026-01-01", "2026-01-31"
)
# ========== 提取前一天特征 ==========
print("\n[3/3] 分析前一天技术特征...")
# 合并所有低准确率日期
all_low_acc_dates = low_acc_2019 + low_acc_2026
print(f"\n总计{len(all_low_acc_dates)}个低准确率交易日")
# 提取前一天的特征
prev_day_features = []
for date in all_low_acc_dates:
# 获取前一天日期
prev_date = date - timedelta(days=1)
# 确定使用哪个数据集
if date in low_acc_2019:
df = df_2019
period = "2019年"
else:
df = df_2026
period = "2026年"
# 获取前一天的数据
prev_day_data = df[df['date'] == prev_date]
if len(prev_day_data) == 0:
continue
# 聚合前一天的技术特征(使用24小时的平均值、最大值、最小值等)
features = {
'date': date,
'prev_date': prev_date,
'period': period,
'accuracy': stats_2019.loc[date, 'accuracy'] if date in low_acc_2019 else stats_2026.loc[date, 'accuracy'],
# 价格特征
'prev_close': prev_day_data['close'].iloc[-1],
'prev_open': prev_day_data['open'].iloc[0],
'prev_price_change_pct': prev_day_data['price_change_pct'].mean(),
'prev_high_low_range_pct': prev_day_data['high_low_range_pct'].mean(),
'prev_body_ratio': prev_day_data['body_ratio'].mean(),
# RSI
'prev_rsi_14': prev_day_data['rsi_14'].iloc[-1],
'prev_rsi_7': prev_day_data['rsi_7'].iloc[-1],
# MACD
'prev_macd': prev_day_data['macd'].iloc[-1] if 'macd' in prev_day_data.columns else None,
'prev_macd_hist': prev_day_data['macd_hist'].iloc[-1] if 'macd_hist' in prev_day_data.columns else None,
# 布林带
'prev_bb_width': prev_day_data['bb_width'].iloc[-1] if 'bb_width' in prev_day_data.columns else None,
'prev_bb_position': prev_day_data['bb_position'].iloc[-1] if 'bb_position' in prev_day_data.columns else None,
# 波动率
'prev_atr_pct': prev_day_data['atr_pct'].iloc[-1] if 'atr_pct' in prev_day_data.columns else None,
'prev_volatility_5': prev_day_data['volatility_5'].iloc[-1] if 'volatility_5' in prev_day_data.columns else None,
'prev_volatility_20': prev_day_data['volatility_20'].iloc[-1] if 'volatility_20' in prev_day_data.columns else None,
# 成交量
'prev_volume_ratio': prev_day_data['volume_ratio'].iloc[-1] if 'volume_ratio' in prev_day_data.columns else None,
# 动量
'prev_momentum_1': prev_day_data['momentum_1'].iloc[-1] if 'momentum_1' in prev_day_data.columns else None,
'prev_momentum_3': prev_day_data['momentum_3'].iloc[-1] if 'momentum_3' in prev_day_data.columns else None,
'prev_momentum_7': prev_day_data['momentum_7'].iloc[-1] if 'momentum_7' in prev_day_data.columns else None,
'prev_momentum_24': prev_day_data['momentum_24'].iloc[-1] if 'momentum_24' in prev_day_data.columns else None,
# 价格相对MA位置
'prev_price_vs_sma5': prev_day_data['price_vs_sma5'].iloc[-1] if 'price_vs_sma5' in prev_day_data.columns else None,
'prev_price_vs_sma20': prev_day_data['price_vs_sma20'].iloc[-1] if 'price_vs_sma20' in prev_day_data.columns else None,
}
prev_day_features.append(features)
# 转换为DataFrame
features_df = pd.DataFrame(prev_day_features)
# ========== 分析共同点 ==========
print("\n" + "="*80)
print("📊 前一天技术特征统计分析")
print("="*80)
print(f"\n分析样本数: {len(features_df)}个低准确率交易日")
print(f" 2019年: {(features_df['period']=='2019年').sum()}天")
print(f" 2026年: {(features_df['period']=='2026年').sum()}天")
# 统计各特征的平均值、中位数、标准差
numeric_cols = [col for col in features_df.columns if col.startswith('prev_') and features_df[col].dtype in [np.float64, np.int64]]
print(f"\n前一天技术特征统计:")
print(f"{'特征':<30} {'平均值':>10} {'中位数':>10} {'标准差':>10} {'共同点':<30}")
print("-" * 95)
for col in numeric_cols:
if features_df[col].notna().sum() > 0:
mean_val = features_df[col].mean()
median_val = features_df[col].median()
std_val = features_df[col].std()
# 判断共同点
common_trait = ""
if 'rsi' in col:
if mean_val > 70:
common_trait = "超买区域"
elif mean_val < 30:
common_trait = "超卖区域"
elif 45 <= mean_val <= 55:
common_trait = "中性区域"
elif 'volatility' in col:
if mean_val > 3:
common_trait = "高波动"
elif mean_val < 1:
common_trait = "低波动"
elif 'momentum' in col:
if mean_val > 2:
common_trait = "强势上涨"
elif mean_val < -2:
common_trait = "强势下跌"
elif abs(mean_val) < 0.5:
common_trait = "横盘震荡"
elif 'bb_position' in col:
if mean_val > 80:
common_trait = "接近布林上轨"
elif mean_val < 20:
common_trait = "接近布林下轨"
elif 'volume_ratio' in col:
if mean_val > 1.5:
common_trait = "成交量放大"
elif mean_val < 0.8:
common_trait = "成交量萎缩"
print(f"{col:<30} {mean_val:>10.2f} {median_val:>10.2f} {std_val:>10.2f} {common_trait:<30}")
# ========== 对比分析 ==========
print("\n" + "="*80)
print("🔍 2019年 vs 2026年对比分析")
print("="*80)
features_2019 = features_df[features_df['period'] == '2019年']
features_2026 = features_df[features_df['period'] == '2026年']
if len(features_2019) > 0 and len(features_2026) > 0:
print(f"\n{'特征':<30} {'2019年平均':>12} {'2026年平均':>12} {'差异':>10}")
print("-" * 70)
for col in numeric_cols[:15]: # 显示前15个特征
if features_2019[col].notna().sum() > 0 and features_2026[col].notna().sum() > 0:
mean_2019 = features_2019[col].mean()
mean_2026 = features_2026[col].mean()
diff = mean_2026 - mean_2019
print(f"{col:<30} {mean_2019:>12.2f} {mean_2026:>12.2f} {diff:>10.2f}")
# ========== 关键共同点总结 ==========
print("\n" + "="*80)
print("💡 关键共同点总结")
print("="*80)
# RSI分析
rsi_mean = features_df['prev_rsi_14'].mean()
print(f"\n1. RSI指标:")
print(f" 平均RSI: {rsi_mean:.1f}")
if rsi_mean > 70:
print(f" 🔴 共同点: 前一天处于超买状态")
elif rsi_mean < 30:
print(f" 🟢 共同点: 前一天处于超卖状态")
else:
print(f" ➖ 共同点: 前一天RSI在中性区域")
# 波动率分析
vol_mean = features_df['prev_volatility_20'].mean()
print(f"\n2. 波动率:")
print(f" 平均波动率: {vol_mean:.2f}%")
if vol_mean > 3:
print(f" ⚠️ 共同点: 前一天波动率较高")
elif vol_mean < 1:
print(f" ✅ 共同点: 前一天波动率较低")
# 动量分析
mom_24_mean = features_df['prev_momentum_24'].mean()
print(f"\n3. 24小时动量:")
print(f" 平均动量: {mom_24_mean:.2f}%")
if abs(mom_24_mean) > 5:
print(f" 🚀 共同点: 前一天有强趋势({'+' if mom_24_mean > 0 else ''}{mom_24_mean:.1f}%)")
else:
print(f" 🔄 共同点: 前一天横盘震荡")
# 布林带位置
bb_pos_mean = features_df['prev_bb_position'].mean()
print(f"\n4. 布林带位置:")
print(f" 平均位置: {bb_pos_mean:.1f}%")
if bb_pos_mean > 80:
print(f" 🔴 共同点: 前一天价格接近布林上轨")
elif bb_pos_mean < 20:
print(f" 🟢 共同点: 前一天价格接近布林下轨")
else:
print(f" ➖ 共同点: 前一天价格在布林带中部")
# 成交量
vol_ratio_mean = features_df['prev_volume_ratio'].mean()
print(f"\n5. 成交量:")
print(f" 平均成交量比: {vol_ratio_mean:.2f}x")
if vol_ratio_mean > 1.5:
print(f" 📈 共同点: 前一天成交量显著放大")
elif vol_ratio_mean < 0.8:
print(f" 📉 共同点: 前一天成交量萎缩")
print("\n" + "="*80)
print("✓ 技术特征分析完成!")
print("="*80)
# 保存详细数据
output_file = "./results/low_accuracy_prev_day_features.csv"
os.makedirs("./results", exist_ok=True)
features_df.to_csv(output_file, index=False)
print(f"\n✓ 详细数据已保存至: {output_file}")
if __name__ == "__main__":
main()
+6 -6
View File
@@ -106,8 +106,8 @@ async def get_monthly_accuracy(
service = get_prediction_service()
service.ensure_predictions_for_month(month)
# 获取统计
stats = PredictionModel.get_monthly_stats(month)
# 获取统计 (使用 V2 方法)
stats = PredictionModel.get_monthly_stats_v2(month)
return {
"success": True,
@@ -135,8 +135,8 @@ async def get_daily_accuracy(
service = get_prediction_service()
service.ensure_predictions_for_month(month)
# 获取统计
result = PredictionModel.get_daily_stats(month=month, page=page, limit=limit)
# 获取统计 (使用 V2 方法)
result = PredictionModel.get_daily_stats_v2(month=month, page=page, limit=limit)
return {
"success": True,
@@ -155,8 +155,8 @@ async def get_hourly_accuracy(date: str):
service = get_prediction_service()
service.ensure_predictions_for_date(date)
# 获取详情
details = PredictionModel.get_hourly_details(date)
# 获取详情 (使用 V2 方法)
details = PredictionModel.get_hourly_details_v2(date)
return {
"success": True,
+363 -175
View File
@@ -4,7 +4,7 @@
import pymysql
import os
from typing import Optional, List, Dict
from datetime import datetime
from datetime import datetime, timedelta
from contextlib import contextmanager
from zoneinfo import ZoneInfo
from dotenv import load_dotenv
@@ -17,12 +17,13 @@ TZ_ET = ZoneInfo('America/New_York') # 美东时间 (Polymarket)
TZ_CST = ZoneInfo('Asia/Shanghai') # 中国标准时间 (前端显示)
def convert_to_cst(timestamp_val) -> str:
def convert_to_cst(timestamp_val, source_tz: str = 'UTC') -> str:
"""
将时间戳转换为 CST 时间字符串
Args:
timestamp_val: datetime 对象或 ISO 格式时间戳字符串 (假设是 UTC)
timestamp_val: datetime 对象或 ISO 格式时间戳字符串
source_tz: 输入时区 - 'UTC'(默认,币安数据), 'ET'(Polymarket), 'CST'(已是北京时间)
Returns:
CST 时间字符串 (YYYY-MM-DD HH:MM:SS)
@@ -37,10 +38,20 @@ def convert_to_cst(timestamp_val) -> str:
# 字符串格式
dt = datetime.fromisoformat(str(timestamp_val).replace('Z', '+00:00'))
# 如果没有时区信息,假设是 UTC
# 根据来源时区设置时区信息
if dt.tzinfo is None:
if source_tz == 'ET':
dt = dt.replace(tzinfo=TZ_ET)
elif source_tz == 'CST':
# 已经是 CST,直接格式化返回
return dt.strftime('%Y-%m-%d %H:%M:%S')
else: # 默认 UTC
dt = dt.replace(tzinfo=TZ_UTC)
# 如果来源已经是 CST,直接格式化
if source_tz == 'CST':
return dt.strftime('%Y-%m-%d %H:%M:%S')
# 转换为 CST
dt_cst = dt.astimezone(TZ_CST)
return dt_cst.strftime('%Y-%m-%d %H:%M:%S')
@@ -77,6 +88,7 @@ def _dict_from_row(row):
def init_model_predictions_table():
"""初始化 model_predictions 表"""
# 兼容旧表,但主要使用 hourly_model_predictions
with get_db_connection() as conn:
cursor = conn.cursor()
cursor.execute('''
@@ -99,6 +111,42 @@ def init_model_predictions_table():
INDEX idx_model (model_name)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
''')
# 确保 hourly_model_predictions 表存在
cursor.execute('''
CREATE TABLE IF NOT EXISTS hourly_model_predictions (
id INT AUTO_INCREMENT PRIMARY KEY,
timestamp DATETIME NOT NULL,
timestamp_et VARCHAR(50),
date VARCHAR(10),
hour INT,
open_price DOUBLE,
close_price DOUBLE,
actual_result INT COMMENT '0=Down, 1=Up',
dlinear_prediction INT,
dlinear_prob DOUBLE,
dlinear_correct INT,
nbeats_prediction INT,
nbeats_prob DOUBLE,
nbeats_correct INT,
ttm_prediction INT,
ttm_prob DOUBLE,
ttm_correct INT,
ensemble_prediction INT,
ensemble_votes VARCHAR(50),
ensemble_correct INT,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE KEY uk_date_hour (date, hour),
INDEX idx_timestamp (timestamp)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
''')
conn.commit()
@@ -122,7 +170,8 @@ class TradeModel:
row = cursor.fetchone()
result = _dict_from_row(row)
if result and result.get('timestamp'):
result['timestamp_cst'] = convert_to_cst(result['timestamp'])
# 交易记录时间已经是 CST,直接格式化
result['timestamp_cst'] = convert_to_cst(result['timestamp'], source_tz='CST')
return result
@staticmethod
@@ -131,12 +180,6 @@ class TradeModel:
day: Optional[str] = None) -> Dict:
"""
获取历史交易记录
Args:
page: 页码
limit: 每页条数
month: 月份筛选 (YYYY-MM)
day: 日期筛选 (YYYY-MM-DD)
"""
offset = (page - 1) * limit
@@ -148,7 +191,7 @@ class TradeModel:
params = []
if month:
where_clauses.append("DATE_FORMAT(timestamp, '%%Y-%%m') = %s")
where_clauses.append("DATE_FORMAT(timestamp, '%Y-%m') = %s")
params.append(month)
if day:
@@ -173,7 +216,8 @@ class TradeModel:
for row in rows:
item = _dict_from_row(row)
if item and item.get('timestamp'):
item['timestamp_cst'] = convert_to_cst(item['timestamp'])
# 交易记录时间已经是 CST,直接格式化
item['timestamp_cst'] = convert_to_cst(item['timestamp'], source_tz='CST')
data_list.append(item)
return {
@@ -186,130 +230,178 @@ class TradeModel:
class PredictionModel:
"""预测记录模型"""
"""预测记录模型 (V2 - 宽表结构)"""
@staticmethod
def get_prediction(date: str, hour: int, model_name: str) -> Optional[Dict]:
"""获取指定时间和模型的预测记录"""
with get_db_connection() as conn:
cursor = conn.cursor()
cursor.execute("""
SELECT * FROM model_predictions
WHERE date = %s AND hour = %s AND model_name = %s
""", (date, hour, model_name))
row = cursor.fetchone()
return _dict_from_row(row)
@staticmethod
def save_prediction(data: Dict) -> int:
def save_hourly_prediction(data: Dict) -> int:
"""
保存预测记录
Args:
data: 预测数据字典
Returns:
插入的记录ID
保存每小时综合预测记录 (宽表)
"""
with get_db_connection() as conn:
cursor = conn.cursor()
sql = """
INSERT INTO model_predictions
(timestamp, timestamp_et, date, hour, model_name,
prediction, probability, actual_result, is_correct,
open_price, close_price)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
INSERT INTO hourly_model_predictions
(timestamp, timestamp_et, date, hour, open_price, close_price, actual_result,
dlinear_prediction, dlinear_prob, dlinear_correct,
nbeats_prediction, nbeats_prob, nbeats_correct,
ttm_prediction, ttm_prob, ttm_correct,
ensemble_prediction, ensemble_votes, ensemble_correct)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
ON DUPLICATE KEY UPDATE
prediction = VALUES(prediction),
probability = VALUES(probability),
actual_result = VALUES(actual_result),
is_correct = VALUES(is_correct),
open_price = VALUES(open_price),
close_price = VALUES(close_price)
close_price = VALUES(close_price),
actual_result = VALUES(actual_result),
dlinear_prediction = VALUES(dlinear_prediction),
dlinear_prob = VALUES(dlinear_prob),
dlinear_correct = VALUES(dlinear_correct),
nbeats_prediction = VALUES(nbeats_prediction),
nbeats_prob = VALUES(nbeats_prob),
nbeats_correct = VALUES(nbeats_correct),
ttm_prediction = VALUES(ttm_prediction),
ttm_prob = VALUES(ttm_prob),
ttm_correct = VALUES(ttm_correct),
ensemble_prediction = VALUES(ensemble_prediction),
ensemble_votes = VALUES(ensemble_votes),
ensemble_correct = VALUES(ensemble_correct)
"""
cursor.execute(sql, (
data.get('timestamp'),
data.get('timestamp_et'),
data.get('date'),
data.get('hour'),
data.get('model_name'),
data.get('prediction'),
data.get('probability'),
data.get('actual_result'),
data.get('is_correct'),
data.get('open_price'),
data.get('close_price')
data.get('close_price'),
data.get('actual_result'),
data.get('dlinear_prediction'),
data.get('dlinear_prob'),
data.get('dlinear_correct'),
data.get('nbeats_prediction'),
data.get('nbeats_prob'),
data.get('nbeats_correct'),
data.get('ttm_prediction'),
data.get('ttm_prob'),
data.get('ttm_correct'),
data.get('ensemble_prediction'),
str(data.get('ensemble_votes')) if data.get('ensemble_votes') else None,
data.get('ensemble_correct')
))
conn.commit()
return cursor.lastrowid or 0
@staticmethod
def update_actual_result(date: str, hour: int, model_name: str,
actual_result: int, close_price: float) -> bool:
"""更新实际结果"""
def update_actual_result_v2(date: str, hour: int, actual_result: int, close_price: float) -> bool:
"""
更新实际结果,并重新计算各模型正确性 (V2)
"""
with get_db_connection() as conn:
cursor = conn.cursor()
# 先获取预测值
# 获取该小时记录
cursor.execute("""
SELECT prediction FROM model_predictions
WHERE date = %s AND hour = %s AND model_name = %s
""", (date, hour, model_name))
SELECT * FROM hourly_model_predictions
WHERE date = %s AND hour = %s
""", (date, hour))
row = cursor.fetchone()
if not row:
return False
prediction = row['prediction']
is_correct = 1 if prediction == actual_result else 0
# 计算正确性
updates = {
'actual_result': actual_result,
'close_price': close_price
}
cursor.execute("""
UPDATE model_predictions
SET actual_result = %s, is_correct = %s, close_price = %s
WHERE date = %s AND hour = %s AND model_name = %s
""", (actual_result, is_correct, close_price, date, hour, model_name))
# DLinear
if row['dlinear_prediction'] is not None:
updates['dlinear_correct'] = 1 if row['dlinear_prediction'] == actual_result else 0
# N-BEATS
if row['nbeats_prediction'] is not None:
updates['nbeats_correct'] = 1 if row['nbeats_prediction'] == actual_result else 0
# TTM
if row['ttm_prediction'] is not None:
updates['ttm_correct'] = 1 if row['ttm_prediction'] == actual_result else 0
# Ensemble
if row['ensemble_prediction'] is not None:
updates['ensemble_correct'] = 1 if row['ensemble_prediction'] == actual_result else 0
# 构建更新SQL
set_clause = ", ".join([f"{k} = %s" for k in updates.keys()])
sql = f"UPDATE hourly_model_predictions SET {set_clause} WHERE date = %s AND hour = %s"
cursor.execute(sql, list(updates.values()) + [date, hour])
conn.commit()
return True
@staticmethod
def get_monthly_stats(month: str) -> Dict:
def get_monthly_stats_v2(month: str) -> Dict:
"""
获取月度统计
Args:
month: 月份 (YYYY-MM)
Returns:
各模型的统计数据
获取月度统计 (V2 - 从宽表)
修改:不使用 SQL 进行时区转换和格式化,而是查询 UTC 时间范围,然后在 Python 中处理
"""
# 计算 UTC 时间范围
try:
year, mon = map(int, month.split('-'))
start_date_cst = datetime(year, mon, 1, tzinfo=TZ_CST)
# 下个月第一天
if mon == 12:
end_date_cst = datetime(year + 1, 1, 1, tzinfo=TZ_CST)
else:
end_date_cst = datetime(year, mon + 1, 1, tzinfo=TZ_CST)
# 转换为 UTC
start_dt_utc = start_date_cst.astimezone(TZ_UTC)
end_dt_utc = end_date_cst.astimezone(TZ_UTC)
start_str = start_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
end_str = end_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
except Exception as e:
print(f"日期解析错误: {e}")
return {}
with get_db_connection() as conn:
cursor = conn.cursor()
sql = """
SELECT
model_name,
COUNT(*) as total,
SUM(CASE WHEN is_correct = 1 THEN 1 ELSE 0 END) as correct,
SUM(CASE WHEN is_correct = 0 THEN 1 ELSE 0 END) as incorrect
FROM model_predictions
WHERE DATE_FORMAT(date, '%%Y-%%m') = %s
AND actual_result IS NOT NULL
GROUP BY model_name
"""
cursor.execute(sql, (month,))
rows = cursor.fetchall()
# 统计各个模型的准确率
models = ['dlinear', 'nbeats', 'ttm', 'ensemble']
result = {}
for row in rows:
row_dict = _dict_from_row(row)
model = row_dict['model_name']
total = row_dict['total']
correct = row_dict['correct'] or 0
incorrect = row_dict['incorrect'] or 0
for model in models:
col_pred = f"{model}_prediction"
col_correct = f"{model}_correct"
# 查询 UTC 时间范围内的所有数据
sql = f"""
SELECT
COUNT({col_pred}) as total,
SUM(CASE WHEN {col_correct} = 1 THEN 1 ELSE 0 END) as correct,
SUM(CASE WHEN {col_correct} = 0 THEN 1 ELSE 0 END) as incorrect
FROM hourly_model_predictions
WHERE timestamp >= %s AND timestamp < %s
AND actual_result IS NOT NULL
"""
cursor.execute(sql, (start_str, end_str))
row = cursor.fetchone()
if row:
total = int(row['total'])
correct = int(row['correct'] or 0)
incorrect = int(row['incorrect'] or 0)
accuracy = (correct / total * 100) if total > 0 else 0
result[model] = {
# 格式化键名以匹配前端
model_key = {
'dlinear': 'DLinear',
'nbeats': 'N-BEATS',
'ttm': 'TTM-R2',
'ensemble': 'Ensemble'
}.get(model, model)
result[model_key] = {
"total": total,
"correct": correct,
"incorrect": incorrect,
@@ -319,66 +411,118 @@ class PredictionModel:
return result
@staticmethod
def get_daily_stats(month: str, page: int = 1, limit: int = 10) -> Dict:
def get_daily_stats_v2(month: str, page: int = 1, limit: int = 10) -> Dict:
"""
获取每日统计
获取每日统计 (V2 - 从宽表)
修改:不使用 SQL 进行时区转换,查询 UTC 范围,然后在 Python 中转换
"""
# 计算 UTC 时间范围
try:
year, mon = map(int, month.split('-'))
start_date_cst = datetime(year, mon, 1, tzinfo=TZ_CST)
if mon == 12:
end_date_cst = datetime(year + 1, 1, 1, tzinfo=TZ_CST)
else:
end_date_cst = datetime(year, mon + 1, 1, tzinfo=TZ_CST)
start_dt_utc = start_date_cst.astimezone(TZ_UTC)
end_dt_utc = end_date_cst.astimezone(TZ_UTC)
start_str = start_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
end_str = end_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
except Exception:
return {"data": [], "page": page, "limit": limit, "total": 0, "total_pages": 0}
Args:
month: 月份 (YYYY-MM)
page: 页码
limit: 每页条数
"""
with get_db_connection() as conn:
cursor = conn.cursor()
# 获取范围内的所有原始数据
# 我们需要 timestamp, actual_result, 和各模型的 prediction/correct
# 为了按 CST 日期分组,我们需要取出每一行,然后在 Python 中转换时间
sql = """
SELECT
date,
model_name,
COUNT(*) as total,
SUM(CASE WHEN is_correct = 1 THEN 1 ELSE 0 END) as correct,
SUM(CASE WHEN is_correct = 0 THEN 1 ELSE 0 END) as incorrect
FROM model_predictions
WHERE DATE_FORMAT(date, '%%Y-%%m') = %s
timestamp,
actual_result,
dlinear_prediction, dlinear_correct,
nbeats_prediction, nbeats_correct,
ttm_prediction, ttm_correct,
ensemble_prediction, ensemble_correct
FROM hourly_model_predictions
WHERE timestamp >= %s AND timestamp < %s
AND actual_result IS NOT NULL
GROUP BY date, model_name
ORDER BY date DESC
ORDER BY timestamp DESC
"""
cursor.execute(sql, (month,))
cursor.execute(sql, (start_str, end_str))
rows = cursor.fetchall()
# 重组数据: 按日期合并不同模型的统计
daily_data = {}
# 在 Python 中处理分组
daily_stats = {} # { "YYYY-MM-DD": { "date": "...", "dlinear": {...}, ... } }
for row in rows:
row_dict = _dict_from_row(row)
date = row_dict['date']
if date not in daily_data:
daily_data[date] = {"date": date}
# 转换 timestamp 到 CST
ts_utc = row['timestamp']
# 确保是带时区的 datetime
if isinstance(ts_utc, str):
ts_utc = datetime.fromisoformat(str(ts_utc).replace('Z', '+00:00'))
if ts_utc.tzinfo is None:
ts_utc = ts_utc.replace(tzinfo=TZ_UTC)
model = row_dict['model_name']
total = row_dict['total']
correct = row_dict['correct'] or 0
incorrect = row_dict['incorrect'] or 0
accuracy = (correct / total * 100) if total > 0 else 0
ts_cst = ts_utc.astimezone(TZ_CST)
date_str = ts_cst.strftime('%Y-%m-%d')
daily_data[date][model] = {
"total": total,
"correct": correct,
"incorrect": incorrect,
"accuracy": round(accuracy, 2)
if date_str not in daily_stats:
daily_stats[date_str] = {
"date": date_str,
"raw_data": {m: {"total": 0, "correct": 0} for m in ['dlinear', 'nbeats', 'ttm', 'ensemble']}
}
# 排序并分页
sorted_dates = sorted(daily_data.keys(), reverse=True)
# 累加统计
models = ['dlinear', 'nbeats', 'ttm', 'ensemble']
for model in models:
pred = row[f"{model}_prediction"]
correct_flag = row[f"{model}_correct"]
if pred is not None:
daily_stats[date_str]['raw_data'][model]['total'] += 1
if correct_flag == 1:
daily_stats[date_str]['raw_data'][model]['correct'] += 1
# 整理为最终格式
sorted_dates = sorted(daily_stats.keys(), reverse=True)
total_count = len(sorted_dates)
# 分页
offset = (page - 1) * limit
paginated_dates = sorted_dates[offset:offset + limit]
data = [daily_data[d] for d in paginated_dates]
result_data = []
for date_key in paginated_dates:
day_item = daily_stats[date_key]
formatted_item = {"date": day_item["date"]}
for model in ['dlinear', 'nbeats', 'ttm', 'ensemble']:
raw = day_item['raw_data'][model]
total = int(raw['total'])
correct = int(raw['correct'])
accuracy = (correct / total * 100) if total > 0 else 0
model_key = {
'dlinear': 'DLinear',
'nbeats': 'N-BEATS',
'ttm': 'TTM-R2',
'ensemble': 'Ensemble'
}.get(model, model)
formatted_item[model_key] = {
"total": total,
"correct": correct,
"accuracy": round(accuracy, 2)
}
result_data.append(formatted_item)
return {
"data": data,
"data": result_data,
"page": page,
"limit": limit,
"total": total_count,
@@ -386,69 +530,113 @@ class PredictionModel:
}
@staticmethod
def get_hourly_details(date: str) -> List[Dict]:
def get_hourly_details_v2(date: str) -> List[Dict]:
"""
获取指定日期的每小时详情
获取指定日期的每小时详情 (V2)
修改:根据传入的 CST 日期计算 UTC 范围,查询该范围内的数据
"""
try:
# date 是 'YYYY-MM-DD' (CST)
date_cst_start = datetime.strptime(date, '%Y-%m-%d').replace(tzinfo=TZ_CST)
date_cst_end = date_cst_start + timedelta(days=1)
start_dt_utc = date_cst_start.astimezone(TZ_UTC)
end_dt_utc = date_cst_end.astimezone(TZ_UTC)
start_str = start_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
end_str = end_dt_utc.strftime('%Y-%m-%d %H:%M:%S')
except Exception as e:
print(f"日期处理错误: {e}")
return []
Args:
date: 日期 (YYYY-MM-DD)
"""
with get_db_connection() as conn:
cursor = conn.cursor()
cursor.execute("""
SELECT * FROM model_predictions
WHERE date = %s
ORDER BY hour, model_name
""", (date,))
SELECT *
FROM hourly_model_predictions
WHERE timestamp >= %s AND timestamp < %s
ORDER BY timestamp ASC
""", (start_str, end_str))
rows = cursor.fetchall()
result = []
# 按小时重组数据
hourly_data = {}
for row in rows:
row_dict = _dict_from_row(row)
hour = row_dict['hour']
if hour not in hourly_data:
# 将 timestamp 转换为 CST 时间
timestamp_cst = convert_to_cst(row_dict.get('timestamp', ''))
hourly_data[hour] = {
"hour": hour,
"timestamp_et": row_dict['timestamp_et'],
"timestamp_cst": timestamp_cst,
"open_price": row_dict['open_price'],
"close_price": row_dict['close_price'],
"actual_result": row_dict['actual_result'],
"models": {}
}
# 计算 CST 小时用于显示排序 (虽然 SQL 已经按 timestamp 排序了,但为了确保前端显示 hour 字段正确)
ts_utc = row['timestamp']
if isinstance(ts_utc, str):
ts_utc = datetime.fromisoformat(str(ts_utc).replace('Z', '+00:00'))
if ts_utc.tzinfo is None:
ts_utc = ts_utc.replace(tzinfo=TZ_UTC)
ts_cst = ts_utc.astimezone(TZ_CST)
hour_cst = ts_cst.hour
model = row_dict['model_name']
hourly_data[hour]["models"][model] = {
"prediction": row_dict['prediction'],
"probability": row_dict['probability'],
"is_correct": row_dict['is_correct']
hour_data = {
"hour": hour_cst,
"timestamp_et": row['timestamp_et'],
"timestamp_cst": convert_to_cst(row.get('timestamp', '')),
"open_price": row['open_price'],
"close_price": row['close_price'],
"actual_result": row['actual_result'],
"models": {
"DLinear": {
"prediction": row['dlinear_prediction'],
"probability": row['dlinear_prob'],
"is_correct": row['dlinear_correct']
},
"N-BEATS": {
"prediction": row['nbeats_prediction'],
"probability": row['nbeats_prob'],
"is_correct": row['nbeats_correct']
},
"TTM-R2": {
"prediction": row['ttm_prediction'],
"probability": row['ttm_prob'],
"is_correct": row['ttm_correct']
},
"Ensemble": {
"prediction": row['ensemble_prediction'],
"is_correct": row['ensemble_correct'],
"votes": row['ensemble_votes']
}
}
}
result.append(hour_data)
# 按小时排序返回
return [hourly_data[h] for h in sorted(hourly_data.keys())]
return result
@staticmethod
def get_missing_hours(date: str, model_name: str) -> List[int]:
def save_prediction(data: Dict) -> int:
"""兼容旧接口 - 如果需要的话,暂时保留,或者可以将旧表写入也放在这里"""
# 旧表写入逻辑... 略,或保持不变
# 为了完整性,我们可以保留对旧表的操作,或者逐步弃用。
# 这里为了安全,保留原方法的签名,实际调用旧表逻辑。
# 但我们主要逻辑将切换到 save_hourly_prediction
# 见 prediction_service.py 的修改
# ... (keep old logic or redirect) ...
# 这里我们假设 prediction_service.py 会改为调用 save_hourly_prediction
# 如果还有地方调用 save_prediction (比如测试代码), 我们可以保留它
return 0
@staticmethod
def update_actual_result(date: str, hour: int, model_name: str,
actual_result: int, close_price: float) -> bool:
"""兼容旧接口"""
return False
@staticmethod
def get_missing_hours(date: str, model_name: str = 'Ensemble') -> List[int]:
"""
获取指定日期缺少预测的小时列表
Args:
date: 日期 (YYYY-MM-DD)
model_name: 模型名称
Returns:
缺少预测的小时列表
获取指定日期缺少预测的小时列表 (V2)
model_name 参数现在主要用于兼容,实际检查整行是否存在
"""
with get_db_connection() as conn:
cursor = conn.cursor()
cursor.execute("""
SELECT hour FROM model_predictions
WHERE date = %s AND model_name = %s
""", (date, model_name))
SELECT hour FROM hourly_model_predictions
WHERE date = %s
""", (date,))
rows = cursor.fetchall()
existing_hours = {row['hour'] for row in rows}
+23 -19
View File
@@ -12,9 +12,13 @@ import httpx
from typing import Optional, Dict, List, Any
from dataclasses import dataclass
from dotenv import load_dotenv
from utils.network_proxy import apply_global_proxy_from_env, get_proxy_url
from utils.project_logger import get_project_logger
# 加载环境变量
load_dotenv()
apply_global_proxy_from_env()
logger = get_project_logger("api.polymarket_client")
@dataclass
@@ -46,7 +50,7 @@ class PolymarketClient:
self.private_key = private_key or os.getenv("POLYMARKET_PRIVATE_KEY")
# 配置代理
proxy = os.getenv("SOCKS5_PROXY") if os.getenv("USE_PROXY", "false").lower() == "true" else None
proxy = get_proxy_url()
# HTTP客户端 - 禁用压缩避免brotli问题
self.http = httpx.Client(
@@ -79,9 +83,9 @@ class PolymarketClient:
# 获取或创建API凭证
self.clob_client.set_api_creds(self.clob_client.create_or_derive_api_creds())
print("✓ CLOB客户端初始化成功")
logger.info("CLOB客户端初始化成功")
except Exception as e:
print(f"⚠️ CLOB客户端初始化失败: {e}")
logger.warning("CLOB客户端初始化失败: %s", e)
self.clob_client = None
def search_markets(self, query: str = "BTC", limit: int = 20) -> List[Dict]:
@@ -111,7 +115,7 @@ class PolymarketClient:
filtered = [m for m in markets if query.lower() in m.get("question", "").lower()]
return filtered
except Exception as e:
print(f"搜索市场失败: {e}")
logger.error("搜索市场失败: %s", e)
return []
def search_btc_1h_markets(self) -> List[Dict]:
@@ -150,7 +154,7 @@ class PolymarketClient:
return btc_1h_markets
except Exception as e:
print(f"搜索BTC 1H市场失败: {e}")
logger.error("搜索BTC 1H市场失败: %s", e)
return []
def get_market_info(self, condition_id: str) -> Optional[MarketInfo]:
@@ -179,7 +183,7 @@ class PolymarketClient:
end_date=data.get("endDate", ""),
)
except Exception as e:
print(f"获取市场信息失败: {e}")
logger.error("获取市场信息失败: %s", e)
return None
def get_orderbook(self, token_id: str) -> Dict:
@@ -200,7 +204,7 @@ class PolymarketClient:
response.raise_for_status()
return response.json()
except Exception as e:
print(f"获取订单簿失败: {e}")
logger.error("获取订单簿失败: %s", e)
return {}
def get_price(self, token_id: str) -> Optional[float]:
@@ -222,7 +226,7 @@ class PolymarketClient:
data = response.json()
return float(data.get("price", 0))
except Exception as e:
print(f"获取价格失败: {e}")
logger.error("获取价格失败: %s", e)
return None
def place_market_order(
@@ -243,7 +247,7 @@ class PolymarketClient:
订单响应
"""
if not self.clob_client:
print("❌ CLOB客户端未初始化,无法下单")
logger.error("CLOB客户端未初始化,无法下单")
return None
try:
@@ -260,10 +264,10 @@ class PolymarketClient:
# 提交订单
response = self.clob_client.post_order(order)
print(f"✓ 订单已提交: {response}")
logger.info("订单已提交: %s", response)
return response
except Exception as e:
print(f"❌ 下单失败: {e}")
logger.error("下单失败: %s", e)
return None
def place_limit_order(
@@ -286,7 +290,7 @@ class PolymarketClient:
订单响应
"""
if not self.clob_client:
print("❌ CLOB客户端未初始化,无法下单")
logger.error("CLOB客户端未初始化,无法下单")
return None
try:
@@ -304,10 +308,10 @@ class PolymarketClient:
# 提交订单
response = self.clob_client.post_order(order)
print(f"✓ 限价单已提交: {response}")
logger.info("限价单已提交: %s", response)
return response
except Exception as e:
print(f"❌ 下单失败: {e}")
logger.error("下单失败: %s", e)
return None
def get_positions(self) -> List[Dict]:
@@ -318,14 +322,14 @@ class PolymarketClient:
仓位列表
"""
if not self.clob_client:
print("❌ CLOB客户端未初始化")
logger.error("CLOB客户端未初始化")
return []
try:
positions = self.clob_client.get_positions()
return positions
except Exception as e:
print(f"获取仓位失败: {e}")
logger.error("获取仓位失败: %s", e)
return []
def cancel_order(self, order_id: str) -> bool:
@@ -339,15 +343,15 @@ class PolymarketClient:
是否成功
"""
if not self.clob_client:
print("❌ CLOB客户端未初始化")
logger.error("CLOB客户端未初始化")
return False
try:
self.clob_client.cancel(order_id)
print(f"✓ 订单已取消: {order_id}")
logger.info("订单已取消: %s", order_id)
return True
except Exception as e:
print(f"❌ 取消订单失败: {e}")
logger.error("取消订单失败: %s", e)
return False
def close(self):
+107 -256
View File
@@ -1,147 +1,42 @@
"""
预测服务 - DLinear 和 TTM-R2 模型预测
预测服务 - 统一模型预测 (DLinear, N-BEATS, TTM-R2)
"""
import os
import sys
import pickle
import threading
from datetime import datetime, timedelta
from typing import Optional, List, Dict, Tuple
from zoneinfo import ZoneInfo
import torch
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
import numpy as np
# 添加项目路径
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from data.fetcher import BinanceDataFetcher
from api.models import PredictionModel
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback: int = 168, forecast_horizon: int = 1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df: pd.DataFrame) -> pd.DataFrame:
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df: pd.DataFrame) -> pd.DataFrame:
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df: pd.DataFrame, fit: bool = True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
class CustomUnpickler(pickle.Unpickler):
"""自定义反序列化器,用于处理 __main__ 模块的类"""
def find_class(self, module, name):
if module == "__main__":
if name == "ClassificationPreprocessor":
return ClassificationPreprocessor
return super().find_class(module, name)
from models.inference import get_inference_manager
class PredictionService:
"""预测服务 - 管理 DLinear 和 TTM-R2 模型"""
"""预测服务 - 管理多模型预测"""
# 类级别的锁,确保多个请求不会同时生成预测
_prediction_lock = None
_prediction_lock = threading.Lock()
def __init__(self):
import threading
# 初始化锁
if PredictionService._prediction_lock is None:
PredictionService._prediction_lock = threading.Lock()
self.device = torch.device('mps' if torch.backends.mps.is_available() else
'cuda' if torch.cuda.is_available() else 'cpu')
self.et_tz = ZoneInfo('America/New_York')
self.fetcher = BinanceDataFetcher()
# 模型和预处理器
self.dlinear_model = None
self.dlinear_preprocessor = None
self.ttm_model = None
self.inference_manager = get_inference_manager()
# 记录正在处理的日期和月份,避免重复处理
self._processing_dates = set()
self._processing_months = set()
self._load_models()
def _load_models(self):
"""加载模型"""
print("正在加载预测模型...")
# 加载 DLinear
from models.dlinear.model import DLinear
dlinear_path = './models/checkpoints/dlinear_classification_2026/best_model.pt'
preprocessor_path = './models/checkpoints/dlinear_classification_2026/preprocessor.pkl'
checkpoint = torch.load(dlinear_path, map_location=self.device, weights_only=False)
self.dlinear_model = DLinear(checkpoint['config'])
self.dlinear_model.load_state_dict(checkpoint['model_state_dict'])
self.dlinear_model = self.dlinear_model.to(self.device).eval()
with open(preprocessor_path, 'rb') as f:
try:
self.dlinear_preprocessor = pickle.load(f)
except AttributeError:
f.seek(0)
self.dlinear_preprocessor = CustomUnpickler(f).load()
print(f" ✓ DLinear 已加载")
# 加载 TTM-R2
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
self.ttm_model = TinyTimeMixerForPrediction.from_pretrained(
"ibm-granite/granite-timeseries-ttm-r2",
trust_remote_code=True
)
self.ttm_model = self.ttm_model.to(self.device).eval()
print(f" ✓ TTM-R2 已加载")
print(f" 设备: {self.device}")
# 预加载模型
print("正在预加载预测模型...")
self.inference_manager.load_all_models()
def _get_btc_data(self, days: int = 30) -> pd.DataFrame:
"""获取BTC历史数据"""
@@ -158,83 +53,22 @@ class PredictionService:
return df
def predict_dlinear(self, df: pd.DataFrame) -> Tuple[int, float]:
"""
使用 DLinear 预测下一小时的涨跌
输入: df 包含历史K线数据 (最后一行是最新的已完成K线)
输出: 预测下一小时相对于 df 最后一行收盘价的涨跌
Returns:
(prediction, probability): 预测结果(0=跌, 1=涨)和概率
"""
# 预处理: 使用全部数据,不去掉最后一行
X, _, _ = self.dlinear_preprocessor.preprocess(df, fit=False)
if len(X) == 0:
"""DLinear 预测代理"""
pred, prob = self.inference_manager.predict_dlinear(df)
if pred is None:
return 0, 0.5
# 取最后一个窗口进行预测
X_last = torch.FloatTensor(X[-1:]).to(self.device)
with torch.no_grad():
logits = self.dlinear_model(X_last).cpu().numpy()
prob = 1 / (1 + np.exp(-logits))
prediction = int(prob.item() > 0.5)
probability = float(prob.item())
return prediction, probability
return pred, prob if prob is not None else 0.5
def predict_ttm(self, df: pd.DataFrame) -> Tuple[int, float]:
"""
使用 TTM-R2 预测下一小时的涨跌
输入: df 包含历史K线数据 (最后一行是最新的已完成K线)
输出: 预测下一小时相对于 df 最后一行收盘价的涨跌
Returns:
(prediction, probability): 预测结果(0=跌, 1=涨)和概率
"""
context_length = 512
all_prices = df['close'].values
# 使用最后 context_length 个价格
if len(all_prices) >= context_length:
context_data = all_prices[-context_length:].reshape(1, -1, 1)
else:
# 数据不足,用最后一个价格填充
padded = np.full(context_length, all_prices[-1])
padded[-len(all_prices):] = all_prices
context_data = padded.reshape(1, -1, 1)
# 基准价格: 最后一行的收盘价 (即预测目标小时的前一小时收盘价)
base_price = float(df['close'].iloc[-1])
context_tensor = torch.FloatTensor(context_data).to(self.device)
with torch.no_grad():
output = self.ttm_model(context_tensor)
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
# 判断涨跌: 预测价格 vs 基准价格
prediction = 1 if pred_price > base_price else 0
# 计算伪概率: 基于价格变化百分比
price_pct = (pred_price - base_price) / base_price
# 将价格变化百分比映射到概率 (sigmoid)
probability = 1 / (1 + np.exp(-price_pct * 100))
return prediction, float(probability)
"""TTM-R2 预测代理"""
pred, prob = self.inference_manager.predict_ttm(df)
if pred is None:
return 0, 0.5
return pred, prob if prob is not None else 0.5
def predict_current_hour(self) -> Dict:
"""
预测当前小时的涨跌
逻辑:
- 获取最新的K线数据 (最后一行是最新已完成的K线)
- 预测下一小时相对于最后一行收盘价的涨跌
Returns:
预测结果字典
预测当前小时的涨跌 (多模型集成)
"""
df = self._get_btc_data()
@@ -248,7 +82,6 @@ class PredictionService:
last_close = float(df['close'].iloc[-1])
# 当前正在形成的K线信息
current_open = float(df['open'].iloc[-1])
current_price = float(df['close'].iloc[-1]) # 当前价格
results = {
@@ -260,49 +93,62 @@ class PredictionService:
"models": {}
}
# DLinear 预测 (预测下一小时相对于 last_close 的涨跌)
dlinear_pred, dlinear_prob = self.predict_dlinear(df)
# 使用 InferenceManager 获取所有预测
predictions = self.inference_manager.predict_all(df)
# 填充结果
if 'dlinear' in predictions:
results["models"]["DLinear"] = {
"prediction": dlinear_pred,
"probability": round(dlinear_prob, 4),
"direction": "UP" if dlinear_pred == 1 else "DOWN"
"prediction": predictions['dlinear']['prediction'],
"probability": round(predictions['dlinear']['probability'], 4),
"direction": "UP" if predictions['dlinear']['prediction'] == 1 else "DOWN"
}
# TTM-R2 预测
ttm_pred, ttm_prob = self.predict_ttm(df)
if 'nbeats' in predictions:
results["models"]["N-BEATS"] = {
"prediction": predictions['nbeats']['prediction'],
"probability": round(predictions['nbeats']['probability'], 4),
"direction": "UP" if predictions['nbeats']['prediction'] == 1 else "DOWN"
}
if 'ttm' in predictions:
results["models"]["TTM-R2"] = {
"prediction": ttm_pred,
"probability": round(ttm_prob, 4),
"direction": "UP" if ttm_pred == 1 else "DOWN"
"prediction": predictions['ttm']['prediction'],
"probability": round(predictions['ttm']['probability'], 4),
"direction": "UP" if predictions['ttm']['prediction'] == 1 else "DOWN"
}
if 'ensemble' in predictions and predictions['ensemble']:
ens_pred = predictions['ensemble']['prediction']
results["models"]["Ensemble"] = {
"prediction": ens_pred,
"probability": 0.0, # 集成暂无概率
"direction": "UP" if ens_pred == 1 else "DOWN",
"votes": predictions['ensemble']['votes']
}
return results
def ensure_predictions_for_date(self, date: str) -> None:
"""
确保指定日期有预测数据,如果没有则生成
确保指定日期有预测数据,如果没有则生成 (V2)
Args:
date: 日期 (YYYY-MM-DD)
"""
# 使用锁避免并发冲突
with PredictionService._prediction_lock:
# 检查是否正在处理该日期
if date in self._processing_dates:
return
# 标记正在处理
self._processing_dates.add(date)
try:
# 检查是否有缺失的小时
for model_name in ["DLinear", "TTM-R2"]:
missing_hours = PredictionModel.get_missing_hours(date, model_name)
# 检查是否有缺失的小时 (只检查宽表是否有记录)
missing_hours = PredictionModel.get_missing_hours(date)
if not missing_hours:
continue
return
print(f"为 {date} 生成 {model_name} 预测 (缺失 {len(missing_hours)} 小时)...")
print(f"为 {date} 生成预测 (缺失 {len(missing_hours)} 小时)...")
# 获取足够的历史数据
target_date = datetime.strptime(date, '%Y-%m-%d')
@@ -310,13 +156,12 @@ class PredictionService:
if df is None or len(df) < 200:
print(f" 警告: 数据不足,无法为 {date} 生成预测")
continue
return
# 为每个缺失的小时生成预测
for hour in missing_hours:
self._generate_prediction_for_hour(df, date, hour, model_name)
self._generate_prediction_for_hour(df, date, hour)
finally:
# 移除处理标记
with PredictionService._prediction_lock:
self._processing_dates.discard(date)
@@ -338,22 +183,9 @@ class PredictionService:
print(f"获取历史数据失败: {e}")
return None
def _generate_prediction_for_hour(self, df: pd.DataFrame, date: str,
hour: int, model_name: str) -> None:
def _generate_prediction_for_hour(self, df: pd.DataFrame, date: str, hour: int) -> None:
"""
为指定小时生成预测
预测逻辑:
- 预测目标: {date} {hour}:00 的K线收盘价涨跌
- 输入数据: {hour-1}:00 及以前的K线数据 (不包含目标小时)
- 涨跌判断: 预测价格 vs 前一小时收盘价 (即 {hour-1}:00 的 close)
- 正确率验证: 目标小时的 close vs 前一小时的 close
Args:
df: 历史K线数据 (UTC时间戳)
date: 目标日期 (美东时间 YYYY-MM-DD)
hour: 目标小时 (美东时间 0-23)
model_name: 模型名称
为指定小时生成预测 (V2)
"""
try:
# 1. 构建目标时间 (美东时间 -> UTC)
@@ -368,7 +200,6 @@ class PredictionService:
# 3. 找到目标小时的K线 (用于验证)
target_mask = df['timestamp_dt'] == target_time_utc
if not target_mask.any():
# 目标小时数据不存在,跳过
return
target_row = df[target_mask].iloc[0]
@@ -379,7 +210,6 @@ class PredictionService:
prev_time_utc = target_time_utc - pd.Timedelta(hours=1)
prev_mask = df['timestamp_dt'] == prev_time_utc
if not prev_mask.any():
# 前一小时数据不存在,跳过
return
prev_row = df[prev_mask].iloc[0]
@@ -390,77 +220,98 @@ class PredictionService:
df_input = df[input_mask].copy()
if len(df_input) < 200:
# 数据不足,跳过
return
# 6. 调用模型预测
if model_name == "DLinear":
prediction, probability = self.predict_dlinear(df_input)
else: # TTM-R2
prediction, probability = self.predict_ttm(df_input)
# 6. 调用多模型预测
preds = self.inference_manager.predict_all(df_input)
# 7. 判断实际涨跌
# Check if the candle is closed (current time > candle open time + 1 hour)
current_now_utc = datetime.now(ZoneInfo('UTC')).replace(tzinfo=None)
candle_close_time = target_time_utc + timedelta(hours=1)
is_closed = current_now_utc >= candle_close_time
if not is_closed:
# 仅保存已完成的K线预测,未完成的跳过 (等待完成后再生成)
# 实时交易逻辑在 auto_trade_polymarket.py 中单独处理
return
# 7. 判断实际涨跌 (目标小时收盘价 vs 前一小时收盘价)
actual_result = 1 if target_close > prev_close else 0
# 8. 判断预测是否正确
is_correct = 1 if prediction == actual_result else 0
# 9. 保存预测结果
# 8. 组装数据 (宽表格式)
prediction_data = {
"timestamp": target_time_utc.isoformat(),
"timestamp_et": target_time_et.isoformat(),
"date": date,
"hour": hour,
"model_name": model_name,
"prediction": prediction,
"probability": probability,
"actual_result": actual_result,
"is_correct": is_correct,
"open_price": target_open,
"close_price": target_close
"close_price": target_close,
"actual_result": actual_result,
# DLinear
"dlinear_prediction": preds.get('dlinear', {}).get('prediction'),
"dlinear_prob": preds.get('dlinear', {}).get('probability'),
"dlinear_correct": 1 if preds.get('dlinear', {}).get('prediction') == actual_result else 0,
# N-BEATS
"nbeats_prediction": preds.get('nbeats', {}).get('prediction'),
"nbeats_prob": preds.get('nbeats', {}).get('probability'),
"nbeats_correct": 1 if preds.get('nbeats', {}).get('prediction') == actual_result else 0,
# TTM-R2
"ttm_prediction": preds.get('ttm', {}).get('prediction'),
"ttm_prob": preds.get('ttm', {}).get('probability'),
"ttm_correct": 1 if preds.get('ttm', {}).get('prediction') == actual_result else 0,
# Ensemble
"ensemble_prediction": preds.get('ensemble', {}).get('prediction'),
"ensemble_votes": preds.get('ensemble', {}).get('votes'),
"ensemble_correct": 1 if preds.get('ensemble', {}).get('prediction') == actual_result else 0,
}
PredictionModel.save_prediction(prediction_data)
PredictionModel.save_hourly_prediction(prediction_data)
except Exception as e:
print(f"生成预测失败 ({date} {hour}:00 {model_name}): {e}")
print(f"生成预测失败 ({date} {hour}:00): {e}")
def ensure_predictions_for_month(self, month: str) -> None:
"""
确保指定月份有预测数据
确保指定月份有预测数据 (CST)
需要检查:
1. 该月的所有日期 (ET)
2. 上个月的最后一天 (ET) -> 因为 ET 上个月最后一天 11:00-23:00 对应 CST 该月第一天 00:00-12:00
Args:
month: 月份 (YYYY-MM)
"""
# 使用锁避免并发冲突
with PredictionService._prediction_lock:
# 检查是否正在处理该月份
if month in self._processing_months:
return
# 标记正在处理
self._processing_months.add(month)
try:
# 解析月份
year, mon = map(int, month.split('-'))
# 计算该月的日期范围
from calendar import monthrange
_, last_day = monthrange(year, mon)
# 对于过去的日期,确保有预测数据
# 1. 确保上个月最后一天有数据 (为了 CST 的开头)
first_day_of_month = datetime(year, mon, 1).date()
prev_day = first_day_of_month - timedelta(days=1)
prev_day_str = prev_day.strftime('%Y-%m-%d')
self.ensure_predictions_for_date(prev_day_str)
today = datetime.now(self.et_tz).date()
# 2. 确保本月所有日期
for day in range(1, last_day + 1):
date = f"{year}-{mon:02d}-{day:02d}"
date_obj = datetime.strptime(date, '%Y-%m-%d').date()
# 只处理过去和今天的日期
if date_obj <= today:
self.ensure_predictions_for_date(date)
finally:
# 移除处理标记
with PredictionService._prediction_lock:
self._processing_months.discard(month)
+132 -78
View File
@@ -74,8 +74,8 @@
</div>
<div class="col-md-2">
<div class="stat-card">
<div class="stat-label">价格</div>
<div class="stat-value">${{ currentTrade.price?.toFixed(3) || '-' }}</div>
<div class="stat-label">策略</div>
<div class="stat-value small">{{ currentTrade.strategy || 'Ensemble' }}</div>
</div>
</div>
<div class="col-md-3">
@@ -116,6 +116,7 @@
<tr>
<th>时间 (北京)</th>
<th>市场</th>
<th>策略</th>
<th>预测</th>
<th>概率</th>
<th>价格</th>
@@ -128,6 +129,9 @@
<td class="text-truncate" style="max-width: 200px;">
{{ trade.market_title }}
</td>
<td>
<span class="badge bg-secondary">{{ trade.strategy || 'Ensemble' }}</span>
</td>
<td :class="getPredictionClass(trade.prediction)">
<i :class="trade.prediction === 'UP' ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
{{ trade.prediction }}
@@ -139,7 +143,7 @@
</td>
</tr>
<tr v-if="!tradeHistory.data?.length">
<td colspan="6" class="text-center text-muted py-4">
<td colspan="7" class="text-center text-muted py-4">
暂无数据
</td>
</tr>
@@ -184,66 +188,80 @@
<!-- 当月汇总卡片 -->
<div class="row mb-4">
<!-- DLinear 卡片 -->
<div class="col-md-6">
<div class="card h-100 border-primary">
<div class="card-header bg-primary text-white">
<i class="bi bi-graph-up-arrow"></i> DLinear 模型
<!-- Ensemble 卡片 -->
<div class="col-md-3">
<div class="card h-100 border-primary shadow-sm">
<div class="card-header bg-primary text-white text-center">
<i class="bi bi-diagram-3"></i> 集成策略 (投票)
</div>
<div class="card-body">
<div class="row text-center" v-if="monthlyStats.DLinear">
<div class="col-3">
<div class="stat-number">{{ monthlyStats.DLinear.total }}</div>
<div class="stat-label">预测次数</div>
</div>
<div class="col-3">
<div class="stat-number text-success">{{ monthlyStats.DLinear.correct }}</div>
<div class="stat-label">正确</div>
</div>
<div class="col-3">
<div class="stat-number text-danger">{{ monthlyStats.DLinear.incorrect }}</div>
<div class="stat-label">错误</div>
</div>
<div class="col-3">
<div class="stat-number text-primary">{{ monthlyStats.DLinear.accuracy }}%</div>
<div class="stat-label">准确率</div>
<div class="text-center" v-if="monthlyStats.Ensemble">
<h2 class="display-6 text-primary fw-bold">{{ monthlyStats.Ensemble.accuracy }}%</h2>
<div class="text-muted mb-2">准确率</div>
<div class="d-flex justify-content-between px-3 border-top pt-2">
<span class="text-success"><i class="bi bi-check-lg"></i> {{ monthlyStats.Ensemble.correct }}</span>
<span class="text-secondary">{{ monthlyStats.Ensemble.total }} 次</span>
<span class="text-danger"><i class="bi bi-x-lg"></i> {{ monthlyStats.Ensemble.incorrect }}</span>
</div>
</div>
<div v-else class="text-center text-muted py-3">
暂无数据
<div v-else class="text-center text-muted py-3">暂无数据</div>
</div>
</div>
</div>
<!-- DLinear 卡片 -->
<div class="col-md-3">
<div class="card h-100 border-secondary">
<div class="card-header bg-secondary text-white text-center">
<i class="bi bi-graph-up-arrow"></i> DLinear
</div>
<div class="card-body">
<div class="text-center" v-if="monthlyStats.DLinear">
<h3 class="text-dark">{{ monthlyStats.DLinear.accuracy }}%</h3>
<div class="text-muted small">准确率</div>
<div class="mt-2 small text-muted">
{{ monthlyStats.DLinear.correct }}/{{ monthlyStats.DLinear.total }}
</div>
</div>
<div v-else class="text-center text-muted py-3">暂无数据</div>
</div>
</div>
</div>
<!-- N-BEATS 卡片 -->
<div class="col-md-3">
<div class="card h-100 border-secondary">
<div class="card-header bg-secondary text-white text-center">
<i class="bi bi-soundwave"></i> N-BEATS
</div>
<div class="card-body">
<div class="text-center" v-if="monthlyStats['N-BEATS']">
<h3 class="text-dark">{{ monthlyStats['N-BEATS'].accuracy }}%</h3>
<div class="text-muted small">准确率</div>
<div class="mt-2 small text-muted">
{{ monthlyStats['N-BEATS'].correct }}/{{ monthlyStats['N-BEATS'].total }}
</div>
</div>
<div v-else class="text-center text-muted py-3">暂无数据</div>
</div>
</div>
</div>
<!-- TTM-R2 卡片 -->
<div class="col-md-6">
<div class="card h-100 border-success">
<div class="card-header bg-success text-white">
<i class="bi bi-cpu"></i> TTM-R2 模型
<div class="col-md-3">
<div class="card h-100 border-secondary">
<div class="card-header bg-secondary text-white text-center">
<i class="bi bi-cpu"></i> TTM-R2
</div>
<div class="card-body">
<div class="row text-center" v-if="monthlyStats['TTM-R2']">
<div class="col-3">
<div class="stat-number">{{ monthlyStats['TTM-R2'].total }}</div>
<div class="stat-label">预测次数</div>
</div>
<div class="col-3">
<div class="stat-number text-success">{{ monthlyStats['TTM-R2'].correct }}</div>
<div class="stat-label">正确</div>
</div>
<div class="col-3">
<div class="stat-number text-danger">{{ monthlyStats['TTM-R2'].incorrect }}</div>
<div class="stat-label">错误</div>
</div>
<div class="col-3">
<div class="stat-number text-primary">{{ monthlyStats['TTM-R2'].accuracy }}%</div>
<div class="stat-label">准确率</div>
<div class="text-center" v-if="monthlyStats['TTM-R2']">
<h3 class="text-dark">{{ monthlyStats['TTM-R2'].accuracy }}%</h3>
<div class="text-muted small">准确率</div>
<div class="mt-2 small text-muted">
{{ monthlyStats['TTM-R2'].correct }}/{{ monthlyStats['TTM-R2'].total }}
</div>
</div>
<div v-else class="text-center text-muted py-3">
暂无数据
</div>
<div v-else class="text-center text-muted py-3">暂无数据</div>
</div>
</div>
</div>
@@ -252,52 +270,61 @@
<!-- 每日准确率表格 -->
<div class="card">
<div class="card-header">
<i class="bi bi-calendar3"></i> 每日准确率
<i class="bi bi-calendar3"></i> 每日准确率概览
</div>
<div class="card-body p-0">
<div class="table-responsive">
<table class="table table-hover mb-0">
<table class="table table-hover mb-0 text-center">
<thead class="table-light">
<tr>
<th>日期</th>
<th>DLinear 预测</th>
<th>DLinear 准确率</th>
<th>TTM-R2 预测</th>
<th>TTM-R2 准确率</th>
<th class="text-start ps-4">日期</th>
<th>集成策略</th>
<th>DLinear</th>
<th>N-BEATS</th>
<th>TTM-R2</th>
<th>操作</th>
</tr>
</thead>
<tbody>
<tr v-for="day in dailyStats.data" :key="day.date">
<td>{{ day.date }}</td>
<td class="text-start ps-4">{{ day.date }}</td>
<!-- Ensemble -->
<td>
<span v-if="day.DLinear">
{{ day.DLinear.correct }}/{{ day.DLinear.total }}
<span v-if="day.Ensemble" :class="getAccuracyClass(day.Ensemble.accuracy)">
{{ day.Ensemble.accuracy }}%
</span>
<span v-else class="text-muted">-</span>
</td>
<!-- DLinear -->
<td>
<span v-if="day.DLinear" :class="getAccuracyClass(day.DLinear.accuracy)">
{{ day.DLinear.accuracy }}%
</span>
<span v-else class="text-muted">-</span>
</td>
<!-- N-BEATS -->
<td>
<span v-if="day['TTM-R2']">
{{ day['TTM-R2'].correct }}/{{ day['TTM-R2'].total }}
<span v-if="day['N-BEATS']" :class="getAccuracyClass(day['N-BEATS'].accuracy)">
{{ day['N-BEATS'].accuracy }}%
</span>
<span v-else class="text-muted">-</span>
</td>
<!-- TTM-R2 -->
<td>
<span v-if="day['TTM-R2']" :class="getAccuracyClass(day['TTM-R2'].accuracy)">
{{ day['TTM-R2'].accuracy }}%
</span>
<span v-else class="text-muted">-</span>
</td>
<td>
<button class="btn btn-sm btn-outline-primary"
@click="showHourlyDetails(day.date)">
<i class="bi bi-eye"></i> 查看
<i class="bi bi-eye"></i> 详情
</button>
</td>
</tr>
@@ -337,52 +364,79 @@
<!-- 每小时详情弹窗 -->
<div class="modal fade" id="hourlyModal" tabindex="-1">
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-dialog modal-xl modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title">
<i class="bi bi-clock"></i> {{ hourlyModal.date }} 每小时预测详情
<i class="bi bi-clock"></i> {{ hourlyModal.date }} 每小时预测详情 (北京时间)
</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
</div>
<div class="modal-body p-0">
<table class="table table-sm table-hover mb-0">
<table class="table table-sm table-hover mb-0 text-center align-middle">
<thead class="table-light sticky-top">
<tr>
<th>时间 (北京)</th>
<th>实际涨跌</th>
<th>时间</th>
<th>开盘</th>
<th>收盘</th>
<th>实际</th>
<th class="table-primary">集成策略</th>
<th>DLinear</th>
<th>N-BEATS</th>
<th>TTM-R2</th>
</tr>
</thead>
<tbody>
<tr v-for="hour in hourlyModal.data" :key="hour.hour">
<td>{{ hour.timestamp_cst || (hour.hour + ':00') }}</td>
<td :class="hour.actual_result === 1 ? 'text-success' : 'text-danger'">
<i :class="hour.actual_result === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
{{ hour.actual_result === 1 ? 'UP' : 'DOWN' }}
<td>{{ hour.timestamp_cst ? hour.timestamp_cst.split(' ')[1] : (hour.hour + ':00') }}</td>
<td>${{ hour.open_price?.toFixed(0) || '-' }}</td>
<td>${{ hour.close_price?.toFixed(0) || '-' }}</td>
<td :class="hour.actual_result === 1 ? 'text-success fw-bold' : (hour.actual_result === 0 ? 'text-danger fw-bold' : 'text-muted')">
<i v-if="hour.actual_result !== null" :class="hour.actual_result === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
{{ hour.actual_result === 1 ? 'UP' : (hour.actual_result === 0 ? 'DOWN' : '-') }}
</td>
<!-- Ensemble -->
<td class="table-primary bg-opacity-10">
<span v-if="hour.models?.Ensemble"
:class="hour.models.Ensemble.is_correct ? 'text-success' : 'text-danger'">
<i :class="hour.models.Ensemble.prediction === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
{{ hour.models.Ensemble.prediction === 1 ? 'UP' : 'DOWN' }}
<i v-if="hour.actual_result !== null" :class="hour.models.Ensemble.is_correct ? 'bi-check-circle-fill' : 'bi-x-circle-fill'"></i>
</span>
</td>
<!-- DLinear -->
<td>
<span v-if="hour.models?.DLinear"
:class="hour.models.DLinear.is_correct ? 'text-success' : 'text-danger'">
<i :class="hour.models.DLinear.prediction === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
{{ hour.models.DLinear.prediction === 1 ? 'UP' : 'DOWN' }}
<i :class="hour.models.DLinear.is_correct ? 'bi-check-circle' : 'bi-x-circle'"></i>
<span class="small text-muted">({{ (hour.models.DLinear.probability * 100).toFixed(0) }}%)</span>
<i v-if="hour.actual_result !== null" :class="hour.models.DLinear.is_correct ? 'bi-check' : 'bi-x'"></i>
</span>
<span v-else class="text-muted">-</span>
</td>
<!-- N-BEATS -->
<td>
<span v-if="hour.models?.['N-BEATS']"
:class="hour.models['N-BEATS'].is_correct ? 'text-success' : 'text-danger'">
<i :class="hour.models['N-BEATS'].prediction === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
<i v-if="hour.actual_result !== null" :class="hour.models['N-BEATS'].is_correct ? 'bi-check' : 'bi-x'"></i>
</span>
</td>
<!-- TTM-R2 -->
<td>
<span v-if="hour.models?.['TTM-R2']"
:class="hour.models['TTM-R2'].is_correct ? 'text-success' : 'text-danger'">
<i :class="hour.models['TTM-R2'].prediction === 1 ? 'bi-arrow-up' : 'bi-arrow-down'"></i>
{{ hour.models['TTM-R2'].prediction === 1 ? 'UP' : 'DOWN' }}
<i :class="hour.models['TTM-R2'].is_correct ? 'bi-check-circle' : 'bi-x-circle'"></i>
<i v-if="hour.actual_result !== null" :class="hour.models['TTM-R2'].is_correct ? 'bi-check' : 'bi-x'"></i>
</span>
<span v-else class="text-muted">-</span>
</td>
</tr>
<tr v-if="!hourlyModal.data?.length">
<td colspan="4" class="text-center text-muted py-4">
<td colspan="8" class="text-center text-muted py-4">
暂无数据
</td>
</tr>
-940
View File
@@ -1,940 +0,0 @@
"""
BTC Hourly 自动交易脚本
基于DLinear模型预测,在Polymarket自动下单BTC小时涨跌
支持:
- 每小时自动寻找对应市场
- 实时价格刷新
- 定时下单
- 详细日志记录
"""
import os
import sys
import argparse
import json
import time
import logging
from logging.handlers import RotatingFileHandler
import schedule
from datetime import datetime, timedelta
from typing import Optional, Tuple, Dict
import pickle
from zoneinfo import ZoneInfo
import torch
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
from dotenv import load_dotenv
import httpx
# 添加项目路径
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from data.fetcher import BinanceDataFetcher
from data.trading_db import TradingDatabase
load_dotenv()
def setup_logging(log_dir: str = "logs", log_level: int = logging.INFO) -> logging.Logger:
"""
配置日志系统
Args:
log_dir: 日志目录
log_level: 日志级别
Returns:
配置好的logger
"""
# 创建日志目录
os.makedirs(log_dir, exist_ok=True)
# 创建logger
logger = logging.getLogger("polymarket_trader")
logger.setLevel(log_level)
# 避免重复添加handler
if logger.handlers:
return logger
# 日志格式
formatter = logging.Formatter(
"%(asctime)s | %(levelname)-8s | %(message)s",
datefmt="%Y-%m-%d %H:%M:%S"
)
# 文件handler (按大小轮转, 最大10MB, 保留5个备份)
log_file = os.path.join(log_dir, "polymarket_trading.log")
file_handler = RotatingFileHandler(
log_file,
maxBytes=10*1024*1024, # 10MB
backupCount=5,
encoding='utf-8'
)
file_handler.setLevel(log_level)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
# 控制台handler
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setLevel(log_level)
console_handler.setFormatter(formatter)
logger.addHandler(console_handler)
return logger
# 初始化全局logger
logger = setup_logging()
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback: int = 168, forecast_horizon: int = 1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df: pd.DataFrame) -> pd.DataFrame:
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df: pd.DataFrame) -> pd.DataFrame:
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df: pd.DataFrame, fit: bool = True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
class PolymarketBTCTrader:
"""Polymarket BTC Hourly交易器"""
GAMMA_API = "https://gamma-api.polymarket.com"
CLOB_API = "https://clob.polymarket.com"
# 月份名称映射
MONTH_NAMES = {
1: 'january', 2: 'february', 3: 'march', 4: 'april',
5: 'may', 6: 'june', 7: 'july', 8: 'august',
9: 'september', 10: 'october', 11: 'november', 12: 'december'
}
def __init__(
self,
private_key: Optional[str] = None,
max_bet: float = 1.0,
max_price: float = 0.50,
dry_run: bool = True,
):
self.private_key = private_key or os.getenv("POLYMARKET_PRIVATE_KEY")
self.max_bet = max_bet
self.max_price = max_price
self.dry_run = dry_run
# HTTP客户端
proxy = os.getenv("SOCKS5_PROXY") if os.getenv("USE_PROXY", "false").lower() == "true" else None
self.http = httpx.Client(
timeout=30.0,
proxy=proxy,
headers={'Accept-Encoding': 'identity'}
)
# 设备
self.device = torch.device('mps' if torch.backends.mps.is_available() else
'cuda' if torch.cuda.is_available() else 'cpu')
# 加载模型
self.model = None
self.preprocessor = None
self._load_model()
# 数据获取器
self.fetcher = BinanceDataFetcher()
# 数据库
self.db = TradingDatabase()
# CLOB客户端(用于下单)
self.clob_client = None
if self.private_key and not self.dry_run:
self._init_clob_client()
# 当前市场缓存
self.current_market = None
self.last_price_update = None
def _load_model(self):
"""加载DLinear模型"""
logger.info("加载DLinear模型...")
from models.dlinear.model import DLinear
checkpoint_path = './models/checkpoints/dlinear_classification_2026/best_model.pt'
preprocessor_path = './models/checkpoints/dlinear_classification_2026/preprocessor.pkl'
checkpoint = torch.load(checkpoint_path, map_location=self.device, weights_only=False)
self.model = DLinear(checkpoint['config'])
self.model.load_state_dict(checkpoint['model_state_dict'])
self.model = self.model.to(self.device).eval()
with open(preprocessor_path, 'rb') as f:
self.preprocessor = pickle.load(f)
logger.info(f"✓ 模型已加载 (设备: {self.device})")
def _init_clob_client(self):
"""初始化CLOB交易客户端"""
try:
from py_clob_client.client import ClobClient
self.clob_client = ClobClient(
host=self.CLOB_API,
key=self.private_key,
chain_id=137, # Polygon
)
self.clob_client.set_api_creds(self.clob_client.create_or_derive_api_creds())
logger.info("✓ CLOB客户端初始化成功")
except Exception as e:
logger.warning(f"⚠️ CLOB客户端初始化失败: {e}")
self.clob_client = None
def _log_to_db(self, result: Dict, market: Optional[Dict]):
"""记录交易结果到数据库"""
try:
# 确定状态
status = 'failure'
if self.dry_run:
status = 'test'
elif result.get('success'):
status = 'success'
elif result.get('error'):
status = 'failure'
elif result.get('prediction'): # 有预测但未交易(如市场未找到)
status = 'in_progress' # 或者 failure? 未找到市场算 failure
if result.get('error'):
status = 'failure'
log_data = {
'timestamp': result.get('timestamp'),
'market_title': result.get('market'),
'market_slug': market.get('slug') if market else result.get('market_slug'),
'model_name': 'DLinear',
'prediction': result.get('prediction'),
'probability': result.get('probability'),
'action': result.get('side'),
'price': result.get('price'),
'amount': result.get('amount'),
'size': result.get('size'),
'status': status,
'error_message': result.get('error'),
'raw_data': result
}
self.db.log_trade(log_data)
logger.info(f"💾 交易记录已保存到数据库")
except Exception as e:
logger.error(f"❌ 保存数据库失败: {e}")
def generate_market_slug(self, target_time: Optional[datetime] = None) -> str:
"""
生成市场slug
格式: bitcoin-up-or-down-{month}-{day}-{hour}am/pm-et
例如: bitcoin-up-or-down-february-3-8am-et
Args:
target_time: 目标时间(ET时区)
for_current_hour: 如果True,使用当前整点;否则使用下一个整点
Returns:
市场slug
"""
# 获取美东时间
et_tz = ZoneInfo('America/New_York')
if target_time is None:
now_et = datetime.now(et_tz)
target_time = now_et.replace(minute=0, second=0, microsecond=0)
month_name = self.MONTH_NAMES[target_time.month]
day = target_time.day
# 转换为12小时制
hour_24 = target_time.hour
if hour_24 == 0:
hour_12 = 12
am_pm = 'am'
elif hour_24 < 12:
hour_12 = hour_24
am_pm = 'am'
elif hour_24 == 12:
hour_12 = 12
am_pm = 'pm'
else:
hour_12 = hour_24 - 12
am_pm = 'pm'
slug = f"bitcoin-up-or-down-{month_name}-{day}-{hour_12}{am_pm}-et"
return slug
def get_market_by_slug(self, slug: str) -> Optional[Dict]:
"""
通过slug获取市场信息
Args:
slug: 市场slug
Returns:
市场信息
"""
try:
response = self.http.get(
f"{self.GAMMA_API}/events",
params={'slug': slug}
)
response.raise_for_status()
events = response.json()
if not events:
return None
event = events[0]
markets = event.get('markets', [])
if not markets:
return None
market = markets[0]
# outcomePrices可能是JSON字符串或列表
outcome_prices_raw = market.get('outcomePrices', ['0', '0'])
if isinstance(outcome_prices_raw, str):
import json as _json
try:
outcome_prices_raw = _json.loads(outcome_prices_raw)
except:
outcome_prices_raw = ['0', '0']
# clobTokenIds也可能是JSON字符串
clob_tokens_raw = market.get('clobTokenIds', [])
if isinstance(clob_tokens_raw, str):
import json as _json
try:
clob_tokens_raw = _json.loads(clob_tokens_raw)
except:
clob_tokens_raw = []
return {
'title': event.get('title', ''),
'slug': slug,
'question': market.get('question', ''),
'condition_id': market.get('conditionId', ''),
'outcomes': market.get('outcomes', []),
'outcome_prices': [float(p) for p in outcome_prices_raw],
'tokens': clob_tokens_raw,
'end_date': event.get('endDate', ''),
'closed': market.get('closed', False),
}
except Exception as e:
logger.error(f"获取市场失败: {e}")
return None
def get_current_hourly_market(self) -> Optional[Dict]:
"""
获取BTC市场
Args:
for_current_hour: 如果True,获取当前整点市场;否则获取下一个整点市场
Returns:
市场信息
"""
slug = self.generate_market_slug()
logger.info(f"搜索市场: {slug}")
market = self.get_market_by_slug(slug)
if market:
self.current_market = market
self.last_price_update = datetime.now()
logger.info(f"✓ 找到市场: {market['title']}")
logger.info(f" Up价格: ${market['outcome_prices'][0]:.3f}")
logger.info(f" Down价格: ${market['outcome_prices'][1]:.3f}")
else:
logger.warning(f"❌ 未找到市场: {slug}")
return market
def refresh_prices(self) -> Optional[Dict]:
"""
刷新当前市场价格
Returns:
更新后的市场信息
"""
if not self.current_market:
self.get_current_hourly_market()
# 使用CLOB API获取实时价格
tokens = self.current_market.get('tokens', [])
if len(tokens) >= 2:
realtime_prices = self.get_realtime_prices(tokens[0], tokens[1])
if realtime_prices:
old_prices = self.current_market['outcome_prices']
new_prices = realtime_prices
self.current_market['outcome_prices'] = new_prices
self.last_price_update = datetime.now()
# 显示价格变化
up_change = new_prices[0] - old_prices[0]
down_change = new_prices[1] - old_prices[1]
logger.info(f"📊 实时价格 [{datetime.now().strftime('%H:%M:%S')}]")
logger.info(f" Up: ${new_prices[0]:.3f} ({up_change:+.3f})")
logger.info(f" Down: ${new_prices[1]:.3f} ({down_change:+.3f})")
return self.current_market
# 备用: 使用Gamma API
slug = self.current_market['slug']
market = self.get_market_by_slug(slug)
if market:
old_prices = self.current_market['outcome_prices']
new_prices = market['outcome_prices']
self.current_market = market
self.last_price_update = datetime.now()
# 显示价格变化
up_change = new_prices[0] - old_prices[0]
down_change = new_prices[1] - old_prices[1]
logger.info(f"📊 价格更新(Gamma) [{datetime.now().strftime('%H:%M:%S')}]")
logger.info(f" Up: ${new_prices[0]:.3f} ({up_change:+.3f})")
logger.info(f" Down: ${new_prices[1]:.3f} ({down_change:+.3f})")
return market
def get_realtime_prices(self, up_token_id: str, down_token_id: str) -> Optional[list]:
"""
从CLOB API获取实时买入价格
使用/price?side=BUY端点获取真实买入价格 (与页面显示一致)
Args:
up_token_id: Up代币ID
down_token_id: Down代币ID
Returns:
[up_price, down_price] 或 None
"""
try:
# 使用/price端点获取买入价格
up_response = self.http.get(
f"{self.CLOB_API}/price",
params={'token_id': up_token_id, 'side': 'BUY'}
)
up_response.raise_for_status()
up_data = up_response.json()
down_response = self.http.get(
f"{self.CLOB_API}/price",
params={'token_id': down_token_id, 'side': 'BUY'}
)
down_response.raise_for_status()
down_data = down_response.json()
# 解析买入价格
up_price = float(up_data.get('price', 0.5))
down_price = float(down_data.get('price', 0.5))
return [up_price, down_price]
except Exception as e:
logger.debug(f"获取实时价格失败: {e}")
return None
def execute_trade(self, prediction: int, market: Dict) -> Dict:
"""
执行交易
Args:
prediction: 预测结果(0=跌, 1=涨)
market: 市场信息
Returns:
交易结果
"""
result = {
'timestamp': datetime.now().isoformat(),
'market': market['title'],
'prediction': 'UP' if prediction == 1 else 'DOWN',
'success': False,
'order': None,
'error': None,
}
if prediction == 1: # 预测涨 -> 买UP
token_id = market['tokens'][0]
price = market['outcome_prices'][0]
side = "UP"
else: # 预测跌 -> 买DOWN
token_id = market['tokens'][1]
price = market['outcome_prices'][1]
side = "DOWN"
result['side'] = side
result['price'] = price
# 检查价格限制
if price > self.max_price:
result['error'] = f"价格${price:.2f}超过限制${self.max_price:.2f}"
logger.error(f"❌ {result['error']}")
return result
# 计算数量
size = self.max_bet / price
result['size'] = size
result['amount'] = self.max_bet
logger.info("📋 交易详情:")
logger.info(f" 方向: {side}")
logger.info(f" 价格: ${price:.3f}")
logger.info(f" 金额: ${self.max_bet:.2f} USDC")
logger.info(f" 数量: {size:.2f} 股")
if self.dry_run:
logger.info("🔸 [模拟模式] 未实际下单")
result['success'] = True
result['order'] = 'DRY_RUN'
return result
# 实际下单
if not self.clob_client:
result['error'] = "CLOB客户端未初始化"
logger.error(f"❌ {result['error']}")
return result
try:
from py_clob_client.order_builder.constants import BUY
order = self.clob_client.create_limit_order(
token_id=token_id,
price=price,
size=size,
side=BUY,
)
response = self.clob_client.post_order(order)
logger.info(f"✓ 订单已提交: {response}")
result['success'] = True
result['order'] = response
except Exception as e:
result['error'] = str(e)
logger.error(f"❌ 下单失败: {e}")
return result
def get_prediction_with_delay(self, delay_seconds: float = 2.0) -> Tuple[int, float]:
"""
延迟后获取K线并预测当前小时的涨跌
注意: 使用上一小时收盘的K线(倒数第二根)来预测当前小时
例如: 22:00整点时,用21:00收盘的K线预测22:00这个小时的涨跌
Args:
delay_seconds: 延迟秒数(等待K线更新)
Returns:
(prediction, probability): 预测结果(0=跌, 1=涨)和概率
"""
logger.info(f"⏳ 等待{delay_seconds}秒获取最新K线...")
time.sleep(delay_seconds)
logger.info("获取最新BTC数据...")
end_date = (datetime.now() + timedelta(days=1)).strftime('%Y-%m-%d')
start_date = (datetime.now() - timedelta(days=30)).strftime('%Y-%m-%d')
df = self.fetcher.fetch_ohlcv(
symbol='BTC',
timeframe='1h',
start_date=start_date,
end_date=end_date,
use_cache=False # 不使用缓存,获取最新数据
)
# 使用倒数第二根K线(上一小时收盘)进行预测
# 例如22:00时,最新K线是14:00 UTC(22:00北京),我们用13:00 UTC(21:00北京)的数据
prev_time = df['timestamp'].iloc[-2]
prev_close = df['close'].iloc[-2]
current_time = df['timestamp'].iloc[-1]
current_close = df['close'].iloc[-1]
logger.info(f" 上一K线时间: {prev_time} (用于预测)")
logger.info(f" 上一收盘价: ${prev_close:,.2f}")
logger.info(f" 当前K线时间: {current_time} (预测目标)")
logger.info(f" 当前收盘价: ${current_close:,.2f} (实际结果)")
# 用除最后一根外的数据进行预测
df_for_prediction = df.iloc[:-1]
X, _, _ = self.preprocessor.preprocess(df_for_prediction, fit=False)
X_last = torch.FloatTensor(X[-1:]).to(self.device)
with torch.no_grad():
logits = self.model(X_last).cpu().numpy()
prob = 1 / (1 + np.exp(-logits))
prediction = int(prob.item() > 0.5)
probability = float(prob.item())
direction = "📈 UP (涨)" if prediction == 1 else "📉 DOWN (跌)"
confidence = probability if prediction == 1 else 1 - probability
logger.info(f"🤖 DLinear预测: {direction}")
logger.info(f" 置信度: {confidence:.2%}")
# 显示预测是否正确 (用于验证)
actual_up = current_close > prev_close
if prediction == 1:
correct = actual_up
else:
correct = not actual_up
logger.info(f" 实际涨跌: {'📈 涨' if actual_up else '📉 跌'} ({'✅ 正确' if correct else '❌ 错误'})")
return prediction, probability
def run_once(self) -> Dict:
"""运行一次交易流程(不等待整点)"""
logger.info("运行单次交易...")
return self.run_at_hour_start(delay_seconds=0)
def run_at_hour_start(self, delay_seconds: float = 2.0, timeout_minutes: int = 50) -> Dict:
"""
整点时执行的交易流程
1. 延迟delay_seconds秒等待K线更新
2. 获取最新K线
3. DLinear预测当前整点的收盘涨跌
4. 获取对应的Polymarket市场
5. 循环检查价格:
- 如果价格 <= max_price: 下单并结束
- 如果价格 > max_price: 等待并重试(直到超时)
Args:
delay_seconds: 整点后延迟秒数
timeout_minutes: 价格等待超时时间(分钟)
Returns:
执行结果
"""
logger.info("=" * 60)
logger.info(f"⏰ 整点交易触发 - {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
logger.info("=" * 60)
logger.info(f" 模式: {'🔸 模拟' if self.dry_run else '💰 真实交易'}")
logger.info(f" 最大下注: ${self.max_bet:.2f} USDC")
logger.info(f" 最高价格: ${self.max_price:.2f}")
logger.info(f" 等待超时: {timeout_minutes}分钟")
result = {
'timestamp': datetime.now().isoformat(),
'success': False,
'error': None,
}
# 1. 延迟后获取预测
prediction, probability = self.get_prediction_with_delay(delay_seconds)
result['prediction'] = 'UP' if prediction == 1 else 'DOWN'
result['probability'] = probability if prediction == 1 else 1 - probability
# 2. 获取初始市场
market = self.get_current_hourly_market()
if not market:
result['error'] = '未找到市场'
logger.error(f"❌ {result['error']}")
self._log_to_db(result, None)
return result
result['market'] = market['title']
result['market_slug'] = market['slug']
if market['closed']:
result['error'] = '市场已关闭'
logger.error(f"❌ {result['error']}")
self._log_to_db(result, market)
return result
# 检查是否已交易过该市场
check_status = 'test' if self.dry_run else 'success'
if self.db.check_existing_trade(market['slug'], status=check_status):
result['error'] = '该市场已存在成功交易记录'
logger.warning(f"⚠️ {result['error']}, 跳过交易")
return result
# 3. 循环检查价格并交易
start_time = datetime.now()
end_time = start_time + timedelta(minutes=timeout_minutes)
logger.info(f"🔄 开始价格监控 (预测: {'UP' if prediction == 1 else 'DOWN'})...")
while datetime.now() < end_time:
# 刷新价格
market = self.refresh_prices()
if not market:
logger.warning("无法获取市场价格,重试中...")
time.sleep(10)
continue
# 检查价格
price_idx = 0 if prediction == 1 else 1
current_price = market['outcome_prices'][price_idx]
logger.info(f" 当前价格: ${current_price:.3f} (目标: <= ${self.max_price:.2f})")
if current_price <= self.max_price:
# 价格符合,执行交易
logger.info("✅ 价格满足条件,正在下单...")
trade_result = self.execute_trade(prediction, market)
result.update(trade_result)
# 记录数据库
self._log_to_db(result, market)
logger.info("=" * 60)
return result
else:
# 价格过高,等待
wait_time = 30 # 等待30秒
elapsed = int((datetime.now() - start_time).total_seconds())
logger.info(f"⏳ 价格过高,继续等待... (已耗时: {elapsed}s)")
time.sleep(wait_time)
# 超时
result['error'] = f'等待价格超时 ({timeout_minutes}分钟)'
result['price'] = market['outcome_prices'][0] if prediction == 1 else market['outcome_prices'][1]
result['side'] = 'UP' if prediction == 1 else 'DOWN'
logger.error(f"❌ {result['error']}")
self._log_to_db(result, market)
logger.info("=" * 60)
return result
def _show_initial_prediction(self):
"""
启动时显示当前预测(不下单)
"""
logger.info("\n📊 当前预测状态:")
end_date = (datetime.now() + timedelta(days=1)).strftime('%Y-%m-%d')
start_date = (datetime.now() - timedelta(days=30)).strftime('%Y-%m-%d')
df = self.fetcher.fetch_ohlcv(
symbol='BTC',
timeframe='1h',
start_date=start_date,
end_date=end_date,
use_cache=False
)
# 显示K线信息
prev_time = df['timestamp'].iloc[-2]
prev_close = df['close'].iloc[-2]
current_time = df['timestamp'].iloc[-1]
current_close = df['close'].iloc[-1]
logger.info(f" 上一K线: {prev_time} 收盘价: ${prev_close:,.2f}")
logger.info(f" 当前K线: {current_time} 收盘价: ${current_close:,.2f}")
# 进行预测
df_for_prediction = df.iloc[:-1]
X, _, _ = self.preprocessor.preprocess(df_for_prediction, fit=False)
X_last = torch.FloatTensor(X[-1:]).to(self.device)
with torch.no_grad():
logits = self.model(X_last).cpu().numpy()
prob = 1 / (1 + np.exp(-logits))
prediction = int(prob.item() > 0.5)
probability = float(prob.item())
direction = "📈 UP (涨)" if prediction == 1 else "📉 DOWN (跌)"
confidence = probability if prediction == 1 else 1 - probability
logger.info(f" 🤖 预测: {direction} (置信度: {confidence:.2%})")
# 显示市场价格
if self.current_market:
up_price = self.current_market['outcome_prices'][0]
down_price = self.current_market['outcome_prices'][1]
if prediction == 1:
logger.info(f" 💰 买入Up价格: ${up_price:.3f} {'✅ 可下单' if up_price <= self.max_price else '❌ 超过限价'}")
else:
logger.info(f" 💰 买入Down价格: ${down_price:.3f} {'✅ 可下单' if down_price <= self.max_price else '❌ 超过限价'}")
# 计算距下一整点时间
now = datetime.now()
next_hour = now.replace(minute=0, second=0, microsecond=0) + timedelta(hours=1)
wait_minutes = int((next_hour - now).total_seconds() / 60)
logger.info(f" ⏰ 距离下一整点: {wait_minutes}分钟")
logger.info("")
def wait_for_next_hour(self) -> datetime:
"""
等待到下一个整点
Returns:
下一个整点时间
"""
now = datetime.now()
next_hour = now.replace(minute=0, second=0, microsecond=0) + timedelta(hours=1)
wait_seconds = (next_hour - now).total_seconds()
logger.info(f"⏳ 等待下一个整点: {next_hour.strftime('%H:%M:%S')}")
logger.info(f" 剩余等待: {int(wait_seconds)}秒")
return next_hour
def run_scheduler(self, interval_minutes: int = 5, delay_seconds: float = 2.0):
"""
定时运行交易器
工作流程:
1. 每个整点+delay_seconds秒触发交易
2. 期间每interval_minutes分钟刷新价格
Args:
interval_minutes: 价格刷新间隔(分钟)
delay_seconds: 整点后延迟秒数
"""
logger.info("=" * 60)
logger.info("🤖 BTC Hourly自动交易已启动")
logger.info("=" * 60)
logger.info(f" 模式: {'🔸 模拟' if self.dry_run else '💰 真实交易'}")
logger.info(f" 下单时机: 每个整点后延迟{delay_seconds}秒")
logger.info(f" 价格刷新: 每{interval_minutes}分钟")
logger.info(f" 最大下注: ${self.max_bet:.2f} USDC")
logger.info(f" 最高价格: ${self.max_price:.2f}")
logger.info(" 按 Ctrl+C 停止")
logger.info("=" * 60)
# 立即获取整点的市场
self.get_current_hourly_market()
# 启动时进行一次预测(不下单,仅显示)
self._show_initial_prediction()
# 每小时整点后执行交易
def hourly_trade():
self.run_at_hour_start(delay_seconds=delay_seconds)
schedule.every().hour.at(":00").do(lambda: time.sleep(delay_seconds) or hourly_trade())
try:
while True:
schedule.run_pending()
time.sleep(1)
except KeyboardInterrupt:
logger.info("✓ 交易器已停止")
def close(self):
"""关闭资源"""
self.http.close()
def main():
parser = argparse.ArgumentParser(description='BTC Hourly自动交易 - 基于DLinear模型')
parser.add_argument('--dry-run', action='store_true', default=True,
help='模拟模式(不实际下单)')
parser.add_argument('--real', action='store_true',
help='真实交易模式')
parser.add_argument('--max-bet', type=float, default=1.0,
help='最大下注金额(USDC)')
parser.add_argument('--max-price', type=float, default=0.50,
help='最高可接受价格')
parser.add_argument('--once', action='store_true',
help='只执行一次(不启动定时器)')
parser.add_argument('--interval', type=int, default=1,
help='价格刷新间隔(分钟)')
parser.add_argument('--check-market', action='store_true',
help='只检查当前市场信息')
args = parser.parse_args()
dry_run = not args.real
# 真实交易确认
if not dry_run:
logger.warning("⚠️ 警告: 即将进行真实交易!")
logger.warning(f" 最大下注: ${args.max_bet:.2f} USDC")
confirm = input("确认继续? (yes/no): ")
if confirm.lower() != 'yes':
logger.info("已取消")
return
trader = PolymarketBTCTrader(
max_bet=args.max_bet,
max_price=args.max_price,
dry_run=dry_run,
)
try:
if args.check_market:
# 只检查市场
trader.get_current_hourly_market()
elif args.once:
# 执行一次
result = trader.run_once()
# 保存日志
log_file = 'trading_log.json'
logs = []
if os.path.exists(log_file):
with open(log_file, 'r') as f:
logs = json.load(f)
logs.append(result)
with open(log_file, 'w') as f:
json.dump(logs, f, indent=2, ensure_ascii=False)
logger.info(f"结果已保存到 {log_file}")
else:
# 定时运行
trader.run_scheduler(interval_minutes=args.interval)
finally:
trader.close()
if __name__ == "__main__":
main()
+4
View File
@@ -0,0 +1,4 @@
{
"account_id": "7ea8a511cad4d467490fe36aa7d6638c",
"project_name": "trades"
}
@@ -0,0 +1,6 @@
{
"account": {
"id": "7ea8a511cad4d467490fe36aa7d6638c",
"name": "Subuhuinew@gmail.com's Account"
}
}
+41
View File
@@ -0,0 +1,41 @@
# Cloudflare Pages 看板
本目录提供一个可直接部署到 Cloudflare Pages 的页面,数据源为 D1(由 MySQL 备份任务同步而来)。
## 目录结构
- `public/`:静态页面资源
- `functions/api/`:Pages Functions,直接查询 D1
## 必要前置
1. 已在 D1 中有备份表:
- `backup_trades`
- `backup_hourly_model_predictions`
2. Pages 项目绑定 D1(绑定名必须是 `DB`)
## 本地开发
```bash
cd cloudflare_pages
npx wrangler pages dev public --d1 DB=<你的D1数据库ID>
```
## 生产部署
1. 在 Cloudflare Pages 新建项目,根目录设置为 `cloudflare_pages`
2. Build command 留空
3. Build output directory 设置为 `public`
4. 在 Pages 项目 Settings -> Functions -> D1 bindings 中添加:
- Variable name: `DB`
- D1 database: 选择你的数据库
## 页面接口
- `/api/status`
- `/api/trades/current`
- `/api/trades/history?page=1&limit=15&month=YYYY-MM&day=YYYY-MM-DD`
- `/api/accuracy/monthly?month=YYYY-MM`
- `/api/accuracy/daily?month=YYYY-MM&page=1&limit=10`
- `/api/accuracy/hourly/YYYY-MM-DD`
@@ -0,0 +1,101 @@
function json(data, status = 200) {
return new Response(JSON.stringify(data), {
status,
headers: { "content-type": "application/json; charset=UTF-8" },
});
}
function toInt(v, d) {
const n = Number(v);
return Number.isFinite(n) && n > 0 ? Math.floor(n) : d;
}
function ratio(correct, total) {
if (!total) return 0;
return Math.round((correct / total) * 10000) / 100;
}
function item(total, correct) {
const t = Number(total || 0);
const c = Number(correct || 0);
return {
total: t,
correct: c,
incorrect: Math.max(0, t - c),
accuracy: ratio(c, t),
};
}
export async function onRequestGet(context) {
const db = context.env.DB;
if (!db) {
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
}
try {
const url = new URL(context.request.url);
const month = (url.searchParams.get("month") || new Date().toISOString().slice(0, 7)).trim();
const page = toInt(url.searchParams.get("page"), 1);
const limit = Math.min(100, toInt(url.searchParams.get("limit"), 10));
const offset = (page - 1) * limit;
const baseWhere = "date LIKE ?";
const monthArg = `${month}%`;
const dataSql = `
SELECT
date,
SUM(CASE WHEN actual_result IS NOT NULL AND ensemble_prediction IS NOT NULL THEN 1 ELSE 0 END) AS ens_total,
SUM(CASE WHEN ensemble_correct = 1 THEN 1 ELSE 0 END) AS ens_correct,
SUM(CASE WHEN actual_result IS NOT NULL AND dlinear_prediction IS NOT NULL THEN 1 ELSE 0 END) AS dl_total,
SUM(CASE WHEN dlinear_correct = 1 THEN 1 ELSE 0 END) AS dl_correct,
SUM(CASE WHEN actual_result IS NOT NULL AND nbeats_prediction IS NOT NULL THEN 1 ELSE 0 END) AS nb_total,
SUM(CASE WHEN nbeats_correct = 1 THEN 1 ELSE 0 END) AS nb_correct,
SUM(CASE WHEN actual_result IS NOT NULL AND ttm_prediction IS NOT NULL THEN 1 ELSE 0 END) AS ttm_total,
SUM(CASE WHEN ttm_correct = 1 THEN 1 ELSE 0 END) AS ttm_correct
FROM backup_hourly_model_predictions
WHERE ${baseWhere}
GROUP BY date
ORDER BY date DESC
LIMIT ? OFFSET ?
`;
const countSql = `
SELECT COUNT(*) AS cnt
FROM (
SELECT date
FROM backup_hourly_model_predictions
WHERE ${baseWhere}
GROUP BY date
) x
`;
const dataResult = await db.prepare(dataSql).bind(monthArg, limit, offset).all();
const countResult = await db.prepare(countSql).bind(monthArg).first();
const rows = (dataResult.results || []).map((r) => ({
date: r.date,
Ensemble: item(r.ens_total, r.ens_correct),
DLinear: item(r.dl_total, r.dl_correct),
"N-BEATS": item(r.nb_total, r.nb_correct),
"TTM-R2": item(r.ttm_total, r.ttm_correct),
}));
const total = Number(countResult?.cnt || 0);
const totalPages = total > 0 ? Math.ceil(total / limit) : 1;
return json({
success: true,
data: rows,
page,
limit,
total,
total_pages: totalPages,
});
} catch (error) {
return json({ success: false, error: String(error) }, 500);
}
}
@@ -0,0 +1,67 @@
function json(data, status = 200) {
return new Response(JSON.stringify(data), {
status,
headers: { "content-type": "application/json; charset=UTF-8" },
});
}
function modelItem(prediction, probability, isCorrect) {
if (prediction === null || prediction === undefined) return null;
return {
prediction: Number(prediction),
probability: probability === null || probability === undefined ? null : Number(probability),
is_correct: isCorrect === null || isCorrect === undefined ? null : Number(isCorrect) === 1,
};
}
export async function onRequestGet(context) {
const db = context.env.DB;
if (!db) {
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
}
try {
const date = context.params.date;
const sql = `
SELECT
timestamp,
hour,
open_price,
close_price,
actual_result,
dlinear_prediction,
dlinear_prob,
dlinear_correct,
nbeats_prediction,
nbeats_prob,
nbeats_correct,
ttm_prediction,
ttm_prob,
ttm_correct,
ensemble_prediction,
ensemble_correct
FROM backup_hourly_model_predictions
WHERE date = ?
ORDER BY hour ASC
`;
const result = await db.prepare(sql).bind(date).all();
const rows = (result.results || []).map((r) => ({
timestamp_cst: r.timestamp,
hour: r.hour,
open_price: r.open_price === null || r.open_price === undefined ? null : Number(r.open_price),
close_price: r.close_price === null || r.close_price === undefined ? null : Number(r.close_price),
actual_result: r.actual_result === null || r.actual_result === undefined ? null : Number(r.actual_result),
models: {
Ensemble: modelItem(r.ensemble_prediction, null, r.ensemble_correct),
DLinear: modelItem(r.dlinear_prediction, r.dlinear_prob, r.dlinear_correct),
"N-BEATS": modelItem(r.nbeats_prediction, r.nbeats_prob, r.nbeats_correct),
"TTM-R2": modelItem(r.ttm_prediction, r.ttm_prob, r.ttm_correct),
},
}));
return json({ success: true, date, data: rows });
} catch (error) {
return json({ success: false, error: String(error) }, 500);
}
}
@@ -0,0 +1,87 @@
function json(data, status = 200) {
return new Response(JSON.stringify(data), {
status,
headers: { "content-type": "application/json; charset=UTF-8" },
});
}
function ratio(correct, total) {
if (!total) return 0;
return Math.round((correct / total) * 10000) / 100;
}
export async function onRequestGet(context) {
const db = context.env.DB;
if (!db) {
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
}
try {
const url = new URL(context.request.url);
const month = (url.searchParams.get("month") || new Date().toISOString().slice(0, 7)).trim();
const sql = `
SELECT
SUM(CASE WHEN actual_result IS NOT NULL AND ensemble_prediction IS NOT NULL THEN 1 ELSE 0 END) AS ens_total,
SUM(CASE WHEN ensemble_correct = 1 THEN 1 ELSE 0 END) AS ens_correct,
SUM(CASE WHEN actual_result IS NOT NULL AND dlinear_prediction IS NOT NULL THEN 1 ELSE 0 END) AS dl_total,
SUM(CASE WHEN dlinear_correct = 1 THEN 1 ELSE 0 END) AS dl_correct,
SUM(CASE WHEN actual_result IS NOT NULL AND nbeats_prediction IS NOT NULL THEN 1 ELSE 0 END) AS nb_total,
SUM(CASE WHEN nbeats_correct = 1 THEN 1 ELSE 0 END) AS nb_correct,
SUM(CASE WHEN actual_result IS NOT NULL AND ttm_prediction IS NOT NULL THEN 1 ELSE 0 END) AS ttm_total,
SUM(CASE WHEN ttm_correct = 1 THEN 1 ELSE 0 END) AS ttm_correct
FROM backup_hourly_model_predictions
WHERE date LIKE ?
`;
const row = await db.prepare(sql).bind(`${month}%`).first();
const ensTotal = Number(row?.ens_total || 0);
const ensCorrect = Number(row?.ens_correct || 0);
const dlTotal = Number(row?.dl_total || 0);
const dlCorrect = Number(row?.dl_correct || 0);
const nbTotal = Number(row?.nb_total || 0);
const nbCorrect = Number(row?.nb_correct || 0);
const ttmTotal = Number(row?.ttm_total || 0);
const ttmCorrect = Number(row?.ttm_correct || 0);
return json({
success: true,
month,
data: {
Ensemble: {
total: ensTotal,
correct: ensCorrect,
incorrect: Math.max(0, ensTotal - ensCorrect),
accuracy: ratio(ensCorrect, ensTotal),
},
DLinear: {
total: dlTotal,
correct: dlCorrect,
incorrect: Math.max(0, dlTotal - dlCorrect),
accuracy: ratio(dlCorrect, dlTotal),
},
"N-BEATS": {
total: nbTotal,
correct: nbCorrect,
incorrect: Math.max(0, nbTotal - nbCorrect),
accuracy: ratio(nbCorrect, nbTotal),
},
"TTM-R2": {
total: ttmTotal,
correct: ttmCorrect,
incorrect: Math.max(0, ttmTotal - ttmCorrect),
accuracy: ratio(ttmCorrect, ttmTotal),
},
},
});
} catch (error) {
return json({ success: false, error: String(error) }, 500);
}
}
+15
View File
@@ -0,0 +1,15 @@
function json(data, status = 200) {
return new Response(JSON.stringify(data), {
status,
headers: { "content-type": "application/json; charset=UTF-8" },
});
}
export async function onRequestGet(context) {
return json({
success: true,
status: "running",
server_time: new Date().toISOString(),
version: "pages-1.0.0",
});
}
@@ -0,0 +1,23 @@
function json(data, status = 200) {
return new Response(JSON.stringify(data), {
status,
headers: { "content-type": "application/json; charset=UTF-8" },
});
}
export async function onRequestGet(context) {
const db = context.env.DB;
if (!db) {
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
}
try {
const row = await db
.prepare("SELECT * FROM backup_trades ORDER BY timestamp DESC LIMIT 1")
.first();
return json({ success: true, data: row || null });
} catch (error) {
return json({ success: false, error: String(error) }, 500);
}
}
@@ -0,0 +1,71 @@
function json(data, status = 200) {
return new Response(JSON.stringify(data), {
status,
headers: { "content-type": "application/json; charset=UTF-8" },
});
}
function toInt(v, d) {
const n = Number(v);
return Number.isFinite(n) && n > 0 ? Math.floor(n) : d;
}
export async function onRequestGet(context) {
const db = context.env.DB;
if (!db) {
return json({ success: false, error: "缺少 D1 绑定 DB" }, 500);
}
try {
const url = new URL(context.request.url);
const page = toInt(url.searchParams.get("page"), 1);
const limit = Math.min(100, toInt(url.searchParams.get("limit"), 10));
const month = (url.searchParams.get("month") || "").trim();
const day = (url.searchParams.get("day") || "").trim();
const offset = (page - 1) * limit;
const where = ["1=1"];
const params = [];
// 时间字段是 YYYY-MM-DD HH:MM:SS 文本,直接前缀过滤可走索引/减少转换成本
if (month) {
where.push("timestamp LIKE ?");
params.push(`${month}%`);
}
if (day) {
where.push("timestamp LIKE ?");
params.push(`${day}%`);
}
const whereSql = where.join(" AND ");
const dataSql = `
SELECT *
FROM backup_trades
WHERE ${whereSql}
ORDER BY timestamp DESC
LIMIT ? OFFSET ?
`;
const countSql = `
SELECT COUNT(*) AS cnt
FROM backup_trades
WHERE ${whereSql}
`;
const dataResult = await db.prepare(dataSql).bind(...params, limit, offset).all();
const countResult = await db.prepare(countSql).bind(...params).first();
const total = Number(countResult?.cnt || 0);
const totalPages = total > 0 ? Math.ceil(total / limit) : 1;
return json({
success: true,
data: dataResult.results || [],
page,
limit,
total,
total_pages: totalPages,
});
} catch (error) {
return json({ success: false, error: String(error) }, 500);
}
}
+200
View File
@@ -0,0 +1,200 @@
const { createApp } = Vue;
const app = createApp({
data() {
return {
loading: false,
activeTab: 'trades',
serverTime: '',
currentTrade: null,
tradeHistory: {
data: [],
page: 1,
limit: 10,
total: 0,
total_pages: 0,
},
tradeFilter: {
month: '',
day: '',
},
accuracyFilter: {
month: '',
},
monthlyStats: {},
dailyStats: {
data: [],
page: 1,
limit: 10,
total: 0,
total_pages: 0,
},
hourlyModal: {
date: '',
data: [],
},
};
},
mounted() {
this.accuracyFilter.month = this.getCurrentMonth();
this.loadCurrentTrade();
this.loadTradeHistory(1);
this.updateServerTime();
setInterval(() => this.updateServerTime(), 60000);
},
methods: {
getCurrentMonth() {
const now = new Date();
return `${now.getFullYear()}-${String(now.getMonth() + 1).padStart(2, '0')}`;
},
updateServerTime() {
const now = new Date();
this.serverTime = now.toLocaleString('zh-CN', {
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
minute: '2-digit',
});
},
formatTime(timestamp) {
if (!timestamp) return '-';
return String(timestamp).slice(0, 19);
},
formatPercent(value) {
if (value === null || value === undefined) return '-';
return `${(Number(value) * 100).toFixed(1)}%`;
},
getPredictionClass(prediction) {
if (prediction === 'UP') return 'text-success fw-bold';
if (prediction === 'DOWN') return 'text-danger fw-bold';
return '';
},
getStatusClass(status) {
switch (status) {
case 'success': return 'text-success';
case 'test': return 'text-info';
case 'failure': return 'text-danger';
case 'in_progress': return 'text-warning';
default: return 'text-muted';
}
},
getStatusText(status) {
switch (status) {
case 'success': return '成功';
case 'test': return '测试';
case 'failure': return '失败';
case 'in_progress': return '进行中';
default: return status || '-';
}
},
getAccuracyClass(accuracy) {
if (accuracy >= 60) return 'text-success fw-bold';
if (accuracy >= 50) return 'text-warning';
return 'text-danger';
},
async fetchAPI(url) {
try {
const response = await fetch(url);
return await response.json();
} catch (error) {
return { success: false, error: String(error) };
}
},
async loadCurrentTrade() {
const data = await this.fetchAPI('/api/trades/current');
if (data.success) {
this.currentTrade = data.data;
}
},
async loadTradeHistory(page) {
if (page < 1) return;
this.loading = true;
let url = `/api/trades/history?page=${page}&limit=${this.tradeHistory.limit}`;
if (this.tradeFilter.month) {
url += `&month=${this.tradeFilter.month}`;
}
if (this.tradeFilter.day) {
url += `&day=${this.tradeFilter.day}`;
}
const data = await this.fetchAPI(url);
if (data.success) {
this.tradeHistory = {
data: data.data,
page: data.page,
limit: data.limit,
total: data.total,
total_pages: data.total_pages,
};
}
this.loading = false;
},
clearTradeFilter() {
this.tradeFilter.month = '';
this.tradeFilter.day = '';
this.loadTradeHistory(1);
},
async loadAccuracyData() {
this.loading = true;
const monthlyData = await this.fetchAPI(`/api/accuracy/monthly?month=${this.accuracyFilter.month}`);
if (monthlyData.success) {
this.monthlyStats = monthlyData.data || {};
}
await this.loadDailyStats(1);
this.loading = false;
},
async loadDailyStats(page) {
if (page < 1) return;
const data = await this.fetchAPI(`/api/accuracy/daily?month=${this.accuracyFilter.month}&page=${page}&limit=10`);
if (data.success) {
this.dailyStats = {
data: data.data,
page: data.page,
limit: data.limit,
total: data.total,
total_pages: data.total_pages,
};
}
},
async showHourlyDetails(date) {
this.loading = true;
this.hourlyModal.date = date;
const data = await this.fetchAPI(`/api/accuracy/hourly/${date}`);
if (data.success) {
this.hourlyModal.data = data.data || [];
}
this.loading = false;
const modal = new bootstrap.Modal(document.getElementById('hourlyModal'));
modal.show();
},
},
watch: {
activeTab(newTab) {
if (newTab === 'accuracy' && !this.dailyStats.data.length) {
this.loadAccuracyData();
}
},
},
});
app.mount('#app');
+251
View File
@@ -0,0 +1,251 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>BTC 预测交易看板 (Pages + D1)</title>
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.11.1/font/bootstrap-icons.css" rel="stylesheet">
<link href="/style.css" rel="stylesheet">
</head>
<body>
<div id="app">
<nav class="navbar navbar-expand-lg navbar-dark bg-dark">
<div class="container">
<a class="navbar-brand" href="#">
<i class="bi bi-currency-bitcoin"></i> BTC 预测交易看板 (Pages)
</a>
<span class="navbar-text text-light">
<i class="bi bi-clock"></i> {{ serverTime }}
</span>
</div>
</nav>
<div class="container mt-4">
<ul class="nav nav-tabs mb-4" role="tablist">
<li class="nav-item">
<button class="nav-link" :class="{ active: activeTab === 'trades' }" @click="activeTab = 'trades'">
<i class="bi bi-graph-up"></i> 交易详情
</button>
</li>
<li class="nav-item">
<button class="nav-link" :class="{ active: activeTab === 'accuracy' }" @click="activeTab = 'accuracy'">
<i class="bi bi-bullseye"></i> 模型准确率
</button>
</li>
</ul>
<div v-show="activeTab === 'trades'">
<div class="card mb-4">
<div class="card-header bg-primary text-white">
<i class="bi bi-lightning-charge"></i> 当前小时交易
</div>
<div class="card-body">
<div v-if="currentTrade" class="row">
<div class="col-md-3">
<div class="stat-card">
<div class="stat-label">市场</div>
<div class="stat-value small">{{ currentTrade.market_title || '-' }}</div>
</div>
</div>
<div class="col-md-2">
<div class="stat-card">
<div class="stat-label">预测方向</div>
<div class="stat-value" :class="getPredictionClass(currentTrade.prediction)">
{{ currentTrade.prediction || '-' }}
</div>
</div>
</div>
<div class="col-md-2">
<div class="stat-card">
<div class="stat-label">概率</div>
<div class="stat-value">{{ formatPercent(currentTrade.probability) }}</div>
</div>
</div>
<div class="col-md-2">
<div class="stat-card">
<div class="stat-label">策略</div>
<div class="stat-value small">{{ currentTrade.strategy || 'Ensemble' }}</div>
</div>
</div>
<div class="col-md-3">
<div class="stat-card">
<div class="stat-label">状态</div>
<div class="stat-value" :class="getStatusClass(currentTrade.status)">
{{ getStatusText(currentTrade.status) }}
</div>
</div>
</div>
</div>
<div v-else class="text-center text-muted py-4">
<i class="bi bi-inbox"></i> 暂无交易记录
</div>
</div>
</div>
<div class="card">
<div class="card-header d-flex justify-content-between align-items-center">
<span><i class="bi bi-clock-history"></i> 历史交易记录</span>
<div class="d-flex gap-2">
<input type="month" class="form-control form-control-sm" v-model="tradeFilter.month" @change="loadTradeHistory(1)" style="width: 150px;">
<input type="date" class="form-control form-control-sm" v-model="tradeFilter.day" @change="loadTradeHistory(1)" style="width: 150px;">
<button class="btn btn-sm btn-outline-secondary" @click="clearTradeFilter">
<i class="bi bi-x-circle"></i> 清除
</button>
</div>
</div>
<div class="card-body p-0">
<div class="table-responsive">
<table class="table table-hover table-striped mb-0">
<thead class="table-light">
<tr>
<th>时间</th>
<th>市场</th>
<th>策略</th>
<th>预测</th>
<th>概率</th>
<th>价格</th>
<th>状态</th>
</tr>
</thead>
<tbody>
<tr v-for="trade in tradeHistory.data" :key="trade.id">
<td>{{ formatTime(trade.timestamp) }}</td>
<td class="text-truncate" style="max-width: 200px;">{{ trade.market_title }}</td>
<td><span class="badge bg-secondary">{{ trade.strategy || 'Ensemble' }}</span></td>
<td :class="getPredictionClass(trade.prediction)">{{ trade.prediction }}</td>
<td>{{ formatPercent(trade.probability) }}</td>
<td>{{ trade.price ?? '-' }}</td>
<td :class="getStatusClass(trade.status)">{{ getStatusText(trade.status) }}</td>
</tr>
<tr v-if="!tradeHistory.data?.length"><td colspan="7" class="text-center text-muted py-4">暂无数据</td></tr>
</tbody>
</table>
</div>
</div>
<div class="card-footer d-flex justify-content-between align-items-center">
<span class="text-muted">共 {{ tradeHistory.total }} 条</span>
<nav>
<ul class="pagination pagination-sm mb-0">
<li class="page-item" :class="{ disabled: tradeHistory.page <= 1 }">
<a class="page-link" href="#" @click.prevent="loadTradeHistory(tradeHistory.page - 1)">上一页</a>
</li>
<li class="page-item disabled"><span class="page-link">{{ tradeHistory.page }} / {{ tradeHistory.total_pages }}</span></li>
<li class="page-item" :class="{ disabled: tradeHistory.page >= tradeHistory.total_pages }">
<a class="page-link" href="#" @click.prevent="loadTradeHistory(tradeHistory.page + 1)">下一页</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div v-show="activeTab === 'accuracy'">
<div class="mb-4">
<div class="d-flex gap-2 align-items-center">
<label class="form-label mb-0">选择月份:</label>
<input type="month" class="form-control" v-model="accuracyFilter.month" @change="loadAccuracyData" style="width: 180px;">
</div>
</div>
<div class="row mb-4">
<div class="col-md-3"><div class="card h-100 border-primary shadow-sm"><div class="card-header bg-primary text-white text-center">集成策略</div><div class="card-body text-center" v-if="monthlyStats.Ensemble"><h2 class="display-6 text-primary fw-bold">{{ monthlyStats.Ensemble.accuracy }}%</h2><div class="text-muted mb-2">准确率</div><div class="small text-muted">{{ monthlyStats.Ensemble.correct }}/{{ monthlyStats.Ensemble.total }}</div></div><div v-else class="text-center text-muted py-3">暂无数据</div></div></div>
<div class="col-md-3"><div class="card h-100 border-secondary"><div class="card-header bg-secondary text-white text-center">DLinear</div><div class="card-body text-center" v-if="monthlyStats.DLinear"><h3 class="text-dark">{{ monthlyStats.DLinear.accuracy }}%</h3><div class="small text-muted">{{ monthlyStats.DLinear.correct }}/{{ monthlyStats.DLinear.total }}</div></div><div v-else class="text-center text-muted py-3">暂无数据</div></div></div>
<div class="col-md-3"><div class="card h-100 border-secondary"><div class="card-header bg-secondary text-white text-center">N-BEATS</div><div class="card-body text-center" v-if="monthlyStats['N-BEATS']"><h3 class="text-dark">{{ monthlyStats['N-BEATS'].accuracy }}%</h3><div class="small text-muted">{{ monthlyStats['N-BEATS'].correct }}/{{ monthlyStats['N-BEATS'].total }}</div></div><div v-else class="text-center text-muted py-3">暂无数据</div></div></div>
<div class="col-md-3"><div class="card h-100 border-secondary"><div class="card-header bg-secondary text-white text-center">TTM-R2</div><div class="card-body text-center" v-if="monthlyStats['TTM-R2']"><h3 class="text-dark">{{ monthlyStats['TTM-R2'].accuracy }}%</h3><div class="small text-muted">{{ monthlyStats['TTM-R2'].correct }}/{{ monthlyStats['TTM-R2'].total }}</div></div><div v-else class="text-center text-muted py-3">暂无数据</div></div></div>
</div>
<div class="card">
<div class="card-header"><i class="bi bi-calendar3"></i> 每日准确率概览</div>
<div class="card-body p-0">
<div class="table-responsive">
<table class="table table-hover mb-0 text-center">
<thead class="table-light">
<tr>
<th class="text-start ps-4">日期</th>
<th>集成策略</th>
<th>DLinear</th>
<th>N-BEATS</th>
<th>TTM-R2</th>
<th>操作</th>
</tr>
</thead>
<tbody>
<tr v-for="day in dailyStats.data" :key="day.date">
<td class="text-start ps-4">{{ day.date }}</td>
<td><span v-if="day.Ensemble" :class="getAccuracyClass(day.Ensemble.accuracy)">{{ day.Ensemble.accuracy }}%</span><span v-else class="text-muted">-</span></td>
<td><span v-if="day.DLinear" :class="getAccuracyClass(day.DLinear.accuracy)">{{ day.DLinear.accuracy }}%</span><span v-else class="text-muted">-</span></td>
<td><span v-if="day['N-BEATS']" :class="getAccuracyClass(day['N-BEATS'].accuracy)">{{ day['N-BEATS'].accuracy }}%</span><span v-else class="text-muted">-</span></td>
<td><span v-if="day['TTM-R2']" :class="getAccuracyClass(day['TTM-R2'].accuracy)">{{ day['TTM-R2'].accuracy }}%</span><span v-else class="text-muted">-</span></td>
<td><button class="btn btn-sm btn-outline-primary" @click="showHourlyDetails(day.date)"><i class="bi bi-eye"></i> 详情</button></td>
</tr>
<tr v-if="!dailyStats.data?.length"><td colspan="6" class="text-center text-muted py-4">暂无数据</td></tr>
</tbody>
</table>
</div>
</div>
<div class="card-footer d-flex justify-content-between align-items-center">
<span class="text-muted">共 {{ dailyStats.total }} 天</span>
<nav>
<ul class="pagination pagination-sm mb-0">
<li class="page-item" :class="{ disabled: dailyStats.page <= 1 }"><a class="page-link" href="#" @click.prevent="loadDailyStats(dailyStats.page - 1)">上一页</a></li>
<li class="page-item disabled"><span class="page-link">{{ dailyStats.page }} / {{ dailyStats.total_pages }}</span></li>
<li class="page-item" :class="{ disabled: dailyStats.page >= dailyStats.total_pages }"><a class="page-link" href="#" @click.prevent="loadDailyStats(dailyStats.page + 1)">下一页</a></li>
</ul>
</nav>
</div>
</div>
</div>
</div>
<div class="modal fade" id="hourlyModal" tabindex="-1">
<div class="modal-dialog modal-xl modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title"><i class="bi bi-clock"></i> {{ hourlyModal.date }} 每小时预测详情</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
</div>
<div class="modal-body p-0">
<table class="table table-sm table-hover mb-0 text-center align-middle">
<thead class="table-light sticky-top">
<tr>
<th>时间</th>
<th>开盘</th>
<th>收盘</th>
<th>实际</th>
<th class="table-primary">集成策略</th>
<th>DLinear</th>
<th>N-BEATS</th>
<th>TTM-R2</th>
</tr>
</thead>
<tbody>
<tr v-for="hour in hourlyModal.data" :key="hour.hour">
<td>{{ hour.timestamp_cst ? hour.timestamp_cst.split(' ')[1] : (hour.hour + ':00') }}</td>
<td>{{ hour.open_price ?? '-' }}</td>
<td>{{ hour.close_price ?? '-' }}</td>
<td :class="hour.actual_result === 1 ? 'text-success fw-bold' : (hour.actual_result === 0 ? 'text-danger fw-bold' : 'text-muted')">{{ hour.actual_result === 1 ? 'UP' : (hour.actual_result === 0 ? 'DOWN' : '-') }}</td>
<td class="table-primary bg-opacity-10"><span v-if="hour.models?.Ensemble" :class="hour.models.Ensemble.is_correct ? 'text-success' : 'text-danger'">{{ hour.models.Ensemble.prediction === 1 ? 'UP' : 'DOWN' }}</span></td>
<td><span v-if="hour.models?.DLinear" :class="hour.models.DLinear.is_correct ? 'text-success' : 'text-danger'">{{ hour.models.DLinear.prediction === 1 ? 'UP' : 'DOWN' }}</span></td>
<td><span v-if="hour.models?.['N-BEATS']" :class="hour.models['N-BEATS'].is_correct ? 'text-success' : 'text-danger'">{{ hour.models['N-BEATS'].prediction === 1 ? 'UP' : 'DOWN' }}</span></td>
<td><span v-if="hour.models?.['TTM-R2']" :class="hour.models['TTM-R2'].is_correct ? 'text-success' : 'text-danger'">{{ hour.models['TTM-R2'].prediction === 1 ? 'UP' : 'DOWN' }}</span></td>
</tr>
<tr v-if="!hourlyModal.data?.length"><td colspan="8" class="text-center text-muted py-4">暂无数据</td></tr>
</tbody>
</table>
</div>
<div class="modal-footer"><button type="button" class="btn btn-secondary" data-bs-dismiss="modal">关闭</button></div>
</div>
</div>
</div>
<div v-if="loading" class="loading-overlay">
<div class="spinner-border text-primary" role="status"><span class="visually-hidden">加载中...</span></div>
</div>
</div>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/js/bootstrap.bundle.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/vue@3.4.15/dist/vue.global.prod.js"></script>
<script src="/app.js"></script>
</body>
</html>
+118
View File
@@ -0,0 +1,118 @@
body {
background-color: #f8f9fa;
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
}
.navbar-brand {
font-weight: 600;
}
.navbar-brand i {
color: #f7931a;
}
.nav-tabs .nav-link {
color: #6c757d;
border: none;
padding: 0.75rem 1.5rem;
}
.nav-tabs .nav-link.active {
color: #0d6efd;
border-bottom: 3px solid #0d6efd;
background: transparent;
}
.nav-tabs .nav-link:hover:not(.active) {
border-bottom: 3px solid #dee2e6;
}
.card {
box-shadow: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);
border-radius: 0.5rem;
}
.card-header {
font-weight: 600;
border-radius: 0.5rem 0.5rem 0 0 !important;
}
.stat-card {
padding: 1rem;
text-align: center;
background: #f8f9fa;
border-radius: 0.5rem;
height: 100%;
}
.stat-label {
font-size: 0.75rem;
color: #6c757d;
text-transform: uppercase;
letter-spacing: 0.5px;
margin-bottom: 0.25rem;
}
.stat-value {
font-size: 1.25rem;
font-weight: 600;
}
.table {
margin-bottom: 0;
}
.table th {
font-weight: 600;
font-size: 0.875rem;
text-transform: uppercase;
letter-spacing: 0.5px;
color: #6c757d;
border-bottom: 2px solid #dee2e6;
}
.table td {
vertical-align: middle;
}
.table-hover tbody tr:hover {
background-color: rgba(13, 110, 253, 0.05);
}
.loading-overlay {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(255, 255, 255, 0.8);
display: flex;
align-items: center;
justify-content: center;
z-index: 9999;
}
.modal-content {
border-radius: 0.5rem;
}
.modal-header {
background: #f8f9fa;
border-radius: 0.5rem 0.5rem 0 0;
}
@media (max-width: 768px) {
.stat-card {
margin-bottom: 1rem;
}
.card-header .d-flex {
flex-direction: column;
gap: 0.5rem !important;
}
.nav-tabs .nav-link {
padding: 0.5rem 1rem;
font-size: 0.875rem;
}
}
+10
View File
@@ -0,0 +1,10 @@
name = "trades"
compatibility_date = "2026-03-10"
[vars]
APP_TITLE = "BTC 预测看板"
[[d1_databases]]
binding = "DB"
database_name = "orders"
database_id = "ae0cd6aa-10da-4459-9768-8e6ffad55a00"
+11
View File
@@ -0,0 +1,11 @@
name = "trades"
compatibility_date = "2026-03-10"
[vars]
APP_TITLE = "BTC 预测看板"
[[d1_databases]]
binding = "DB"
database_name = "orders"
database_id = "ae0cd6aa-10da-4459-9768-8e6ffad55a00"
+380
View File
@@ -0,0 +1,380 @@
"""对比 test 旧逻辑 vs auto_trade 逻辑的数据处理差异
两种预处理方式:
- test旧逻辑:整个df一次性预处理,无create_labels,按索引遍历窗口
- auto_trade逻辑:截取到当前K线的df,含create_labels,取X[-1]预测
对比内容:
1. 预处理后的df行数差异
2. 同一时间点的特征值差异(RSI、MACD等)
3. scaler transform后的值差异
4. 最终输入窗口的数值差异
5. 模型输出logits/概率/预测结果差异
"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import numpy as np
import pandas as pd
import pickle
import pandas_ta as ta
from datetime import timedelta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
# ==================== pickle反序列化需要的占位类 ====================
class ClassificationPreprocessor:
"""pkl文件中保存的类,pickle.load时需要能找到这个类名"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
# ==================== 两种预处理器 ====================
class TestPreprocessor:
"""test旧逻辑:无create_labels"""
def __init__(self, lookback=168):
self.lookback = lookback
self.scaler = StandardScaler()
def preprocess(self, df, fit=True):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
else:
scaled_features = self.scaler.transform(df[feature_cols])
X = []
valid_indices = []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
valid_indices.append(i + self.lookback - 1)
return np.array(X), df, valid_indices, scaled_features
class AutoTradePreprocessor:
"""auto_trade逻辑:含create_labels"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
def preprocess(self, df, fit=True):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
# create_labels: shift + 砍最后一行
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
df = df[:-self.forecast_horizon]
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
else:
scaled_features = self.scaler.transform(df[feature_cols])
X = []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
return np.array(X), df, scaled_features
def utc_to_beijing(ts):
return ts + timedelta(hours=8)
def main():
print("=" * 100)
print("数据处理对比:test旧逻辑 vs auto_trade逻辑")
print("=" * 100)
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
# 加载模型和训练时的scaler
from models.dlinear.model import DLinear
checkpoint = torch.load('./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device, weights_only=False)
model = DLinear(checkpoint['config']).to(device).eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
trained_preprocessor = pickle.load(f)
# 提取训练时的scaler参数
trained_scaler = trained_preprocessor.scaler
print(f"训练时scaler均值: {trained_scaler.mean_}")
print(f"训练时scaler方差: {trained_scaler.scale_}")
# 获取数据
print("\n获取数据...")
fetcher = BinanceDataFetcher()
df_full = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', end_date='2026-03-01', use_cache=False)
print(f"总行数: {len(df_full)}")
# 选几个2月10日(北京时间)的时间点做对比
# 北京时间02-10 00:00 = UTC 02-09 16:00
test_targets_utc = [
'2026-02-09 16:00:00', # 北京 02-10 00:00
'2026-02-09 20:00:00', # 北京 02-10 04:00
'2026-02-10 00:00:00', # 北京 02-10 08:00
'2026-02-10 04:00:00', # 北京 02-10 12:00
'2026-02-10 08:00:00', # 北京 02-10 16:00
'2026-02-10 12:00:00', # 北京 02-10 20:00
]
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
for target_utc_str in test_targets_utc:
target_utc = pd.Timestamp(target_utc_str)
target_bj = utc_to_beijing(target_utc)
# 找到目标K线在df中的索引
target_mask = df_full['timestamp'] == target_utc
if not target_mask.any():
print(f"\n⚠ 找不到 {target_utc_str},跳过")
continue
target_idx = df_full[target_mask].index[0]
input_idx = target_idx - 1 # 输入的最后一根K线
input_row = df_full.iloc[input_idx]
target_row = df_full.iloc[target_idx]
print(f"\n{'=' * 100}")
print(f"目标K线: 北京{target_bj.strftime('%m-%d %H:%M')} (UTC {target_utc_str})")
print(f"输入末根: UTC {input_row['timestamp']} O:{input_row['open']:.2f} H:{input_row['high']:.2f} L:{input_row['low']:.2f} C:{input_row['close']:.2f}")
print(f"目标K线: UTC {target_row['timestamp']} 收盘:{target_row['close']:.2f}")
print("-" * 100)
# ===== 方式A: test旧逻辑 =====
# 对整个df一次性预处理,按索引找到对应窗口
test_prep = TestPreprocessor(lookback=168)
test_prep.scaler = trained_scaler # 使用训练时的scaler
X_test, df_test, idx_test, scaled_test = test_prep.preprocess(df_full, fit=False)
# 找到input_idx对应的窗口
# idx_test[i] = i + 168 - 1,所以 input_idx 在 df_test 中的位置需要查找
# df_test 是 dropna 后 reset_index 的,timestamp 可能对应不同的行号
test_input_mask = df_test['timestamp'] == input_row['timestamp']
if test_input_mask.any():
test_df_idx = df_test[test_input_mask].index[0]
# 对应的窗口序号
test_window_i = test_df_idx - 168 + 1
if 0 <= test_window_i < len(X_test):
X_test_window = X_test[test_window_i]
else:
X_test_window = None
print(" [test] 窗口索引越界")
else:
test_df_idx = None
X_test_window = None
print(" [test] 在df_test中找不到对应时间")
# ===== 方式B: auto_trade逻辑 =====
# 截取到input_idx为止,含create_labels
df_up_to_t = df_full.iloc[:input_idx + 1]
auto_prep = AutoTradePreprocessor(lookback=168)
auto_prep.scaler = trained_scaler # 使用训练时的scaler
X_auto, df_auto, scaled_auto = auto_prep.preprocess(df_up_to_t, fit=False)
if len(X_auto) > 0:
X_auto_window = X_auto[-1] # 取最后一个窗口
else:
X_auto_window = None
print(" [auto] X为空")
# ===== 对比 =====
# 1. df行数对比
print(f"\n[1] 预处理后df行数")
print(f" test: {len(df_test)} 行 (整个df, 无create_labels)")
print(f" auto: {len(df_auto)} 行 (截取到input, 含create_labels砍尾)")
# 2. 最后一行时间对比
print(f"\n[2] 预处理后df最后一行")
print(f" test最后一行: {df_test.iloc[-1]['timestamp']} close={df_test.iloc[-1]['close']:.2f}")
print(f" auto最后一行: {df_auto.iloc[-1]['timestamp']} close={df_auto.iloc[-1]['close']:.2f}")
# 3. 输入末根K线的原始特征对比
if test_df_idx is not None:
test_features = df_test.loc[test_df_idx, feature_cols]
auto_last_row = df_auto.iloc[-1]
auto_features = auto_last_row[feature_cols]
print(f"\n[3] 输入末根K线原始特征值")
print(f" {'特征':<15} {'test值':>15} {'auto值':>15} {'差异':>15}")
print(f" {'-'*60}")
for col in feature_cols:
tv = test_features[col]
av = auto_features[col]
diff = tv - av
marker = " ⚠" if abs(diff) > 1e-6 else ""
print(f" {col:<15} {tv:>15.6f} {av:>15.6f} {diff:>15.8f}{marker}")
# 4. scaler transform 后的值对比(输入末根K线)
if test_df_idx is not None and X_test_window is not None and X_auto_window is not None:
test_scaled_last = scaled_test[test_df_idx]
auto_scaled_last = scaled_auto[-1]
print(f"\n[4] scaler transform 后的值(输入末根K线)")
print(f" {'特征':<15} {'test值':>15} {'auto值':>15} {'差异':>15}")
print(f" {'-'*60}")
for j, col in enumerate(feature_cols):
tv = test_scaled_last[j]
av = auto_scaled_last[j]
diff = tv - av
marker = " ⚠" if abs(diff) > 1e-6 else ""
print(f" {col:<15} {tv:>15.6f} {av:>15.6f} {diff:>15.8f}{marker}")
# 5. 输入窗口对比(整个168步窗口)
if X_test_window is not None and X_auto_window is not None:
window_diff = X_test_window - X_auto_window
max_diff = np.max(np.abs(window_diff))
mean_diff = np.mean(np.abs(window_diff))
nonzero_count = np.count_nonzero(np.abs(window_diff) > 1e-6)
print(f"\n[5] 输入窗口对比 (168步 × 9特征 = 1512个值)")
print(f" 最大绝对差异: {max_diff:.10f}")
print(f" 平均绝对差异: {mean_diff:.10f}")
print(f" 差异>1e-6的值: {nonzero_count} / 1512")
if nonzero_count > 0:
# 找到差异最大的位置
max_pos = np.unravel_index(np.argmax(np.abs(window_diff)), window_diff.shape)
print(f" 最大差异位置: 步={max_pos[0]} 特征={feature_cols[max_pos[1]]}")
print(f" test值: {X_test_window[max_pos]:.10f}")
print(f" auto值: {X_auto_window[max_pos]:.10f}")
# 6. 模型推理对比
if X_test_window is not None and X_auto_window is not None:
with torch.no_grad():
test_tensor = torch.FloatTensor(X_test_window[np.newaxis]).to(device)
auto_tensor = torch.FloatTensor(X_auto_window[np.newaxis]).to(device)
test_logits = model(test_tensor).cpu().numpy().item()
auto_logits = model(auto_tensor).cpu().numpy().item()
test_prob = 1 / (1 + np.exp(-test_logits))
auto_prob = 1 / (1 + np.exp(-auto_logits))
test_pred = "涨" if test_prob > 0.5 else "跌"
auto_pred = "涨" if auto_prob > 0.5 else "跌"
match = "✓ 一致" if test_pred == auto_pred else "✗ 不一致"
print(f"\n[6] 模型推理结果")
print(f" {'':15} {'test':>15} {'auto':>15} {'差异':>15}")
print(f" {'-'*60}")
print(f" {'logits':<15} {test_logits:>15.8f} {auto_logits:>15.8f} {test_logits-auto_logits:>15.8f}")
print(f" {'概率':<15} {test_prob:>15.8f} {auto_prob:>15.8f} {test_prob-auto_prob:>15.8f}")
print(f" {'预测':<15} {test_pred:>15} {auto_pred:>15} {match:>15}")
# ===== 全量对比:2月所有小时 =====
print(f"\n\n{'=' * 100}")
print("全量对比:2月份所有小时预测结果差异统计")
print("=" * 100)
# test方式:一次性预处理
test_prep_full = TestPreprocessor(lookback=168)
test_prep_full.scaler = trained_scaler
X_test_full, df_test_full, idx_test_full, _ = test_prep_full.preprocess(df_full, fit=False)
# 构建 timestamp -> test窗口索引 的映射
test_ts_map = {}
for i, idx in enumerate(idx_test_full):
ts = df_test_full.iloc[idx]['timestamp']
test_ts_map[ts] = i
total = 0
same = 0
diff_list = []
for target_idx in range(1, len(df_full)):
target_time_utc = df_full.iloc[target_idx]['timestamp']
target_time_bj = utc_to_beijing(target_time_utc)
if not (target_time_bj.year == 2026 and target_time_bj.month == 2):
continue
input_idx = target_idx - 1
input_ts = df_full.iloc[input_idx]['timestamp']
# test方式
test_i = test_ts_map.get(input_ts)
if test_i is None or test_i >= len(X_test_full):
continue
# auto方式
df_up_to_t = df_full.iloc[:input_idx + 1]
auto_prep_tmp = AutoTradePreprocessor(lookback=168)
auto_prep_tmp.scaler = trained_scaler
X_auto_tmp, _, _ = auto_prep_tmp.preprocess(df_up_to_t, fit=False)
if len(X_auto_tmp) == 0:
continue
with torch.no_grad():
test_logits = model(torch.FloatTensor(X_test_full[test_i:test_i+1]).to(device)).cpu().numpy().item()
auto_logits = model(torch.FloatTensor(X_auto_tmp[-1:]).to(device)).cpu().numpy().item()
test_prob = 1 / (1 + np.exp(-test_logits))
auto_prob = 1 / (1 + np.exp(-auto_logits))
test_pred = 1 if test_prob > 0.5 else 0
auto_pred = 1 if auto_prob > 0.5 else 0
total += 1
if test_pred == auto_pred:
same += 1
else:
diff_list.append({
'time_bj': target_time_bj.strftime('%m-%d %H:%M'),
'test_prob': test_prob,
'auto_prob': auto_prob,
'test_pred': '涨' if test_pred == 1 else '跌',
'auto_pred': '涨' if auto_pred == 1 else '跌',
})
print(f"总预测数: {total}")
print(f"结果一致: {same} ({same/total:.1%})")
print(f"结果不同: {len(diff_list)} ({len(diff_list)/total:.1%})")
if diff_list:
print(f"\n不一致的时间点:")
print(f" {'北京时间':<15} {'test概率':>10} {'test预测':>8} {'auto概率':>10} {'auto预测':>8}")
print(f" {'-'*55}")
for d in diff_list:
print(f" {d['time_bj']:<15} {d['test_prob']:>10.6f} {d['test_pred']:>8} {d['auto_prob']:>10.6f} {d['auto_pred']:>8}")
print("=" * 100)
if __name__ == "__main__":
main()
-460
View File
@@ -1,460 +0,0 @@
"""按天统计双模型和三模型的准确率"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import pickle
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df, fit=True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df = df[:-self.forecast_horizon].reset_index(drop=True)
prices = df['close'].values.reshape(-1, 1)
future_prices = df['future_price'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
X, y = [], []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
y.append(scaled_future[i + self.lookback, 0])
return np.array(X), np.array(y), df
def inverse_transform_price(self, scaled_price):
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
def main():
print("="*80)
print("按天统计:双模型交集 vs 三模型投票 每日准确率分析")
print("="*80)
# ========== 加载模型 ==========
print("\n[1/4] 加载模型...")
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
# 加载DLinear
from models.dlinear.model import DLinear
dlinear_checkpoint = torch.load(
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device
)
dlinear_model = DLinear(dlinear_checkpoint['config'])
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
dlinear_model = dlinear_model.to(device)
dlinear_model.eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
# 加载N-BEATS
from models.nbeats.model import NBeats
nbeats_checkpoint = torch.load(
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
map_location=device
)
nbeats_model = NBeats(nbeats_checkpoint['config'])
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
nbeats_model = nbeats_model.to(device)
nbeats_model.eval()
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
# 加载TTM-R2
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
"ibm-granite/granite-timeseries-ttm-r2",
trust_remote_code=True
)
ttm_model = ttm_model.to(device)
ttm_model.eval()
print(f"✓ 三个模型已加载")
# ========== 获取数据 ==========
print("\n[2/4] 获取2026年测试数据...")
fetcher = BinanceDataFetcher()
try:
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-26',
end_date='2026-02-02', use_cache=False)
except:
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', use_cache=False)
print(f"✓ 测试数据: {len(df_2026)} 行")
# ========== 预处理 ==========
print("\n[3/4] 预处理数据...")
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
min_samples = min(len(X_dlinear), len(X_nbeats))
X_dlinear = X_dlinear[:min_samples]
y_dlinear = y_dlinear[:min_samples]
X_nbeats = X_nbeats[:min_samples]
y_nbeats = y_nbeats[:min_samples]
# ========== 预测 ==========
print("\n[4/4] 执行预测...")
# DLinear预测
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
with torch.no_grad():
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits))
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten()
# N-BEATS预测
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
with torch.no_grad():
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
# TTM-R2预测
context_length = 512
ttm_preds_list = []
all_prices = df_2026['close'].values
all_open_prices = df_2026['open'].values
for i in range(min_samples):
start_idx = i
end_idx = start_idx + context_length
if end_idx > len(all_prices):
context_data = all_prices[-context_length:].reshape(1, -1, 1)
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
context_tensor = torch.FloatTensor(context_data).to(device)
with torch.no_grad():
output = ttm_model(context_tensor)
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
ttm_preds = np.array(ttm_preds_list)
print(f"✓ 预测完成")
# ========== 构建结果DataFrame ==========
# 获取对应的时间戳(第i个样本对应df_2026的第168+i根K线)
timestamps = df_2026['timestamp'].iloc[168:168+min_samples].values
# 转换为美国东部时间 (ET)
timestamps_utc = pd.to_datetime(timestamps)
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
dates = timestamps_et.date
# 双模型交集
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
two_model_preds = np.full(len(dlinear_preds), -1)
two_model_preds[two_model_agree_up] = 1
two_model_preds[two_model_agree_down] = 0
# 三模型多数投票
vote_sum = dlinear_preds + nbeats_preds + ttm_preds
three_model_preds = np.where(vote_sum >= 2, 1, 0)
# 真实标签
true_labels = y_dlinear
# 创建DataFrame
results_df = pd.DataFrame({
'date': dates,
'timestamp': timestamps,
'timestamp_et': timestamps_et,
'true_label': true_labels,
'dlinear_pred': dlinear_preds,
'nbeats_pred': nbeats_preds,
'ttm_pred': ttm_preds,
'two_model_pred': two_model_preds,
'three_model_pred': three_model_preds,
'two_model_correct': (two_model_preds == true_labels) & (two_model_preds != -1),
'three_model_correct': (three_model_preds == true_labels)
})
# 添加周几信息(美国东部时间)
results_df['day_of_week'] = pd.to_datetime(results_df['timestamp_et']).dt.day_name()
results_df['is_weekend'] = pd.to_datetime(results_df['timestamp_et']).dt.dayofweek >= 5 # 5=Saturday, 6=Sunday
# 2026年1月美国重要经济数据发布日期(美国东部时间)
# 数据来源:美联储、劳工部等官方日历
us_economic_events = {
'2026-01-08': ['初请失业金人数'],
'2026-01-09': ['非农就业报告 (NFP)', '失业率'],
'2026-01-13': ['CPI通胀数据'],
'2026-01-14': ['PPI生产者物价指数'],
'2026-01-15': ['零售销售数据'],
'2026-01-16': ['密歇根大学消费者信心指数'],
'2026-01-22': ['初请失业金人数'],
'2026-01-27': ['耐用品订单'],
'2026-01-28': ['FOMC利率决议', 'GDP初值'],
'2026-01-29': ['PCE物价指数', '个人收入与支出'],
}
# ========== 按天统计 ==========
print("\n" + "="*80)
print("📊 每日准确率统计 (美国东部时间)")
print("="*80)
daily_stats = results_df.groupby('date').agg({
'true_label': 'count', # 总样本数
'two_model_pred': lambda x: (x != -1).sum(), # 双模型有效预测数
'two_model_correct': 'sum', # 双模型正确数
'three_model_correct': 'sum', # 三模型正确数
'is_weekend': 'first', # 是否周末
'day_of_week': 'first' # 星期几
})
daily_stats.columns = ['total_samples', 'two_model_valid', 'two_model_correct',
'three_model_correct', 'is_weekend', 'day_of_week']
# 计算准确率
daily_stats['two_model_accuracy'] = daily_stats['two_model_correct'] / daily_stats['two_model_valid'] * 100
daily_stats['three_model_accuracy'] = daily_stats['three_model_correct'] / daily_stats['total_samples'] * 100
daily_stats['two_model_coverage'] = daily_stats['two_model_valid'] / daily_stats['total_samples'] * 100
# 填充NaN(当天没有有效预测)
daily_stats['two_model_accuracy'] = daily_stats['two_model_accuracy'].fillna(0)
# 添加美国经济数据发布标记
daily_stats['us_events'] = daily_stats.index.map(
lambda x: us_economic_events.get(str(x), [])
)
daily_stats['has_us_data'] = daily_stats['us_events'].apply(lambda x: len(x) > 0)
print(f"\n每日详细统计:")
print(f"{'日期':<12} {'星期':<10} {'周末':<4} {'美国数据':<4} {'样本':>4} "
f"{'双模型%':>8} {'覆盖%':>6} {'三模型%':>8} {'差异':>7}")
print("-" * 95)
for date, row in daily_stats.iterrows():
diff = row['two_model_accuracy'] - row['three_model_accuracy']
diff_str = f"{diff:+.1f}" if row['two_model_valid'] > 0 else "N/A"
weekend_mark = "✓" if row['is_weekend'] else ""
us_data_mark = "📊" if row['has_us_data'] else ""
# 根据准确率设置颜色标记
two_acc = row['two_model_accuracy']
three_acc = row['three_model_accuracy']
two_mark = "🔥" if two_acc >= 70 else ("✅" if two_acc >= 60 else ("⚠️" if two_acc < 40 else ""))
three_mark = "🔥" if three_acc >= 70 else ("✅" if three_acc >= 60 else ("⚠️" if three_acc < 40 else ""))
print(f"{date} {row['day_of_week'][:3]:>8} {weekend_mark:>4} {us_data_mark:>4} "
f"{row['total_samples']:>4.0f} "
f"{row['two_model_accuracy']:>7.1f}{two_mark} "
f"{row['two_model_coverage']:>5.1f} "
f"{row['three_model_accuracy']:>7.1f}{three_mark} "
f"{diff_str:>7}")
# 如果有美国经济数据发布,显示详情
if row['has_us_data']:
events_str = ", ".join(row['us_events'])
print(f" └─ 📊 {events_str}")
# ========== 周末 vs 工作日分析 ==========
print("\n" + "="*80)
print("📅 周末 vs 工作日准确率对比")
print("="*80)
weekend_stats = daily_stats[daily_stats['is_weekend']]
weekday_stats = daily_stats[~daily_stats['is_weekend']]
if len(weekend_stats) > 0:
print(f"\n🌴 周末 ({len(weekend_stats)}天):")
print(f" 双模型平均准确率: {weekend_stats['two_model_accuracy'].mean():.2f}%")
print(f" 三模型平均准确率: {weekend_stats['three_model_accuracy'].mean():.2f}%")
print(f" 平均覆盖率: {weekend_stats['two_model_coverage'].mean():.2f}%")
if len(weekday_stats) > 0:
print(f"\n💼 工作日 ({len(weekday_stats)}天):")
print(f" 双模型平均准确率: {weekday_stats['two_model_accuracy'].mean():.2f}%")
print(f" 三模型平均准确率: {weekday_stats['three_model_accuracy'].mean():.2f}%")
print(f" 平均覆盖率: {weekday_stats['two_model_coverage'].mean():.2f}%")
# ========== 美国数据发布日 vs 普通日分析 ==========
print("\n" + "="*80)
print("📊 美国经济数据发布日 vs 普通日准确率对比")
print("="*80)
us_data_days = daily_stats[daily_stats['has_us_data']]
normal_days = daily_stats[~daily_stats['has_us_data']]
if len(us_data_days) > 0:
print(f"\n📊 美国数据发布日 ({len(us_data_days)}天):")
print(f" 双模型平均准确率: {us_data_days['two_model_accuracy'].mean():.2f}%")
print(f" 三模型平均准确率: {us_data_days['three_model_accuracy'].mean():.2f}%")
print(f" 平均覆盖率: {us_data_days['two_model_coverage'].mean():.2f}%")
print(f"\n 具体日期:")
for date in us_data_days.index:
events = ", ".join(us_data_days.loc[date, 'us_events'])
two_acc = us_data_days.loc[date, 'two_model_accuracy']
three_acc = us_data_days.loc[date, 'three_model_accuracy']
print(f" {date}: 双{two_acc:.1f}% / 三{three_acc:.1f}% - {events}")
if len(normal_days) > 0:
print(f"\n📅 普通日 ({len(normal_days)}天):")
print(f" 双模型平均准确率: {normal_days['two_model_accuracy'].mean():.2f}%")
print(f" 三模型平均准确率: {normal_days['three_model_accuracy'].mean():.2f}%")
print(f" 平均覆盖率: {normal_days['two_model_coverage'].mean():.2f}%")
# ========== 分类统计 ==========
print("\n" + "="*80)
print("📈 准确率分类统计")
print("="*80)
# 双模型交集分类
two_model_high = daily_stats[daily_stats['two_model_accuracy'] >= 60]
two_model_medium = daily_stats[(daily_stats['two_model_accuracy'] >= 45) &
(daily_stats['two_model_accuracy'] < 60)]
two_model_low = daily_stats[(daily_stats['two_model_accuracy'] > 0) &
(daily_stats['two_model_accuracy'] < 45)]
# 三模型投票分类
three_model_high = daily_stats[daily_stats['three_model_accuracy'] >= 60]
three_model_medium = daily_stats[(daily_stats['three_model_accuracy'] >= 45) &
(daily_stats['three_model_accuracy'] < 60)]
three_model_low = daily_stats[daily_stats['three_model_accuracy'] < 45]
print(f"\n🔵 双模型交集:")
print(f" 高准确率 (>=60%): {len(two_model_high)}天")
if len(two_model_high) > 0:
print(f" 日期: {', '.join([str(d) for d in two_model_high.index])}")
print(f" 平均准确率: {two_model_high['two_model_accuracy'].mean():.1f}%")
print(f"\n 中等准确率 (45-60%): {len(two_model_medium)}天")
if len(two_model_medium) > 0:
print(f" 日期: {', '.join([str(d) for d in two_model_medium.index])}")
print(f" 平均准确率: {two_model_medium['two_model_accuracy'].mean():.1f}%")
print(f"\n 低准确率 (<45%): {len(two_model_low)}天")
if len(two_model_low) > 0:
print(f" 日期: {', '.join([str(d) for d in two_model_low.index])}")
print(f" 平均准确率: {two_model_low['two_model_accuracy'].mean():.1f}%")
print(f"\n🟢 三模型投票:")
print(f" 高准确率 (>=60%): {len(three_model_high)}天")
if len(three_model_high) > 0:
print(f" 日期: {', '.join([str(d) for d in three_model_high.index])}")
print(f" 平均准确率: {three_model_high['three_model_accuracy'].mean():.1f}%")
print(f"\n 中等准确率 (45-60%): {len(three_model_medium)}天")
if len(three_model_medium) > 0:
print(f" 日期: {', '.join([str(d) for d in three_model_medium.index])}")
print(f" 平均准确率: {three_model_medium['three_model_accuracy'].mean():.1f}%")
print(f"\n 低准确率 (<45%): {len(three_model_low)}天")
if len(three_model_low) > 0:
print(f" 日期: {', '.join([str(d) for d in three_model_low.index])}")
print(f" 平均准确率: {three_model_low['three_model_accuracy'].mean():.1f}%")
# ========== 总结 ==========
print("\n" + "="*80)
print("📊 总体统计")
print("="*80)
print(f"\n双模型交集:")
print(f" 总体准确率: {(daily_stats['two_model_correct'].sum() / daily_stats['two_model_valid'].sum() * 100):.2f}%")
print(f" 总体覆盖率: {(daily_stats['two_model_valid'].sum() / daily_stats['total_samples'].sum() * 100):.2f}%")
print(f" 最高日准确率: {daily_stats['two_model_accuracy'].max():.1f}% ({daily_stats['two_model_accuracy'].idxmax()})")
print(f" 最低日准确率: {daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].min():.1f}% ({daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].idxmin()})")
print(f"\n三模型投票:")
print(f" 总体准确率: {(daily_stats['three_model_correct'].sum() / daily_stats['total_samples'].sum() * 100):.2f}%")
print(f" 覆盖率: 100%")
print(f" 最高日准确率: {daily_stats['three_model_accuracy'].max():.1f}% ({daily_stats['three_model_accuracy'].idxmax()})")
print(f" 最低日准确率: {daily_stats['three_model_accuracy'].min():.1f}% ({daily_stats['three_model_accuracy'].idxmin()})")
print("\n" + "="*80)
print("✓ 每日准确率统计完成!")
print("="*80)
if __name__ == "__main__":
main()
+379
View File
@@ -0,0 +1,379 @@
#!/usr/bin/env python3
"""
每日数据报告定时推送程序
- 每天北京时间 8 点推送昨天的 24 小时预测统计
- 包含各模型准确率、逐小时预测结果详情
"""
import schedule
import time
import logging
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
from typing import Dict, List
from utils.send_msg import lark_notifier
from api.models import PredictionModel, convert_to_cst, get_db_connection, TZ_CST, TZ_UTC
# 配置日志
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler('logs/daily_report.log'),
logging.StreamHandler()
]
)
logger = logging.getLogger(__name__)
class DailyReportSender:
"""每日数据报告发送器"""
def __init__(self):
self.model = PredictionModel()
self.lark = lark_notifier
def get_yesterday_failed_trades(self) -> List[Dict]:
"""
获取昨天的失败交易订单
Returns:
[
{
'timestamp': '2026-02-27 10:30:00',
'market_title': 'BTC 价格',
'action': 'BUY',
'status': 'failed',
'error_message': '余额不足'
},
...
]
"""
try:
# 获取昨天的日期(CST)
now_cst = datetime.now(TZ_CST)
yesterday_cst = now_cst - timedelta(days=1)
yesterday_date = yesterday_cst.strftime('%Y-%m-%d')
# 计算 CST 时间范围
start_cst = datetime.strptime(f"{yesterday_date} 00:00:00", '%Y-%m-%d %H:%M:%S').replace(tzinfo=TZ_CST)
end_cst = start_cst + timedelta(days=1)
start_str = start_cst.strftime('%Y-%m-%d %H:%M:%S')
end_str = end_cst.strftime('%Y-%m-%d %H:%M:%S')
# 从 trades 表查询失败订单
with get_db_connection() as conn:
cursor = conn.cursor()
sql = """
SELECT
timestamp,
market_title,
action,
status,
error_message
FROM trades
WHERE timestamp >= %s AND timestamp < %s
AND status != 'MATCHED'
ORDER BY timestamp DESC
"""
cursor.execute(sql, (start_str, end_str))
rows = cursor.fetchall()
return rows if rows else []
except Exception as e:
logger.error(f"获取昨天失败交易失败: {e}", exc_info=True)
return []
def get_yesterday_statistics(self) -> Dict:
"""
获取昨天的 24 小时预测统计和详细数据
Returns:
{
'date': 'YYYY-MM-DD',
'models': {
'DLinear': {'total': 24, 'correct': 14, 'accuracy': 58.3},
'N-BEATS': {...},
'TTM-R2': {...},
'Ensemble': {...}
},
'hourly_details': [
{
'hour': '00:00-01:00',
'actual': '涨',
'dlinear': {'pred': '涨', 'correct': True},
'nbeats': {...},
'ttm': {...},
'ensemble': {...}
},
...
]
}
"""
try:
# 获取昨天的日期(CST)
now_cst = datetime.now(TZ_CST)
yesterday_cst = now_cst - timedelta(days=1)
yesterday_date = yesterday_cst.strftime('%Y-%m-%d')
# 计算 UTC 时间范围
start_cst = datetime.strptime(f"{yesterday_date} 00:00:00", '%Y-%m-%d %H:%M:%S').replace(tzinfo=TZ_CST)
end_cst = start_cst + timedelta(days=1)
start_utc = start_cst.astimezone(TZ_UTC)
end_utc = end_cst.astimezone(TZ_UTC)
start_str = start_utc.strftime('%Y-%m-%d %H:%M:%S')
end_str = end_utc.strftime('%Y-%m-%d %H:%M:%S')
# 查询数据库
with get_db_connection() as conn:
cursor = conn.cursor()
sql = """
SELECT
timestamp,
actual_result,
dlinear_prediction, dlinear_correct,
nbeats_prediction, nbeats_correct,
ttm_prediction, ttm_correct,
ensemble_prediction, ensemble_correct
FROM hourly_model_predictions
WHERE timestamp >= %s AND timestamp < %s
ORDER BY timestamp ASC
"""
cursor.execute(sql, (start_str, end_str))
rows = cursor.fetchall()
# 整理统计数据
models = ['dlinear', 'nbeats', 'ttm', 'ensemble']
model_names = {
'dlinear': 'DLinear',
'nbeats': 'N-BEATS',
'ttm': 'TTM-R2',
'ensemble': 'Ensemble'
}
# 初始化统计
stats = {
'date': yesterday_date,
'models': {},
'hourly_details': []
}
for model in models:
stats['models'][model_names[model]] = {
'total': 0,
'correct': 0,
'accuracy': 0.0
}
# 处理每行数据
for idx, row in enumerate(rows):
if row.get('actual_result') is None:
continue
# 获取时间戳(转为 CST 显示)
ts_utc = row['timestamp']
if isinstance(ts_utc, str):
ts_utc = datetime.fromisoformat(str(ts_utc).replace('Z', '+00:00'))
if ts_utc.tzinfo is None:
ts_utc = ts_utc.replace(tzinfo=TZ_UTC)
ts_cst = ts_utc.astimezone(TZ_CST)
hour_start = ts_cst.strftime('%H:%M')
hour_end = (ts_cst + timedelta(hours=1)).strftime('%H:%M')
hour_label = f"{hour_start}-{hour_end}"
# 实际结果
actual_result = row['actual_result']
actual_label = '涨' if actual_result == 1 else '跌'
# 每小时预测详情
hourly_item = {
'hour': hour_label,
'actual': actual_label
}
# 逐模型处理预测结果
for model in models:
pred = row[f"{model}_prediction"]
correct = row[f"{model}_correct"]
# 更新统计
if pred is not None:
stats['models'][model_names[model]]['total'] += 1
if correct == 1:
stats['models'][model_names[model]]['correct'] += 1
# 添加到小时详情
pred_label = '涨' if pred == 1 else ('跌' if pred == 0 else '无')
correct_flag = correct == 1 if correct is not None else None
hourly_item[model] = {
'pred': pred_label,
'correct': '✓' if correct_flag else ('✗' if correct_flag is False else '-')
}
stats['hourly_details'].append(hourly_item)
# 计算准确率百分比
for model_name in stats['models']:
total = stats['models'][model_name]['total']
correct = stats['models'][model_name]['correct']
if total > 0:
stats['models'][model_name]['accuracy'] = round(correct / total * 100, 2)
return stats
except Exception as e:
logger.error(f"获取昨天统计数据失败: {e}", exc_info=True)
return None
def format_report(self, stats: Dict, failed_trades: List[Dict] = None) -> str:
"""
格式化报告为飞书 Markdown 格式
Args:
stats: get_yesterday_statistics 返回的数据字典
failed_trades: 失败订单列表
Returns:
Markdown 格式的报告文本
"""
if not stats:
return "❌ 无法获取昨天的统计数据,请检查数据库连接"
date = stats['date']
models = stats['models']
# 构建报告头
report = f"**【BTC 日报 - {date}】**\n\n"
# 模型准确率统计部分
report += "**📊 模型准确率统计(24小时)**\n"
for model_name in ['DLinear', 'N-BEATS', 'TTM-R2', 'Ensemble']:
data = models.get(model_name, {})
total = data.get('total', 0)
correct = data.get('correct', 0)
accuracy = data.get('accuracy', 0)
# 进度条可视化
bar_length = 10
filled = int(accuracy / 100 * bar_length)
bar = '█' * filled + '░' * (bar_length - filled)
report += f"• **{model_name}**: {correct}/{total} {bar} {accuracy}%\n"
report += "\n"
# 交易失败订单详情
report += "**⚠️ 交易失败订单**\n"
if failed_trades and len(failed_trades) > 0:
report += f"昨天共有 {len(failed_trades)} 个失败订单\n"
for idx, trade in enumerate(failed_trades[:10], 1): # 仅显示前10个
timestamp = trade.get('timestamp', '未知')
market_title = trade.get('market_title', '未知市场')
action = trade.get('action', '未知')
status = trade.get('status', '未知')
error_msg = trade.get('error_message', '无详细信息')
# 简化错误信息 - 仅取第一行
if error_msg:
error_msg = error_msg.split('\n')[0][:50] # 仅取前50字
else:
error_msg = '无错误信息'
report += f"{idx}. [{timestamp}] {market_title} ({action}) → {error_msg}\n"
if len(failed_trades) > 10:
report += f"... 还有 {len(failed_trades) - 10} 个失败订单\n"
else:
report += "昨日无失败订单,运营正常 ✓\n"
report += "\n生成时间: " + datetime.now(TZ_CST).strftime('%Y-%m-%d %H:%M:%S CST')
return report
def send_daily_report(self):
"""获取昨天统计并推送"""
try:
logger.info("开始推送每日报告...")
stats = self.get_yesterday_statistics()
if not stats:
logger.warning("无法获取昨天统计数据")
self.lark.send_alert(
title="⚠️ 每日报告生成失败",
content="无法获取昨天的统计数据,请检查数据库连接"
)
return
# 获取失败订单
failed_trades = self.get_yesterday_failed_trades()
# 格式化报告
report = self.format_report(stats, failed_trades)
# 推送到飞书
self.lark.send_alert(
title="📊 BTC 每日数据报告",
content=report
)
logger.info(f"每日报告已送出: {stats['date']}")
except Exception as e:
logger.error(f"推送每日报告异常: {e}", exc_info=True)
self.lark.send_alert(
title="❌ 每日报告推送异常",
content=f"错误详情: {str(e)}"
)
def schedule_daily_report():
"""
启动每日报告定时任务
北京时间每天 8 点执行
"""
sender = DailyReportSender()
sender.send_daily_report() # 启动时立即发送一次(可选)
# 使用 schedule 库在北京时间 8 点触发
# 由于 schedule 库基于本地时间,当前环境如果是 macOS 默认时间,
# 我们使用一个包装器来处理 CST 时间转换
def cst_8am_job():
"""北京时间 8 点执行的任务"""
now_cst = datetime.now(TZ_CST)
if now_cst.hour == 8 and now_cst.minute < 1:
# 确保只在 08:00-08:01 间隔执行一次(避免重复)
sender.send_daily_report()
# 每分钟检查一次是否需要执行 (确保能在 8 点执行)
schedule.every(1).minutes.do(cst_8am_job)
logger.info("每日报告定时任务已启动(北京时间每天 8 点执行)")
# 启动调度循环
while True:
schedule.run_pending()
time.sleep(1)
def run_scheduler_background():
"""
在后台线程中运行调度器(供 start_system.py 调用)
"""
import threading
scheduler_thread = threading.Thread(target=schedule_daily_report, daemon=True)
scheduler_thread.start()
logger.info("每日报告调度器已在后台启动")
return scheduler_thread
if __name__ == '__main__':
# 支持单独运行以及从其他模块导入
logger.info("启动每日报告系统...")
schedule_daily_report()
+3 -1
View File
@@ -26,7 +26,9 @@ class DatabaseManager:
password = os.getenv('MYSQL_PASSWORD')
database = os.getenv('MYSQL_DB')
self._engine = create_engine(
f"mysql+pymysql://{user}:{password}@{host}:{port}/{database}?charset=utf8mb4"
f"mysql+pymysql://{user}:{password}@{host}:{port}/{database}?charset=utf8mb4",
pool_pre_ping=True,
pool_recycle=3600
)
return self._engine
+36 -21
View File
@@ -1,6 +1,7 @@
"""从Binance获取加密货币OHLCV数据"""
import os
import time
from datetime import datetime, timedelta
from pathlib import Path
from typing import Optional, Literal
@@ -11,9 +12,14 @@ import pytz
from dotenv import load_dotenv
from data.database import DatabaseManager
from utils.network_proxy import apply_global_proxy_from_env, get_proxy_url
from utils.project_logger import get_project_logger
from utils.send_msg import lark_notifier
# 加载环境变量
load_dotenv()
apply_global_proxy_from_env()
logger = get_project_logger("data.fetcher")
class BinanceDataFetcher:
@@ -30,17 +36,16 @@ class BinanceDataFetcher:
Args:
cache_dir: 缓存/数据库目录
"""
# 配置代理
# 统一读取代理配置,兼容全局代理环境变量。
proxies = None
use_proxy = os.getenv('USE_PROXY', 'false').lower() == 'true'
proxy_url = get_proxy_url()
if use_proxy:
socks5_proxy = os.getenv('SOCKS5_PROXY', 'socks5://127.0.0.1:7890')
if proxy_url:
proxies = {
'http': socks5_proxy,
'https': socks5_proxy
'http': proxy_url,
'https': proxy_url
}
print(f"✓ 使用SOCKS5代理: {socks5_proxy}")
logger.info("使用代理: %s", proxy_url)
try:
# 初始化Binance客户端(带代理)
@@ -51,13 +56,10 @@ class BinanceDataFetcher:
# 测试连接
self.client.ping()
print("✓ 已成功连接到Binance API")
logger.info("已成功连接到Binance API")
except Exception as e:
print(f"⚠ Binance API连接失败: {e}")
print("提示:")
print(" 1. 检查代理设置 (.env文件)")
print(" 2. 确保代理服务正在运行")
print(" 3. 或者使用已缓存的数据")
logger.warning("Binance API连接失败: %s", e)
logger.warning("提示: 检查代理设置、代理服务状态,或使用缓存数据")
self.client = None
# 初始化数据库管理器
@@ -69,7 +71,8 @@ class BinanceDataFetcher:
timeframe: Literal['1h', '15m'],
start_date: Optional[str] = None,
end_date: Optional[str] = None,
use_cache: bool = True
use_cache: bool = True,
max_retries: int = 3 # [新增] 最大重试次数参数
) -> pd.DataFrame:
"""获取加密货币对的OHLCV数据
@@ -109,13 +112,13 @@ class BinanceDataFetcher:
# 简单判断覆盖范围 (容忍少量缺失)
if min_ts <= start_dt + timedelta(days=2) and max_ts >= end_dt - timedelta(days=2):
print(f"从数据库加载数据:{symbol} {timeframe} ({len(df)} rows)")
logger.info("从数据库加载数据: %s %s (%s rows)", symbol, timeframe, len(df))
return df
# 2. 从API获取数据
if self.client is None:
if not df.empty:
print(f"⚠ API不可用,返回数据库现有数据 ({len(df)} rows)")
logger.warning("API不可用,返回数据库现有数据 (%s rows)", len(df))
return df
raise ConnectionError(
@@ -126,9 +129,10 @@ class BinanceDataFetcher:
pair = f"{symbol}USDT"
interval = self.INTERVALS[timeframe]
print(f"获取 {symbol} {timeframe} 数据,时间范围:{start_dt.date()} 至 {end_dt.date()}...")
logger.info("获取 %s %s 数据,时间范围: %s 至 %s", symbol, timeframe, start_dt.date(), end_dt.date())
# 获取K线数据
klines = []
for attempt in range(max_retries):
try:
klines = self.client.get_historical_klines(
symbol=pair,
@@ -136,13 +140,24 @@ class BinanceDataFetcher:
start_str=str(int(start_dt.timestamp() * 1000)),
end_str=str(int(end_dt.timestamp() * 1000))
)
# 如果成功获取,跳出循环
break
except Exception as e:
# 如果是最后一次尝试,且依然失败
if attempt == max_retries - 1:
logger.error("第 %s 次尝试失败,已达到最大重试次数。错误: %s", attempt + 1, e)
if not df.empty:
print(f"⚠ API请求失败 ({e}),返回数据库现有数据")
logger.warning("API请求最终失败,返回数据库现有数据")
return df
raise e
raise e # 如果没有缓存数据,抛出异常
# 计算等待时间 (指数退避: 2s, 4s, 8s...)
wait_time = 2 * (2 ** attempt)
logger.warning("请求失败(尝试 %s/%s),%s 秒后重试,错误: %s", attempt + 1, max_retries, wait_time, e)
time.sleep(wait_time)
if not klines:
lark_notifier.send_alert(f"数据获取失败: {symbol} {timeframe}", "连续3次请求或解析发生异常,已放弃重试")
if not df.empty:
return df
raise ValueError(f"未获取到 {symbol} {timeframe} 的数据")
@@ -164,7 +179,7 @@ class BinanceDataFetcher:
# 按时间排序
new_df = new_df.sort_values('timestamp').reset_index(drop=True)
print(f"已获取 {len(new_df)} 行数据")
logger.info("已获取 %s 行数据", len(new_df))
# 保存到数据库
self.db_manager.save_ohlcv(new_df, symbol, timeframe)
+49 -3
View File
@@ -53,12 +53,28 @@ class TradingDatabase:
error_message TEXT,
raw_data JSON,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
order_id VARCHAR(100),
INDEX idx_market_slug (market_slug),
INDEX idx_status (status)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
''')
conn.commit()
# 检查并添加 order_id 列 (Migration)
self._ensure_column_exists(conn, 'trades', 'order_id', 'VARCHAR(100)')
def _ensure_column_exists(self, conn, table_name, column_name, column_type):
"""确保表中存在指定列"""
try:
cursor = conn.cursor()
cursor.execute(f"SHOW COLUMNS FROM {table_name} LIKE '{column_name}'")
if not cursor.fetchone():
print(f"正在添加列 {column_name} 到表 {table_name}...")
cursor.execute(f"ALTER TABLE {table_name} ADD COLUMN {column_name} {column_type}")
conn.commit()
except Exception as e:
print(f"检查/添加列失败: {e}")
def log_trade(self, trade_data: dict):
"""记录交易信息"""
try:
@@ -72,8 +88,8 @@ class TradingDatabase:
INSERT INTO trades (
timestamp, market_title, market_slug, model_name,
prediction, probability, action, price, amount, size,
status, error_message, raw_data
) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
status, strategy, error_message, raw_data, order_id
) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
'''
cursor.execute(sql, (
@@ -88,8 +104,10 @@ class TradingDatabase:
trade_data.get('amount'),
trade_data.get('size'),
trade_data.get('status'),
trade_data.get('strategy'),
trade_data.get('error_message'),
raw_data_json
raw_data_json,
trade_data.get('order_id')
))
conn.commit()
except Exception as e:
@@ -108,3 +126,31 @@ class TradingDatabase:
except Exception as e:
print(f"检查已存在交易失败: {e}")
return False
def get_existing_trade(self, market_slug: str) -> dict:
"""获取指定市场的最近一次交易记录"""
try:
with self.get_connection() as conn:
cursor = conn.cursor()
# Get the latest trade for this market
sql = 'SELECT * FROM trades WHERE market_slug = %s ORDER BY id DESC LIMIT 1'
cursor.execute(sql, (market_slug,))
return cursor.fetchone()
except Exception as e:
print(f"获取交易记录失败: {e}")
return None
def update_trade_status(self, trade_id: int, status: str, error_message: str = None):
"""更新交易状态"""
try:
with self.get_connection() as conn:
cursor = conn.cursor()
if error_message:
sql = 'UPDATE trades SET status = %s, error_message = %s WHERE id = %s'
cursor.execute(sql, (status, error_message, trade_id))
else:
sql = 'UPDATE trades SET status = %s WHERE id = %s'
cursor.execute(sql, (status, trade_id))
conn.commit()
except Exception as e:
print(f"更新交易状态失败: {e}")
-406
View File
@@ -1,406 +0,0 @@
"""分析DLinear输出概率值的偏向性及其对集成策略的影响"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import pickle
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df, fit=True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df = df[:-self.forecast_horizon].reset_index(drop=True)
prices = df['close'].values.reshape(-1, 1)
future_prices = df['future_price'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
X, y = [], []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
y.append(scaled_future[i + self.lookback, 0])
return np.array(X), np.array(y), df
def inverse_transform_price(self, scaled_price):
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
def main():
print("="*80)
print("DLinear输出概率值分布分析 - 以0.1阶梯统计")
print("="*80)
# ========== 加载模型 ==========
print("\n[1/4] 加载模型...")
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
# 加载DLinear
from models.dlinear.model import DLinear
dlinear_checkpoint = torch.load(
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device
)
dlinear_model = DLinear(dlinear_checkpoint['config'])
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
dlinear_model = dlinear_model.to(device)
dlinear_model.eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
# 加载N-BEATS
from models.nbeats.model import NBeats
nbeats_checkpoint = torch.load(
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
map_location=device
)
nbeats_model = NBeats(nbeats_checkpoint['config'])
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
nbeats_model = nbeats_model.to(device)
nbeats_model.eval()
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
# 加载TTM-R2
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
"ibm-granite/granite-timeseries-ttm-r2",
trust_remote_code=True
)
ttm_model = ttm_model.to(device)
ttm_model.eval()
print(f"✓ 三个模型已加载")
# ========== 获取数据 ==========
print("\n[2/4] 获取2026年测试数据...")
fetcher = BinanceDataFetcher()
try:
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
end_date='2026-12-31', use_cache=False)
except:
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', use_cache=False)
print(f"✓ 测试数据: {len(df_2026)} 行")
# ========== 预处理 ==========
print("\n[3/4] 预处理数据...")
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
min_samples = min(len(X_dlinear), len(X_nbeats))
X_dlinear = X_dlinear[:min_samples]
y_dlinear = y_dlinear[:min_samples]
X_nbeats = X_nbeats[:min_samples]
y_nbeats = y_nbeats[:min_samples]
# ========== 预测并获取DLinear概率值 ==========
print("\n[4/4] 执行预测并分析DLinear输出概率...")
# DLinear预测(保留概率值)
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
with torch.no_grad():
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits.flatten())) # sigmoid概率
dlinear_preds = (dlinear_probs > 0.5).astype(int)
# N-BEATS预测
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
with torch.no_grad():
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
# TTM-R2预测
context_length = 512
ttm_preds_list = []
all_prices = df_2026['close'].values
all_open_prices = df_2026['open'].values
for i in range(min_samples):
start_idx = i
end_idx = start_idx + context_length
if end_idx > len(all_prices):
context_data = all_prices[-context_length:].reshape(1, -1, 1)
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
context_tensor = torch.FloatTensor(context_data).to(device)
with torch.no_grad():
output = ttm_model(context_tensor)
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
ttm_preds = np.array(ttm_preds_list)
# 双模型交集
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
two_model_valid = two_model_agree_up | two_model_agree_down
# 三模型多数投票
vote_sum = dlinear_preds + nbeats_preds + ttm_preds
three_model_preds = np.where(vote_sum >= 2, 1, 0)
# 真实标签
true_labels = y_dlinear
print(f"✓ 预测完成,开始分析DLinear概率分布...")
# ========== 按0.1阶梯分析 ==========
print("\n" + "="*80)
print("📊 DLinear输出概率分布分析(0.1阶梯)")
print("="*80)
# 定义概率区间
prob_bins = np.arange(0, 1.1, 0.1)
prob_labels = [f"{i/10:.1f}-{(i+1)/10:.1f}" for i in range(10)]
# 为每个样本分配到概率区间
prob_bin_indices = np.digitize(dlinear_probs, prob_bins) - 1
prob_bin_indices = np.clip(prob_bin_indices, 0, 9) # 确保在0-9范围内
# 初始化统计
stats = []
for i, bin_label in enumerate(prob_labels):
mask = prob_bin_indices == i
count = mask.sum()
if count == 0:
continue
# 该区间的真实标签
bin_true = true_labels[mask]
# 该区间DLinear的预测
bin_dlinear_pred = dlinear_preds[mask]
# 该区间在双模型交集中的表现
bin_two_model_valid = two_model_valid[mask]
bin_two_model_pred = np.full(count, -1)
bin_two_model_pred[two_model_agree_up[mask]] = 1
bin_two_model_pred[two_model_agree_down[mask]] = 0
# 该区间在三模型投票中的表现
bin_three_model_pred = three_model_preds[mask]
# 计算统计
dlinear_acc = (bin_dlinear_pred == bin_true).mean() * 100 if count > 0 else 0
two_model_valid_count = bin_two_model_valid.sum()
two_model_acc = ((bin_two_model_pred == bin_true) & (bin_two_model_pred != -1)).sum() / two_model_valid_count * 100 if two_model_valid_count > 0 else 0
two_model_coverage = two_model_valid_count / count * 100
three_model_acc = (bin_three_model_pred == bin_true).mean() * 100 if count > 0 else 0
# 平均概率值
avg_prob = dlinear_probs[mask].mean()
# 预测倾向
pred_up = bin_dlinear_pred.mean() * 100
stats.append({
'bin': bin_label,
'count': count,
'avg_prob': avg_prob,
'pred_up_pct': pred_up,
'dlinear_acc': dlinear_acc,
'two_model_acc': two_model_acc,
'two_model_cov': two_model_coverage,
'three_model_acc': three_model_acc
})
# 转换为DataFrame
stats_df = pd.DataFrame(stats)
# 显示统计表
print("\n概率区间分析:")
print(f"{'区间':<10} {'样本数':>6} {'平均概率':>8} {'预测↑%':>7} "
f"{'DL准确率':>9} {'双模准确率':>10} {'双模覆盖':>8} {'三模准确率':>10}")
print("-" * 95)
for _, row in stats_df.iterrows():
# 根据准确率添加标记
dl_mark = "🔥" if row['dlinear_acc'] >= 70 else ("✅" if row['dlinear_acc'] >= 60 else ("⚠️" if row['dlinear_acc'] < 45 else ""))
two_mark = "🔥" if row['two_model_acc'] >= 70 else ("✅" if row['two_model_acc'] >= 60 else ("⚠️" if row['two_model_acc'] < 45 else ""))
three_mark = "🔥" if row['three_model_acc'] >= 70 else ("✅" if row['three_model_acc'] >= 60 else ("⚠️" if row['three_model_acc'] < 45 else ""))
print(f"{row['bin']:<10} {row['count']:>6} {row['avg_prob']:>8.3f} {row['pred_up_pct']:>6.1f}% "
f"{row['dlinear_acc']:>8.1f}{dl_mark} "
f"{row['two_model_acc']:>9.1f}{two_mark} "
f"{row['two_model_cov']:>7.1f}% "
f"{row['three_model_acc']:>9.1f}{three_mark}")
# ========== 关键发现 ==========
print("\n" + "="*80)
print("🔍 关键发现")
print("="*80)
# 极端概率区间(高置信度)
high_conf_up = stats_df[stats_df['avg_prob'] >= 0.65]
high_conf_down = stats_df[stats_df['avg_prob'] <= 0.35]
print(f"\n📈 高置信度看涨区间 (概率≥0.65):")
if len(high_conf_up) > 0:
total_samples = high_conf_up['count'].sum()
avg_dl_acc = (high_conf_up['dlinear_acc'] * high_conf_up['count']).sum() / total_samples
avg_two_acc = (high_conf_up['two_model_acc'] * high_conf_up['count']).sum() / total_samples
avg_three_acc = (high_conf_up['three_model_acc'] * high_conf_up['count']).sum() / total_samples
print(f" 样本数: {total_samples} ({total_samples/len(dlinear_probs)*100:.1f}%)")
print(f" DLinear平均准确率: {avg_dl_acc:.2f}%")
print(f" 双模型平均准确率: {avg_two_acc:.2f}%")
print(f" 三模型平均准确率: {avg_three_acc:.2f}%")
else:
print(" 无数据")
print(f"\n📉 高置信度看跌区间 (概率≤0.35):")
if len(high_conf_down) > 0:
total_samples = high_conf_down['count'].sum()
avg_dl_acc = (high_conf_down['dlinear_acc'] * high_conf_down['count']).sum() / total_samples
avg_two_acc = (high_conf_down['two_model_acc'] * high_conf_down['count']).sum() / total_samples
avg_three_acc = (high_conf_down['three_model_acc'] * high_conf_down['count']).sum() / total_samples
print(f" 样本数: {total_samples} ({total_samples/len(dlinear_probs)*100:.1f}%)")
print(f" DLinear平均准确率: {avg_dl_acc:.2f}%")
print(f" 双模型平均准确率: {avg_two_acc:.2f}%")
print(f" 三模型平均准确率: {avg_three_acc:.2f}%")
else:
print(" 无数据")
# 中等置信度区间
mid_conf = stats_df[(stats_df['avg_prob'] > 0.45) & (stats_df['avg_prob'] < 0.55)]
print(f"\n⚖️ 中等置信度区间 (0.45-0.55):")
if len(mid_conf) > 0:
total_samples = mid_conf['count'].sum()
avg_dl_acc = (mid_conf['dlinear_acc'] * mid_conf['count']).sum() / total_samples
avg_two_acc = (mid_conf['two_model_acc'] * mid_conf['count']).sum() / total_samples if total_samples > 0 else 0
avg_three_acc = (mid_conf['three_model_acc'] * mid_conf['count']).sum() / total_samples
print(f" 样本数: {total_samples} ({total_samples/len(dlinear_probs)*100:.1f}%)")
print(f" DLinear平均准确率: {avg_dl_acc:.2f}%")
print(f" 双模型平均准确率: {avg_two_acc:.2f}%")
print(f" 三模型平均准确率: {avg_three_acc:.2f}%")
print(f" ⚠️ 建议: 低置信度区间,考虑弃权或减仓")
else:
print(" 无数据")
# 找出表现最好的概率区间
best_bin = stats_df.loc[stats_df['dlinear_acc'].idxmax()]
worst_bin = stats_df.loc[stats_df['dlinear_acc'].idxmin()]
print(f"\n🏆 DLinear表现最佳区间:")
print(f" 区间: {best_bin['bin']}")
print(f" 准确率: {best_bin['dlinear_acc']:.1f}%")
print(f" 样本数: {best_bin['count']}")
print(f"\n💀 DLinear表现最差区间:")
print(f" 区间: {worst_bin['bin']}")
print(f" 准确率: {worst_bin['dlinear_acc']:.1f}%")
print(f" 样本数: {worst_bin['count']}")
# ========== 概率分布可视化 ==========
print("\n" + "="*80)
print("📊 DLinear概率分布直方图")
print("="*80)
print("\n样本数分布:")
max_count = stats_df['count'].max()
for _, row in stats_df.iterrows():
bar_length = int(row['count'] / max_count * 40)
bar = "█" * bar_length
print(f"{row['bin']:<10} {bar} {row['count']}")
print("\n" + "="*80)
print("✓ DLinear概率分布分析完成!")
print("="*80)
if __name__ == "__main__":
main()
+295
View File
@@ -0,0 +1,295 @@
# Windows 环境安装指南
本指南帮助您在 Windows 系统上部署加密货币预测系统。
---
## 一、前置要求
- Windows 10/11 (64位)
- 至少 8GB 内存
- 至少 10GB 可用磁盘空间
- 网络连接(需访问 GitHub、PyPI、Binance API)
---
## 二、安装 Miniconda
### 1. 下载 Miniconda
访问 [Miniconda 官网](https://docs.conda.io/en/latest/miniconda.html) 下载 Windows 64-bit 安装包,或直接下载:
```
https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe
```
### 2. 安装
1. 双击运行安装程序
2. 选择 "Just Me" (仅为当前用户安装)
3. 安装路径建议使用默认值或 `C:\Miniconda3`
4. **重要**:勾选 "Add Miniconda3 to my PATH environment variable"
5. 完成安装
### 3. 验证安装
打开 **命令提示符 (CMD)** 或 **PowerShell**:
```powershell
conda --version
# 应显示类似: conda 24.x.x
```
---
## 三、创建 Conda 环境
### 1. 创建 Python 3.12 环境
```powershell
conda create -n crypto_pred_312 python=3.12 -y
```
### 2. 激活环境
```powershell
conda activate crypto_pred_312
```
激活后,命令行前缀应显示 `(crypto_pred_312)`
---
## 四、克隆项目
### 1. 安装 Git (如未安装)
下载并安装 [Git for Windows](https://git-scm.com/download/win)
### 2. 克隆仓库
```powershell
cd C:\Projects # 或您喜欢的目录
git clone http://192.168.50.24:3000/subuhui/crpto_predictor.git
cd crpto_predictor
```
---
## 五、安装依赖
### 1. 安装 PyTorch (CPU 版本)
```powershell
# CPU 版本 PyTorch(Windows 不使用 GPU)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
```
### 2. 安装项目依赖
```powershell
pip install -r requirements.txt
```
### 3. 安装 IBM 时序模型 (TTM-R2)
```powershell
# PyPI 包名是 granite-tsfm,导入时用 tsfm_public
pip install granite-tsfm
```
验证安装:
```powershell
python -c "from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction; print('TTM-R2 安装成功')"
```
---
## 六、配置环境变量
### 1. 复制环境变量模板
```powershell
copy .env.example .env
```
### 2. 编辑 .env 文件
使用记事本或 VSCode 编辑 `.env` 文件:
```ini
# MySQL 数据库配置
MYSQL_HOST=your_mysql_host
MYSQL_PORT=3306
MYSQL_USER=your_username
MYSQL_PASSWORD=your_password
MYSQL_DB=crypto_predictor
# Binance API (可选,用于实时数据)
BINANCE_API_KEY=your_api_key
BINANCE_API_SECRET=your_api_secret
# Polymarket 配置 (可选,用于自动交易)
POLYMARKET_PRIVATE_KEY=your_private_key
DRY_RUN=true # 设为 true 进行模拟交易
# 代理配置 (可选)
USE_PROXY=false
SOCKS5_PROXY=socks5://127.0.0.1:1080
```
---
## 七、数据库配置
### 1. 安装 MySQL
下载并安装 [MySQL Community Server](https://dev.mysql.com/downloads/mysql/)
或使用 Docker:
```powershell
docker run -d --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=yourpassword mysql:8.0
```
### 2. 创建数据库
```sql
CREATE DATABASE crypto_predictor CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
```
### 3. 初始化表结构
首次运行时会自动创建表,或手动执行:
```powershell
python -c "from api.models import init_model_predictions_table; from data.trading_db import TradingDatabase; TradingDatabase()"
```
---
## 八、验证安装
### 1. 测试 PyTorch
```powershell
python -c "import torch; print(f'PyTorch: {torch.__version__}'); print(f'Device: cpu')"
```
### 2. 测试模型加载
```powershell
python -c "from training.config import TrainingConfig; c = TrainingConfig(); print(f'Device: {c.get_device()}')"
# 应输出: Device: cpu
```
### 3. 测试数据库连接
```powershell
python -c "from api.models import get_db_connection; conn = get_db_connection(); print('数据库连接成功!')"
```
---
## 九、运行系统
### 方式一:双击批处理脚本 (推荐)
双击项目根目录的 `start_system.bat`
### 方式二:PowerShell 脚本
```powershell
.\start_system.ps1
```
### 方式三:Python 脚本
```powershell
python start_system.py
```
### 方式四:分别启动
```powershell
# 终端 1: API 服务
python -m uvicorn api.main:app --host 0.0.0.0 --port 8000
# 终端 2: 自动交易
python auto_trade_polymarket.py
```
---
## 十、访问系统
- **前端看板**: http://localhost:8000
- **API 文档**: http://localhost:8000/docs
---
## 常见问题
### Q1: `conda activate` 不生效
在 PowerShell 中需要先初始化 conda:
```powershell
conda init powershell
# 重启 PowerShell
```
### Q2: pip 安装超时
使用国内镜像:
```powershell
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
```
### Q3: PyTorch 安装失败
确保使用 CPU 版本的安装命令,不要安装 CUDA 版本。
### Q4: 数据库连接失败
1. 检查 MySQL 服务是否运行
2. 检查 `.env` 中的数据库配置
3. 检查防火墙是否允许 3306 端口
### Q5: 模型文件缺失
确保 `models/checkpoints/` 目录包含预训练模型文件。如缺失,联系管理员获取。
---
## 目录结构
```
crpto_predictor/
├── api/ # API 服务
│ ├── main.py # FastAPI 入口
│ ├── models.py # 数据库模型
│ ├── prediction_service.py # 预测服务
│ └── static/ # 前端静态文件
├── models/ # 模型定义和权重
│ ├── checkpoints/ # 模型权重文件
│ └── dlinear/ # DLinear 模型
├── data/ # 数据层
├── training/ # 训练配置
├── start_system.py # 跨平台启动脚本
├── start_system.bat # Windows CMD 脚本
├── start_system.ps1 # Windows PowerShell 脚本
├── auto_trade_polymarket.py # 自动交易程序
├── requirements.txt # 依赖列表
└── .env # 环境变量 (需自行创建)
```
---
## 更新项目
```powershell
git pull origin main
pip install -r requirements.txt
```
---
*如有问题,请联系管理员。*
-395
View File
@@ -1,395 +0,0 @@
"""四模型集成每日准确率分析 - 多时间段测试(2019年1月、2020年1月、2026年1月)"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import pickle
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df, fit=True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df = df[:-self.forecast_horizon].reset_index(drop=True)
prices = df['close'].values.reshape(-1, 1)
future_prices = df['future_price'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
X, y = [], []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
y.append(scaled_future[i + self.lookback, 0])
return np.array(X), np.array(y), df
def inverse_transform_price(self, scaled_price):
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
def predict_period(period_name, start_date, end_date, dlinear_model, nbeats_model, ttm_model, timesfm_model,
dlinear_preprocessor, nbeats_preprocessor, device):
"""预测单个时间段的数据"""
print(f"\n{'='*80}")
print(f"📅 {period_name} ({start_date} 至 {end_date})")
print(f"{'='*80}")
# 获取数据
fetcher = BinanceDataFetcher()
df = fetcher.fetch_ohlcv('BTC', '1h', start_date=start_date, end_date=end_date, use_cache=False)
print(f"✓ 数据: {len(df)} 行")
# 预处理
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df, fit=False)
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df, fit=False)
min_samples = min(len(X_dlinear), len(X_nbeats))
X_dlinear = X_dlinear[:min_samples]
y_dlinear = y_dlinear[:min_samples]
X_nbeats = X_nbeats[:min_samples]
print(f"✓ 样本数: {min_samples}")
# DLinear预测
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
with torch.no_grad():
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits))
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten()
# N-BEATS预测
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
with torch.no_grad():
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
# TTM-R2预测
context_length = 512
ttm_preds_list = []
all_prices = df['close'].values
all_open_prices = df['open'].values
for i in range(min_samples):
start_idx = i
end_idx = start_idx + context_length
if end_idx > len(all_prices):
context_data = all_prices[-context_length:].reshape(1, -1, 1)
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
context_tensor = torch.FloatTensor(context_data).to(device)
with torch.no_grad():
output = ttm_model(context_tensor)
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
ttm_preds = np.array(ttm_preds_list)
# TimesFM预测
timesfm_preds_list = []
for i in range(min_samples):
start_idx = i
end_idx = start_idx + 512
if end_idx > len(all_prices):
context_prices = all_prices[-512:]
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_prices = all_prices[start_idx:end_idx]
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
try:
point_forecast, _ = timesfm_model.forecast(horizon=1, inputs=[context_prices])
pred_price = point_forecast[0, 0]
timesfm_preds_list.append(1 if pred_price > current_open_price else 0)
except:
timesfm_preds_list.append(np.random.randint(0, 2))
timesfm_preds = np.array(timesfm_preds_list)
# 集成策略
true_labels = y_dlinear
# 双模型交集
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
two_model_preds = np.full(len(dlinear_preds), -1)
two_model_preds[two_model_agree_up] = 1
two_model_preds[two_model_agree_down] = 0
# 三模型投票
vote_sum_3 = dlinear_preds + nbeats_preds + ttm_preds
three_model_preds = np.where(vote_sum_3 >= 2, 1, 0)
# 四模型投票
vote_sum_4 = dlinear_preds + nbeats_preds + ttm_preds + timesfm_preds
four_model_preds = np.where(vote_sum_4 >= 3, 1, 0)
# 构建每日结果DataFrame
timestamps = df['timestamp'].iloc[168:168+min_samples].values
timestamps_utc = pd.to_datetime(timestamps)
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
dates = timestamps_et.date
results_df = pd.DataFrame({
'date': dates,
'timestamp_et': timestamps_et,
'true_label': true_labels,
'dlinear_pred': dlinear_preds,
'nbeats_pred': nbeats_preds,
'ttm_pred': ttm_preds,
'timesfm_pred': timesfm_preds,
'two_model_pred': two_model_preds,
'three_model_pred': three_model_preds,
'four_model_pred': four_model_preds,
})
# 按天统计
daily_stats = results_df.groupby('date').agg({
'true_label': 'count',
'dlinear_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
'nbeats_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
'ttm_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
'timesfm_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
'two_model_pred': lambda x: ((x == results_df.loc[x.index, 'true_label']) & (x != -1)).sum(),
'three_model_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
'four_model_pred': lambda x: (x == results_df.loc[x.index, 'true_label']).sum(),
})
daily_stats.columns = ['total', 'dlinear_correct', 'nbeats_correct', 'ttm_correct',
'timesfm_correct', 'two_model_correct', 'three_model_correct', 'four_model_correct']
# 计算每天的有效预测数(双模型)
two_model_valid = results_df.groupby('date')['two_model_pred'].apply(lambda x: (x != -1).sum())
daily_stats['two_model_valid'] = two_model_valid
# 计算准确率
daily_stats['dlinear_acc'] = daily_stats['dlinear_correct'] / daily_stats['total'] * 100
daily_stats['nbeats_acc'] = daily_stats['nbeats_correct'] / daily_stats['total'] * 100
daily_stats['ttm_acc'] = daily_stats['ttm_correct'] / daily_stats['total'] * 100
daily_stats['timesfm_acc'] = daily_stats['timesfm_correct'] / daily_stats['total'] * 100
daily_stats['two_model_acc'] = daily_stats['two_model_correct'] / daily_stats['two_model_valid'] * 100
daily_stats['three_model_acc'] = daily_stats['three_model_correct'] / daily_stats['total'] * 100
daily_stats['four_model_acc'] = daily_stats['four_model_correct'] / daily_stats['total'] * 100
daily_stats['two_model_acc'] = daily_stats['two_model_acc'].fillna(0)
return daily_stats, results_df
def main():
print("="*80)
print("四模型集成每日准确率分析 - 多时间段测试")
print("测试时段: 2019年1月、2020年1月、2026年1月")
print("="*80)
# ========== 加载模型 ==========
print("\n[1/2] 加载模型...")
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
from models.dlinear.model import DLinear
dlinear_checkpoint = torch.load(
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device, weights_only=False
)
dlinear_model = DLinear(dlinear_checkpoint['config'])
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
dlinear_model = dlinear_model.to(device)
dlinear_model.eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
from models.nbeats.model import NBeats
nbeats_checkpoint = torch.load(
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
map_location=device, weights_only=False
)
nbeats_model = NBeats(nbeats_checkpoint['config'])
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
nbeats_model = nbeats_model.to(device)
nbeats_model.eval()
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
"ibm-granite/granite-timeseries-ttm-r2",
trust_remote_code=True
)
ttm_model = ttm_model.to(device)
ttm_model.eval()
import timesfm
timesfm_model = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
"google/timesfm-2.5-200m-pytorch",
torch_compile=False
)
timesfm_model.compile(
timesfm.ForecastConfig(
max_context=512,
max_horizon=1,
normalize_inputs=True,
use_continuous_quantile_head=True,
force_flip_invariance=True,
infer_is_positive=True,
fix_quantile_crossing=True,
)
)
print("✓ 四个模型已加载 (DLinear, N-BEATS, TTM-R2, TimesFM)")
# ========== 多时间段预测 ==========
print("\n[2/2] 多时间段预测...")
periods = [
("2019年1月", "2019-01-01", "2019-01-31"),
("2020年1月", "2020-01-01", "2020-01-31"),
("2026年1月", "2026-01-01", "2026-01-31"),
]
all_period_stats = {}
for period_name, start_date, end_date in periods:
daily_stats, results_df = predict_period(
period_name, start_date, end_date,
dlinear_model, nbeats_model, ttm_model, timesfm_model,
dlinear_preprocessor, nbeats_preprocessor, device
)
all_period_stats[period_name] = daily_stats
# 显示每日准确率
print(f"\n每日准确率明细:")
print(f"{'日期':<12} {'样本':>4} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'TimesFM':>8} {'双模型':>8} {'三模型':>8} {'四模型':>8}")
print("-" * 95)
for date, row in daily_stats.iterrows():
print(f"{date} {row['total']:>4.0f} "
f"{row['dlinear_acc']:>7.1f}% "
f"{row['nbeats_acc']:>7.1f}% "
f"{row['ttm_acc']:>7.1f}% "
f"{row['timesfm_acc']:>7.1f}% "
f"{row['two_model_acc']:>7.1f}% "
f"{row['three_model_acc']:>7.1f}% "
f"{row['four_model_acc']:>7.1f}%")
# 统计总体
print(f"\n{period_name} 总体统计:")
print(f" DLinear: {daily_stats['dlinear_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
print(f" N-BEATS: {daily_stats['nbeats_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
print(f" TTM-R2: {daily_stats['ttm_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
print(f" TimesFM: {daily_stats['timesfm_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
print(f" 双模型: {daily_stats['two_model_correct'].sum() / daily_stats['two_model_valid'].sum() * 100:.2f}% (覆盖率: {daily_stats['two_model_valid'].sum() / daily_stats['total'].sum() * 100:.1f}%)")
print(f" 三模型: {daily_stats['three_model_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
print(f" 四模型: {daily_stats['four_model_correct'].sum() / daily_stats['total'].sum() * 100:.2f}%")
# ========== 跨时期对比 ==========
print("\n" + "="*80)
print("📊 跨时期对比总结")
print("="*80)
print(f"\n{'时期':<12} {'DLinear':>9} {'N-BEATS':>9} {'TTM-R2':>9} {'TimesFM':>9} {'双模型':>9} {'三模型':>9} {'四模型':>9}")
print("-" * 95)
for period_name in ["2019年1月", "2020年1月", "2026年1月"]:
stats = all_period_stats[period_name]
print(f"{period_name:<12} "
f"{stats['dlinear_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
f"{stats['nbeats_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
f"{stats['ttm_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
f"{stats['timesfm_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
f"{stats['two_model_correct'].sum() / stats['two_model_valid'].sum() * 100:>8.2f}% "
f"{stats['three_model_correct'].sum() / stats['total'].sum() * 100:>8.2f}% "
f"{stats['four_model_correct'].sum() / stats['total'].sum() * 100:>8.2f}%")
print("\n" + "="*80)
print("✓ 多时间段每日准确率分析完成!")
print("="*80)
if __name__ == "__main__":
main()
+289
View File
@@ -0,0 +1,289 @@
"""2026年1月BTC数据 - DLinear 准确率测试
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
价格对比:预测结果与预测K线(t+1)的收盘价做对比
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口
输出:每小时详细数据、每天准确率、总准确率、美股开盘标记
"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import numpy as np
import pandas as pd
import pickle
import pandas_ta as ta
from datetime import timedelta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
# ==================== 时区与美股判断 ====================
def utc_to_beijing(ts):
"""UTC时间转北京时间(UTC+8)"""
return ts + timedelta(hours=8)
def is_us_stock_open_by_utc(utc_ts):
"""根据UTC时间判断美股当天是否开盘
美股交易日基于美东时间(ET),UTC-5(冬令时)
2026年1月在冬令时范围内(夏令时3月第二个周日开始)
所以美东 = UTC - 5h
"""
et_ts = utc_ts - timedelta(hours=5)
et_date = pd.Timestamp(et_ts.date())
us_holidays_jan_2026 = {
pd.Timestamp('2026-01-01'), # 元旦 New Year's Day
pd.Timestamp('2026-01-19'), # 马丁·路德·金日 MLK Day
}
if et_date.weekday() >= 5:
return False
if et_date in us_holidays_jan_2026:
return False
return True
# ==================== pickle反序列化占位类 ====================
class ClassificationPreprocessor:
"""pickle.load时需要能找到这个类名"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
# ==================== 工具函数 ====================
def fmt_pred(val):
"""格式化预测结果"""
if val is None or (isinstance(val, float) and np.isnan(val)):
return "N/A"
return "涨" if int(val) == 1 else "跌"
def fmt_result(pred, actual):
"""格式化对错"""
if pred is None or (isinstance(pred, float) and np.isnan(pred)):
return " "
return "✓" if int(pred) == int(actual) else "✗"
def main():
print("=" * 90)
print("2026年1月 BTC 1h 涨跌预测准确率测试 (DLinear)")
print("输入:预测K线的前一根 | 对比:预测结果 vs 预测K线收盘价 | 时间:北京时间(UTC+8)")
print("=" * 90)
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
print(f"使用设备: {device}")
# ========== 步骤1: 加载模型 ==========
print("\n[1/4] 加载模型...")
from models.dlinear.model import DLinear
checkpoint = torch.load('./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device, weights_only=False)
model = DLinear(checkpoint['config'])
model.load_state_dict(checkpoint['model_state_dict'])
model = model.to(device).eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
preprocessor = pickle.load(f)
print(" ✓ DLinear已加载")
# ========== 步骤2: 获取测试数据 ==========
print("\n[2/4] 获取数据...")
fetcher = BinanceDataFetcher()
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', end_date='2026-03-01', use_cache=False)
print(f"数据范围(UTC): {df['timestamp'].min()} -> {df['timestamp'].max()}")
print(f"总行数: {len(df)}")
# ========== 步骤3: 预处理与预测 ==========
print("\n[3/4] 运行预测...")
# DLinear 整体预处理(无create_labels,整个df一次性处理)
lookback = preprocessor.lookback
df_dl = df.copy()
df_dl['rsi'] = ta.rsi(df_dl['close'], length=14)
macd = ta.macd(df_dl['close'])
if macd is not None and not macd.empty:
df_dl['macd'] = macd.iloc[:, 0]
df_dl['macd_hist'] = macd.iloc[:, 1]
df_dl['macd_signal'] = macd.iloc[:, 2]
df_dl = df_dl.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
scaled = preprocessor.scaler.transform(df_dl[feature_cols])
X_dl = []
idx_dl = []
for i in range(len(scaled) - lookback):
X_dl.append(scaled[i:i + lookback])
idx_dl.append(i + lookback - 1)
X_dl = np.array(X_dl)
# 构建 timestamp -> 窗口索引映射
dl_ts_map = {}
for i, idx in enumerate(idx_dl):
ts = df_dl.iloc[idx]['timestamp']
dl_ts_map[ts] = i
# 找出1月(北京时间)的所有预测目标K线索引
jan_targets = []
for target_idx in range(1, len(df)):
target_time_utc = df.iloc[target_idx]['timestamp']
target_time_bj = utc_to_beijing(target_time_utc)
if target_time_bj.year == 2026 and target_time_bj.month == 2:
jan_targets.append(target_idx)
print(f"1月(北京时间)目标K线数: {len(jan_targets)}")
results = []
for target_idx in jan_targets:
target_time_utc = df.iloc[target_idx]['timestamp']
target_time_bj = utc_to_beijing(target_time_utc)
# 输入K线索引(前一根)
input_idx = target_idx - 1
current_close = df.iloc[input_idx]['close']
target_close = df.iloc[target_idx]['close']
actual_label = 1 if target_close > current_close else 0
# --- DLinear 预测 ---
dl_pred = None
dl_prob = None
input_ts = df.iloc[input_idx]['timestamp']
dl_i = dl_ts_map.get(input_ts)
if dl_i is not None and dl_i < len(X_dl):
x_tensor = torch.FloatTensor(X_dl[dl_i:dl_i+1]).to(device)
with torch.no_grad():
logits = model(x_tensor).cpu().numpy()
prob = 1 / (1 + np.exp(-logits))
dl_pred = int(prob.item() > 0.5)
dl_prob = float(prob.item())
results.append({
'time_utc': target_time_utc,
'time_bj': target_time_bj,
'current_close': current_close,
'target_close': target_close,
'actual': actual_label,
'dlinear': dl_pred,
'dl_prob': dl_prob,
})
# ========== 步骤4: 输出结果 ==========
if not results:
print("没有找到符合条件的测试样本")
return
df_res = pd.DataFrame(results)
df_res['date_bj'] = df_res['time_bj'].dt.date
df_res['us_open'] = df_res['time_utc'].apply(is_us_stock_open_by_utc)
total_samples = len(df_res)
print(f"\n[4/4] 准确率统计 (2026年1月,北京时间)")
print(f"总样本数: {total_samples}")
# ========== 每小时详细数据 ==========
print("\n" + "=" * 90)
print("每小时详细数据")
print("=" * 90)
hourly_header = (
f"{'北京时间':<18} {'前收盘':>10} {'现收盘':>10} {'实际':>4} "
f"{'DL':>3} {'':>1} {'概率':>6}"
)
for date_bj, day_group in df_res.groupby('date_bj'):
us_open = day_group['us_open'].iloc[0]
us_str = "美股开盘" if us_open else "美股休市"
dl_day_correct = (day_group['dlinear'] == day_group['actual']).sum()
dl_day_total = day_group['dlinear'].notna().sum()
dl_day_acc = dl_day_correct / dl_day_total if dl_day_total > 0 else 0
print(f"\n--- {date_bj} ({us_str}) --- DL准确率:{dl_day_acc:.1%} ({dl_day_correct}/{dl_day_total})")
print(hourly_header)
print("-" * 90)
for _, row in day_group.iterrows():
time_str = row['time_bj'].strftime('%m-%d %H:%M')
actual_str = fmt_pred(row['actual'])
dl_str = fmt_pred(row['dlinear'])
dl_r = fmt_result(row['dlinear'], row['actual'])
prob_str = f"{row['dl_prob']:.3f}" if row['dl_prob'] is not None else " N/A"
print(
f"{time_str:<18} {row['current_close']:>10.2f} {row['target_close']:>10.2f} {actual_str:>4} "
f"{dl_str:>3} {dl_r:>1} {prob_str:>6}"
)
# ========== 每日准确率汇总表 ==========
print("\n\n" + "=" * 90)
print("每日准确率汇总 (北京时间)")
print("=" * 90)
header = f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8}"
print(header)
print("-" * 90)
for date_bj, group in df_res.groupby('date_bj'):
total = len(group)
us_open = group['us_open'].iloc[0]
us_str = "开盘" if us_open else "休市"
dl_valid = group.dropna(subset=['dlinear'])
dl_acc = (dl_valid['dlinear'] == dl_valid['actual']).mean() if len(dl_valid) > 0 else 0
dl_str = f"{dl_acc:.1%}"
print(f"{str(date_bj):<14} {total:>4} {us_str:>6} {dl_str:>8}")
print("-" * 90)
# --- 总准确率 ---
dl_valid_all = df_res.dropna(subset=['dlinear'])
dl_total = (dl_valid_all['dlinear'] == dl_valid_all['actual']).mean() if len(dl_valid_all) > 0 else 0
print(f"{'总计':<14} {total_samples:>4} {'':>6} {dl_total:.2%}")
print("=" * 90)
# --- 美股开盘日 vs 休市日对比 ---
print("\n[美股开盘日 vs 休市日 准确率对比]")
print("-" * 60)
for label, flag in [("开盘日", True), ("休市日", False)]:
subset = df_res[df_res['us_open'] == flag]
if len(subset) == 0:
continue
n = len(subset)
dl_s = subset.dropna(subset=['dlinear'])
dl_a = (dl_s['dlinear'] == dl_s['actual']).mean() if len(dl_s) > 0 else 0
print(f" {label} ({n}样本): DLinear={dl_a:.2%}")
print("-" * 60)
# --- 预测分布统计 ---
print("\n[预测分布统计]")
print("-" * 60)
actual_up = (df_res['actual'] == 1).sum()
actual_down = (df_res['actual'] == 0).sum()
print(f" 实际: 上涨={actual_up} ({actual_up/total_samples:.1%}) 下跌={actual_down} ({actual_down/total_samples:.1%})")
dl_valid = df_res['dlinear'].dropna()
if len(dl_valid) > 0:
up = (dl_valid == 1).sum()
down = (dl_valid == 0).sum()
print(f" DLinear: 预测上涨={up} ({up/len(dl_valid):.1%}) 预测下跌={down} ({down/len(dl_valid):.1%})")
print("=" * 90)
if __name__ == "__main__":
main()
-454
View File
@@ -1,454 +0,0 @@
"""五模型集成预测:DLinear + N-BEATS + TTM-R2 + TimesFM 2.5 + CatBoost"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import pickle
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
from catboost import CatBoostClassifier
from data.fetcher import BinanceDataFetcher
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df, fit=True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df = df[:-self.forecast_horizon].reset_index(drop=True)
prices = df['close'].values.reshape(-1, 1)
future_prices = df['future_price'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
X, y = [], []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
y.append(scaled_future[i + self.lookback, 0])
return np.array(X), np.array(y), df
def inverse_transform_price(self, scaled_price):
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
def add_catboost_features(df):
"""为CatBoost添加丰富的技术指标特征"""
df = df.copy()
# 基础价格特征
df['price_change'] = df['close'] - df['open']
df['price_change_pct'] = (df['close'] - df['open']) / df['open']
df['high_low_range'] = df['high'] - df['low']
df['high_low_range_pct'] = (df['high'] - df['low']) / df['low']
# 移动平均线
df['sma_7'] = ta.sma(df['close'], length=7)
df['sma_25'] = ta.sma(df['close'], length=25)
df['sma_99'] = ta.sma(df['close'], length=99)
df['ema_12'] = ta.ema(df['close'], length=12)
df['ema_26'] = ta.ema(df['close'], length=26)
# RSI
df['rsi_14'] = ta.rsi(df['close'], length=14)
df['rsi_7'] = ta.rsi(df['close'], length=7)
# MACD
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
# 布林带
bbands = ta.bbands(df['close'], length=20)
if bbands is not None and not bbands.empty:
df['bb_upper'] = bbands.iloc[:, 0]
df['bb_mid'] = bbands.iloc[:, 1]
df['bb_lower'] = bbands.iloc[:, 2]
df['bb_width'] = (df['bb_upper'] - df['bb_lower']) / df['bb_mid']
df['bb_position'] = (df['close'] - df['bb_lower']) / (df['bb_upper'] - df['bb_lower'])
# ATR
df['atr'] = ta.atr(df['high'], df['low'], df['close'], length=14)
# 成交量特征
df['volume_sma_7'] = ta.sma(df['volume'], length=7)
df['volume_ratio'] = df['volume'] / df['volume_sma_7']
# OBV
df['obv'] = ta.obv(df['close'], df['volume'])
# Stochastic
stoch = ta.stoch(df['high'], df['low'], df['close'])
if stoch is not None and not stoch.empty:
df['stoch_k'] = stoch.iloc[:, 0]
df['stoch_d'] = stoch.iloc[:, 1]
# ADX
adx = ta.adx(df['high'], df['low'], df['close'])
if adx is not None and not adx.empty:
df['adx'] = adx.iloc[:, 0]
# 价格动量
df['momentum_1'] = df['close'].pct_change(1)
df['momentum_3'] = df['close'].pct_change(3)
df['momentum_7'] = df['close'].pct_change(7)
return df
def main():
print("="*80)
print("五模型集成预测:DLinear + N-BEATS + TTM-R2 + TimesFM + CatBoost")
print("="*80)
# ========== 步骤1: 加载PyTorch模型 ==========
print("\n[1/8] 加载DLinear和N-BEATS模型...")
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
# 加载DLinear
from models.dlinear.model import DLinear
dlinear_checkpoint = torch.load(
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device
)
dlinear_model = DLinear(dlinear_checkpoint['config'])
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
dlinear_model = dlinear_model.to(device)
dlinear_model.eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
print(f"✓ DLinear已加载")
# 加载N-BEATS
from models.nbeats.model import NBeats
nbeats_checkpoint = torch.load(
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
map_location=device
)
nbeats_model = NBeats(nbeats_checkpoint['config'])
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
nbeats_model = nbeats_model.to(device)
nbeats_model.eval()
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
print(f"✓ N-BEATS已加载")
# ========== 步骤2: 加载TTM-R2 ==========
print("\n[2/8] 加载TTM-R2零样本模型...")
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
"ibm-granite/granite-timeseries-ttm-r2",
trust_remote_code=True
)
ttm_model = ttm_model.to(device)
ttm_model.eval()
print(f"✓ TTM-R2已加载")
# ========== 步骤3: 加载TimesFM 2.5 ==========
print("\n[3/8] 加载TimesFM 2.5模型...")
import timesfm
timesfm_model = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
"google/timesfm-2.5-200m-pytorch",
torch_compile=False
)
timesfm_model.compile(
timesfm.ForecastConfig(
max_context=512,
max_horizon=1,
normalize_inputs=True,
use_continuous_quantile_head=True,
force_flip_invariance=True,
infer_is_positive=True,
fix_quantile_crossing=True,
)
)
print(f"✓ TimesFM 2.5已加载")
# ========== 步骤4: 加载CatBoost ==========
print("\n[4/8] 加载CatBoost模型...")
catboost_model = CatBoostClassifier()
catboost_model.load_model('./models/checkpoints/catboost_classification_2026/catboost_model.cbm')
with open('./models/checkpoints/catboost_classification_2026/metadata.pkl', 'rb') as f:
catboost_metadata = pickle.load(f)
catboost_feature_cols = catboost_metadata['feature_cols']
print(f"✓ CatBoost已加载 (特征数: {len(catboost_feature_cols)})")
# ========== 步骤5: 获取测试数据 ==========
print("\n[5/8] 获取2026年1月测试数据...")
fetcher = BinanceDataFetcher()
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
end_date='2026-01-31', use_cache=False)
print(f"✓ 数据获取完成: {len(df_2026)} 行")
# ========== 步骤6: 预处理 ==========
print("\n[6/8] 预处理数据...")
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
# CatBoost特征工程
df_catboost = add_catboost_features(df_2026)
df_catboost = df_catboost.dropna().reset_index(drop=True)
X_catboost = df_catboost[catboost_feature_cols]
# 确保样本数量一致
min_samples = min(len(X_dlinear), len(X_nbeats), len(X_catboost))
X_dlinear = X_dlinear[:min_samples]
y_dlinear = y_dlinear[:min_samples]
X_nbeats = X_nbeats[:min_samples]
X_catboost = X_catboost.iloc[:min_samples]
print(f"✓ 对齐样本数: {min_samples}")
# ========== 步骤7: 五个模型分别预测 ==========
print("\n[7/8] 五个模型分别预测...")
# 7.1 DLinear预测
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
with torch.no_grad():
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits))
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten()
print(f"✓ DLinear: 涨{(dlinear_preds==1).sum()} ({(dlinear_preds==1).sum()/len(dlinear_preds)*100:.1f}%)")
# 7.2 N-BEATS预测
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
with torch.no_grad():
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
print(f"✓ N-BEATS: 涨{(nbeats_preds==1).sum()} ({(nbeats_preds==1).sum()/len(nbeats_preds)*100:.1f}%)")
# 7.3 TTM-R2预测
context_length = 512
ttm_preds_list = []
all_prices = df_2026['close'].values
all_open_prices = df_2026['open'].values
for i in range(min_samples):
start_idx = i
end_idx = start_idx + context_length
if end_idx > len(all_prices):
context_data = all_prices[-context_length:].reshape(1, -1, 1)
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
context_tensor = torch.FloatTensor(context_data).to(device)
with torch.no_grad():
output = ttm_model(context_tensor)
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
ttm_preds = np.array(ttm_preds_list)
print(f"✓ TTM-R2: 涨{(ttm_preds==1).sum()} ({(ttm_preds==1).sum()/len(ttm_preds)*100:.1f}%)")
# 7.4 TimesFM 2.5预测
timesfm_preds_list = []
for i in range(min_samples):
start_idx = i
end_idx = start_idx + 512
if end_idx > len(all_prices):
context_prices = all_prices[-512:]
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_prices = all_prices[start_idx:end_idx]
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
try:
point_forecast, _ = timesfm_model.forecast(
horizon=1,
inputs=[context_prices],
)
pred_price = point_forecast[0, 0]
timesfm_preds_list.append(1 if pred_price > current_open_price else 0)
except:
timesfm_preds_list.append(np.random.randint(0, 2))
timesfm_preds = np.array(timesfm_preds_list)
print(f"✓ TimesFM: 涨{(timesfm_preds==1).sum()} ({(timesfm_preds==1).sum()/len(timesfm_preds)*100:.1f}%)")
# 7.5 CatBoost预测
catboost_preds = catboost_model.predict(X_catboost).astype(int)
print(f"✓ CatBoost: 涨{(catboost_preds==1).sum()} ({(catboost_preds==1).sum()/len(catboost_preds)*100:.1f}%)")
# ========== 步骤8: 集成策略评估 ==========
print("\n[8/8] 集成策略评估...")
true_labels = y_dlinear
# 策略1: 双模型交集(DLinear + N-BEATS)
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
two_model_preds = np.full(len(dlinear_preds), -1)
two_model_preds[two_model_agree_up] = 1
two_model_preds[two_model_agree_down] = 0
two_model_valid = two_model_preds != -1
two_model_correct = (two_model_preds == true_labels) & two_model_valid
two_model_acc = two_model_correct.sum() / two_model_valid.sum() * 100 if two_model_valid.sum() > 0 else 0
two_model_coverage = two_model_valid.sum() / len(true_labels) * 100
# 策略2: 三模型投票(DLinear + N-BEATS + TTM-R2)
vote_sum_3 = dlinear_preds + nbeats_preds + ttm_preds
three_model_preds = np.where(vote_sum_3 >= 2, 1, 0)
three_model_acc = (three_model_preds == true_labels).mean() * 100
# 策略3: 四模型投票
vote_sum_4 = dlinear_preds + nbeats_preds + ttm_preds + timesfm_preds
four_model_preds = np.where(vote_sum_4 >= 3, 1, 0)
four_model_acc = (four_model_preds == true_labels).mean() * 100
# 策略4: 五模型强投票(≥3票)
vote_sum_5 = dlinear_preds + nbeats_preds + ttm_preds + timesfm_preds + catboost_preds
five_model_preds_strong = np.where(vote_sum_5 >= 3, 1, 0)
five_model_acc_strong = (five_model_preds_strong == true_labels).mean() * 100
# 策略5: 五模型弱投票(≥2票)
five_model_preds_weak = np.where(vote_sum_5 >= 2, 1, 0)
five_model_acc_weak = (five_model_preds_weak == true_labels).mean() * 100
# ========== 结果展示 ==========
print("\n" + "="*80)
print("📊 五模型集成策略对比结果")
print("="*80)
print(f"\n单模型性能:")
print(f" DLinear: {(dlinear_preds == true_labels).mean() * 100:.2f}%")
print(f" CatBoost: {(catboost_preds == true_labels).mean() * 100:.2f}%")
print(f" TimesFM 2.5: {(timesfm_preds == true_labels).mean() * 100:.2f}%")
print(f" TTM-R2: {(ttm_preds == true_labels).mean() * 100:.2f}%")
print(f" N-BEATS: {(nbeats_preds == true_labels).mean() * 100:.2f}%")
print(f"\n集成策略性能:")
print(f" 双模型交集 (DL+NB): {two_model_acc:.2f}% (覆盖率: {two_model_coverage:.2f}%)")
print(f" 三模型投票 (DL+NB+TTM, ≥2票): {three_model_acc:.2f}% (覆盖率: 100%)")
print(f" 四模型投票 (前4, ≥3票): {four_model_acc:.2f}% (覆盖率: 100%)")
print(f" 五模型强投票 (所有, ≥3票): {five_model_acc_strong:.2f}% (覆盖率: 100%)")
print(f" 五模型弱投票 (所有, ≥2票): {five_model_acc_weak:.2f}% (覆盖率: 100%)")
# 投票分布分析
print(f"\n五模型投票分布:")
for votes in range(6):
count = (vote_sum_5 == votes).sum()
pct = count / len(vote_sum_5) * 100
bars = "█" * int(pct / 2)
print(f" {votes}票同意: {count:3d}次 ({pct:5.1f}%) {bars}")
# 模型一致性分析
print(f"\n模型一致性分析:")
all_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1) & (ttm_preds == 1) & (timesfm_preds == 1) & (catboost_preds == 1)
all_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0) & (ttm_preds == 0) & (timesfm_preds == 0) & (catboost_preds == 0)
all_agree = all_agree_up | all_agree_down
print(f" 五模型完全一致: {all_agree.sum()}次 ({all_agree.sum()/len(all_agree)*100:.1f}%)")
if all_agree.sum() > 0:
all_agree_acc = ((five_model_preds_strong[all_agree] == true_labels[all_agree]).mean() * 100)
print(f" 一致时准确率: {all_agree_acc:.2f}%")
print("\n" + "="*80)
print("✓ 五模型集成预测完成!")
print("="*80)
if __name__ == "__main__":
main()
-385
View File
@@ -1,385 +0,0 @@
"""四模型集成预测:DLinear + N-BEATS + TTM-R2 + TimesFM 2.5"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import pickle
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df, fit=True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df = df[:-self.forecast_horizon].reset_index(drop=True)
prices = df['close'].values.reshape(-1, 1)
future_prices = df['future_price'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
X, y = [], []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
y.append(scaled_future[i + self.lookback, 0])
return np.array(X), np.array(y), df
def inverse_transform_price(self, scaled_price):
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
def main():
print("="*80)
print("四模型集成预测:DLinear + N-BEATS + TTM-R2 + TimesFM 2.5")
print("="*80)
# ========== 步骤1: 加载DLinear和N-BEATS模型 ==========
print("\n[1/7] 加载DLinear和N-BEATS模型...")
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
# 加载DLinear
from models.dlinear.model import DLinear
dlinear_checkpoint = torch.load(
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device
)
dlinear_model = DLinear(dlinear_checkpoint['config'])
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
dlinear_model = dlinear_model.to(device)
dlinear_model.eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
print(f"✓ DLinear已加载")
# 加载N-BEATS
from models.nbeats.model import NBeats
nbeats_checkpoint = torch.load(
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
map_location=device
)
nbeats_model = NBeats(nbeats_checkpoint['config'])
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
nbeats_model = nbeats_model.to(device)
nbeats_model.eval()
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
print(f"✓ N-BEATS已加载")
# ========== 步骤2: 加载TTM-R2 ==========
print("\n[2/7] 加载TTM-R2零样本模型...")
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
"ibm-granite/granite-timeseries-ttm-r2",
trust_remote_code=True
)
ttm_model = ttm_model.to(device)
ttm_model.eval()
print(f"✓ TTM-R2已加载 (参数量: {sum(p.numel() for p in ttm_model.parameters())})")
# ========== 步骤3: 加载TimesFM 2.5 ==========
print("\n[3/7] 加载TimesFM 2.5模型...")
import timesfm
timesfm_model = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
"google/timesfm-2.5-200m-pytorch",
torch_compile=False # 推理时不使用torch.compile
)
# 配置TimesFM用于BTC 1小时预测
timesfm_model.compile(
timesfm.ForecastConfig(
max_context=512, # 使用512小时历史数据
max_horizon=1, # 预测未来1小时
normalize_inputs=True,
use_continuous_quantile_head=True,
force_flip_invariance=True,
infer_is_positive=True,
fix_quantile_crossing=True,
)
)
print(f"✓ TimesFM 2.5已加载并配置")
print(f" 上下文长度: 512小时")
print(f" 预测长度: 1小时")
# ========== 步骤4: 获取测试数据 ==========
print("\n[4/7] 获取2026年1月测试数据...")
fetcher = BinanceDataFetcher()
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
end_date='2026-01-30', use_cache=False)
print(f"✓ 数据获取完成: {len(df_2026)} 行")
print(f"✓ 时间范围: {df_2026['timestamp'].min()} 至 {df_2026['timestamp'].max()}")
# ========== 步骤5: 预处理 ==========
print("\n[5/7] 预处理数据...")
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
# 确保样本数量一致
min_samples = min(len(X_dlinear), len(X_nbeats))
X_dlinear = X_dlinear[:min_samples]
y_dlinear = y_dlinear[:min_samples]
X_nbeats = X_nbeats[:min_samples]
y_nbeats = y_nbeats[:min_samples]
print(f"✓ 对齐样本数: {min_samples}")
# ========== 步骤6: 四个模型分别预测 ==========
print("\n[6/7] 四个模型分别预测...")
# 6.1 DLinear预测
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
with torch.no_grad():
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits)) # sigmoid
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten() # 1=涨, 0=跌
print(f"✓ DLinear预测完成")
print(f" 预测上涨: {(dlinear_preds==1).sum()} ({(dlinear_preds==1).sum()/len(dlinear_preds)*100:.1f}%)")
print(f" 预测下跌: {(dlinear_preds==0).sum()} ({(dlinear_preds==0).sum()/len(dlinear_preds)*100:.1f}%)")
# 6.2 N-BEATS预测
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
with torch.no_grad():
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
# 还原真实价格并使用当前K线open价格判断
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int) # 1=涨, 0=跌
print(f"✓ N-BEATS预测完成 (使用当前K线open价格判断)")
print(f" 预测上涨: {(nbeats_preds==1).sum()} ({(nbeats_preds==1).sum()/len(nbeats_preds)*100:.1f}%)")
print(f" 预测下跌: {(nbeats_preds==0).sum()} ({(nbeats_preds==0).sum()/len(nbeats_preds)*100:.1f}%)")
# 6.3 TTM-R2预测
print(f"✓ TTM-R2预测中...")
context_length = 512
ttm_preds_list = []
all_prices = df_2026['close'].values
all_open_prices = df_2026['open'].values
for i in range(min_samples):
start_idx = i
end_idx = start_idx + context_length
if end_idx > len(all_prices):
context_data = all_prices[-context_length:].reshape(1, -1, 1)
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
context_tensor = torch.FloatTensor(context_data).to(device)
with torch.no_grad():
output = ttm_model(context_tensor)
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
ttm_preds = np.array(ttm_preds_list)
print(f"✓ TTM-R2预测完成")
print(f" 预测上涨: {(ttm_preds==1).sum()} ({(ttm_preds==1).sum()/len(ttm_preds)*100:.1f}%)")
print(f" 预测下跌: {(ttm_preds==0).sum()} ({(ttm_preds==0).sum()/len(ttm_preds)*100:.1f}%)")
# 6.4 TimesFM 2.5预测
print(f"✓ TimesFM 2.5预测中...")
timesfm_preds_list = []
for i in range(min_samples):
# TimesFM使用512小时上下文
start_idx = i
end_idx = start_idx + 512
if end_idx > len(all_prices):
# 如果数据不足,使用最后512个数据点
context_prices = all_prices[-512:]
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_prices = all_prices[start_idx:end_idx]
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
# TimesFM的forecast方法接受list of arrays作为inputs
try:
point_forecast, _ = timesfm_model.forecast(
horizon=1,
inputs=[context_prices], # list包含单个序列
)
# point_forecast形状: (1, 1) - (num_series, horizon)
pred_price = point_forecast[0, 0]
# 与当前K线open价格比较
timesfm_preds_list.append(1 if pred_price > current_open_price else 0)
except Exception as e:
print(f" ⚠️ TimesFM预测第{i}个样本失败: {e}")
# 使用随机预测作为fallback
timesfm_preds_list.append(np.random.randint(0, 2))
timesfm_preds = np.array(timesfm_preds_list)
print(f"✓ TimesFM 2.5预测完成")
print(f" 预测上涨: {(timesfm_preds==1).sum()} ({(timesfm_preds==1).sum()/len(timesfm_preds)*100:.1f}%)")
print(f" 预测下跌: {(timesfm_preds==0).sum()} ({(timesfm_preds==0).sum()/len(timesfm_preds)*100:.1f}%)")
# ========== 步骤7: 集成策略评估 ==========
print("\n[7/7] 集成策略评估...")
true_labels = y_dlinear
# 策略1: 双模型交集(DLinear + N-BEATS)
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
two_model_preds = np.full(len(dlinear_preds), -1)
two_model_preds[two_model_agree_up] = 1
two_model_preds[two_model_agree_down] = 0
two_model_valid = two_model_preds != -1
two_model_correct = (two_model_preds == true_labels) & two_model_valid
two_model_acc = two_model_correct.sum() / two_model_valid.sum() * 100 if two_model_valid.sum() > 0 else 0
two_model_coverage = two_model_valid.sum() / len(true_labels) * 100
# 策略2: 三模型多数投票(DLinear + N-BEATS + TTM-R2)
vote_sum_3 = dlinear_preds + nbeats_preds + ttm_preds
three_model_preds = np.where(vote_sum_3 >= 2, 1, 0)
three_model_acc = (three_model_preds == true_labels).mean() * 100
# 策略3: 四模型多数投票(至少3个模型同意)
vote_sum_4 = dlinear_preds + nbeats_preds + ttm_preds + timesfm_preds
four_model_preds = np.where(vote_sum_4 >= 3, 1, 0)
four_model_acc = (four_model_preds == true_labels).mean() * 100
# 策略4: 四模型半数以上投票(至少2个模型同意)
four_model_preds_half = np.where(vote_sum_4 >= 2, 1, 0)
four_model_acc_half = (four_model_preds_half == true_labels).mean() * 100
# ========== 结果展示 ==========
print("\n" + "="*80)
print("📊 集成策略对比结果")
print("="*80)
print(f"\n单模型性能:")
print(f" DLinear: {(dlinear_preds == true_labels).mean() * 100:.2f}%")
print(f" N-BEATS: {(nbeats_preds == true_labels).mean() * 100:.2f}%")
print(f" TTM-R2: {(ttm_preds == true_labels).mean() * 100:.2f}%")
print(f" TimesFM 2.5: {(timesfm_preds == true_labels).mean() * 100:.2f}%")
print(f"\n集成策略性能:")
print(f" 双模型交集 (DL+NB): {two_model_acc:.2f}% (覆盖率: {two_model_coverage:.2f}%)")
print(f" 三模型投票 (DL+NB+TTM, ≥2票): {three_model_acc:.2f}% (覆盖率: 100%)")
print(f" 四模型强投票 (所有, ≥3票): {four_model_acc:.2f}% (覆盖率: 100%)")
print(f" 四模型弱投票 (所有, ≥2票): {four_model_acc_half:.2f}% (覆盖率: 100%)")
# 投票分布分析
print(f"\n四模型投票分布:")
for votes in range(5):
count = (vote_sum_4 == votes).sum()
pct = count / len(vote_sum_4) * 100
bars = "█" * int(pct / 2)
print(f" {votes}票同意: {count:3d}次 ({pct:5.1f}%) {bars}")
# 模型一致性分析
print(f"\n模型一致性分析:")
all_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1) & (ttm_preds == 1) & (timesfm_preds == 1)
all_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0) & (ttm_preds == 0) & (timesfm_preds == 0)
all_agree = all_agree_up | all_agree_down
print(f" 四模型完全一致: {all_agree.sum()}次 ({all_agree.sum()/len(all_agree)*100:.1f}%)")
if all_agree.sum() > 0:
all_agree_acc = ((four_model_preds[all_agree] == true_labels[all_agree]).mean() * 100)
print(f" 一致时准确率: {all_agree_acc:.2f}%")
print("\n" + "="*80)
print("✓ 四模型集成预测完成!")
print("="*80)
if __name__ == "__main__":
main()
+247
View File
@@ -0,0 +1,247 @@
"""2026年2月BTC数据 - DLinear, N-BEATS, TTM-R2 准确率测试 (使用统一推理管理器)
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
价格对比:预测结果与预测K线(t+1)的收盘价做对比
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
"""
import sys
import os
import pandas as pd
import numpy as np
from datetime import timedelta
from typing import Dict, List, Any
# 添加项目路径
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from data.fetcher import BinanceDataFetcher
from models.inference import get_inference_manager
# ==================== 时区与美股判断 ====================
def utc_to_beijing(ts):
"""UTC时间转北京时间(UTC+8)"""
return ts + timedelta(hours=8)
def is_us_stock_open_by_utc(utc_ts):
"""根据UTC时间判断美股当天是否开盘"""
et_ts = utc_ts - timedelta(hours=5)
et_date = pd.Timestamp(et_ts.date())
us_holidays_feb_2026 = {
pd.Timestamp('2026-02-16'), # 总统日 Presidents' Day
}
if et_date.weekday() >= 5:
return False
if et_date in us_holidays_feb_2026:
return False
return True
def fmt_pred(val):
"""格式化预测结果"""
if val is None or (isinstance(val, float) and np.isnan(val)):
return "N/A"
return "涨" if int(val) == 1 else "跌"
def fmt_result(pred, actual):
"""格式化对错"""
if pred is None or (isinstance(pred, float) and np.isnan(pred)):
return " "
return "✓" if int(pred) == int(actual) else "✗"
def main():
print("=" * 110)
print("2026年2月 BTC 1h 涨跌预测准确率测试 (DLinear, N-BEATS, TTM-R2)")
print("基于统一推理管理器 (models.inference)")
print("=" * 110)
# ========== 步骤1: 加载模型 ==========
print("\n[1/4] 加载模型...")
manager = get_inference_manager()
manager.load_all_models()
# ========== 步骤2: 获取测试数据 ==========
print("\n[2/4] 获取2026年2月数据...")
fetcher = BinanceDataFetcher()
# 获取足够长的数据以供lookback
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-12-01', end_date='2026-03-01', use_cache=False)
print(f"数据范围(UTC): {df['timestamp'].min()} -> {df['timestamp'].max()}")
print(f"总行数: {len(df)}")
# ========== 步骤3: 运行预测 ==========
print("\n[3/4] 运行预测 (逐小时回测)...")
# 找出2月(北京时间)的所有预测目标K线索引
jan_targets = []
for target_idx in range(200, len(df)): # 从200开始确保有足够lookback
target_time_utc = df.iloc[target_idx]['timestamp']
target_time_bj = utc_to_beijing(target_time_utc)
if target_time_bj.year == 2026 and target_time_bj.month == 2:
jan_targets.append(target_idx)
print(f"2月(北京时间)目标K线数: {len(jan_targets)}")
results = []
for i, target_idx in enumerate(jan_targets):
if i % 50 == 0:
print(f" 处理进度: {i}/{len(jan_targets)}...")
target_time_utc = df.iloc[target_idx]['timestamp']
target_time_bj = utc_to_beijing(target_time_utc)
# 输入数据:截止到 target_idx - 1
# 注意:predict_all 需要传入DataFrame,它会取最后一行作为 "当前"
# 所以我们需要传入 df.iloc[:target_idx] (不包含 target_idx)
# Python切片是 [start:end),所以 [:target_idx] 包含 0 到 target_idx-1
df_input = df.iloc[:target_idx].copy()
# 实际结果
current_close = df.iloc[target_idx-1]['close']
target_close = df.iloc[target_idx]['close']
actual_label = 1 if target_close > current_close else 0
# 预测
preds = manager.predict_all(df_input)
dl_pred = preds.get('dlinear', {}).get('prediction')
nb_pred = preds.get('nbeats', {}).get('prediction')
ttm_pred = preds.get('ttm', {}).get('prediction')
ens_pred = preds.get('ensemble', {}).get('prediction')
results.append({
'time_utc': target_time_utc,
'time_bj': target_time_bj,
'current_close': current_close,
'target_close': target_close,
'actual': actual_label,
'dlinear': dl_pred,
'nbeats': nb_pred,
'ttm': ttm_pred,
'ensemble': ens_pred,
'us_open': is_us_stock_open_by_utc(target_time_utc)
})
# ========== 步骤4: 输出结果 ==========
if not results:
print("没有找到符合条件的测试样本")
return
df_res = pd.DataFrame(results)
df_res['date_bj'] = df_res['time_bj'].dt.date
total_samples = len(df_res)
print(f"\n[4/4] 准确率统计 (2026年2月,北京时间)")
print(f"总样本数: {total_samples}")
# ========== 每小时详细数据 ==========
print("\n" + "=" * 110)
print("每小时详细数据")
print("=" * 110)
hourly_header = (
f"{'北京时间':<18} {'前收盘':>10} {'现收盘':>10} {'实际':>4} "
f"{'DL':>3} {'':>1} {'NB':>3} {'':>1} {'TTM':>3} {'':>1} {'集成':>4} {'':<1}"
)
for date_bj, day_group in df_res.groupby('date_bj'):
us_open = day_group['us_open'].iloc[0]
us_str = "美股开盘" if us_open else "美股休市"
dl_day_acc = (day_group['dlinear'] == day_group['actual']).mean()
nb_valid = day_group.dropna(subset=['nbeats'])
nb_day_acc = (nb_valid['nbeats'] == nb_valid['actual']).mean() if len(nb_valid) > 0 else None
ttm_valid = day_group.dropna(subset=['ttm'])
ttm_day_acc = (ttm_valid['ttm'] == ttm_valid['actual']).mean() if len(ttm_valid) > 0 else None
ens_valid = day_group.dropna(subset=['ensemble'])
ens_day_acc = (ens_valid['ensemble'] == ens_valid['actual']).mean() if len(ens_valid) > 0 else None
nb_acc_str = f"{nb_day_acc:.1%}" if nb_day_acc is not None else "N/A"
ttm_acc_str = f"{ttm_day_acc:.1%}" if ttm_day_acc is not None else "N/A"
ens_acc_str = f"{ens_day_acc:.1%}" if ens_day_acc is not None else "N/A"
print(f"\n--- {date_bj} ({us_str}) --- DL:{dl_day_acc:.1%} | NB:{nb_acc_str} | TTM:{ttm_acc_str} | 集成:{ens_acc_str}")
print(hourly_header)
print("-" * 110)
for _, row in day_group.iterrows():
time_str = row['time_bj'].strftime('%m-%d %H:%M')
actual_str = fmt_pred(row['actual'])
dl_str = fmt_pred(row['dlinear'])
dl_r = fmt_result(row['dlinear'], row['actual'])
nb_str = fmt_pred(row['nbeats'])
nb_r = fmt_result(row['nbeats'], row['actual'])
ttm_str = fmt_pred(row['ttm'])
ttm_r = fmt_result(row['ttm'], row['actual'])
ens_str = fmt_pred(row['ensemble'])
ens_r = fmt_result(row['ensemble'], row['actual'])
print(
f"{time_str:<18} {row['current_close']:>10.2f} {row['target_close']:>10.2f} {actual_str:>4} "
f"{dl_str:>3} {dl_r:>1} {nb_str:>3} {nb_r:>1} {ttm_str:>3} {ttm_r:>1} {ens_str:>4} {ens_r:<1}"
)
# ========== 每日准确率汇总表 ==========
print("\n\n" + "=" * 110)
print("每日准确率汇总 (北京时间)")
print("=" * 110)
header = f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'集成(投票)':>10}"
print(header)
print("-" * 110)
for date_bj, group in df_res.groupby('date_bj'):
total = len(group)
us_open = group['us_open'].iloc[0]
us_str = "开盘" if us_open else "休市"
dl_acc = (group['dlinear'] == group['actual']).mean()
nb_valid = group.dropna(subset=['nbeats'])
nb_acc = (nb_valid['nbeats'] == nb_valid['actual']).mean() if len(nb_valid) > 0 else None
ttm_valid = group.dropna(subset=['ttm'])
ttm_acc = (ttm_valid['ttm'] == ttm_valid['actual']).mean() if len(ttm_valid) > 0 else None
ens_valid = group.dropna(subset=['ensemble'])
ens_acc = (ens_valid['ensemble'] == ens_valid['actual']).mean() if len(ens_valid) > 0 else None
dl_str = f"{dl_acc:.1%}"
nb_str = f"{nb_acc:.1%}" if nb_acc is not None else "N/A"
ttm_str = f"{ttm_acc:.1%}" if ttm_acc is not None else "N/A"
ens_str = f"{ens_acc:.1%}" if ens_acc is not None else "N/A"
print(f"{str(date_bj):<14} {total:>4} {us_str:>6} {dl_str:>8} {nb_str:>8} {ttm_str:>8} {ens_str:>10}")
print("-" * 110)
# --- 总准确率 ---
dl_total = (df_res['dlinear'] == df_res['actual']).mean()
nb_all = df_res.dropna(subset=['nbeats'])
nb_total = (nb_all['nbeats'] == nb_all['actual']).mean() if len(nb_all) > 0 else None
ttm_all = df_res.dropna(subset=['ttm'])
ttm_total = (ttm_all['ttm'] == ttm_all['actual']).mean() if len(ttm_all) > 0 else None
ens_all = df_res.dropna(subset=['ensemble'])
ens_total = (ens_all['ensemble'] == ens_all['actual']).mean() if len(ens_all) > 0 else None
nb_t_str = f"{nb_total:.2%}" if nb_total is not None else "N/A"
ttm_t_str = f"{ttm_total:.2%}" if ttm_total is not None else "N/A"
ens_t_str = f"{ens_total:.2%}" if ens_total is not None else "N/A"
print(f"{'总计':<14} {total_samples:>4} {'':>6} {dl_total:.2%} {nb_t_str:>8} {ttm_t_str:>8} {ens_t_str:>10}")
print("=" * 110)
if __name__ == "__main__":
main()
+427
View File
@@ -0,0 +1,427 @@
"""2026年2月BTC数据 - DLinear, N-BEATS, TTM-R2 准确率测试
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
价格对比:预测结果与预测K线(t+1)的收盘价做对比
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口
输出:每小时详细数据、每天准确率、总准确率、三模型集成、美股开盘标记
"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import numpy as np
import pandas as pd
import pickle
import pandas_ta as ta
from datetime import timedelta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
# ==================== 时区与美股判断 ====================
def utc_to_beijing(ts):
"""UTC时间转北京时间(UTC+8)"""
return ts + timedelta(hours=8)
def is_us_stock_open_by_utc(utc_ts):
"""根据UTC时间判断美股当天是否开盘
美股交易日基于美东时间(ET),UTC-5(冬令时)
2026年2月在冬令时范围内(夏令时3月第二个周日开始)
所以美东 = UTC - 5h
"""
et_ts = utc_ts - timedelta(hours=5)
et_date = pd.Timestamp(et_ts.date())
us_holidays_feb_2026 = {
pd.Timestamp('2026-02-16'), # 总统日 Presidents' Day
}
if et_date.weekday() >= 5:
return False
if et_date in us_holidays_feb_2026:
return False
return True
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器 - pickle反序列化占位类"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
prices = df['close'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
X = []
valid_indices = []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
valid_indices.append(i + self.lookback - 1)
return np.array(X), df, valid_indices
def fmt_pred(val):
"""格式化预测结果"""
if val is None or (isinstance(val, float) and np.isnan(val)):
return "N/A"
return "涨" if int(val) == 1 else "跌"
def fmt_result(pred, actual):
"""格式化对错"""
if pred is None or (isinstance(pred, float) and np.isnan(pred)):
return " "
return "✓" if int(pred) == int(actual) else "✗"
def main():
print("=" * 110)
print("2026年2月 BTC 1h 涨跌预测准确率测试 (DLinear, N-BEATS, TTM-R2)")
print("输入:预测K线的前一根 | 对比:预测结果 vs 预测K线收盘价 | 时间:北京时间(UTC+8)")
print("DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口")
print("=" * 110)
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
print(f"使用设备: {device}")
# ========== 步骤1: 加载模型 ==========
print("\n[1/4] 加载模型...")
from models.dlinear.model import DLinear
dlinear_checkpoint = torch.load('./models/checkpoints/dlinear_classification_2026/best_model.pt', map_location=device, weights_only=False)
dlinear_model = DLinear(dlinear_checkpoint['config'])
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
dlinear_model = dlinear_model.to(device).eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
print(" ✓ DLinear已加载")
from models.nbeats.model import NBeats
nbeats_checkpoint = torch.load('./models/checkpoints/nbeats_btc_1h/best_model.pt', map_location=device, weights_only=False)
nbeats_model = NBeats(nbeats_checkpoint['config']).to(device).eval()
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
print(" ✓ N-BEATS已加载")
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
"ibm-granite/granite-timeseries-ttm-r2",
trust_remote_code=True
).to(device).eval()
print(" ✓ TTM-R2已加载")
# ========== 步骤2: 获取测试数据 ==========
print("\n[2/4] 获取2026年1月数据...")
fetcher = BinanceDataFetcher()
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-10-01', end_date='2026-02-01', use_cache=False)
print(f"数据范围(UTC): {df['timestamp'].min()} -> {df['timestamp'].max()}")
print(f"总行数: {len(df)}")
# ========== 步骤3: 预处理与预测 ==========
print("\n[3/4] 运行预测...")
# DLinear 整体预处理(无create_labels,整个df一次性处理)
dl_lookback = dlinear_preprocessor.lookback
df_dl = df.copy()
df_dl['rsi'] = ta.rsi(df_dl['close'], length=14)
macd = ta.macd(df_dl['close'])
if macd is not None and not macd.empty:
df_dl['macd'] = macd.iloc[:, 0]
df_dl['macd_hist'] = macd.iloc[:, 1]
df_dl['macd_signal'] = macd.iloc[:, 2]
df_dl = df_dl.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
scaled_dl = dlinear_preprocessor.scaler.transform(df_dl[feature_cols])
X_dl = []
idx_dl = []
for i in range(len(scaled_dl) - dl_lookback):
X_dl.append(scaled_dl[i:i + dl_lookback])
idx_dl.append(i + dl_lookback - 1)
X_dl = np.array(X_dl)
# 构建 timestamp -> 窗口索引映射
dl_ts_map = {}
for i, idx in enumerate(idx_dl):
ts = df_dl.iloc[idx]['timestamp']
dl_ts_map[ts] = i
# N-BEATS 整体预处理(N-BEATS只用close,不受create_labels影响)
X_nb, df_nb, idx_nb = nbeats_preprocessor.preprocess(df, fit=False)
# TTM-R2 整体预处理
ttm_context_len = 512
scaler_ttm = StandardScaler()
scaled_close_ttm = scaler_ttm.fit_transform(df[['close']])
X_ttm = []
idx_ttm = []
for i in range(len(scaled_close_ttm) - ttm_context_len):
X_ttm.append(scaled_close_ttm[i:i+ttm_context_len])
idx_ttm.append(i + ttm_context_len - 1)
X_ttm = np.array(X_ttm)
idx_ttm_map = {v: k for k, v in enumerate(idx_ttm)}
# 找出2月(北京时间)的所有预测目标K线索引
# df 中每一行都是一根已收盘的K线,timestamp 是 UTC
jan_targets = []
for target_idx in range(1, len(df)):
target_time_utc = df.iloc[target_idx]['timestamp']
target_time_bj = utc_to_beijing(target_time_utc)
if target_time_bj.year == 2026 and target_time_bj.month == 1:
jan_targets.append(target_idx)
print(f"1月(北京时间)目标K线数: {len(jan_targets)}")
results = []
for target_idx in jan_targets:
target_time_utc = df.iloc[target_idx]['timestamp']
target_time_bj = utc_to_beijing(target_time_utc)
# 输入K线索引(前一根)
input_idx = target_idx - 1
current_close = df.iloc[input_idx]['close']
target_close = df.iloc[target_idx]['close']
actual_label = 1 if target_close > current_close else 0
# --- DLinear 预测(整个df一次性预处理,按索引查找窗口) ---
dl_pred = None
dl_prob = None
input_ts = df.iloc[input_idx]['timestamp']
dl_i = dl_ts_map.get(input_ts)
if dl_i is not None and dl_i < len(X_dl):
x_dl_tensor = torch.FloatTensor(X_dl[dl_i:dl_i+1]).to(device)
with torch.no_grad():
logits = dlinear_model(x_dl_tensor).cpu().numpy()
prob = 1 / (1 + np.exp(-logits))
dl_pred = int(prob.item() > 0.5)
dl_prob = float(prob.item())
# --- N-BEATS 预测 ---
nb_pred = None
if input_idx in idx_nb:
nb_i = idx_nb.index(input_idx)
x_nb_tensor = torch.FloatTensor(X_nb[nb_i:nb_i+1]).to(device)
with torch.no_grad():
nb_out = nbeats_model(x_nb_tensor).cpu().numpy()
if nb_out.ndim == 1:
pred_price_scaled = nb_out[0]
else:
pred_price_scaled = nb_out[0, 0]
current_price_scaled = X_nb[nb_i, -1]
nb_pred = 1 if pred_price_scaled > current_price_scaled else 0
# --- TTM-R2 预测 ---
ttm_pred = None
ttm_i = idx_ttm_map.get(input_idx)
if ttm_i is not None:
x_ttm_tensor = torch.FloatTensor(X_ttm[ttm_i:ttm_i+1]).to(device)
with torch.no_grad():
ttm_out = ttm_model(x_ttm_tensor)
ttm_pred_scaled = ttm_out.prediction_outputs[0, 0, 0].cpu().numpy()
ttm_current_scaled = X_ttm[ttm_i, -1, 0]
ttm_pred = 1 if ttm_pred_scaled > ttm_current_scaled else 0
# 集成预测(多数投票)
votes = [v for v in [dl_pred, nb_pred, ttm_pred] if v is not None]
ens_pred = (1 if sum(votes) >= 2 else 0) if len(votes) == 3 else None
results.append({
'time_utc': target_time_utc,
'time_bj': target_time_bj,
'current_close': current_close,
'target_close': target_close,
'actual': actual_label,
'dlinear': dl_pred,
'nbeats': nb_pred,
'ttm': ttm_pred,
'ensemble': ens_pred,
})
# ========== 步骤4: 输出结果 ==========
if not results:
print("没有找到符合条件的测试样本")
return
df_res = pd.DataFrame(results)
df_res['date_bj'] = df_res['time_bj'].dt.date
df_res['us_open'] = df_res['time_utc'].apply(is_us_stock_open_by_utc)
total_samples = len(df_res)
print(f"\n[4/4] 准确率统计 (2026年2月,北京时间)")
print(f"总样本数: {total_samples}")
# ========== 每小时详细数据 ==========
print("\n" + "=" * 110)
print("每小时详细数据")
print("=" * 110)
hourly_header = (
f"{'北京时间':<18} {'前收盘':>10} {'现收盘':>10} {'实际':>4} "
f"{'DL':>3} {'':>1} {'NB':>3} {'':>1} {'TTM':>3} {'':>1} {'集成':>4} {'':<1}"
)
for date_bj, day_group in df_res.groupby('date_bj'):
us_open = day_group['us_open'].iloc[0]
us_str = "美股开盘" if us_open else "美股休市"
dl_day_acc = (day_group['dlinear'] == day_group['actual']).mean()
nb_day_valid = day_group.dropna(subset=['nbeats'])
nb_day_acc = (nb_day_valid['nbeats'] == nb_day_valid['actual']).mean() if len(nb_day_valid) > 0 else None
ttm_day_valid = day_group.dropna(subset=['ttm'])
ttm_day_acc = (ttm_day_valid['ttm'] == ttm_day_valid['actual']).mean() if len(ttm_day_valid) > 0 else None
ens_day_valid = day_group.dropna(subset=['ensemble'])
ens_day_acc = (ens_day_valid['ensemble'] == ens_day_valid['actual']).mean() if len(ens_day_valid) > 0 else None
nb_acc_str = f"{nb_day_acc:.1%}" if nb_day_acc is not None else "N/A"
ttm_acc_str = f"{ttm_day_acc:.1%}" if ttm_day_acc is not None else "N/A"
ens_acc_str = f"{ens_day_acc:.1%}" if ens_day_acc is not None else "N/A"
print(f"\n--- {date_bj} ({us_str}) --- DL:{dl_day_acc:.1%} | NB:{nb_acc_str} | TTM:{ttm_acc_str} | 集成:{ens_acc_str}")
print(hourly_header)
print("-" * 110)
for _, row in day_group.iterrows():
time_str = row['time_bj'].strftime('%m-%d %H:%M')
actual_str = fmt_pred(row['actual'])
dl_str = fmt_pred(row['dlinear'])
dl_r = fmt_result(row['dlinear'], row['actual'])
nb_str = fmt_pred(row['nbeats'])
nb_r = fmt_result(row['nbeats'], row['actual'])
ttm_str = fmt_pred(row['ttm'])
ttm_r = fmt_result(row['ttm'], row['actual'])
ens_str = fmt_pred(row['ensemble'])
ens_r = fmt_result(row['ensemble'], row['actual'])
print(
f"{time_str:<18} {row['current_close']:>10.2f} {row['target_close']:>10.2f} {actual_str:>4} "
f"{dl_str:>3} {dl_r:>1} {nb_str:>3} {nb_r:>1} {ttm_str:>3} {ttm_r:>1} {ens_str:>4} {ens_r:<1}"
)
# ========== 每日准确率汇总表 ==========
print("\n\n" + "=" * 110)
print("每日准确率汇总 (北京时间)")
print("=" * 110)
header = f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'集成(投票)':>10}"
print(header)
print("-" * 110)
for date_bj, group in df_res.groupby('date_bj'):
total = len(group)
us_open = group['us_open'].iloc[0]
us_str = "开盘" if us_open else "休市"
dl_acc = (group['dlinear'] == group['actual']).mean()
nb_valid = group.dropna(subset=['nbeats'])
nb_acc = (nb_valid['nbeats'] == nb_valid['actual']).mean() if len(nb_valid) > 0 else None
ttm_valid = group.dropna(subset=['ttm'])
ttm_acc = (ttm_valid['ttm'] == ttm_valid['actual']).mean() if len(ttm_valid) > 0 else None
ens_valid = group.dropna(subset=['ensemble'])
ens_acc = (ens_valid['ensemble'] == ens_valid['actual']).mean() if len(ens_valid) > 0 else None
dl_str = f"{dl_acc:.1%}"
nb_str = f"{nb_acc:.1%}" if nb_acc is not None else "N/A"
ttm_str = f"{ttm_acc:.1%}" if ttm_acc is not None else "N/A"
ens_str = f"{ens_acc:.1%}" if ens_acc is not None else "N/A"
print(f"{str(date_bj):<14} {total:>4} {us_str:>6} {dl_str:>8} {nb_str:>8} {ttm_str:>8} {ens_str:>10}")
print("-" * 110)
# --- 总准确率 ---
dl_total = (df_res['dlinear'] == df_res['actual']).mean()
nb_all = df_res.dropna(subset=['nbeats'])
nb_total = (nb_all['nbeats'] == nb_all['actual']).mean() if len(nb_all) > 0 else None
ttm_all = df_res.dropna(subset=['ttm'])
ttm_total = (ttm_all['ttm'] == ttm_all['actual']).mean() if len(ttm_all) > 0 else None
ens_all = df_res.dropna(subset=['ensemble'])
ens_total = (ens_all['ensemble'] == ens_all['actual']).mean() if len(ens_all) > 0 else None
nb_t_str = f"{nb_total:.2%}" if nb_total is not None else "N/A"
ttm_t_str = f"{ttm_total:.2%}" if ttm_total is not None else "N/A"
ens_t_str = f"{ens_total:.2%}" if ens_total is not None else "N/A"
print(f"{'总计':<14} {total_samples:>4} {'':>6} {dl_total:.2%} {nb_t_str:>8} {ttm_t_str:>8} {ens_t_str:>10}")
print("=" * 110)
# --- 美股开盘日 vs 休市日对比 ---
print("\n[美股开盘日 vs 休市日 准确率对比]")
print("-" * 80)
for label, flag in [("开盘日", True), ("休市日", False)]:
subset = df_res[df_res['us_open'] == flag]
if len(subset) == 0:
continue
n = len(subset)
dl_a = (subset['dlinear'] == subset['actual']).mean()
nb_s = subset.dropna(subset=['nbeats'])
nb_a = (nb_s['nbeats'] == nb_s['actual']).mean() if len(nb_s) > 0 else None
ttm_s = subset.dropna(subset=['ttm'])
ttm_a = (ttm_s['ttm'] == ttm_s['actual']).mean() if len(ttm_s) > 0 else None
ens_s = subset.dropna(subset=['ensemble'])
ens_a = (ens_s['ensemble'] == ens_s['actual']).mean() if len(ens_s) > 0 else None
nb_str = f"{nb_a:.2%}" if nb_a is not None else "N/A"
ttm_str = f"{ttm_a:.2%}" if ttm_a is not None else "N/A"
ens_str = f"{ens_a:.2%}" if ens_a is not None else "N/A"
print(f" {label} ({n}样本): DLinear={dl_a:.2%} N-BEATS={nb_str} TTM-R2={ttm_str} 集成={ens_str}")
print("-" * 80)
# --- 预测分布统计 ---
print("\n[预测分布统计]")
print("-" * 80)
actual_up = (df_res['actual'] == 1).sum()
actual_down = (df_res['actual'] == 0).sum()
print(f" 实际: 上涨={actual_up} ({actual_up/total_samples:.1%}) 下跌={actual_down} ({actual_down/total_samples:.1%})")
for name, col in [("DLinear", "dlinear"), ("N-BEATS", "nbeats"), ("TTM-R2", "ttm"), ("集成", "ensemble")]:
valid = df_res[col].dropna()
if len(valid) == 0:
continue
up = (valid == 1).sum()
down = (valid == 0).sum()
print(f" {name}: 预测上涨={up} ({up/len(valid):.1%}) 预测下跌={down} ({down/len(valid):.1%})")
print("=" * 110)
if __name__ == "__main__":
main()
+488
View File
@@ -0,0 +1,488 @@
"""2026年2月BTC数据 - DLinear, N-BEATS, TTM-R2 准确率测试 (ONNX版)
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
价格对比:预测结果与预测K线(t+1)的收盘价做对比
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口
输出:每小时详细数据、每天准确率、总准确率、三模型集成、美股开盘标记
"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import numpy as np
import pandas as pd
import pickle
import pandas_ta as ta
import onnxruntime as ort
from datetime import timedelta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
# ==================== 时区与美股判断 ====================
def utc_to_beijing(ts):
"""UTC时间转北京时间(UTC+8)"""
return ts + timedelta(hours=8)
def is_us_stock_open_by_utc(utc_ts):
"""根据UTC时间判断美股当天是否开盘
美股交易日基于美东时间(ET),UTC-5(冬令时)
2026年2月在冬令时范围内(夏令时3月第二个周日开始)
所以美东 = UTC - 5h
"""
et_ts = utc_ts - timedelta(hours=5)
et_date = pd.Timestamp(et_ts.date())
us_holidays_feb_2026 = {
pd.Timestamp('2026-02-16'), # 总统日 Presidents' Day
}
if et_date.weekday() >= 5:
return False
if et_date in us_holidays_feb_2026:
return False
return True
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器 - pickle反序列化占位类"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
prices = df['close'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
X = []
valid_indices = []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
valid_indices.append(i + self.lookback - 1)
return np.array(X), df, valid_indices
def fmt_pred(val):
"""格式化预测结果"""
if val is None or (isinstance(val, float) and np.isnan(val)):
return "N/A"
return "涨" if int(val) == 1 else "跌"
def fmt_result(pred, actual):
"""格式化对错"""
if pred is None or (isinstance(pred, float) and np.isnan(pred)):
return " "
return "✓" if int(pred) == int(actual) else "✗"
def main():
print("=" * 110)
print("2026年3月 BTC 1h 涨跌预测准确率测试 (DLinear, N-BEATS, TTM-R2) - ONNX Version")
print("输入:预测K线的前一根 | 对比:预测结果 vs 预测K线收盘价 | 时间:北京时间(UTC+8)")
print("DLinear预测逻辑:整个df一次性预处理,按索引遍历窗口")
print("=" * 110)
# ========== 步骤1: 加载模型 ==========
print("\n[1/4] 加载 ONNX 模型...")
# Load DLinear ONNX
try:
dlinear_session = ort.InferenceSession('./models/checkpoints/dlinear_classification_2026/model.onnx')
print(" ✓ DLinear (ONNX) 已加载")
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
except Exception as e:
print(f" ✗ DLinear 加载失败: {e}")
return
# Load N-BEATS ONNX
try:
nbeats_session = ort.InferenceSession('./models/checkpoints/nbeats_btc_1h/model.onnx')
print(" ✓ N-BEATS (ONNX) 已加载")
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
except Exception as e:
print(f" ✗ N-BEATS 加载失败: {e}")
return
# Load TTM-R2 ONNX
ttm_session = None
try:
temp_session = ort.InferenceSession('./models/checkpoints/ttm_r2_onnx/ttm_r2_crypto_channel_mixing.onnx')
# Test inference with dummy input
print(" Testing TTM-R2 ONNX inference...")
dummy_input = np.random.randn(1, 512, 1).astype(np.float32)
ort_inputs = {}
input_names = [i.name for i in temp_session.get_inputs()]
print(f"Model inputs: {input_names}")
# 根据 session 中的名称映射输入
for input_meta in temp_session.get_inputs():
name = input_meta.name
if 'past_values' in name or name == 'input':
ort_inputs[name] = dummy_input
elif 'mask' in name:
# 创建全 1 mask
mask = np.ones_like(dummy_input)
# 检查期望的类型
if input_meta.type == 'tensor(int64)':
mask = mask.astype(np.int64)
else:
mask = mask.astype(np.float32)
ort_inputs[name] = mask
print(f"Session run input keys: {list(ort_inputs.keys())}")
for k, v in ort_inputs.items():
print(f"Input {k} shape: {v.shape}, dtype: {v.dtype}")
temp_session.run(None, ort_inputs)
ttm_session = temp_session
print(" ✓ TTM-R2 (ONNX) 已加载且可用 (Optimizations Disabled)")
except Exception as e:
print(f" ⚠ TTM-R2 (ONNX) 加载或推理失败: {e}")
print(" 将在本次测试中跳过 TTM-R2 模型")
ttm_session = None
# ========== 步骤2: 获取测试数据 ==========
print("\n[2/4] 获取2026年3月数据...")
fetcher = BinanceDataFetcher()
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01', end_date='2026-04-01', use_cache=False)
print(f"数据范围(UTC): {df['timestamp'].min()} -> {df['timestamp'].max()}")
print(f"总行数: {len(df)}")
# ========== 步骤3: 预处理与预测 ==========
print("\n[3/4] 运行预测...")
# DLinear 整体预处理(无create_labels,整个df一次性处理)
dl_lookback = dlinear_preprocessor.lookback
df_dl = df.copy()
df_dl['rsi'] = ta.rsi(df_dl['close'], length=14)
macd = ta.macd(df_dl['close'])
if macd is not None and not macd.empty:
df_dl['macd'] = macd.iloc[:, 0]
df_dl['macd_hist'] = macd.iloc[:, 1]
df_dl['macd_signal'] = macd.iloc[:, 2]
df_dl = df_dl.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
scaled_dl = dlinear_preprocessor.scaler.transform(df_dl[feature_cols])
X_dl = []
idx_dl = []
for i in range(len(scaled_dl) - dl_lookback):
X_dl.append(scaled_dl[i:i + dl_lookback])
idx_dl.append(i + dl_lookback - 1)
X_dl = np.array(X_dl, dtype=np.float32)
# 构建 timestamp -> 窗口索引映射
dl_ts_map = {}
for i, idx in enumerate(idx_dl):
ts = df_dl.iloc[idx]['timestamp']
dl_ts_map[ts] = i
# N-BEATS 整体预处理(N-BEATS只用close,不受create_labels影响)
X_nb, df_nb, idx_nb = nbeats_preprocessor.preprocess(df, fit=False)
X_nb = X_nb.astype(np.float32) # Ensure float32
# TTM-R2 整体预处理
ttm_context_len = 512
scaler_ttm = StandardScaler()
scaled_close_ttm = scaler_ttm.fit_transform(df[['close']])
X_ttm = []
idx_ttm = []
for i in range(len(scaled_close_ttm) - ttm_context_len):
X_ttm.append(scaled_close_ttm[i:i+ttm_context_len])
idx_ttm.append(i + ttm_context_len - 1)
X_ttm = np.array(X_ttm, dtype=np.float32)
idx_ttm_map = {v: k for k, v in enumerate(idx_ttm)}
# 找出3月(北京时间)的所有预测目标K线索引
# df 中每一行都是一根已收盘的K线,timestamp 是 UTC
jan_targets = []
for target_idx in range(1, len(df)):
# 同样在筛选样本时也将时间向后移动一个小时
target_time_utc = df.iloc[target_idx]['timestamp'] + timedelta(hours=1)
target_time_bj = utc_to_beijing(target_time_utc)
if target_time_bj.year == 2026 and target_time_bj.month == 3:
jan_targets.append(target_idx)
print(f"3月(北京时间)目标K线数: {len(jan_targets)}")
results = []
for target_idx in jan_targets:
# 原始timestamp表示k线的开始时间,因此预测的结果实际对应下一小时的k线。
# 所以展示时需要将时间向后移动1小时。
target_time_utc = df.iloc[target_idx]['timestamp'] + timedelta(hours=1)
target_time_bj = utc_to_beijing(target_time_utc)
# 现在我们将预测目标设为下一根k线(即target_idx+1),
# 输入使用它的前一根(等于原先的target_idx)。
predicted_idx = target_idx + 1
if predicted_idx >= len(df):
# 已经到达数据末尾,无法构建完整样本,跳过
continue
input_idx = target_idx
# 开盘/收盘对应预测k线
open_price = df.iloc[predicted_idx]['open']
close_price = df.iloc[predicted_idx]['close']
# 实际涨跌以预测k线的开盘为基准
actual_label = 1 if close_price > open_price else 0
# --- DLinear 预测 (ONNX) ---
dl_pred = None
dl_prob = None
input_ts = df.iloc[input_idx]['timestamp']
dl_i = dl_ts_map.get(input_ts)
if dl_i is not None and dl_i < len(X_dl):
x_dl_input = X_dl[dl_i:dl_i+1] # (1, 168, 9)
# ONNX Inference
# Input name: 'input'
logits = dlinear_session.run(['output'], {'input': x_dl_input})[0]
prob = 1 / (1 + np.exp(-logits))
dl_pred = int(prob.item() > 0.5)
dl_prob = float(prob.item())
# --- N-BEATS 预测 (ONNX) ---
nb_pred = None
if input_idx in idx_nb:
nb_i = idx_nb.index(input_idx)
x_nb_input = X_nb[nb_i:nb_i+1] # (1, 168)
# ONNX Inference
# Input name: 'input'
nb_out = nbeats_session.run(['forecast'], {'input': x_nb_input})[0]
if nb_out.ndim == 1:
pred_price_scaled = nb_out[0]
else:
pred_price_scaled = nb_out[0, 0]
current_price_scaled = X_nb[nb_i, -1]
nb_pred = 1 if pred_price_scaled > current_price_scaled else 0
# --- TTM-R2 预测 (ONNX) ---
ttm_pred = None
ttm_i = idx_ttm_map.get(input_idx)
if ttm_i is not None and ttm_session is not None:
x_ttm_input = X_ttm[ttm_i:ttm_i+1] # (1, 512, 1)
# Debug shape
# print(f"TTM input shape: {x_ttm_input.shape}, dtype: {x_ttm_input.dtype}")
# ONNX Inference
# Input name: 'past_values'
# Try transposing to (1, 1, 512) just in case
# x_ttm_input = x_ttm_input.transpose(0, 2, 1)
ttm_out = ttm_session.run(None, {'past_values': x_ttm_input})[0]
ttm_pred_scaled = ttm_out[0, 0, 0]
ttm_current_scaled = X_ttm[ttm_i, -1, 0]
ttm_pred = 1 if ttm_pred_scaled > ttm_current_scaled else 0
# 集成预测(多数投票)
votes = [v for v in [dl_pred, nb_pred, ttm_pred] if v is not None]
ens_pred = (1 if sum(votes) >= 2 else 0) if len(votes) == 3 else None
results.append({
'time_utc': target_time_utc,
'time_bj': target_time_bj,
'open': open_price,
'close': close_price,
'actual': actual_label,
'dlinear': dl_pred,
'nbeats': nb_pred,
'ttm': ttm_pred,
'ensemble': ens_pred,
})
# ========== 步骤4: 输出结果 ==========
if not results:
print("没有找到符合条件的测试样本")
return
df_res = pd.DataFrame(results)
df_res['date_bj'] = df_res['time_bj'].dt.date
df_res['us_open'] = df_res['time_utc'].apply(is_us_stock_open_by_utc)
total_samples = len(df_res)
print(f"\n[4/4] 准确率统计 (2026年3月,北京时间)")
print(f"总样本数: {total_samples}")
# ========== 每小时详细数据 ==========
print("\n" + "=" * 110)
print("每小时详细数据")
print("=" * 110)
# 改为显示开盘/收盘
hourly_header = (
f"{'北京时间':<18} {'开盘':>10} {'收盘':>10} {'实际':>4} "
f"{'DL':>3} {'':>1} {'NB':>3} {'':>1} {'TTM':>3} {'':>1} {'集成':>4} {'':<1}"
)
for date_bj, day_group in df_res.groupby('date_bj'):
us_open = day_group['us_open'].iloc[0]
us_str = "美股开盘" if us_open else "美股休市"
dl_day_acc = (day_group['dlinear'] == day_group['actual']).mean()
nb_day_valid = day_group.dropna(subset=['nbeats'])
nb_day_acc = (nb_day_valid['nbeats'] == nb_day_valid['actual']).mean() if len(nb_day_valid) > 0 else None
ttm_day_valid = day_group.dropna(subset=['ttm'])
ttm_day_acc = (ttm_day_valid['ttm'] == ttm_day_valid['actual']).mean() if len(ttm_day_valid) > 0 else None
ens_day_valid = day_group.dropna(subset=['ensemble'])
ens_day_acc = (ens_day_valid['ensemble'] == ens_day_valid['actual']).mean() if len(ens_day_valid) > 0 else None
nb_acc_str = f"{nb_day_acc:.1%}" if nb_day_acc is not None else "N/A"
ttm_acc_str = f"{ttm_day_acc:.1%}" if ttm_day_acc is not None else "N/A"
ens_acc_str = f"{ens_day_acc:.1%}" if ens_day_acc is not None else "N/A"
print(f"\n--- {date_bj} ({us_str}) --- DL:{dl_day_acc:.1%} | NB:{nb_acc_str} | TTM:{ttm_acc_str} | 集成:{ens_acc_str}")
print(hourly_header)
print("-" * 110)
for _, row in day_group.iterrows():
time_str = row['time_bj'].strftime('%m-%d %H:%M')
actual_str = fmt_pred(row['actual'])
dl_str = fmt_pred(row['dlinear'])
dl_r = fmt_result(row['dlinear'], row['actual'])
nb_str = fmt_pred(row['nbeats'])
nb_r = fmt_result(row['nbeats'], row['actual'])
ttm_str = fmt_pred(row['ttm'])
ttm_r = fmt_result(row['ttm'], row['actual'])
ens_str = fmt_pred(row['ensemble'])
ens_r = fmt_result(row['ensemble'], row['actual'])
print(
f"{time_str:<18} {row['open']:>10.2f} {row['close']:>10.2f} {actual_str:>4} "
f"{dl_str:>3} {dl_r:>1} {nb_str:>3} {nb_r:>1} {ttm_str:>3} {ttm_r:>1} {ens_str:>4} {ens_r:<1}"
)
# ========== 每日准确率汇总表 ==========
print("\n\n" + "=" * 110)
print("每日准确率汇总 (北京时间)")
print("=" * 110)
header = f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'集成(投票)':>10}"
print(header)
print("-" * 110)
for date_bj, group in df_res.groupby('date_bj'):
total = len(group)
us_open = group['us_open'].iloc[0]
us_str = "开盘" if us_open else "休市"
dl_acc = (group['dlinear'] == group['actual']).mean()
nb_valid = group.dropna(subset=['nbeats'])
nb_acc = (nb_valid['nbeats'] == nb_valid['actual']).mean() if len(nb_valid) > 0 else None
ttm_valid = group.dropna(subset=['ttm'])
ttm_acc = (ttm_valid['ttm'] == ttm_valid['actual']).mean() if len(ttm_valid) > 0 else None
ens_valid = group.dropna(subset=['ensemble'])
ens_acc = (ens_valid['ensemble'] == ens_valid['actual']).mean() if len(ens_valid) > 0 else None
dl_str = f"{dl_acc:.1%}"
nb_str = f"{nb_acc:.1%}" if nb_acc is not None else "N/A"
ttm_str = f"{ttm_acc:.1%}" if ttm_acc is not None else "N/A"
ens_str = f"{ens_acc:.1%}" if ens_acc is not None else "N/A"
print(f"{str(date_bj):<14} {total:>4} {us_str:>6} {dl_str:>8} {nb_str:>8} {ttm_str:>8} {ens_str:>10}")
print("-" * 110)
# --- 总准确率 ---
dl_total = (df_res['dlinear'] == df_res['actual']).mean()
nb_all = df_res.dropna(subset=['nbeats'])
nb_total = (nb_all['nbeats'] == nb_all['actual']).mean() if len(nb_all) > 0 else None
ttm_all = df_res.dropna(subset=['ttm'])
ttm_total = (ttm_all['ttm'] == ttm_all['actual']).mean() if len(ttm_all) > 0 else None
ens_all = df_res.dropna(subset=['ensemble'])
ens_total = (ens_all['ensemble'] == ens_all['actual']).mean() if len(ens_all) > 0 else None
nb_t_str = f"{nb_total:.2%}" if nb_total is not None else "N/A"
ttm_t_str = f"{ttm_total:.2%}" if ttm_total is not None else "N/A"
ens_t_str = f"{ens_total:.2%}" if ens_total is not None else "N/A"
print(f"{'总计':<14} {total_samples:>4} {'':>6} {dl_total:.2%} {nb_t_str:>8} {ttm_t_str:>8} {ens_t_str:>10}")
print("=" * 110)
# --- 美股开盘日 vs 休市日对比 ---
print("\n[美股开盘日 vs 休市日 准确率对比]")
print("-" * 80)
for label, flag in [("开盘日", True), ("休市日", False)]:
subset = df_res[df_res['us_open'] == flag]
if len(subset) == 0:
continue
n = len(subset)
dl_a = (subset['dlinear'] == subset['actual']).mean()
nb_s = subset.dropna(subset=['nbeats'])
nb_a = (nb_s['nbeats'] == nb_s['actual']).mean() if len(nb_s) > 0 else None
ttm_s = subset.dropna(subset=['ttm'])
ttm_a = (ttm_s['ttm'] == ttm_s['actual']).mean() if len(ttm_s) > 0 else None
ens_s = subset.dropna(subset=['ensemble'])
ens_a = (ens_s['ensemble'] == ens_s['actual']).mean() if len(ens_s) > 0 else None
nb_str = f"{nb_a:.2%}" if nb_a is not None else "N/A"
ttm_str = f"{ttm_a:.2%}" if ttm_a is not None else "N/A"
ens_str = f"{ens_a:.2%}" if ens_a is not None else "N/A"
print(f" {label} ({n}样本): DLinear={dl_a:.2%} N-BEATS={nb_str} TTM-R2={ttm_str} 集成={ens_str}")
print("-" * 80)
# --- 预测分布统计 ---
print("\n[预测分布统计]")
print("-" * 80)
actual_up = (df_res['actual'] == 1).sum()
actual_down = (df_res['actual'] == 0).sum()
print(f" 实际: 上涨={actual_up} ({actual_up/total_samples:.1%}) 下跌={actual_down} ({actual_down/total_samples:.1%})")
for name, col in [("DLinear", "dlinear"), ("N-BEATS", "nbeats"), ("TTM-R2", "ttm"), ("集成", "ensemble")]:
valid = df_res[col].dropna()
if len(valid) == 0:
continue
up = (valid == 1).sum()
down = (valid == 0).sum()
print(f" {name}: 预测上涨={up} ({up/len(valid):.1%}) 预测下跌={down} ({down/len(valid):.1%})")
print("=" * 110)
if __name__ == "__main__":
main()
+387
View File
@@ -0,0 +1,387 @@
"""2026年2月BTC数据 - DLinear, N-BEATS, TTM-R2, DeepSeek-R1(高并发版) 准确率测试
输入数据:预测K线的前一根(即用截止到t的数据预测t+1)
价格对比:预测结果与预测K线(t+1)的收盘价做对比
时间:币安数据UTC,输出转为北京时间(UTC+8),按北京时间分天
输出:每小时详细数据、每天准确率、总准确率、四模型集成、美股开盘标记
"""
import sys
import os
import json
import re
import time
import asyncio
import aiohttp
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import numpy as np
import pandas as pd
import pickle
import pandas_ta as ta
from datetime import timedelta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
from utils.network_proxy import apply_global_proxy_from_env
apply_global_proxy_from_env()
# ==================== 时区与美股判断 ====================
def utc_to_beijing(ts):
return ts + timedelta(hours=8)
def is_us_stock_open_by_utc(utc_ts):
et_ts = utc_ts - timedelta(hours=5)
et_date = pd.Timestamp(et_ts.date())
us_holidays_feb_2026 = {pd.Timestamp('2026-02-16')}
if et_date.weekday() >= 5 or et_date in us_holidays_feb_2026:
return False
return True
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
class NBeatsPreprocessor:
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
prices = df['close'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
X, valid_indices = [], []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
valid_indices.append(i + self.lookback - 1)
return np.array(X), df, valid_indices
def fmt_pred(val):
if val is None or (isinstance(val, float) and np.isnan(val)): return "N/A"
return "涨" if int(val) == 1 else "跌"
def fmt_result(pred, actual):
if pred is None or (isinstance(pred, float) and np.isnan(pred)): return " "
return "✓" if int(pred) == int(actual) else "✗"
from datetime import datetime
import json
import re
import time
import asyncio
from openai import AsyncOpenAI
import openai
# ==================== DeepSeek 异步并发核心逻辑 (双重锁绝对防 429) ====================
def get_log_time():
return datetime.now().strftime("%H:%M:%S")
async def fetch_deepseek_single(client, messages, item_id, rate_lock, global_state, sem):
"""
单次异步API调用:并发数控制 + 漏桶控速 + 指数退避重试
"""
attempt = 0
while True:
attempt += 1
# 0. 并发锁:保证同一时刻,最多只有 N 个请求在英伟达服务器上运行
async with sem:
# 1. 速率锁:保证就算允许并发,发包的瞬间也要间隔至少 2.1 秒 (约 28次/分钟,留足余量)
async with rate_lock:
now = time.time()
elapsed = now - global_state['last_call_time']
if elapsed < 2.1:
await asyncio.sleep(2.1 - elapsed)
global_state['last_call_time'] = time.time()
print(f"[{get_log_time()}] 🚀 [任务 {item_id}] 发起请求 (第 {attempt} 次尝试)...")
try:
# 2. 调用 API (注意:请确保 model 是你刚才探测到的最新可用模型名)
completion = await client.chat.completions.create(
model="deepseek-ai/deepseek-v3.2", # <--- 如果旧名字报错410,请务必替换为最新名字
messages=messages,
temperature=0.6,
top_p=0.7,
max_tokens=1024,
timeout=60.0
)
content = completion.choices[0].message.content
match = re.search(r'\{.*\}', content, re.DOTALL)
if match:
res = json.loads(match.group(0))
pred = res.get('prediction')
if pred in [0, 1]:
pred_str = "涨" if pred == 1 else "跌"
print(f"[{get_log_time()}] ✅ [任务 {item_id}] 成功获取预测: {pred_str}")
return item_id, int(pred)
else:
print(f"[{get_log_time()}] ⚠️ [任务 {item_id}] 异常预测值: {pred},重新请求...")
continue
else:
print(f"[{get_log_time()}] ⚠️ [任务 {item_id}] JSON 解析失败,重新请求...")
continue
except openai.RateLimitError:
# 指数退避:第一次等 5 秒,第二次等 10 秒,第三次等 20 秒...
wait_time = 5 * (2 ** (attempt - 1))
wait_time = min(wait_time, 60) # 最多等 60 秒
print(f"[{get_log_time()}] 🛑 [任务 {item_id}] 触发限流 (429),休眠 {wait_time} 秒后重试...")
await asyncio.sleep(wait_time)
except openai.APITimeoutError:
print(f"[{get_log_time()}] ⏳ [任务 {item_id}] 请求超时 (>60s),重新请求...")
await asyncio.sleep(2)
except openai.APIStatusError as e:
if e.status_code in [400, 401, 403, 404, 410]:
print(f"[{get_log_time()}] 🚨 [任务 {item_id}] 致命错误 (HTTP {e.status_code}): {e.message}。放弃。")
return item_id, None
print(f"[{get_log_time()}] ❌ [任务 {item_id}] API 报错 (HTTP {e.status_code}),休眠 3 秒后重试...")
await asyncio.sleep(3)
except Exception as e:
print(f"[{get_log_time()}] 💥 [任务 {item_id}] 未知错误: {e},休眠 3 秒后重新请求...")
await asyncio.sleep(3)
async def run_all_deepseek_async(tasks_data, api_key):
"""统筹所有的异步请求"""
total = len(tasks_data)
# === 核心参数配置 ===
MAX_CONCURRENT = 8 # 绝对并发上限:同一时刻最多 2 个请求在处理
print("\n" + "="*80)
print(f"[{get_log_time()}] 开始 DeepSeek 异步并发请求 (双重防 429 版)")
print(f"总任务数: {total} 个 | 严格并发数: {MAX_CONCURRENT} | 发包间隔: 2.1 秒")
print("="*80 + "\n")
client = AsyncOpenAI(
base_url="https://integrate.api.nvidia.com/v1",
api_key=api_key
)
sem = asyncio.Semaphore(MAX_CONCURRENT)
rate_lock = asyncio.Lock()
global_state = {'last_call_time': 0.0}
ds_results_dict = {}
tasks = []
for item in tasks_data:
task = asyncio.create_task(
fetch_deepseek_single(client, item['messages'], item['id'], rate_lock, global_state, sem)
)
tasks.append(task)
completed = 0
for coro in asyncio.as_completed(tasks):
item_id, pred = await coro
ds_results_dict[item_id] = pred
completed += 1
if completed % 10 == 0 or completed == total:
print(f"\n[{get_log_time()}] 📊 【全局进度】 DeepSeek 已完成: {completed}/{total} ({(completed/total):.1%})\n")
return ds_results_dict
# ==================== 主程序 ====================
def main():
print("=" * 115)
print("BTC 1h 涨跌预测准确率测试 (DLinear, N-BEATS, TTM-R2, DeepSeek-R1 异步版)")
print("=" * 115)
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
print(f"使用设备: {device}")
# ========== 步骤1: 加载模型 ==========
print("\n[1/4] 加载本地模型...")
from models.dlinear.model import DLinear
dlinear_checkpoint = torch.load('./models/checkpoints/dlinear_classification_2026/best_model.pt', map_location=device, weights_only=False)
dlinear_model = DLinear(dlinear_checkpoint['config']).to(device).eval()
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
from models.nbeats.model import NBeats
nbeats_checkpoint = torch.load('./models/checkpoints/nbeats_btc_1h/best_model.pt', map_location=device, weights_only=False)
nbeats_model = NBeats(nbeats_checkpoint['config']).to(device).eval()
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
ttm_model = TinyTimeMixerForPrediction.from_pretrained("ibm-granite/granite-timeseries-ttm-r2", trust_remote_code=True).to(device).eval()
ds_api_key = os.environ.get("NVIDIA_API_KEY", "")
if not ds_api_key:
print(" ⚠️ 未检测到 NVIDIA_API_KEY,跳过 DeepSeek 测试。")
else:
print(" ✓ DeepSeek-R1 (异步并发) 已就绪")
# ========== 步骤2: 获取历史数据 ==========
print("\n[2/4] 获取历史数据并计算指标...")
fetcher = BinanceDataFetcher()
df = fetcher.fetch_ohlcv('BTC', '1h', start_date='2025-10-01', end_date='2026-03-01', use_cache=False)
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd_hist'] = macd.iloc[:, 1]
# ========== 步骤3: 本地预处理与预测 ==========
print("\n[3/4] 运行本地预测 & 构建 API 任务队列...")
# DLinear预处理
df_dl = df.copy()
df_dl['rsi'] = ta.rsi(df_dl['close'], length=14)
macd_dl = ta.macd(df_dl['close'])
if macd_dl is not None and not macd_dl.empty:
df_dl['macd'], df_dl['macd_hist'], df_dl['macd_signal'] = macd_dl.iloc[:, 0], macd_dl.iloc[:, 1], macd_dl.iloc[:, 2]
df_dl = df_dl.dropna().reset_index(drop=True)
scaled_dl = dlinear_preprocessor.scaler.transform(df_dl[['open', 'high', 'low', 'close', 'volume', 'rsi', 'macd', 'macd_hist', 'macd_signal']])
X_dl, dl_ts_map = [], {}
for i in range(len(scaled_dl) - dlinear_preprocessor.lookback):
X_dl.append(scaled_dl[i:i + dlinear_preprocessor.lookback])
dl_ts_map[df_dl.iloc[i + dlinear_preprocessor.lookback - 1]['timestamp']] = i
X_dl = np.array(X_dl)
# N-BEATS预处理
X_nb, df_nb, idx_nb = nbeats_preprocessor.preprocess(df, fit=False)
# TTM预处理
scaler_ttm = StandardScaler()
scaled_close_ttm = scaler_ttm.fit_transform(df[['close']])
X_ttm, idx_ttm_map = [], {}
for i in range(len(scaled_close_ttm) - 512):
X_ttm.append(scaled_close_ttm[i:i+512])
idx_ttm_map[i + 511] = i
X_ttm = np.array(X_ttm)
target_targets = []
for target_idx in range(1, len(df)):
target_time_bj = utc_to_beijing(df.iloc[target_idx]['timestamp'])
if target_time_bj.year == 2026 and target_time_bj.month == 2: # 锁定 2026年 2月 数据
target_targets.append(target_idx)
temp_results = []
deepseek_tasks_data = []
for target_idx in target_targets:
input_idx = target_idx - 1
current_close, target_close = df.iloc[input_idx]['close'], df.iloc[target_idx]['close']
actual_label = 1 if target_close > current_close else 0
# DL
dl_pred, input_ts = None, df.iloc[input_idx]['timestamp']
if (dl_i := dl_ts_map.get(input_ts)) is not None and dl_i < len(X_dl):
with torch.no_grad():
logits = dlinear_model(torch.FloatTensor(X_dl[dl_i:dl_i+1]).to(device)).cpu().numpy()
dl_pred = int((1 / (1 + np.exp(-logits))).item() > 0.5)
# NB
nb_pred = None
if input_idx in idx_nb:
nb_i = idx_nb.index(input_idx)
with torch.no_grad():
nb_out = nbeats_model(torch.FloatTensor(X_nb[nb_i:nb_i+1]).to(device)).cpu().numpy()
nb_pred = 1 if (nb_out[0] if nb_out.ndim == 1 else nb_out[0, 0]) > X_nb[nb_i, -1] else 0
# TTM
ttm_pred = None
if (ttm_i := idx_ttm_map.get(input_idx)) is not None:
with torch.no_grad():
ttm_pred_scaled = ttm_model(torch.FloatTensor(X_ttm[ttm_i:ttm_i+1]).to(device)).prediction_outputs[0, 0, 0].cpu().numpy()
ttm_pred = 1 if ttm_pred_scaled > X_ttm[ttm_i, -1, 0] else 0
# 构造 DeepSeek Payload
if ds_api_key:
recent_k = df.iloc[max(0, input_idx-14) : input_idx+1]
if len(recent_k) >= 5:
kline_text = "".join([f"时间: {row['timestamp']}, 开盘: {row['open']:.2f}, 收盘: {row['close']:.2f}, RSI: {row.get('rsi', 0):.2f}\n" for _, row in recent_k.iterrows()])
sys_prompt = '你是一位顶级量化分析师。请分析BTC 1小时K线,预测下一小时涨跌。必须输出合法的JSON格式:{"prediction": 1(看涨)或0(看跌), "reason": "理由"}'
messages = [
{"role": "system", "content": sys_prompt},
{"role": "user", "content": f"近15周期数据:\n{kline_text}\n输出JSON:"}
]
deepseek_tasks_data.append({'id': target_idx, 'messages': messages})
temp_results.append({
'id': target_idx, 'time_utc': df.iloc[target_idx]['timestamp'], 'time_bj': utc_to_beijing(df.iloc[target_idx]['timestamp']),
'current_close': current_close, 'target_close': target_close, 'actual': actual_label,
'dlinear': dl_pred, 'nbeats': nb_pred, 'ttm': ttm_pred, 'deepseek': None
})
# ========== 异步执行 DeepSeek 并合入数据 ==========
if deepseek_tasks_data:
# 并发数设为 5,对 NVIDIA 免费/基础层级比较安全
ds_predictions = asyncio.run(run_all_deepseek_async(deepseek_tasks_data, ds_api_key))
for res in temp_results:
if res['id'] in ds_predictions:
res['deepseek'] = ds_predictions[res['id']]
# 结算集成模型结果 (四投票)
final_results = []
for res in temp_results:
votes = [v for v in [res['dlinear'], res['nbeats'], res['ttm'], res['deepseek']] if v is not None]
if votes:
vote_sum = sum(votes)
if vote_sum > len(votes) / 2: res['ensemble'] = 1
elif vote_sum < len(votes) / 2: res['ensemble'] = 0
else: res['ensemble'] = res['dlinear'] if res['dlinear'] is not None else 0
else:
res['ensemble'] = None
final_results.append(res)
# ========== 步骤4: 输出结果 ==========
if not final_results:
print("没有找到符合条件的测试样本")
return
df_res = pd.DataFrame(final_results)
df_res['date_bj'] = df_res['time_bj'].dt.date
df_res['us_open'] = df_res['time_utc'].apply(is_us_stock_open_by_utc)
# === (下方打印逻辑与原版一致,缩略优化版) ===
print(f"\n[4/4] 准确率统计 (北京时间)\n总样本数: {len(df_res)}")
print("\n" + "=" * 115 + "\n每小时详细数据\n" + "=" * 115)
hourly_h = f"{'北京时间':<18} {'前收盘':>10} {'现收盘':>10} {'实际':>4} {'DL':>3} {'':>1} {'NB':>3} {'':>1} {'TTM':>3} {'':>1} {'DS':>3} {'':>1} {'集成':>4} {'':<1}"
for date_bj, day_group in df_res.groupby('date_bj'):
us_str = "美股开盘" if day_group['us_open'].iloc[0] else "美股休市"
def s_acc(c): return (day_group.dropna(subset=[c])[c] == day_group.dropna(subset=[c])['actual']).mean() if len(day_group.dropna(subset=[c])) > 0 else None
print(f"\n--- {date_bj} ({us_str}) --- DL:{s_acc('dlinear'):.1%} | NB:{s_acc('nbeats'):.1%} | TTM:{s_acc('ttm'):.1%} | DS:{s_acc('deepseek'):.1%} | 集成:{s_acc('ensemble'):.1%}")
print(hourly_h + "\n" + "-" * 115)
for _, r in day_group.iterrows():
print(f"{r['time_bj'].strftime('%m-%d %H:%M'):<18} {r['current_close']:>10.2f} {r['target_close']:>10.2f} {fmt_pred(r['actual']):>4} "
f"{fmt_pred(r['dlinear']):>3} {fmt_result(r['dlinear'],r['actual']):>1} {fmt_pred(r['nbeats']):>3} {fmt_result(r['nbeats'],r['actual']):>1} "
f"{fmt_pred(r['ttm']):>3} {fmt_result(r['ttm'],r['actual']):>1} {fmt_pred(r['deepseek']):>3} {fmt_result(r['deepseek'],r['actual']):>1} "
f"{fmt_pred(r['ensemble']):>4} {fmt_result(r['ensemble'],r['actual']):<1}")
print("\n\n" + "=" * 115 + "\n每日准确率汇总 (北京时间)\n" + "=" * 115)
print(f"{'日期':<14} {'样本':>4} {'美股':>6} {'DLinear':>8} {'N-BEATS':>8} {'TTM-R2':>8} {'DeepSeek':>8} {'集成':>10}\n" + "-" * 115)
for date_bj, g in df_res.groupby('date_bj'):
def g_acc(c): return (g.dropna(subset=[c])[c] == g.dropna(subset=[c])['actual']).mean() if len(g.dropna(subset=[c])) > 0 else None
print(f"{str(date_bj):<14} {len(g):>4} {'开盘' if g['us_open'].iloc[0] else '休市':>6} {g_acc('dlinear'):>8.1%} {g_acc('nbeats'):>8.1%} {g_acc('ttm'):>8.1%} {g_acc('deepseek'):>8.1%} {g_acc('ensemble'):>10.1%}".replace('nan%', 'N/A'))
def t_acc(c): return (df_res.dropna(subset=[c])[c] == df_res.dropna(subset=[c])['actual']).mean() if len(df_res.dropna(subset=[c])) > 0 else None
print("-" * 115)
print(f"{'总计':<14} {len(df_res):>4} {'':>6} {t_acc('dlinear'):>8.2%} {t_acc('nbeats'):>8.2%} {t_acc('ttm'):>8.2%} {t_acc('deepseek'):>8.2%} {t_acc('ensemble'):>10.2%}".replace('nan%', 'N/A'))
print("=" * 115)
if __name__ == "__main__":
main()
-295
View File
@@ -1,295 +0,0 @@
"""将Google TimesFM 2.5模型导出为ONNX格式 - 使用正确的API"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import numpy as np
def export_timesfm_to_onnx():
print("="*80)
print("导出TimesFM 2.5 (200M) 模型为ONNX格式")
print("="*80)
# ========== 步骤1: 加载TimesFM模型 ==========
print("\n[1/6] 加载TimesFM 2.5模型...")
try:
import timesfm
# 使用正确的from_pretrained方法加载模型
print(" 从HuggingFace加载预训练模型...")
print(" 模型ID: google/timesfm-2.5-200m-pytorch")
model = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
"google/timesfm-2.5-200m-pytorch",
torch_compile=False # 导出ONNX时不使用torch.compile
)
print("✓ TimesFM 2.5模型已加载")
except ImportError as e:
print(f"✗ TimesFM未安装或导入失败: {e}")
print(" 请运行: pip install git+https://github.com/google-research/timesfm.git")
return
except Exception as e:
print(f"✗ 加载模型失败: {e}")
import traceback
traceback.print_exc()
return
# ========== 步骤2: 编译模型配置 ==========
print("\n[2/6] 配置模型...")
try:
# 配置模型
model.compile(
timesfm.ForecastConfig(
max_context=512, # BTC历史数据512小时
max_horizon=24, # 预测未来24小时
normalize_inputs=True,
use_continuous_quantile_head=True,
force_flip_invariance=True,
infer_is_positive=True,
fix_quantile_crossing=True,
)
)
print("✓ 模型配置完成")
print(f" 最大上下文长度: 512")
print(f" 最大预测长度: 24")
print(f" 归一化输入: True")
print(f" 使用分位数预测: True")
except Exception as e:
print(f"✗ 配置失败: {e}")
import traceback
traceback.print_exc()
return
# ========== 步骤3: 测试模型 ==========
print("\n[3/6] 测试模型推理...")
try:
# 创建测试数据
test_input = [
np.linspace(85000, 90000, 512), # 模拟BTC价格从85k到90k
np.sin(np.linspace(0, 20, 512)) * 1000 + 87000, # 模拟震荡价格
]
# 测试推理
point_forecast, quantile_forecast = model.forecast(
horizon=24,
inputs=test_input,
)
print("✓ 模型推理成功")
print(f" 点预测形状: {point_forecast.shape}") # (2, 24)
print(f" 分位数预测形状: {quantile_forecast.shape}") # (2, 24, 11)
except Exception as e:
print(f"⚠️ 模型测试失败: {e}")
import traceback
traceback.print_exc()
print(" 继续尝试导出...")
# ========== 步骤4: 提取核心模型并导出ONNX ==========
print("\n[4/6] 提取核心模型...")
try:
# TimesFM的核心模型在model.model中
core_model = model.model
core_model.eval()
print("✓ 核心模型已提取")
print(f" 模型类型: {type(core_model)}")
# 创建ONNX导出包装器
class TimesFMONNXWrapper(torch.nn.Module):
"""TimesFM ONNX导出包装器"""
def __init__(self, timesfm_core_model):
super().__init__()
self.model = timesfm_core_model
def forward(self, inputs, masks=None):
"""
Args:
inputs: (batch_size, context_length) - 输入时间序列
masks: (batch_size, context_length) - 可选mask,1=有效,0=无效
Returns:
point_forecast: (batch_size, horizon_length) - 点预测
"""
# 如果没有提供masks,创建全1的mask
if masks is None:
masks = torch.ones_like(inputs)
# 调用TimesFM核心模型
# 注意:这里需要根据实际的forward签名调整
outputs, _ = self.model(inputs, masks, decode_caches=None)
# outputs包含: (input_embeddings, output_embeddings, output_ts, output_quantile_spread)
# output_ts是点预测
point_forecast = outputs[2]
return point_forecast
wrapped_model = TimesFMONNXWrapper(core_model)
wrapped_model.eval()
print("✓ ONNX包装器已创建")
except Exception as e:
print(f"✗ 提取核心模型失败: {e}")
import traceback
traceback.print_exc()
return
# ========== 步骤5: 导出为ONNX ==========
print("\n[5/6] 导出模型为ONNX格式...")
onnx_path = "./models/timesfm_2p5_200m.onnx"
os.makedirs("./models", exist_ok=True)
try:
# 准备示例输入
batch_size = 1
context_len = 512
dummy_input = torch.randn(batch_size, context_len, dtype=torch.float32)
dummy_mask = torch.ones(batch_size, context_len, dtype=torch.float32)
print(f" 示例输入形状: inputs={dummy_input.shape}, masks={dummy_mask.shape}")
print(" 开始导出ONNX...")
# 导出为ONNX
torch.onnx.export(
wrapped_model,
(dummy_input, dummy_mask),
onnx_path,
export_params=True,
opset_version=14,
do_constant_folding=True,
input_names=['inputs', 'masks'],
output_names=['point_forecast'],
dynamic_axes={
'inputs': {0: 'batch_size', 1: 'context_length'},
'masks': {0: 'batch_size', 1: 'context_length'},
'point_forecast': {0: 'batch_size', 1: 'horizon_length'}
},
verbose=False
)
print(f"✓ ONNX模型已导出")
print(f" 保存路径: {onnx_path}")
print(f" 文件大小: {os.path.getsize(onnx_path) / (1024**2):.2f} MB")
except Exception as e:
print(f"✗ ONNX导出失败: {e}")
import traceback
traceback.print_exc()
# 尝试简化版本
print("\n尝试简化导出方案...")
try:
# 只导出单输入版本
torch.onnx.export(
wrapped_model,
dummy_input,
onnx_path,
export_params=True,
opset_version=14,
input_names=['inputs'],
output_names=['predictions'],
)
print(f"✓ 简化版ONNX模型已导出: {onnx_path}")
except Exception as e2:
print(f"✗ 简化导出也失败: {e2}")
return
# ========== 步骤6: 验证ONNX模型 ==========
print("\n[6/6] 验证ONNX模型...")
try:
import onnx
import onnxruntime as ort
# 加载并检查ONNX模型
onnx_model = onnx.load(onnx_path)
onnx.checker.check_model(onnx_model)
print("✓ ONNX模型结构验证通过")
# 显示模型信息
print(f"\n模型信息:")
print(f" 输入:")
for input_tensor in onnx_model.graph.input:
dims = [d.dim_value if d.dim_value > 0 else 'dynamic' for d in input_tensor.type.tensor_type.shape.dim]
print(f" - {input_tensor.name}: {dims}")
print(f" 输出:")
for output_tensor in onnx_model.graph.output:
dims = [d.dim_value if d.dim_value > 0 else 'dynamic' for d in output_tensor.type.tensor_type.shape.dim]
print(f" - {output_tensor.name}: {dims}")
# 创建ONNX Runtime推理会话
print("\n 创建ONNX Runtime推理会话...")
ort_session = ort.InferenceSession(onnx_path)
# 准备测试输入
test_input = np.random.randn(1, 512).astype(np.float32)
test_mask = np.ones((1, 512), dtype=np.float32)
# 运行推理
print(" 运行推理测试...")
outputs = ort_session.run(
None,
{'inputs': test_input, 'masks': test_mask}
)
print(f"✓ ONNX推理成功")
print(f" 输出数量: {len(outputs)}")
for i, output in enumerate(outputs):
print(f" 输出{i}形状: {output.shape}")
except ImportError:
print("⚠️ 未安装onnx或onnxruntime,跳过验证")
print(" 安装命令: pip install onnx onnxruntime")
except Exception as e:
print(f"⚠️ ONNX验证失败: {e}")
import traceback
traceback.print_exc()
print(" 但模型文件已保存,可以尝试手动验证")
# ========== 总结 ==========
print("\n" + "="*80)
print("✓ TimesFM 2.5模型导出完成!")
print("="*80)
if os.path.exists(onnx_path):
print(f"\n保存位置: {onnx_path}")
print(f"文件大小: {os.path.getsize(onnx_path) / (1024**2):.2f} MB")
print(f"\n使用方法:")
print(f"```python")
print(f"import onnxruntime as ort")
print(f"import numpy as np")
print(f"")
print(f"# 加载模型")
print(f"session = ort.InferenceSession('{onnx_path}')")
print(f"")
print(f"# 准备输入(512小时BTC历史价格)")
print(f"btc_prices = np.array([85000, 85100, ...]) # 512个价格点")
print(f"inputs = btc_prices.reshape(1, -1).astype(np.float32)")
print(f"masks = np.ones((1, 512), dtype=np.float32)")
print(f"")
print(f"# 推理")
print(f"outputs = session.run(None, {{'inputs': inputs, 'masks': masks}})")
print(f"predictions = outputs[0] # 未来24小时价格预测")
print(f"```")
print("\n" + "="*80)
if __name__ == "__main__":
export_timesfm_to_onnx()
+58
View File
@@ -0,0 +1,58 @@
@echo off
chcp 65001 >nul
REM ============================================
REM Windows 环境安装脚本
REM 安装所有依赖和模型
REM ============================================
echo ==========================================
echo 加密货币预测系统 - Windows 安装脚本
echo ==========================================
echo.
REM 检查 conda
where conda >nul 2>&1
if %errorlevel% neq 0 (
echo [错误] 未找到 conda
echo 请先安装 Miniconda: https://docs.conda.io/en/latest/miniconda.html
pause
exit /b 1
)
echo [1/6] 创建 conda 环境...
call conda create -n crypto_pred_312 python=3.12 -y
if %errorlevel% neq 0 (
echo [警告] 环境可能已存在,继续...
)
echo.
echo [2/6] 激活环境...
call conda activate crypto_pred_312
echo.
echo [3/6] 安装 PyTorch (CPU 版本)...
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
echo.
echo [4/5] 安装项目依赖...
pip install -r requirements.txt
echo.
echo [5/5] 验证安装...
python -c "import torch; print(f'PyTorch: {torch.__version__}')"
python -c "from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction; print('TTM-R2: 已安装')"
python -c "import pandas_ta; print('pandas-ta: 已安装')"
python -c "from training.config import TrainingConfig; c=TrainingConfig(); print(f'设备: {c.get_device()}')"
echo.
echo ==========================================
echo 安装完成!
echo ==========================================
echo.
echo 下一步:
echo 1. 复制 .env.example 为 .env 并配置数据库
echo 2. 双击 start_system.bat 启动系统
echo 3. 访问 http://localhost:8000
echo.
pause
Binary file not shown.
Binary file not shown.
+426
View File
@@ -0,0 +1,426 @@
"""
模型推理服务 - 统一管理 DLinear, N-BEATS, TTM-R2 模型加载与预测
"""
import os
import sys
import pickle
import torch
import numpy as np
import pandas as pd
import pandas_ta as ta
import onnxruntime as ort
from typing import Dict, List, Optional, Tuple, Union, Any
from sklearn.preprocessing import StandardScaler
# 添加项目根目录到 sys.path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
class CustomUnpickler(pickle.Unpickler):
"""自定义反序列化器,用于处理 __main__ 模块的类重定向"""
def find_class(self, module, name):
# 将 __main__ 中的类重定向到当前模块
if module == "__main__":
if name == "ClassificationPreprocessor":
return ClassificationPreprocessor
if name == "NBeatsPreprocessor":
return NBeatsPreprocessor
return super().find_class(module, name)
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback: int = 168, forecast_horizon: int = 1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
self.feature_columns = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
def add_features(self, df: pd.DataFrame) -> pd.DataFrame:
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df: pd.DataFrame) -> pd.DataFrame:
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df: pd.DataFrame, fit: bool = True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
if fit:
scaled_features = self.scaler.fit_transform(df[self.feature_columns])
self.is_fitted = True
else:
scaled_features = self.scaler.transform(df[self.feature_columns])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
def preprocess_for_inference(self, df: pd.DataFrame):
"""推理专用预处理:整个df一次性处理,不生成标签"""
df = self.add_features(df)
df = df.dropna().reset_index(drop=True)
# 确保有足够的数据
if len(df) == 0:
return np.array([]), df, []
scaled_features = self.scaler.transform(df[self.feature_columns])
X = []
valid_indices = []
if len(scaled_features) < self.lookback:
return np.array([]), df, []
# 注意:这里生成的窗口数量应该是 len(scaled) - lookback + 1
# 例如 len=168, lookback=168 -> range(1) -> i=0 -> [0:168] (1个窗口)
for i in range(len(scaled_features) - self.lookback + 1):
window = scaled_features[i:i + self.lookback]
X.append(window)
valid_indices.append(i + self.lookback - 1)
return np.array(X), df, valid_indices
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback: int = 168, forecast_horizon: int = 1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df: pd.DataFrame, fit: bool = True):
df = df.copy()
prices = df['close'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
X = []
valid_indices = []
if len(scaled_prices) < self.lookback:
return np.array([]), df, []
for i in range(len(scaled_prices) - self.lookback + 1):
X.append(scaled_prices[i:i + self.lookback, 0])
valid_indices.append(i + self.lookback - 1)
return np.array(X), df, valid_indices
class InferenceManager:
"""统一模型推理管理器"""
_instance = None
def __new__(cls, *args, **kwargs):
if cls._instance is None:
cls._instance = super(InferenceManager, cls).__new__(cls)
cls._instance.initialized = False
return cls._instance
def __init__(self, device_str: str = None):
if self.initialized:
return
import platform
if device_str:
self.device = torch.device(device_str)
elif platform.system() == 'Darwin' and torch.backends.mps.is_available():
self.device = torch.device('mps')
else:
self.device = torch.device('cpu')
print(f"InferenceManager 使用设备: {self.device}")
self.models = {}
self.preprocessors = {}
# 配置路径
self.configs = {
'dlinear': {
'model_path': './models/checkpoints/dlinear_classification_2026/model.onnx',
'preprocessor_path': './models/checkpoints/dlinear_classification_2026/preprocessor.pkl'
},
'nbeats': {
'model_path': './models/checkpoints/nbeats_btc_1h/model.onnx',
'preprocessor_path': './models/checkpoints/nbeats_btc_1h/preprocessor.pkl'
},
'ttm': {
'model_path': './models/checkpoints/ttm_r2_onnx/ttm_r2_crypto_channel_mixing.onnx'
}
}
self.initialized = True
def load_model(self, model_name: str):
"""加载指定模型"""
if model_name in self.models:
return
print(f"正在加载 {model_name}...")
try:
if model_name == 'dlinear':
self._load_dlinear()
elif model_name == 'nbeats':
self._load_nbeats()
elif model_name == 'ttm':
self._load_ttm()
else:
print(f"未知模型: {model_name}")
except Exception as e:
print(f"加载 {model_name} 失败: {e}")
import traceback
traceback.print_exc()
def load_all_models(self):
"""加载所有模型"""
self.load_model('dlinear')
self.load_model('nbeats')
self.load_model('ttm')
def _load_dlinear(self):
config = self.configs['dlinear']
if not os.path.exists(config['model_path']):
print(f"DLinear 模型文件不存在: {config['model_path']}")
return
try:
self.models['dlinear'] = ort.InferenceSession(config['model_path'])
with open(config['preprocessor_path'], 'rb') as f:
try:
self.preprocessors['dlinear'] = pickle.load(f)
except AttributeError:
f.seek(0)
self.preprocessors['dlinear'] = CustomUnpickler(f).load()
print(" ✓ DLinear (ONNX) 已加载")
except Exception as e:
print(f"DLinear 加载失败: {e}")
def _load_nbeats(self):
config = self.configs['nbeats']
if not os.path.exists(config['model_path']):
print(f"N-BEATS 模型文件不存在: {config['model_path']}")
return
try:
self.models['nbeats'] = ort.InferenceSession(config['model_path'])
with open(config['preprocessor_path'], 'rb') as f:
try:
self.preprocessors['nbeats'] = pickle.load(f)
except AttributeError:
f.seek(0)
self.preprocessors['nbeats'] = CustomUnpickler(f).load()
print(" ✓ N-BEATS (ONNX) 已加载")
except Exception as e:
print(f"N-BEATS 加载失败: {e}")
def _load_ttm(self):
config = self.configs['ttm']
try:
self.models['ttm'] = ort.InferenceSession(config['model_path'])
print(" ✓ TTM-R2 (ONNX) 已加载")
except Exception as e:
print(f"TTM 加载失败: {e}")
def predict_dlinear(self, df: pd.DataFrame) -> Tuple[Optional[int], Optional[float]]:
"""
DLinear预测 (输入df包含历史数据,预测下一时刻)
"""
if 'dlinear' not in self.models:
self.load_model('dlinear')
if 'dlinear' not in self.models:
return None, None
session = self.models['dlinear']
preprocessor = self.preprocessors['dlinear']
# 预处理
X, _, _ = preprocessor.preprocess_for_inference(df)
if len(X) == 0:
return None, None
# 取最后一个窗口
# ONNX 需要 float32
X_last = X[-1:].astype(np.float32)
try:
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name
logits = session.run([output_name], {input_name: X_last})[0]
prob = 1 / (1 + np.exp(-logits))
prediction = int(prob.item() > 0.5)
probability = float(prob.item())
return prediction, probability
except Exception as e:
print(f"DLinear 预测出错: {e}")
return None, None
def predict_nbeats(self, df: pd.DataFrame) -> Tuple[Optional[int], Optional[float]]:
"""
N-BEATS预测
"""
if 'nbeats' not in self.models:
self.load_model('nbeats')
if 'nbeats' not in self.models:
return None, None
session = self.models['nbeats']
preprocessor = self.preprocessors['nbeats']
# 预处理
X, _, _ = preprocessor.preprocess(df, fit=False)
if len(X) == 0:
return None, None
# 取最后一个窗口
X_last = X[-1:].astype(np.float32)
current_price_scaled = X[-1, -1] # 窗口中最后一个点的缩放价格
try:
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name
output = session.run([output_name], {input_name: X_last})[0]
# output shape: (batch, forecast_horizon) or (batch, 1)
if output.ndim == 1:
pred_price_scaled = output[0]
else:
pred_price_scaled = output[0, 0]
prediction = 1 if pred_price_scaled > current_price_scaled else 0
# N-BEATS 回归模型没有直接概率
probability = 0.6 if prediction == 1 else 0.4 # 占位
return prediction, probability
except Exception as e:
print(f"N-BEATS 预测出错: {e}")
return None, None
def predict_ttm(self, df: pd.DataFrame) -> Tuple[Optional[int], Optional[float]]:
"""
TTM-R2预测
"""
if 'ttm' not in self.models:
self.load_model('ttm')
if 'ttm' not in self.models:
return None, None
session = self.models['ttm']
context_length = 512
all_prices = df['close'].values
if len(all_prices) == 0:
return None, None
# TTM 上下文准备
if len(all_prices) >= context_length:
context_data = all_prices[-context_length:].reshape(1, -1, 1)
else:
padded = np.full(context_length, all_prices[-1])
padded[-len(all_prices):] = all_prices
context_data = padded.reshape(1, -1, 1)
# ONNX input
context_data = context_data.astype(np.float32)
base_price = float(df['close'].iloc[-1])
try:
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name
output = session.run([output_name], {input_name: context_data})[0]
# output shape might be (batch, prediction_length, features)
# prediction_length=96 usually for TTM zero-shot
# We want the first prediction step [0, 0, 0]
pred_price = output[0, 0, 0]
prediction = 1 if pred_price > base_price else 0
# 简单的概率模拟
price_pct = (pred_price - base_price) / base_price
probability = 1 / (1 + np.exp(-price_pct * 100))
return prediction, float(probability)
except Exception as e:
print(f"TTM 预测出错: {e}")
return None, None
def predict_all(self, df: pd.DataFrame) -> Dict[str, Any]:
"""
使用所有加载的模型进行预测
"""
results = {}
# DLinear
dl_pred, dl_prob = self.predict_dlinear(df)
if dl_pred is not None:
results['dlinear'] = {'prediction': dl_pred, 'probability': dl_prob}
# N-BEATS
nb_pred, nb_prob = self.predict_nbeats(df)
if nb_pred is not None:
results['nbeats'] = {'prediction': nb_pred, 'probability': nb_prob}
# TTM
ttm_pred, ttm_prob = self.predict_ttm(df)
if ttm_pred is not None:
results['ttm'] = {'prediction': ttm_pred, 'probability': ttm_prob}
# 集成 (投票)
votes = []
if dl_pred is not None: votes.append(dl_pred)
if nb_pred is not None: votes.append(nb_pred)
if ttm_pred is not None: votes.append(ttm_pred)
if votes:
# 简单投票: > 50% 胜出
vote_sum = sum(votes)
if vote_sum > len(votes) / 2:
ensemble_pred = 1
elif vote_sum < len(votes) / 2:
ensemble_pred = 0
else:
# 平票时,如果有 DLinear,优先 DLinear,否则 0 (保守)
ensemble_pred = dl_pred if dl_pred is not None else 0
results['ensemble'] = {'prediction': ensemble_pred, 'votes': votes}
else:
results['ensemble'] = None
return results
def get_inference_manager() -> InferenceManager:
"""获取单例"""
return InferenceManager()
+371
View File
@@ -0,0 +1,371 @@
#!/usr/bin/env python3
"""
MySQL 数据定时备份到 Cloudflare D1
- 定时:每小时 XX:30
- 模式:按游标列增量同步(timestamp/created_at/updated_at/id)
"""
import json
import os
import threading
import time
from datetime import datetime
from decimal import Decimal
from typing import Any
import httpx
import schedule
from dotenv import load_dotenv
from api.models import get_db_connection
from utils.project_logger import get_project_logger
load_dotenv()
logger = get_project_logger("mysql_d1_backup")
class MySQLD1BackupService:
"""MySQL -> D1 备份服务。"""
def __init__(self):
self.enabled = os.getenv("D1_BACKUP_ENABLED", "false").lower() == "true"
self.api_token = os.getenv("CF_D1_API_TOKEN", "")
self.account_id = os.getenv("CF_D1_ACCOUNT_ID", "")
self.database_id = os.getenv("CF_D1_DATABASE_ID", "")
self.batch_size = int(os.getenv("D1_BACKUP_BATCH_SIZE", "500"))
self.max_sql_params = int(os.getenv("D1_MAX_SQL_PARAMS", "90"))
self.tables_config = os.getenv("D1_BACKUP_TABLES", "").strip()
self._job_lock = threading.Lock()
def is_config_ready(self) -> bool:
"""检查 D1 配置是否完整。"""
if not self.enabled:
return False
return bool(self.api_token and self.account_id and self.database_id)
def _d1_endpoint(self) -> str:
return (
f"https://api.cloudflare.com/client/v4/accounts/"
f"{self.account_id}/d1/database/{self.database_id}/query"
)
def _query_d1(self, sql: str, params: list[Any] | None = None) -> dict:
"""执行 D1 SQL。"""
headers = {
"Authorization": f"Bearer {self.api_token}",
"Content-Type": "application/json",
}
statement: dict[str, Any] = {"sql": sql}
if params is not None:
statement["params"] = params
with httpx.Client(timeout=30) as client:
# Cloudflare D1 REST API 对 /query 更稳定的格式是语句数组
resp = client.post(self._d1_endpoint(), headers=headers, json=[statement])
if resp.status_code >= 400:
# 兼容部分环境下使用对象格式的情况,失败时回退重试一次
fallback_resp = client.post(self._d1_endpoint(), headers=headers, json=statement)
if fallback_resp.status_code >= 400:
sql_preview = " ".join(sql.strip().split())[:180]
raise RuntimeError(
f"D1 HTTP {fallback_resp.status_code},sql={sql_preview},响应={fallback_resp.text[:500]}"
)
data = fallback_resp.json()
else:
data = resp.json()
if not data.get("success", False):
raise RuntimeError(f"D1 SQL 执行失败: {data}")
return data
def _d1_results(self, response: dict) -> list[dict]:
"""解析 D1 响应结果。"""
result = response.get("result", [])
if not result:
return []
first = result[0] if isinstance(result, list) else result
return first.get("results", []) if isinstance(first, dict) else []
def _normalize_value(self, value: Any) -> Any:
"""把 MySQL 字段值转换为可写入 D1 的类型。"""
if value is None:
return None
if isinstance(value, (datetime,)):
return value.strftime("%Y-%m-%d %H:%M:%S")
if isinstance(value, Decimal):
return float(value)
if isinstance(value, bytes):
return value.hex()
if isinstance(value, (dict, list)):
return json.dumps(value, ensure_ascii=False)
return value
def _sqlite_type_from_mysql(self, mysql_type: str) -> str:
t = mysql_type.lower()
if any(x in t for x in ["int", "bit", "bool"]):
return "INTEGER"
if any(x in t for x in ["double", "float", "decimal", "numeric", "real"]):
return "REAL"
if any(x in t for x in ["blob", "binary", "varbinary"]):
return "BLOB"
return "TEXT"
def _get_mysql_tables(self) -> list[str]:
"""获取待备份表。"""
if self.tables_config:
return [x.strip() for x in self.tables_config.split(",") if x.strip()]
with get_db_connection() as conn:
cursor = conn.cursor()
cursor.execute("SHOW TABLES")
rows = cursor.fetchall()
tables: list[str] = []
for row in rows:
# DictCursor 下 SHOW TABLES 返回单键字典
table_name = list(row.values())[0]
tables.append(str(table_name))
return tables
def _get_mysql_columns(self, table_name: str) -> list[dict]:
with get_db_connection() as conn:
cursor = conn.cursor()
cursor.execute(f"SHOW COLUMNS FROM `{table_name}`")
return list(cursor.fetchall())
def _guess_cursor_column(self, columns: list[dict]) -> str | None:
"""选择增量同步游标列。"""
column_names = {c["Field"] for c in columns}
for candidate in ["timestamp", "created_at", "updated_at", "id"]:
if candidate in column_names:
return candidate
return None
def _ensure_d1_meta_tables(self):
"""创建 D1 元数据表。"""
self._query_d1(
"""
CREATE TABLE IF NOT EXISTS backup_sync_state (
table_name TEXT PRIMARY KEY,
cursor_col TEXT,
last_cursor TEXT,
updated_at TEXT
)
"""
)
self._query_d1(
"""
CREATE TABLE IF NOT EXISTS backup_run_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
run_at TEXT,
table_name TEXT,
synced_rows INTEGER,
status TEXT,
message TEXT
)
"""
)
def _ensure_d1_table(self, table_name: str, columns: list[dict]):
"""创建 D1 备份表。"""
table_sql_name = f"backup_{table_name}"
col_defs: list[str] = []
primary_keys: list[str] = []
for col in columns:
field = col["Field"]
col_type = self._sqlite_type_from_mysql(col["Type"])
col_defs.append(f'"{field}" {col_type}')
if col.get("Key") == "PRI":
primary_keys.append(field)
# SQLite/D1 对复合主键需要表级 PRIMARY KEY 约束,不能多列各自 PRIMARY KEY
if primary_keys:
pk_cols = ", ".join([f'"{k}"' for k in primary_keys])
col_defs.append(f"PRIMARY KEY ({pk_cols})")
create_sql = f"CREATE TABLE IF NOT EXISTS \"{table_sql_name}\" ({', '.join(col_defs)})"
self._query_d1(create_sql)
def _get_last_cursor(self, table_name: str) -> str | None:
"""读取 D1 已同步游标。"""
resp = self._query_d1(
"SELECT last_cursor FROM backup_sync_state WHERE table_name = ?",
[table_name],
)
rows = self._d1_results(resp)
if not rows:
return None
return rows[0].get("last_cursor")
def _update_last_cursor(self, table_name: str, cursor_col: str, cursor_value: Any):
"""更新 D1 已同步游标。"""
now_str = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
self._query_d1(
"""
INSERT INTO backup_sync_state(table_name, cursor_col, last_cursor, updated_at)
VALUES (?, ?, ?, ?)
ON CONFLICT(table_name) DO UPDATE SET
cursor_col = excluded.cursor_col,
last_cursor = excluded.last_cursor,
updated_at = excluded.updated_at
""",
[table_name, cursor_col, str(cursor_value), now_str],
)
def _log_run(self, table_name: str, synced_rows: int, status: str, message: str):
"""记录备份执行日志。"""
run_at = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
self._query_d1(
"INSERT INTO backup_run_log(run_at, table_name, synced_rows, status, message) VALUES (?, ?, ?, ?, ?)",
[run_at, table_name, synced_rows, status, message[:1000]],
)
def _fetch_mysql_rows(
self,
table_name: str,
cursor_col: str | None,
last_cursor: str | None,
) -> list[dict]:
"""按游标从 MySQL 抽取增量数据。"""
with get_db_connection() as conn:
cursor = conn.cursor()
if cursor_col and last_cursor:
sql = (
f"SELECT * FROM `{table_name}` WHERE `{cursor_col}` > %s "
f"ORDER BY `{cursor_col}` ASC LIMIT %s"
)
cursor.execute(sql, (last_cursor, self.batch_size))
elif cursor_col:
sql = f"SELECT * FROM `{table_name}` ORDER BY `{cursor_col}` ASC LIMIT %s"
cursor.execute(sql, (self.batch_size,))
else:
sql = f"SELECT * FROM `{table_name}` LIMIT %s"
cursor.execute(sql, (self.batch_size,))
return list(cursor.fetchall())
def _upsert_rows_to_d1(self, table_name: str, columns: list[dict], rows: list[dict]):
"""把数据批量 UPSERT 到 D1。"""
if not rows:
return
table_sql_name = f"backup_{table_name}"
col_names = [c["Field"] for c in columns]
col_sql = ", ".join([f'"{c}"' for c in col_names])
# 避免 D1 的 SQL 变量数量限制触发 400
# 通过环境变量 D1_MAX_SQL_PARAMS 控制,默认保守值 90
chunk_size = max(1, self.max_sql_params // max(1, len(col_names)))
for i in range(0, len(rows), chunk_size):
chunk_rows = rows[i:i + chunk_size]
values_clause: list[str] = []
params: list[Any] = []
for row in chunk_rows:
placeholders = ", ".join(["?" for _ in col_names])
values_clause.append(f"({placeholders})")
for col in col_names:
params.append(self._normalize_value(row.get(col)))
sql = (
f"INSERT OR REPLACE INTO \"{table_sql_name}\" ({col_sql}) "
f"VALUES {', '.join(values_clause)}"
)
self._query_d1(sql, params)
def backup_table_once(self, table_name: str) -> int:
"""单表执行一次增量备份,返回同步行数。"""
columns = self._get_mysql_columns(table_name)
if not columns:
return 0
cursor_col = self._guess_cursor_column(columns)
self._ensure_d1_table(table_name, columns)
total_synced = 0
last_cursor = self._get_last_cursor(table_name)
while True:
rows = self._fetch_mysql_rows(table_name, cursor_col, last_cursor)
if not rows:
break
self._upsert_rows_to_d1(table_name, columns, rows)
total_synced += len(rows)
if cursor_col:
last_cursor = self._normalize_value(rows[-1].get(cursor_col))
self._update_last_cursor(table_name, cursor_col, last_cursor)
else:
# 无游标列时仅做一批,避免重复全量
break
if len(rows) < self.batch_size:
break
return total_synced
def run_backup_once(self):
"""执行一次全表备份任务。"""
if not self.is_config_ready():
logger.warning("D1 备份未启用或配置不完整,跳过本次任务")
return
if not self._job_lock.acquire(blocking=False):
logger.warning("上一次 D1 备份尚未完成,跳过本次任务")
return
try:
self._ensure_d1_meta_tables()
tables = self._get_mysql_tables()
logger.info("D1 备份任务开始,待同步表: %s", ", ".join(tables))
for table_name in tables:
try:
synced = self.backup_table_once(table_name)
logger.info("表 %s 同步完成,本次同步 %d 行", table_name, synced)
self._log_run(table_name, synced, "success", "ok")
except Exception as table_error:
logger.exception("表 %s 同步失败", table_name)
self._log_run(table_name, 0, "failed", str(table_error))
except Exception:
logger.exception("D1 备份任务执行失败")
finally:
self._job_lock.release()
def schedule_mysql_d1_backup():
"""启动 D1 备份定时任务(每小时 :30)。"""
service = MySQLD1BackupService()
if not service.is_config_ready():
logger.warning("D1 备份调度未启动:请检查 D1_BACKUP_ENABLED 与 Cloudflare 参数")
return
# 启动后先执行一次,避免首次备份必须等待到下一个 :30
service.run_backup_once()
# 使用独立调度器,避免与项目内其他定时任务共享全局调度状态
local_scheduler = schedule.Scheduler()
local_scheduler.every().hour.at(":30").do(service.run_backup_once)
logger.info("MySQL -> D1 定时备份已启动(每小时 :30 执行)")
while True:
local_scheduler.run_pending()
time.sleep(1)
def run_mysql_d1_backup_scheduler_background():
"""后台线程启动 D1 备份调度器。"""
service = MySQLD1BackupService()
if not service.is_config_ready():
logger.warning("D1 备份配置不完整,后台线程不启动")
return None
scheduler_thread = threading.Thread(target=schedule_mysql_d1_backup, daemon=True)
scheduler_thread.start()
logger.info("D1 备份调度器已在后台启动")
return scheduler_thread
if __name__ == "__main__":
schedule_mysql_d1_backup()
+52 -28
View File
@@ -1,45 +1,69 @@
# Core dependencies
torch>=2.0.0
torchvision
torchaudio
python-binance>=1.0.19
# ==============================================
# 加密货币预测系统依赖
# Python 3.12 | 支持 macOS (MPS) 和 Windows (CPU)
# ==============================================
# ----------------------------------------------
# PyTorch (根据平台单独安装)
# ----------------------------------------------
# macOS: pip install torch torchvision torchaudio
# Windows CPU: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
# 注意:requirements.txt 中不包含 torch,需要手动安装
# ----------------------------------------------
# 数据处理
# ----------------------------------------------
pandas>=2.0.0
numpy>=1.24.0
scikit-learn>=1.3.0
pytz
python-dotenv>=1.0.0
# ONNX
onnx>=1.14.0
onnxruntime>=1.15.0
# ----------------------------------------------
# 时序模型
# ----------------------------------------------
# IBM Granite TTM-R2 (导入名: tsfm_public)
granite-tsfm>=0.3.0
# Model-specific dependencies
darts>=0.27.0
u8darts[torch]>=0.27.0
tsfm_public>=0.2.0
gluonts[torch]>=0.14.0
# ----------------------------------------------
# 技术指标
# ----------------------------------------------
pandas-ta>=0.3.14b
# API dependencies
# ----------------------------------------------
# API 服务
# ----------------------------------------------
fastapi>=0.104.0
uvicorn[standard]>=0.24.0
pydantic>=2.0.0
python-multipart>=0.0.6
# Technical indicators
pandas-ta>=0.3.14b
# ----------------------------------------------
# HTTP 客户端
# ----------------------------------------------
httpx>=0.25.0
# Development tools
jupyter>=1.0.0
matplotlib>=3.7.0
seaborn>=0.12.0
pytest>=7.4.0
black>=23.0.0
flake8>=6.0.0
# ----------------------------------------------
# 数据源
# ----------------------------------------------
python-binance>=1.0.19
# Utilities
tqdm>=4.66.0
# ----------------------------------------------
# 任务调度
# ----------------------------------------------
schedule>=1.2.0
# Database
# ----------------------------------------------
# 数据库
# ----------------------------------------------
pymysql>=1.1.0
sqlalchemy>=2.0.0
cryptography>=41.0.0
# ----------------------------------------------
# 环境变量
# ----------------------------------------------
python-dotenv>=1.0.0
# ----------------------------------------------
# 进度条
# ----------------------------------------------
tqdm>=4.66.0
-458
View File
@@ -1,458 +0,0 @@
"""保存所有测试结果到项目 - 生成完整报告"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import pickle
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import confusion_matrix
from data.fetcher import BinanceDataFetcher
import json
from datetime import datetime
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df, fit=True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df = df[:-self.forecast_horizon].reset_index(drop=True)
prices = df['close'].values.reshape(-1, 1)
future_prices = df['future_price'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
X, y = [], []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
y.append(scaled_future[i + self.lookback, 0])
return np.array(X), np.array(y), df
def inverse_transform_price(self, scaled_price):
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
def main():
print("="*80)
print("保存完整测试结果到项目")
print("="*80)
# 创建结果目录
results_dir = "./results"
os.makedirs(results_dir, exist_ok=True)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
# ========== 加载模型 ==========
print("\n[1/5] 加载模型...")
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
from models.dlinear.model import DLinear
dlinear_checkpoint = torch.load(
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device
)
dlinear_model = DLinear(dlinear_checkpoint['config'])
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
dlinear_model = dlinear_model.to(device)
dlinear_model.eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
from models.nbeats.model import NBeats
nbeats_checkpoint = torch.load(
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
map_location=device
)
nbeats_model = NBeats(nbeats_checkpoint['config'])
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
nbeats_model = nbeats_model.to(device)
nbeats_model.eval()
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
"ibm-granite/granite-timeseries-ttm-r2",
trust_remote_code=True
)
ttm_model = ttm_model.to(device)
ttm_model.eval()
print("✓ 三个模型已加载")
# ========== 获取数据 ==========
print("\n[2/5] 获取2026年1月测试数据...")
fetcher = BinanceDataFetcher()
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
end_date='2026-01-30', use_cache=False)
print(f"✓ 数据: {len(df_2026)} 行, {df_2026['timestamp'].min()} 至 {df_2026['timestamp'].max()}")
# ========== 预处理和预测 ==========
print("\n[3/5] 预处理和预测...")
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
min_samples = min(len(X_dlinear), len(X_nbeats))
X_dlinear = X_dlinear[:min_samples]
y_dlinear = y_dlinear[:min_samples]
X_nbeats = X_nbeats[:min_samples]
# DLinear预测
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
with torch.no_grad():
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits.flatten()))
dlinear_preds = (dlinear_probs > 0.5).astype(int)
# N-BEATS预测
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
with torch.no_grad():
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
# TTM-R2预测
context_length = 512
ttm_preds_list = []
all_prices = df_2026['close'].values
all_open_prices = df_2026['open'].values
for i in range(min_samples):
start_idx = i
end_idx = start_idx + context_length
if end_idx > len(all_prices):
context_data = all_prices[-context_length:].reshape(1, -1, 1)
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
context_tensor = torch.FloatTensor(context_data).to(device)
with torch.no_grad():
output = ttm_model(context_tensor)
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
ttm_preds = np.array(ttm_preds_list)
# 双模型交集
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
two_model_preds = np.full(len(dlinear_preds), -1)
two_model_preds[two_model_agree_up] = 1
two_model_preds[two_model_agree_down] = 0
# 三模型投票
vote_sum = dlinear_preds + nbeats_preds + ttm_preds
three_model_preds = np.where(vote_sum >= 2, 1, 0)
true_labels = y_dlinear
print(f"✓ 预测完成,样本数: {min_samples}")
# ========== 生成详细结果 ==========
print("\n[4/5] 生成详细结果...")
# 时间戳和日期
timestamps = df_2026['timestamp'].iloc[168:168+min_samples].values
timestamps_utc = pd.to_datetime(timestamps)
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
dates = timestamps_et.date
# 创建详细结果DataFrame
results_df = pd.DataFrame({
'timestamp_utc': timestamps_utc,
'timestamp_et': timestamps_et,
'date': dates,
'true_label': true_labels,
'dlinear_prob': dlinear_probs,
'dlinear_pred': dlinear_preds,
'nbeats_pred': nbeats_preds,
'ttm_pred': ttm_preds,
'two_model_pred': two_model_preds,
'three_model_pred': three_model_preds,
'dlinear_correct': dlinear_preds == true_labels,
'two_model_correct': (two_model_preds == true_labels) & (two_model_preds != -1),
'three_model_correct': three_model_preds == true_labels
})
# 按天统计
daily_stats = results_df.groupby('date').agg({
'true_label': 'count',
'two_model_pred': lambda x: (x != -1).sum(),
'two_model_correct': 'sum',
'three_model_correct': 'sum',
'dlinear_correct': 'sum'
})
daily_stats.columns = ['total_samples', 'two_model_valid', 'two_model_correct',
'three_model_correct', 'dlinear_correct']
daily_stats['dlinear_accuracy'] = daily_stats['dlinear_correct'] / daily_stats['total_samples'] * 100
daily_stats['two_model_accuracy'] = daily_stats['two_model_correct'] / daily_stats['two_model_valid'] * 100
daily_stats['three_model_accuracy'] = daily_stats['three_model_correct'] / daily_stats['total_samples'] * 100
daily_stats['two_model_coverage'] = daily_stats['two_model_valid'] / daily_stats['total_samples'] * 100
daily_stats['two_model_accuracy'] = daily_stats['two_model_accuracy'].fillna(0)
# ========== 保存结果 ==========
print("\n[5/5] 保存结果文件...")
# 1. 保存逐小时详细结果
hourly_file = f"{results_dir}/hourly_predictions_2026_01.csv"
results_df.to_csv(hourly_file, index=False)
print(f"✓ 保存逐小时预测: {hourly_file}")
# 2. 保存每日统计
daily_file = f"{results_dir}/daily_statistics_2026_01.csv"
daily_stats.to_csv(daily_file)
print(f"✓ 保存每日统计: {daily_file}")
# 3. 生成总结报告
summary = {
"测试期间": {
"开始日期": str(df_2026['timestamp'].min()),
"结束日期": str(df_2026['timestamp'].max()),
"总天数": len(daily_stats),
"总样本数": int(min_samples)
},
"单模型性能": {
"DLinear": {
"准确率": f"{(dlinear_preds == true_labels).mean() * 100:.2f}%",
"预测上涨比例": f"{(dlinear_preds == 1).mean() * 100:.1f}%"
},
"N-BEATS": {
"准确率": f"{(nbeats_preds == true_labels).mean() * 100:.2f}%",
"预测上涨比例": f"{(nbeats_preds == 1).mean() * 100:.1f}%"
},
"TTM-R2": {
"准确率": f"{(ttm_preds == true_labels).mean() * 100:.2f}%",
"预测上涨比例": f"{(ttm_preds == 1).mean() * 100:.1f}%"
}
},
"集成策略性能": {
"双模型交集": {
"准确率": f"{((two_model_preds == true_labels) & (two_model_preds != -1)).sum() / (two_model_preds != -1).sum() * 100:.2f}%",
"覆盖率": f"{(two_model_preds != -1).mean() * 100:.2f}%",
"有效预测数": int((two_model_preds != -1).sum())
},
"三模型投票": {
"准确率": f"{(three_model_preds == true_labels).mean() * 100:.2f}%",
"覆盖率": "100.00%",
"有效预测数": int(len(three_model_preds))
}
},
"DLinear概率分布": {
"0.4-0.5区间": f"{((dlinear_probs >= 0.4) & (dlinear_probs < 0.5)).sum()}个 ({((dlinear_probs >= 0.4) & (dlinear_probs < 0.5)).mean() * 100:.1f}%)",
"0.5-0.6区间": f"{((dlinear_probs >= 0.5) & (dlinear_probs < 0.6)).sum()}个 ({((dlinear_probs >= 0.5) & (dlinear_probs < 0.6)).mean() * 100:.1f}%)",
"0.6-0.7区间": f"{((dlinear_probs >= 0.6) & (dlinear_probs < 0.7)).sum()}个 ({((dlinear_probs >= 0.6) & (dlinear_probs < 0.7)).mean() * 100:.1f}%)",
"平均概率": f"{dlinear_probs.mean():.3f}"
},
"最佳/最差交易日": {
"双模型最佳日": {
"日期": str(daily_stats['two_model_accuracy'].idxmax()),
"准确率": f"{daily_stats['two_model_accuracy'].max():.1f}%"
},
"双模型最差日": {
"日期": str(daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].idxmin()),
"准确率": f"{daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].min():.1f}%"
},
"三模型最佳日": {
"日期": str(daily_stats['three_model_accuracy'].idxmax()),
"准确率": f"{daily_stats['three_model_accuracy'].max():.1f}%"
},
"三模型最差日": {
"日期": str(daily_stats['three_model_accuracy'].idxmin()),
"准确率": f"{daily_stats['three_model_accuracy'].min():.1f}%"
}
}
}
summary_file = f"{results_dir}/summary_report_2026_01.json"
with open(summary_file, 'w', encoding='utf-8') as f:
json.dump(summary, f, indent=2, ensure_ascii=False)
print(f"✓ 保存总结报告: {summary_file}")
# 4. 生成Markdown报告
markdown_report = f"""# 加密货币价格预测模型测试报告
## 测试概况
- **测试期间**: 2026年1月1日 - 2026年1月30日
- **总天数**: {len(daily_stats)}天
- **总样本数**: {min_samples}个小时
- **生成时间**: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
---
## 单模型性能
### DLinear分类模型
- **准确率**: {(dlinear_preds == true_labels).mean() * 100:.2f}%
- **预测上涨比例**: {(dlinear_preds == 1).mean() * 100:.1f}%
- **预测下跌比例**: {(dlinear_preds == 0).mean() * 100:.1f}%
### N-BEATS回归模型
- **准确率**: {(nbeats_preds == true_labels).mean() * 100:.2f}%
- **预测上涨比例**: {(nbeats_preds == 1).mean() * 100:.1f}%
- **预测下跌比例**: {(nbeats_preds == 0).mean() * 100:.1f}%
### TTM-R2零样本模型
- **准确率**: {(ttm_preds == true_labels).mean() * 100:.2f}%
- **预测上涨比例**: {(ttm_preds == 1).mean() * 100:.1f}%
- **预测下跌比例**: {(ttm_preds == 0).mean() * 100:.1f}%
---
## 集成策略性能
### 双模型交集(DLinear + N-BEATS)
- **准确率**: {((two_model_preds == true_labels) & (two_model_preds != -1)).sum() / (two_model_preds != -1).sum() * 100:.2f}%
- **覆盖率**: {(two_model_preds != -1).mean() * 100:.2f}%
- **有效预测数**: {(two_model_preds != -1).sum()}次
### 三模型多数投票(DLinear + N-BEATS + TTM-R2)
- **准确率**: {(three_model_preds == true_labels).mean() * 100:.2f}%
- **覆盖率**: 100.00%
- **有效预测数**: {len(three_model_preds)}次
---
## DLinear概率分布分析
| 概率区间 | 样本数 | 占比 |
|---------|--------|------|
| 0.4-0.5 | {((dlinear_probs >= 0.4) & (dlinear_probs < 0.5)).sum()} | {((dlinear_probs >= 0.4) & (dlinear_probs < 0.5)).mean() * 100:.1f}% |
| 0.5-0.6 | {((dlinear_probs >= 0.5) & (dlinear_probs < 0.6)).sum()} | {((dlinear_probs >= 0.5) & (dlinear_probs < 0.6)).mean() * 100:.1f}% |
| 0.6-0.7 | {((dlinear_probs >= 0.6) & (dlinear_probs < 0.7)).sum()} | {((dlinear_probs >= 0.6) & (dlinear_probs < 0.7)).mean() * 100:.1f}% |
**平均概率**: {dlinear_probs.mean():.3f}
---
## 最佳/最差交易日
### 双模型交集
- **最佳日**: {daily_stats['two_model_accuracy'].idxmax()} ({daily_stats['two_model_accuracy'].max():.1f}%)
- **最差日**: {daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].idxmin()} ({daily_stats[daily_stats['two_model_accuracy'] > 0]['two_model_accuracy'].min():.1f}%)
### 三模型投票
- **最佳日**: {daily_stats['three_model_accuracy'].idxmax()} ({daily_stats['three_model_accuracy'].max():.1f}%)
- **最差日**: {daily_stats['three_model_accuracy'].idxmin()} ({daily_stats['three_model_accuracy'].min():.1f}%)
---
## 结论
1. **最佳单模型**: DLinear ({(dlinear_preds == true_labels).mean() * 100:.2f}%)
2. **最佳集成策略**: 双模型交集 ({((two_model_preds == true_labels) & (two_model_preds != -1)).sum() / (two_model_preds != -1).sum() * 100:.2f}%)
3. **DLinear置信度**: 99.2%的预测集中在0.45-0.55区间,缺乏高置信度预测
## 数据文件
- 逐小时预测结果: `hourly_predictions_2026_01.csv`
- 每日统计数据: `daily_statistics_2026_01.csv`
- 总结报告: `summary_report_2026_01.json`
---
生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
"""
markdown_file = f"{results_dir}/README_2026_01.md"
with open(markdown_file, 'w', encoding='utf-8') as f:
f.write(markdown_report)
print(f"✓ 保存Markdown报告: {markdown_file}")
print("\n" + "="*80)
print("✓ 所有结果已保存到 ./results/ 目录")
print("="*80)
print(f"\n保存的文件:")
print(f" 1. {hourly_file}")
print(f" 2. {daily_file}")
print(f" 3. {summary_file}")
print(f" 4. {markdown_file}")
print("\n" + "="*80)
if __name__ == "__main__":
main()
+134
View File
@@ -0,0 +1,134 @@
"""导出 DLinear, N-BEATS 和 TTM-R2 模型为 ONNX 格式。"""
import sys
import os
import torch
import numpy as np
from pathlib import Path
# 添加项目根目录到路径
project_root = Path(__file__).parent.parent
sys.path.insert(0, str(project_root))
from models.dlinear.model import DLinear
from models.nbeats.model import NBeats
from tsfm_public.toolkit.get_model import get_model
def export_dlinear():
print("\n" + "="*50)
print("导出 DLinear 模型...")
try:
checkpoint_path = project_root / "models/checkpoints/dlinear_classification_2026/best_model.pt"
output_path = checkpoint_path.parent / "model.onnx"
if not checkpoint_path.exists():
print(f"跳过: 找不到检查点 {checkpoint_path}")
return
print(f"加载模型: {checkpoint_path}")
model = DLinear.load(str(checkpoint_path), device='cpu')
print(f"模型配置: lookback={model.config.lookback}, input_dim={model.config.input_dim}")
model.to_onnx(str(output_path))
print(f"✓ ONNX 模型已保存到: {output_path}")
except Exception as e:
print(f"导出 DLinear 失败: {e}")
def export_nbeats():
print("\n" + "="*50)
print("导出 N-BEATS 模型...")
try:
checkpoint_path = project_root / "models/checkpoints/nbeats_btc_1h/best_model.pt"
output_path = checkpoint_path.parent / "model.onnx"
if not checkpoint_path.exists():
print(f"跳过: 找不到检查点 {checkpoint_path}")
return
print(f"加载模型: {checkpoint_path}")
checkpoint = torch.load(str(checkpoint_path), map_location='cpu')
model = NBeats(checkpoint['config'])
model.eval()
# N-BEATS input: (batch_size, input_size)
# input_size default is 168
input_size = checkpoint['config'].input_size
dummy_input = torch.randn(1, input_size)
print(f"导出 ONNX (input_size={input_size})...")
torch.onnx.export(
model,
dummy_input,
str(output_path),
export_params=True,
opset_version=17,
do_constant_folding=True,
input_names=['input'],
output_names=['forecast'],
dynamic_axes={'input': {0: 'batch_size'}, 'forecast': {0: 'batch_size'}}
)
print(f"✓ ONNX 模型已保存到: {output_path}")
except Exception as e:
print(f"导出 N-BEATS 失败: {e}")
import traceback
traceback.print_exc()
def export_ttm_r2():
print("\n" + "="*50)
print("导出 TTM-R2 模型...")
try:
model_name = "ibm-granite/granite-timeseries-ttm-r2"
output_dir = project_root / "models/checkpoints/ttm_r2_onnx"
output_path = output_dir / "model.onnx"
output_dir.mkdir(parents=True, exist_ok=True)
print(f"加载模型: {model_name}")
base_model = get_model(
model_path=model_name,
context_length=512,
prediction_length=96,
)
base_model.eval()
print(f"TTM Config: {base_model.config}")
# TTM input: (batch_size, context_length, num_features)
# Context length 512, features 1
dummy_input = torch.randn(1, 512, 1)
with torch.no_grad():
output = base_model(dummy_input)
if hasattr(output, 'prediction_outputs'):
test_output = output.prediction_outputs
else:
test_output = output
print(f"✓ 输出形状: {test_output.shape}")
print(f"导出 ONNX (固定 Batch Size=1)...")
torch.onnx.export(
base_model,
dummy_input,
str(output_path),
export_params=True,
opset_version=17, # Revert to 14 as it is standard
do_constant_folding=True,
input_names=['past_values'],
output_names=['predictions'],
# Remove dynamic axes to fix potential shape inference issues
dynamic_axes={'past_values': {0: 'batch_size'}, 'predictions': {0: 'batch_size'}},
verbose=False
)
print(f"✓ ONNX 模型已保存到: {output_path}")
except Exception as e:
print(f"导出 TTM-R2 失败: {e}")
import traceback
traceback.print_exc()
def main():
export_dlinear()
export_nbeats()
#export_ttm_r2()
if __name__ == "__main__":
main()
+91
View File
@@ -0,0 +1,91 @@
import os
file_path = 'trader/auto_trade_polymarket.py'
with open(file_path, 'r', encoding='utf-8') as f:
lines = f.readlines()
new_lines = []
inserted = False
# Code to insert
insert_code = """
# === 新增: 保存初始预测和状态 ===
try:
# 确定最终预测 (用于日志)
final_pred = 0
if 'ensemble' in all_preds and all_preds['ensemble']:
final_pred = all_preds['ensemble']['prediction']
elif 'dlinear' in all_preds:
final_pred = all_preds['dlinear']['prediction']
et_tz = ZoneInfo('America/New_York')
current_time_utc = pd.Timestamp(current_time)
current_time_et = current_time_utc.tz_localize('UTC').tz_convert(et_tz)
# 1. 保存预测到 model_predictions
common_data = {
"timestamp": current_time_utc.isoformat(),
"timestamp_et": current_time_et.isoformat(),
"date": current_time_et.strftime('%Y-%m-%d'),
"hour": current_time_et.hour,
"actual_result": None,
"is_correct": None,
"open_price": float(df['open'].iloc[-1]),
"close_price": None,
}
def save_single_pred(name, pred_res):
data = common_data.copy()
data['model_name'] = name
data['prediction'] = pred_res['prediction']
data['probability'] = pred_res.get('probability', 0.5)
PredictionModel.save_prediction(data)
if 'dlinear' in all_preds: save_single_pred("DLinear", all_preds['dlinear'])
if 'nbeats' in all_preds: save_single_pred("N-BEATS", all_preds['nbeats'])
if 'ttm' in all_preds: save_single_pred("TTM-R2", all_preds['ttm'])
if 'ensemble' in all_preds and all_preds['ensemble']: save_single_pred("Ensemble", all_preds['ensemble'])
logger.info(f"💾 [Init] 初始预测已保存到数据库")
# 2. 记录当前监控状态到 trades (作为 'in_progress')
status = 'in_progress'
market_title = self.current_market['title'] if self.current_market else "Waiting for Market..."
market_slug = self.current_market['slug'] if self.current_market else None
log_data = {
'timestamp': datetime.now().isoformat(),
'market_title': market_title,
'market_slug': market_slug,
'model_name': 'Ensemble',
'prediction': 'UP' if final_pred == 1 else 'DOWN',
'probability': 0.0,
'action': 'MONITOR',
'price': 0,
'amount': 0,
'size': 0,
'status': status,
'error_message': 'System Started - Monitoring',
'raw_data': {}
}
self.db.log_trade(log_data)
logger.info(f"💾 [Init] 初始监控状态已记录")
except Exception as e:
logger.error(f"保存初始状态失败: {e}")
import traceback
traceback.print_exc()
"""
for line in lines:
if "def wait_for_next_hour(self)" in line and not inserted:
new_lines.append(insert_code)
inserted = True
new_lines.append(line)
with open(file_path, 'w', encoding='utf-8') as f:
f.writelines(new_lines)
print("Successfully patched trader/auto_trade_polymarket.py")
-451
View File
@@ -1,451 +0,0 @@
"""智能休息策略回测:基于历史准确率动态决定是否交易"""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import torch
import pickle
import numpy as np
import pandas as pd
import pandas_ta as ta
from sklearn.preprocessing import StandardScaler
from data.fetcher import BinanceDataFetcher
# ==================== 预处理器定义 ====================
class ClassificationPreprocessor:
"""DLinear分类预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.scaler = StandardScaler()
self.is_fitted = False
def add_features(self, df):
df = df.copy()
df['rsi'] = ta.rsi(df['close'], length=14)
macd = ta.macd(df['close'])
if macd is not None and not macd.empty:
df['macd'] = macd.iloc[:, 0]
df['macd_hist'] = macd.iloc[:, 1]
df['macd_signal'] = macd.iloc[:, 2]
return df
def create_labels(self, df):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df['future_return'] = (df['future_price'] - df['close']) / df['close']
df['label'] = (df['future_return'] > 0).astype(int)
return df[:-self.forecast_horizon]
def preprocess(self, df, fit=True):
df = self.add_features(df)
df = self.create_labels(df)
df = df.dropna().reset_index(drop=True)
feature_cols = ['open', 'high', 'low', 'close', 'volume',
'rsi', 'macd', 'macd_hist', 'macd_signal']
if fit:
scaled_features = self.scaler.fit_transform(df[feature_cols])
self.is_fitted = True
self.feature_columns = feature_cols
else:
scaled_features = self.scaler.transform(df[feature_cols])
X, y = [], []
for i in range(len(scaled_features) - self.lookback):
X.append(scaled_features[i:i + self.lookback])
y.append(df['label'].iloc[i + self.lookback])
return np.array(X), np.array(y), df
class NBeatsPreprocessor:
"""N-BEATS预处理器"""
def __init__(self, lookback=168, forecast_horizon=1):
self.lookback = lookback
self.forecast_horizon = forecast_horizon
self.price_scaler = StandardScaler()
self.is_fitted = False
def preprocess(self, df, fit=True):
df = df.copy()
df['future_price'] = df['close'].shift(-self.forecast_horizon)
df = df[:-self.forecast_horizon].reset_index(drop=True)
prices = df['close'].values.reshape(-1, 1)
future_prices = df['future_price'].values.reshape(-1, 1)
if fit:
scaled_prices = self.price_scaler.fit_transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
self.is_fitted = True
else:
scaled_prices = self.price_scaler.transform(prices)
scaled_future = self.price_scaler.transform(future_prices)
X, y = [], []
for i in range(len(scaled_prices) - self.lookback):
X.append(scaled_prices[i:i + self.lookback, 0])
y.append(scaled_future[i + self.lookback, 0])
return np.array(X), np.array(y), df
def inverse_transform_price(self, scaled_price):
return self.price_scaler.inverse_transform(scaled_price.reshape(-1, 1)).flatten()
def main():
print("="*80)
print("智能休息策略回测:基于历史准确率动态决定是否交易")
print("="*80)
print("\n策略规则:")
print(" 1. 如果前一天准确率超过70% → 当天休息")
print(" 2. 如果连续两天准确率超过60% → 第三天休息")
print(" 3. 其他情况 → 正常交易")
print("="*80)
# ========== 加载模型 ==========
print("\n[1/4] 加载模型...")
device = torch.device('mps' if torch.backends.mps.is_available() else 'cpu')
# 加载DLinear
from models.dlinear.model import DLinear
dlinear_checkpoint = torch.load(
'./models/checkpoints/dlinear_classification_2026/best_model.pt',
map_location=device
)
dlinear_model = DLinear(dlinear_checkpoint['config'])
dlinear_model.load_state_dict(dlinear_checkpoint['model_state_dict'])
dlinear_model = dlinear_model.to(device)
dlinear_model.eval()
with open('./models/checkpoints/dlinear_classification_2026/preprocessor.pkl', 'rb') as f:
dlinear_preprocessor = pickle.load(f)
# 加载N-BEATS
from models.nbeats.model import NBeats
nbeats_checkpoint = torch.load(
'./models/checkpoints/nbeats_btc_1h/best_model.pt',
map_location=device
)
nbeats_model = NBeats(nbeats_checkpoint['config'])
nbeats_model.load_state_dict(nbeats_checkpoint['model_state_dict'])
nbeats_model = nbeats_model.to(device)
nbeats_model.eval()
with open('./models/checkpoints/nbeats_btc_1h/preprocessor.pkl', 'rb') as f:
nbeats_preprocessor = pickle.load(f)
# 加载TTM-R2
from tsfm_public.models.tinytimemixer import TinyTimeMixerForPrediction
ttm_model = TinyTimeMixerForPrediction.from_pretrained(
"ibm-granite/granite-timeseries-ttm-r2",
trust_remote_code=True
)
ttm_model = ttm_model.to(device)
ttm_model.eval()
print(f"✓ 三个模型已加载")
# ========== 获取数据 ==========
print("\n[2/4] 获取2026年1月测试数据(1月1日-1月30日)...")
fetcher = BinanceDataFetcher()
df_2026 = fetcher.fetch_ohlcv('BTC', '1h', start_date='2026-01-01',
end_date='2026-01-30', use_cache=False)
print(f"✓ 测试数据: {len(df_2026)} 行")
print(f"✓ 时间范围: {df_2026['timestamp'].min()} 至 {df_2026['timestamp'].max()}")
# ========== 预处理 ==========
print("\n[3/4] 预处理数据...")
X_dlinear, y_dlinear, df_dlinear = dlinear_preprocessor.preprocess(df_2026, fit=False)
X_nbeats, y_nbeats, df_nbeats = nbeats_preprocessor.preprocess(df_2026, fit=False)
min_samples = min(len(X_dlinear), len(X_nbeats))
X_dlinear = X_dlinear[:min_samples]
y_dlinear = y_dlinear[:min_samples]
X_nbeats = X_nbeats[:min_samples]
y_nbeats = y_nbeats[:min_samples]
# ========== 预测 ==========
print("\n[4/4] 执行预测...")
# DLinear预测
X_dlinear_tensor = torch.FloatTensor(X_dlinear).to(device)
with torch.no_grad():
dlinear_logits = dlinear_model(X_dlinear_tensor).cpu().numpy()
dlinear_probs = 1 / (1 + np.exp(-dlinear_logits))
dlinear_preds = (dlinear_probs > 0.5).astype(int).flatten()
# N-BEATS预测
X_nbeats_tensor = torch.FloatTensor(X_nbeats).to(device)
with torch.no_grad():
nbeats_pred_scaled = nbeats_model(X_nbeats_tensor).cpu().numpy()
nbeats_pred_prices = nbeats_preprocessor.inverse_transform_price(nbeats_pred_scaled)
current_open_prices = df_nbeats['open'].iloc[168:168+len(nbeats_pred_prices)].values
nbeats_preds = (nbeats_pred_prices > current_open_prices).astype(int)
# TTM-R2预测
context_length = 512
ttm_preds_list = []
all_prices = df_2026['close'].values
all_open_prices = df_2026['open'].values
for i in range(min_samples):
start_idx = i
end_idx = start_idx + context_length
if end_idx > len(all_prices):
context_data = all_prices[-context_length:].reshape(1, -1, 1)
current_open_price = all_open_prices[min(168 + i, len(all_open_prices) - 1)]
else:
context_data = all_prices[start_idx:end_idx].reshape(1, -1, 1)
current_open_price = all_open_prices[end_idx] if end_idx < len(all_open_prices) else all_open_prices[-1]
context_tensor = torch.FloatTensor(context_data).to(device)
with torch.no_grad():
output = ttm_model(context_tensor)
pred_price = output.prediction_outputs.cpu().numpy()[0, 0, 0]
ttm_preds_list.append(1 if pred_price > current_open_price else 0)
ttm_preds = np.array(ttm_preds_list)
# 双模型交集
two_model_agree_up = (dlinear_preds == 1) & (nbeats_preds == 1)
two_model_agree_down = (dlinear_preds == 0) & (nbeats_preds == 0)
two_model_preds = np.full(len(dlinear_preds), -1)
two_model_preds[two_model_agree_up] = 1
two_model_preds[two_model_agree_down] = 0
# 三模型多数投票
vote_sum = dlinear_preds + nbeats_preds + ttm_preds
three_model_preds = np.where(vote_sum >= 2, 1, 0)
# 真实标签
true_labels = y_dlinear
# ========== 构建每日数据 ==========
timestamps = df_2026['timestamp'].iloc[168:168+min_samples].values
timestamps_utc = pd.to_datetime(timestamps)
timestamps_et = timestamps_utc.tz_localize('UTC').tz_convert('America/New_York')
dates = timestamps_et.date
results_df = pd.DataFrame({
'date': dates,
'timestamp_et': timestamps_et,
'true_label': true_labels,
'two_model_pred': two_model_preds,
'three_model_pred': three_model_preds,
'two_model_correct': (two_model_preds == true_labels) & (two_model_preds != -1),
'three_model_correct': (three_model_preds == true_labels)
})
# 按天统计基础数据
daily_stats = results_df.groupby('date').agg({
'true_label': 'count',
'two_model_pred': lambda x: (x != -1).sum(),
'two_model_correct': 'sum',
'three_model_correct': 'sum'
})
daily_stats.columns = ['total_samples', 'two_model_valid', 'two_model_correct', 'three_model_correct']
# 计算每日准确率
daily_stats['two_model_accuracy'] = daily_stats['two_model_correct'] / daily_stats['two_model_valid'] * 100
daily_stats['three_model_accuracy'] = daily_stats['three_model_correct'] / daily_stats['total_samples'] * 100
daily_stats['two_model_accuracy'] = daily_stats['two_model_accuracy'].fillna(0)
# ========== 应用智能休息策略 ==========
print("\n" + "="*80)
print("📊 应用智能休息策略")
print("="*80)
# 为每一天添加"是否休息"标记
daily_stats['two_model_rest'] = False
daily_stats['three_model_rest'] = False
daily_stats['rest_reason'] = ''
dates_list = daily_stats.index.tolist()
for i, date in enumerate(dates_list):
if i == 0:
continue # 第一天没有历史数据,不休息
# 获取前一天和前两天的数据
prev_date = dates_list[i-1]
prev2_date = dates_list[i-2] if i >= 2 else None
# 双模型休息逻辑
# 规则1:前一天准确率超过70%
if daily_stats.loc[prev_date, 'two_model_accuracy'] > 70:
daily_stats.loc[date, 'two_model_rest'] = True
daily_stats.loc[date, 'rest_reason'] = f'前日准确率{daily_stats.loc[prev_date, "two_model_accuracy"]:.1f}%>70%'
# 规则2:连续两天准确率超过60%
elif (prev2_date is not None and
daily_stats.loc[prev_date, 'two_model_accuracy'] > 60 and
daily_stats.loc[prev2_date, 'two_model_accuracy'] > 60):
daily_stats.loc[date, 'two_model_rest'] = True
daily_stats.loc[date, 'rest_reason'] = f'连续2天>60% ({daily_stats.loc[prev2_date, "two_model_accuracy"]:.1f}%, {daily_stats.loc[prev_date, "two_model_accuracy"]:.1f}%)'
# 三模型休息逻辑
# 规则1:前一天准确率超过70%
if daily_stats.loc[prev_date, 'three_model_accuracy'] > 70:
daily_stats.loc[date, 'three_model_rest'] = True
if not daily_stats.loc[date, 'rest_reason']:
daily_stats.loc[date, 'rest_reason'] = f'三模前日{daily_stats.loc[prev_date, "three_model_accuracy"]:.1f}%>70%'
# 规则2:连续两天准确率超过60%
elif (prev2_date is not None and
daily_stats.loc[prev_date, 'three_model_accuracy'] > 60 and
daily_stats.loc[prev2_date, 'three_model_accuracy'] > 60):
daily_stats.loc[date, 'three_model_rest'] = True
if not daily_stats.loc[date, 'rest_reason']:
daily_stats.loc[date, 'rest_reason'] = f'三模连续2天>60%'
# ========== 统计休息天数 ==========
two_model_rest_days = daily_stats['two_model_rest'].sum()
three_model_rest_days = daily_stats['three_model_rest'].sum()
total_days = len(daily_stats)
print(f"\n休息天数统计:")
print(f" 总交易日: {total_days}天")
print(f" 双模型休息: {two_model_rest_days}天 ({two_model_rest_days/total_days*100:.1f}%)")
print(f" 三模型休息: {three_model_rest_days}天 ({three_model_rest_days/total_days*100:.1f}%)")
# ========== 计算休息策略的实际表现 ==========
print("\n" + "="*80)
print("📈 策略对比:正常交易 vs 智能休息")
print("="*80)
# 双模型策略
print(f"\n🔵 双模型交集策略:")
# 正常交易(不休息)
two_normal_correct = daily_stats['two_model_correct'].sum()
two_normal_total = daily_stats['two_model_valid'].sum()
two_normal_acc = two_normal_correct / two_normal_total * 100 if two_normal_total > 0 else 0
print(f"\n 📊 正常交易(不休息):")
print(f" 交易天数: {total_days}天")
print(f" 有效预测: {two_normal_total}次")
print(f" 正确预测: {two_normal_correct}次")
print(f" 准确率: {two_normal_acc:.2f}%")
# 智能休息
two_rest_mask = ~daily_stats['two_model_rest']
two_rest_correct = daily_stats.loc[two_rest_mask, 'two_model_correct'].sum()
two_rest_total = daily_stats.loc[two_rest_mask, 'two_model_valid'].sum()
two_rest_acc = two_rest_correct / two_rest_total * 100 if two_rest_total > 0 else 0
two_trade_days = two_rest_mask.sum()
print(f"\n 🛌 智能休息策略:")
print(f" 交易天数: {two_trade_days}天 (休息{two_model_rest_days}天)")
print(f" 有效预测: {two_rest_total}次")
print(f" 正确预测: {two_rest_correct}次")
print(f" 准确率: {two_rest_acc:.2f}%")
two_acc_diff = two_rest_acc - two_normal_acc
two_diff_mark = "✅" if two_acc_diff > 0 else ("⚠️" if two_acc_diff < 0 else "")
print(f"\n 准确率变化: {two_acc_diff:+.2f}% {two_diff_mark}")
# 休息日的实际表现
if two_model_rest_days > 0:
two_rest_days_mask = daily_stats['two_model_rest']
two_rest_day_correct = daily_stats.loc[two_rest_days_mask, 'two_model_correct'].sum()
two_rest_day_total = daily_stats.loc[two_rest_days_mask, 'two_model_valid'].sum()
two_rest_day_acc = two_rest_day_correct / two_rest_day_total * 100 if two_rest_day_total > 0 else 0
print(f"\n 🔍 被休息的{two_model_rest_days}天实际表现:")
print(f" 准确率: {two_rest_day_acc:.2f}%")
avoided_loss = "避免了低准确率" if two_rest_day_acc < two_normal_acc else "⚠️ 错过了好机会"
print(f" 评价: {avoided_loss}")
# 三模型策略
print(f"\n🟢 三模型投票策略:")
# 正常交易
three_normal_correct = daily_stats['three_model_correct'].sum()
three_normal_total = daily_stats['total_samples'].sum()
three_normal_acc = three_normal_correct / three_normal_total * 100
print(f"\n 📊 正常交易(不休息):")
print(f" 交易天数: {total_days}天")
print(f" 有效预测: {three_normal_total}次")
print(f" 正确预测: {three_normal_correct}次")
print(f" 准确率: {three_normal_acc:.2f}%")
# 智能休息
three_rest_mask = ~daily_stats['three_model_rest']
three_rest_correct = daily_stats.loc[three_rest_mask, 'three_model_correct'].sum()
three_rest_total = daily_stats.loc[three_rest_mask, 'total_samples'].sum()
three_rest_acc = three_rest_correct / three_rest_total * 100 if three_rest_total > 0 else 0
three_trade_days = three_rest_mask.sum()
print(f"\n 🛌 智能休息策略:")
print(f" 交易天数: {three_trade_days}天 (休息{three_model_rest_days}天)")
print(f" 有效预测: {three_rest_total}次")
print(f" 正确预测: {three_rest_correct}次")
print(f" 准确率: {three_rest_acc:.2f}%")
three_acc_diff = three_rest_acc - three_normal_acc
three_diff_mark = "✅" if three_acc_diff > 0 else ("⚠️" if three_acc_diff < 0 else "")
print(f"\n 准确率变化: {three_acc_diff:+.2f}% {three_diff_mark}")
# 休息日的实际表现
if three_model_rest_days > 0:
three_rest_days_mask = daily_stats['three_model_rest']
three_rest_day_correct = daily_stats.loc[three_rest_days_mask, 'three_model_correct'].sum()
three_rest_day_total = daily_stats.loc[three_rest_days_mask, 'total_samples'].sum()
three_rest_day_acc = three_rest_day_correct / three_rest_day_total * 100
print(f"\n 🔍 被休息的{three_model_rest_days}天实际表现:")
print(f" 准确率: {three_rest_day_acc:.2f}%")
avoided_loss = "避免了低准确率" if three_rest_day_acc < three_normal_acc else "⚠️ 错过了好机会"
print(f" 评价: {avoided_loss}")
# ========== 详细日志 ==========
print("\n" + "="*80)
print("📅 每日详细记录")
print("="*80)
print(f"\n{'日期':<12} {'双模准确率':>10} {'三模准确率':>10} {'休息状态':<12} {'休息原因':<40}")
print("-" * 95)
for date, row in daily_stats.iterrows():
two_acc = row['two_model_accuracy']
three_acc = row['three_model_accuracy']
rest_status = ""
if row['two_model_rest']:
rest_status += "双休 "
if row['three_model_rest']:
rest_status += "三休"
if not rest_status:
rest_status = "正常交易"
# 准确率标记
two_mark = "🔥" if two_acc >= 70 else ("✅" if two_acc >= 60 else "")
three_mark = "🔥" if three_acc >= 70 else ("✅" if three_acc >= 60 else "")
print(f"{date} {two_acc:>9.1f}{two_mark} {three_acc:>9.1f}{three_mark} "
f"{rest_status:<12} {row['rest_reason']:<40}")
print("\n" + "="*80)
print("✓ 智能休息策略回测完成!")
print("="*80)
if __name__ == "__main__":
main()
+130 -12
View File
@@ -4,34 +4,90 @@
同时启动:
1. FastAPI 后端服务 (提供前端看板 API)
2. Polymarket 自动交易程序
3. 每日数据报告推送程序 (北京时间每天 8 点)
支持平台:macOS, Linux, Windows
"""
import subprocess
import sys
import time
import signal
import os
import platform
from datetime import datetime
from dotenv import load_dotenv
# 颜色代码
from utils.network_proxy import apply_global_proxy_from_env
from utils.project_logger import get_project_logger
try:
from daily_report_sender import run_scheduler_background
except ImportError:
# 如果模块导入失败,定义一个空函数避免启动失败
def run_scheduler_background():
return None
try:
from mysql_d1_backup_scheduler import run_mysql_d1_backup_scheduler_background
except ImportError:
# 如果模块导入失败,定义一个空函数避免启动失败
def run_mysql_d1_backup_scheduler_background():
return None
load_dotenv()
apply_global_proxy_from_env()
logger = get_project_logger("start_system")
# 检测操作系统
IS_WINDOWS = platform.system() == 'Windows'
# 颜色代码 (Windows CMD 默认不支持 ANSI,但现代 Windows Terminal 支持)
if IS_WINDOWS:
# 尝试启用 Windows ANSI 支持
try:
import ctypes
kernel32 = ctypes.windll.kernel32
kernel32.SetConsoleMode(kernel32.GetStdHandle(-11), 7)
ANSI_SUPPORTED = True
except Exception:
ANSI_SUPPORTED = False
else:
ANSI_SUPPORTED = True
if ANSI_SUPPORTED:
GREEN = '\033[92m'
YELLOW = '\033[93m'
RED = '\033[91m'
RESET = '\033[0m'
else:
GREEN = YELLOW = RED = RESET = ''
def log(message, color=RESET):
timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
print(f"{color}[{timestamp}] {message}{RESET}")
logger.info(message)
def start_process(command, name):
"""启动子进程"""
"""启动子进程(跨平台兼容)"""
log(f"正在启动 {name}...", YELLOW)
try:
if IS_WINDOWS:
# Windows: 不使用 preexec_fn,使用 CREATE_NEW_PROCESS_GROUP
process = subprocess.Popen(
command,
shell=True,
stdout=sys.stdout,
stderr=sys.stderr,
preexec_fn=os.setsid # 创建新的进程组,方便统一杀掉
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP
)
else:
# macOS / Linux: 使用 setsid 创建新进程组
process = subprocess.Popen(
command,
shell=True,
stdout=sys.stdout,
stderr=sys.stderr,
preexec_fn=os.setsid
)
log(f"{name} 已启动 (PID: {process.pid})", GREEN)
return process
@@ -39,6 +95,43 @@ def start_process(command, name):
log(f"启动 {name} 失败: {e}", RED)
return None
def terminate_process(process, name):
"""终止进程(跨平台兼容)"""
if process is None:
return
try:
if IS_WINDOWS:
# Windows: 使用 terminate() 或发送 CTRL_BREAK_EVENT
process.terminate()
else:
# macOS / Linux: 终止整个进程组
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
log(f"{name} 已停止", GREEN)
except Exception as e:
log(f"停止 {name} 时出错: {e}", RED)
# 强制杀死
try:
process.kill()
except Exception:
pass
def send_alert(content):
"""发送告警消息 (使用 curl)"""
try:
url = "https://zhiyinzhe.xyz/sendMsg/api/send"
cmd = [
"curl", "-X", "POST", url,
"-F", "bot_name=weibo",
"-F", "receive_id=on_f972f5f8c4de6ac967574b97dc8c1d82",
"-F", "receive_id_type=union_id",
"-F", f"content={content}"
]
# 使用 run 等待请求完成,确保告警发出
subprocess.run(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=10)
log(f"已发送告警: {content}", GREEN)
except Exception as e:
log(f"发送告警失败: {e}", RED)
def main():
# 激活 Conda 环境
# 注意:在子进程中直接使用当前已激活的 python 解释器路径,避免 conda activate 问题
@@ -54,9 +147,32 @@ def main():
# 2. 启动自动交易程序
# 使用 -u 参数禁用输出缓冲
trade_cmd = f"{python_exec} -u auto_trade_polymarket.py"
trade_cmd = f"{python_exec} -u trader/auto_trade_polymarket.py"
trade_process = start_process(trade_cmd, "Polymarket 自动交易程序")
# 3. 启动每日数据报告推送程序(后台线程)
# 北京时间每天 8 点推送昨天的数据统计
try:
report_thread = run_scheduler_background()
if report_thread:
log("每日数据报告推送程序已启动", GREEN)
else:
log("⚠️ 每日数据报告推送程序启动失败", YELLOW)
except Exception as e:
log(f"⚠️ 启动每日报告程序异常: {e}", YELLOW)
report_thread = None
# 4. 启动 MySQL -> Cloudflare D1 定时备份(每小时 :30)
try:
d1_backup_thread = run_mysql_d1_backup_scheduler_background()
if d1_backup_thread:
log("MySQL -> D1 定时备份程序已启动(每小时 :30)", GREEN)
else:
log("D1 定时备份未启动(可能未启用或配置不完整)", YELLOW)
except Exception as e:
log(f"⚠️ 启动 D1 定时备份程序异常: {e}", YELLOW)
d1_backup_thread = None
log("系统启动完成!", GREEN)
log("前端访问地址: http://localhost:8000", GREEN)
log("按 Ctrl+C 停止所有服务...", YELLOW)
@@ -64,11 +180,15 @@ def main():
try:
# 监控子进程状态
while True:
if api_process.poll() is not None:
log("API 服务意外退出", RED)
if api_process and api_process.poll() is not None:
msg = "API 服务意外退出"
log(msg, RED)
send_alert(msg)
break
if trade_process.poll() is not None:
log("自动交易程序意外退出", RED)
if trade_process and trade_process.poll() is not None:
msg = "自动交易程序意外退出"
log(msg, RED)
send_alert(msg)
break
time.sleep(1)
@@ -76,10 +196,8 @@ def main():
log("\n正在停止服务...", YELLOW)
finally:
# 优雅退出
if api_process:
os.killpg(os.getpgid(api_process.pid), signal.SIGTERM)
if trade_process:
os.killpg(os.getpgid(trade_process.pid), signal.SIGTERM)
terminate_process(api_process, "API 服务")
terminate_process(trade_process, "自动交易程序")
log("所有服务已停止", GREEN)
File diff suppressed because it is too large. Load diff
+54
View File
@@ -9,5 +9,59 @@
"side": "DOWN",
"price": 0.505,
"probability": 0.5563886165618896
},
{
"timestamp": "2026-02-11T22:32:46.922211",
"success": false,
"error": "该市场已存在成功交易记录",
"prediction": "DOWN",
"probability": 0.0,
"market": "Bitcoin Up or Down - February 11, 9AM ET",
"market_slug": "bitcoin-up-or-down-february-11-9am-et"
},
{
"timestamp": "2026-02-11T23:18:40.579373",
"success": false,
"error": "该市场已存在成功交易记录",
"prediction": "UP",
"probability": 0.6666666666666666,
"market": "Bitcoin Up or Down - February 11, 10AM ET",
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
},
{
"timestamp": "2026-02-11T23:18:59.059130",
"success": false,
"error": "该市场已存在成功交易记录",
"prediction": "UP",
"probability": 0.6666666666666666,
"market": "Bitcoin Up or Down - February 11, 10AM ET",
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
},
{
"timestamp": "2026-02-11T23:19:50.797784",
"success": false,
"error": "该市场已存在成功交易记录",
"prediction": "UP",
"probability": 0.6666666666666666,
"market": "Bitcoin Up or Down - February 11, 10AM ET",
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
},
{
"timestamp": "2026-02-11T23:31:03.776709",
"success": false,
"error": "该市场已存在成功交易记录",
"prediction": "UP",
"probability": 0.6666666666666666,
"market": "Bitcoin Up or Down - February 11, 10AM ET",
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
},
{
"timestamp": "2026-02-11T23:35:39.565123",
"success": false,
"error": "该市场已存在成功交易记录",
"prediction": "UP",
"probability": 0.6666666666666666,
"market": "Bitcoin Up or Down - February 11, 10AM ET",
"market_slug": "bitcoin-up-or-down-february-11-10am-et"
}
]
+8 -3
View File
@@ -38,19 +38,24 @@ class TrainingConfig:
Returns:
Device string ('mps', 'cuda', or 'cpu')
Note:
macOS 使用 MPS,Windows/Linux 使用 CPU
"""
import torch
import platform
if self.device != 'auto':
return self.device
# Auto-detect best available device
# 根据操作系统选择设备:macOS 用 MPS,Windows/Linux 用 CPU
if platform.system() == 'Darwin': # macOS
if torch.backends.mps.is_available():
return 'mps'
elif torch.cuda.is_available():
return 'cuda'
else:
return 'cpu'
else: # Windows / Linux
return 'cpu'
if __name__ == "__main__":
+53
View File
@@ -0,0 +1,53 @@
"""统一网络代理配置工具。"""
import os
from typing import Optional
def get_proxy_url() -> Optional[str]:
"""从环境变量读取代理地址,统一兼容不同变量名。"""
use_proxy = os.getenv("USE_PROXY", "false").lower() == "true"
if not use_proxy:
return None
return (
os.getenv("SOCKS5_PROXY")
or os.getenv("ALL_PROXY")
or os.getenv("HTTPS_PROXY")
or os.getenv("HTTP_PROXY")
)
def get_send_msg_proxy_url() -> Optional[str]:
"""读取 send_msg 独立代理,未配置时回退到全局代理。"""
send_msg_proxy = os.getenv("SEND_MSG_PROXY")
if send_msg_proxy:
return send_msg_proxy
send_msg_proxy_port = os.getenv("SEND_MSG_PROXY_PORT")
if send_msg_proxy_port:
return f"socks5://127.0.0.1:{send_msg_proxy_port}"
return get_proxy_url()
def apply_global_proxy_from_env() -> Optional[str]:
"""将代理写入全局环境变量,确保 requests/httpx/curl 等默认走同一代理。"""
proxy_url = get_proxy_url()
if not proxy_url:
return None
os.environ["HTTP_PROXY"] = proxy_url
os.environ["HTTPS_PROXY"] = proxy_url
os.environ["ALL_PROXY"] = proxy_url
os.environ["http_proxy"] = proxy_url
os.environ["https_proxy"] = proxy_url
os.environ["all_proxy"] = proxy_url
# 保留本地直连,避免影响本机服务调用。
if not os.getenv("NO_PROXY"):
os.environ["NO_PROXY"] = "127.0.0.1,localhost"
if not os.getenv("no_proxy"):
os.environ["no_proxy"] = os.environ["NO_PROXY"]
return proxy_url
+40
View File
@@ -0,0 +1,40 @@
"""项目统一日志工具。"""
import logging
import os
from logging.handlers import RotatingFileHandler
def get_project_logger(name: str) -> logging.Logger:
"""返回统一配置的日志实例,避免重复添加处理器。"""
logger = logging.getLogger(name)
if logger.handlers:
return logger
os.makedirs("logs", exist_ok=True)
logger.setLevel(logging.INFO)
logger.propagate = False
formatter = logging.Formatter(
"%(asctime)s | %(name)s | %(levelname)s | %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
file_handler = RotatingFileHandler(
"logs/project.log",
maxBytes=10 * 1024 * 1024,
backupCount=5,
encoding="utf-8",
)
file_handler.setFormatter(formatter)
file_handler.setLevel(logging.INFO)
console_handler = logging.StreamHandler()
console_handler.setFormatter(formatter)
console_handler.setLevel(logging.INFO)
logger.addHandler(file_handler)
logger.addHandler(console_handler)
return logger
+54
View File
@@ -0,0 +1,54 @@
import requests
from utils.network_proxy import apply_global_proxy_from_env, get_send_msg_proxy_url
from utils.project_logger import get_project_logger
apply_global_proxy_from_env()
logger = get_project_logger("send_msg")
class LarkNotifier:
"""飞书消息发送工具类"""
def __init__(self, bot_name="weibo", receive_id="on_f972f5f8c4de6ac967574b97dc8c1d82"):
# 将基础配置放在初始化中,方便后续修改或通过环境变量传入
self.url = "https://zhiyinzhe.xyz/sendMsg/api/send"
self.bot_name = bot_name
self.receive_id = receive_id
# 使用 Session 可以复用底层 TCP 连接,提升发送效率
self.session = requests.Session()
proxy_url = get_send_msg_proxy_url()
if proxy_url:
self.session.proxies.update({
"http": proxy_url,
"https": proxy_url,
})
logger.info("send_msg 使用独立代理: %s", proxy_url)
else:
logger.info("send_msg 未启用代理")
def send_alert(self, title: str, content: str):
"""
发送通用文本告警
:param title: 告警标题 (如 "⚠️ 市场获取失败")
:param content: 告警具体内容 (如 "Slug: xxx \n 原因: xxx")
"""
payload = {
"bot_name": self.bot_name,
"receive_id": self.receive_id,
"receive_id_type": "union_id",
"content": f"{title}\n{content}"
}
try:
# 使用 data=payload 发送 Form Data
response = self.session.post(self.url, data=payload, timeout=5)
response.raise_for_status()
logger.info("告警发送成功: %s", title)
except Exception as e:
# 告警系统自身的错误只记录日志,绝不抛出异常影响主程序
logger.error("发送飞书告警失败: %s", e)
# 实例化一个全局单例供其他模块直接导入使用
lark_notifier = LarkNotifier()