feat: study modes (fast/parallel) and course-container completion fix
- settings: study.mode fast|normal and study.parallel 1-5 - normal mode: one headless browser per video, visibility keep-alive, auto-degrade when the platform enforces order - report study/log/detail/save before isComplete; drop lastProgressTime on completion - complete periodType 10/13 container nodes and recompute counters via PUT calFinalScore - capture mode: POST /api/capture records all XHR/fetch including new tabs - UI: study mode in settings, capture / stop-capture menu items
This commit is contained in:
study-tool
@@ -0,0 +1,24 @@
|
||||
# 构建上下文里不需要的东西。
|
||||
# data/ 必须排除：里面是本地账号库和 settings.json（含 api_key），
|
||||
# 运行期由 volume 挂进来，不能打进镜像层。
|
||||
|
||||
.git
|
||||
.gitignore
|
||||
__pycache__/
|
||||
**/__pycache__/
|
||||
*.py[cod]
|
||||
*.pyo
|
||||
.venv
|
||||
venv/
|
||||
env/
|
||||
.idea/
|
||||
.vscode/
|
||||
data/
|
||||
*.db
|
||||
*.db-wal
|
||||
*.db-shm
|
||||
*.log
|
||||
*.imported
|
||||
.env
|
||||
docker-compose.override.yml
|
||||
node_modules/
|
||||
@@ -0,0 +1,18 @@
|
||||
# 复制成 .env 后按需修改：cp .env.example .env
|
||||
# docker compose 会自动读取同目录的 .env
|
||||
|
||||
# 远程访问令牌：非空时，页面和所有接口都要带 ?token=<值>（首次带参数访问会记住 cookie）。
|
||||
# 生成一个：python -c "import secrets;print(secrets.token_urlsafe(24))"
|
||||
AUTH_TOKEN=
|
||||
|
||||
# 监听端口（宿主机）
|
||||
PORT=8848
|
||||
|
||||
# 绑定地址：0.0.0.0 = 所有网卡（远程管理）；127.0.0.1 = 只允许本机 + 反代转发
|
||||
BIND_ADDR=0.0.0.0
|
||||
|
||||
# 每个账号保留的运行日志行数 / 课程页快照条数
|
||||
RUN_LOG_KEEP=2000
|
||||
SNAPSHOT_KEEP=200
|
||||
|
||||
TZ=Asia/Shanghai
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
# 运行期数据：SQLite 库（含明文账号密码）、导入备份、调试快照
|
||||
data/
|
||||
*.db
|
||||
*.db-wal
|
||||
*.db-shm
|
||||
*.imported
|
||||
|
||||
# 本地环境
|
||||
.env
|
||||
.venv/
|
||||
venv/
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
.playwright-mcp/
|
||||
node_modules/
|
||||
@@ -0,0 +1,38 @@
|
||||
# AGENTS.md
|
||||
|
||||
本文档用于约束本项目中的 AI / 自动化开发行为。开发时优先遵循本文件，其次遵循用户当前消息。
|
||||
|
||||
## 基本原则
|
||||
|
||||
- 先读现有代码，再动手修改，优先沿用项目已有结构和写法。
|
||||
- 写代码保持最少行数，能简单实现就不要引入复杂抽象。
|
||||
- 标准格式、协议、解析、压缩、加密、日期等通用能力优先使用成熟稳定的库，不要手写底层实现，除非用户明确要求或项目已有实现必须沿用。
|
||||
- 不要为了“兼容更多场景”写大量分支，只实现当前明确需要的功能。
|
||||
- 项目尚未上线，不需要兼容旧数据；表结构或字段调整时直接按新设计修改，不写旧字段兼容、数据迁移兜底或删除旧表的清理逻辑，除非用户明确要求。
|
||||
- 每次写完代码，不需要检查语法，不需要执行构建，用户会自己做。
|
||||
- 不要改无关文件，不要顺手重构。
|
||||
- 任何文件编辑、改动、删除或格式化前，必须先询问用户是否允许；只有得到明确同意后才能修改。
|
||||
- 如果工作区已有用户改动，不要回滚，不要覆盖；只在必要范围内追加修改。
|
||||
|
||||
## 反复提醒沉淀
|
||||
|
||||
- 如果开发过程中总是遇到某个问题，或者用户反复提醒同一个注意事项，需要把该注意事项补充到本文件。
|
||||
- 补充时写成明确、可执行的规则，避免只写模糊描述。
|
||||
- 新规则应放到最相关的章节；找不到合适章节时放到“项目注意事项”。
|
||||
|
||||
## 文档规范
|
||||
|
||||
- README 保持简洁，只放项目介绍、核心功能、快速开始和文档入口。
|
||||
- 后续待办写到 `docs/progress/todo.md`。
|
||||
- 已实现但还需要用户测试确认的事项写到 `OPTIMIZATION.md`。
|
||||
- 每次任务完成前，都要根据实际变更检查并更新 `OPTIMIZATION.md` 和 `OPTIMIZATION.md`；如果功能或待办没有变化，也要确认无需修改。
|
||||
- 文档不要写过期日期；除非用户明确要求记录具体时间。
|
||||
|
||||
## 发版本流程
|
||||
|
||||
- 发版本时，先把 `CHANGELOG.md` 的 `Unreleased` 变更整理成新的版本记录，并保留空的 `Unreleased` 标题。
|
||||
- 每次提交必须将当前版本号进行提升，更新 `metadata.yaml`里面的`version`中的版本号。
|
||||
- 将当前未提交的代码全部提交到 Git。
|
||||
- 提交完成后，给当前提交打最新版本号对应的 tag，例如 `v0.0.5`。
|
||||
- 发版本流程中不要执行编译、测试或构建，除非用户明确要求。
|
||||
- 如果触发重要的决策，你需要将代码提交到`git`中，并且将本次已经修改的内容进行总结
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
# 多账号学习控制台 —— 运行镜像
|
||||
# 构建: docker build -t study-tool:latest .
|
||||
# 瘦身: docker build --build-arg WITH_PLAYWRIGHT=0 -t study-tool:latest . （不要直链视频播放）
|
||||
FROM python:3.12-slim-bookworm
|
||||
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PIP_NO_CACHE_DIR=1 \
|
||||
PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
||||
DEBIAN_FRONTEND=noninteractive \
|
||||
TZ=Asia/Shanghai \
|
||||
HOST=0.0.0.0 \
|
||||
PORT=8848 \
|
||||
DATA_DIR=/app/data \
|
||||
DB_PATH=/app/data/study.db \
|
||||
PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# nodejs: desenc.js 要用（登录表单 rsa 字段走站点原版 DES 算法）
|
||||
# curl: 健康检查用
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends nodejs ca-certificates curl tzdata \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY requirements.txt ./
|
||||
RUN pip install -r requirements.txt
|
||||
|
||||
# 直链视频（fileType=other）必须用真浏览器播到结尾；不需要就 WITH_PLAYWRIGHT=0
|
||||
ARG WITH_PLAYWRIGHT=1
|
||||
RUN if [ "$WITH_PLAYWRIGHT" = "1" ]; then \
|
||||
playwright install --with-deps chromium && rm -rf /var/lib/apt/lists/* ; \
|
||||
fi
|
||||
|
||||
COPY . .
|
||||
RUN mkdir -p /app/data
|
||||
|
||||
EXPOSE 8848
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
|
||||
CMD curl -fsS "http://127.0.0.1:${PORT}/api/health" || exit 1
|
||||
|
||||
CMD ["python", "app.py"]
|
||||
@@ -0,0 +1,207 @@
|
||||
# 多账号学习控制台
|
||||
|
||||
东莞市教师培训平台（jspx.dgjy.net / study.dgjy.net）的多账号挂机控制台。账号、会话、课程缓存、运行日志、系统设置全部存在服务端 SQLite，浏览器只是界面，换电脑、远程打开都在。
|
||||
|
||||
## 启动
|
||||
|
||||
### 方式一：本机（Windows）
|
||||
|
||||
```bat
|
||||
start.bat
|
||||
```
|
||||
|
||||
或手动：
|
||||
|
||||
```bat
|
||||
pip install -r requirements.txt
|
||||
python app.py
|
||||
```
|
||||
|
||||
浏览器打开 <http://127.0.0.1:8848>（`localhost` 也可以，服务同时监听 `::1`）。
|
||||
|
||||
换端口 / 对外开放：
|
||||
|
||||
```bat
|
||||
set PORT=8849 && python app.py
|
||||
set HOST=0.0.0.0 && python app.py
|
||||
```
|
||||
|
||||
### 方式二：Docker（推荐用于远程管理）
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
# 编辑 .env：至少设一个 AUTH_TOKEN（远程管理必须有，否则谁都能读到里面的账号密码）
|
||||
# AUTH_TOKEN=$(python -c "import secrets;print(secrets.token_urlsafe(24))")
|
||||
|
||||
docker compose up -d --build
|
||||
docker compose logs -f # 看启动日志
|
||||
```
|
||||
|
||||
打开 `http://<服务器IP>:8848/?token=<AUTH_TOKEN>`，令牌会写进 cookie，之后正常用。
|
||||
|
||||
| 操作 | 命令 |
|
||||
| --- | --- |
|
||||
| 起 / 重建 | `docker compose up -d --build` |
|
||||
| 停 | `docker compose down` |
|
||||
| 看日志 | `docker compose logs -f --tail=200` |
|
||||
| 备份数据 | `cp data/study.db data/study.db.bak`（服务运行中也可拷，WAL 模式） |
|
||||
| 进容器 | `docker compose exec study-tool bash` |
|
||||
| 换端口 | `.env` 里改 `PORT`，再 `docker compose up -d` |
|
||||
|
||||
环境变量：
|
||||
|
||||
| 变量 | 默认 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `HOST` | `127.0.0.1` | 监听地址；Docker 里是 `0.0.0.0` |
|
||||
| `PORT` | `8848` | 端口 |
|
||||
| `DB_PATH` | `<项目>/data/study.db` | SQLite 库文件 |
|
||||
| `AUTH_TOKEN` | 空 | 非空 = 页面/接口需要 `?token=` 或 cookie；`/api/health` 永远放行 |
|
||||
| `RUN_LOG_KEEP` | `2000` | 每个账号保留的日志行数 |
|
||||
| `SNAPSHOT_KEEP` | `200` | 课程页快照保留条数 |
|
||||
| `NODE_BIN` | 自动找 | node.exe 路径（算登录表单的 DES） |
|
||||
|
||||
镜像默认装 Chromium（直链视频要真播到结尾）。不需要就用
|
||||
`docker compose build --build-arg WITH_PLAYWRIGHT=0`，镜像能小一大截。
|
||||
|
||||
## 数据存储（SQLite3）
|
||||
|
||||
单文件 `data/study.db`（WAL 模式，读写不互斥）。表：
|
||||
|
||||
| 表 | 内容 |
|
||||
| --- | --- |
|
||||
| `accounts` | 账号 / 密码 / 会话 cookie / 课程缓存 / 勾选 / 自动交测验·反馈表开关 |
|
||||
| `settings` | 大模型配置（`base_url` / `api_key` / `model` / 温度 / 超时 / 提示词） |
|
||||
| `kv` | 通用缓存（例如 study 平台试出来的接口前缀、`study_video.py` 的会话） |
|
||||
| `login_log` | 每次登录尝试（平台原话，`/api/loginlog` 读它） |
|
||||
| `run_log` | 每个账号的运行日志（`/api/log/history?key=<id>&tail=200`） |
|
||||
| `run_state` | 运行状态快照，进程重启后 `/api/runs` 仍能看到，未跑完的标 `interrupted` |
|
||||
| `snapshots` | 站点改版时存下的课程页原始 HTML（`/api/snapshots`、`/api/snapshot/<id>`） |
|
||||
|
||||
旧的 `data/settings.json`、`data/logs/*.log`、`data/study_session.json` 在首次启动时自动导入，
|
||||
原文件改名为 `*.imported`（回滚旧版本时改名回去即可）。
|
||||
|
||||
密码是明文存在库里的——会话失效时后台要能自己重登。所以这个库等价于账号密码本：
|
||||
不要提交进 git，Docker 部署时务必设 `AUTH_TOKEN`，公网前面再加一层 HTTPS 反代。
|
||||
|
||||
## 用法
|
||||
|
||||
1. 右上角「添加账号」，填账号密码（存进服务端 SQLite）。
|
||||
2. 点「登录」→ 自动读 `/admin/main.action` 的「在学课程」。
|
||||
3. 勾选课程 → 「开始学习」→ 卡片底部「课程日志」展开看实时进度。
|
||||
4. 「自动交反馈表」默认开；「自动交测验」默认关（答题正确率无法保证）。
|
||||
|
||||
## 测验
|
||||
|
||||
卡片上「测验」→ 弹窗按表格列出题号 / 题型 / 题干 / 选项 →「AI 答题」按系统设置里的模型填答案 → 核对后「提交」。
|
||||
|
||||
- **老平台（jspx）**：`showTestQums.action` 取题，`endTest.action` 交卷，回分从 `doTest.action` 抓
|
||||
- **新平台（study.dgjy.net）**：
|
||||
- 取题 `GET /train/questions/front/testpaper/getById?testpaperResultId=…`
|
||||
- 还没生成答卷时先自动创建（不用再手工去页面点一次「开始考试」）：
|
||||
`PUT /train/questions/front/testpaper/start/do`
|
||||
`{"periodId": "<课时 id>", "trainCourseId": "<trainCourseId>"}`
|
||||
→ `data.testpaperResultId`
|
||||
- 交卷 `PUT /train/questions/front/result/submitExam`
|
||||
`{"resultId": "<resultId>", "resultItemList": [{"fileIds": [], "itemId": "<itemResultId>", "replay": [0,2] | null}]}`
|
||||
- `replay` 是**0 基选项下标**数组：历史答案会自动回填，提交时字母答案按选项顺序换成下标
|
||||
- 交卷后看答案 `GET /train/questions/front/testpaper/teacher/review?testpaperResultId=…`
|
||||
返回 `data.questions[].data[]` 里的 `answer`（标准答案）、`replay`（学生答案）、
|
||||
`answerScore`、`analysis`。只有测验配置 `examAfterWatchAnswer=1` 时才在页面展示
|
||||
- 重考：课程树里的 `allowExamCount` / `surplusExamCount` 是总次数 / 剩余次数
|
||||
（平台一般 3 次）。`surplusExamCount > 0` 时再调一次 `start/do` 会生成新答卷，
|
||||
不需要老师重置；次数用完平台会拒绝
|
||||
- 判断题（`JUDGMENT`）接口里没有选项（`metasData` 只有一个 null 占位），页面会补出
|
||||
`A 正确 / B 错误`，交卷时映射成 `[0]` / `[1]`；模型写「正确/对/√」也会收敛成 A、B
|
||||
- 题组结构 `data.questions[].data[]`，鉴权用 `Authorization: Bearer <Middle-Token>`
|
||||
|
||||
已交卷的测验会显示「已完成 · 得分 X · 及格 Y · 已通过」并禁用「AI 答题」「提交」，
|
||||
按钮文字变成「已交卷」、旁边写「已交卷 · 不可重复提交」——不会出现点了没反应的情况。
|
||||
如果后端发现平台已经收卷，会直接提示「已经交过卷了，不能重复提交」，不会偷偷覆盖原成绩。
|
||||
|
||||
## 系统设置
|
||||
|
||||
右上角「设置」→ 大模型：`base_url`（OpenAI 兼容，写到 `/v1` 为止）、`api_key`、`model`、
|
||||
温度、超时、答题提示词。配置存 SQLite 的 `settings` 表，`api_key` 不回显，留空表示不修改。
|
||||
「测试连接」发一条 8 token 的请求验证连通性。
|
||||
|
||||
## 目录
|
||||
|
||||
```
|
||||
app.py 服务端：登录链路 / 课程抓取 / 课时上报 / 测验 / 反馈表 / HTTP 接口
|
||||
store.py SQLite3 存储层：账号 / 设置 / 登录记录 / 运行日志 / 状态 / 缓存 / 快照
|
||||
static/index.html 前端：账号卡片 + 卡片内「课程日志」折叠面板（无框架，原生 JS）
|
||||
des.js desenc.js DES 加密（登录表单 rsa 字段，站点原版算法）
|
||||
start.bat 一键启动（Windows）
|
||||
Dockerfile Docker 镜像（python:3.12-slim + nodejs + chromium）
|
||||
docker-compose.yml Docker 编排：端口、数据卷、AUTH_TOKEN、健康检查
|
||||
.env.example 远程管理用的环境变量样例（复制成 .env）
|
||||
requirements.txt 依赖
|
||||
data/study.db SQLite 库（账号/设置/日志/快照）
|
||||
study_video.py 独立命令行工具：按 trainCourseId 直接刷视频课时（会话也存 study.db）
|
||||
dgjy-toolkit.user.js 浏览器油猴脚本（独立使用，与网页端无关）
|
||||
```
|
||||
|
||||
## 登录链路
|
||||
|
||||
```
|
||||
GET tyyh.dgjy.net/tpass/login 取 lt / execution
|
||||
POST tyyh.dgjy.net/tpass/login un pd ul pl rsa(des) lt
|
||||
→ CASTGC
|
||||
GET dj.dgjy.net/.../authorize 302 + JS 中转页
|
||||
GET passport.dgjy.net/scriptLogin JS 返回 verifyURL
|
||||
GET dj.dgjy.net/fpf/login/verify.action 302
|
||||
GET jspx.dgjy.net/tstudy/passport/login.action 下发 jspx JSESSIONID
|
||||
GET jspx.dgjy.net/admin/main.action table-results 课程表
|
||||
```
|
||||
|
||||
`requests` 不执行 JS，中间两跳是手工解析 `scriptLogin.src` 和 `verifyURL` 走完的。
|
||||
|
||||
新版平台（慕课）另走一条 OAuth 码换 token 的链，拿到 `study.dgjy.net` 的 `Middle-Token`，见 `ensure_study_session()`。
|
||||
|
||||
## 接口
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/` | 控制台页面 |
|
||||
| GET | `/api/health` | 存活探测：pid / port / runs / SQLite 路径与行数（不需要 token） |
|
||||
| GET | `/api/accounts` | 账号列表（会话、课程缓存、勾选，**不含密码**） |
|
||||
| POST | `/api/accounts` | 新增/更新账号；`password` 不传 = 保留库里原密码 |
|
||||
| GET | `/api/accounts/{id}` | 单个账号 |
|
||||
| DELETE | `/api/accounts/{id}` | 删除账号（连带清掉它的运行状态与日志） |
|
||||
| POST | `/api/login` | `{account_id}`（密码从库里取）或 `{username,password}` → cookies + 课程列表 |
|
||||
| POST | `/api/courses` | 用已有 cookies 刷新课程 |
|
||||
| POST | `/api/discover` | 按 planId/projectId 查课程详情 |
|
||||
| POST | `/api/study` | 开始挂机 |
|
||||
| POST | `/api/stop` | 停止 |
|
||||
| POST | `/api/quiz` | 取一次测验的题目与选项 |
|
||||
| POST | `/api/quiz/answer` | 把题目交给大模型，返回 `{题号: 答案}` |
|
||||
| POST | `/api/quiz/submit` | 按题号合并答案后交卷 |
|
||||
| GET/POST | `/api/settings` | 读取 / 保存大模型配置（key 脱敏） |
|
||||
| POST | `/api/llm/test` | 测试模型连通性 |
|
||||
| GET | `/api/log` | 增量日志 `?key=&since=`（内存里的实时日志） |
|
||||
| GET | `/api/log/history` | SQLite 里的历史日志 `?key=&tail=`；不带 key 列出各账号行数 |
|
||||
| DELETE | `/api/log/history` | 清掉某个账号的历史日志 `?key=` |
|
||||
| GET | `/api/runs` | 所有账号运行状态（重启后从 SQLite 恢复） |
|
||||
| GET | `/api/loginlog` | 登录尝试记录 `?tail=` |
|
||||
| GET | `/api/snapshots` | 课程页快照列表 |
|
||||
| GET | `/api/snapshot/{id}` | 取回某份快照的原始 HTML |
|
||||
|
||||
远程调用时（设了 `AUTH_TOKEN`）带上 `-H "X-Auth-Token: $TOKEN"`：
|
||||
|
||||
```bash
|
||||
curl -H "X-Auth-Token: $TOKEN" http://host:8848/api/health
|
||||
curl -X POST -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/json" \
|
||||
-d '{"account_id":"a1"}' http://host:8848/api/login
|
||||
```
|
||||
|
||||
## 排查
|
||||
|
||||
登录失败先看 `data/study.db` 的 `login_log` 表（或 `GET /api/loginlog`）最后一行：
|
||||
|
||||
- `您还剩N次机会…` — 平台判密码错，跟链路无关
|
||||
- `SSO 校验未通过` / `jspx 没有建立会话` — 授权链断了，站点改版
|
||||
- `Incoming markup is of an invalid type` — 代码把布尔当 HTML 用了
|
||||
- 页面能开、接口全 `ERR_CONNECTION_REFUSED` — 服务没起或端口被占，`netstat -ano | findstr :8848`
|
||||
- 容器里起来了但连不上 — `docker compose logs` 看是否 `Address already in use`，`.env` 换 `PORT`
|
||||
- 页面 401 / 空白提示需要访问令牌 — 用 `/?token=<AUTH_TOKEN>` 打开一次
|
||||
|
||||
@@ -0,0 +1,867 @@
|
||||
/**
|
||||
* DES加密解密
|
||||
* @Copyright Copyright (c) 2006
|
||||
* @author Guapo
|
||||
* @see DESCore
|
||||
*/
|
||||
|
||||
/*
|
||||
* encrypt the string to string made up of hex
|
||||
* return the encrypted string
|
||||
*/
|
||||
function strEnc(data,firstKey,secondKey,thirdKey){
|
||||
|
||||
var leng = data.length;
|
||||
var encData = "";
|
||||
var firstKeyBt,secondKeyBt,thirdKeyBt,firstLength,secondLength,thirdLength;
|
||||
if(firstKey != null && firstKey != ""){
|
||||
firstKeyBt = getKeyBytes(firstKey);
|
||||
firstLength = firstKeyBt.length;
|
||||
}
|
||||
if(secondKey != null && secondKey != ""){
|
||||
secondKeyBt = getKeyBytes(secondKey);
|
||||
secondLength = secondKeyBt.length;
|
||||
}
|
||||
if(thirdKey != null && thirdKey != ""){
|
||||
thirdKeyBt = getKeyBytes(thirdKey);
|
||||
thirdLength = thirdKeyBt.length;
|
||||
}
|
||||
|
||||
if(leng > 0){
|
||||
if(leng < 4){
|
||||
var bt = strToBt(data);
|
||||
var encByte ;
|
||||
if(firstKey != null && firstKey !="" && secondKey != null && secondKey != "" && thirdKey != null && thirdKey != ""){
|
||||
var tempBt;
|
||||
var x,y,z;
|
||||
tempBt = bt;
|
||||
for(x = 0;x < firstLength ;x ++){
|
||||
tempBt = enc(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
for(y = 0;y < secondLength ;y ++){
|
||||
tempBt = enc(tempBt,secondKeyBt[y]);
|
||||
}
|
||||
for(z = 0;z < thirdLength ;z ++){
|
||||
tempBt = enc(tempBt,thirdKeyBt[z]);
|
||||
}
|
||||
encByte = tempBt;
|
||||
}else{
|
||||
if(firstKey != null && firstKey !="" && secondKey != null && secondKey != ""){
|
||||
var tempBt;
|
||||
var x,y;
|
||||
tempBt = bt;
|
||||
for(x = 0;x < firstLength ;x ++){
|
||||
tempBt = enc(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
for(y = 0;y < secondLength ;y ++){
|
||||
tempBt = enc(tempBt,secondKeyBt[y]);
|
||||
}
|
||||
encByte = tempBt;
|
||||
}else{
|
||||
if(firstKey != null && firstKey !=""){
|
||||
var tempBt;
|
||||
var x = 0;
|
||||
tempBt = bt;
|
||||
for(x = 0;x < firstLength ;x ++){
|
||||
tempBt = enc(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
encByte = tempBt;
|
||||
}
|
||||
}
|
||||
}
|
||||
encData = bt64ToHex(encByte);
|
||||
}else{
|
||||
var iterator = parseInt(leng/4);
|
||||
var remainder = leng%4;
|
||||
var i=0;
|
||||
for(i = 0;i < iterator;i++){
|
||||
var tempData = data.substring(i*4+0,i*4+4);
|
||||
var tempByte = strToBt(tempData);
|
||||
var encByte ;
|
||||
if(firstKey != null && firstKey !="" && secondKey != null && secondKey != "" && thirdKey != null && thirdKey != ""){
|
||||
var tempBt;
|
||||
var x,y,z;
|
||||
tempBt = tempByte;
|
||||
for(x = 0;x < firstLength ;x ++){
|
||||
tempBt = enc(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
for(y = 0;y < secondLength ;y ++){
|
||||
tempBt = enc(tempBt,secondKeyBt[y]);
|
||||
}
|
||||
for(z = 0;z < thirdLength ;z ++){
|
||||
tempBt = enc(tempBt,thirdKeyBt[z]);
|
||||
}
|
||||
encByte = tempBt;
|
||||
}else{
|
||||
if(firstKey != null && firstKey !="" && secondKey != null && secondKey != ""){
|
||||
var tempBt;
|
||||
var x,y;
|
||||
tempBt = tempByte;
|
||||
for(x = 0;x < firstLength ;x ++){
|
||||
tempBt = enc(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
for(y = 0;y < secondLength ;y ++){
|
||||
tempBt = enc(tempBt,secondKeyBt[y]);
|
||||
}
|
||||
encByte = tempBt;
|
||||
}else{
|
||||
if(firstKey != null && firstKey !=""){
|
||||
var tempBt;
|
||||
var x;
|
||||
tempBt = tempByte;
|
||||
for(x = 0;x < firstLength ;x ++){
|
||||
tempBt = enc(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
encByte = tempBt;
|
||||
}
|
||||
}
|
||||
}
|
||||
encData += bt64ToHex(encByte);
|
||||
}
|
||||
if(remainder > 0){
|
||||
var remainderData = data.substring(iterator*4+0,leng);
|
||||
var tempByte = strToBt(remainderData);
|
||||
var encByte ;
|
||||
if(firstKey != null && firstKey !="" && secondKey != null && secondKey != "" && thirdKey != null && thirdKey != ""){
|
||||
var tempBt;
|
||||
var x,y,z;
|
||||
tempBt = tempByte;
|
||||
for(x = 0;x < firstLength ;x ++){
|
||||
tempBt = enc(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
for(y = 0;y < secondLength ;y ++){
|
||||
tempBt = enc(tempBt,secondKeyBt[y]);
|
||||
}
|
||||
for(z = 0;z < thirdLength ;z ++){
|
||||
tempBt = enc(tempBt,thirdKeyBt[z]);
|
||||
}
|
||||
encByte = tempBt;
|
||||
}else{
|
||||
if(firstKey != null && firstKey !="" && secondKey != null && secondKey != ""){
|
||||
var tempBt;
|
||||
var x,y;
|
||||
tempBt = tempByte;
|
||||
for(x = 0;x < firstLength ;x ++){
|
||||
tempBt = enc(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
for(y = 0;y < secondLength ;y ++){
|
||||
tempBt = enc(tempBt,secondKeyBt[y]);
|
||||
}
|
||||
encByte = tempBt;
|
||||
}else{
|
||||
if(firstKey != null && firstKey !=""){
|
||||
var tempBt;
|
||||
var x;
|
||||
tempBt = tempByte;
|
||||
for(x = 0;x < firstLength ;x ++){
|
||||
tempBt = enc(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
encByte = tempBt;
|
||||
}
|
||||
}
|
||||
}
|
||||
encData += bt64ToHex(encByte);
|
||||
}
|
||||
}
|
||||
}
|
||||
return encData;
|
||||
}
|
||||
|
||||
/*
|
||||
* decrypt the encrypted string to the original string
|
||||
*
|
||||
* return the original string
|
||||
*/
|
||||
function strDec(data,firstKey,secondKey,thirdKey){
|
||||
var leng = data.length;
|
||||
var decStr = "";
|
||||
var firstKeyBt,secondKeyBt,thirdKeyBt,firstLength,secondLength,thirdLength;
|
||||
if(firstKey != null && firstKey != ""){
|
||||
firstKeyBt = getKeyBytes(firstKey);
|
||||
firstLength = firstKeyBt.length;
|
||||
}
|
||||
if(secondKey != null && secondKey != ""){
|
||||
secondKeyBt = getKeyBytes(secondKey);
|
||||
secondLength = secondKeyBt.length;
|
||||
}
|
||||
if(thirdKey != null && thirdKey != ""){
|
||||
thirdKeyBt = getKeyBytes(thirdKey);
|
||||
thirdLength = thirdKeyBt.length;
|
||||
}
|
||||
|
||||
var iterator = parseInt(leng/16);
|
||||
var i=0;
|
||||
for(i = 0;i < iterator;i++){
|
||||
var tempData = data.substring(i*16+0,i*16+16);
|
||||
var strByte = hexToBt64(tempData);
|
||||
var intByte = new Array(64);
|
||||
var j = 0;
|
||||
for(j = 0;j < 64; j++){
|
||||
intByte[j] = parseInt(strByte.substring(j,j+1));
|
||||
}
|
||||
var decByte;
|
||||
if(firstKey != null && firstKey !="" && secondKey != null && secondKey != "" && thirdKey != null && thirdKey != ""){
|
||||
var tempBt;
|
||||
var x,y,z;
|
||||
tempBt = intByte;
|
||||
for(x = thirdLength - 1;x >= 0;x --){
|
||||
tempBt = dec(tempBt,thirdKeyBt[x]);
|
||||
}
|
||||
for(y = secondLength - 1;y >= 0;y --){
|
||||
tempBt = dec(tempBt,secondKeyBt[y]);
|
||||
}
|
||||
for(z = firstLength - 1;z >= 0 ;z --){
|
||||
tempBt = dec(tempBt,firstKeyBt[z]);
|
||||
}
|
||||
decByte = tempBt;
|
||||
}else{
|
||||
if(firstKey != null && firstKey !="" && secondKey != null && secondKey != ""){
|
||||
var tempBt;
|
||||
var x,y,z;
|
||||
tempBt = intByte;
|
||||
for(x = secondLength - 1;x >= 0 ;x --){
|
||||
tempBt = dec(tempBt,secondKeyBt[x]);
|
||||
}
|
||||
for(y = firstLength - 1;y >= 0 ;y --){
|
||||
tempBt = dec(tempBt,firstKeyBt[y]);
|
||||
}
|
||||
decByte = tempBt;
|
||||
}else{
|
||||
if(firstKey != null && firstKey !=""){
|
||||
var tempBt;
|
||||
var x,y,z;
|
||||
tempBt = intByte;
|
||||
for(x = firstLength - 1;x >= 0 ;x --){
|
||||
tempBt = dec(tempBt,firstKeyBt[x]);
|
||||
}
|
||||
decByte = tempBt;
|
||||
}
|
||||
}
|
||||
}
|
||||
decStr += byteToString(decByte);
|
||||
}
|
||||
return decStr;
|
||||
}
|
||||
/*
|
||||
* chang the string into the bit array
|
||||
*
|
||||
* return bit array(it's length % 64 = 0)
|
||||
*/
|
||||
function getKeyBytes(key){
|
||||
var keyBytes = new Array();
|
||||
var leng = key.length;
|
||||
var iterator = parseInt(leng/4);
|
||||
var remainder = leng%4;
|
||||
var i = 0;
|
||||
for(i = 0;i < iterator; i ++){
|
||||
keyBytes[i] = strToBt(key.substring(i*4+0,i*4+4));
|
||||
}
|
||||
if(remainder > 0){
|
||||
keyBytes[i] = strToBt(key.substring(i*4+0,leng));
|
||||
}
|
||||
return keyBytes;
|
||||
}
|
||||
|
||||
/*
|
||||
* chang the string(it's length <= 4) into the bit array
|
||||
*
|
||||
* return bit array(it's length = 64)
|
||||
*/
|
||||
function strToBt(str){
|
||||
var leng = str.length;
|
||||
var bt = new Array(64);
|
||||
if(leng < 4){
|
||||
var i=0,j=0,p=0,q=0;
|
||||
for(i = 0;i<leng;i++){
|
||||
var k = str.charCodeAt(i);
|
||||
for(j=0;j<16;j++){
|
||||
var pow=1,m=0;
|
||||
for(m=15;m>j;m--){
|
||||
pow *= 2;
|
||||
}
|
||||
bt[16*i+j]=parseInt(k/pow)%2;
|
||||
}
|
||||
}
|
||||
for(p = leng;p<4;p++){
|
||||
var k = 0;
|
||||
for(q=0;q<16;q++){
|
||||
var pow=1,m=0;
|
||||
for(m=15;m>q;m--){
|
||||
pow *= 2;
|
||||
}
|
||||
bt[16*p+q]=parseInt(k/pow)%2;
|
||||
}
|
||||
}
|
||||
}else{
|
||||
for(i = 0;i<4;i++){
|
||||
var k = str.charCodeAt(i);
|
||||
for(j=0;j<16;j++){
|
||||
var pow=1;
|
||||
for(m=15;m>j;m--){
|
||||
pow *= 2;
|
||||
}
|
||||
bt[16*i+j]=parseInt(k/pow)%2;
|
||||
}
|
||||
}
|
||||
}
|
||||
return bt;
|
||||
}
|
||||
|
||||
/*
|
||||
* chang the bit(it's length = 4) into the hex
|
||||
*
|
||||
* return hex
|
||||
*/
|
||||
function bt4ToHex(binary) {
|
||||
var hex;
|
||||
switch (binary) {
|
||||
case "0000" : hex = "0"; break;
|
||||
case "0001" : hex = "1"; break;
|
||||
case "0010" : hex = "2"; break;
|
||||
case "0011" : hex = "3"; break;
|
||||
case "0100" : hex = "4"; break;
|
||||
case "0101" : hex = "5"; break;
|
||||
case "0110" : hex = "6"; break;
|
||||
case "0111" : hex = "7"; break;
|
||||
case "1000" : hex = "8"; break;
|
||||
case "1001" : hex = "9"; break;
|
||||
case "1010" : hex = "A"; break;
|
||||
case "1011" : hex = "B"; break;
|
||||
case "1100" : hex = "C"; break;
|
||||
case "1101" : hex = "D"; break;
|
||||
case "1110" : hex = "E"; break;
|
||||
case "1111" : hex = "F"; break;
|
||||
}
|
||||
return hex;
|
||||
}
|
||||
|
||||
/*
|
||||
* chang the hex into the bit(it's length = 4)
|
||||
*
|
||||
* return the bit(it's length = 4)
|
||||
*/
|
||||
function hexToBt4(hex) {
|
||||
var binary;
|
||||
switch (hex) {
|
||||
case "0" : binary = "0000"; break;
|
||||
case "1" : binary = "0001"; break;
|
||||
case "2" : binary = "0010"; break;
|
||||
case "3" : binary = "0011"; break;
|
||||
case "4" : binary = "0100"; break;
|
||||
case "5" : binary = "0101"; break;
|
||||
case "6" : binary = "0110"; break;
|
||||
case "7" : binary = "0111"; break;
|
||||
case "8" : binary = "1000"; break;
|
||||
case "9" : binary = "1001"; break;
|
||||
case "A" : binary = "1010"; break;
|
||||
case "B" : binary = "1011"; break;
|
||||
case "C" : binary = "1100"; break;
|
||||
case "D" : binary = "1101"; break;
|
||||
case "E" : binary = "1110"; break;
|
||||
case "F" : binary = "1111"; break;
|
||||
}
|
||||
return binary;
|
||||
}
|
||||
|
||||
/*
|
||||
* chang the bit(it's length = 64) into the string
|
||||
*
|
||||
* return string
|
||||
*/
|
||||
function byteToString(byteData){
|
||||
var str="";
|
||||
for(i = 0;i<4;i++){
|
||||
var count=0;
|
||||
for(j=0;j<16;j++){
|
||||
var pow=1;
|
||||
for(m=15;m>j;m--){
|
||||
pow*=2;
|
||||
}
|
||||
count+=byteData[16*i+j]*pow;
|
||||
}
|
||||
if(count != 0){
|
||||
str+=String.fromCharCode(count);
|
||||
}
|
||||
}
|
||||
return str;
|
||||
}
|
||||
|
||||
function bt64ToHex(byteData){
|
||||
var hex = "";
|
||||
for(i = 0;i<16;i++){
|
||||
var bt = "";
|
||||
for(j=0;j<4;j++){
|
||||
bt += byteData[i*4+j];
|
||||
}
|
||||
hex+=bt4ToHex(bt);
|
||||
}
|
||||
return hex;
|
||||
}
|
||||
|
||||
function hexToBt64(hex){
|
||||
var binary = "";
|
||||
for(i = 0;i<16;i++){
|
||||
binary+=hexToBt4(hex.substring(i,i+1));
|
||||
}
|
||||
return binary;
|
||||
}
|
||||
|
||||
/*
|
||||
* the 64 bit des core arithmetic
|
||||
*/
|
||||
|
||||
function enc(dataByte,keyByte){
|
||||
var keys = generateKeys(keyByte);
|
||||
var ipByte = initPermute(dataByte);
|
||||
var ipLeft = new Array(32);
|
||||
var ipRight = new Array(32);
|
||||
var tempLeft = new Array(32);
|
||||
var i = 0,j = 0,k = 0,m = 0, n = 0;
|
||||
for(k = 0;k < 32;k ++){
|
||||
ipLeft[k] = ipByte[k];
|
||||
ipRight[k] = ipByte[32+k];
|
||||
}
|
||||
for(i = 0;i < 16;i ++){
|
||||
for(j = 0;j < 32;j ++){
|
||||
tempLeft[j] = ipLeft[j];
|
||||
ipLeft[j] = ipRight[j];
|
||||
}
|
||||
var key = new Array(48);
|
||||
for(m = 0;m < 48;m ++){
|
||||
key[m] = keys[i][m];
|
||||
}
|
||||
var tempRight = xor(pPermute(sBoxPermute(xor(expandPermute(ipRight),key))), tempLeft);
|
||||
for(n = 0;n < 32;n ++){
|
||||
ipRight[n] = tempRight[n];
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
var finalData =new Array(64);
|
||||
for(i = 0;i < 32;i ++){
|
||||
finalData[i] = ipRight[i];
|
||||
finalData[32+i] = ipLeft[i];
|
||||
}
|
||||
return finallyPermute(finalData);
|
||||
}
|
||||
|
||||
function dec(dataByte,keyByte){
|
||||
var keys = generateKeys(keyByte);
|
||||
var ipByte = initPermute(dataByte);
|
||||
var ipLeft = new Array(32);
|
||||
var ipRight = new Array(32);
|
||||
var tempLeft = new Array(32);
|
||||
var i = 0,j = 0,k = 0,m = 0, n = 0;
|
||||
for(k = 0;k < 32;k ++){
|
||||
ipLeft[k] = ipByte[k];
|
||||
ipRight[k] = ipByte[32+k];
|
||||
}
|
||||
for(i = 15;i >= 0;i --){
|
||||
for(j = 0;j < 32;j ++){
|
||||
tempLeft[j] = ipLeft[j];
|
||||
ipLeft[j] = ipRight[j];
|
||||
}
|
||||
var key = new Array(48);
|
||||
for(m = 0;m < 48;m ++){
|
||||
key[m] = keys[i][m];
|
||||
}
|
||||
|
||||
var tempRight = xor(pPermute(sBoxPermute(xor(expandPermute(ipRight),key))), tempLeft);
|
||||
for(n = 0;n < 32;n ++){
|
||||
ipRight[n] = tempRight[n];
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
var finalData =new Array(64);
|
||||
for(i = 0;i < 32;i ++){
|
||||
finalData[i] = ipRight[i];
|
||||
finalData[32+i] = ipLeft[i];
|
||||
}
|
||||
return finallyPermute(finalData);
|
||||
}
|
||||
|
||||
function initPermute(originalData){
|
||||
var ipByte = new Array(64);
|
||||
for (i = 0, m = 1, n = 0; i < 4; i++, m += 2, n += 2) {
|
||||
for (j = 7, k = 0; j >= 0; j--, k++) {
|
||||
ipByte[i * 8 + k] = originalData[j * 8 + m];
|
||||
ipByte[i * 8 + k + 32] = originalData[j * 8 + n];
|
||||
}
|
||||
}
|
||||
return ipByte;
|
||||
}
|
||||
|
||||
function expandPermute(rightData){
|
||||
var epByte = new Array(48);
|
||||
for (i = 0; i < 8; i++) {
|
||||
if (i == 0) {
|
||||
epByte[i * 6 + 0] = rightData[31];
|
||||
} else {
|
||||
epByte[i * 6 + 0] = rightData[i * 4 - 1];
|
||||
}
|
||||
epByte[i * 6 + 1] = rightData[i * 4 + 0];
|
||||
epByte[i * 6 + 2] = rightData[i * 4 + 1];
|
||||
epByte[i * 6 + 3] = rightData[i * 4 + 2];
|
||||
epByte[i * 6 + 4] = rightData[i * 4 + 3];
|
||||
if (i == 7) {
|
||||
epByte[i * 6 + 5] = rightData[0];
|
||||
} else {
|
||||
epByte[i * 6 + 5] = rightData[i * 4 + 4];
|
||||
}
|
||||
}
|
||||
return epByte;
|
||||
}
|
||||
|
||||
function xor(byteOne,byteTwo){
|
||||
var xorByte = new Array(byteOne.length);
|
||||
for(i = 0;i < byteOne.length; i ++){
|
||||
xorByte[i] = byteOne[i] ^ byteTwo[i];
|
||||
}
|
||||
return xorByte;
|
||||
}
|
||||
|
||||
function sBoxPermute(expandByte){
|
||||
|
||||
var sBoxByte = new Array(32);
|
||||
var binary = "";
|
||||
var s1 = [
|
||||
[14, 4, 13, 1, 2, 15, 11, 8, 3, 10, 6, 12, 5, 9, 0, 7],
|
||||
[0, 15, 7, 4, 14, 2, 13, 1, 10, 6, 12, 11, 9, 5, 3, 8],
|
||||
[4, 1, 14, 8, 13, 6, 2, 11, 15, 12, 9, 7, 3, 10, 5, 0],
|
||||
[15, 12, 8, 2, 4, 9, 1, 7, 5, 11, 3, 14, 10, 0, 6, 13 ]];
|
||||
|
||||
/* Table - s2 */
|
||||
var s2 = [
|
||||
[15, 1, 8, 14, 6, 11, 3, 4, 9, 7, 2, 13, 12, 0, 5, 10],
|
||||
[3, 13, 4, 7, 15, 2, 8, 14, 12, 0, 1, 10, 6, 9, 11, 5],
|
||||
[0, 14, 7, 11, 10, 4, 13, 1, 5, 8, 12, 6, 9, 3, 2, 15],
|
||||
[13, 8, 10, 1, 3, 15, 4, 2, 11, 6, 7, 12, 0, 5, 14, 9 ]];
|
||||
|
||||
/* Table - s3 */
|
||||
var s3= [
|
||||
[10, 0, 9, 14, 6, 3, 15, 5, 1, 13, 12, 7, 11, 4, 2, 8],
|
||||
[13, 7, 0, 9, 3, 4, 6, 10, 2, 8, 5, 14, 12, 11, 15, 1],
|
||||
[13, 6, 4, 9, 8, 15, 3, 0, 11, 1, 2, 12, 5, 10, 14, 7],
|
||||
[1, 10, 13, 0, 6, 9, 8, 7, 4, 15, 14, 3, 11, 5, 2, 12 ]];
|
||||
/* Table - s4 */
|
||||
var s4 = [
|
||||
[7, 13, 14, 3, 0, 6, 9, 10, 1, 2, 8, 5, 11, 12, 4, 15],
|
||||
[13, 8, 11, 5, 6, 15, 0, 3, 4, 7, 2, 12, 1, 10, 14, 9],
|
||||
[10, 6, 9, 0, 12, 11, 7, 13, 15, 1, 3, 14, 5, 2, 8, 4],
|
||||
[3, 15, 0, 6, 10, 1, 13, 8, 9, 4, 5, 11, 12, 7, 2, 14 ]];
|
||||
|
||||
/* Table - s5 */
|
||||
var s5 = [
|
||||
[2, 12, 4, 1, 7, 10, 11, 6, 8, 5, 3, 15, 13, 0, 14, 9],
|
||||
[14, 11, 2, 12, 4, 7, 13, 1, 5, 0, 15, 10, 3, 9, 8, 6],
|
||||
[4, 2, 1, 11, 10, 13, 7, 8, 15, 9, 12, 5, 6, 3, 0, 14],
|
||||
[11, 8, 12, 7, 1, 14, 2, 13, 6, 15, 0, 9, 10, 4, 5, 3 ]];
|
||||
|
||||
/* Table - s6 */
|
||||
var s6 = [
|
||||
[12, 1, 10, 15, 9, 2, 6, 8, 0, 13, 3, 4, 14, 7, 5, 11],
|
||||
[10, 15, 4, 2, 7, 12, 9, 5, 6, 1, 13, 14, 0, 11, 3, 8],
|
||||
[9, 14, 15, 5, 2, 8, 12, 3, 7, 0, 4, 10, 1, 13, 11, 6],
|
||||
[4, 3, 2, 12, 9, 5, 15, 10, 11, 14, 1, 7, 6, 0, 8, 13 ]];
|
||||
|
||||
/* Table - s7 */
|
||||
var s7 = [
|
||||
[4, 11, 2, 14, 15, 0, 8, 13, 3, 12, 9, 7, 5, 10, 6, 1],
|
||||
[13, 0, 11, 7, 4, 9, 1, 10, 14, 3, 5, 12, 2, 15, 8, 6],
|
||||
[1, 4, 11, 13, 12, 3, 7, 14, 10, 15, 6, 8, 0, 5, 9, 2],
|
||||
[6, 11, 13, 8, 1, 4, 10, 7, 9, 5, 0, 15, 14, 2, 3, 12]];
|
||||
|
||||
/* Table - s8 */
|
||||
var s8 = [
|
||||
[13, 2, 8, 4, 6, 15, 11, 1, 10, 9, 3, 14, 5, 0, 12, 7],
|
||||
[1, 15, 13, 8, 10, 3, 7, 4, 12, 5, 6, 11, 0, 14, 9, 2],
|
||||
[7, 11, 4, 1, 9, 12, 14, 2, 0, 6, 10, 13, 15, 3, 5, 8],
|
||||
[2, 1, 14, 7, 4, 10, 8, 13, 15, 12, 9, 0, 3, 5, 6, 11]];
|
||||
|
||||
for(m=0;m<8;m++){
|
||||
var i=0,j=0;
|
||||
i = expandByte[m*6+0]*2+expandByte[m*6+5];
|
||||
j = expandByte[m * 6 + 1] * 2 * 2 * 2
|
||||
+ expandByte[m * 6 + 2] * 2* 2
|
||||
+ expandByte[m * 6 + 3] * 2
|
||||
+ expandByte[m * 6 + 4];
|
||||
switch (m) {
|
||||
case 0 :
|
||||
binary = getBoxBinary(s1[i][j]);
|
||||
break;
|
||||
case 1 :
|
||||
binary = getBoxBinary(s2[i][j]);
|
||||
break;
|
||||
case 2 :
|
||||
binary = getBoxBinary(s3[i][j]);
|
||||
break;
|
||||
case 3 :
|
||||
binary = getBoxBinary(s4[i][j]);
|
||||
break;
|
||||
case 4 :
|
||||
binary = getBoxBinary(s5[i][j]);
|
||||
break;
|
||||
case 5 :
|
||||
binary = getBoxBinary(s6[i][j]);
|
||||
break;
|
||||
case 6 :
|
||||
binary = getBoxBinary(s7[i][j]);
|
||||
break;
|
||||
case 7 :
|
||||
binary = getBoxBinary(s8[i][j]);
|
||||
break;
|
||||
}
|
||||
sBoxByte[m*4+0] = parseInt(binary.substring(0,1));
|
||||
sBoxByte[m*4+1] = parseInt(binary.substring(1,2));
|
||||
sBoxByte[m*4+2] = parseInt(binary.substring(2,3));
|
||||
sBoxByte[m*4+3] = parseInt(binary.substring(3,4));
|
||||
}
|
||||
return sBoxByte;
|
||||
}
|
||||
|
||||
function pPermute(sBoxByte){
|
||||
var pBoxPermute = new Array(32);
|
||||
pBoxPermute[ 0] = sBoxByte[15];
|
||||
pBoxPermute[ 1] = sBoxByte[ 6];
|
||||
pBoxPermute[ 2] = sBoxByte[19];
|
||||
pBoxPermute[ 3] = sBoxByte[20];
|
||||
pBoxPermute[ 4] = sBoxByte[28];
|
||||
pBoxPermute[ 5] = sBoxByte[11];
|
||||
pBoxPermute[ 6] = sBoxByte[27];
|
||||
pBoxPermute[ 7] = sBoxByte[16];
|
||||
pBoxPermute[ 8] = sBoxByte[ 0];
|
||||
pBoxPermute[ 9] = sBoxByte[14];
|
||||
pBoxPermute[10] = sBoxByte[22];
|
||||
pBoxPermute[11] = sBoxByte[25];
|
||||
pBoxPermute[12] = sBoxByte[ 4];
|
||||
pBoxPermute[13] = sBoxByte[17];
|
||||
pBoxPermute[14] = sBoxByte[30];
|
||||
pBoxPermute[15] = sBoxByte[ 9];
|
||||
pBoxPermute[16] = sBoxByte[ 1];
|
||||
pBoxPermute[17] = sBoxByte[ 7];
|
||||
pBoxPermute[18] = sBoxByte[23];
|
||||
pBoxPermute[19] = sBoxByte[13];
|
||||
pBoxPermute[20] = sBoxByte[31];
|
||||
pBoxPermute[21] = sBoxByte[26];
|
||||
pBoxPermute[22] = sBoxByte[ 2];
|
||||
pBoxPermute[23] = sBoxByte[ 8];
|
||||
pBoxPermute[24] = sBoxByte[18];
|
||||
pBoxPermute[25] = sBoxByte[12];
|
||||
pBoxPermute[26] = sBoxByte[29];
|
||||
pBoxPermute[27] = sBoxByte[ 5];
|
||||
pBoxPermute[28] = sBoxByte[21];
|
||||
pBoxPermute[29] = sBoxByte[10];
|
||||
pBoxPermute[30] = sBoxByte[ 3];
|
||||
pBoxPermute[31] = sBoxByte[24];
|
||||
return pBoxPermute;
|
||||
}
|
||||
|
||||
function finallyPermute(endByte){
|
||||
var fpByte = new Array(64);
|
||||
fpByte[ 0] = endByte[39];
|
||||
fpByte[ 1] = endByte[ 7];
|
||||
fpByte[ 2] = endByte[47];
|
||||
fpByte[ 3] = endByte[15];
|
||||
fpByte[ 4] = endByte[55];
|
||||
fpByte[ 5] = endByte[23];
|
||||
fpByte[ 6] = endByte[63];
|
||||
fpByte[ 7] = endByte[31];
|
||||
fpByte[ 8] = endByte[38];
|
||||
fpByte[ 9] = endByte[ 6];
|
||||
fpByte[10] = endByte[46];
|
||||
fpByte[11] = endByte[14];
|
||||
fpByte[12] = endByte[54];
|
||||
fpByte[13] = endByte[22];
|
||||
fpByte[14] = endByte[62];
|
||||
fpByte[15] = endByte[30];
|
||||
fpByte[16] = endByte[37];
|
||||
fpByte[17] = endByte[ 5];
|
||||
fpByte[18] = endByte[45];
|
||||
fpByte[19] = endByte[13];
|
||||
fpByte[20] = endByte[53];
|
||||
fpByte[21] = endByte[21];
|
||||
fpByte[22] = endByte[61];
|
||||
fpByte[23] = endByte[29];
|
||||
fpByte[24] = endByte[36];
|
||||
fpByte[25] = endByte[ 4];
|
||||
fpByte[26] = endByte[44];
|
||||
fpByte[27] = endByte[12];
|
||||
fpByte[28] = endByte[52];
|
||||
fpByte[29] = endByte[20];
|
||||
fpByte[30] = endByte[60];
|
||||
fpByte[31] = endByte[28];
|
||||
fpByte[32] = endByte[35];
|
||||
fpByte[33] = endByte[ 3];
|
||||
fpByte[34] = endByte[43];
|
||||
fpByte[35] = endByte[11];
|
||||
fpByte[36] = endByte[51];
|
||||
fpByte[37] = endByte[19];
|
||||
fpByte[38] = endByte[59];
|
||||
fpByte[39] = endByte[27];
|
||||
fpByte[40] = endByte[34];
|
||||
fpByte[41] = endByte[ 2];
|
||||
fpByte[42] = endByte[42];
|
||||
fpByte[43] = endByte[10];
|
||||
fpByte[44] = endByte[50];
|
||||
fpByte[45] = endByte[18];
|
||||
fpByte[46] = endByte[58];
|
||||
fpByte[47] = endByte[26];
|
||||
fpByte[48] = endByte[33];
|
||||
fpByte[49] = endByte[ 1];
|
||||
fpByte[50] = endByte[41];
|
||||
fpByte[51] = endByte[ 9];
|
||||
fpByte[52] = endByte[49];
|
||||
fpByte[53] = endByte[17];
|
||||
fpByte[54] = endByte[57];
|
||||
fpByte[55] = endByte[25];
|
||||
fpByte[56] = endByte[32];
|
||||
fpByte[57] = endByte[ 0];
|
||||
fpByte[58] = endByte[40];
|
||||
fpByte[59] = endByte[ 8];
|
||||
fpByte[60] = endByte[48];
|
||||
fpByte[61] = endByte[16];
|
||||
fpByte[62] = endByte[56];
|
||||
fpByte[63] = endByte[24];
|
||||
return fpByte;
|
||||
}
|
||||
|
||||
function getBoxBinary(i) {
|
||||
var binary = "";
|
||||
switch (i) {
|
||||
case 0 :binary = "0000";break;
|
||||
case 1 :binary = "0001";break;
|
||||
case 2 :binary = "0010";break;
|
||||
case 3 :binary = "0011";break;
|
||||
case 4 :binary = "0100";break;
|
||||
case 5 :binary = "0101";break;
|
||||
case 6 :binary = "0110";break;
|
||||
case 7 :binary = "0111";break;
|
||||
case 8 :binary = "1000";break;
|
||||
case 9 :binary = "1001";break;
|
||||
case 10 :binary = "1010";break;
|
||||
case 11 :binary = "1011";break;
|
||||
case 12 :binary = "1100";break;
|
||||
case 13 :binary = "1101";break;
|
||||
case 14 :binary = "1110";break;
|
||||
case 15 :binary = "1111";break;
|
||||
}
|
||||
return binary;
|
||||
}
|
||||
/*
|
||||
* generate 16 keys for xor
|
||||
*
|
||||
*/
|
||||
function generateKeys(keyByte){
|
||||
var key = new Array(56);
|
||||
var keys = new Array();
|
||||
|
||||
keys[ 0] = new Array();
|
||||
keys[ 1] = new Array();
|
||||
keys[ 2] = new Array();
|
||||
keys[ 3] = new Array();
|
||||
keys[ 4] = new Array();
|
||||
keys[ 5] = new Array();
|
||||
keys[ 6] = new Array();
|
||||
keys[ 7] = new Array();
|
||||
keys[ 8] = new Array();
|
||||
keys[ 9] = new Array();
|
||||
keys[10] = new Array();
|
||||
keys[11] = new Array();
|
||||
keys[12] = new Array();
|
||||
keys[13] = new Array();
|
||||
keys[14] = new Array();
|
||||
keys[15] = new Array();
|
||||
var loop = [1,1,2,2,2,2,2,2,1,2,2,2,2,2,2,1];
|
||||
|
||||
for(i=0;i<7;i++){
|
||||
for(j=0,k=7;j<8;j++,k--){
|
||||
key[i*8+j]=keyByte[8*k+i];
|
||||
}
|
||||
}
|
||||
|
||||
var i = 0;
|
||||
for(i = 0;i < 16;i ++){
|
||||
var tempLeft=0;
|
||||
var tempRight=0;
|
||||
for(j = 0; j < loop[i];j ++){
|
||||
tempLeft = key[0];
|
||||
tempRight = key[28];
|
||||
for(k = 0;k < 27 ;k ++){
|
||||
key[k] = key[k+1];
|
||||
key[28+k] = key[29+k];
|
||||
}
|
||||
key[27]=tempLeft;
|
||||
key[55]=tempRight;
|
||||
}
|
||||
var tempKey = new Array(48);
|
||||
tempKey[ 0] = key[13];
|
||||
tempKey[ 1] = key[16];
|
||||
tempKey[ 2] = key[10];
|
||||
tempKey[ 3] = key[23];
|
||||
tempKey[ 4] = key[ 0];
|
||||
tempKey[ 5] = key[ 4];
|
||||
tempKey[ 6] = key[ 2];
|
||||
tempKey[ 7] = key[27];
|
||||
tempKey[ 8] = key[14];
|
||||
tempKey[ 9] = key[ 5];
|
||||
tempKey[10] = key[20];
|
||||
tempKey[11] = key[ 9];
|
||||
tempKey[12] = key[22];
|
||||
tempKey[13] = key[18];
|
||||
tempKey[14] = key[11];
|
||||
tempKey[15] = key[ 3];
|
||||
tempKey[16] = key[25];
|
||||
tempKey[17] = key[ 7];
|
||||
tempKey[18] = key[15];
|
||||
tempKey[19] = key[ 6];
|
||||
tempKey[20] = key[26];
|
||||
tempKey[21] = key[19];
|
||||
tempKey[22] = key[12];
|
||||
tempKey[23] = key[ 1];
|
||||
tempKey[24] = key[40];
|
||||
tempKey[25] = key[51];
|
||||
tempKey[26] = key[30];
|
||||
tempKey[27] = key[36];
|
||||
tempKey[28] = key[46];
|
||||
tempKey[29] = key[54];
|
||||
tempKey[30] = key[29];
|
||||
tempKey[31] = key[39];
|
||||
tempKey[32] = key[50];
|
||||
tempKey[33] = key[44];
|
||||
tempKey[34] = key[32];
|
||||
tempKey[35] = key[47];
|
||||
tempKey[36] = key[43];
|
||||
tempKey[37] = key[48];
|
||||
tempKey[38] = key[38];
|
||||
tempKey[39] = key[55];
|
||||
tempKey[40] = key[33];
|
||||
tempKey[41] = key[52];
|
||||
tempKey[42] = key[45];
|
||||
tempKey[43] = key[41];
|
||||
tempKey[44] = key[49];
|
||||
tempKey[45] = key[35];
|
||||
tempKey[46] = key[28];
|
||||
tempKey[47] = key[31];
|
||||
switch(i){
|
||||
case 0: for(m=0;m < 48 ;m++){ keys[ 0][m] = tempKey[m]; } break;
|
||||
case 1: for(m=0;m < 48 ;m++){ keys[ 1][m] = tempKey[m]; } break;
|
||||
case 2: for(m=0;m < 48 ;m++){ keys[ 2][m] = tempKey[m]; } break;
|
||||
case 3: for(m=0;m < 48 ;m++){ keys[ 3][m] = tempKey[m]; } break;
|
||||
case 4: for(m=0;m < 48 ;m++){ keys[ 4][m] = tempKey[m]; } break;
|
||||
case 5: for(m=0;m < 48 ;m++){ keys[ 5][m] = tempKey[m]; } break;
|
||||
case 6: for(m=0;m < 48 ;m++){ keys[ 6][m] = tempKey[m]; } break;
|
||||
case 7: for(m=0;m < 48 ;m++){ keys[ 7][m] = tempKey[m]; } break;
|
||||
case 8: for(m=0;m < 48 ;m++){ keys[ 8][m] = tempKey[m]; } break;
|
||||
case 9: for(m=0;m < 48 ;m++){ keys[ 9][m] = tempKey[m]; } break;
|
||||
case 10: for(m=0;m < 48 ;m++){ keys[10][m] = tempKey[m]; } break;
|
||||
case 11: for(m=0;m < 48 ;m++){ keys[11][m] = tempKey[m]; } break;
|
||||
case 12: for(m=0;m < 48 ;m++){ keys[12][m] = tempKey[m]; } break;
|
||||
case 13: for(m=0;m < 48 ;m++){ keys[13][m] = tempKey[m]; } break;
|
||||
case 14: for(m=0;m < 48 ;m++){ keys[14][m] = tempKey[m]; } break;
|
||||
case 15: for(m=0;m < 48 ;m++){ keys[15][m] = tempKey[m]; } break;
|
||||
}
|
||||
}
|
||||
return keys;
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
// Compute the site's strEnc(data, k1, k2, k3) using the original des.js.
|
||||
// usage: node desenc.js <data> [k1] [k2] [k3]
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
|
||||
const desPath = path.join(__dirname, 'des.js');
|
||||
const code = fs.readFileSync(desPath, 'utf8');
|
||||
const ctx = { console };
|
||||
vm.createContext(ctx);
|
||||
vm.runInContext(code, ctx);
|
||||
|
||||
const [data, k1 = '1', k2 = '2', k3 = '3'] = process.argv.slice(2);
|
||||
if (data === undefined) {
|
||||
console.error('usage: node desenc.js <data> [k1] [k2] [k3]');
|
||||
process.exit(2);
|
||||
}
|
||||
const out = ctx.strEnc(data, k1, k2, k3);
|
||||
process.stdout.write(String(out));
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,39 @@
|
||||
# 多账号学习控制台 —— Docker 部署
|
||||
# docker compose up -d --build 起服务
|
||||
# docker compose logs -f 看日志
|
||||
# docker compose down 停服务（data/ 卷保留，账号和设置不丢）
|
||||
#
|
||||
# 远程管理：确保 .env 里设了 AUTH_TOKEN，再用 http://<服务器IP>:8848/?token=<AUTH_TOKEN> 打开
|
||||
services:
|
||||
study-tool:
|
||||
build:
|
||||
context: .
|
||||
args:
|
||||
# 直链视频（fileType=other）要 Chromium 真播到结尾；不要就把这里改成 "0"（镜像小很多）
|
||||
WITH_PLAYWRIGHT: "1"
|
||||
image: study-tool:latest
|
||||
container_name: study-tool
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "${BIND_ADDR:-0.0.0.0}:${PORT:-8848}:8848"
|
||||
environment:
|
||||
HOST: 0.0.0.0
|
||||
PORT: "8848"
|
||||
DATA_DIR: /app/data
|
||||
DB_PATH: /app/data/study.db
|
||||
# 远程访问令牌；留空 = 不校验（只在自己内网用）。改这里要同步 .env
|
||||
AUTH_TOKEN: ${AUTH_TOKEN:-}
|
||||
# 每个账号保留的日志行数 / 页面快照条数
|
||||
RUN_LOG_KEEP: ${RUN_LOG_KEEP:-2000}
|
||||
SNAPSHOT_KEEP: ${SNAPSHOT_KEEP:-200}
|
||||
TZ: ${TZ:-Asia/Shanghai}
|
||||
volumes:
|
||||
# 唯一的持久化目录：SQLite 库（study.db + -wal/-shm）都在这里
|
||||
- ./data:/app/data
|
||||
# Chromium 在容器里默认只有 64MB /dev/shm，跑直链视频会崩
|
||||
shm_size: "1gb"
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "10m"
|
||||
max-file: "3"
|
||||
@@ -0,0 +1,215 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* 莞易学（study.dgjy.net）学习链路的 axios 实现。
|
||||
*
|
||||
* 只做三件事：查进度 / 课时上报 / 完成后回读复核。
|
||||
* 不负责登录（Middle-Token 从浏览器 cookie 里复制），也不假装能绕过服务端校验。
|
||||
*
|
||||
* 用法：
|
||||
* MIDDLE_TOKEN='<document.cookie 里 Middle-Token 的值>' \
|
||||
* node js/study-axios.mjs status 123456
|
||||
*
|
||||
* MIDDLE_TOKEN=... node js/study-axios.mjs watch 123456
|
||||
* MIDDLE_TOKEN=... node js/study-axios.mjs watch 123456 --beat 2 --limit 900
|
||||
*
|
||||
* 依赖：axios（npm i axios），Node >= 18。
|
||||
*/
|
||||
|
||||
import axios from 'axios';
|
||||
|
||||
const BASE = process.env.DGJY_BASE || 'https://study.dgjy.net/train';
|
||||
const TOKEN = process.env.MIDDLE_TOKEN || '';
|
||||
|
||||
if (!TOKEN) {
|
||||
console.error('缺少 MIDDLE_TOKEN（浏览器 F12 → Application → Cookies → study.dgjy.net → Middle-Token）');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const http = axios.create({
|
||||
baseURL: BASE,
|
||||
timeout: 20000,
|
||||
headers: {
|
||||
'Content-Type': 'application/json;charset=utf-8',
|
||||
Authorization: `Bearer ${TOKEN}`,
|
||||
'User-Agent':
|
||||
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36',
|
||||
Referer: 'https://study.dgjy.net/training/',
|
||||
},
|
||||
});
|
||||
|
||||
// 站点统一返回 {code, msg, data}，HTTP 200 也可能是业务失败
|
||||
http.interceptors.response.use(
|
||||
(r) => {
|
||||
const b = r.data;
|
||||
if (!b || typeof b !== 'object') throw new Error(`${r.config.url} → 非 JSON 响应`);
|
||||
if (b.code !== 200) throw new Error(`${r.config.url} → code=${b.code} ${b.msg || b.message || ''}`);
|
||||
return b.data;
|
||||
},
|
||||
(e) => {
|
||||
const u = e.config ? `${e.config.method} ${e.config.url}` : '';
|
||||
const s = e.response ? `HTTP ${e.response.status}` : e.code || e.message;
|
||||
throw new Error(`${u} → ${s}`);
|
||||
},
|
||||
);
|
||||
|
||||
// ---------------------------------------------------------------- 接口
|
||||
|
||||
const getTrain = (id) =>
|
||||
http.get('/course/front/trainCourse/getById', { params: { id } });
|
||||
|
||||
const getTree = (courseId) =>
|
||||
http.get('/course/front/period/list/tree/studyLog', { params: { courseId, isFront: 1 } });
|
||||
|
||||
const getPeriod = (periodId) =>
|
||||
http.get('/course/front/period/getById', { params: { periodId: Number(periodId) } });
|
||||
|
||||
// 播放前置校验：{ canPlay, periodId, timeRemaining }
|
||||
// canPlay=false 表示「你有未完成课时，请按顺序播放」
|
||||
const getLimit = (periodId) =>
|
||||
http.get('/course/front/period/studyLimit', { params: { periodId: Number(periodId) } });
|
||||
|
||||
// 播放器每 2 秒发一次 isComplete=0，播到结尾发一次 isComplete=1（服务端据此写 completeTime）
|
||||
const postLog = (periodId, isComplete, lastProgressTime) =>
|
||||
http.post('/course/front/period/study/log', {
|
||||
periodId: Number(periodId),
|
||||
isComplete: isComplete ? 1 : 0,
|
||||
lastProgressTime: String(Math.round(lastProgressTime || 0)),
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------- 工具
|
||||
|
||||
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
|
||||
|
||||
const CN = { 1: '视频', 2: '回放', 5: '考试', 6: '作业', 12: '评价', 17: '培训评价' };
|
||||
|
||||
function flatten(nodes, out = []) {
|
||||
for (const n of nodes || []) {
|
||||
if (n.periodType || n.type === '3') out.push(n);
|
||||
flatten(n.children || n.periodList || [], out);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const isDone = (p) => Boolean(p.studyLog && p.studyLog.completeTime);
|
||||
const mark = (p) => (isDone(p) ? '✔ 完成' : p.studyLog ? '◐ 学习中' : '○ 未学');
|
||||
|
||||
// ---------------------------------------------------------------- 命令
|
||||
|
||||
async function status(trainCourseId) {
|
||||
const t = await getTrain(trainCourseId);
|
||||
const st = t.study || {};
|
||||
console.log(
|
||||
`培训：${t.courseName || t.name || '?'}\n` +
|
||||
`进度：${st.periodStudy ?? '?'}/${st.periodTotal ?? '?'} 分数：${st.courseStudyScore ?? '-'}`,
|
||||
);
|
||||
|
||||
const tree = await getTree(t.id || trainCourseId);
|
||||
for (const p of flatten(tree)) {
|
||||
const type = CN[String(p.periodType)] || `type${p.periodType}`;
|
||||
console.log(
|
||||
`${mark(p)} id=${p.id} ${type} ${p.name} fileType=${(p.file && p.file.fileType) || '-'}` +
|
||||
(p.studyLog && p.studyLog.lastProgressTime ? ` last=${p.studyLog.lastProgressTime}` : ''),
|
||||
);
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
async function watch(trainCourseId, opts) {
|
||||
const t = await getTrain(trainCourseId);
|
||||
const tree = await getTree(t.id || trainCourseId);
|
||||
const all = flatten(tree);
|
||||
const pending = all.filter((p) => !isDone(p) && ['1', '2', '3'].includes(String(p.periodType)));
|
||||
|
||||
if (!pending.length) {
|
||||
console.log('没有待学的视频课时');
|
||||
return;
|
||||
}
|
||||
|
||||
for (const p of pending) {
|
||||
const label = `[${p.id}] ${p.name}`;
|
||||
|
||||
try {
|
||||
const limit = await getLimit(p.id);
|
||||
if (limit && limit.canPlay === false) {
|
||||
console.log(`${label} 服务端要求按顺序播放，先学课时 ${limit.periodId}，跳过`);
|
||||
continue;
|
||||
}
|
||||
if (limit && limit.timeRemaining) console.log(`${label} 今日剩余播放时长 ${limit.timeRemaining}`);
|
||||
} catch (e) {
|
||||
console.log(`${label} 前置校验不可用（继续尝试）：${e.message}`);
|
||||
}
|
||||
|
||||
const fileType = (p.file && p.file.fileType) || '';
|
||||
if (fileType === 'mooc_video' || fileType === 'cloud_video') {
|
||||
console.log(`${label} fileType=${fileType}：这种课时的进度由播放器上报，纯接口大概率写不进 completeTime`);
|
||||
}
|
||||
|
||||
let dur = Number((p.file && p.file.duration) || p.duration || 0);
|
||||
if (!dur) {
|
||||
try {
|
||||
const d = await getPeriod(p.id);
|
||||
dur = Number((d.file && d.file.duration) || d.duration || 0);
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
dur = Math.min(dur || 600, Number(opts.limit) || 900);
|
||||
console.log(`${label} 开始上报，总时长 ${dur}s，心跳 ${opts.beat}s`);
|
||||
|
||||
let failed = false;
|
||||
for (let cur = 0; cur <= dur; cur += opts.beat) {
|
||||
await sleep(opts.beat * 1000);
|
||||
try {
|
||||
await postLog(p.id, 0, cur);
|
||||
} catch (e) {
|
||||
console.log(`${label} 心跳失败：${e.message}`);
|
||||
failed = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (failed) continue;
|
||||
|
||||
try {
|
||||
await postLog(p.id, 1, dur);
|
||||
} catch (e) {
|
||||
console.log(`${label} 完成上报失败：${e.message}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
await sleep(1000);
|
||||
try {
|
||||
const d = await getPeriod(p.id);
|
||||
const sl = d.studyLog || {};
|
||||
if (sl.completeTime) console.log(`${label} ✔ 服务端已写 completeTime=${sl.completeTime}`);
|
||||
else console.log(`${label} ✘ 接口 200 但 completeTime 仍为空（进度 ${sl.progress ?? '?'}）`);
|
||||
} catch (e) {
|
||||
console.log(`${label} 回读失败：${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
const after = await getTrain(trainCourseId);
|
||||
const st = after.study || {};
|
||||
console.log(`最终进度：${st.periodStudy ?? '?'}/${st.periodTotal ?? '?'}`);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- CLI
|
||||
|
||||
const [cmd, id, ...rest] = process.argv.slice(2);
|
||||
const argv = {};
|
||||
for (let i = 0; i < rest.length; i += 2) {
|
||||
if (rest[i].startsWith('--')) argv[rest[i].slice(2)] = rest[i + 1];
|
||||
}
|
||||
|
||||
const opts = { beat: Number(argv.beat) || 2, limit: Number(argv.limit) || 900 };
|
||||
|
||||
try {
|
||||
if (cmd === 'status') await status(id);
|
||||
else if (cmd === 'watch') await watch(id, opts);
|
||||
else {
|
||||
console.log('用法：node js/study-axios.mjs <status|watch> <trainCourseId> [--beat 2] [--limit 900]');
|
||||
process.exit(2);
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('失败：', e.message);
|
||||
process.exit(1);
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
fastapi>=0.110
|
||||
uvicorn>=0.27
|
||||
requests>=2.31
|
||||
beautifulsoup4>=4.12
|
||||
lxml>=5.0
|
||||
playwright>=1.40
|
||||
@@ -0,0 +1,39 @@
|
||||
@echo off
|
||||
chcp 65001 >nul
|
||||
title 多账号学习控制台 · 东莞市教师培训平台
|
||||
cd /d "%~dp0"
|
||||
|
||||
echo ============================================
|
||||
echo 多账号学习控制台 jspx.dgjy.net
|
||||
echo ============================================
|
||||
echo.
|
||||
|
||||
where python >nul 2>nul
|
||||
if errorlevel 1 (
|
||||
echo [错误] 未找到 python，请先安装 Python 3.10+ 并加入 PATH
|
||||
pause
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
where node >nul 2>nul
|
||||
if errorlevel 1 (
|
||||
if not exist "%NODE_BIN%" (
|
||||
echo [警告] 未在 PATH 中找到 node。若登录报错，请设置环境变量 NODE_BIN 指向 node.exe
|
||||
)
|
||||
)
|
||||
|
||||
python -c "import fastapi, uvicorn, requests, bs4" >nul 2>nul
|
||||
if errorlevel 1 (
|
||||
echo [安装依赖] fastapi uvicorn requests beautifulsoup4 lxml
|
||||
python -m pip install -r requirements.txt
|
||||
)
|
||||
|
||||
echo.
|
||||
echo 启动中... 浏览器请打开: http://127.0.0.1:8848
|
||||
echo 关闭本窗口即停止服务。
|
||||
echo.
|
||||
|
||||
start "" http://127.0.0.1:8848
|
||||
python app.py
|
||||
|
||||
pause
|
||||
+1201
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,486 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""SQLite3 存储层 —— 账号 / 设置 / 登录记录 / 运行日志 / 运行状态 / 缓存 / 页面快照
|
||||
|
||||
为什么用 SQLite 而不是散落的文件
|
||||
* 远程管理：Docker 里只挂一个 data 卷，所有状态都在这一个文件里，备份 = 拷一个文件
|
||||
* 并发：running 时有几十个线程写日志，文件追加要自己加锁，SQLite WAL 直接解决
|
||||
* 查询：/api/log/history 能按账号取最近 N 行，不需要自己维护文件轮转
|
||||
|
||||
表
|
||||
settings 系统配置（name -> JSON），目前只有 llm
|
||||
kv 通用缓存（任意 key -> JSON），例如 StudyClient 试出来的接口前缀
|
||||
accounts 账号 / 密码 / 会话 cookie / 课程缓存 / 勾选 / 开关
|
||||
login_log 每次登录尝试（平台原话）
|
||||
run_log 每个账号（key）的运行日志
|
||||
run_state 运行状态快照，进程重启后 /api/runs 还能看到上次跑到哪
|
||||
snapshots 课程页原始 HTML（站点改版时定位选择器用）
|
||||
|
||||
并发模型
|
||||
* WAL + busy_timeout=15s：读写互不阻塞，写锁竞争自动重试
|
||||
* 每个线程一条连接（threading.local）：ThreadPoolExecutor 里多线程写日志不会串
|
||||
* 本进程内的写操作再用一把 RLock 串起来，避免多线程 commit 抖动（读不加锁）
|
||||
* 首次调用自动建库建表，并把旧的 data/settings.json、data/logs/*.log 导入后改名
|
||||
（原文件保留为 *.imported，回滚旧版本时改名回去即可）
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sqlite3
|
||||
import threading
|
||||
import time
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
DATA_DIR = os.environ.get("DATA_DIR") or os.path.join(HERE, "data")
|
||||
DB_PATH = os.environ.get("DB_PATH") or os.path.join(DATA_DIR, "study.db")
|
||||
|
||||
# 旧的文件式存储位置（只在首次启动时导入一次）
|
||||
LEGACY_SETTINGS = os.path.join(DATA_DIR, "settings.json")
|
||||
LEGACY_LOGDIR = os.path.join(DATA_DIR, "logs")
|
||||
|
||||
# 每个账号保留的日志行数，超出就丢最旧的（远程长期挂机，不能无限涨）
|
||||
RUN_LOG_KEEP = int(os.environ.get("RUN_LOG_KEEP", "2000") or 2000)
|
||||
# 页面快照保留条数
|
||||
SNAPSHOT_KEEP = int(os.environ.get("SNAPSHOT_KEEP", "200") or 200)
|
||||
|
||||
SCHEMA = """
|
||||
CREATE TABLE IF NOT EXISTS settings (
|
||||
name TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
updated REAL NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS kv (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT,
|
||||
updated REAL NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS accounts (
|
||||
id TEXT PRIMARY KEY,
|
||||
username TEXT,
|
||||
password TEXT,
|
||||
note TEXT,
|
||||
user TEXT,
|
||||
cookies TEXT,
|
||||
courses TEXT,
|
||||
selected TEXT,
|
||||
status TEXT,
|
||||
last_error TEXT,
|
||||
last_error_at REAL,
|
||||
opt_quiz INTEGER DEFAULT 0,
|
||||
opt_feedback INTEGER DEFAULT 1,
|
||||
login_at REAL,
|
||||
created REAL,
|
||||
updated REAL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS login_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
ts TEXT,
|
||||
username TEXT,
|
||||
ok INTEGER,
|
||||
code TEXT,
|
||||
elapsed REAL,
|
||||
stage TEXT,
|
||||
error TEXT
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS run_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
key TEXT NOT NULL,
|
||||
ts REAL,
|
||||
line TEXT
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_run_log_key ON run_log(key, id);
|
||||
CREATE TABLE IF NOT EXISTS run_state (
|
||||
key TEXT PRIMARY KEY,
|
||||
state TEXT,
|
||||
updated REAL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS snapshots (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
plan_id TEXT,
|
||||
project_id TEXT,
|
||||
url TEXT,
|
||||
html TEXT,
|
||||
created REAL
|
||||
);
|
||||
"""
|
||||
|
||||
_local = threading.local()
|
||||
_lock = threading.RLock() # 只保护“写”和一次性初始化
|
||||
_ready = False
|
||||
_write_count = {}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 基础
|
||||
|
||||
def dumps(o):
|
||||
return json.dumps(o, ensure_ascii=False, separators=(",", ":"), default=str)
|
||||
|
||||
|
||||
def loads(s, default=None):
|
||||
if not s:
|
||||
return default
|
||||
try:
|
||||
return json.loads(s)
|
||||
except Exception:
|
||||
return default
|
||||
|
||||
|
||||
def conn():
|
||||
"""取当前线程的连接（没有就建一条），顺带保证表结构已建好。"""
|
||||
global _ready
|
||||
c = getattr(_local, "conn", None)
|
||||
if c is None:
|
||||
os.makedirs(os.path.dirname(os.path.abspath(DB_PATH)), exist_ok=True)
|
||||
# check_same_thread=False：连接跟着线程走，线程退出后由 GC 关闭，
|
||||
# 关闭动作可能发生在别的线程里，默认的 same-thread 检查会直接报错。
|
||||
c = sqlite3.connect(DB_PATH, timeout=15, check_same_thread=False)
|
||||
c.row_factory = sqlite3.Row
|
||||
c.execute("PRAGMA journal_mode=WAL")
|
||||
c.execute("PRAGMA synchronous=NORMAL")
|
||||
c.execute("PRAGMA busy_timeout=15000")
|
||||
_local.conn = c
|
||||
if not _ready:
|
||||
with _lock:
|
||||
if not _ready:
|
||||
c.executescript(SCHEMA)
|
||||
c.commit()
|
||||
_migrate(c)
|
||||
_ready = True
|
||||
return c
|
||||
|
||||
|
||||
def query(sql, params=()):
|
||||
return conn().execute(sql, params).fetchall()
|
||||
|
||||
|
||||
def query_one(sql, params=()):
|
||||
rows = query(sql, params)
|
||||
return rows[0] if rows else None
|
||||
|
||||
|
||||
def write(sql, params=()):
|
||||
with _lock:
|
||||
c = conn()
|
||||
cur = c.execute(sql, params)
|
||||
c.commit()
|
||||
return cur
|
||||
|
||||
|
||||
def init():
|
||||
"""显式初始化（启动时调用一次，确保库文件已建好）。"""
|
||||
conn()
|
||||
|
||||
|
||||
def db_info():
|
||||
try:
|
||||
size = os.path.getsize(DB_PATH) if os.path.exists(DB_PATH) else 0
|
||||
except Exception:
|
||||
size = 0
|
||||
counts = {}
|
||||
for t in ("accounts", "run_log", "login_log", "snapshots", "run_state", "kv", "settings"):
|
||||
try:
|
||||
counts[t] = (query_one("SELECT COUNT(*) AS n FROM %s" % t) or {"n": 0})["n"]
|
||||
except Exception:
|
||||
counts[t] = -1
|
||||
return {"path": DB_PATH, "size": size, "counts": counts}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 迁移
|
||||
|
||||
def _migrate(c):
|
||||
"""把旧的文件式存储导入 SQLite；成功后把原文件改名成 *.imported。"""
|
||||
# 1) data/settings.json -> settings
|
||||
if os.path.exists(LEGACY_SETTINGS):
|
||||
try:
|
||||
with open(LEGACY_SETTINGS, encoding="utf-8") as f:
|
||||
data = json.load(f) or {}
|
||||
if isinstance(data.get("llm"), dict) and get_setting("llm") is None:
|
||||
set_setting("llm", data["llm"])
|
||||
os.replace(LEGACY_SETTINGS, LEGACY_SETTINGS + ".imported")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 2) data/logs/login.log -> login_log
|
||||
if os.path.isdir(LEGACY_LOGDIR):
|
||||
for name in sorted(os.listdir(LEGACY_LOGDIR)):
|
||||
if not name.endswith(".log"):
|
||||
continue
|
||||
path = os.path.join(LEGACY_LOGDIR, name)
|
||||
try:
|
||||
with open(path, encoding="utf-8", errors="replace") as f:
|
||||
lines = [x.rstrip("\n") for x in f if x.strip()]
|
||||
except Exception:
|
||||
continue
|
||||
if name == "login.log":
|
||||
for ln in lines:
|
||||
p = ln.split("\t")
|
||||
if len(p) < 4:
|
||||
continue
|
||||
el = re.sub(r"[^0-9.]", "", p[4]) if len(p) > 4 else "0"
|
||||
login_log_add(p[1], p[2] == "OK", p[3],
|
||||
float(el or 0), p[5] if len(p) > 5 else "",
|
||||
p[6] if len(p) > 6 else "", ts=p[0])
|
||||
else:
|
||||
# data/logs/<key>.log -> run_log（key 是文件名去掉 .log）
|
||||
key = name[:-4]
|
||||
for ln in lines:
|
||||
run_log_add(key, ln, ts=None)
|
||||
try:
|
||||
os.replace(path, path + ".imported")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 3) data/study_session.json（study_video.py 的会话）-> kv
|
||||
legacy_session = os.path.join(DATA_DIR, "study_session.json")
|
||||
if os.path.exists(legacy_session) and kv_get("study_session") is None:
|
||||
try:
|
||||
with open(legacy_session, encoding="utf-8") as f:
|
||||
kv_set("study_session", json.load(f))
|
||||
os.replace(legacy_session, legacy_session + ".imported")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ settings
|
||||
|
||||
def get_setting(name, default=None):
|
||||
r = query_one("SELECT value FROM settings WHERE name=?", (name,))
|
||||
return loads(r["value"], default) if r else default
|
||||
|
||||
|
||||
def set_setting(name, value):
|
||||
write("INSERT INTO settings(name, value, updated) VALUES(?,?,?) "
|
||||
"ON CONFLICT(name) DO UPDATE SET value=excluded.value, updated=excluded.updated",
|
||||
(name, dumps(value), time.time()))
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ kv 缓存
|
||||
|
||||
def kv_get(key, default=None):
|
||||
r = query_one("SELECT value FROM kv WHERE key=?", (key,))
|
||||
return loads(r["value"], default) if r else default
|
||||
|
||||
|
||||
def kv_set(key, value):
|
||||
write("INSERT INTO kv(key, value, updated) VALUES(?,?,?) "
|
||||
"ON CONFLICT(key) DO UPDATE SET value=excluded.value, updated=excluded.updated",
|
||||
(key, dumps(value), time.time()))
|
||||
|
||||
|
||||
def kv_del(key):
|
||||
write("DELETE FROM kv WHERE key=?", (key,))
|
||||
|
||||
|
||||
def kv_all():
|
||||
return {r["key"]: loads(r["value"]) for r in query("SELECT key, value FROM kv")}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ accounts
|
||||
|
||||
def _acc_row(r):
|
||||
"""DB 行 -> 前端用的记录（不含密码）。"""
|
||||
return {
|
||||
"id": r["id"],
|
||||
"username": r["username"] or "",
|
||||
"note": r["note"] or "",
|
||||
"user": r["user"] or "",
|
||||
"cookies": loads(r["cookies"], []) or [],
|
||||
"courses": loads(r["courses"], []) or [],
|
||||
"selected": loads(r["selected"], []) or [],
|
||||
"status": r["status"] or "idle",
|
||||
"lastError": r["last_error"] or "",
|
||||
"lastErrorAt": r["last_error_at"] or 0,
|
||||
"optQuiz": bool(r["opt_quiz"]),
|
||||
"optFeedback": bool(r["opt_feedback"]),
|
||||
"loginAt": r["login_at"] or 0,
|
||||
"created": r["created"] or 0,
|
||||
"updated": r["updated"] or 0,
|
||||
"hasPassword": bool(r["password"]),
|
||||
}
|
||||
|
||||
|
||||
def account_list():
|
||||
return [_acc_row(r) for r in query("SELECT * FROM accounts ORDER BY created, id")]
|
||||
|
||||
|
||||
def account_get(account_id):
|
||||
if not account_id:
|
||||
return None
|
||||
r = query_one("SELECT * FROM accounts WHERE id=?", (str(account_id),))
|
||||
return _acc_row(r) if r else None
|
||||
|
||||
|
||||
def account_password(account_id):
|
||||
if not account_id:
|
||||
return ""
|
||||
r = query_one("SELECT password FROM accounts WHERE id=?", (str(account_id),))
|
||||
return (r["password"] or "") if r else ""
|
||||
|
||||
|
||||
def account_exists(account_id):
|
||||
return bool(account_id) and query_one("SELECT 1 FROM accounts WHERE id=?",
|
||||
(str(account_id),)) is not None
|
||||
|
||||
|
||||
def account_upsert(rec):
|
||||
"""新增/更新账号。rec['password'] 为空(None)时保留库里原有密码。
|
||||
|
||||
明文密码要留在库里：会话失效时后台要能自己重登，不能等用户回来输密码。
|
||||
"""
|
||||
now = time.time()
|
||||
aid = str(rec.get("id") or "").strip()
|
||||
if not aid:
|
||||
aid = "a%s%04d" % (int(now * 1000), int(time.time() * 10000) % 10000)
|
||||
pwd = rec.get("password")
|
||||
if pwd is not None:
|
||||
pwd = str(pwd)
|
||||
selected = loads(dumps(rec.get("selected") or []), []) or []
|
||||
write(
|
||||
"""INSERT INTO accounts
|
||||
(id, username, password, note, user, cookies, courses, selected, status,
|
||||
last_error, last_error_at, opt_quiz, opt_feedback, login_at, created, updated)
|
||||
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
username = excluded.username,
|
||||
password = COALESCE(excluded.password, accounts.password),
|
||||
note = excluded.note,
|
||||
user = excluded.user,
|
||||
cookies = excluded.cookies,
|
||||
courses = excluded.courses,
|
||||
selected = excluded.selected,
|
||||
status = excluded.status,
|
||||
last_error = excluded.last_error,
|
||||
last_error_at = excluded.last_error_at,
|
||||
opt_quiz = excluded.opt_quiz,
|
||||
opt_feedback = excluded.opt_feedback,
|
||||
login_at = excluded.login_at,
|
||||
updated = excluded.updated""",
|
||||
(aid, rec.get("username") or "", pwd, rec.get("note") or "", rec.get("user") or "",
|
||||
dumps(rec.get("cookies") or []), dumps(rec.get("courses") or []), dumps(selected),
|
||||
rec.get("status") or "idle", rec.get("lastError") or "",
|
||||
float(rec.get("lastErrorAt") or 0),
|
||||
1 if rec.get("optQuiz") is True else 0,
|
||||
0 if rec.get("optFeedback") is False else 1,
|
||||
float(rec.get("loginAt") or 0),
|
||||
float(rec.get("created") or now), now))
|
||||
return aid
|
||||
|
||||
|
||||
def account_save_login(account_id, user, cookies, courses):
|
||||
"""登录成功后把新的会话写回账号（服务端自己记，不依赖前端回传）。"""
|
||||
rec = account_get(account_id)
|
||||
if not rec:
|
||||
return
|
||||
courses = courses or []
|
||||
plans = {c.get("plan_id") for c in courses}
|
||||
rec.update({
|
||||
"user": user or rec.get("user") or "",
|
||||
"cookies": cookies or [],
|
||||
"courses": courses,
|
||||
"selected": [p for p in (rec.get("selected") or []) if p in plans],
|
||||
"status": "logged_in",
|
||||
"lastError": "",
|
||||
"lastErrorAt": 0,
|
||||
"loginAt": time.time(),
|
||||
})
|
||||
rec.pop("hasPassword", None)
|
||||
account_upsert(rec)
|
||||
|
||||
|
||||
def account_delete(account_id):
|
||||
aid = str(account_id or "")
|
||||
write("DELETE FROM accounts WHERE id=?", (aid,))
|
||||
write("DELETE FROM run_state WHERE key=?", (aid,))
|
||||
write("DELETE FROM run_log WHERE key=?", (aid,))
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 登录记录
|
||||
|
||||
def login_log_add(username, ok, code, elapsed, stage="", error="", ts=None):
|
||||
write("INSERT INTO login_log(ts, username, ok, code, elapsed, stage, error) VALUES(?,?,?,?,?,?,?)",
|
||||
(ts or time.strftime("%Y-%m-%d %H:%M:%S"), username, 1 if ok else 0,
|
||||
str(code or ""), float(elapsed or 0), str(stage or ""),
|
||||
str(error or "").replace("\n", " ")[:500]))
|
||||
|
||||
|
||||
def login_log_tail(n=40):
|
||||
"""返回旧版文本行格式（ts\\t用户名\\tOK\\tcode\\t耗时\\tstage\\t错误），前端不用改。"""
|
||||
n = max(1, min(int(n or 40), 500))
|
||||
rows = query("SELECT * FROM login_log ORDER BY id DESC LIMIT ?", (n,))
|
||||
out = []
|
||||
for r in reversed(rows):
|
||||
out.append("%s\t%s\t%s\t%s\t%.1fs\t%s\t%s" % (
|
||||
r["ts"], r["username"], "OK" if r["ok"] else "FAIL", r["code"],
|
||||
r["elapsed"] or 0, r["stage"] or "", r["error"] or ""))
|
||||
return out
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 运行日志
|
||||
|
||||
def run_log_add(key, line, ts=None):
|
||||
key = str(key or "")
|
||||
write("INSERT INTO run_log(key, ts, line) VALUES(?,?,?)", (key, ts or time.time(), line))
|
||||
n = _write_count.get("run_log", 0) + 1
|
||||
_write_count["run_log"] = n
|
||||
if n % 200 == 0: # 每 200 行清一次，避免每行都跑一遍 DELETE
|
||||
write("""DELETE FROM run_log WHERE key=? AND id NOT IN
|
||||
(SELECT id FROM run_log WHERE key=? ORDER BY id DESC LIMIT ?)""",
|
||||
(key, key, RUN_LOG_KEEP))
|
||||
|
||||
|
||||
def run_log_tail(key, n=200):
|
||||
n = max(1, min(int(n or 200), RUN_LOG_KEEP))
|
||||
rows = query("SELECT line FROM run_log WHERE key=? ORDER BY id DESC LIMIT ?", (str(key), n))
|
||||
return [r["line"] for r in reversed(rows)]
|
||||
|
||||
|
||||
def run_log_keys():
|
||||
return [{"key": r["key"], "lines": r["n"],
|
||||
"updated": r["ts"]} for r in query(
|
||||
"SELECT key, COUNT(*) AS n, MAX(ts) AS ts FROM run_log GROUP BY key ORDER BY ts DESC")]
|
||||
|
||||
|
||||
def run_log_clear(key):
|
||||
write("DELETE FROM run_log WHERE key=?", (str(key),))
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 运行状态
|
||||
|
||||
def run_state_set(key, state):
|
||||
st = dict(state or {})
|
||||
st.pop("log", None) # 日志单独进 run_log 表，不在这里重复一份
|
||||
write("INSERT INTO run_state(key, state, updated) VALUES(?,?,?) "
|
||||
"ON CONFLICT(key) DO UPDATE SET state=excluded.state, updated=excluded.updated",
|
||||
(str(key), dumps(st), time.time()))
|
||||
|
||||
|
||||
def run_state_all():
|
||||
out = {}
|
||||
for r in query("SELECT key, state FROM run_state"):
|
||||
st = loads(r["state"], None)
|
||||
if isinstance(st, dict):
|
||||
out[r["key"]] = st
|
||||
return out
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 页面快照
|
||||
|
||||
def snapshot_add(plan_id, project_id, url, html):
|
||||
cur = write("INSERT INTO snapshots(plan_id, project_id, url, html, created) VALUES(?,?,?,?,?)",
|
||||
(str(plan_id or ""), str(project_id or ""), url or "", html or "", time.time()))
|
||||
sid = cur.lastrowid
|
||||
write("""DELETE FROM snapshots WHERE id NOT IN
|
||||
(SELECT id FROM snapshots ORDER BY id DESC LIMIT ?)""", (SNAPSHOT_KEEP,))
|
||||
return sid
|
||||
|
||||
|
||||
def snapshot_get(sid):
|
||||
r = query_one("SELECT * FROM snapshots WHERE id=?", (int(sid),))
|
||||
return dict(r) if r else None
|
||||
|
||||
|
||||
def snapshot_list(limit=50):
|
||||
limit = max(1, min(int(limit or 50), 500))
|
||||
rows = query("SELECT id, plan_id, project_id, url, created, LENGTH(html) AS size "
|
||||
"FROM snapshots ORDER BY id DESC LIMIT ?", (limit,))
|
||||
return [dict(r) for r in rows]
|
||||
+298
@@ -0,0 +1,298 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
study.dgjy.net（莞易学 / 东莞教师培训）视频课时 —— 自动完成 + 完成度核验
|
||||
|
||||
机制（逆向结论）
|
||||
* 播放器是网易云课堂 EduPlayer，进度上报接口：
|
||||
POST /train/course/front/period/study/log
|
||||
body {"periodId":4133,"isComplete":0|1,"lastProgressTime":"<秒>"}
|
||||
* onTimeupdate 时若 currentTime > maxTime 才上报（isComplete=0）
|
||||
* 视频播到结尾触发 onPlayEnd -> POST {"periodId":x,"isComplete":1} ← 这才是"完成"
|
||||
* onSeek 守卫：仅当 n.newData !== duration 且 allowVideoDrag === "0" 且 newData > maxTime
|
||||
时才会重新拉签名把进度拉回去。所以「拖到正好结尾」不会被拦。
|
||||
* 上报后 10 秒自动跳到下一课时。
|
||||
|
||||
完成度核验（唯一可信来源）
|
||||
GET /train/course/front/period/list/tree/studyLog?courseId=<MOOC课程ID>&isFront=1
|
||||
每个课时节点里的 studyLog.completeTime 非空 = 平台真的记为完成。
|
||||
|
||||
用法
|
||||
python study_video.py verify <trainCourseId> # 只核验，不动数据
|
||||
python study_video.py run <trainCourseId> # 完成所有未完成的视频课时
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
|
||||
import requests
|
||||
|
||||
import store
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
BASE = "https://study.dgjy.net"
|
||||
KV_SESSION = "study_session" # 会话存在 SQLite 的 kv 表（和网页端同一份库）
|
||||
UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
|
||||
"(KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36 Edg/154.0.0.0")
|
||||
CDP = "http://127.0.0.1:9222"
|
||||
|
||||
|
||||
def load_session():
|
||||
"""从 SQLite 读上次抓到的会话；没有返回 None。"""
|
||||
sess = store.kv_get(KV_SESSION)
|
||||
return sess if isinstance(sess, dict) and sess.get("token") else None
|
||||
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 会话 / 认证
|
||||
|
||||
def capture_session():
|
||||
"""从已登录的浏览器里取 Middle-Token 与 cookie。"""
|
||||
from playwright.sync_api import sync_playwright
|
||||
with sync_playwright() as p:
|
||||
b = p.chromium.connect_over_cdp(CDP)
|
||||
cookies = b.contexts[0].cookies()
|
||||
token = ""
|
||||
for c in cookies:
|
||||
if c["name"] == "Middle-Token" and "study.dgjy.net" in (c.get("domain") or ""):
|
||||
token = c["value"]
|
||||
if not token:
|
||||
raise SystemExit("没找到 Middle-Token —— 请先在浏览器里登录 study.dgjy.net")
|
||||
sess = {"token": token,
|
||||
"cookies": [{"name": c["name"], "value": c["value"],
|
||||
"domain": c["domain"], "path": c["path"]} for c in cookies]}
|
||||
store.kv_set(KV_SESSION, sess)
|
||||
return sess
|
||||
|
||||
|
||||
def api(sess=None):
|
||||
if sess is None:
|
||||
sess = load_session() or capture_session()
|
||||
s = requests.Session()
|
||||
s.headers.update({"User-Agent": UA, "Accept": "application/json, text/plain, */*",
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": "Bearer " + sess["token"],
|
||||
"Referer": BASE + "/training/courseVideo"})
|
||||
for c in sess["cookies"]:
|
||||
if "dgjy.net" in (c.get("domain") or ""):
|
||||
s.cookies.set(c["name"], c["value"] or "",
|
||||
domain=(c.get("domain") or "").lstrip("."), path=c.get("path") or "/")
|
||||
return s
|
||||
|
||||
|
||||
def check_auth(s):
|
||||
r = s.get(BASE + "/train/getInfo", timeout=30)
|
||||
try:
|
||||
return r.json().get("code") == 200
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 课程结构
|
||||
|
||||
def course_info(s, train_id):
|
||||
return s.get(BASE + "/train/course/front/trainCourse/getById?id=%s" % train_id,
|
||||
timeout=30).json().get("data") or {}
|
||||
|
||||
|
||||
def period_tree(s, course_id, is_front="1"):
|
||||
r = s.get(BASE + "/train/course/front/period/list/tree/studyLog?courseId=%s&isFront=%s"
|
||||
% (course_id, is_front), timeout=30)
|
||||
return r.json().get("data") or []
|
||||
|
||||
|
||||
def flatten(nodes, out=None, parent=""):
|
||||
out = [] if out is None else out
|
||||
for n in nodes or []:
|
||||
sl = n.get("studyLog") or {}
|
||||
out.append({
|
||||
"periodId": n.get("id"),
|
||||
"name": n.get("name") or "",
|
||||
"parent": parent,
|
||||
"periodType": str(n.get("periodType") or ""),
|
||||
"duration": n.get("duration"),
|
||||
"periodScore": n.get("periodScore"),
|
||||
"allowVideoDrag": n.get("allowVideoDrag"),
|
||||
"completeLimit": n.get("completeLimit"),
|
||||
"kcdm": n.get("kcdm"),
|
||||
"completeTime": sl.get("completeTime"),
|
||||
"logPeriodScore": sl.get("logPeriodScore"),
|
||||
"lastProgressTime": sl.get("lastProgressTime"),
|
||||
})
|
||||
flatten(n.get("children"), out, n.get("name") or parent)
|
||||
return out
|
||||
|
||||
|
||||
def video_periods(s, train_id):
|
||||
"""返回 (培训课程信息, MOOC课程ID, 视频课时列表)"""
|
||||
info = course_info(s, train_id)
|
||||
top = flatten(period_tree(s, train_id, ""))
|
||||
mooc = ""
|
||||
for t in top:
|
||||
if t["kcdm"]:
|
||||
mooc = t["kcdm"]
|
||||
break
|
||||
vids = []
|
||||
if mooc:
|
||||
vids = [p for p in flatten(period_tree(s, mooc, "1")) if p["periodType"] == "2"]
|
||||
return info, mooc, vids
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 核验
|
||||
|
||||
def report(info, vids):
|
||||
print()
|
||||
print("课程：%s" % info.get("courseName", "?"))
|
||||
st = info.get("study") or {}
|
||||
print("进度：%s/%s 课时 规则 completeRule=%s" %
|
||||
(st.get("periodStudy"), st.get("periodTotal"), info.get("completeRule")))
|
||||
print()
|
||||
print("%-8s %-40s %-8s %-6s %-20s %s" %
|
||||
("periodId", "课时名", "时长(s)", "拖拽", "completeTime", "判定"))
|
||||
done = 0
|
||||
for v in vids:
|
||||
ok = bool(v["completeTime"])
|
||||
done += ok
|
||||
print("%-8s %-40s %-8s %-6s %-20s %s" %
|
||||
(v["periodId"], v["name"][:40], v["duration"], v["allowVideoDrag"],
|
||||
v["completeTime"] or "-", "✅ 已完成" if ok else "❌ 未完成"))
|
||||
print()
|
||||
print("视频课时：%d/%d 已完成" % (done, len(vids)))
|
||||
return done, len(vids)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 自动完成
|
||||
|
||||
def complete_one(pg, train_id, train_course, period_id, timeout=180):
|
||||
"""打开视频页 -> 拖到结尾 -> 播放 -> 等 isComplete=1 上报。返回 True/False。"""
|
||||
url = ("%s/training/courseVideo?classhour=%d&isJoin=1&needJoin=1&courseId=%s&isFront=1"
|
||||
"&trainCourseId=%s" % (BASE, period_id, train_course, train_id))
|
||||
posted = {"done": False}
|
||||
|
||||
def on_req(r):
|
||||
try:
|
||||
if "study/log" in r.url and "detail" not in r.url:
|
||||
body = r.post_data or ""
|
||||
if '"isComplete":1' in body.replace(" ", ""):
|
||||
posted["done"] = True
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
pg.on("request", on_req)
|
||||
pg.goto(url, wait_until="domcontentloaded", timeout=60000)
|
||||
|
||||
dur = None
|
||||
for _ in range(40):
|
||||
time.sleep(1.5)
|
||||
st = pg.evaluate("""() => {
|
||||
const v = Array.from(document.querySelectorAll('video')).filter(x=>x.duration>100)[0];
|
||||
return v ? {dur: v.duration, ct: v.currentTime} : null;
|
||||
}""")
|
||||
if st and st["dur"] > 100:
|
||||
dur = st["dur"]
|
||||
break
|
||||
if not dur:
|
||||
print(" 播放器未就绪")
|
||||
return False
|
||||
|
||||
# 拖到结尾前 4 秒（绕开 onSeek 守卫，等价于"拖到正好结尾"）
|
||||
pg.evaluate("""() => {
|
||||
const v = Array.from(document.querySelectorAll('video')).filter(x=>x.duration>100)[0];
|
||||
v.muted = true;
|
||||
v.currentTime = Math.max(0, v.duration - 4);
|
||||
}""")
|
||||
time.sleep(1)
|
||||
|
||||
btn = pg.query_selector(".j-bigplaybtn") or pg.query_selector(".j-playbtn")
|
||||
if btn and btn.bounding_box():
|
||||
bx = btn.bounding_box()
|
||||
pg.mouse.click(bx["x"] + bx["width"] / 2, bx["y"] + bx["height"] / 2)
|
||||
|
||||
t0 = time.time()
|
||||
while time.time() - t0 < timeout:
|
||||
time.sleep(2)
|
||||
if posted["done"]:
|
||||
return True
|
||||
st = pg.evaluate("""() => {
|
||||
const v = Array.from(document.querySelectorAll('video')).filter(x=>x.duration>100)[0];
|
||||
return v ? {ct: Math.round(v.currentTime), ended: v.ended, paused: v.paused} : null;
|
||||
}""")
|
||||
# 防挂机弹窗「是否继续观看视频?」
|
||||
try:
|
||||
ok = pg.query_selector(".el-message-box__btns .el-button--primary")
|
||||
if ok:
|
||||
ok.click()
|
||||
except Exception:
|
||||
pass
|
||||
# 播放结束但还没上报 -> 再补一次播放
|
||||
if st and st["ended"] and not posted["done"]:
|
||||
btn = pg.query_selector(".j-bigplaybtn") or pg.query_selector(".j-playbtn")
|
||||
if btn and btn.bounding_box():
|
||||
bx = btn.bounding_box()
|
||||
pg.mouse.click(bx["x"] + bx["width"] / 2, bx["y"] + bx["height"] / 2)
|
||||
return posted["done"]
|
||||
|
||||
|
||||
def run(train_id, only_missing=True):
|
||||
sess = load_session() or capture_session()
|
||||
s = api(sess)
|
||||
if not check_auth(s):
|
||||
print("会话失效，重新抓取…")
|
||||
sess = capture_session()
|
||||
s = api(sess)
|
||||
if not check_auth(s):
|
||||
raise SystemExit("认证失败：请先在浏览器登录 study.dgjy.net")
|
||||
|
||||
info, mooc, vids = video_periods(s, train_id)
|
||||
print("=== 处理前核验 ===")
|
||||
report(info, vids)
|
||||
|
||||
todo = [v for v in vids if not v["completeTime"]]
|
||||
if not todo:
|
||||
print("\n所有视频课时均已完成，无需处理。")
|
||||
return
|
||||
|
||||
print("\n待完成 %d 个课时：" % len(todo))
|
||||
for v in todo:
|
||||
print(" %s %s (%ss)" % (v["periodId"], v["name"][:40], v["duration"]))
|
||||
|
||||
from playwright.sync_api import sync_playwright
|
||||
with sync_playwright() as p:
|
||||
b = p.chromium.connect_over_cdp(CDP)
|
||||
ctx = b.contexts[0]
|
||||
pg = ctx.new_page()
|
||||
for i, v in enumerate(todo, 1):
|
||||
print("\n[%d/%d] %s %s" % (i, len(todo), v["periodId"], v["name"][:40]))
|
||||
try:
|
||||
ok = complete_one(pg, train_id, train_id, v["periodId"])
|
||||
except Exception as e:
|
||||
print(" 异常:", e)
|
||||
ok = False
|
||||
print(" 上报 isComplete=1 :", "✅" if ok else "❌")
|
||||
time.sleep(4)
|
||||
try:
|
||||
pg.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
print("\n=== 处理后核验（以平台数据为准）===")
|
||||
time.sleep(3)
|
||||
info2, _, vids2 = video_periods(s, train_id)
|
||||
report(info2, vids2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 3:
|
||||
print(__doc__)
|
||||
sys.exit(1)
|
||||
mode, tid = sys.argv[1], sys.argv[2]
|
||||
if mode == "verify":
|
||||
sess = load_session() or capture_session()
|
||||
s = api(sess)
|
||||
info, mooc, vids = video_periods(s, tid)
|
||||
print("MOOC 课程ID:", mooc)
|
||||
report(info, vids)
|
||||
elif mode == "run":
|
||||
run(tid)
|
||||
else:
|
||||
print(__doc__)
|
||||
@@ -0,0 +1,79 @@
|
||||
# TODO
|
||||
|
||||
## 待修复：首次进入课程会弹提示挡住，必须手动点一下才能进课程
|
||||
|
||||
**状态**：暂停（2026-10-27 18:27）。现象确认存在，但还没抓到具体弹窗，修复方案未定。
|
||||
|
||||
### 现象
|
||||
|
||||
- 一个课程/课时**第一次**进入时，页面会先出提示（弹层/遮罩/确认框），不点它进不去。
|
||||
- 手动点一下之后就能正常进入、正常播放；再进同一门课就不再出现。
|
||||
- 因为卡在这一步，自动学习流程跑不下去，只能人工点。
|
||||
|
||||
### 需要先确认（下一步的起点）
|
||||
|
||||
1. 那个提示到底是哪一层的东西，三选一：
|
||||
- 培训平台（`study.dgjy.net`）自己的「学习须知 / 继续学习 / 我已知晓」弹层；
|
||||
- 播放页 `courseVideo` 里的播放前确认（点「开始学习」才 load video）；
|
||||
- 浏览器层的 autoplay / 全屏 / 权限提示。
|
||||
2. 弹层出现的确切条件（首次进课程？首次进课时？换设备/换浏览器算不算首次？）。
|
||||
3. 点掉它之后前端发了什么请求（有没有 `period/study/log`、`student/join` 之类）。
|
||||
|
||||
### 已排查到的线索
|
||||
|
||||
- 触发点在「进入课程 → 课时」这条路径上，接口层面看不出来：
|
||||
`app.py` 的学习主流程是纯 HTTP（`study_complete_period` → `period/study/log`），
|
||||
只有 `kind == "direct"` 的直链视频才用 Playwright 开真浏览器（`play_period_to_end`，app.py:1664）。
|
||||
- `play_period_to_end` 里已经有一处「盲点确定」的补丁，说明历史上就撞过类似弹层：
|
||||
|
||||
```python
|
||||
ok = pg.query_selector(".el-message-box__btns .el-button--primary")
|
||||
if ok:
|
||||
ok.click()
|
||||
```
|
||||
|
||||
但它只在视频开始播放**之后**每 2 秒点一次，且只认 Element-UI 的 message-box；
|
||||
如果是进入页面时的 `el-dialog` 遮罩，这个选择器打不中。
|
||||
- 新版平台课程页地址（`discover()` 拿到的 `page_url`，日志里 token 打码）：
|
||||
`https://study.dgjy.net/training/trainDetails?id=<trainCourseId>&isFront=1`
|
||||
- 课程页点「继续学习」后跳到课时列表页：
|
||||
`https://study.dgjy.net/training/curriculum?id=<moocCourseId>&isFront=1&courseId=<trainCourseId>`
|
||||
视频课时在 `period/list/tree/studyLog` 里，`periodType=2`，`file.fileType=mooc_video`。
|
||||
- 播放页拼法见 app.py:1675：
|
||||
`https://study.dgjy.net/training/courseVideo?classhour=<periodId>&isJoin=1&needJoin=1&courseId=<trainCourseId>&isFront=1&trainCourseId=<trainCourseId>`
|
||||
|
||||
### 为什么这次没抓到
|
||||
|
||||
演示账号（库里唯一账号）的两门课 12 + 14 个课时**全部已 `completeTime` 有值**，
|
||||
「首次进入」的前置状态已经不在了，弹层复现不出来。
|
||||
下次要么换一个没学过的账号/课程，要么把某个课时的 `studyLog` 清掉再试。
|
||||
|
||||
### 下次的复现 + 取证办法
|
||||
|
||||
用带界面的 Playwright 窗口，把登录后的 cookie 注进去，人工在窗口里点到弹层出现：
|
||||
|
||||
1. `python -c` 跑一遍 `app.login(user, pw)`，把 `sess.cookies`（只取 `*.dgjy.net`）写成 JSON；
|
||||
2. `launch_persistent_context(headless=False)`，`ctx.add_cookies(...)`，`goto` 课程页；
|
||||
3. 同时挂：
|
||||
- `page.on("request"/"response")` 全量记录（过滤掉 png/css/font）；
|
||||
- `document.addEventListener('click', ..., true)` 记录 `target` 的 CSS 路径和文字；
|
||||
- 每 2 秒 `page.screenshot()` 一张；
|
||||
- 每 2 秒扫一次可见的 `.el-message-box / .el-dialog / [role=dialog]`，把 `innerText` 落日志。
|
||||
4. 人工点掉弹层 → 看日志里「点击前后出现了哪个 dialog 文本 + 紧随其后的请求」，就知道该自动点哪个。
|
||||
|
||||
### 修好后要做到
|
||||
|
||||
- 自动学习时不再需要人工点一下：进课程/进课时后若出现确认弹层，自动识别并点掉，
|
||||
再继续走 `period/study/log` 或 `courseVideo` 播放。
|
||||
- 只点「继续/确定/我已知晓」这类无副作用的按钮，别乱点「提交/评价」。
|
||||
- 日志里留下「已自动关闭弹层：<文字>」，方便出问题时对账。
|
||||
|
||||
### 相关代码位置
|
||||
|
||||
| 位置 | 作用 |
|
||||
| --- | --- |
|
||||
| `app.py:1664` `play_period_to_end` | 直链视频真播；已有盲点 `.el-message-box__btns .el-button--primary` |
|
||||
| `app.py:1530` `run_study_course` | 新版平台课程主流程（反馈表 / 测验 / 视频） |
|
||||
| `app.py:1013` `discover_study` | 读课程结构，产出 `page_url` / `train_id` / `courseware` |
|
||||
| `app.py:1128` `study_period_log` | 课时上报接口，`isComplete` 走这里 |
|
||||
| `app.py:633` `_discover_once` | jspx 课程页 → study `trainDetails` 的跳转与 token 落 cookie |
|
||||
Reference in New Issue
Block a user