refactor(platforms): 平台能力改为子类实现,注册不再逐项绑定函数
- Platform 由 dataclass 字段绑定改为基类:id / label / entries 是类属性,能力是子类方法, 未实现的阶段保持基类 None,capabilities() 与 /api/platforms 输出不变 - gdgxk:session(原 new_session)/ login / list_courses / discover / study / _relogin 收进 class Gdgxk(Platform),解析与上报 helper 仍是模块级函数 - guanyixue:改成 class Guanyixue(Platform),方法转调 jspx / study - 删除 src/study_tool/pugins/ 草稿目录(内容与 platforms/gdgxk.py 重复, pugin.json 里按名字声明 study / entries 的绑定方式作废,元数据改由类属性承载) - 引擎 / 路由 / quiz / contract / 前端未改动 - 同步 docs/architecture.md、AGENTS.md、docs/OPTIMIZATION.md
This commit is contained in:
@@ -31,8 +31,8 @@
|
|||||||
## 代码结构 / 平台插件
|
## 代码结构 / 平台插件
|
||||||
|
|
||||||
- 业务代码都在 `src/study_tool/` 包里;根目录只放启动壳 `app.py`、`pyproject.toml`、`static/`、`data/`、`script/` 这类运行时资源。散落的 JS 脚本统一收进 `script/`,不参与运行的参考脚本放 `script/reference/`。
|
- 业务代码都在 `src/study_tool/` 包里;根目录只放启动壳 `app.py`、`pyproject.toml`、`static/`、`data/`、`script/` 这类运行时资源。散落的 JS 脚本统一收进 `script/`,不参与运行的参考脚本放 `script/reference/`。
|
||||||
- 平台代码只能放在 `src/study_tool/platforms/` 下;一个平台只注册一次(`Platform` + `registry.register`),并在 `Platform.entries` 里声明登录 / 学习入口。同一平台的不同页面代际可以拆成实现模块(如莞易学的 `jspx.py` / `study.py`),由平台模块按 `course["generation"]` 分发。
|
- 平台代码只能放在 `src/study_tool/platforms/` 下;一个平台 = 一个 `Platform` 子类(类属性 `id` / `label` / `entries`),能力只能通过「在子类里实现同名方法」绑定,禁止再写 `Platform(id=..., study=..., session=...)` 这种逐项传函数的注册;模块末尾 `PLATFORM = registry.register(<子类>())`,一个平台只注册一次。同一平台的不同页面代际可以拆成实现模块(如莞易学的 `jspx.py` / `study.py`),由平台模块按 `course["generation"]` 分发。
|
||||||
- 新增平台的标准步骤:写 `platforms/<名字>.py`(实现 `study(key, s, d, **opts)`、声明 `entries`,需要登录 / 课程列表 / 发现时再实现对应可选入口)→ 在 `platforms/__init__.py` 的 `load_builtin()` 里 import → 结束。不要改 `engine.py`、`routes/`、前端。
|
- 新增平台的标准步骤:写 `platforms/<名字>.py`(`class X(Platform)`,声明 `id` / `label` / `entries`,实现 `study(self, key, s, d, **opts)`,需要登录 / 课程列表 / 发现时再实现 `session` / `dump_cookies` / `login` / `list_courses` / `discover` / `renew_session` / `profile` 对应方法,没有的阶段不写)→ 在 `platforms/__init__.py` 的 `load_builtin()` 里 import → 结束。不要改 `engine.py`、`routes/`、前端。
|
||||||
- 引擎、路由、前端都不允许直接 import 某个具体平台模块;需要平台能力时用 `registry.get(course["platform"])`;账号级操作(登录 / 刷新课程 / 发现课程 / 会话)按 `accounts.platform` 用 `registry.get()` 取 `Platform` 上的 `session` / `dump_cookies` / `login` / `list_courses` / `discover` / `renew_session` / `profile`;「添加账号」的平台下拉来自 `GET /api/platforms`。
|
- 引擎、路由、前端都不允许直接 import 某个具体平台模块;需要平台能力时用 `registry.get(course["platform"])`;账号级操作(登录 / 刷新课程 / 发现课程 / 会话)按 `accounts.platform` 用 `registry.get()` 取 `Platform` 上的 `session` / `dump_cookies` / `login` / `list_courses` / `discover` / `renew_session` / `profile`;「添加账号」的平台下拉来自 `GET /api/platforms`。
|
||||||
- 前端源码在 `web/`(React + TypeScript + Vite + Tailwind CSS),构建产物输出到 `static/` 并提交进仓库;改完前端必须执行 `cd web && pnpm build`,不要直接改 `static/` 里的产物。
|
- 前端源码在 `web/`(React + TypeScript + Vite + Tailwind CSS),构建产物输出到 `static/` 并提交进仓库;改完前端必须执行 `cd web && pnpm build`,不要直接改 `static/` 里的产物。
|
||||||
- 新代码的 import 一律显式(如 `from study_tool.config import JSPX`),不要再新增 `from xxx import *`;历史遗留的 star import 只保留在拆分前就存在的模块里。
|
- 新代码的 import 一律显式(如 `from study_tool.config import JSPX`),不要再新增 `from xxx import *`;历史遗留的 star import 只保留在拆分前就存在的模块里。
|
||||||
|
|||||||
@@ -2,6 +2,28 @@
|
|||||||
|
|
||||||
已实现、但还需要你实测确认的事项。确认没问题后把对应条目删掉。
|
已实现、但还需要你实测确认的事项。确认没问题后把对应条目删掉。
|
||||||
|
|
||||||
|
## 平台契约改为子类实现(注册方式调整)
|
||||||
|
|
||||||
|
**状态**:已实现,待实测确认。
|
||||||
|
|
||||||
|
### 做了什么
|
||||||
|
|
||||||
|
- `Platform` 从 dataclass「注册时逐项传函数」改成基类:`id` / `label` / `entries` 是类属性,
|
||||||
|
能力是子类方法;没实现的阶段保持基类的 `None`,`capabilities()` 与 `/api/platforms` 不变。
|
||||||
|
- `platforms/gdgxk.py`:`session`(原 `new_session`)/ `login` / `list_courses` / `discover` /
|
||||||
|
`study` 等收进 `class Gdgxk(Platform)`;解析 / 上报类 helper 仍是模块级函数。
|
||||||
|
- `platforms/guanyixue.py`:改成 `class Guanyixue(Platform)`,方法转调 `jspx` / `study`。
|
||||||
|
- 删掉 `src/study_tool/pugins/` 草稿目录:它和 `platforms/gdgxk.py` 内容重复,
|
||||||
|
`pugin.json` 里「按名字声明 `study` / `entries`」的绑定方式作废,元数据改为类属性。
|
||||||
|
- 引擎 / 路由 / `quiz.py` / 前端 / `platforms/contract.py` 均未改动。
|
||||||
|
|
||||||
|
### 怎么测
|
||||||
|
|
||||||
|
1. `GET /api/platforms` 仍返回 `gdgxk` 与 `guanyixue`,`capabilities` 与改动前一致
|
||||||
|
(广东公需课:`login` / `session` / `renew` / `profile` / `courses` / `discover` / `study`)。
|
||||||
|
2. 莞易学:刷新课程 → 发现课程 → 开始学习照常(旧版 / 新版都跑一次)。
|
||||||
|
3. 广东公需课:登录 → 刷新课程 → 开始学习照常。
|
||||||
|
|
||||||
## 平台流程下沉(契约第二步)
|
## 平台流程下沉(契约第二步)
|
||||||
|
|
||||||
**状态**:已实现,待实测确认。
|
**状态**:已实现,待实测确认。
|
||||||
|
|||||||
+34
-25
@@ -61,24 +61,24 @@ pf = registry.get(d.get("platform"))
|
|||||||
pf.study(key, s, d, workers=..., cap=..., enable_quiz=..., ...)
|
pf.study(key, s, d, workers=..., cap=..., enable_quiz=..., ...)
|
||||||
```
|
```
|
||||||
|
|
||||||
`Platform` 契约(见 `platforms/base.py`):
|
`Platform` 契约(见 `platforms/base.py`):一个平台 = 一个 `Platform` 子类,类属性放元数据,
|
||||||
|
自己确实有的阶段实现成方法;注册时只实例化子类,不再逐项绑定函数。
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@dataclass
|
|
||||||
class Platform:
|
class Platform:
|
||||||
id: str # course["platform"] 的取值
|
id = "" # course["platform"] 的取值
|
||||||
label: str # 日志 / 界面展示名,如「莞易学」
|
label = "" # 日志 / 界面展示名,如「莞易学」
|
||||||
study: Callable # (key, s, d, **opts) -> None
|
entries = () # 登录入口 / 学习入口的 URL 片段
|
||||||
entries: tuple[str, ...] # 登录入口 / 学习入口的 URL 片段
|
# 槽位:子类实现哪个阶段就覆盖成方法(第一个参数是 self);保持 None = 该平台没有这项能力
|
||||||
# 账号级能力:添加账号时选中的平台决定登录 / 刷新课程 / 发现课程走哪套实现
|
study = None # study(self, key, s, d, **opts) -> None
|
||||||
session: Callable # (cookies) -> requests.Session
|
session = None # session(self, cookies) -> requests.Session
|
||||||
dump_cookies: Callable # (session) -> [cookie]
|
dump_cookies = None # dump_cookies(self, session) -> [cookie]
|
||||||
login: Callable # (username, password) -> (session, user)
|
login = None # login(self, username, password) -> (session, user)
|
||||||
list_courses: Callable # (session) -> [course]
|
list_courses = None # list_courses(self, session) -> [course]
|
||||||
discover: Callable # (session, plan_id, project_id, name) -> 课程描述
|
discover = None # discover(self, session, plan_id, project_id, name) -> 课程描述
|
||||||
renew_session: Callable # (session) -> bool,无密码续期
|
renew_session = None # renew_session(self, session) -> bool,无密码续期
|
||||||
profile: Callable # (session) -> 显示名
|
profile = None # profile(self, session) -> 显示名
|
||||||
# 槽位为 None = 该平台没有这项能力;capabilities() 按非空槽位推导
|
# capabilities() 按非空槽位推导能力名
|
||||||
```
|
```
|
||||||
|
|
||||||
`registry.detect(url, html)` 按 `entries` 命中平台;「添加账号」时选择的平台写进
|
`registry.detect(url, html)` 按 `entries` 命中平台;「添加账号」时选择的平台写进
|
||||||
@@ -130,19 +130,28 @@ quiz_key / cookies / plan / project / name`。插件可以只取自己需要的
|
|||||||
from study_tool.platforms import registry
|
from study_tool.platforms import registry
|
||||||
from study_tool.platforms.base import Platform
|
from study_tool.platforms.base import Platform
|
||||||
|
|
||||||
def _study(key, s, d, **opts):
|
class MyPlatform(Platform):
|
||||||
# 用 s(requests.Session,带该账号 cookie)把课程活动跑完
|
id = "<平台 id>"
|
||||||
# 日志一律 runtime.log(key, ...)
|
label = "展示名"
|
||||||
...
|
entries = ("<登录入口域名>", "<学习入口域名>")
|
||||||
|
|
||||||
PLATFORM = registry.register(Platform(
|
def session(self, cookies=None): ...
|
||||||
id="<平台 id>", label="展示名", study=_study,
|
def dump_cookies(self, s): ...
|
||||||
entries=("<登录入口域名>", "<学习入口域名>"),
|
def login(self, username, password): ...
|
||||||
session=<new_session>, dump_cookies=<dump_cookies>, login=<login>,
|
def list_courses(self, s): ...
|
||||||
list_courses=<list_courses>, discover=<discover>,
|
def discover(self, s, plan_id, project_id, name=""): ...
|
||||||
))
|
def profile(self, s): ...
|
||||||
|
|
||||||
|
def study(self, key, s, d, **opts):
|
||||||
|
# 用 s(requests.Session,带该账号 cookie)把课程活动跑完
|
||||||
|
# 日志一律 runtime.log(key, ...)
|
||||||
|
...
|
||||||
|
|
||||||
|
PLATFORM = registry.register(MyPlatform())
|
||||||
```
|
```
|
||||||
|
|
||||||
|
平台确实没有的阶段不要写,保持基类的 `None`,引擎按能力判断走不走那个分支。
|
||||||
|
|
||||||
2. 在 `platforms/__init__.py` 的 `load_builtin()` 里加一行 `from study_tool.platforms import <名字>`。
|
2. 在 `platforms/__init__.py` 的 `load_builtin()` 里加一行 `from study_tool.platforms import <名字>`。
|
||||||
|
|
||||||
3. 平台注册后会自动出现在「添加账号」的平台下拉框;账号选了该平台,登录 / 刷新课程 /
|
3. 平台注册后会自动出现在「添加账号」的平台下拉框;账号选了该平台,登录 / 刷新课程 /
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
"""平台插件包。
|
"""平台插件包。
|
||||||
|
|
||||||
内置平台在 `load_builtin()` 里导入(导入即注册)。新增平台:
|
内置平台在 `load_builtin()` 里导入(导入即注册)。新增平台:
|
||||||
1. 新建 `platforms/<名字>.py`,实现 `Platform.study`、声明 `entries` 登录 / 学习入口,
|
1. 新建 `platforms/<名字>.py`,写一个 `Platform` 子类:类属性声明 `id` / `label` /
|
||||||
并 `registry.register(...)`;
|
`entries`,自己有的阶段实现成方法(见 `base.py`),末尾
|
||||||
|
`PLATFORM = registry.register(<子类>())`;
|
||||||
2. 在下面的 `load_builtin()` 里加一行 import;
|
2. 在下面的 `load_builtin()` 里加一行 import;
|
||||||
3. 路由 / 引擎不用改 —— 它们只认 `course["platform"]`。
|
3. 路由 / 引擎不用改 —— 它们只认 `course["platform"]`。
|
||||||
"""
|
"""
|
||||||
|
|||||||
@@ -1,41 +1,59 @@
|
|||||||
"""平台插件契约。
|
"""平台插件契约:一个平台 = 一个 `Platform` 子类。
|
||||||
|
|
||||||
新增一个平台 = 在 `platforms/` 下加一个模块,构造 `Platform` 并 `registry.register()`,
|
子类用类属性声明元数据,用**方法**实现自己确实有的阶段;没实现的阶段不要写,
|
||||||
然后把它加进 `platforms/__init__.py` 的 `load_builtin()`。引擎只按
|
保持基类的 `None`(`capabilities()` 按非空槽位推导,引擎按能力决定走不走这个分支):
|
||||||
`course["platform"]` 去注册表取实现,不再 import 具体平台。
|
|
||||||
|
class Gdgxk(Platform):
|
||||||
|
id = "gdgxk"
|
||||||
|
label = "广东公需课"
|
||||||
|
entries = ("jsglpt.gds.edu.cn", "jsxx.gds.edu.cn")
|
||||||
|
|
||||||
|
def session(self, cookies=None): ...
|
||||||
|
def login(self, username, password): ...
|
||||||
|
def study(self, key, s, d, **opts): ...
|
||||||
|
|
||||||
|
PLATFORM = registry.register(Gdgxk())
|
||||||
|
|
||||||
|
新增平台 = 在 `platforms/` 下加一个这样的模块,再把它加进 `platforms/__init__.py`
|
||||||
|
的 `load_builtin()`。引擎只按 `course["platform"]` 去注册表取实现,不 import 具体平台。
|
||||||
|
|
||||||
`entries` 声明登录入口 / 学习入口的 URL 片段,`registry.detect(url, html)` 用它识别
|
`entries` 声明登录入口 / 学习入口的 URL 片段,`registry.detect(url, html)` 用它识别
|
||||||
页面属于哪个平台;同一平台的不同代页面(如莞易学的旧版 / 新版)用 `course["generation"]`
|
页面属于哪个平台;同一平台的不同代页面(如莞易学的旧版 / 新版)用 `course["generation"]`
|
||||||
区分,由平台内部自行分发。
|
区分,由平台内部自行分发。
|
||||||
|
|
||||||
Platform.study 的签名约定:
|
槽位方法签名(第一个参数是 self,其余同自由函数):
|
||||||
|
|
||||||
study(key, s, d, **opts)
|
study(key, s, d, **opts)
|
||||||
key 运行 key(账号 id,日志也用它)
|
key 运行 key(账号 id,日志也用它)
|
||||||
s requests.Session(带该账号 cookie)
|
s requests.Session(带该账号 cookie)
|
||||||
d discover() 返回的课程描述(course/courseware/feedback/test/platform/generation/...)
|
d discover() 返回的课程描述(course/courseware/feedback/test/platform/generation/...)
|
||||||
opts workers / cap / enable_quiz / enable_feedback / mode / parallel /
|
opts workers / cap / enable_quiz / enable_feedback / mode / parallel /
|
||||||
quiz_key / cookies / plan / project / name
|
quiz_key / cookies / plan / project / name
|
||||||
|
session(cookies) -> requests.Session
|
||||||
|
dump_cookies(session) -> [cookie]
|
||||||
|
login(username, password) -> (session, user)
|
||||||
|
list_courses(session) -> [course]
|
||||||
|
discover(session, plan_id, project_id, name) -> 课程描述
|
||||||
|
renew_session(session) -> bool,无密码续期
|
||||||
|
profile(session) -> 显示名
|
||||||
"""
|
"""
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Callable, Optional
|
|
||||||
|
|
||||||
from study_tool.platforms.contract import capabilities_of
|
from study_tool.platforms.contract import capabilities_of
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class Platform:
|
class Platform:
|
||||||
id: str # course["platform"] 的取值
|
id = "" # course["platform"] 的取值
|
||||||
label: str # 日志 / 界面里的展示名
|
label = "" # 日志 / 界面里的展示名
|
||||||
study: Optional[Callable] = None # 学一门课;None = 这个平台还没实现
|
entries = () # 登录 / 学习入口 URL 片段,用于识别平台
|
||||||
entries: tuple = () # 登录 / 学习入口 URL 片段,用于识别平台
|
|
||||||
# 账号级能力:添加账号时选了哪个平台,登录 / 发现 / 刷新课程就走哪个平台的实现
|
# 槽位:子类实现哪个阶段就把它覆盖成方法;保持 None = 该平台没有这项能力
|
||||||
session: Optional[Callable] = None # (cookies) -> requests.Session
|
study = None # 学一门课;None = 这个平台还没实现
|
||||||
dump_cookies: Optional[Callable] = None # (session) -> [cookie]
|
session = None
|
||||||
login: Optional[Callable] = None # (username, password) -> (session, user)
|
dump_cookies = None
|
||||||
list_courses: Optional[Callable] = None # (session) -> [course]
|
login = None
|
||||||
discover: Optional[Callable] = None # (session, plan_id, project_id, name) -> 课程描述
|
list_courses = None
|
||||||
renew_session: Optional[Callable] = None # (session) -> bool,无密码续期
|
discover = None
|
||||||
profile: Optional[Callable] = None # (session) -> 显示名
|
renew_session = None
|
||||||
|
profile = None
|
||||||
|
|
||||||
def matches(self, url="", html=""):
|
def matches(self, url="", html=""):
|
||||||
"""登录入口或学习入口命中任意片段,就认为页面属于本平台。"""
|
"""登录入口或学习入口命中任意片段,就认为页面属于本平台。"""
|
||||||
|
|||||||
+158
-169
@@ -28,7 +28,6 @@ X = "https://jsxx.gds.edu.cn"
|
|||||||
PLATFORM_ID = "gdgxk"
|
PLATFORM_ID = "gdgxk"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def _xhr(s, method, url, **kw):
|
def _xhr(s, method, url, **kw):
|
||||||
"""这些上报接口必须带 Referer(指向课程页),否则直接 500。"""
|
"""这些上报接口必须带 Referer(指向课程页),否则直接 500。"""
|
||||||
kw.setdefault("headers", {})["X-Requested-With"] = "XMLHttpRequest"
|
kw.setdefault("headers", {})["X-Requested-With"] = "XMLHttpRequest"
|
||||||
@@ -44,23 +43,6 @@ def _b64(text):
|
|||||||
return base64.b64encode(str(text).encode("utf-8")).decode()
|
return base64.b64encode(str(text).encode("utf-8")).decode()
|
||||||
|
|
||||||
|
|
||||||
def new_session(cookies=None):
|
|
||||||
s = requests.Session()
|
|
||||||
s.headers.update({"User-Agent": UA, "Accept-Language": "zh-CN,zh;q=0.9"})
|
|
||||||
for c in (cookies or []):
|
|
||||||
if "gds.edu.cn" not in (c.get("domain") or ""):
|
|
||||||
continue
|
|
||||||
s.cookies.set(c["name"], c.get("value") or "",
|
|
||||||
domain=(c.get("domain") or "").lstrip("."),
|
|
||||||
path=c.get("path") or "/")
|
|
||||||
return s
|
|
||||||
|
|
||||||
|
|
||||||
def dump_cookies(s):
|
|
||||||
return [{"name": c.name, "value": c.value, "domain": c.domain, "path": c.path}
|
|
||||||
for c in s.cookies]
|
|
||||||
|
|
||||||
|
|
||||||
def _sso(s):
|
def _sso(s):
|
||||||
"""jsglpt 会话换 ITLMS 票,票按 `ITLMS-xxx||ticketId` 两段交给 jsxx。
|
"""jsglpt 会话换 ITLMS 票,票按 `ITLMS-xxx||ticketId` 两段交给 jsxx。
|
||||||
|
|
||||||
@@ -104,44 +86,6 @@ def _login_error(s):
|
|||||||
return (m.group(1).strip() if m else "")
|
return (m.group(1).strip() if m else "")
|
||||||
|
|
||||||
|
|
||||||
def login(username, password):
|
|
||||||
s = new_session()
|
|
||||||
s.get(G + "/login.jsp", timeout=30)
|
|
||||||
err = ""
|
|
||||||
for _ in range(5):
|
|
||||||
data = {
|
|
||||||
"loginUser.userName": _b64(username),
|
|
||||||
"loginUser.loginType": "normal",
|
|
||||||
"loginUser.password": _b64(password),
|
|
||||||
"captcha": _ocr().classification(_captcha(s)),
|
|
||||||
}
|
|
||||||
r = s.post(G + "/loginIn.do", data=data, allow_redirects=False, timeout=30)
|
|
||||||
if r.status_code == 302 and "/default.jsp" in (r.headers.get("location") or ""):
|
|
||||||
_sso(s)
|
|
||||||
return s, profile(s) or username
|
|
||||||
err = _login_error(s)
|
|
||||||
if "验证码" not in err: # 不是验证码的问题,重试没意义
|
|
||||||
break
|
|
||||||
raise RuntimeError("登录失败:%s" % (err or "账号或密码不对"))
|
|
||||||
|
|
||||||
|
|
||||||
def renew_session(s):
|
|
||||||
try:
|
|
||||||
_sso(s)
|
|
||||||
return True
|
|
||||||
except Exception:
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def profile(s):
|
|
||||||
html = s.get(G + "/teacher/index.do", timeout=30).text
|
|
||||||
m = re.search(r"你好[,,]?\s*([^\s<]+)\s*老师", re.sub(r"\s+", " ", html))
|
|
||||||
if m:
|
|
||||||
return m.group(1)
|
|
||||||
m = re.search(r'class="user"[^>]*>\s*<[^>]*>\s*([^\s<]{2,20})', html)
|
|
||||||
return m.group(1) if m else ""
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_courses(html):
|
def _parse_courses(html):
|
||||||
"""课程列表页:每个 `showCourse('id','名称')` 前面就是这张卡片的信息。"""
|
"""课程列表页:每个 `showCourse('id','名称')` 前面就是这张卡片的信息。"""
|
||||||
out = []
|
out = []
|
||||||
@@ -163,16 +107,6 @@ def _parse_courses(html):
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def list_courses(s):
|
|
||||||
html = s.get(X + "/uc/store/courseRegister", timeout=30).text
|
|
||||||
out = _parse_courses(html)
|
|
||||||
if not out:
|
|
||||||
if "logout" not in html:
|
|
||||||
raise _expired("学习平台会话已失效,请重新登录")
|
|
||||||
return []
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def parse_course(html):
|
def parse_course(html):
|
||||||
"""从学习页里抠出心跳需要的一切:aid、课件 id、md5Uid、要求时长、已看秒数。"""
|
"""从学习页里抠出心跳需要的一切:aid、课件 id、md5Uid、要求时长、已看秒数。"""
|
||||||
d = {}
|
d = {}
|
||||||
@@ -241,51 +175,11 @@ def _progress(s, cid):
|
|||||||
return {"total": total or len(acts), "done": done, "acts": acts}
|
return {"total": total or len(acts), "done": done, "acts": acts}
|
||||||
|
|
||||||
|
|
||||||
def discover(s, plan_id, project_id, name=""):
|
|
||||||
cid = plan_id
|
|
||||||
page = X + "/study/course/" + cid
|
|
||||||
s.headers["Referer"] = page
|
|
||||||
r = s.get(page + "?v=1", timeout=30)
|
|
||||||
if "loginBtn" in r.text and "md5Uid" not in r.text:
|
|
||||||
raise _expired("学习平台会话已失效,请重新登录")
|
|
||||||
# 一门课是「章 / 节 / 活动」的目录,不是单个视频;活动清单和完成情况都从进度页取
|
|
||||||
pr = _progress(s, cid)
|
|
||||||
if not pr["acts"]:
|
|
||||||
raise RuntimeError("进度页里没有找到活动列表,页面结构可能变了")
|
|
||||||
courseware = [{
|
|
||||||
"aid": a["aid"], "title": a["title"], "done": a["done"],
|
|
||||||
"href": X + "/" + a["aid"] + "/study/course/" + cid,
|
|
||||||
"total": 0, "viewTime": 0,
|
|
||||||
} for a in pr["acts"]]
|
|
||||||
return {
|
|
||||||
"user": profile(s),
|
|
||||||
"course": name or "公需课程",
|
|
||||||
"progress": "已完成 %d/%d 个活动" % (pr["done"], pr["total"]),
|
|
||||||
"info": page,
|
|
||||||
"courseware": courseware,
|
|
||||||
"feedback": [], "test": [],
|
|
||||||
"platform": PLATFORM_ID, "generation": "gxk",
|
|
||||||
"plan_id": cid, "project_id": "",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _stopped(key):
|
def _stopped(key):
|
||||||
with runtime.RUNS_LOCK:
|
with runtime.RUNS_LOCK:
|
||||||
return (runtime.RUNS.get(key) or {}).get("status") == "stopped"
|
return (runtime.RUNS.get(key) or {}).get("status") == "stopped"
|
||||||
|
|
||||||
|
|
||||||
def _relogin(key):
|
|
||||||
"""掉线后按库里存的账号密码重登,返回新会话。"""
|
|
||||||
acc = store.account_get(key) or {}
|
|
||||||
pw = store.account_password(key)
|
|
||||||
if not pw:
|
|
||||||
raise _expired("会话掉线,且没有保存密码,无法自动重连")
|
|
||||||
runtime.log(key, "会话掉线,正在用账号密码自动重新登录…", level="warn")
|
|
||||||
s, _ = login(acc.get("username") or "", pw)
|
|
||||||
runtime.log(key, "重新登录成功,继续学习")
|
|
||||||
return s
|
|
||||||
|
|
||||||
|
|
||||||
def _study_one(key, s, cw, speed, beat):
|
def _study_one(key, s, cw, speed, beat):
|
||||||
"""打开活动页 → 注册播放 → 1x 心跳;会话失效时抛 SessionExpired 交给上层重登。"""
|
"""打开活动页 → 注册播放 → 1x 心跳;会话失效时抛 SessionExpired 交给上层重登。"""
|
||||||
try:
|
try:
|
||||||
@@ -312,56 +206,6 @@ def _study_one(key, s, cw, speed, beat):
|
|||||||
runtime.log(key, "服务端已记 %d 秒,达到要求,等平台刷新完成状态" % cur, level="warn")
|
runtime.log(key, "服务端已记 %d 秒,达到要求,等平台刷新完成状态" % cur, level="warn")
|
||||||
|
|
||||||
|
|
||||||
def study(key, s, d, **opts):
|
|
||||||
"""按进度页的活动清单逐个心跳;平台只按真实时间放行,所以默认 1 倍速慢慢刷。"""
|
|
||||||
speed = float(opts.get("speed") or 1)
|
|
||||||
beat = int(opts.get("beat") or 60)
|
|
||||||
cid = d["plan_id"]
|
|
||||||
total = len(d["courseware"])
|
|
||||||
runtime.set_state(key, status="studying", total=total, done=0)
|
|
||||||
skipped = []
|
|
||||||
for i, cw in enumerate(d["courseware"], 1):
|
|
||||||
if _stopped(key):
|
|
||||||
runtime.log(key, "已按要求停止")
|
|
||||||
return
|
|
||||||
if cw.get("done"):
|
|
||||||
runtime.log(key, "活动 %d/%d 已完成,跳过:%s" % (i, total, cw["title"]))
|
|
||||||
runtime.set_state(key, done=i)
|
|
||||||
continue
|
|
||||||
for attempt in (1, 2):
|
|
||||||
try:
|
|
||||||
_study_one(key, s, cw, speed, beat)
|
|
||||||
break
|
|
||||||
except Exception as e:
|
|
||||||
expired = e.__class__.__name__ == "SessionExpired"
|
|
||||||
if expired and attempt == 1:
|
|
||||||
s = _relogin(key) # 掉线:重登后把当前活动重新走一遍
|
|
||||||
continue
|
|
||||||
if expired:
|
|
||||||
raise
|
|
||||||
runtime.log(key, "活动「%s」跳过:%s" % (cw["title"], e), level="warn")
|
|
||||||
skipped.append(cw["title"])
|
|
||||||
break
|
|
||||||
runtime.set_state(key, done=i)
|
|
||||||
# 以平台进度页为准收尾:视频秒数够 ≠ 课程完成,考核没做就是没完成
|
|
||||||
try:
|
|
||||||
pr = _progress(s, cid)
|
|
||||||
except Exception as e:
|
|
||||||
if e.__class__.__name__ != "SessionExpired":
|
|
||||||
raise
|
|
||||||
s = _relogin(key)
|
|
||||||
pr = _progress(s, cid)
|
|
||||||
act_total = pr["total"] or total
|
|
||||||
runtime.set_state(key, status="done", done=pr["done"], total=act_total)
|
|
||||||
if pr["done"] >= act_total:
|
|
||||||
runtime.log(key, "平台进度:%d/%d 个活动已完成" % (pr["done"], act_total))
|
|
||||||
else:
|
|
||||||
runtime.log(key, "平台进度:%d/%d 个活动已完成,剩余需人工处理%s"
|
|
||||||
% (pr["done"], act_total,
|
|
||||||
("(" + "、".join(skipped) + ")") if skipped else ""),
|
|
||||||
level="warn")
|
|
||||||
|
|
||||||
|
|
||||||
def _learn(key, s, cw, speed, beat):
|
def _learn(key, s, cw, speed, beat):
|
||||||
"""心跳上报。平台只按真实流逝时间放行(1x),所以目标值按墙钟推,不靠每拍累加。"""
|
"""心跳上报。平台只按真实流逝时间放行(1x),所以目标值按墙钟推,不靠每拍累加。"""
|
||||||
uid, total = cw["uid"], cw["total"]
|
uid, total = cw["uid"], cw["total"]
|
||||||
@@ -397,16 +241,161 @@ def _learn(key, s, cw, speed, beat):
|
|||||||
runtime.log(key, "课件完成:%s(%d 秒)" % (cw["title"], total))
|
runtime.log(key, "课件完成:%s(%d 秒)" % (cw["title"], total))
|
||||||
|
|
||||||
|
|
||||||
PLATFORM = registry.register(Platform(
|
class Gdgxk(Platform):
|
||||||
id=PLATFORM_ID,
|
"""广东公需课:jsglpt 登录(账号 / 密码都是 base64 + 图形验证码),jsxx 学习。
|
||||||
label="广东公需课",
|
|
||||||
study=study,
|
账号级能力都实现在这里;反馈表 / 测验这类平台没有的阶段不实现,保持基类的 None。
|
||||||
entries=("jsglpt.gds.edu.cn", "jsxx.gds.edu.cn"),
|
"""
|
||||||
session=new_session,
|
|
||||||
dump_cookies=dump_cookies,
|
id = PLATFORM_ID
|
||||||
login=login,
|
label = "广东公需课"
|
||||||
list_courses=list_courses,
|
entries = ("jsglpt.gds.edu.cn", "jsxx.gds.edu.cn")
|
||||||
discover=discover,
|
|
||||||
renew_session=renew_session,
|
def session(self, cookies=None):
|
||||||
profile=profile,
|
s = requests.Session()
|
||||||
))
|
s.headers.update({"User-Agent": UA, "Accept-Language": "zh-CN,zh;q=0.9"})
|
||||||
|
for c in (cookies or []):
|
||||||
|
if "gds.edu.cn" not in (c.get("domain") or ""):
|
||||||
|
continue
|
||||||
|
s.cookies.set(c["name"], c.get("value") or "",
|
||||||
|
domain=(c.get("domain") or "").lstrip("."),
|
||||||
|
path=c.get("path") or "/")
|
||||||
|
return s
|
||||||
|
|
||||||
|
def dump_cookies(self, s):
|
||||||
|
return [{"name": c.name, "value": c.value, "domain": c.domain, "path": c.path}
|
||||||
|
for c in s.cookies]
|
||||||
|
|
||||||
|
def login(self, username, password):
|
||||||
|
s = self.session()
|
||||||
|
s.get(G + "/login.jsp", timeout=30)
|
||||||
|
err = ""
|
||||||
|
for _ in range(5):
|
||||||
|
data = {
|
||||||
|
"loginUser.userName": _b64(username),
|
||||||
|
"loginUser.loginType": "normal",
|
||||||
|
"loginUser.password": _b64(password),
|
||||||
|
"captcha": _ocr().classification(_captcha(s)),
|
||||||
|
}
|
||||||
|
r = s.post(G + "/loginIn.do", data=data, allow_redirects=False, timeout=30)
|
||||||
|
if r.status_code == 302 and "/default.jsp" in (r.headers.get("location") or ""):
|
||||||
|
_sso(s)
|
||||||
|
return s, self.profile(s) or username
|
||||||
|
err = _login_error(s)
|
||||||
|
if "验证码" not in err: # 不是验证码的问题,重试没意义
|
||||||
|
break
|
||||||
|
raise RuntimeError("登录失败:%s" % (err or "账号或密码不对"))
|
||||||
|
|
||||||
|
def renew_session(self, s):
|
||||||
|
try:
|
||||||
|
_sso(s)
|
||||||
|
return True
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
def profile(self, s):
|
||||||
|
html = s.get(G + "/teacher/index.do", timeout=30).text
|
||||||
|
m = re.search(r"你好[,,]?\s*([^\s<]+)\s*老师", re.sub(r"\s+", " ", html))
|
||||||
|
if m:
|
||||||
|
return m.group(1)
|
||||||
|
m = re.search(r'class="user"[^>]*>\s*<[^>]*>\s*([^\s<]{2,20})', html)
|
||||||
|
return m.group(1) if m else ""
|
||||||
|
|
||||||
|
def list_courses(self, s):
|
||||||
|
html = s.get(X + "/uc/store/courseRegister", timeout=30).text
|
||||||
|
out = _parse_courses(html)
|
||||||
|
if not out:
|
||||||
|
if "logout" not in html:
|
||||||
|
raise _expired("学习平台会话已失效,请重新登录")
|
||||||
|
return []
|
||||||
|
return out
|
||||||
|
|
||||||
|
def discover(self, s, plan_id, project_id, name=""):
|
||||||
|
cid = plan_id
|
||||||
|
page = X + "/study/course/" + cid
|
||||||
|
s.headers["Referer"] = page
|
||||||
|
r = s.get(page + "?v=1", timeout=30)
|
||||||
|
if "loginBtn" in r.text and "md5Uid" not in r.text:
|
||||||
|
raise _expired("学习平台会话已失效,请重新登录")
|
||||||
|
# 一门课是「章 / 节 / 活动」的目录,不是单个视频;活动清单和完成情况都从进度页取
|
||||||
|
pr = _progress(s, cid)
|
||||||
|
if not pr["acts"]:
|
||||||
|
raise RuntimeError("进度页里没有找到活动列表,页面结构可能变了")
|
||||||
|
courseware = [{
|
||||||
|
"aid": a["aid"], "title": a["title"], "done": a["done"],
|
||||||
|
"href": X + "/" + a["aid"] + "/study/course/" + cid,
|
||||||
|
"total": 0, "viewTime": 0,
|
||||||
|
} for a in pr["acts"]]
|
||||||
|
return {
|
||||||
|
"user": self.profile(s),
|
||||||
|
"course": name or "公需课程",
|
||||||
|
"progress": "已完成 %d/%d 个活动" % (pr["done"], pr["total"]),
|
||||||
|
"info": page,
|
||||||
|
"courseware": courseware,
|
||||||
|
"feedback": [], "test": [],
|
||||||
|
"platform": PLATFORM_ID, "generation": "gxk",
|
||||||
|
"plan_id": cid, "project_id": "",
|
||||||
|
}
|
||||||
|
|
||||||
|
def study(self, key, s, d, **opts):
|
||||||
|
"""按进度页的活动清单逐个心跳;平台只按真实时间放行,所以默认 1 倍速慢慢刷。"""
|
||||||
|
speed = float(opts.get("speed") or 1)
|
||||||
|
beat = int(opts.get("beat") or 60)
|
||||||
|
cid = d["plan_id"]
|
||||||
|
total = len(d["courseware"])
|
||||||
|
runtime.set_state(key, status="studying", total=total, done=0)
|
||||||
|
skipped = []
|
||||||
|
for i, cw in enumerate(d["courseware"], 1):
|
||||||
|
if _stopped(key):
|
||||||
|
runtime.log(key, "已按要求停止")
|
||||||
|
return
|
||||||
|
if cw.get("done"):
|
||||||
|
runtime.log(key, "活动 %d/%d 已完成,跳过:%s" % (i, total, cw["title"]))
|
||||||
|
runtime.set_state(key, done=i)
|
||||||
|
continue
|
||||||
|
for attempt in (1, 2):
|
||||||
|
try:
|
||||||
|
_study_one(key, s, cw, speed, beat)
|
||||||
|
break
|
||||||
|
except Exception as e:
|
||||||
|
expired = e.__class__.__name__ == "SessionExpired"
|
||||||
|
if expired and attempt == 1:
|
||||||
|
s = self._relogin(key) # 掉线:重登后把当前活动重新走一遍
|
||||||
|
continue
|
||||||
|
if expired:
|
||||||
|
raise
|
||||||
|
runtime.log(key, "活动「%s」跳过:%s" % (cw["title"], e), level="warn")
|
||||||
|
skipped.append(cw["title"])
|
||||||
|
break
|
||||||
|
runtime.set_state(key, done=i)
|
||||||
|
# 以平台进度页为准收尾:视频秒数够 ≠ 课程完成,考核没做就是没完成
|
||||||
|
try:
|
||||||
|
pr = _progress(s, cid)
|
||||||
|
except Exception as e:
|
||||||
|
if e.__class__.__name__ != "SessionExpired":
|
||||||
|
raise
|
||||||
|
s = self._relogin(key)
|
||||||
|
pr = _progress(s, cid)
|
||||||
|
act_total = pr["total"] or total
|
||||||
|
runtime.set_state(key, status="done", done=pr["done"], total=act_total)
|
||||||
|
if pr["done"] >= act_total:
|
||||||
|
runtime.log(key, "平台进度:%d/%d 个活动已完成" % (pr["done"], act_total))
|
||||||
|
else:
|
||||||
|
runtime.log(key, "平台进度:%d/%d 个活动已完成,剩余需人工处理%s"
|
||||||
|
% (pr["done"], act_total,
|
||||||
|
("(" + "、".join(skipped) + ")") if skipped else ""),
|
||||||
|
level="warn")
|
||||||
|
|
||||||
|
def _relogin(self, key):
|
||||||
|
"""掉线后按库里存的账号密码重登,返回新会话。"""
|
||||||
|
acc = store.account_get(key) or {}
|
||||||
|
pw = store.account_password(key)
|
||||||
|
if not pw:
|
||||||
|
raise _expired("会话掉线,且没有保存密码,无法自动重连")
|
||||||
|
runtime.log(key, "会话掉线,正在用账号密码自动重新登录…", level="warn")
|
||||||
|
s, _ = self.login(acc.get("username") or "", pw)
|
||||||
|
runtime.log(key, "重新登录成功,继续学习")
|
||||||
|
return s
|
||||||
|
|
||||||
|
|
||||||
|
PLATFORM = registry.register(Gdgxk())
|
||||||
|
|||||||
@@ -2,31 +2,44 @@
|
|||||||
|
|
||||||
账号在「添加账号」里选择平台(`accounts.platform`),登录 / 刷新课程 / 发现课程都会
|
账号在「添加账号」里选择平台(`accounts.platform`),登录 / 刷新课程 / 发现课程都会
|
||||||
走这里的 `login` / `list_courses` / `discover`;学习时 `discover` 会把页面代际写进
|
走这里的 `login` / `list_courses` / `discover`;学习时 `discover` 会把页面代际写进
|
||||||
`course["generation"]`(legacy / spa),由 `_study` 分发到对应实现。
|
`course["generation"]`(legacy / spa),由 `study` 分发到对应实现。
|
||||||
新增其他平台时,照本文件写一个模块并在 `Platform.entries` 声明登录 / 学习入口。
|
新增其他平台时,照本文件写一个 `Platform` 子类并在 `entries` 声明登录 / 学习入口。
|
||||||
"""
|
"""
|
||||||
from study_tool.platforms import jspx, registry, study
|
from study_tool.platforms import jspx, registry, study
|
||||||
from study_tool.platforms.base import Platform
|
from study_tool.platforms.base import Platform
|
||||||
|
|
||||||
ID = "guanyixue"
|
|
||||||
LABEL = "莞易学"
|
class Guanyixue(Platform):
|
||||||
# 登录入口 / 学习入口;命中任意一个就归到莞易学
|
id = "guanyixue"
|
||||||
ENTRIES = ("jspx.dgjy.net", "study.dgjy.net", "dgjy.net")
|
label = "莞易学"
|
||||||
|
# 登录入口 / 学习入口;命中任意一个就归到莞易学
|
||||||
|
entries = ("jspx.dgjy.net", "study.dgjy.net", "dgjy.net")
|
||||||
|
|
||||||
|
def session(self, cookies=None):
|
||||||
|
return jspx.new_session(cookies)
|
||||||
|
|
||||||
|
def dump_cookies(self, s):
|
||||||
|
return jspx.dump_cookies(s)
|
||||||
|
|
||||||
|
def login(self, username, password):
|
||||||
|
return jspx.login(username, password)
|
||||||
|
|
||||||
|
def list_courses(self, s):
|
||||||
|
return jspx.scrape_courses(s)
|
||||||
|
|
||||||
|
def discover(self, s, plan_id, project_id, name=""):
|
||||||
|
return jspx.discover(s, plan_id, project_id, name)
|
||||||
|
|
||||||
|
def renew_session(self, s):
|
||||||
|
return jspx.ensure_jspx_session(s)
|
||||||
|
|
||||||
|
def profile(self, s):
|
||||||
|
return jspx.whoami(jspx.fetch_admin(s))
|
||||||
|
|
||||||
|
def study(self, key, s, d, **opts):
|
||||||
|
if d.get("generation") == "spa":
|
||||||
|
return study._study(key, s, d, **opts)
|
||||||
|
return jspx._study(key, s, d, **opts)
|
||||||
|
|
||||||
|
|
||||||
def _profile(s):
|
PLATFORM = registry.register(Guanyixue())
|
||||||
return jspx.whoami(jspx.fetch_admin(s))
|
|
||||||
|
|
||||||
|
|
||||||
def _study(key, s, d, **opts):
|
|
||||||
if d.get("generation") == "spa":
|
|
||||||
return study._study(key, s, d, **opts)
|
|
||||||
return jspx._study(key, s, d, **opts)
|
|
||||||
|
|
||||||
|
|
||||||
PLATFORM = registry.register(Platform(
|
|
||||||
id=ID, label=LABEL, study=_study, entries=ENTRIES,
|
|
||||||
session=jspx.new_session, dump_cookies=jspx.dump_cookies,
|
|
||||||
login=jspx.login, list_courses=jspx.scrape_courses, discover=jspx.discover,
|
|
||||||
renew_session=jspx.ensure_jspx_session, profile=_profile,
|
|
||||||
))
|
|
||||||
|
|||||||
Reference in New Issue
Block a user