Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0543694c1e | ||
|
|
8fbabd3c11 |
@@ -0,0 +1,49 @@
|
|||||||
|
---
|
||||||
|
description: MapMil platform architecture, boundaries, and agent workflow
|
||||||
|
alwaysApply: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# MapMil Platform
|
||||||
|
|
||||||
|
Monorepo: **ЦА** (analytics) + **ЦП** (parsing) + **contracts** + **ПИ** (distribution API).
|
||||||
|
|
||||||
|
## Data flow
|
||||||
|
|
||||||
|
```
|
||||||
|
CA admin → Redis cp:jobs:{family} → CP adapter → POST /internal/ingest → CA DB → map / ПИ
|
||||||
|
```
|
||||||
|
|
||||||
|
| Center | Path | Role |
|
||||||
|
|--------|------|------|
|
||||||
|
| ЦА API | `centers/analytics/api/` | FastAPI, ingest, scheduler, admin |
|
||||||
|
| ЦА UI | `centers/analytics/frontend/` | Vue + Leaflet admin |
|
||||||
|
| ЦП | `centers/parsing/workers/` | Adapter registry, batch + Telegram listener |
|
||||||
|
| Contracts | `contracts/` | Shared schemas — change first |
|
||||||
|
|
||||||
|
## Hard boundaries
|
||||||
|
|
||||||
|
- ЦП **never** touches CA database — only `POST /internal/ingest` and `PATCH /internal/jobs/{id}`
|
||||||
|
- `source_type` events must match adapter type; stable `source_url` required (CA dedup)
|
||||||
|
- `source_config` validated via `contracts/sources.py` on admin CRUD
|
||||||
|
- Telegram session: `data/telegram.session` — local only, **never commit**
|
||||||
|
- `.env` and secrets — **never commit**
|
||||||
|
|
||||||
|
## CP adapter model
|
||||||
|
|
||||||
|
Each `source_type` implements `SourceAdapter.run(job_id, source_config, *, ctx) -> tuple[list[dict], str | None]`.
|
||||||
|
|
||||||
|
Output dicts are `IngestEventItem`-shaped. Routing: `contracts/queues.py` (`SOURCE_FAMILY` → `cp:jobs:{telegram|web|nlp}`).
|
||||||
|
|
||||||
|
Worker images: `cp-workers` (telegram), `cp-workers-web` (crawl4ai), `cp-workers-nlp` (viina).
|
||||||
|
|
||||||
|
## Agent workflow
|
||||||
|
|
||||||
|
1. **Contracts first** if data shape or routing changes
|
||||||
|
2. **One zone per task** — do not mix contracts + UI + infra in one unfocused change
|
||||||
|
3. **Follow existing adapters** — copy `viina.py` or `telegram.py`, not greenfield
|
||||||
|
4. **Do not touch** Telegram listener unless explicitly asked
|
||||||
|
5. **Verify**: `docker compose config`, then targeted service rebuild
|
||||||
|
|
||||||
|
Docs: `README.md`, `centers/parsing/ARCHITECTURE.md`, `AGENTS.md`.
|
||||||
|
|
||||||
|
For new `source_type`: use skill `add-parser-adapter`.
|
||||||
@@ -0,0 +1,142 @@
|
|||||||
|
---
|
||||||
|
name: add-parser-adapter
|
||||||
|
description: >-
|
||||||
|
Adds a new CP source_type end-to-end in MapMil: contracts schema and queue
|
||||||
|
family, SourceAdapter implementation, registry, optional Docker worker image,
|
||||||
|
and ParsersView.vue form fields. Use when adding a parser source, adapter,
|
||||||
|
source_type, or new ingest channel to the parsing center.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Add Parser Adapter
|
||||||
|
|
||||||
|
End-to-end checklist for a new `source_type` in ЦП. Work in order; do not skip contracts.
|
||||||
|
|
||||||
|
## Inputs to clarify
|
||||||
|
|
||||||
|
- `source_type` slug (lowercase, e.g. `rss`)
|
||||||
|
- Worker **family** (`telegram` | `web` | `nlp`) — pick by deps:
|
||||||
|
- `telegram` — Telethon, shared session
|
||||||
|
- `web` — browser/HTTP heavy (Crawl4AI, Playwright)
|
||||||
|
- `nlp` — lightweight HTTP + text (VIINA-style)
|
||||||
|
- `source_config` fields for admin UI
|
||||||
|
- Whether existing worker image can host it or needs new `requirements-*.txt` service
|
||||||
|
|
||||||
|
## Checklist
|
||||||
|
|
||||||
|
### 1. Contracts
|
||||||
|
|
||||||
|
**`contracts/sources.py`**
|
||||||
|
|
||||||
|
- Add `{Name}SourceConfig(BaseModel)` with validators
|
||||||
|
- Register in `CONFIG_MODELS["{source_type}"]`
|
||||||
|
|
||||||
|
**`contracts/queues.py`**
|
||||||
|
|
||||||
|
- Add `SOURCE_FAMILY["{source_type}"] = "{family}"`
|
||||||
|
|
||||||
|
No CA code changes needed for routing — `enqueue_job` uses `queue_key_for_source`.
|
||||||
|
|
||||||
|
### 2. Adapter
|
||||||
|
|
||||||
|
**`centers/parsing/workers/workers/adapters/{name}.py`**
|
||||||
|
|
||||||
|
Copy pattern from `viina.py` (text/HTTP) or `telegram.py` (Telethon):
|
||||||
|
|
||||||
|
```python
|
||||||
|
class MyAdapter:
|
||||||
|
source_type = "{source_type}"
|
||||||
|
|
||||||
|
async def run(self, job_id, source_config, *, ctx: WorkerContext) -> tuple[list[dict], str | None]:
|
||||||
|
cfg = MySourceConfig.model_validate(source_config or {})
|
||||||
|
# fetch / parse → list of IngestEventItem-shaped dicts
|
||||||
|
return events, None # or ([], "error message")
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- Validate config with Pydantic model from `contracts/sources`
|
||||||
|
- Every event: stable `source_url`, `source_type="{source_type}"`
|
||||||
|
- Optional fields: `title`, `description`, `locality`, `latitude`, `longitude`, `event_date`, `region`, `topic`, `tags`, `metadata`
|
||||||
|
- Return `([], error)` on fatal config errors; partial fetch errors can go in metadata
|
||||||
|
|
||||||
|
**`workers/adapters/registry.py`**
|
||||||
|
|
||||||
|
- Add lazy factory in `_factories()`
|
||||||
|
- Factory name key must match `source_type`
|
||||||
|
|
||||||
|
### 3. Dependencies & Docker
|
||||||
|
|
||||||
|
If adapter fits an existing family, only update `ENABLED_ADAPTERS` in compose (usually not needed — one adapter per service).
|
||||||
|
|
||||||
|
If new deps are heavy or conflict:
|
||||||
|
|
||||||
|
1. `centers/parsing/workers/requirements-{family}.txt` (or new file)
|
||||||
|
2. New or updated service in `docker-compose.yml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
cp-workers-{family}:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: centers/parsing/workers/Dockerfile
|
||||||
|
args:
|
||||||
|
REQUIREMENTS_FILE: requirements-{family}.txt
|
||||||
|
INSTALL_PLAYWRIGHT: "0" # "1" only for browser adapters
|
||||||
|
environment:
|
||||||
|
ENABLED_ADAPTERS: {source_type}
|
||||||
|
WORKER_FAMILIES: {family}
|
||||||
|
CA_API_URL: http://ca-api:8000
|
||||||
|
REDIS_URL: redis://redis:6379/0
|
||||||
|
INTERNAL_TOKEN: dev-internal-token
|
||||||
|
depends_on: [redis, ca-api]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Admin UI
|
||||||
|
|
||||||
|
**`centers/analytics/frontend/src/views/ParsersView.vue`**
|
||||||
|
|
||||||
|
- Add option to source type `<select>`
|
||||||
|
- Extend form state + `buildSourceConfig()` for new fields
|
||||||
|
- Add create/edit `<template>` blocks (mirror `crawl4ai` or `viina`)
|
||||||
|
- Extend validation in submit handler
|
||||||
|
|
||||||
|
TypeScript unions: update `"telegram" | "crawl4ai" | "viina"` wherever used.
|
||||||
|
|
||||||
|
No CA router changes if `source_type` is registered in `CONFIG_MODELS` — `_validated_source_config` picks it up automatically.
|
||||||
|
|
||||||
|
### 5. Documentation
|
||||||
|
|
||||||
|
- Add row to adapter table in `centers/parsing/ARCHITECTURE.md`
|
||||||
|
- One-line mention in `README.md` if user-facing
|
||||||
|
|
||||||
|
### 6. Verify
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose config
|
||||||
|
docker compose up --build cp-workers-{family} # or relevant service
|
||||||
|
curl -X POST http://localhost:8080/admin/jobs \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"source_type":"{source_type}","source_config":{...},"interval_seconds":3600}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Check CA logs for ingest and `/admin/events` for new rows.
|
||||||
|
|
||||||
|
## Reference files
|
||||||
|
|
||||||
|
| Concern | File |
|
||||||
|
|---------|------|
|
||||||
|
| Config schema | `contracts/sources.py` |
|
||||||
|
| Queue routing | `contracts/queues.py` |
|
||||||
|
| Ingest shape | `contracts/ingest.py` |
|
||||||
|
| Adapter protocol | `workers/adapters/base.py` |
|
||||||
|
| Simple adapter | `workers/adapters/viina.py` |
|
||||||
|
| LLM adapter | `workers/adapters/crawl4ai_adapter.py` |
|
||||||
|
| Telegram + listener | `workers/adapters/telegram.py` |
|
||||||
|
| Enqueue | `centers/analytics/api/app/services/jobs.py` |
|
||||||
|
| Admin validation | `centers/analytics/api/app/routers/admin.py` |
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- Implementing adapter before `CONFIG_MODELS` entry
|
||||||
|
- Hardcoding Redis queue names — use `queue_key_for_source`
|
||||||
|
- Skipping `source_url` stability (breaks dedup)
|
||||||
|
- Importing heavy deps at module top in `registry.py` — use lazy factories
|
||||||
|
- Editing Telegram listener when adding unrelated sources
|
||||||
@@ -11,3 +11,12 @@ TELEGRAM_SESSION_PATH=/data/telegram.session
|
|||||||
# Platform internals
|
# Platform internals
|
||||||
INTERNAL_TOKEN=dev-internal-token
|
INTERNAL_TOKEN=dev-internal-token
|
||||||
TEST_PI_API_KEY=test-pi-api-key-change-me
|
TEST_PI_API_KEY=test-pi-api-key-change-me
|
||||||
|
|
||||||
|
# DeepSeek LLM for extract_mode=llm (Telegram unstructured + Crawl4AI LLM)
|
||||||
|
# DEEPSEEK_API_KEY=sk-...
|
||||||
|
# DEEPSEEK_BASE_URL=https://api.deepseek.com
|
||||||
|
# DEEPSEEK_MODEL=deepseek-chat
|
||||||
|
|
||||||
|
# CP adapter workers (set in docker-compose; override locally if needed)
|
||||||
|
# ENABLED_ADAPTERS=telegram
|
||||||
|
# WORKER_FAMILIES=telegram
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
.env
|
.env
|
||||||
data/
|
data/
|
||||||
|
mapmil-secrets.tar.gz
|
||||||
__pycache__/
|
__pycache__/
|
||||||
*.pyc
|
*.pyc
|
||||||
node_modules/
|
node_modules/
|
||||||
|
|||||||
@@ -0,0 +1,100 @@
|
|||||||
|
# MapMil — guide for AI agents
|
||||||
|
|
||||||
|
Quick orientation for Cursor and other coding agents working in this repository.
|
||||||
|
|
||||||
|
## What this is
|
||||||
|
|
||||||
|
**MapMil** is a monorepo platform: parsing center (ЦП) collects events from external sources, analytics center (ЦА) stores them and serves a map + admin UI, external consumers (ПИ) read via distribution API.
|
||||||
|
|
||||||
|
```
|
||||||
|
ЦА (centers/analytics) ←→ Redis ←→ ЦП (centers/parsing)
|
||||||
|
↓ ingest ↑ adapters
|
||||||
|
PostgreSQL + map UI Telegram / Crawl4AI / VIINA
|
||||||
|
↓
|
||||||
|
GET /api/v1/events → ПИ
|
||||||
|
```
|
||||||
|
|
||||||
|
## Repository map
|
||||||
|
|
||||||
|
| Path | Stack | Purpose |
|
||||||
|
|------|-------|---------|
|
||||||
|
| `contracts/` | Pydantic | `ingest`, `jobs`, `sources`, `queues` — shared truth |
|
||||||
|
| `centers/analytics/api/` | FastAPI, SQLAlchemy | Admin API, ingest, scheduler, map queries |
|
||||||
|
| `centers/analytics/frontend/` | Vue 3, Leaflet | Map, parsers, events, analytics, consumers |
|
||||||
|
| `centers/parsing/workers/` | Python async | CP workers, adapter registry, LLM extract |
|
||||||
|
| `docker-compose.yml` | Docker | `ca-*`, `cp-workers*`, `redis` |
|
||||||
|
| `data/` | Telethon session | **Gitignored** — do not commit |
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp .env.example .env # then set TELEGRAM_API_ID/HASH, optional DEEPSEEK_API_KEY
|
||||||
|
docker compose up --build # full stack → http://localhost:8080
|
||||||
|
docker compose config # validate compose
|
||||||
|
docker compose up --build cp-workers-web # single worker rebuild
|
||||||
|
```
|
||||||
|
|
||||||
|
Health: `curl http://localhost:8080/api/health`
|
||||||
|
|
||||||
|
Create Telegram parser:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8080/admin/jobs \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"source_type":"telegram","source_config":{"channel":"example","limit":50}}'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
### Contracts-first
|
||||||
|
|
||||||
|
Any change to event shape, `source_config`, or queue routing starts in `contracts/`, then CA/CP/UI.
|
||||||
|
|
||||||
|
### CP adapters
|
||||||
|
|
||||||
|
- Protocol: `workers/adapters/base.py` (`SourceAdapter`)
|
||||||
|
- Registry: `workers/adapters/registry.py` (`ENABLED_ADAPTERS`)
|
||||||
|
- Queue family: `contracts/queues.py` (`SOURCE_FAMILY`)
|
||||||
|
- Config schema: `contracts/sources.py` (`CONFIG_MODELS`)
|
||||||
|
|
||||||
|
### Ingest output
|
||||||
|
|
||||||
|
Adapters return dicts matching `IngestEventItem` (`contracts/ingest.py`). Required: `source_url`. Dedup in CA by `source_url`.
|
||||||
|
|
||||||
|
### LLM extract (runtime, not dev)
|
||||||
|
|
||||||
|
`extract_mode: llm` uses DeepSeek via `workers/llm_extract.py`. Key: `DEEPSEEK_API_KEY` in `.env`. Do not confuse with Cursor dev agents.
|
||||||
|
|
||||||
|
## Common tasks
|
||||||
|
|
||||||
|
| Task | Start here | Skill |
|
||||||
|
|------|------------|-------|
|
||||||
|
| New parser source | `contracts/sources.py` | `add-parser-adapter` |
|
||||||
|
| Admin API change | `centers/analytics/api/app/routers/` | — |
|
||||||
|
| Map / UI | `centers/analytics/frontend/src/` | — |
|
||||||
|
| New worker image | `docker-compose.yml` + `requirements-*.txt` | `add-parser-adapter` |
|
||||||
|
|
||||||
|
## Do not
|
||||||
|
|
||||||
|
- Commit `.env`, `data/`, or API keys
|
||||||
|
- Add direct DB access from CP workers
|
||||||
|
- Change `IngestEventItem` without checking ingest + map + ПИ impact
|
||||||
|
- Modify Telegram listener/session logic unless the task requires it
|
||||||
|
- Reintroduce legacy root `backend/` or `frontend/` (removed; use `centers/`)
|
||||||
|
|
||||||
|
## Cursor setup
|
||||||
|
|
||||||
|
- Rule: `.cursor/rules/platform.mdc` (always on)
|
||||||
|
- Skill: `.cursor/skills/add-parser-adapter/` for end-to-end new sources
|
||||||
|
- Deep docs: `README.md`, `centers/parsing/ARCHITECTURE.md`
|
||||||
|
|
||||||
|
## Commit style
|
||||||
|
|
||||||
|
Follow existing history — imperative, concise:
|
||||||
|
|
||||||
|
```
|
||||||
|
Add CP source adapter registry with multi-worker queues and LLM extract.
|
||||||
|
Add Telegram listener, scheduler, and extended parsers admin UI.
|
||||||
|
```
|
||||||
|
|
||||||
|
One logical change per commit; do not bundle unrelated zones.
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# MapMil Platform (ЦП → ЦА → ПИ)
|
# MapMil Platform (ЦП → ЦА → ПИ)
|
||||||
|
|
||||||
Единая платформа: ЦА (аналитика и карта) + ЦП (парсинг Telegram).
|
Единая платформа: ЦА (аналитика и карта) + ЦП (адаптеры парсинга: Telegram, Crawl4AI, VIINA).
|
||||||
|
|
||||||
## Архитектура
|
## Архитектура
|
||||||
|
|
||||||
@@ -10,7 +10,7 @@ flowchart LR
|
|||||||
CP[ЦП Parsing Center]
|
CP[ЦП Parsing Center]
|
||||||
PI[ПИ External Consumers]
|
PI[ПИ External Consumers]
|
||||||
|
|
||||||
CA -->|jobs via Redis| CP
|
CA -->|jobs via Redis families| CP
|
||||||
CP -->|POST /internal/ingest| CA
|
CP -->|POST /internal/ingest| CA
|
||||||
CA -->|GET /api/v1/events| PI
|
CA -->|GET /api/v1/events| PI
|
||||||
```
|
```
|
||||||
@@ -18,8 +18,10 @@ flowchart LR
|
|||||||
| Центр | Контейнеры | Назначение |
|
| Центр | Контейнеры | Назначение |
|
||||||
|-------|------------|------------|
|
|-------|------------|------------|
|
||||||
| **ЦА** | `ca-db`, `ca-api`, `ca-frontend` | PostgreSQL, ingest API, карта, distribution API |
|
| **ЦА** | `ca-db`, `ca-api`, `ca-frontend` | PostgreSQL, ingest API, карта, distribution API |
|
||||||
| **ЦП** | `cp-workers` | Парсинг Telegram: real-time listener (Telethon) + batch-задания из Redis |
|
| **ЦП** | `cp-workers`, `cp-workers-web`, `cp-workers-nlp` | Адаптеры: Telegram / Crawl4AI / VIINA |
|
||||||
| **Общее** | `redis` | Очередь заданий ЦА → ЦП |
|
| **Общее** | `redis` | Очереди `cp:jobs:{telegram\|web\|nlp}` |
|
||||||
|
|
||||||
|
Подробности ЦП: [`centers/parsing/ARCHITECTURE.md`](centers/parsing/ARCHITECTURE.md).
|
||||||
|
|
||||||
## Структура monorepo
|
## Структура monorepo
|
||||||
|
|
||||||
@@ -30,11 +32,12 @@ MapMil/
|
|||||||
│ │ ├── api/ # CA backend (FastAPI + PostgreSQL)
|
│ │ ├── api/ # CA backend (FastAPI + PostgreSQL)
|
||||||
│ │ └── frontend/ # CA admin UI (Vue + Leaflet)
|
│ │ └── frontend/ # CA admin UI (Vue + Leaflet)
|
||||||
│ └── parsing/
|
│ └── parsing/
|
||||||
│ └── workers/ # CP workers (Telethon)
|
│ ├── ARCHITECTURE.md
|
||||||
├── contracts/ # Shared schemas (ingest, jobs)
|
│ └── workers/ # CP workers + adapters
|
||||||
|
├── contracts/ # Shared schemas (ingest, jobs, sources, queues)
|
||||||
├── data/ # telegram.session (локально, не в git)
|
├── data/ # telegram.session (локально, не в git)
|
||||||
├── docker-compose.yml
|
├── docker-compose.yml
|
||||||
└── .env # TELEGRAM_API_ID, TELEGRAM_API_HASH, …
|
└── .env
|
||||||
```
|
```
|
||||||
|
|
||||||
## Быстрый старт
|
## Быстрый старт
|
||||||
@@ -67,7 +70,7 @@ docker compose up --build
|
|||||||
| Раздел | Путь | Описание |
|
| Раздел | Путь | Описание |
|
||||||
|--------|------|----------|
|
|--------|------|----------|
|
||||||
| **Карта** | `/` | Интерактивная карта событий: навигация по датам (flatpickr), пресеты периода, фильтры региона/темы/источника, подложки Яндекс/OSM/Topo/ESRI, линейка, полноэкранный режим, центрирование по координатам и городам, поиск населённых пунктов (Nominatim). CRUD для ручных объектов (ПКМ). Поддерживает `?eventId=` |
|
| **Карта** | `/` | Интерактивная карта событий: навигация по датам (flatpickr), пресеты периода, фильтры региона/темы/источника, подложки Яндекс/OSM/Topo/ESRI, линейка, полноэкранный режим, центрирование по координатам и городам, поиск населённых пунктов (Nominatim). CRUD для ручных объектов (ПКМ). Поддерживает `?eventId=` |
|
||||||
| **Парсеры** | `/parsers` | Telegram-парсеры с периодическим запуском, настройка интервала, редактирование и удаление; дубликаты по `source_url` не записываются |
|
| **Парсеры** | `/parsers` | Адаптеры `telegram` / `crawl4ai` / `viina`, интервал, CRUD; дедуп по `source_url` |
|
||||||
| **События** | `/events` | Фильтрация, пагинация, просмотр деталей, ссылка «На карте» для событий с координатами |
|
| **События** | `/events` | Фильтрация, пагинация, просмотр деталей, ссылка «На карте» для событий с координатами |
|
||||||
| **Аналитика** | `/analytics` | KPI-карточки, график динамики ingest за 30 дней, топ населённых пунктов и регионов |
|
| **Аналитика** | `/analytics` | KPI-карточки, график динамики ingest за 30 дней, топ населённых пунктов и регионов |
|
||||||
| **ПИ** | `/consumers` | CRUD подписчиков distribution API, ротация ключей, тест среза через `/api/v1/events` |
|
| **ПИ** | `/consumers` | CRUD подписчиков distribution API, ротация ключей, тест среза через `/api/v1/events` |
|
||||||
@@ -198,7 +201,3 @@ docker compose down
|
|||||||
```
|
```
|
||||||
|
|
||||||
Данные PostgreSQL сохраняются в volume `pgdata`.
|
Данные PostgreSQL сохраняются в volume `pgdata`.
|
||||||
|
|
||||||
## Legacy
|
|
||||||
|
|
||||||
Старые каталоги `backend/` и `frontend/` в корне оставлены для справки; активная разработка — в `centers/`.
|
|
||||||
|
|||||||
@@ -1,14 +0,0 @@
|
|||||||
FROM python:3.12-slim
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
|
|
||||||
COPY requirements.txt .
|
|
||||||
RUN pip install --no-cache-dir -r requirements.txt
|
|
||||||
|
|
||||||
COPY app/ ./app/
|
|
||||||
|
|
||||||
RUN mkdir -p /data
|
|
||||||
|
|
||||||
EXPOSE 8000
|
|
||||||
|
|
||||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
import os
|
|
||||||
from sqlalchemy import create_engine
|
|
||||||
from sqlalchemy.orm import DeclarativeBase, sessionmaker
|
|
||||||
|
|
||||||
DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:////data/mapmil.db")
|
|
||||||
|
|
||||||
engine = create_engine(
|
|
||||||
DATABASE_URL,
|
|
||||||
connect_args={"check_same_thread": False},
|
|
||||||
)
|
|
||||||
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
|
||||||
|
|
||||||
|
|
||||||
class Base(DeclarativeBase):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def get_db():
|
|
||||||
db = SessionLocal()
|
|
||||||
try:
|
|
||||||
yield db
|
|
||||||
finally:
|
|
||||||
db.close()
|
|
||||||
@@ -1,195 +0,0 @@
|
|||||||
from contextlib import asynccontextmanager
|
|
||||||
|
|
||||||
from fastapi import Depends, FastAPI, File, HTTPException, UploadFile
|
|
||||||
from fastapi.middleware.cors import CORSMiddleware
|
|
||||||
from fastapi.responses import FileResponse
|
|
||||||
from sqlalchemy.orm import Session
|
|
||||||
|
|
||||||
from .database import Base, engine, get_db
|
|
||||||
from .models import MapObject, ObjectMedia
|
|
||||||
from .schemas import MapObjectCreate, MapObjectRead, MapObjectUpdate, ObjectMediaRead
|
|
||||||
from .seed import seed_objects
|
|
||||||
from .storage import (
|
|
||||||
ALLOWED_CONTENT_TYPES,
|
|
||||||
MAX_FILE_SIZE,
|
|
||||||
build_stored_name,
|
|
||||||
ensure_upload_dir,
|
|
||||||
is_allowed_content_type,
|
|
||||||
media_file_path,
|
|
||||||
remove_media_file,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def media_to_read(media: ObjectMedia) -> ObjectMediaRead:
|
|
||||||
return ObjectMediaRead(
|
|
||||||
id=media.id,
|
|
||||||
object_id=media.object_id,
|
|
||||||
original_name=media.original_name,
|
|
||||||
content_type=media.content_type,
|
|
||||||
size=media.size,
|
|
||||||
created_at=media.created_at,
|
|
||||||
url=f"/api/media/{media.id}/file",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@asynccontextmanager
|
|
||||||
async def lifespan(_: FastAPI):
|
|
||||||
ensure_upload_dir()
|
|
||||||
Base.metadata.create_all(bind=engine)
|
|
||||||
db = next(get_db())
|
|
||||||
try:
|
|
||||||
seed_objects(db)
|
|
||||||
finally:
|
|
||||||
db.close()
|
|
||||||
yield
|
|
||||||
|
|
||||||
|
|
||||||
app = FastAPI(title="MapMil API", lifespan=lifespan)
|
|
||||||
|
|
||||||
app.add_middleware(
|
|
||||||
CORSMiddleware,
|
|
||||||
allow_origins=["*"],
|
|
||||||
allow_credentials=True,
|
|
||||||
allow_methods=["*"],
|
|
||||||
allow_headers=["*"],
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/health")
|
|
||||||
def health():
|
|
||||||
return {"status": "ok"}
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/objects", response_model=list[MapObjectRead])
|
|
||||||
def list_objects(db: Session = Depends(get_db)):
|
|
||||||
return db.query(MapObject).order_by(MapObject.id).all()
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/objects/{object_id}", response_model=MapObjectRead)
|
|
||||||
def get_object(object_id: int, db: Session = Depends(get_db)):
|
|
||||||
obj = db.query(MapObject).filter(MapObject.id == object_id).first()
|
|
||||||
if not obj:
|
|
||||||
raise HTTPException(status_code=404, detail="Объект не найден")
|
|
||||||
return obj
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/api/objects", response_model=MapObjectRead, status_code=201)
|
|
||||||
def create_object(payload: MapObjectCreate, db: Session = Depends(get_db)):
|
|
||||||
data = payload.model_dump()
|
|
||||||
created_at = data.pop("created_at", None)
|
|
||||||
obj = MapObject(**data)
|
|
||||||
if created_at is not None:
|
|
||||||
obj.created_at = created_at
|
|
||||||
db.add(obj)
|
|
||||||
db.commit()
|
|
||||||
db.refresh(obj)
|
|
||||||
return obj
|
|
||||||
|
|
||||||
|
|
||||||
@app.patch("/api/objects/{object_id}", response_model=MapObjectRead)
|
|
||||||
def update_object(
|
|
||||||
object_id: int,
|
|
||||||
payload: MapObjectUpdate,
|
|
||||||
db: Session = Depends(get_db),
|
|
||||||
):
|
|
||||||
obj = db.query(MapObject).filter(MapObject.id == object_id).first()
|
|
||||||
if not obj:
|
|
||||||
raise HTTPException(status_code=404, detail="Объект не найден")
|
|
||||||
|
|
||||||
for field, value in payload.model_dump(exclude_unset=True).items():
|
|
||||||
setattr(obj, field, value)
|
|
||||||
|
|
||||||
db.commit()
|
|
||||||
db.refresh(obj)
|
|
||||||
return obj
|
|
||||||
|
|
||||||
|
|
||||||
@app.delete("/api/objects/{object_id}", status_code=204)
|
|
||||||
def delete_object(object_id: int, db: Session = Depends(get_db)):
|
|
||||||
obj = db.query(MapObject).filter(MapObject.id == object_id).first()
|
|
||||||
if not obj:
|
|
||||||
raise HTTPException(status_code=404, detail="Объект не найден")
|
|
||||||
|
|
||||||
for media in obj.media:
|
|
||||||
remove_media_file(media.stored_name)
|
|
||||||
|
|
||||||
db.delete(obj)
|
|
||||||
db.commit()
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/objects/{object_id}/media", response_model=list[ObjectMediaRead])
|
|
||||||
def list_object_media(object_id: int, db: Session = Depends(get_db)):
|
|
||||||
obj = db.query(MapObject).filter(MapObject.id == object_id).first()
|
|
||||||
if not obj:
|
|
||||||
raise HTTPException(status_code=404, detail="Объект не найден")
|
|
||||||
|
|
||||||
media_items = (
|
|
||||||
db.query(ObjectMedia)
|
|
||||||
.filter(ObjectMedia.object_id == object_id)
|
|
||||||
.order_by(ObjectMedia.id)
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
return [media_to_read(item) for item in media_items]
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/api/objects/{object_id}/media", response_model=ObjectMediaRead, status_code=201)
|
|
||||||
async def upload_object_media(
|
|
||||||
object_id: int,
|
|
||||||
file: UploadFile = File(...),
|
|
||||||
db: Session = Depends(get_db),
|
|
||||||
):
|
|
||||||
obj = db.query(MapObject).filter(MapObject.id == object_id).first()
|
|
||||||
if not obj:
|
|
||||||
raise HTTPException(status_code=404, detail="Объект не найден")
|
|
||||||
|
|
||||||
content_type = file.content_type or "application/octet-stream"
|
|
||||||
if not is_allowed_content_type(content_type):
|
|
||||||
raise HTTPException(
|
|
||||||
status_code=400,
|
|
||||||
detail=f"Неподдерживаемый тип файла. Разрешены: {', '.join(sorted(ALLOWED_CONTENT_TYPES))}",
|
|
||||||
)
|
|
||||||
|
|
||||||
content = await file.read()
|
|
||||||
if len(content) > MAX_FILE_SIZE:
|
|
||||||
raise HTTPException(status_code=400, detail="Файл слишком большой (макс. 20 МБ)")
|
|
||||||
|
|
||||||
original_name = file.filename or "file"
|
|
||||||
stored_name = build_stored_name(original_name)
|
|
||||||
path = media_file_path(stored_name)
|
|
||||||
path.write_bytes(content)
|
|
||||||
|
|
||||||
media = ObjectMedia(
|
|
||||||
object_id=object_id,
|
|
||||||
stored_name=stored_name,
|
|
||||||
original_name=original_name,
|
|
||||||
content_type=content_type,
|
|
||||||
size=len(content),
|
|
||||||
)
|
|
||||||
db.add(media)
|
|
||||||
db.commit()
|
|
||||||
db.refresh(media)
|
|
||||||
return media_to_read(media)
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/media/{media_id}/file")
|
|
||||||
def get_media_file(media_id: int, db: Session = Depends(get_db)):
|
|
||||||
media = db.query(ObjectMedia).filter(ObjectMedia.id == media_id).first()
|
|
||||||
if not media:
|
|
||||||
raise HTTPException(status_code=404, detail="Медиафайл не найден")
|
|
||||||
|
|
||||||
path = media_file_path(media.stored_name)
|
|
||||||
if not path.exists():
|
|
||||||
raise HTTPException(status_code=404, detail="Файл не найден на диске")
|
|
||||||
|
|
||||||
return FileResponse(path, media_type=media.content_type, filename=media.original_name)
|
|
||||||
|
|
||||||
|
|
||||||
@app.delete("/api/media/{media_id}", status_code=204)
|
|
||||||
def delete_media(media_id: int, db: Session = Depends(get_db)):
|
|
||||||
media = db.query(ObjectMedia).filter(ObjectMedia.id == media_id).first()
|
|
||||||
if not media:
|
|
||||||
raise HTTPException(status_code=404, detail="Медиафайл не найден")
|
|
||||||
|
|
||||||
remove_media_file(media.stored_name)
|
|
||||||
db.delete(media)
|
|
||||||
db.commit()
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
from datetime import datetime, timezone
|
|
||||||
|
|
||||||
from sqlalchemy import DateTime, Float, ForeignKey, Integer, String, Text
|
|
||||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
|
||||||
|
|
||||||
from .database import Base
|
|
||||||
|
|
||||||
|
|
||||||
class MapObject(Base):
|
|
||||||
__tablename__ = "map_objects"
|
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, index=True)
|
|
||||||
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
|
||||||
description: Mapped[str] = mapped_column(Text, default="")
|
|
||||||
type: Mapped[str] = mapped_column(String(50), nullable=False)
|
|
||||||
latitude: Mapped[float] = mapped_column(Float, nullable=False)
|
|
||||||
longitude: Mapped[float] = mapped_column(Float, nullable=False)
|
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
|
||||||
DateTime,
|
|
||||||
default=lambda: datetime.now(timezone.utc),
|
|
||||||
)
|
|
||||||
|
|
||||||
media: Mapped[list["ObjectMedia"]] = relationship(
|
|
||||||
back_populates="object",
|
|
||||||
cascade="all, delete-orphan",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class ObjectMedia(Base):
|
|
||||||
__tablename__ = "object_media"
|
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, index=True)
|
|
||||||
object_id: Mapped[int] = mapped_column(
|
|
||||||
ForeignKey("map_objects.id", ondelete="CASCADE"),
|
|
||||||
nullable=False,
|
|
||||||
index=True,
|
|
||||||
)
|
|
||||||
stored_name: Mapped[str] = mapped_column(String(255), nullable=False)
|
|
||||||
original_name: Mapped[str] = mapped_column(String(255), nullable=False)
|
|
||||||
content_type: Mapped[str] = mapped_column(String(100), nullable=False)
|
|
||||||
size: Mapped[int] = mapped_column(Integer, nullable=False)
|
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
|
||||||
DateTime,
|
|
||||||
default=lambda: datetime.now(timezone.utc),
|
|
||||||
)
|
|
||||||
|
|
||||||
object: Mapped["MapObject"] = relationship(back_populates="media")
|
|
||||||
@@ -1,48 +0,0 @@
|
|||||||
from datetime import datetime
|
|
||||||
from typing import Literal
|
|
||||||
|
|
||||||
from pydantic import BaseModel, ConfigDict, Field
|
|
||||||
|
|
||||||
ObjectType = Literal["point", "marker", "zone", "other"]
|
|
||||||
|
|
||||||
|
|
||||||
class MapObjectCreate(BaseModel):
|
|
||||||
name: str = Field(min_length=1, max_length=255)
|
|
||||||
description: str = ""
|
|
||||||
type: ObjectType
|
|
||||||
latitude: float = Field(ge=-90, le=90)
|
|
||||||
longitude: float = Field(ge=-180, le=180)
|
|
||||||
created_at: datetime | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class MapObjectRead(BaseModel):
|
|
||||||
model_config = ConfigDict(from_attributes=True)
|
|
||||||
|
|
||||||
id: int
|
|
||||||
name: str
|
|
||||||
description: str
|
|
||||||
type: ObjectType
|
|
||||||
latitude: float
|
|
||||||
longitude: float
|
|
||||||
created_at: datetime
|
|
||||||
|
|
||||||
|
|
||||||
class MapObjectUpdate(BaseModel):
|
|
||||||
name: str | None = Field(default=None, min_length=1, max_length=255)
|
|
||||||
description: str | None = None
|
|
||||||
type: ObjectType | None = None
|
|
||||||
latitude: float | None = Field(default=None, ge=-90, le=90)
|
|
||||||
longitude: float | None = Field(default=None, ge=-180, le=180)
|
|
||||||
created_at: datetime | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class ObjectMediaRead(BaseModel):
|
|
||||||
model_config = ConfigDict(from_attributes=True)
|
|
||||||
|
|
||||||
id: int
|
|
||||||
object_id: int
|
|
||||||
original_name: str
|
|
||||||
content_type: str
|
|
||||||
size: int
|
|
||||||
created_at: datetime
|
|
||||||
url: str
|
|
||||||
@@ -1,54 +0,0 @@
|
|||||||
from sqlalchemy.orm import Session
|
|
||||||
|
|
||||||
from .models import MapObject
|
|
||||||
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
|
|
||||||
from sqlalchemy.orm import Session
|
|
||||||
|
|
||||||
from .models import MapObject
|
|
||||||
|
|
||||||
SEED_OBJECTS = [
|
|
||||||
{
|
|
||||||
"name": "Красная площадь",
|
|
||||||
"description": "Главная площадь Москвы, исторический центр города.",
|
|
||||||
"type": "marker",
|
|
||||||
"latitude": 55.7539,
|
|
||||||
"longitude": 37.6208,
|
|
||||||
"created_at": datetime(2018, 5, 9, 12, 0, tzinfo=timezone.utc),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "ВДНХ",
|
|
||||||
"description": "Выставка достижений народного хозяйства — крупный выставочный комплекс.",
|
|
||||||
"type": "zone",
|
|
||||||
"latitude": 55.8298,
|
|
||||||
"longitude": 37.6361,
|
|
||||||
"created_at": datetime(2020, 8, 15, 10, 30, tzinfo=timezone.utc),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "МГУ",
|
|
||||||
"description": "Московский государственный университет имени М.В. Ломоносова.",
|
|
||||||
"type": "point",
|
|
||||||
"latitude": 55.7033,
|
|
||||||
"longitude": 37.5307,
|
|
||||||
"created_at": datetime(2022, 2, 12, 14, 0, tzinfo=timezone.utc),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "Парк Горького",
|
|
||||||
"description": "Центральный парк культуры и отдыха имени М. Горького.",
|
|
||||||
"type": "other",
|
|
||||||
"latitude": 55.7312,
|
|
||||||
"longitude": 37.6013,
|
|
||||||
"created_at": datetime(2024, 6, 1, 9, 0, tzinfo=timezone.utc),
|
|
||||||
},
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def seed_objects(db: Session) -> None:
|
|
||||||
if db.query(MapObject).count() > 0:
|
|
||||||
return
|
|
||||||
|
|
||||||
for item in SEED_OBJECTS:
|
|
||||||
db.add(MapObject(**item))
|
|
||||||
|
|
||||||
db.commit()
|
|
||||||
@@ -1,39 +0,0 @@
|
|||||||
import os
|
|
||||||
import uuid
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
UPLOAD_DIR = Path(os.getenv("UPLOAD_DIR", "/data/uploads"))
|
|
||||||
|
|
||||||
ALLOWED_CONTENT_TYPES = {
|
|
||||||
"image/jpeg",
|
|
||||||
"image/png",
|
|
||||||
"image/gif",
|
|
||||||
"image/webp",
|
|
||||||
"video/mp4",
|
|
||||||
"video/webm",
|
|
||||||
}
|
|
||||||
|
|
||||||
MAX_FILE_SIZE = 20 * 1024 * 1024
|
|
||||||
|
|
||||||
|
|
||||||
def ensure_upload_dir() -> None:
|
|
||||||
UPLOAD_DIR.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
|
|
||||||
def is_allowed_content_type(content_type: str) -> bool:
|
|
||||||
return content_type in ALLOWED_CONTENT_TYPES
|
|
||||||
|
|
||||||
|
|
||||||
def build_stored_name(original_name: str) -> str:
|
|
||||||
suffix = Path(original_name).suffix.lower()
|
|
||||||
return f"{uuid.uuid4().hex}{suffix}"
|
|
||||||
|
|
||||||
|
|
||||||
def media_file_path(stored_name: str) -> Path:
|
|
||||||
return UPLOAD_DIR / stored_name
|
|
||||||
|
|
||||||
|
|
||||||
def remove_media_file(stored_name: str) -> None:
|
|
||||||
path = media_file_path(stored_name)
|
|
||||||
if path.exists():
|
|
||||||
path.unlink()
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
fastapi==0.115.6
|
|
||||||
uvicorn[standard]==0.34.0
|
|
||||||
sqlalchemy==2.0.36
|
|
||||||
pydantic==2.10.3
|
|
||||||
python-multipart==0.0.20
|
|
||||||
@@ -2,13 +2,16 @@ FROM python:3.12-slim
|
|||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
COPY requirements.txt .
|
COPY centers/analytics/api/requirements.txt .
|
||||||
RUN pip install --no-cache-dir -r requirements.txt
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
COPY app/ ./app/
|
COPY contracts/ ./contracts/
|
||||||
|
COPY centers/analytics/api/app/ ./app/
|
||||||
|
|
||||||
RUN mkdir -p /data
|
RUN mkdir -p /data
|
||||||
|
|
||||||
|
ENV PYTHONPATH=/app
|
||||||
|
|
||||||
EXPOSE 8000
|
EXPOSE 8000
|
||||||
|
|
||||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||||
|
|||||||
@@ -1,8 +1,17 @@
|
|||||||
from contextlib import asynccontextmanager
|
from contextlib import asynccontextmanager
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
from fastapi import FastAPI
|
from fastapi import FastAPI
|
||||||
from fastapi.middleware.cors import CORSMiddleware
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
|
||||||
|
# Monorepo / Docker: contracts live next to app or at repo root
|
||||||
|
_HERE = Path(__file__).resolve()
|
||||||
|
for _candidate in (_HERE.parent, *_HERE.parents):
|
||||||
|
if (_candidate / "contracts").is_dir() and str(_candidate) not in sys.path:
|
||||||
|
sys.path.insert(0, str(_candidate))
|
||||||
|
break
|
||||||
|
|
||||||
from .database import Base, engine, get_db
|
from .database import Base, engine, get_db
|
||||||
from .routers import admin, internal, map, objects, v1
|
from .routers import admin, internal, map, objects, v1
|
||||||
from .seed import seed_objects, seed_test_consumer
|
from .seed import seed_objects, seed_test_consumer
|
||||||
|
|||||||
@@ -37,11 +37,23 @@ from ..services.jobs import enqueue_job
|
|||||||
router = APIRouter(prefix="/admin", tags=["admin"])
|
router = APIRouter(prefix="/admin", tags=["admin"])
|
||||||
|
|
||||||
|
|
||||||
|
def _validated_source_config(source_type: str, source_config: dict) -> dict:
|
||||||
|
try:
|
||||||
|
from contracts.queues import family_for_source
|
||||||
|
from contracts.sources import parse_source_config
|
||||||
|
|
||||||
|
family_for_source(source_type)
|
||||||
|
return parse_source_config(source_type, source_config or {}).model_dump()
|
||||||
|
except Exception as exc:
|
||||||
|
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
@router.post("/jobs", response_model=ParseJobRead, status_code=201)
|
@router.post("/jobs", response_model=ParseJobRead, status_code=201)
|
||||||
def create_parse_job(payload: ParseJobCreate, db: Session = Depends(get_db)):
|
def create_parse_job(payload: ParseJobCreate, db: Session = Depends(get_db)):
|
||||||
|
source_config = _validated_source_config(payload.source_type, payload.source_config)
|
||||||
job = ParseJob(
|
job = ParseJob(
|
||||||
source_type=payload.source_type,
|
source_type=payload.source_type,
|
||||||
source_config=payload.source_config,
|
source_config=source_config,
|
||||||
schedule=payload.schedule,
|
schedule=payload.schedule,
|
||||||
interval_seconds=payload.interval_seconds,
|
interval_seconds=payload.interval_seconds,
|
||||||
is_active=payload.is_active,
|
is_active=payload.is_active,
|
||||||
@@ -96,7 +108,7 @@ def update_parse_job(
|
|||||||
raise HTTPException(status_code=404, detail="Job not found")
|
raise HTTPException(status_code=404, detail="Job not found")
|
||||||
|
|
||||||
if payload.source_config is not None:
|
if payload.source_config is not None:
|
||||||
job.source_config = payload.source_config
|
job.source_config = _validated_source_config(job.source_type, payload.source_config)
|
||||||
if payload.interval_seconds is not None:
|
if payload.interval_seconds is not None:
|
||||||
job.interval_seconds = payload.interval_seconds
|
job.interval_seconds = payload.interval_seconds
|
||||||
if payload.is_active is not None:
|
if payload.is_active is not None:
|
||||||
|
|||||||
@@ -1,10 +1,22 @@
|
|||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
import redis
|
import redis
|
||||||
|
|
||||||
|
# Allow importing shared contracts from monorepo root in local runs
|
||||||
|
_HERE = Path(__file__).resolve()
|
||||||
|
for _candidate in (_HERE.parent, *_HERE.parents):
|
||||||
|
if (_candidate / "contracts").is_dir():
|
||||||
|
if str(_candidate) not in sys.path:
|
||||||
|
sys.path.insert(0, str(_candidate))
|
||||||
|
break
|
||||||
|
|
||||||
|
from contracts.queues import queue_key_for_source # noqa: E402
|
||||||
|
from contracts.sources import parse_source_config # noqa: E402
|
||||||
|
|
||||||
REDIS_URL = os.getenv("REDIS_URL", "redis://redis:6379/0")
|
REDIS_URL = os.getenv("REDIS_URL", "redis://redis:6379/0")
|
||||||
JOB_QUEUE_KEY = "cp:jobs"
|
|
||||||
|
|
||||||
|
|
||||||
def get_redis() -> redis.Redis:
|
def get_redis() -> redis.Redis:
|
||||||
@@ -12,9 +24,17 @@ def get_redis() -> redis.Redis:
|
|||||||
|
|
||||||
|
|
||||||
def enqueue_job(job_id: int, source_type: str, source_config: dict) -> None:
|
def enqueue_job(job_id: int, source_type: str, source_config: dict) -> None:
|
||||||
|
# Validate known configs early; unknown types still raise from queue_key_for_source
|
||||||
|
try:
|
||||||
|
parse_source_config(source_type, source_config or {})
|
||||||
|
except Exception:
|
||||||
|
# Keep enqueue resilient for legacy rows; worker validates again
|
||||||
|
pass
|
||||||
|
|
||||||
payload = {
|
payload = {
|
||||||
"job_id": job_id,
|
"job_id": job_id,
|
||||||
"source_type": source_type,
|
"source_type": source_type,
|
||||||
"source_config": source_config,
|
"source_config": source_config or {},
|
||||||
}
|
}
|
||||||
get_redis().rpush(JOB_QUEUE_KEY, json.dumps(payload))
|
key = queue_key_for_source(source_type)
|
||||||
|
get_redis().rpush(key, json.dumps(payload))
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { onMounted, onUnmounted, ref } from "vue";
|
import { computed, onMounted, onUnmounted, ref } from "vue";
|
||||||
import { createJob, deleteJob, fetchJobs, retryJob, updateJob } from "../api/admin";
|
import { createJob, deleteJob, fetchJobs, retryJob, updateJob } from "../api/admin";
|
||||||
import type { ParseJob } from "../types/admin";
|
import type { ParseJob } from "../types/admin";
|
||||||
|
|
||||||
@@ -12,22 +12,39 @@ const INTERVAL_OPTIONS = [
|
|||||||
{ value: 86400, label: "24 ч" },
|
{ value: 86400, label: "24 ч" },
|
||||||
];
|
];
|
||||||
|
|
||||||
|
const SOURCE_TYPES = [
|
||||||
|
{ value: "telegram", label: "Telegram" },
|
||||||
|
{ value: "crawl4ai", label: "Crawl4AI (web)" },
|
||||||
|
{ value: "viina", label: "VIINA (news NLP)" },
|
||||||
|
] as const;
|
||||||
|
|
||||||
const jobs = ref<ParseJob[]>([]);
|
const jobs = ref<ParseJob[]>([]);
|
||||||
const loading = ref(true);
|
const loading = ref(true);
|
||||||
const error = ref("");
|
const error = ref("");
|
||||||
const submitting = ref(false);
|
const submitting = ref(false);
|
||||||
const editingJob = ref<ParseJob | null>(null);
|
const editingJob = ref<ParseJob | null>(null);
|
||||||
|
|
||||||
const editForm = ref({
|
const editForm = ref({
|
||||||
channel: "",
|
channel: "",
|
||||||
limit: 50,
|
limit: 50,
|
||||||
|
urlsText: "",
|
||||||
|
textsText: "",
|
||||||
|
domain_profile: "generic_news",
|
||||||
|
input_mode: "urls" as "urls" | "texts" | "mixed",
|
||||||
|
extract_mode: "heuristic" as "heuristic" | "llm",
|
||||||
interval_seconds: 3600,
|
interval_seconds: 3600,
|
||||||
is_active: true,
|
is_active: true,
|
||||||
});
|
});
|
||||||
|
|
||||||
const form = ref({
|
const form = ref({
|
||||||
source_type: "telegram",
|
source_type: "telegram" as "telegram" | "crawl4ai" | "viina",
|
||||||
channel: "",
|
channel: "",
|
||||||
limit: 50,
|
limit: 50,
|
||||||
|
urlsText: "",
|
||||||
|
textsText: "",
|
||||||
|
domain_profile: "generic_news",
|
||||||
|
input_mode: "urls" as "urls" | "texts" | "mixed",
|
||||||
|
extract_mode: "heuristic" as "heuristic" | "llm",
|
||||||
interval_seconds: 3600,
|
interval_seconds: 3600,
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -42,6 +59,38 @@ function limitFromConfig(config: Record<string, unknown>): number {
|
|||||||
return typeof limit === "number" ? limit : 50;
|
return typeof limit === "number" ? limit : 50;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function urlsFromConfig(config: Record<string, unknown>): string[] {
|
||||||
|
return Array.isArray(config.urls)
|
||||||
|
? config.urls.filter((u): u is string => typeof u === "string")
|
||||||
|
: [];
|
||||||
|
}
|
||||||
|
|
||||||
|
function textsFromConfig(config: Record<string, unknown>): string[] {
|
||||||
|
return Array.isArray(config.texts)
|
||||||
|
? config.texts.filter((t): t is string => typeof t === "string")
|
||||||
|
: [];
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseLines(text: string): string[] {
|
||||||
|
return text
|
||||||
|
.split(/\r?\n/)
|
||||||
|
.map((line) => line.trim())
|
||||||
|
.filter(Boolean);
|
||||||
|
}
|
||||||
|
|
||||||
|
function sourceSummary(job: ParseJob): string {
|
||||||
|
const cfg = job.source_config;
|
||||||
|
const mode = cfg.extract_mode === "llm" ? " [LLM]" : "";
|
||||||
|
if (job.source_type === "telegram") {
|
||||||
|
return (channelFromConfig(cfg) || "—") + mode;
|
||||||
|
}
|
||||||
|
const urls = urlsFromConfig(cfg);
|
||||||
|
if (urls.length) return (urls.length === 1 ? urls[0] : `${urls.length} URL`) + mode;
|
||||||
|
const texts = textsFromConfig(cfg);
|
||||||
|
if (texts.length) return `${texts.length} текст(ов)`;
|
||||||
|
return "—";
|
||||||
|
}
|
||||||
|
|
||||||
function formatInterval(seconds: number): string {
|
function formatInterval(seconds: number): string {
|
||||||
const opt = INTERVAL_OPTIONS.find((item) => item.value === seconds);
|
const opt = INTERVAL_OPTIONS.find((item) => item.value === seconds);
|
||||||
if (opt) return opt.label;
|
if (opt) return opt.label;
|
||||||
@@ -50,6 +99,53 @@ function formatInterval(seconds: number): string {
|
|||||||
return `${Math.round(seconds / 86400)} д`;
|
return `${Math.round(seconds / 86400)} д`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function buildSourceConfig(
|
||||||
|
sourceType: string,
|
||||||
|
data: {
|
||||||
|
channel: string;
|
||||||
|
limit: number;
|
||||||
|
urlsText: string;
|
||||||
|
textsText: string;
|
||||||
|
domain_profile: string;
|
||||||
|
input_mode: "urls" | "texts" | "mixed";
|
||||||
|
extract_mode: "heuristic" | "llm";
|
||||||
|
},
|
||||||
|
): Record<string, unknown> {
|
||||||
|
if (sourceType === "telegram") {
|
||||||
|
return {
|
||||||
|
channel: data.channel.trim(),
|
||||||
|
limit: data.limit,
|
||||||
|
extract_mode: data.extract_mode,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (sourceType === "crawl4ai") {
|
||||||
|
return {
|
||||||
|
urls: parseLines(data.urlsText),
|
||||||
|
domain_profile: data.domain_profile.trim() || "generic_news",
|
||||||
|
extract_mode: data.extract_mode,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
urls: parseLines(data.urlsText),
|
||||||
|
texts: parseLines(data.textsText),
|
||||||
|
input_mode: data.input_mode,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const formHint = computed(() => {
|
||||||
|
if (form.value.source_type === "telegram") {
|
||||||
|
return form.value.extract_mode === "llm"
|
||||||
|
? "LLM (DeepSeek): неструктурированные посты канала разбираются в locality/date/coords/description."
|
||||||
|
: "Активные Telegram-парсеры подхватываются real-time listener; batch забирает последние N постов (эвристики).";
|
||||||
|
}
|
||||||
|
if (form.value.source_type === "crawl4ai") {
|
||||||
|
return form.value.extract_mode === "llm"
|
||||||
|
? "Crawl4AI + LLM (DeepSeek): страница → structured event. Нужен DEEPSEEK_API_KEY и cp-workers-web."
|
||||||
|
: "Crawl4AI обходит URL и нормализует страницы эвристиками. Обрабатывает cp-workers-web.";
|
||||||
|
}
|
||||||
|
return "VIINA-адаптер извлекает инциденты из новостных URL/текстов. Обрабатывает cp-workers-nlp.";
|
||||||
|
});
|
||||||
|
|
||||||
async function loadJobs() {
|
async function loadJobs() {
|
||||||
try {
|
try {
|
||||||
jobs.value = await fetchJobs();
|
jobs.value = await fetchJobs();
|
||||||
@@ -62,24 +158,43 @@ async function loadJobs() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async function handleSubmit() {
|
async function handleSubmit() {
|
||||||
if (!form.value.channel.trim()) {
|
if (form.value.source_type === "telegram" && !form.value.channel.trim()) {
|
||||||
error.value = "Укажите канал Telegram";
|
error.value = "Укажите канал Telegram";
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
if (form.value.source_type === "crawl4ai" && !parseLines(form.value.urlsText).length) {
|
||||||
|
error.value = "Укажите хотя бы один URL";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (form.value.source_type === "viina") {
|
||||||
|
const urls = parseLines(form.value.urlsText);
|
||||||
|
const texts = parseLines(form.value.textsText);
|
||||||
|
if (form.value.input_mode === "urls" && !urls.length) {
|
||||||
|
error.value = "Укажите URL для VIINA";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (form.value.input_mode === "texts" && !texts.length) {
|
||||||
|
error.value = "Укажите тексты для VIINA";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (form.value.input_mode === "mixed" && !urls.length && !texts.length) {
|
||||||
|
error.value = "Укажите URL или тексты";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
submitting.value = true;
|
submitting.value = true;
|
||||||
error.value = "";
|
error.value = "";
|
||||||
try {
|
try {
|
||||||
await createJob({
|
await createJob({
|
||||||
source_type: form.value.source_type,
|
source_type: form.value.source_type,
|
||||||
source_config: {
|
source_config: buildSourceConfig(form.value.source_type, form.value),
|
||||||
channel: form.value.channel.trim(),
|
|
||||||
limit: form.value.limit,
|
|
||||||
},
|
|
||||||
interval_seconds: form.value.interval_seconds,
|
interval_seconds: form.value.interval_seconds,
|
||||||
is_active: true,
|
is_active: true,
|
||||||
});
|
});
|
||||||
form.value.channel = "";
|
form.value.channel = "";
|
||||||
|
form.value.urlsText = "";
|
||||||
|
form.value.textsText = "";
|
||||||
await loadJobs();
|
await loadJobs();
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
error.value = err instanceof Error ? err.message : "Не удалось создать задание";
|
error.value = err instanceof Error ? err.message : "Не удалось создать задание";
|
||||||
@@ -93,6 +208,17 @@ function openEdit(job: ParseJob) {
|
|||||||
editForm.value = {
|
editForm.value = {
|
||||||
channel: channelFromConfig(job.source_config),
|
channel: channelFromConfig(job.source_config),
|
||||||
limit: limitFromConfig(job.source_config),
|
limit: limitFromConfig(job.source_config),
|
||||||
|
urlsText: urlsFromConfig(job.source_config).join("\n"),
|
||||||
|
textsText: textsFromConfig(job.source_config).join("\n"),
|
||||||
|
domain_profile:
|
||||||
|
typeof job.source_config.domain_profile === "string"
|
||||||
|
? job.source_config.domain_profile
|
||||||
|
: "generic_news",
|
||||||
|
input_mode:
|
||||||
|
job.source_config.input_mode === "texts" || job.source_config.input_mode === "mixed"
|
||||||
|
? job.source_config.input_mode
|
||||||
|
: "urls",
|
||||||
|
extract_mode: job.source_config.extract_mode === "llm" ? "llm" : "heuristic",
|
||||||
interval_seconds: job.interval_seconds,
|
interval_seconds: job.interval_seconds,
|
||||||
is_active: job.is_active,
|
is_active: job.is_active,
|
||||||
};
|
};
|
||||||
@@ -104,7 +230,9 @@ function closeEdit() {
|
|||||||
|
|
||||||
async function handleSaveEdit() {
|
async function handleSaveEdit() {
|
||||||
if (!editingJob.value) return;
|
if (!editingJob.value) return;
|
||||||
if (!editForm.value.channel.trim()) {
|
const sourceType = editingJob.value.source_type;
|
||||||
|
|
||||||
|
if (sourceType === "telegram" && !editForm.value.channel.trim()) {
|
||||||
error.value = "Укажите канал Telegram";
|
error.value = "Укажите канал Telegram";
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -113,10 +241,7 @@ async function handleSaveEdit() {
|
|||||||
error.value = "";
|
error.value = "";
|
||||||
try {
|
try {
|
||||||
await updateJob(editingJob.value.id, {
|
await updateJob(editingJob.value.id, {
|
||||||
source_config: {
|
source_config: buildSourceConfig(sourceType, editForm.value),
|
||||||
channel: editForm.value.channel.trim(),
|
|
||||||
limit: editForm.value.limit,
|
|
||||||
},
|
|
||||||
interval_seconds: editForm.value.interval_seconds,
|
interval_seconds: editForm.value.interval_seconds,
|
||||||
is_active: editForm.value.is_active,
|
is_active: editForm.value.is_active,
|
||||||
});
|
});
|
||||||
@@ -130,7 +255,7 @@ async function handleSaveEdit() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async function handleDelete(job: ParseJob) {
|
async function handleDelete(job: ParseJob) {
|
||||||
if (!window.confirm(`Удалить парсер #${job.id} (${channelFromConfig(job.source_config)})?`)) {
|
if (!window.confirm(`Удалить парсер #${job.id} (${sourceSummary(job)})?`)) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
error.value = "";
|
error.value = "";
|
||||||
@@ -181,18 +306,21 @@ onUnmounted(() => {
|
|||||||
<section class="card">
|
<section class="card">
|
||||||
<h3>Новый парсер</h3>
|
<h3>Новый парсер</h3>
|
||||||
<p class="hint">
|
<p class="hint">
|
||||||
Активные парсеры подхватываются real-time listener (новые посты сразу в БД).
|
{{ formHint }}
|
||||||
Дополнительно batch-прогон по интервалу забирает последние N постов.
|
|
||||||
Дубликаты по <code>source_url</code> не записываются.
|
Дубликаты по <code>source_url</code> не записываются.
|
||||||
</p>
|
</p>
|
||||||
<form class="admin-form" @submit.prevent="handleSubmit">
|
<form class="admin-form" @submit.prevent="handleSubmit">
|
||||||
<div class="form-row">
|
<div class="form-row">
|
||||||
<label>
|
<label>
|
||||||
Тип источника
|
Тип источника
|
||||||
<select v-model="form.source_type" disabled>
|
<select v-model="form.source_type">
|
||||||
<option value="telegram">Telegram</option>
|
<option v-for="opt in SOURCE_TYPES" :key="opt.value" :value="opt.value">
|
||||||
|
{{ opt.label }}
|
||||||
|
</option>
|
||||||
</select>
|
</select>
|
||||||
</label>
|
</label>
|
||||||
|
|
||||||
|
<template v-if="form.source_type === 'telegram'">
|
||||||
<label>
|
<label>
|
||||||
Канал
|
Канал
|
||||||
<input v-model="form.channel" type="text" placeholder="creamy_caprice" required />
|
<input v-model="form.channel" type="text" placeholder="creamy_caprice" required />
|
||||||
@@ -201,6 +329,52 @@ onUnmounted(() => {
|
|||||||
Лимит
|
Лимит
|
||||||
<input v-model.number="form.limit" type="number" min="1" max="1000" />
|
<input v-model.number="form.limit" type="number" min="1" max="1000" />
|
||||||
</label>
|
</label>
|
||||||
|
<label>
|
||||||
|
Извлечение
|
||||||
|
<select v-model="form.extract_mode">
|
||||||
|
<option value="heuristic">Эвристики (структурированные посты)</option>
|
||||||
|
<option value="llm">LLM DeepSeek (неструктурированные)</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<template v-else-if="form.source_type === 'crawl4ai'">
|
||||||
|
<label class="span-2">
|
||||||
|
URL (по одному в строке)
|
||||||
|
<textarea v-model="form.urlsText" rows="3" placeholder="https://example.com/news/…" required />
|
||||||
|
</label>
|
||||||
|
<label>
|
||||||
|
Domain profile
|
||||||
|
<input v-model="form.domain_profile" type="text" placeholder="generic_news" />
|
||||||
|
</label>
|
||||||
|
<label>
|
||||||
|
Извлечение
|
||||||
|
<select v-model="form.extract_mode">
|
||||||
|
<option value="heuristic">Эвристики (regex)</option>
|
||||||
|
<option value="llm">LLM DeepSeek (Crawl4AI)</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<template v-else>
|
||||||
|
<label>
|
||||||
|
Режим ввода
|
||||||
|
<select v-model="form.input_mode">
|
||||||
|
<option value="urls">URLs</option>
|
||||||
|
<option value="texts">Texts</option>
|
||||||
|
<option value="mixed">Mixed</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<label v-if="form.input_mode !== 'texts'" class="span-2">
|
||||||
|
URL (по одному в строке)
|
||||||
|
<textarea v-model="form.urlsText" rows="3" placeholder="https://example.com/article…" />
|
||||||
|
</label>
|
||||||
|
<label v-if="form.input_mode !== 'urls'" class="span-2">
|
||||||
|
Тексты (по одному блоку в строке)
|
||||||
|
<textarea v-model="form.textsText" rows="3" placeholder="Текст статьи…" />
|
||||||
|
</label>
|
||||||
|
</template>
|
||||||
|
|
||||||
<label>
|
<label>
|
||||||
Интервал
|
Интервал
|
||||||
<select v-model.number="form.interval_seconds">
|
<select v-model.number="form.interval_seconds">
|
||||||
@@ -226,8 +400,8 @@ onUnmounted(() => {
|
|||||||
<thead>
|
<thead>
|
||||||
<tr>
|
<tr>
|
||||||
<th>ID</th>
|
<th>ID</th>
|
||||||
<th>Канал</th>
|
<th>Тип</th>
|
||||||
<th>Лимит</th>
|
<th>Источник</th>
|
||||||
<th>Интервал</th>
|
<th>Интервал</th>
|
||||||
<th>Активен</th>
|
<th>Активен</th>
|
||||||
<th>Статус</th>
|
<th>Статус</th>
|
||||||
@@ -242,8 +416,8 @@ onUnmounted(() => {
|
|||||||
</tr>
|
</tr>
|
||||||
<tr v-for="job in jobs" :key="job.id">
|
<tr v-for="job in jobs" :key="job.id">
|
||||||
<td>{{ job.id }}</td>
|
<td>{{ job.id }}</td>
|
||||||
<td>{{ channelFromConfig(job.source_config) }}</td>
|
<td>{{ job.source_type }}</td>
|
||||||
<td>{{ limitFromConfig(job.source_config) }}</td>
|
<td class="source-cell">{{ sourceSummary(job) }}</td>
|
||||||
<td>{{ formatInterval(job.interval_seconds) }}</td>
|
<td>{{ formatInterval(job.interval_seconds) }}</td>
|
||||||
<td>{{ job.is_active ? "да" : "нет" }}</td>
|
<td>{{ job.is_active ? "да" : "нет" }}</td>
|
||||||
<td>
|
<td>
|
||||||
@@ -278,9 +452,10 @@ onUnmounted(() => {
|
|||||||
|
|
||||||
<div v-if="editingJob" class="modal-backdrop" @click.self="closeEdit">
|
<div v-if="editingJob" class="modal-backdrop" @click.self="closeEdit">
|
||||||
<div class="modal card">
|
<div class="modal card">
|
||||||
<h3>Редактировать парсер #{{ editingJob.id }}</h3>
|
<h3>Редактировать парсер #{{ editingJob.id }} ({{ editingJob.source_type }})</h3>
|
||||||
<form class="admin-form" @submit.prevent="handleSaveEdit">
|
<form class="admin-form" @submit.prevent="handleSaveEdit">
|
||||||
<div class="form-row">
|
<div class="form-row">
|
||||||
|
<template v-if="editingJob.source_type === 'telegram'">
|
||||||
<label>
|
<label>
|
||||||
Канал
|
Канал
|
||||||
<input v-model="editForm.channel" type="text" required />
|
<input v-model="editForm.channel" type="text" required />
|
||||||
@@ -289,6 +464,49 @@ onUnmounted(() => {
|
|||||||
Лимит
|
Лимит
|
||||||
<input v-model.number="editForm.limit" type="number" min="1" max="1000" />
|
<input v-model.number="editForm.limit" type="number" min="1" max="1000" />
|
||||||
</label>
|
</label>
|
||||||
|
<label>
|
||||||
|
Извлечение
|
||||||
|
<select v-model="editForm.extract_mode">
|
||||||
|
<option value="heuristic">Эвристики</option>
|
||||||
|
<option value="llm">LLM DeepSeek</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
</template>
|
||||||
|
<template v-else-if="editingJob.source_type === 'crawl4ai'">
|
||||||
|
<label class="span-2">
|
||||||
|
URL (по одному в строке)
|
||||||
|
<textarea v-model="editForm.urlsText" rows="3" required />
|
||||||
|
</label>
|
||||||
|
<label>
|
||||||
|
Domain profile
|
||||||
|
<input v-model="editForm.domain_profile" type="text" />
|
||||||
|
</label>
|
||||||
|
<label>
|
||||||
|
Извлечение
|
||||||
|
<select v-model="editForm.extract_mode">
|
||||||
|
<option value="heuristic">Эвристики</option>
|
||||||
|
<option value="llm">LLM DeepSeek</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
</template>
|
||||||
|
<template v-else>
|
||||||
|
<label>
|
||||||
|
Режим ввода
|
||||||
|
<select v-model="editForm.input_mode">
|
||||||
|
<option value="urls">URLs</option>
|
||||||
|
<option value="texts">Texts</option>
|
||||||
|
<option value="mixed">Mixed</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<label v-if="editForm.input_mode !== 'texts'" class="span-2">
|
||||||
|
URL
|
||||||
|
<textarea v-model="editForm.urlsText" rows="3" />
|
||||||
|
</label>
|
||||||
|
<label v-if="editForm.input_mode !== 'urls'" class="span-2">
|
||||||
|
Тексты
|
||||||
|
<textarea v-model="editForm.textsText" rows="3" />
|
||||||
|
</label>
|
||||||
|
</template>
|
||||||
<label>
|
<label>
|
||||||
Интервал
|
Интервал
|
||||||
<select v-model.number="editForm.interval_seconds">
|
<select v-model.number="editForm.interval_seconds">
|
||||||
@@ -322,6 +540,26 @@ onUnmounted(() => {
|
|||||||
line-height: 1.5;
|
line-height: 1.5;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.form-row .span-2 {
|
||||||
|
grid-column: span 2;
|
||||||
|
}
|
||||||
|
|
||||||
|
.form-row textarea {
|
||||||
|
width: 100%;
|
||||||
|
font: inherit;
|
||||||
|
padding: 0.4rem 0.5rem;
|
||||||
|
border: 1px solid #d1d5db;
|
||||||
|
border-radius: 4px;
|
||||||
|
resize: vertical;
|
||||||
|
}
|
||||||
|
|
||||||
|
.source-cell {
|
||||||
|
max-width: 280px;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
.actions-cell {
|
.actions-cell {
|
||||||
white-space: nowrap;
|
white-space: nowrap;
|
||||||
}
|
}
|
||||||
@@ -346,8 +584,10 @@ onUnmounted(() => {
|
|||||||
}
|
}
|
||||||
|
|
||||||
.modal {
|
.modal {
|
||||||
width: min(520px, 92vw);
|
width: min(560px, 92vw);
|
||||||
margin: 0;
|
margin: 0;
|
||||||
|
max-height: 90vh;
|
||||||
|
overflow: auto;
|
||||||
}
|
}
|
||||||
|
|
||||||
.checkbox-row {
|
.checkbox-row {
|
||||||
|
|||||||
@@ -0,0 +1,146 @@
|
|||||||
|
# Архитектура парсинга (ЦП)
|
||||||
|
|
||||||
|
Центр парсинга (**ЦП**) — сервисы `cp-workers*` с **реестром адаптеров** источников. Собирают события и отправляют их в ЦА через internal API.
|
||||||
|
|
||||||
|
## Роль в платформе
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
CA[ЦА ca-api]
|
||||||
|
Redis[(Redis cp:jobs:family)]
|
||||||
|
CP[ЦП adapters]
|
||||||
|
Src[Sources]
|
||||||
|
|
||||||
|
CA -->|"RPUSH JobPayload"| Redis
|
||||||
|
Redis -->|"BLPOP"| CP
|
||||||
|
CP --> Src
|
||||||
|
CP -->|"POST /internal/ingest"| CA
|
||||||
|
CP -->|"PATCH /internal/jobs/{id}"| CA
|
||||||
|
CP -->|"GET /internal/listener/subscriptions"| CA
|
||||||
|
```
|
||||||
|
|
||||||
|
| Направление | Механизм | Назначение |
|
||||||
|
|-------------|----------|------------|
|
||||||
|
| ЦА → ЦП | Redis `cp:jobs:{family}` | Batch-задания по семейству адаптеров |
|
||||||
|
| ЦП → источники | Адаптер (`telegram` / `crawl4ai` / `viina`) | Fetch + extract |
|
||||||
|
| ЦП → ЦА | `POST /internal/ingest` | Запись событий (`IngestEventItem`) |
|
||||||
|
| ЦП → ЦА | `PATCH /internal/jobs/{id}` | Статус batch-задания |
|
||||||
|
| ЦП ← ЦА | `GET /internal/listener/subscriptions` | Каналы real-time (только Telegram) |
|
||||||
|
|
||||||
|
Общие контракты: [`contracts/jobs.py`](../../contracts/jobs.py), [`contracts/ingest.py`](../../contracts/ingest.py), [`contracts/sources.py`](../../contracts/sources.py), [`contracts/queues.py`](../../contracts/queues.py).
|
||||||
|
|
||||||
|
## Адаптеры источников
|
||||||
|
|
||||||
|
Каждый `source_type` реализует `SourceAdapter`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def run(job_id, source_config, *, ctx) -> tuple[list[dict], str | None]
|
||||||
|
```
|
||||||
|
|
||||||
|
Выход — список dict в форме `IngestEventItem`. ЦА **не** знает про Crawl4AI/VIINA.
|
||||||
|
|
||||||
|
| source_type | Семейство / очередь | Сервис | Зависимости |
|
||||||
|
|-------------|---------------------|--------|-------------|
|
||||||
|
| `telegram` | `telegram` → `cp:jobs:telegram` | `cp-workers` | Telethon |
|
||||||
|
| `crawl4ai` | `web` → `cp:jobs:web` | `cp-workers-web` | Crawl4AI + Playwright |
|
||||||
|
| `viina` | `nlp` → `cp:jobs:nlp` | `cp-workers-nlp` | httpx + BeautifulSoup |
|
||||||
|
|
||||||
|
Маршрутизация при enqueue в ЦА: [`queue_key_for_source`](../../contracts/queues.py).
|
||||||
|
Воркер слушает `WORKER_FAMILIES` / очереди своих `ENABLED_ADAPTERS`. Чужой job → requeue в нужную очередь.
|
||||||
|
|
||||||
|
Правила для всех адаптеров:
|
||||||
|
|
||||||
|
- стабильный `source_url` (дедуп в ЦА);
|
||||||
|
- `source_type` события = тип адаптера;
|
||||||
|
- `source_config` валидируется схемами из `contracts/sources.py`.
|
||||||
|
|
||||||
|
### LLM-режим (`extract_mode: llm`)
|
||||||
|
|
||||||
|
Для неструктурированных Telegram-постов и Crawl4AI:
|
||||||
|
|
||||||
|
- ключ `DEEPSEEK_API_KEY` в `.env` (воркеры `cp-workers` / `cp-workers-web`);
|
||||||
|
- Telegram: текст поста → DeepSeek JSON → `IngestEventItem`;
|
||||||
|
- Crawl4AI: страница → `LLMExtractionStrategy` (DeepSeek) с fallback на тот же DeepSeek по markdown;
|
||||||
|
- в UI «Парсеры»: поле **Извлечение** = LLM DeepSeek.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
Job[JobPayload]
|
||||||
|
Reg[AdapterRegistry]
|
||||||
|
TG[TelegramAdapter]
|
||||||
|
C4[Crawl4AIAdapter]
|
||||||
|
VI[ViinaAdapter]
|
||||||
|
Ingest[IngestEventItem]
|
||||||
|
|
||||||
|
Job --> Reg
|
||||||
|
Reg --> TG --> Ingest
|
||||||
|
Reg --> C4 --> Ingest
|
||||||
|
Reg --> VI --> Ingest
|
||||||
|
```
|
||||||
|
|
||||||
|
## Структура каталога
|
||||||
|
|
||||||
|
```
|
||||||
|
centers/parsing/
|
||||||
|
├── ARCHITECTURE.md
|
||||||
|
└── workers/
|
||||||
|
├── Dockerfile # context = repo root; ARG REQUIREMENTS_FILE
|
||||||
|
├── requirements.txt # telegram
|
||||||
|
├── requirements-web.txt # crawl4ai
|
||||||
|
├── requirements-nlp.txt # viina
|
||||||
|
├── worker.py
|
||||||
|
└── workers/
|
||||||
|
├── adapters/
|
||||||
|
│ ├── base.py # SourceAdapter, WorkerContext
|
||||||
|
│ ├── registry.py # ENABLED_ADAPTERS
|
||||||
|
│ ├── telegram.py
|
||||||
|
│ ├── crawl4ai_adapter.py
|
||||||
|
│ └── viina.py
|
||||||
|
├── converter.py
|
||||||
|
├── parsers/telegram_events.py
|
||||||
|
└── sources/ # Telethon session / listener / client
|
||||||
|
```
|
||||||
|
|
||||||
|
## Режимы работы
|
||||||
|
|
||||||
|
| Режим | Условие | Поведение |
|
||||||
|
|-------|---------|-----------|
|
||||||
|
| Listener + batch | `ENABLED_ADAPTERS` включает `telegram` и `TELEGRAM_LISTENER_ENABLED=true` | Shared Telethon + listener + `worker_loop` |
|
||||||
|
| Только batch | listener выключен или нет telegram | Только `BLPOP` по очередям семейства |
|
||||||
|
|
||||||
|
## Поток batch-заданий
|
||||||
|
|
||||||
|
1. ЦА `enqueue_job` → Redis `cp:jobs:{family}` с `{ job_id, source_type, source_config }`
|
||||||
|
2. Воркер семейства: `BLPOP` → `handle_job` → `registry.get(source_type).run(...)`
|
||||||
|
3. `POST /internal/ingest` + статус job
|
||||||
|
|
||||||
|
Legacy-ключ `cp:jobs` по-прежнему дренируется telegram-воркером (совместимость).
|
||||||
|
|
||||||
|
## Telegram real-time
|
||||||
|
|
||||||
|
`TelegramListener` без изменений: подписки из ЦА, ingest с `listener: true` (статус `ParseJob` не трогается).
|
||||||
|
|
||||||
|
## Переменные окружения
|
||||||
|
|
||||||
|
| Переменная | Назначение |
|
||||||
|
|------------|------------|
|
||||||
|
| `ENABLED_ADAPTERS` | Список адаптеров через запятую (`telegram`, `crawl4ai`, `viina`) |
|
||||||
|
| `WORKER_FAMILIES` | Какие семейства очередей слушать (`telegram`, `web`, `nlp`) |
|
||||||
|
| `REDIS_URL` / `CA_API_URL` / `INTERNAL_TOKEN` | Как раньше |
|
||||||
|
| `TELEGRAM_*` | Только для `cp-workers` |
|
||||||
|
|
||||||
|
## Как добавить новый источник
|
||||||
|
|
||||||
|
1. Схема `source_config` в [`contracts/sources.py`](../../contracts/sources.py) + запись в `SOURCE_FAMILY` ([`queues.py`](../../contracts/queues.py))
|
||||||
|
2. Класс адаптера в `workers/adapters/` + factory в `registry.py`
|
||||||
|
3. При тяжёлых deps — `requirements-*.txt` и сервис в `docker-compose.yml`
|
||||||
|
4. Поля формы в UI «Парсеры»
|
||||||
|
|
||||||
|
## Связанные части ЦА
|
||||||
|
|
||||||
|
| Файл ЦА | Роль |
|
||||||
|
|---------|------|
|
||||||
|
| `services/jobs.py` | `enqueue_job` → `cp:jobs:{family}` |
|
||||||
|
| `routers/admin.py` | CRUD + валидация `source_config` |
|
||||||
|
| `services/ingest.py` | Сохранение Event + карта |
|
||||||
|
| UI `/parsers` | Выбор `telegram` / `crawl4ai` / `viina` |
|
||||||
@@ -2,11 +2,20 @@ FROM python:3.12-slim
|
|||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
COPY requirements.txt .
|
ARG REQUIREMENTS_FILE=requirements.txt
|
||||||
|
|
||||||
|
COPY centers/parsing/workers/${REQUIREMENTS_FILE} ./requirements.txt
|
||||||
RUN pip install --no-cache-dir -r requirements.txt
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
COPY workers/ ./workers/
|
# Optional Playwright browsers for Crawl4AI web workers
|
||||||
COPY worker.py .
|
ARG INSTALL_PLAYWRIGHT=0
|
||||||
|
RUN if [ "$INSTALL_PLAYWRIGHT" = "1" ]; then \
|
||||||
|
python -m playwright install --with-deps chromium; \
|
||||||
|
fi
|
||||||
|
|
||||||
|
COPY contracts/ ./contracts/
|
||||||
|
COPY centers/parsing/workers/workers/ ./workers/
|
||||||
|
COPY centers/parsing/workers/worker.py .
|
||||||
|
|
||||||
ENV PYTHONPATH=/app
|
ENV PYTHONPATH=/app
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,5 @@
|
|||||||
|
httpx==0.28.1
|
||||||
|
redis==5.2.1
|
||||||
|
beautifulsoup4==4.12.3
|
||||||
|
lxml==5.3.0
|
||||||
|
pydantic==2.10.3
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
httpx==0.28.1
|
||||||
|
redis==5.2.1
|
||||||
|
beautifulsoup4==4.12.3
|
||||||
|
lxml==5.3.0
|
||||||
|
pydantic==2.10.3
|
||||||
|
crawl4ai>=0.4.0,<0.7
|
||||||
|
playwright>=1.40.0
|
||||||
@@ -4,3 +4,4 @@ telethon==1.44.0
|
|||||||
python-socks[asyncio]==2.7.1
|
python-socks[asyncio]==2.7.1
|
||||||
beautifulsoup4==4.12.3
|
beautifulsoup4==4.12.3
|
||||||
lxml==5.3.0
|
lxml==5.3.0
|
||||||
|
pydantic==2.10.3
|
||||||
|
|||||||
@@ -1,29 +1,38 @@
|
|||||||
"""CP worker: Redis jobs + real-time Telethon listener."""
|
"""CP worker: Redis jobs via adapter registry + optional Telethon listener."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Monorepo local runs / Docker: ensure contracts/ is importable
|
||||||
|
_HERE = Path(__file__).resolve()
|
||||||
|
for _candidate in (_HERE.parent, *_HERE.parents):
|
||||||
|
if (_candidate / "contracts").is_dir():
|
||||||
|
if str(_candidate) not in sys.path:
|
||||||
|
sys.path.insert(0, str(_candidate))
|
||||||
|
break
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
import redis
|
import redis
|
||||||
|
|
||||||
from workers.converter import event_record_to_ingest
|
from contracts.queues import (
|
||||||
from workers.parsers.telegram_events import parse_event_posts
|
LEGACY_JOB_QUEUE_KEY,
|
||||||
from workers.sources.telegram_client import (
|
family_for_source,
|
||||||
TelegramAuthError,
|
queue_key_for_family,
|
||||||
TelegramConfigError,
|
queue_key_for_source,
|
||||||
fetch_channel_posts,
|
|
||||||
normalize_channel,
|
|
||||||
)
|
)
|
||||||
from workers.sources.telegram_listener import TelegramListener
|
from workers.adapters import build_registry
|
||||||
from workers.sources.telegram_session import close_shared_client, get_shared_client
|
from workers.adapters.base import WorkerContext
|
||||||
|
|
||||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
|
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
|
||||||
logger = logging.getLogger("cp-worker")
|
logger = logging.getLogger("cp-worker")
|
||||||
|
|
||||||
REDIS_URL = os.getenv("REDIS_URL", "redis://redis:6379/0")
|
REDIS_URL = os.getenv("REDIS_URL", "redis://redis:6379/0")
|
||||||
JOB_QUEUE_KEY = "cp:jobs"
|
|
||||||
CA_API_URL = os.getenv("CA_API_URL", "http://ca-api:8000")
|
CA_API_URL = os.getenv("CA_API_URL", "http://ca-api:8000")
|
||||||
INTERNAL_TOKEN = os.getenv("INTERNAL_TOKEN", "dev-internal-token")
|
INTERNAL_TOKEN = os.getenv("INTERNAL_TOKEN", "dev-internal-token")
|
||||||
POLL_TIMEOUT = int(os.getenv("WORKER_POLL_TIMEOUT", "5"))
|
POLL_TIMEOUT = int(os.getenv("WORKER_POLL_TIMEOUT", "5"))
|
||||||
@@ -33,32 +42,31 @@ LISTENER_ENABLED = os.getenv("TELEGRAM_LISTENER_ENABLED", "true").lower() not in
|
|||||||
"no",
|
"no",
|
||||||
"off",
|
"off",
|
||||||
)
|
)
|
||||||
|
# Comma-separated families this process polls (default: derived from adapters)
|
||||||
|
WORKER_FAMILIES = os.getenv("WORKER_FAMILIES", "").strip()
|
||||||
|
|
||||||
|
REGISTRY = build_registry()
|
||||||
|
|
||||||
|
|
||||||
def get_redis() -> redis.Redis:
|
def get_redis() -> redis.Redis:
|
||||||
return redis.from_url(REDIS_URL, decode_responses=True)
|
return redis.from_url(REDIS_URL, decode_responses=True)
|
||||||
|
|
||||||
|
|
||||||
async def process_telegram_job(
|
def _poll_keys() -> list[str]:
|
||||||
job_id: int,
|
if WORKER_FAMILIES:
|
||||||
source_config: dict,
|
families = [f.strip() for f in WORKER_FAMILIES.split(",") if f.strip()]
|
||||||
*,
|
else:
|
||||||
client=None,
|
families = sorted(
|
||||||
) -> tuple[list[dict], str | None]:
|
{
|
||||||
channel = source_config.get("channel", "creamy_caprice")
|
family_for_source(source_type)
|
||||||
limit = int(source_config.get("limit", 100))
|
for source_type in REGISTRY.enabled_types()
|
||||||
|
}
|
||||||
try:
|
)
|
||||||
username = normalize_channel(channel)
|
keys = [queue_key_for_family(f) for f in families]
|
||||||
posts = await fetch_channel_posts(username, limit=limit, client=client)
|
# Backward compatible: telegram workers also drain legacy cp:jobs
|
||||||
except (TelegramConfigError, TelegramAuthError, ValueError) as exc:
|
if "telegram" in families and LEGACY_JOB_QUEUE_KEY not in keys:
|
||||||
return [], str(exc)
|
keys.append(LEGACY_JOB_QUEUE_KEY)
|
||||||
except Exception as exc:
|
return keys
|
||||||
return [], f"Telegram: {exc}"
|
|
||||||
|
|
||||||
records = parse_event_posts(posts)
|
|
||||||
events = [event_record_to_ingest(r) for r in records]
|
|
||||||
return events, None
|
|
||||||
|
|
||||||
|
|
||||||
async def post_ingest(job_id: int, events: list[dict]) -> None:
|
async def post_ingest(job_id: int, events: list[dict]) -> None:
|
||||||
@@ -95,18 +103,35 @@ async def patch_job_status(job_id: int, status: str, error: str | None = None) -
|
|||||||
response.raise_for_status()
|
response.raise_for_status()
|
||||||
|
|
||||||
|
|
||||||
async def handle_job(payload: dict, *, tg_client=None) -> None:
|
async def handle_job(payload: dict, *, ctx: WorkerContext) -> None:
|
||||||
job_id = payload["job_id"]
|
job_id = payload["job_id"]
|
||||||
source_type = payload["source_type"]
|
source_type = payload["source_type"]
|
||||||
source_config = payload.get("source_config", {})
|
source_config = payload.get("source_config", {})
|
||||||
|
|
||||||
logger.info("Processing job %s (%s)", job_id, source_type)
|
logger.info("Processing job %s (%s)", job_id, source_type)
|
||||||
await patch_job_status(job_id, "running")
|
|
||||||
|
|
||||||
if source_type == "telegram":
|
adapter = REGISTRY.get(source_type)
|
||||||
events, error = await process_telegram_job(job_id, source_config, client=tg_client)
|
if adapter is None:
|
||||||
else:
|
try:
|
||||||
events, error = [], f"Unsupported source_type: {source_type}"
|
target = queue_key_for_source(source_type)
|
||||||
|
except ValueError:
|
||||||
|
await patch_job_status(
|
||||||
|
job_id,
|
||||||
|
"failed",
|
||||||
|
error=f"Unsupported source_type: {source_type}",
|
||||||
|
)
|
||||||
|
return
|
||||||
|
get_redis().rpush(target, json.dumps(payload))
|
||||||
|
logger.warning(
|
||||||
|
"Job %s (%s) not enabled here; requeued to %s",
|
||||||
|
job_id,
|
||||||
|
source_type,
|
||||||
|
target,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
await patch_job_status(job_id, "running")
|
||||||
|
events, error = await adapter.run(job_id, source_config, ctx=ctx)
|
||||||
|
|
||||||
if error:
|
if error:
|
||||||
logger.error("Job %s failed: %s", job_id, error)
|
logger.error("Job %s failed: %s", job_id, error)
|
||||||
@@ -123,18 +148,23 @@ async def handle_job(payload: dict, *, tg_client=None) -> None:
|
|||||||
await patch_job_status(job_id, "failed", error=str(exc))
|
await patch_job_status(job_id, "failed", error=str(exc))
|
||||||
|
|
||||||
|
|
||||||
async def worker_loop(*, tg_client=None) -> None:
|
async def worker_loop(*, ctx: WorkerContext) -> None:
|
||||||
r = get_redis()
|
r = get_redis()
|
||||||
logger.info("CP worker started, polling %s", JOB_QUEUE_KEY)
|
keys = _poll_keys()
|
||||||
|
logger.info(
|
||||||
|
"CP worker started, polling %s (adapters: %s)",
|
||||||
|
keys,
|
||||||
|
", ".join(REGISTRY.enabled_types()) or "(none)",
|
||||||
|
)
|
||||||
|
|
||||||
while True:
|
while True:
|
||||||
try:
|
try:
|
||||||
item = await asyncio.to_thread(r.blpop, JOB_QUEUE_KEY, POLL_TIMEOUT)
|
item = await asyncio.to_thread(r.blpop, keys, POLL_TIMEOUT)
|
||||||
if not item:
|
if not item:
|
||||||
continue
|
continue
|
||||||
_, raw = item
|
_, raw = item
|
||||||
payload = json.loads(raw)
|
payload = json.loads(raw)
|
||||||
await handle_job(payload, tg_client=tg_client)
|
await handle_job(payload, ctx=ctx)
|
||||||
except redis.RedisError as exc:
|
except redis.RedisError as exc:
|
||||||
logger.error("Redis error: %s", exc)
|
logger.error("Redis error: %s", exc)
|
||||||
await asyncio.sleep(3)
|
await asyncio.sleep(3)
|
||||||
@@ -144,9 +174,18 @@ async def worker_loop(*, tg_client=None) -> None:
|
|||||||
|
|
||||||
|
|
||||||
async def run_with_listener() -> None:
|
async def run_with_listener() -> None:
|
||||||
|
if "telegram" not in REGISTRY:
|
||||||
|
logger.warning("Listener requested but telegram adapter is not enabled")
|
||||||
|
await worker_loop(ctx=WorkerContext())
|
||||||
|
return
|
||||||
|
|
||||||
|
from workers.sources.telegram_listener import TelegramListener
|
||||||
|
from workers.sources.telegram_session import close_shared_client, get_shared_client
|
||||||
|
|
||||||
client = await get_shared_client()
|
client = await get_shared_client()
|
||||||
listener = TelegramListener(client)
|
listener = TelegramListener(client)
|
||||||
worker_task = asyncio.create_task(worker_loop(tg_client=client))
|
ctx = WorkerContext(tg_client=client)
|
||||||
|
worker_task = asyncio.create_task(worker_loop(ctx=ctx))
|
||||||
listener_task = asyncio.create_task(listener.run())
|
listener_task = asyncio.create_task(listener.run())
|
||||||
|
|
||||||
logger.info("Telegram listener enabled (shared session with batch worker)")
|
logger.info("Telegram listener enabled (shared session with batch worker)")
|
||||||
@@ -160,12 +199,12 @@ async def run_with_listener() -> None:
|
|||||||
|
|
||||||
async def run_batch_only() -> None:
|
async def run_batch_only() -> None:
|
||||||
logger.info("Telegram listener disabled")
|
logger.info("Telegram listener disabled")
|
||||||
await worker_loop(tg_client=None)
|
await worker_loop(ctx=WorkerContext(tg_client=None))
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
def main() -> None:
|
||||||
try:
|
try:
|
||||||
if LISTENER_ENABLED:
|
if LISTENER_ENABLED and "telegram" in REGISTRY:
|
||||||
asyncio.run(run_with_listener())
|
asyncio.run(run_with_listener())
|
||||||
else:
|
else:
|
||||||
asyncio.run(run_batch_only())
|
asyncio.run(run_batch_only())
|
||||||
|
|||||||
@@ -0,0 +1,5 @@
|
|||||||
|
"""CP source adapters (telegram, crawl4ai, viina, …)."""
|
||||||
|
|
||||||
|
from workers.adapters.registry import AdapterRegistry, build_registry
|
||||||
|
|
||||||
|
__all__ = ["AdapterRegistry", "build_registry"]
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
"""Source adapter protocol and shared worker context."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any, Protocol
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class WorkerContext:
|
||||||
|
"""Runtime deps shared by adapters (optional Telegram client, etc.)."""
|
||||||
|
|
||||||
|
tg_client: Any | None = None
|
||||||
|
extras: dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class SourceAdapter(Protocol):
|
||||||
|
source_type: str
|
||||||
|
|
||||||
|
async def run(
|
||||||
|
self,
|
||||||
|
job_id: int,
|
||||||
|
source_config: dict,
|
||||||
|
*,
|
||||||
|
ctx: WorkerContext,
|
||||||
|
) -> tuple[list[dict], str | None]:
|
||||||
|
"""Return ingest-ready dicts (IngestEventItem-shaped) or an error string."""
|
||||||
|
...
|
||||||
@@ -0,0 +1,286 @@
|
|||||||
|
"""Crawl4AI web adapter: crawl URLs → map fields → ingest events."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
from typing import Any
|
||||||
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
|
from contracts.sources import Crawl4AISourceConfig
|
||||||
|
from workers.adapters.base import WorkerContext
|
||||||
|
from workers.llm_extract import (
|
||||||
|
DEFAULT_INSTRUCTION,
|
||||||
|
crawl4ai_llm_config,
|
||||||
|
extract_event_fields,
|
||||||
|
fields_to_ingest,
|
||||||
|
llm_enabled,
|
||||||
|
parse_coords,
|
||||||
|
parse_date,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger("cp-worker.crawl4ai")
|
||||||
|
|
||||||
|
COORDS_RE = re.compile(r"(-?\d{1,3}\.\d+)\s*,\s*(-?\d{1,3}\.\d+)")
|
||||||
|
DATE_RE = re.compile(
|
||||||
|
r"\b(\d{1,2}[./]\d{1,2}[./]\d{2,4}|\d{4}-\d{2}-\d{2})\b",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Crawl4AIAdapter:
|
||||||
|
source_type = "crawl4ai"
|
||||||
|
|
||||||
|
async def run(
|
||||||
|
self,
|
||||||
|
job_id: int,
|
||||||
|
source_config: dict,
|
||||||
|
*,
|
||||||
|
ctx: WorkerContext,
|
||||||
|
) -> tuple[list[dict], str | None]:
|
||||||
|
try:
|
||||||
|
cfg = Crawl4AISourceConfig.model_validate(source_config or {})
|
||||||
|
except Exception as exc:
|
||||||
|
return [], f"Invalid crawl4ai source_config: {exc}"
|
||||||
|
|
||||||
|
try:
|
||||||
|
from crawl4ai import AsyncWebCrawler # type: ignore
|
||||||
|
except ImportError:
|
||||||
|
return [], (
|
||||||
|
"crawl4ai is not installed in this worker image. "
|
||||||
|
"Use cp-workers-web (ENABLED_ADAPTERS=crawl4ai)."
|
||||||
|
)
|
||||||
|
|
||||||
|
if cfg.extract_mode == "llm" and not llm_enabled():
|
||||||
|
return [], "extract_mode=llm requires DEEPSEEK_API_KEY in worker env"
|
||||||
|
|
||||||
|
events: list[dict] = []
|
||||||
|
errors: list[str] = []
|
||||||
|
|
||||||
|
async with AsyncWebCrawler(verbose=False) as crawler:
|
||||||
|
for url in cfg.urls:
|
||||||
|
try:
|
||||||
|
if cfg.extract_mode == "llm":
|
||||||
|
item, err = await _crawl_with_llm(crawler, url, cfg)
|
||||||
|
else:
|
||||||
|
item, err = await _crawl_heuristic(crawler, url, cfg)
|
||||||
|
if err:
|
||||||
|
errors.append(err)
|
||||||
|
if item:
|
||||||
|
events.append(item)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.exception("Crawl4AI failed for %s", url)
|
||||||
|
errors.append(f"{url}: {exc}")
|
||||||
|
|
||||||
|
if not events and errors:
|
||||||
|
return [], "; ".join(errors)
|
||||||
|
return events, None
|
||||||
|
|
||||||
|
|
||||||
|
async def _crawl_heuristic(crawler: Any, url: str, cfg: Crawl4AISourceConfig):
|
||||||
|
result = await crawler.arun(url=url)
|
||||||
|
markdown = _result_markdown(result)
|
||||||
|
if not markdown:
|
||||||
|
return None, f"{url}: empty crawl result"
|
||||||
|
return (
|
||||||
|
_markdown_to_ingest(
|
||||||
|
url=url,
|
||||||
|
markdown=markdown,
|
||||||
|
extract_schema=cfg.extract_schema,
|
||||||
|
domain_profile=cfg.domain_profile,
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _crawl_with_llm(crawler: Any, url: str, cfg: Crawl4AISourceConfig):
|
||||||
|
"""Prefer Crawl4AI LLMExtractionStrategy; fall back to DeepSeek on markdown."""
|
||||||
|
instruction = cfg.instruction or DEFAULT_INSTRUCTION
|
||||||
|
try:
|
||||||
|
from crawl4ai import CacheMode, CrawlerRunConfig # type: ignore
|
||||||
|
from crawl4ai import LLMExtractionStrategy # type: ignore
|
||||||
|
|
||||||
|
strategy = LLMExtractionStrategy(
|
||||||
|
llm_config=crawl4ai_llm_config(),
|
||||||
|
schema=_pydantic_like_schema(cfg.extract_schema),
|
||||||
|
extraction_type="schema",
|
||||||
|
instruction=instruction,
|
||||||
|
input_format="markdown",
|
||||||
|
apply_chunking=False,
|
||||||
|
extra_args={"temperature": 0.1, "max_tokens": 1200},
|
||||||
|
)
|
||||||
|
run_config = CrawlerRunConfig(
|
||||||
|
extraction_strategy=strategy,
|
||||||
|
cache_mode=CacheMode.BYPASS,
|
||||||
|
)
|
||||||
|
result = await crawler.arun(url=url, config=run_config)
|
||||||
|
markdown = _result_markdown(result)
|
||||||
|
fields = _parse_extracted_content(getattr(result, "extracted_content", None))
|
||||||
|
if fields:
|
||||||
|
return (
|
||||||
|
fields_to_ingest(
|
||||||
|
source_type="crawl4ai",
|
||||||
|
source_url=url,
|
||||||
|
raw_text=markdown or json.dumps(fields, ensure_ascii=False),
|
||||||
|
fields=fields,
|
||||||
|
domain_profile=cfg.domain_profile,
|
||||||
|
extra_metadata={"extractor": "crawl4ai_llm"},
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
if markdown:
|
||||||
|
# Fallback: same DeepSeek path as Telegram
|
||||||
|
fields = await extract_event_fields(
|
||||||
|
markdown,
|
||||||
|
extract_schema=cfg.extract_schema,
|
||||||
|
instruction=instruction,
|
||||||
|
)
|
||||||
|
if fields.get("is_event", True):
|
||||||
|
return (
|
||||||
|
fields_to_ingest(
|
||||||
|
source_type="crawl4ai",
|
||||||
|
source_url=url,
|
||||||
|
raw_text=markdown,
|
||||||
|
fields=fields,
|
||||||
|
domain_profile=cfg.domain_profile,
|
||||||
|
extra_metadata={"extractor": "deepseek_fallback"},
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
return None, f"{url}: LLM marked as non-event"
|
||||||
|
return None, f"{url}: empty LLM extraction"
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("Crawl4AI LLMStrategy failed for %s: %s; trying DeepSeek on markdown", url, exc)
|
||||||
|
result = await crawler.arun(url=url)
|
||||||
|
markdown = _result_markdown(result)
|
||||||
|
if not markdown:
|
||||||
|
return None, f"{url}: empty crawl result ({exc})"
|
||||||
|
fields = await extract_event_fields(
|
||||||
|
markdown,
|
||||||
|
extract_schema=cfg.extract_schema,
|
||||||
|
instruction=instruction,
|
||||||
|
)
|
||||||
|
if not fields.get("is_event", True):
|
||||||
|
return None, None
|
||||||
|
return (
|
||||||
|
fields_to_ingest(
|
||||||
|
source_type="crawl4ai",
|
||||||
|
source_url=url,
|
||||||
|
raw_text=markdown,
|
||||||
|
fields=fields,
|
||||||
|
domain_profile=cfg.domain_profile,
|
||||||
|
extra_metadata={"extractor": "deepseek_fallback", "llm_strategy_error": str(exc)},
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _pydantic_like_schema(extract_schema: dict[str, str]) -> dict:
|
||||||
|
properties = {
|
||||||
|
key: {"type": "string", "description": desc}
|
||||||
|
for key, desc in extract_schema.items()
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"title": "MapEvent",
|
||||||
|
"type": "object",
|
||||||
|
"properties": properties,
|
||||||
|
"required": list(extract_schema.keys()),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_extracted_content(raw: Any) -> dict[str, Any]:
|
||||||
|
if not raw:
|
||||||
|
return {}
|
||||||
|
try:
|
||||||
|
data = json.loads(raw) if isinstance(raw, str) else raw
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return {}
|
||||||
|
if isinstance(data, list) and data:
|
||||||
|
data = data[0]
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
return {}
|
||||||
|
# Unwrap common nesting
|
||||||
|
if isinstance(data.get("fields"), dict):
|
||||||
|
data = data["fields"]
|
||||||
|
return {k: str(v).strip() if v is not None else "" for k, v in data.items()}
|
||||||
|
|
||||||
|
|
||||||
|
def _result_markdown(result: Any) -> str:
|
||||||
|
markdown = getattr(result, "markdown", None) or ""
|
||||||
|
if hasattr(markdown, "raw_markdown"):
|
||||||
|
return str(markdown.raw_markdown or "")
|
||||||
|
if isinstance(markdown, str):
|
||||||
|
return markdown
|
||||||
|
fit = getattr(result, "fit_markdown", None)
|
||||||
|
return str(fit or "")
|
||||||
|
|
||||||
|
|
||||||
|
def _markdown_to_ingest(
|
||||||
|
*,
|
||||||
|
url: str,
|
||||||
|
markdown: str,
|
||||||
|
extract_schema: dict[str, str],
|
||||||
|
domain_profile: str,
|
||||||
|
) -> dict:
|
||||||
|
fields = _extract_fields(markdown, extract_schema)
|
||||||
|
lat, lng = parse_coords(fields.get("coords", ""))
|
||||||
|
description = fields.get("description") or markdown[:4000]
|
||||||
|
locality = fields.get("locality") or ""
|
||||||
|
title = fields.get("title") or locality or _title_from_url(url) or description.splitlines()[0][:120]
|
||||||
|
event_date = parse_date(fields.get("event_date", ""))
|
||||||
|
|
||||||
|
return {
|
||||||
|
"source_type": "crawl4ai",
|
||||||
|
"source_url": url,
|
||||||
|
"raw_text": markdown[:20000],
|
||||||
|
"title": title[:255],
|
||||||
|
"description": description[:8000],
|
||||||
|
"locality": locality,
|
||||||
|
"latitude": lat,
|
||||||
|
"longitude": lng,
|
||||||
|
"event_date": event_date.isoformat() if event_date else None,
|
||||||
|
"region": locality or None,
|
||||||
|
"topic": fields.get("topic") or domain_profile,
|
||||||
|
"tags": ["crawl4ai", domain_profile],
|
||||||
|
"metadata": {
|
||||||
|
"domain_profile": domain_profile,
|
||||||
|
"extract_schema": extract_schema,
|
||||||
|
"extracted": fields,
|
||||||
|
"extract_mode": "heuristic",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _extract_fields(markdown: str, schema: dict[str, str]) -> dict[str, str]:
|
||||||
|
fields: dict[str, str] = {}
|
||||||
|
for key in schema:
|
||||||
|
if key == "coords":
|
||||||
|
match = COORDS_RE.search(markdown)
|
||||||
|
fields[key] = match.group(0) if match else ""
|
||||||
|
elif key == "event_date":
|
||||||
|
match = DATE_RE.search(markdown)
|
||||||
|
fields[key] = match.group(1) if match else ""
|
||||||
|
elif key == "description":
|
||||||
|
fields[key] = markdown.strip()[:4000]
|
||||||
|
elif key == "locality":
|
||||||
|
fields[key] = _guess_locality(markdown)
|
||||||
|
elif key == "title":
|
||||||
|
fields[key] = _guess_locality(markdown)
|
||||||
|
else:
|
||||||
|
fields[key] = ""
|
||||||
|
return fields
|
||||||
|
|
||||||
|
|
||||||
|
def _guess_locality(markdown: str) -> str:
|
||||||
|
for line in markdown.splitlines():
|
||||||
|
stripped = line.strip().lstrip("#").strip()
|
||||||
|
if 2 <= len(stripped) <= 80 and not COORDS_RE.search(stripped):
|
||||||
|
return stripped
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def _title_from_url(url: str) -> str:
|
||||||
|
path = urlparse(url).path.rstrip("/")
|
||||||
|
if not path:
|
||||||
|
return urlparse(url).netloc
|
||||||
|
return path.rsplit("/", 1)[-1].replace("-", " ").replace("_", " ")
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
"""Registry of source adapters enabled for this worker process."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
from workers.adapters.base import SourceAdapter
|
||||||
|
|
||||||
|
logger = logging.getLogger("cp-worker.adapters")
|
||||||
|
|
||||||
|
AdapterFactory = Callable[[], SourceAdapter]
|
||||||
|
|
||||||
|
|
||||||
|
class AdapterRegistry:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._adapters: dict[str, SourceAdapter] = {}
|
||||||
|
|
||||||
|
def register(self, adapter: SourceAdapter) -> None:
|
||||||
|
self._adapters[adapter.source_type] = adapter
|
||||||
|
logger.info("Registered adapter: %s", adapter.source_type)
|
||||||
|
|
||||||
|
def get(self, source_type: str) -> SourceAdapter | None:
|
||||||
|
return self._adapters.get(source_type)
|
||||||
|
|
||||||
|
def enabled_types(self) -> list[str]:
|
||||||
|
return sorted(self._adapters.keys())
|
||||||
|
|
||||||
|
def __contains__(self, source_type: str) -> bool:
|
||||||
|
return source_type in self._adapters
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_enabled_adapters() -> set[str] | None:
|
||||||
|
raw = os.getenv("ENABLED_ADAPTERS", "").strip()
|
||||||
|
if not raw:
|
||||||
|
return None
|
||||||
|
return {part.strip() for part in raw.split(",") if part.strip()}
|
||||||
|
|
||||||
|
|
||||||
|
def _factories() -> dict[str, AdapterFactory]:
|
||||||
|
# Lazy imports so telegram-only / web-only / nlp-only images do not need all deps
|
||||||
|
def telegram() -> SourceAdapter:
|
||||||
|
from workers.adapters.telegram import TelegramAdapter
|
||||||
|
|
||||||
|
return TelegramAdapter()
|
||||||
|
|
||||||
|
def crawl4ai() -> SourceAdapter:
|
||||||
|
from workers.adapters.crawl4ai_adapter import Crawl4AIAdapter
|
||||||
|
|
||||||
|
return Crawl4AIAdapter()
|
||||||
|
|
||||||
|
def viina() -> SourceAdapter:
|
||||||
|
from workers.adapters.viina import ViinaAdapter
|
||||||
|
|
||||||
|
return ViinaAdapter()
|
||||||
|
|
||||||
|
return {
|
||||||
|
"telegram": telegram,
|
||||||
|
"crawl4ai": crawl4ai,
|
||||||
|
"viina": viina,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_registry() -> AdapterRegistry:
|
||||||
|
"""Register adapters filtered by ENABLED_ADAPTERS (comma-separated).
|
||||||
|
|
||||||
|
Empty ENABLED_ADAPTERS → attempt to load every known adapter; skip import failures.
|
||||||
|
"""
|
||||||
|
enabled = _parse_enabled_adapters()
|
||||||
|
factories = _factories()
|
||||||
|
names = sorted(enabled) if enabled is not None else sorted(factories)
|
||||||
|
|
||||||
|
registry = AdapterRegistry()
|
||||||
|
for name in names:
|
||||||
|
factory = factories.get(name)
|
||||||
|
if factory is None:
|
||||||
|
logger.warning("Unknown adapter in ENABLED_ADAPTERS: %s", name)
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
registry.register(factory())
|
||||||
|
except Exception:
|
||||||
|
logger.exception("Failed to load adapter %s", name)
|
||||||
|
|
||||||
|
if not registry.enabled_types():
|
||||||
|
logger.warning("No adapters enabled (ENABLED_ADAPTERS=%r)", os.getenv("ENABLED_ADAPTERS"))
|
||||||
|
else:
|
||||||
|
logger.info("Enabled adapters: %s", ", ".join(registry.enabled_types()))
|
||||||
|
return registry
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
"""Telegram batch adapter (Telethon history → ingest events)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from contracts.sources import TelegramSourceConfig
|
||||||
|
from workers.adapters.base import WorkerContext
|
||||||
|
from workers.converter import event_record_to_ingest
|
||||||
|
from workers.llm_extract import (
|
||||||
|
DEFAULT_EXTRACT_SCHEMA,
|
||||||
|
DEFAULT_INSTRUCTION,
|
||||||
|
extract_event_fields,
|
||||||
|
fields_to_ingest,
|
||||||
|
llm_enabled,
|
||||||
|
)
|
||||||
|
from workers.parsers.telegram_events import parse_event_posts
|
||||||
|
from workers.sources.telegram_client import (
|
||||||
|
TelegramAuthError,
|
||||||
|
TelegramConfigError,
|
||||||
|
fetch_channel_posts,
|
||||||
|
normalize_channel,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger("cp-worker.telegram")
|
||||||
|
|
||||||
|
|
||||||
|
class TelegramAdapter:
|
||||||
|
source_type = "telegram"
|
||||||
|
|
||||||
|
async def run(
|
||||||
|
self,
|
||||||
|
job_id: int,
|
||||||
|
source_config: dict,
|
||||||
|
*,
|
||||||
|
ctx: WorkerContext,
|
||||||
|
) -> tuple[list[dict], str | None]:
|
||||||
|
try:
|
||||||
|
cfg = TelegramSourceConfig.model_validate(source_config or {})
|
||||||
|
except Exception as exc:
|
||||||
|
return [], f"Invalid telegram source_config: {exc}"
|
||||||
|
|
||||||
|
try:
|
||||||
|
username = normalize_channel(cfg.channel)
|
||||||
|
posts = await fetch_channel_posts(
|
||||||
|
username,
|
||||||
|
limit=cfg.limit,
|
||||||
|
client=ctx.tg_client,
|
||||||
|
)
|
||||||
|
except (TelegramConfigError, TelegramAuthError, ValueError) as exc:
|
||||||
|
return [], str(exc)
|
||||||
|
except Exception as exc:
|
||||||
|
return [], f"Telegram: {exc}"
|
||||||
|
|
||||||
|
if cfg.extract_mode == "llm":
|
||||||
|
if not llm_enabled():
|
||||||
|
return [], "extract_mode=llm requires DEEPSEEK_API_KEY in worker env"
|
||||||
|
return await _extract_posts_with_llm(posts, cfg)
|
||||||
|
|
||||||
|
records = parse_event_posts(posts)
|
||||||
|
events = [event_record_to_ingest(r) for r in records]
|
||||||
|
return events, None
|
||||||
|
|
||||||
|
|
||||||
|
async def _extract_posts_with_llm(posts, cfg: TelegramSourceConfig) -> tuple[list[dict], str | None]:
|
||||||
|
schema = cfg.extract_schema or DEFAULT_EXTRACT_SCHEMA
|
||||||
|
instruction = cfg.instruction or DEFAULT_INSTRUCTION
|
||||||
|
events: list[dict] = []
|
||||||
|
errors: list[str] = []
|
||||||
|
|
||||||
|
for post in posts:
|
||||||
|
text = (post.text or "").strip()
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
fields = await extract_event_fields(
|
||||||
|
text,
|
||||||
|
extract_schema=schema,
|
||||||
|
instruction=instruction,
|
||||||
|
)
|
||||||
|
if not fields.get("is_event", True):
|
||||||
|
continue
|
||||||
|
events.append(
|
||||||
|
fields_to_ingest(
|
||||||
|
source_type="telegram",
|
||||||
|
source_url=post.url,
|
||||||
|
raw_text=text,
|
||||||
|
fields=fields,
|
||||||
|
domain_profile="telegram_llm",
|
||||||
|
extra_metadata={
|
||||||
|
"channel": post.channel,
|
||||||
|
"message_id": post.id,
|
||||||
|
"post_date": post.date.isoformat() if post.date else None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.exception("LLM extract failed for %s", post.url)
|
||||||
|
errors.append(f"{post.url}: {exc}")
|
||||||
|
|
||||||
|
if not events and errors:
|
||||||
|
return [], "; ".join(errors[:5])
|
||||||
|
return events, None
|
||||||
@@ -0,0 +1,155 @@
|
|||||||
|
"""VIINA-style news incident adapter: fetch/text → extract → ingest."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from email.utils import parsedate_to_datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from bs4 import BeautifulSoup
|
||||||
|
|
||||||
|
from contracts.sources import ViinaSourceConfig
|
||||||
|
from workers.adapters.base import WorkerContext
|
||||||
|
|
||||||
|
logger = logging.getLogger("cp-worker.viina")
|
||||||
|
|
||||||
|
COORDS_RE = re.compile(r"(-?\d{1,3}\.\d+)\s*,\s*(-?\d{1,3}\.\d+)")
|
||||||
|
DATE_RE = re.compile(
|
||||||
|
r"\b(\d{1,2}[./]\d{1,2}[./]\d{2,4}|\d{4}-\d{2}-\d{2})\b",
|
||||||
|
)
|
||||||
|
# Lightweight incident cues inspired by VIINA-style violent-event coding
|
||||||
|
INCIDENT_CUES = re.compile(
|
||||||
|
r"\b(attack|shelling|strike|explosion|casualty|killed|wounded|"
|
||||||
|
r"обстрел|удар|взрыв|погибли|ранены|атака)\b",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ViinaAdapter:
|
||||||
|
source_type = "viina"
|
||||||
|
|
||||||
|
async def run(
|
||||||
|
self,
|
||||||
|
job_id: int,
|
||||||
|
source_config: dict,
|
||||||
|
*,
|
||||||
|
ctx: WorkerContext,
|
||||||
|
) -> tuple[list[dict], str | None]:
|
||||||
|
try:
|
||||||
|
cfg = ViinaSourceConfig.model_validate(source_config or {})
|
||||||
|
except Exception as exc:
|
||||||
|
return [], f"Invalid viina source_config: {exc}"
|
||||||
|
|
||||||
|
articles: list[tuple[str, str]] = []
|
||||||
|
errors: list[str] = []
|
||||||
|
|
||||||
|
if cfg.input_mode in ("urls", "mixed"):
|
||||||
|
for url in cfg.urls:
|
||||||
|
try:
|
||||||
|
text = await _fetch_article_text(url)
|
||||||
|
if text:
|
||||||
|
articles.append((url, text))
|
||||||
|
else:
|
||||||
|
errors.append(f"{url}: empty article")
|
||||||
|
except Exception as exc:
|
||||||
|
logger.exception("VIINA fetch failed for %s", url)
|
||||||
|
errors.append(f"{url}: {exc}")
|
||||||
|
|
||||||
|
if cfg.input_mode in ("texts", "mixed"):
|
||||||
|
for idx, text in enumerate(cfg.texts):
|
||||||
|
articles.append((f"viina:text:{job_id}:{idx}", text))
|
||||||
|
|
||||||
|
events = [_article_to_ingest(source_url, text) for source_url, text in articles]
|
||||||
|
# Keep articles without strong cues — still useful raw intelligence
|
||||||
|
if not events and errors:
|
||||||
|
return [], "; ".join(errors)
|
||||||
|
return events, None
|
||||||
|
|
||||||
|
|
||||||
|
async def _fetch_article_text(url: str) -> str:
|
||||||
|
async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
|
||||||
|
response = await client.get(
|
||||||
|
url,
|
||||||
|
headers={"User-Agent": "MapMil-CP-Viina/1.0"},
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
content_type = response.headers.get("content-type", "")
|
||||||
|
if "html" in content_type or url.endswith(".html"):
|
||||||
|
return _html_to_text(response.text)
|
||||||
|
return response.text.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _html_to_text(html: str) -> str:
|
||||||
|
soup = BeautifulSoup(html, "lxml")
|
||||||
|
for tag in soup(["script", "style", "noscript", "nav", "footer", "header"]):
|
||||||
|
tag.decompose()
|
||||||
|
article = soup.find("article") or soup.find("main") or soup.body
|
||||||
|
if article is None:
|
||||||
|
return soup.get_text("\n", strip=True)
|
||||||
|
return article.get_text("\n", strip=True)
|
||||||
|
|
||||||
|
|
||||||
|
def _article_to_ingest(source_url: str, text: str) -> dict[str, Any]:
|
||||||
|
lat, lng = _parse_coords(text)
|
||||||
|
date_match = DATE_RE.search(text)
|
||||||
|
event_date = _parse_date(date_match.group(1) if date_match else "")
|
||||||
|
cue = INCIDENT_CUES.search(text)
|
||||||
|
title = next((ln.strip() for ln in text.splitlines() if ln.strip()), source_url)[:120]
|
||||||
|
locality = _guess_locality(text)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"source_type": "viina",
|
||||||
|
"source_url": source_url,
|
||||||
|
"raw_text": text[:20000],
|
||||||
|
"title": title,
|
||||||
|
"description": text[:8000],
|
||||||
|
"locality": locality,
|
||||||
|
"latitude": lat,
|
||||||
|
"longitude": lng,
|
||||||
|
"event_date": event_date.isoformat() if event_date else None,
|
||||||
|
"region": locality or None,
|
||||||
|
"topic": "violent_incident" if cue else "news",
|
||||||
|
"tags": ["viina", "news"] + ([cue.group(0).lower()] if cue else []),
|
||||||
|
"metadata": {
|
||||||
|
"extractor": "viina_heuristic",
|
||||||
|
"incident_cue": cue.group(0) if cue else None,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_coords(raw: str) -> tuple[float | None, float | None]:
|
||||||
|
match = COORDS_RE.search(raw or "")
|
||||||
|
if not match:
|
||||||
|
return None, None
|
||||||
|
return float(match.group(1)), float(match.group(2))
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_date(raw: str) -> datetime | None:
|
||||||
|
if not raw:
|
||||||
|
return None
|
||||||
|
raw = raw.strip()
|
||||||
|
for fmt in ("%d.%m.%Y", "%d.%m.%y", "%d/%m/%Y", "%d/%m/%y", "%Y-%m-%d"):
|
||||||
|
try:
|
||||||
|
return datetime.strptime(raw, fmt).replace(tzinfo=timezone.utc)
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
dt = parsedate_to_datetime(raw)
|
||||||
|
if dt.tzinfo is None:
|
||||||
|
dt = dt.replace(tzinfo=timezone.utc)
|
||||||
|
return dt
|
||||||
|
except (TypeError, ValueError, IndexError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _guess_locality(text: str) -> str:
|
||||||
|
for line in text.splitlines()[:15]:
|
||||||
|
stripped = line.strip()
|
||||||
|
if 2 <= len(stripped) <= 60 and not DATE_RE.search(stripped):
|
||||||
|
if INCIDENT_CUES.search(stripped):
|
||||||
|
continue
|
||||||
|
return stripped
|
||||||
|
return ""
|
||||||
@@ -0,0 +1,179 @@
|
|||||||
|
"""DeepSeek / OpenAI-compatible LLM extraction for unstructured text."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
logger = logging.getLogger("cp-worker.llm")
|
||||||
|
|
||||||
|
COORDS_RE = re.compile(r"(-?\d{1,3}\.\d+)\s*,\s*(-?\d{1,3}\.\d+)")
|
||||||
|
|
||||||
|
DEFAULT_EXTRACT_SCHEMA: dict[str, str] = {
|
||||||
|
"title": "string — short event title",
|
||||||
|
"locality": "string — place / settlement name",
|
||||||
|
"event_date": "string — date as DD.MM.YYYY or YYYY-MM-DD if known",
|
||||||
|
"description": "string — concise event summary",
|
||||||
|
"coords": "string — latitude, longitude if present else empty",
|
||||||
|
"topic": "string — short topic tag",
|
||||||
|
}
|
||||||
|
|
||||||
|
DEFAULT_INSTRUCTION = (
|
||||||
|
"Extract structured military/news event fields from the text. "
|
||||||
|
"If the text is not an event, return is_event=false. "
|
||||||
|
"Respond with a single JSON object only."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def llm_enabled() -> bool:
|
||||||
|
return bool(os.getenv("DEEPSEEK_API_KEY", "").strip())
|
||||||
|
|
||||||
|
|
||||||
|
def llm_settings() -> dict[str, str]:
|
||||||
|
return {
|
||||||
|
"api_key": os.getenv("DEEPSEEK_API_KEY", "").strip(),
|
||||||
|
"base_url": os.getenv("DEEPSEEK_BASE_URL", "https://api.deepseek.com").rstrip("/"),
|
||||||
|
"model": os.getenv("DEEPSEEK_MODEL", "deepseek-chat"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
async def extract_event_fields(
|
||||||
|
text: str,
|
||||||
|
*,
|
||||||
|
extract_schema: dict[str, str] | None = None,
|
||||||
|
instruction: str | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Ask DeepSeek to fill schema fields from free text. Returns dict (+ is_event)."""
|
||||||
|
settings = llm_settings()
|
||||||
|
if not settings["api_key"]:
|
||||||
|
raise RuntimeError(
|
||||||
|
"DEEPSEEK_API_KEY is not set. Add it to .env for LLM extract_mode."
|
||||||
|
)
|
||||||
|
|
||||||
|
schema = extract_schema or DEFAULT_EXTRACT_SCHEMA
|
||||||
|
instr = instruction or DEFAULT_INSTRUCTION
|
||||||
|
schema_lines = "\n".join(f"- {k}: {v}" for k, v in schema.items())
|
||||||
|
user_prompt = (
|
||||||
|
f"{instr}\n\n"
|
||||||
|
f"Fields to extract:\n{schema_lines}\n\n"
|
||||||
|
'Return JSON: {"is_event": true|false, "fields": {<field>: <string>}}\n\n'
|
||||||
|
f"Text:\n{text[:12000]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
payload = {
|
||||||
|
"model": settings["model"],
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"content": (
|
||||||
|
"You extract structured event data for a geoint map. "
|
||||||
|
"Output valid JSON only, no markdown."
|
||||||
|
),
|
||||||
|
},
|
||||||
|
{"role": "user", "content": user_prompt},
|
||||||
|
],
|
||||||
|
"temperature": 0.1,
|
||||||
|
"response_format": {"type": "json_object"},
|
||||||
|
}
|
||||||
|
|
||||||
|
url = f"{settings['base_url']}/chat/completions"
|
||||||
|
async with httpx.AsyncClient(timeout=90.0) as client:
|
||||||
|
response = await client.post(
|
||||||
|
url,
|
||||||
|
headers={
|
||||||
|
"Authorization": f"Bearer {settings['api_key']}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
json=payload,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
data = response.json()
|
||||||
|
|
||||||
|
content = data["choices"][0]["message"]["content"]
|
||||||
|
parsed = json.loads(content)
|
||||||
|
fields = parsed.get("fields") if isinstance(parsed.get("fields"), dict) else parsed
|
||||||
|
if not isinstance(fields, dict):
|
||||||
|
fields = {}
|
||||||
|
# Normalize to strings for known keys
|
||||||
|
result = {key: str(fields.get(key) or "").strip() for key in schema}
|
||||||
|
result["is_event"] = bool(parsed.get("is_event", True))
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def fields_to_ingest(
|
||||||
|
*,
|
||||||
|
source_type: str,
|
||||||
|
source_url: str,
|
||||||
|
raw_text: str,
|
||||||
|
fields: dict[str, Any],
|
||||||
|
domain_profile: str = "llm",
|
||||||
|
extra_metadata: dict | None = None,
|
||||||
|
) -> dict:
|
||||||
|
lat, lng = parse_coords(str(fields.get("coords") or ""))
|
||||||
|
description = str(fields.get("description") or raw_text)[:8000]
|
||||||
|
locality = str(fields.get("locality") or "")
|
||||||
|
title = str(fields.get("title") or locality or description.splitlines()[0][:120])
|
||||||
|
topic = str(fields.get("topic") or domain_profile)
|
||||||
|
event_date = parse_date(str(fields.get("event_date") or ""))
|
||||||
|
|
||||||
|
meta = {
|
||||||
|
"extract_mode": "llm",
|
||||||
|
"extracted": {k: fields.get(k) for k in fields if k != "is_event"},
|
||||||
|
}
|
||||||
|
if extra_metadata:
|
||||||
|
meta.update(extra_metadata)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"source_type": source_type,
|
||||||
|
"source_url": source_url,
|
||||||
|
"raw_text": raw_text[:20000],
|
||||||
|
"title": title[:255],
|
||||||
|
"description": description,
|
||||||
|
"locality": locality,
|
||||||
|
"latitude": lat,
|
||||||
|
"longitude": lng,
|
||||||
|
"event_date": event_date.isoformat() if event_date else None,
|
||||||
|
"region": locality or None,
|
||||||
|
"topic": topic,
|
||||||
|
"tags": [source_type, "llm", domain_profile],
|
||||||
|
"metadata": meta,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_coords(raw: str) -> tuple[float | None, float | None]:
|
||||||
|
match = COORDS_RE.search(raw or "")
|
||||||
|
if not match:
|
||||||
|
return None, None
|
||||||
|
return float(match.group(1)), float(match.group(2))
|
||||||
|
|
||||||
|
|
||||||
|
def parse_date(raw: str) -> datetime | None:
|
||||||
|
if not raw:
|
||||||
|
return None
|
||||||
|
raw = raw.strip()
|
||||||
|
for fmt in ("%d.%m.%Y", "%d.%m.%y", "%d/%m/%Y", "%d/%m/%y", "%Y-%m-%d"):
|
||||||
|
try:
|
||||||
|
return datetime.strptime(raw, fmt).replace(tzinfo=timezone.utc)
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def crawl4ai_llm_config():
|
||||||
|
"""Build Crawl4AI LLMConfig for DeepSeek (OpenAI-compatible)."""
|
||||||
|
from crawl4ai import LLMConfig # type: ignore
|
||||||
|
|
||||||
|
settings = llm_settings()
|
||||||
|
if not settings["api_key"]:
|
||||||
|
raise RuntimeError("DEEPSEEK_API_KEY is not set")
|
||||||
|
return LLMConfig(
|
||||||
|
provider=f"openai/{settings['model']}",
|
||||||
|
api_token=settings["api_key"],
|
||||||
|
base_url=settings["base_url"],
|
||||||
|
)
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""Shared CA ↔ CP contracts (jobs, ingest, source configs, queues)."""
|
||||||
|
|||||||
@@ -2,8 +2,13 @@
|
|||||||
|
|
||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from .queues import queue_key_for_source
|
||||||
|
|
||||||
|
|
||||||
class JobPayload(BaseModel):
|
class JobPayload(BaseModel):
|
||||||
job_id: int
|
job_id: int
|
||||||
source_type: str
|
source_type: str
|
||||||
source_config: dict = Field(default_factory=dict)
|
source_config: dict = Field(default_factory=dict)
|
||||||
|
|
||||||
|
def queue_key(self) -> str:
|
||||||
|
return queue_key_for_source(self.source_type)
|
||||||
|
|||||||
@@ -0,0 +1,32 @@
|
|||||||
|
"""Redis queue routing for CP adapter families."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
# source_type → worker family (separate Redis list + Docker image)
|
||||||
|
SOURCE_FAMILY: dict[str, str] = {
|
||||||
|
"telegram": "telegram",
|
||||||
|
"crawl4ai": "web",
|
||||||
|
"viina": "nlp",
|
||||||
|
}
|
||||||
|
|
||||||
|
LEGACY_JOB_QUEUE_KEY = "cp:jobs"
|
||||||
|
QUEUE_PREFIX = "cp:jobs"
|
||||||
|
|
||||||
|
|
||||||
|
def family_for_source(source_type: str) -> str:
|
||||||
|
try:
|
||||||
|
return SOURCE_FAMILY[source_type]
|
||||||
|
except KeyError as exc:
|
||||||
|
raise ValueError(f"Unknown source_type: {source_type}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def queue_key_for_source(source_type: str) -> str:
|
||||||
|
return f"{QUEUE_PREFIX}:{family_for_source(source_type)}"
|
||||||
|
|
||||||
|
|
||||||
|
def queue_key_for_family(family: str) -> str:
|
||||||
|
return f"{QUEUE_PREFIX}:{family}"
|
||||||
|
|
||||||
|
|
||||||
|
def known_source_types() -> list[str]:
|
||||||
|
return sorted(SOURCE_FAMILY.keys())
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
"""Per-source_type source_config schemas (CA admin + CP adapters)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field, field_validator, model_validator
|
||||||
|
|
||||||
|
|
||||||
|
class TelegramSourceConfig(BaseModel):
|
||||||
|
channel: str = Field(min_length=1)
|
||||||
|
limit: int = Field(default=100, ge=1, le=1000)
|
||||||
|
# heuristic = legacy telegram_events parser; llm = DeepSeek structured extract
|
||||||
|
extract_mode: Literal["heuristic", "llm"] = "heuristic"
|
||||||
|
extract_schema: dict[str, str] | None = None
|
||||||
|
instruction: str | None = None
|
||||||
|
|
||||||
|
@field_validator("channel")
|
||||||
|
@classmethod
|
||||||
|
def strip_channel(cls, value: str) -> str:
|
||||||
|
return value.strip()
|
||||||
|
|
||||||
|
|
||||||
|
class Crawl4AISourceConfig(BaseModel):
|
||||||
|
urls: list[str] = Field(min_length=1)
|
||||||
|
extract_schema: dict[str, str] = Field(
|
||||||
|
default_factory=lambda: {
|
||||||
|
"title": "string",
|
||||||
|
"locality": "string",
|
||||||
|
"event_date": "string",
|
||||||
|
"description": "string",
|
||||||
|
"coords": "string",
|
||||||
|
"topic": "string",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
domain_profile: str = "generic_news"
|
||||||
|
extract_mode: Literal["heuristic", "llm"] = "heuristic"
|
||||||
|
instruction: str | None = None
|
||||||
|
|
||||||
|
@field_validator("urls")
|
||||||
|
@classmethod
|
||||||
|
def non_empty_urls(cls, value: list[str]) -> list[str]:
|
||||||
|
cleaned = [u.strip() for u in value if u and u.strip()]
|
||||||
|
if not cleaned:
|
||||||
|
raise ValueError("urls must contain at least one URL")
|
||||||
|
return cleaned
|
||||||
|
|
||||||
|
|
||||||
|
class ViinaSourceConfig(BaseModel):
|
||||||
|
urls: list[str] = Field(default_factory=list)
|
||||||
|
texts: list[str] = Field(default_factory=list)
|
||||||
|
input_mode: Literal["urls", "texts", "mixed"] = "urls"
|
||||||
|
|
||||||
|
@field_validator("urls")
|
||||||
|
@classmethod
|
||||||
|
def strip_urls(cls, value: list[str]) -> list[str]:
|
||||||
|
return [u.strip() for u in value if u and u.strip()]
|
||||||
|
|
||||||
|
@field_validator("texts")
|
||||||
|
@classmethod
|
||||||
|
def strip_texts(cls, value: list[str]) -> list[str]:
|
||||||
|
return [t.strip() for t in value if t and t.strip()]
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def require_inputs(self) -> "ViinaSourceConfig":
|
||||||
|
if self.input_mode == "urls" and not self.urls:
|
||||||
|
raise ValueError("urls required when input_mode=urls")
|
||||||
|
if self.input_mode == "texts" and not self.texts:
|
||||||
|
raise ValueError("texts required when input_mode=texts")
|
||||||
|
if self.input_mode == "mixed" and not self.urls and not self.texts:
|
||||||
|
raise ValueError("urls or texts required when input_mode=mixed")
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
|
CONFIG_MODELS: dict[str, type[BaseModel]] = {
|
||||||
|
"telegram": TelegramSourceConfig,
|
||||||
|
"crawl4ai": Crawl4AISourceConfig,
|
||||||
|
"viina": ViinaSourceConfig,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_source_config(source_type: str, raw: dict) -> BaseModel:
|
||||||
|
model = CONFIG_MODELS.get(source_type)
|
||||||
|
if model is None:
|
||||||
|
raise ValueError(f"Unknown source_type: {source_type}")
|
||||||
|
return model.model_validate(raw or {})
|
||||||
+51
-2
@@ -19,7 +19,9 @@ services:
|
|||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
ca-api:
|
ca-api:
|
||||||
build: ./centers/analytics/api
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: centers/analytics/api/Dockerfile
|
||||||
environment:
|
environment:
|
||||||
DATABASE_URL: postgresql://mapmil:mapmil@ca-db:5432/mapmil
|
DATABASE_URL: postgresql://mapmil:mapmil@ca-db:5432/mapmil
|
||||||
REDIS_URL: redis://redis:6379/0
|
REDIS_URL: redis://redis:6379/0
|
||||||
@@ -43,13 +45,20 @@ services:
|
|||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
cp-workers:
|
cp-workers:
|
||||||
build: ./centers/parsing/workers
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: centers/parsing/workers/Dockerfile
|
||||||
|
args:
|
||||||
|
REQUIREMENTS_FILE: requirements.txt
|
||||||
|
INSTALL_PLAYWRIGHT: "0"
|
||||||
env_file:
|
env_file:
|
||||||
- .env
|
- .env
|
||||||
environment:
|
environment:
|
||||||
TELEGRAM_SESSION_PATH: /data/telegram.session
|
TELEGRAM_SESSION_PATH: /data/telegram.session
|
||||||
TELEGRAM_LISTENER_ENABLED: "true"
|
TELEGRAM_LISTENER_ENABLED: "true"
|
||||||
TELEGRAM_LISTENER_REFRESH_SECONDS: "60"
|
TELEGRAM_LISTENER_REFRESH_SECONDS: "60"
|
||||||
|
ENABLED_ADAPTERS: telegram
|
||||||
|
WORKER_FAMILIES: telegram
|
||||||
CA_API_URL: http://ca-api:8000
|
CA_API_URL: http://ca-api:8000
|
||||||
REDIS_URL: redis://redis:6379/0
|
REDIS_URL: redis://redis:6379/0
|
||||||
INTERNAL_TOKEN: dev-internal-token
|
INTERNAL_TOKEN: dev-internal-token
|
||||||
@@ -60,6 +69,46 @@ services:
|
|||||||
- ca-api
|
- ca-api
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
|
cp-workers-web:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: centers/parsing/workers/Dockerfile
|
||||||
|
args:
|
||||||
|
REQUIREMENTS_FILE: requirements-web.txt
|
||||||
|
INSTALL_PLAYWRIGHT: "1"
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
environment:
|
||||||
|
TELEGRAM_LISTENER_ENABLED: "false"
|
||||||
|
ENABLED_ADAPTERS: crawl4ai
|
||||||
|
WORKER_FAMILIES: web
|
||||||
|
CA_API_URL: http://ca-api:8000
|
||||||
|
REDIS_URL: redis://redis:6379/0
|
||||||
|
INTERNAL_TOKEN: dev-internal-token
|
||||||
|
depends_on:
|
||||||
|
- redis
|
||||||
|
- ca-api
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
cp-workers-nlp:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: centers/parsing/workers/Dockerfile
|
||||||
|
args:
|
||||||
|
REQUIREMENTS_FILE: requirements-nlp.txt
|
||||||
|
INSTALL_PLAYWRIGHT: "0"
|
||||||
|
environment:
|
||||||
|
TELEGRAM_LISTENER_ENABLED: "false"
|
||||||
|
ENABLED_ADAPTERS: viina
|
||||||
|
WORKER_FAMILIES: nlp
|
||||||
|
CA_API_URL: http://ca-api:8000
|
||||||
|
REDIS_URL: redis://redis:6379/0
|
||||||
|
INTERNAL_TOKEN: dev-internal-token
|
||||||
|
depends_on:
|
||||||
|
- redis
|
||||||
|
- ca-api
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
pgdata:
|
pgdata:
|
||||||
ca_uploads:
|
ca_uploads:
|
||||||
|
|||||||
@@ -1,18 +0,0 @@
|
|||||||
FROM node:22-alpine AS build
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
|
|
||||||
COPY package.json ./
|
|
||||||
RUN npm install
|
|
||||||
|
|
||||||
COPY . .
|
|
||||||
RUN npm run build
|
|
||||||
|
|
||||||
FROM nginx:alpine
|
|
||||||
|
|
||||||
COPY nginx.conf /etc/nginx/conf.d/default.conf
|
|
||||||
COPY --from=build /app/dist /usr/share/nginx/html
|
|
||||||
|
|
||||||
EXPOSE 80
|
|
||||||
|
|
||||||
CMD ["nginx", "-g", "daemon off;"]
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
<!DOCTYPE html>
|
|
||||||
<html lang="ru">
|
|
||||||
<head>
|
|
||||||
<meta charset="UTF-8" />
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
|
||||||
<title>MapMil</title>
|
|
||||||
<link
|
|
||||||
rel="stylesheet"
|
|
||||||
href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css"
|
|
||||||
integrity="sha256-p4NxAoJBhIIN+hmNHrzRCf9tD/miZyoHS5obTRR9BMY="
|
|
||||||
crossorigin=""
|
|
||||||
/>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<div id="app"></div>
|
|
||||||
<script type="module" src="/src/main.ts"></script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
server {
|
|
||||||
listen 80;
|
|
||||||
server_name localhost;
|
|
||||||
root /usr/share/nginx/html;
|
|
||||||
index index.html;
|
|
||||||
|
|
||||||
location /api/ {
|
|
||||||
client_max_body_size 50M;
|
|
||||||
proxy_pass http://backend:8000/api/;
|
|
||||||
proxy_set_header Host $host;
|
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
|
||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
|
||||||
}
|
|
||||||
|
|
||||||
location / {
|
|
||||||
try_files $uri $uri/ /index.html;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "mapmil-frontend",
|
|
||||||
"private": true,
|
|
||||||
"version": "1.0.0",
|
|
||||||
"type": "module",
|
|
||||||
"scripts": {
|
|
||||||
"dev": "vite",
|
|
||||||
"build": "vite build",
|
|
||||||
"preview": "vite preview"
|
|
||||||
},
|
|
||||||
"dependencies": {
|
|
||||||
"leaflet": "^1.9.4",
|
|
||||||
"vue": "^3.5.13"
|
|
||||||
},
|
|
||||||
"devDependencies": {
|
|
||||||
"@types/leaflet": "^1.9.15",
|
|
||||||
"@vitejs/plugin-vue": "^5.2.1",
|
|
||||||
"typescript": "~5.6.3",
|
|
||||||
"vite": "^6.0.3",
|
|
||||||
"vue-tsc": "^2.1.10"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,349 +0,0 @@
|
|||||||
<script setup lang="ts">
|
|
||||||
import { computed, onMounted, ref, watch } from "vue";
|
|
||||||
import {
|
|
||||||
createObject,
|
|
||||||
deleteObject,
|
|
||||||
fetchObjects,
|
|
||||||
updateObject,
|
|
||||||
uploadObjectMedia,
|
|
||||||
} from "./api/objects";
|
|
||||||
import ContextMenu from "./components/ContextMenu.vue";
|
|
||||||
import CreateObjectModal from "./components/CreateObjectModal.vue";
|
|
||||||
import EditObjectModal from "./components/EditObjectModal.vue";
|
|
||||||
import MapView from "./components/MapView.vue";
|
|
||||||
import ObjectPanel from "./components/ObjectPanel.vue";
|
|
||||||
import TimelineBar from "./components/TimelineBar.vue";
|
|
||||||
import type { MapObject, MapObjectCreate, ObjectType } from "./types/object";
|
|
||||||
|
|
||||||
const objects = ref<MapObject[]>([]);
|
|
||||||
const selectedObject = ref<MapObject | null>(null);
|
|
||||||
const loading = ref(true);
|
|
||||||
const error = ref("");
|
|
||||||
const timelinePosition = ref(Date.now());
|
|
||||||
|
|
||||||
const contextMenu = ref<{
|
|
||||||
visible: boolean;
|
|
||||||
x: number;
|
|
||||||
y: number;
|
|
||||||
latitude: number;
|
|
||||||
longitude: number;
|
|
||||||
target: "map" | "object";
|
|
||||||
object: MapObject | null;
|
|
||||||
}>({
|
|
||||||
visible: false,
|
|
||||||
x: 0,
|
|
||||||
y: 0,
|
|
||||||
latitude: 0,
|
|
||||||
longitude: 0,
|
|
||||||
target: "map",
|
|
||||||
object: null,
|
|
||||||
});
|
|
||||||
|
|
||||||
const createModal = ref({
|
|
||||||
visible: false,
|
|
||||||
latitude: 0,
|
|
||||||
longitude: 0,
|
|
||||||
});
|
|
||||||
|
|
||||||
const editModal = ref({
|
|
||||||
visible: false,
|
|
||||||
object: null as MapObject | null,
|
|
||||||
});
|
|
||||||
|
|
||||||
const selectedId = computed(() => selectedObject.value?.id ?? null);
|
|
||||||
|
|
||||||
function objectTime(obj: MapObject): number {
|
|
||||||
return new Date(obj.created_at).getTime();
|
|
||||||
}
|
|
||||||
|
|
||||||
function replaceObject(updated: MapObject) {
|
|
||||||
objects.value = objects.value.map((obj) => (obj.id === updated.id ? updated : obj));
|
|
||||||
if (selectedObject.value?.id === updated.id) {
|
|
||||||
selectedObject.value = updated;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
const timelineBounds = computed(() => {
|
|
||||||
if (objects.value.length === 0) {
|
|
||||||
const now = Date.now();
|
|
||||||
return { min: now, max: now };
|
|
||||||
}
|
|
||||||
|
|
||||||
const times = objects.value.map(objectTime);
|
|
||||||
return {
|
|
||||||
min: Math.min(...times),
|
|
||||||
max: Math.max(...times),
|
|
||||||
};
|
|
||||||
});
|
|
||||||
|
|
||||||
const visibleObjects = computed(() =>
|
|
||||||
objects.value.filter((obj) => objectTime(obj) <= timelinePosition.value),
|
|
||||||
);
|
|
||||||
|
|
||||||
function syncTimelineToMax() {
|
|
||||||
timelinePosition.value = timelineBounds.value.max;
|
|
||||||
}
|
|
||||||
|
|
||||||
async function loadObjects() {
|
|
||||||
loading.value = true;
|
|
||||||
error.value = "";
|
|
||||||
try {
|
|
||||||
objects.value = await fetchObjects();
|
|
||||||
syncTimelineToMax();
|
|
||||||
} catch (err) {
|
|
||||||
error.value = err instanceof Error ? err.message : "Не удалось загрузить объекты";
|
|
||||||
} finally {
|
|
||||||
loading.value = false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function handleSelectObject(obj: MapObject) {
|
|
||||||
selectedObject.value = obj;
|
|
||||||
}
|
|
||||||
|
|
||||||
function handleMapContextMenu(payload: {
|
|
||||||
x: number;
|
|
||||||
y: number;
|
|
||||||
latitude: number;
|
|
||||||
longitude: number;
|
|
||||||
object: MapObject | null;
|
|
||||||
}) {
|
|
||||||
contextMenu.value = {
|
|
||||||
visible: true,
|
|
||||||
x: payload.x,
|
|
||||||
y: payload.y,
|
|
||||||
latitude: payload.latitude,
|
|
||||||
longitude: payload.longitude,
|
|
||||||
target: payload.object ? "object" : "map",
|
|
||||||
object: payload.object,
|
|
||||||
};
|
|
||||||
|
|
||||||
if (payload.object) {
|
|
||||||
selectedObject.value = payload.object;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function closeContextMenu() {
|
|
||||||
contextMenu.value.visible = false;
|
|
||||||
}
|
|
||||||
|
|
||||||
function openCreateModal() {
|
|
||||||
createModal.value = {
|
|
||||||
visible: true,
|
|
||||||
latitude: contextMenu.value.latitude,
|
|
||||||
longitude: contextMenu.value.longitude,
|
|
||||||
};
|
|
||||||
closeContextMenu();
|
|
||||||
}
|
|
||||||
|
|
||||||
function closeCreateModal() {
|
|
||||||
createModal.value.visible = false;
|
|
||||||
}
|
|
||||||
|
|
||||||
function openEditModal() {
|
|
||||||
if (!contextMenu.value.object) return;
|
|
||||||
|
|
||||||
editModal.value = {
|
|
||||||
visible: true,
|
|
||||||
object: contextMenu.value.object,
|
|
||||||
};
|
|
||||||
closeContextMenu();
|
|
||||||
}
|
|
||||||
|
|
||||||
function closeEditModal() {
|
|
||||||
editModal.value.visible = false;
|
|
||||||
editModal.value.object = null;
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleCreateObject(payload: {
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
type: ObjectType;
|
|
||||||
created_at?: string;
|
|
||||||
files: File[];
|
|
||||||
}) {
|
|
||||||
const data: MapObjectCreate = {
|
|
||||||
name: payload.name,
|
|
||||||
description: payload.description,
|
|
||||||
type: payload.type,
|
|
||||||
latitude: createModal.value.latitude,
|
|
||||||
longitude: createModal.value.longitude,
|
|
||||||
created_at: payload.created_at,
|
|
||||||
};
|
|
||||||
|
|
||||||
const created = await createObject(data);
|
|
||||||
|
|
||||||
for (const file of payload.files) {
|
|
||||||
await uploadObjectMedia(created.id, file);
|
|
||||||
}
|
|
||||||
|
|
||||||
objects.value = [...objects.value, created];
|
|
||||||
selectedObject.value = created;
|
|
||||||
timelinePosition.value = objectTime(created);
|
|
||||||
closeCreateModal();
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleEditObject(payload: {
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
type: ObjectType;
|
|
||||||
created_at: string;
|
|
||||||
}) {
|
|
||||||
if (!editModal.value.object) return;
|
|
||||||
|
|
||||||
const updated = await updateObject(editModal.value.object.id, payload);
|
|
||||||
replaceObject(updated);
|
|
||||||
timelinePosition.value = objectTime(updated);
|
|
||||||
closeEditModal();
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleDeleteObject() {
|
|
||||||
const object = contextMenu.value.object;
|
|
||||||
if (!object) return;
|
|
||||||
|
|
||||||
const confirmed = window.confirm(`Удалить объект «${object.name}»?`);
|
|
||||||
if (!confirmed) return;
|
|
||||||
|
|
||||||
closeContextMenu();
|
|
||||||
|
|
||||||
try {
|
|
||||||
await deleteObject(object.id);
|
|
||||||
objects.value = objects.value.filter((item) => item.id !== object.id);
|
|
||||||
|
|
||||||
if (selectedObject.value?.id === object.id) {
|
|
||||||
selectedObject.value = null;
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
error.value = err instanceof Error ? err.message : "Не удалось удалить объект";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleMoveObject(payload: {
|
|
||||||
object: MapObject;
|
|
||||||
latitude: number;
|
|
||||||
longitude: number;
|
|
||||||
}) {
|
|
||||||
try {
|
|
||||||
const updated = await updateObject(payload.object.id, {
|
|
||||||
latitude: payload.latitude,
|
|
||||||
longitude: payload.longitude,
|
|
||||||
});
|
|
||||||
replaceObject(updated);
|
|
||||||
} catch (err) {
|
|
||||||
error.value = err instanceof Error ? err.message : "Не удалось переместить объект";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
watch(visibleObjects, (visible) => {
|
|
||||||
if (
|
|
||||||
selectedObject.value &&
|
|
||||||
!visible.some((obj) => obj.id === selectedObject.value?.id)
|
|
||||||
) {
|
|
||||||
selectedObject.value = null;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
onMounted(loadObjects);
|
|
||||||
</script>
|
|
||||||
|
|
||||||
<template>
|
|
||||||
<div class="app" @click="closeContextMenu">
|
|
||||||
<header class="header">
|
|
||||||
<h1>MapMil</h1>
|
|
||||||
<span v-if="loading" class="status">Загрузка...</span>
|
|
||||||
<span v-else-if="error" class="status error">{{ error }}</span>
|
|
||||||
<span v-else class="status">
|
|
||||||
{{ visibleObjects.length }} / {{ objects.length }} объектов на карте
|
|
||||||
</span>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<main class="main">
|
|
||||||
<MapView
|
|
||||||
:objects="visibleObjects"
|
|
||||||
:selected-id="selectedId"
|
|
||||||
@select="handleSelectObject"
|
|
||||||
@contextmenu="handleMapContextMenu"
|
|
||||||
@move="handleMoveObject"
|
|
||||||
/>
|
|
||||||
<ObjectPanel :object="selectedObject" />
|
|
||||||
</main>
|
|
||||||
|
|
||||||
<TimelineBar
|
|
||||||
v-if="!loading && objects.length > 0"
|
|
||||||
v-model="timelinePosition"
|
|
||||||
:min="timelineBounds.min"
|
|
||||||
:max="timelineBounds.max"
|
|
||||||
:objects="objects"
|
|
||||||
:visible-count="visibleObjects.length"
|
|
||||||
/>
|
|
||||||
|
|
||||||
<ContextMenu
|
|
||||||
v-if="contextMenu.visible"
|
|
||||||
:x="contextMenu.x"
|
|
||||||
:y="contextMenu.y"
|
|
||||||
:target="contextMenu.target"
|
|
||||||
:object-name="contextMenu.object?.name"
|
|
||||||
@create="openCreateModal"
|
|
||||||
@edit="openEditModal"
|
|
||||||
@delete="handleDeleteObject"
|
|
||||||
/>
|
|
||||||
|
|
||||||
<CreateObjectModal
|
|
||||||
v-if="createModal.visible"
|
|
||||||
:latitude="createModal.latitude"
|
|
||||||
:longitude="createModal.longitude"
|
|
||||||
@close="closeCreateModal"
|
|
||||||
@submit="handleCreateObject"
|
|
||||||
/>
|
|
||||||
|
|
||||||
<EditObjectModal
|
|
||||||
v-if="editModal.visible && editModal.object"
|
|
||||||
:object="editModal.object"
|
|
||||||
@close="closeEditModal"
|
|
||||||
@submit="handleEditObject"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
</template>
|
|
||||||
|
|
||||||
<style scoped>
|
|
||||||
.app {
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
height: 100%;
|
|
||||||
}
|
|
||||||
|
|
||||||
.header {
|
|
||||||
position: relative;
|
|
||||||
z-index: 5;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 1rem;
|
|
||||||
padding: 0.75rem 1.25rem;
|
|
||||||
background: #fff;
|
|
||||||
border-bottom: 1px solid #e0e0e0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.header h1 {
|
|
||||||
margin: 0;
|
|
||||||
font-size: 1.25rem;
|
|
||||||
font-weight: 600;
|
|
||||||
}
|
|
||||||
|
|
||||||
.status {
|
|
||||||
font-size: 0.875rem;
|
|
||||||
color: #666;
|
|
||||||
}
|
|
||||||
|
|
||||||
.status.error {
|
|
||||||
color: #c62828;
|
|
||||||
}
|
|
||||||
|
|
||||||
.main {
|
|
||||||
position: relative;
|
|
||||||
z-index: 1;
|
|
||||||
display: flex;
|
|
||||||
flex: 1;
|
|
||||||
min-height: 0;
|
|
||||||
overflow: hidden;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
import type { MapObject, MapObjectCreate, MapObjectUpdate, ObjectMedia } from "../types/object";
|
|
||||||
|
|
||||||
const API_BASE = "/api";
|
|
||||||
|
|
||||||
async function request<T>(url: string, options?: RequestInit): Promise<T> {
|
|
||||||
const headers = new Headers(options?.headers);
|
|
||||||
const isFormData = options?.body instanceof FormData;
|
|
||||||
|
|
||||||
if (!isFormData && !headers.has("Content-Type")) {
|
|
||||||
headers.set("Content-Type", "application/json");
|
|
||||||
}
|
|
||||||
|
|
||||||
const response = await fetch(`${API_BASE}${url}`, {
|
|
||||||
...options,
|
|
||||||
headers,
|
|
||||||
});
|
|
||||||
|
|
||||||
if (!response.ok) {
|
|
||||||
const message = await response.text();
|
|
||||||
throw new Error(message || `Ошибка запроса: ${response.status}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (response.status === 204) {
|
|
||||||
return undefined as T;
|
|
||||||
}
|
|
||||||
|
|
||||||
return response.json() as Promise<T>;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function fetchObjects(): Promise<MapObject[]> {
|
|
||||||
return request<MapObject[]>("/objects");
|
|
||||||
}
|
|
||||||
|
|
||||||
export function fetchObject(id: number): Promise<MapObject> {
|
|
||||||
return request<MapObject>(`/objects/${id}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function createObject(payload: MapObjectCreate): Promise<MapObject> {
|
|
||||||
return request<MapObject>("/objects", {
|
|
||||||
method: "POST",
|
|
||||||
body: JSON.stringify(payload),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
export function updateObject(id: number, payload: MapObjectUpdate): Promise<MapObject> {
|
|
||||||
return request<MapObject>(`/objects/${id}`, {
|
|
||||||
method: "PATCH",
|
|
||||||
body: JSON.stringify(payload),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
export function deleteObject(id: number): Promise<void> {
|
|
||||||
return request<void>(`/objects/${id}`, {
|
|
||||||
method: "DELETE",
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
export function fetchObjectMedia(objectId: number): Promise<ObjectMedia[]> {
|
|
||||||
return request<ObjectMedia[]>(`/objects/${objectId}/media`);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function uploadObjectMedia(objectId: number, file: File): Promise<ObjectMedia> {
|
|
||||||
const form = new FormData();
|
|
||||||
form.append("file", file);
|
|
||||||
|
|
||||||
return request<ObjectMedia>(`/objects/${objectId}/media`, {
|
|
||||||
method: "POST",
|
|
||||||
body: form,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
export function deleteObjectMedia(mediaId: number): Promise<void> {
|
|
||||||
return request<void>(`/media/${mediaId}`, {
|
|
||||||
method: "DELETE",
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
export function formatFileSize(bytes: number): string {
|
|
||||||
if (bytes < 1024) return `${bytes} Б`;
|
|
||||||
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} КБ`;
|
|
||||||
return `${(bytes / (1024 * 1024)).toFixed(1)} МБ`;
|
|
||||||
}
|
|
||||||
@@ -1,81 +0,0 @@
|
|||||||
<script setup lang="ts">
|
|
||||||
defineProps<{
|
|
||||||
x: number;
|
|
||||||
y: number;
|
|
||||||
target: "map" | "object";
|
|
||||||
objectName?: string;
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const emit = defineEmits<{
|
|
||||||
create: [];
|
|
||||||
edit: [];
|
|
||||||
delete: [];
|
|
||||||
}>();
|
|
||||||
</script>
|
|
||||||
|
|
||||||
<template>
|
|
||||||
<div
|
|
||||||
class="context-menu"
|
|
||||||
:style="{ left: `${x}px`, top: `${y}px` }"
|
|
||||||
@click.stop
|
|
||||||
>
|
|
||||||
<p v-if="target === 'object' && objectName" class="title">{{ objectName }}</p>
|
|
||||||
|
|
||||||
<template v-if="target === 'map'">
|
|
||||||
<button type="button" @click="emit('create')">Создать объект</button>
|
|
||||||
</template>
|
|
||||||
|
|
||||||
<template v-else>
|
|
||||||
<button type="button" @click="emit('edit')">Редактировать</button>
|
|
||||||
<button type="button" class="danger" @click="emit('delete')">Удалить</button>
|
|
||||||
</template>
|
|
||||||
</div>
|
|
||||||
</template>
|
|
||||||
|
|
||||||
<style scoped>
|
|
||||||
.context-menu {
|
|
||||||
position: fixed;
|
|
||||||
z-index: 1000;
|
|
||||||
min-width: 200px;
|
|
||||||
background: #fff;
|
|
||||||
border: 1px solid #d0d0d0;
|
|
||||||
border-radius: 6px;
|
|
||||||
box-shadow: 0 4px 16px rgba(0, 0, 0, 0.12);
|
|
||||||
overflow: hidden;
|
|
||||||
}
|
|
||||||
|
|
||||||
.title {
|
|
||||||
margin: 0;
|
|
||||||
padding: 0.5rem 1rem;
|
|
||||||
font-size: 0.75rem;
|
|
||||||
font-weight: 600;
|
|
||||||
color: #666;
|
|
||||||
background: #f8f8f8;
|
|
||||||
border-bottom: 1px solid #eee;
|
|
||||||
overflow: hidden;
|
|
||||||
text-overflow: ellipsis;
|
|
||||||
white-space: nowrap;
|
|
||||||
}
|
|
||||||
|
|
||||||
button {
|
|
||||||
display: block;
|
|
||||||
width: 100%;
|
|
||||||
padding: 0.625rem 1rem;
|
|
||||||
border: none;
|
|
||||||
background: #fff;
|
|
||||||
text-align: left;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
button:hover {
|
|
||||||
background: #f0f4ff;
|
|
||||||
}
|
|
||||||
|
|
||||||
button.danger {
|
|
||||||
color: #b91c1c;
|
|
||||||
}
|
|
||||||
|
|
||||||
button.danger:hover {
|
|
||||||
background: #fee2e2;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
@@ -1,253 +0,0 @@
|
|||||||
<script setup lang="ts">
|
|
||||||
import { computed, ref } from "vue";
|
|
||||||
import { MEDIA_ACCEPT, OBJECT_TYPE_LABELS, OBJECT_TYPES, type ObjectType } from "../types/object";
|
|
||||||
|
|
||||||
const props = defineProps<{
|
|
||||||
latitude: number;
|
|
||||||
longitude: number;
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const emit = defineEmits<{
|
|
||||||
close: [];
|
|
||||||
submit: [payload: {
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
type: ObjectType;
|
|
||||||
created_at?: string;
|
|
||||||
files: File[];
|
|
||||||
}];
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const name = ref("");
|
|
||||||
const description = ref("");
|
|
||||||
const type = ref<ObjectType>("marker");
|
|
||||||
const createdAt = ref(toLocalDateTimeValue(new Date()));
|
|
||||||
const selectedFiles = ref<File[]>([]);
|
|
||||||
const error = ref("");
|
|
||||||
const submitting = ref(false);
|
|
||||||
|
|
||||||
const fileLabel = computed(() => {
|
|
||||||
if (selectedFiles.value.length === 0) return "Файлы не выбраны";
|
|
||||||
return selectedFiles.value.map((file) => file.name).join(", ");
|
|
||||||
});
|
|
||||||
|
|
||||||
function toLocalDateTimeValue(date: Date): string {
|
|
||||||
const pad = (value: number) => String(value).padStart(2, "0");
|
|
||||||
return `${date.getFullYear()}-${pad(date.getMonth() + 1)}-${pad(date.getDate())}T${pad(date.getHours())}:${pad(date.getMinutes())}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
function handleFileChange(event: Event) {
|
|
||||||
const input = event.target as HTMLInputElement;
|
|
||||||
selectedFiles.value = input.files ? Array.from(input.files) : [];
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleSubmit() {
|
|
||||||
if (!name.value.trim()) {
|
|
||||||
error.value = "Введите название объекта";
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!createdAt.value) {
|
|
||||||
error.value = "Укажите дату создания";
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
error.value = "";
|
|
||||||
submitting.value = true;
|
|
||||||
|
|
||||||
try {
|
|
||||||
emit("submit", {
|
|
||||||
name: name.value.trim(),
|
|
||||||
description: description.value.trim(),
|
|
||||||
type: type.value,
|
|
||||||
created_at: new Date(createdAt.value).toISOString(),
|
|
||||||
files: selectedFiles.value,
|
|
||||||
});
|
|
||||||
} catch (err) {
|
|
||||||
error.value = err instanceof Error ? err.message : "Не удалось создать объект";
|
|
||||||
} finally {
|
|
||||||
submitting.value = false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
</script>
|
|
||||||
|
|
||||||
<template>
|
|
||||||
<div class="overlay" @click.self="emit('close')">
|
|
||||||
<div class="modal" role="dialog" aria-labelledby="create-title">
|
|
||||||
<header>
|
|
||||||
<h2 id="create-title">Создать объект</h2>
|
|
||||||
<button type="button" class="close" aria-label="Закрыть" @click="emit('close')">
|
|
||||||
×
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<form @submit.prevent="handleSubmit">
|
|
||||||
<p class="coords">
|
|
||||||
Координаты: {{ props.latitude.toFixed(6) }}, {{ props.longitude.toFixed(6) }}
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Название *
|
|
||||||
<input v-model="name" type="text" placeholder="Название объекта" required />
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Тип *
|
|
||||||
<select v-model="type" required>
|
|
||||||
<option v-for="item in OBJECT_TYPES" :key="item" :value="item">
|
|
||||||
{{ OBJECT_TYPE_LABELS[item] }}
|
|
||||||
</option>
|
|
||||||
</select>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Дата создания *
|
|
||||||
<input v-model="createdAt" type="datetime-local" required />
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Описание
|
|
||||||
<textarea
|
|
||||||
v-model="description"
|
|
||||||
rows="4"
|
|
||||||
placeholder="Описание объекта"
|
|
||||||
/>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Медиафайлы
|
|
||||||
<input
|
|
||||||
type="file"
|
|
||||||
multiple
|
|
||||||
:accept="MEDIA_ACCEPT"
|
|
||||||
@change="handleFileChange"
|
|
||||||
/>
|
|
||||||
<span class="hint">{{ fileLabel }}</span>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<p v-if="error" class="error">{{ error }}</p>
|
|
||||||
|
|
||||||
<footer>
|
|
||||||
<button type="button" class="secondary" @click="emit('close')">Отмена</button>
|
|
||||||
<button type="submit" :disabled="submitting">
|
|
||||||
{{ submitting ? "Создание..." : "Создать" }}
|
|
||||||
</button>
|
|
||||||
</footer>
|
|
||||||
</form>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</template>
|
|
||||||
|
|
||||||
<style scoped>
|
|
||||||
.overlay {
|
|
||||||
position: fixed;
|
|
||||||
inset: 0;
|
|
||||||
z-index: 1100;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: center;
|
|
||||||
background: rgba(0, 0, 0, 0.4);
|
|
||||||
}
|
|
||||||
|
|
||||||
.modal {
|
|
||||||
width: min(480px, calc(100vw - 2rem));
|
|
||||||
max-height: calc(100vh - 2rem);
|
|
||||||
overflow-y: auto;
|
|
||||||
background: #fff;
|
|
||||||
border-radius: 8px;
|
|
||||||
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.2);
|
|
||||||
}
|
|
||||||
|
|
||||||
header {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: space-between;
|
|
||||||
padding: 1rem 1.25rem;
|
|
||||||
border-bottom: 1px solid #e8e8e8;
|
|
||||||
}
|
|
||||||
|
|
||||||
header h2 {
|
|
||||||
margin: 0;
|
|
||||||
font-size: 1.125rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.close {
|
|
||||||
border: none;
|
|
||||||
background: none;
|
|
||||||
font-size: 1.5rem;
|
|
||||||
line-height: 1;
|
|
||||||
cursor: pointer;
|
|
||||||
color: #666;
|
|
||||||
}
|
|
||||||
|
|
||||||
form {
|
|
||||||
padding: 1.25rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.coords {
|
|
||||||
margin: 0 0 1rem;
|
|
||||||
font-size: 0.875rem;
|
|
||||||
color: #666;
|
|
||||||
}
|
|
||||||
|
|
||||||
label {
|
|
||||||
display: block;
|
|
||||||
margin-bottom: 1rem;
|
|
||||||
font-size: 0.875rem;
|
|
||||||
font-weight: 500;
|
|
||||||
}
|
|
||||||
|
|
||||||
input,
|
|
||||||
select,
|
|
||||||
textarea {
|
|
||||||
display: block;
|
|
||||||
width: 100%;
|
|
||||||
margin-top: 0.375rem;
|
|
||||||
padding: 0.5rem 0.75rem;
|
|
||||||
border: 1px solid #ccc;
|
|
||||||
border-radius: 4px;
|
|
||||||
}
|
|
||||||
|
|
||||||
textarea {
|
|
||||||
resize: vertical;
|
|
||||||
}
|
|
||||||
|
|
||||||
.hint {
|
|
||||||
display: block;
|
|
||||||
margin-top: 0.375rem;
|
|
||||||
font-size: 0.75rem;
|
|
||||||
font-weight: 400;
|
|
||||||
color: #666;
|
|
||||||
}
|
|
||||||
|
|
||||||
.error {
|
|
||||||
margin: 0 0 1rem;
|
|
||||||
color: #c62828;
|
|
||||||
font-size: 0.875rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
footer {
|
|
||||||
display: flex;
|
|
||||||
justify-content: flex-end;
|
|
||||||
gap: 0.5rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
button {
|
|
||||||
padding: 0.5rem 1rem;
|
|
||||||
border: none;
|
|
||||||
border-radius: 4px;
|
|
||||||
background: #2563eb;
|
|
||||||
color: #fff;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
button.secondary {
|
|
||||||
background: #e8e8e8;
|
|
||||||
color: #333;
|
|
||||||
}
|
|
||||||
|
|
||||||
button:disabled {
|
|
||||||
opacity: 0.6;
|
|
||||||
cursor: not-allowed;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
@@ -1,239 +0,0 @@
|
|||||||
<script setup lang="ts">
|
|
||||||
import { ref, watch } from "vue";
|
|
||||||
import { OBJECT_TYPE_LABELS, OBJECT_TYPES, type MapObject, type ObjectType } from "../types/object";
|
|
||||||
|
|
||||||
const props = defineProps<{
|
|
||||||
object: MapObject;
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const emit = defineEmits<{
|
|
||||||
close: [];
|
|
||||||
submit: [payload: {
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
type: ObjectType;
|
|
||||||
created_at: string;
|
|
||||||
}];
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const name = ref("");
|
|
||||||
const description = ref("");
|
|
||||||
const type = ref<ObjectType>("marker");
|
|
||||||
const createdAt = ref("");
|
|
||||||
const error = ref("");
|
|
||||||
const submitting = ref(false);
|
|
||||||
|
|
||||||
function toLocalDateTimeValue(iso: string): string {
|
|
||||||
const date = new Date(iso);
|
|
||||||
const pad = (value: number) => String(value).padStart(2, "0");
|
|
||||||
return `${date.getFullYear()}-${pad(date.getMonth() + 1)}-${pad(date.getDate())}T${pad(date.getHours())}:${pad(date.getMinutes())}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
function resetForm() {
|
|
||||||
name.value = props.object.name;
|
|
||||||
description.value = props.object.description;
|
|
||||||
type.value = props.object.type;
|
|
||||||
createdAt.value = toLocalDateTimeValue(props.object.created_at);
|
|
||||||
error.value = "";
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleSubmit() {
|
|
||||||
if (!name.value.trim()) {
|
|
||||||
error.value = "Введите название объекта";
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!createdAt.value) {
|
|
||||||
error.value = "Укажите дату создания";
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
error.value = "";
|
|
||||||
submitting.value = true;
|
|
||||||
|
|
||||||
try {
|
|
||||||
emit("submit", {
|
|
||||||
name: name.value.trim(),
|
|
||||||
description: description.value.trim(),
|
|
||||||
type: type.value,
|
|
||||||
created_at: new Date(createdAt.value).toISOString(),
|
|
||||||
});
|
|
||||||
} catch (err) {
|
|
||||||
error.value = err instanceof Error ? err.message : "Не удалось сохранить изменения";
|
|
||||||
} finally {
|
|
||||||
submitting.value = false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
watch(() => props.object, resetForm, { immediate: true });
|
|
||||||
</script>
|
|
||||||
|
|
||||||
<template>
|
|
||||||
<div class="overlay" @click.self="emit('close')">
|
|
||||||
<div class="modal" role="dialog" aria-labelledby="edit-title">
|
|
||||||
<header>
|
|
||||||
<h2 id="edit-title">Редактировать объект</h2>
|
|
||||||
<button type="button" class="close" aria-label="Закрыть" @click="emit('close')">
|
|
||||||
×
|
|
||||||
</button>
|
|
||||||
</header>
|
|
||||||
|
|
||||||
<form @submit.prevent="handleSubmit">
|
|
||||||
<p class="coords">
|
|
||||||
Координаты: {{ object.latitude.toFixed(6) }}, {{ object.longitude.toFixed(6) }}
|
|
||||||
<span class="hint">(измените через «Переместить» на карте)</span>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Название *
|
|
||||||
<input v-model="name" type="text" placeholder="Название объекта" required />
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Тип *
|
|
||||||
<select v-model="type" required>
|
|
||||||
<option v-for="item in OBJECT_TYPES" :key="item" :value="item">
|
|
||||||
{{ OBJECT_TYPE_LABELS[item] }}
|
|
||||||
</option>
|
|
||||||
</select>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Дата создания *
|
|
||||||
<input v-model="createdAt" type="datetime-local" required />
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<label>
|
|
||||||
Описание
|
|
||||||
<textarea
|
|
||||||
v-model="description"
|
|
||||||
rows="4"
|
|
||||||
placeholder="Описание объекта"
|
|
||||||
/>
|
|
||||||
</label>
|
|
||||||
|
|
||||||
<p v-if="error" class="error">{{ error }}</p>
|
|
||||||
|
|
||||||
<footer>
|
|
||||||
<button type="button" class="secondary" @click="emit('close')">Отмена</button>
|
|
||||||
<button type="submit" :disabled="submitting">
|
|
||||||
{{ submitting ? "Сохранение..." : "Сохранить" }}
|
|
||||||
</button>
|
|
||||||
</footer>
|
|
||||||
</form>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</template>
|
|
||||||
|
|
||||||
<style scoped>
|
|
||||||
.overlay {
|
|
||||||
position: fixed;
|
|
||||||
inset: 0;
|
|
||||||
z-index: 1100;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: center;
|
|
||||||
background: rgba(0, 0, 0, 0.4);
|
|
||||||
}
|
|
||||||
|
|
||||||
.modal {
|
|
||||||
width: min(480px, calc(100vw - 2rem));
|
|
||||||
max-height: calc(100vh - 2rem);
|
|
||||||
overflow-y: auto;
|
|
||||||
background: #fff;
|
|
||||||
border-radius: 8px;
|
|
||||||
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.2);
|
|
||||||
}
|
|
||||||
|
|
||||||
header {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: space-between;
|
|
||||||
padding: 1rem 1.25rem;
|
|
||||||
border-bottom: 1px solid #e8e8e8;
|
|
||||||
}
|
|
||||||
|
|
||||||
header h2 {
|
|
||||||
margin: 0;
|
|
||||||
font-size: 1.125rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.close {
|
|
||||||
border: none;
|
|
||||||
background: none;
|
|
||||||
font-size: 1.5rem;
|
|
||||||
line-height: 1;
|
|
||||||
cursor: pointer;
|
|
||||||
color: #666;
|
|
||||||
}
|
|
||||||
|
|
||||||
form {
|
|
||||||
padding: 1.25rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.coords {
|
|
||||||
margin: 0 0 1rem;
|
|
||||||
font-size: 0.875rem;
|
|
||||||
color: #666;
|
|
||||||
line-height: 1.5;
|
|
||||||
}
|
|
||||||
|
|
||||||
.hint {
|
|
||||||
display: block;
|
|
||||||
font-size: 0.75rem;
|
|
||||||
color: #999;
|
|
||||||
}
|
|
||||||
|
|
||||||
label {
|
|
||||||
display: block;
|
|
||||||
margin-bottom: 1rem;
|
|
||||||
font-size: 0.875rem;
|
|
||||||
font-weight: 500;
|
|
||||||
}
|
|
||||||
|
|
||||||
input,
|
|
||||||
select,
|
|
||||||
textarea {
|
|
||||||
display: block;
|
|
||||||
width: 100%;
|
|
||||||
margin-top: 0.375rem;
|
|
||||||
padding: 0.5rem 0.75rem;
|
|
||||||
border: 1px solid #ccc;
|
|
||||||
border-radius: 4px;
|
|
||||||
}
|
|
||||||
|
|
||||||
textarea {
|
|
||||||
resize: vertical;
|
|
||||||
}
|
|
||||||
|
|
||||||
.error {
|
|
||||||
margin: 0 0 1rem;
|
|
||||||
color: #c62828;
|
|
||||||
font-size: 0.875rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
footer {
|
|
||||||
display: flex;
|
|
||||||
justify-content: flex-end;
|
|
||||||
gap: 0.5rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
button {
|
|
||||||
padding: 0.5rem 1rem;
|
|
||||||
border: none;
|
|
||||||
border-radius: 4px;
|
|
||||||
background: #2563eb;
|
|
||||||
color: #fff;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
button.secondary {
|
|
||||||
background: #e8e8e8;
|
|
||||||
color: #333;
|
|
||||||
}
|
|
||||||
|
|
||||||
button:disabled {
|
|
||||||
opacity: 0.6;
|
|
||||||
cursor: not-allowed;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
@@ -1,232 +0,0 @@
|
|||||||
<script setup lang="ts">
|
|
||||||
import L from "leaflet";
|
|
||||||
import { onMounted, onUnmounted, ref, watch } from "vue";
|
|
||||||
import type { MapObject } from "../types/object";
|
|
||||||
|
|
||||||
const props = defineProps<{
|
|
||||||
objects: MapObject[];
|
|
||||||
selectedId: number | null;
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const emit = defineEmits<{
|
|
||||||
select: [object: MapObject];
|
|
||||||
contextmenu: [payload: {
|
|
||||||
x: number;
|
|
||||||
y: number;
|
|
||||||
latitude: number;
|
|
||||||
longitude: number;
|
|
||||||
object: MapObject | null;
|
|
||||||
}];
|
|
||||||
move: [payload: { object: MapObject; latitude: number; longitude: number }];
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const mapContainer = ref<HTMLElement | null>(null);
|
|
||||||
|
|
||||||
let map: L.Map | null = null;
|
|
||||||
let markersLayer: L.LayerGroup | null = null;
|
|
||||||
const markerById = new Map<number, L.Marker>();
|
|
||||||
|
|
||||||
const defaultIcon = L.icon({
|
|
||||||
iconUrl: "https://unpkg.com/leaflet@1.9.4/dist/images/marker-icon.png",
|
|
||||||
iconRetinaUrl: "https://unpkg.com/leaflet@1.9.4/dist/images/marker-icon-2x.png",
|
|
||||||
shadowUrl: "https://unpkg.com/leaflet@1.9.4/dist/images/marker-shadow.png",
|
|
||||||
iconSize: [25, 41],
|
|
||||||
iconAnchor: [12, 41],
|
|
||||||
popupAnchor: [1, -34],
|
|
||||||
shadowSize: [41, 41],
|
|
||||||
});
|
|
||||||
|
|
||||||
const selectedIcon = L.icon({
|
|
||||||
iconUrl: "https://raw.githubusercontent.com/pointhi/leaflet-color-markers/master/img/marker-icon-orange.png",
|
|
||||||
iconRetinaUrl: "https://raw.githubusercontent.com/pointhi/leaflet-color-markers/master/img/marker-icon-2x-orange.png",
|
|
||||||
shadowUrl: "https://unpkg.com/leaflet@1.9.4/dist/images/marker-shadow.png",
|
|
||||||
iconSize: [25, 41],
|
|
||||||
iconAnchor: [12, 41],
|
|
||||||
popupAnchor: [1, -34],
|
|
||||||
shadowSize: [41, 41],
|
|
||||||
});
|
|
||||||
|
|
||||||
function isDraggable(obj: MapObject): boolean {
|
|
||||||
return obj.id === props.selectedId;
|
|
||||||
}
|
|
||||||
|
|
||||||
function getMarkerIcon(obj: MapObject): L.Icon {
|
|
||||||
if (obj.id === props.selectedId) return selectedIcon;
|
|
||||||
return defaultIcon;
|
|
||||||
}
|
|
||||||
|
|
||||||
function bindMarker(marker: L.Marker, obj: MapObject) {
|
|
||||||
marker.off("click");
|
|
||||||
marker.off("contextmenu");
|
|
||||||
marker.off("dragend");
|
|
||||||
|
|
||||||
marker.on("click", () => emit("select", obj));
|
|
||||||
|
|
||||||
marker.on("contextmenu", (event: L.LeafletMouseEvent) => {
|
|
||||||
L.DomEvent.stopPropagation(event.originalEvent);
|
|
||||||
L.DomEvent.preventDefault(event.originalEvent);
|
|
||||||
|
|
||||||
emit("contextmenu", {
|
|
||||||
x: event.originalEvent.clientX,
|
|
||||||
y: event.originalEvent.clientY,
|
|
||||||
latitude: obj.latitude,
|
|
||||||
longitude: obj.longitude,
|
|
||||||
object: obj,
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
const draggable = isDraggable(obj);
|
|
||||||
if (marker.dragging) {
|
|
||||||
if (draggable) {
|
|
||||||
marker.dragging.enable();
|
|
||||||
} else {
|
|
||||||
marker.dragging.disable();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (draggable) {
|
|
||||||
marker.on("dragend", () => {
|
|
||||||
const latlng = marker.getLatLng();
|
|
||||||
emit("move", {
|
|
||||||
object: obj,
|
|
||||||
latitude: latlng.lat,
|
|
||||||
longitude: latlng.lng,
|
|
||||||
});
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function syncMarkers() {
|
|
||||||
if (!map || !markersLayer) return;
|
|
||||||
|
|
||||||
const currentIds = new Set(props.objects.map((obj) => obj.id));
|
|
||||||
|
|
||||||
for (const [id, marker] of markerById) {
|
|
||||||
if (!currentIds.has(id)) {
|
|
||||||
markersLayer.removeLayer(marker);
|
|
||||||
markerById.delete(id);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
for (const obj of props.objects) {
|
|
||||||
const draggable = isDraggable(obj);
|
|
||||||
let marker = markerById.get(obj.id);
|
|
||||||
|
|
||||||
if (!marker) {
|
|
||||||
marker = L.marker([obj.latitude, obj.longitude], {
|
|
||||||
icon: getMarkerIcon(obj),
|
|
||||||
draggable,
|
|
||||||
});
|
|
||||||
marker.addTo(markersLayer);
|
|
||||||
markerById.set(obj.id, marker);
|
|
||||||
} else {
|
|
||||||
const latlng = marker.getLatLng();
|
|
||||||
const positionChanged =
|
|
||||||
Math.abs(latlng.lat - obj.latitude) > 1e-8 ||
|
|
||||||
Math.abs(latlng.lng - obj.longitude) > 1e-8;
|
|
||||||
|
|
||||||
if (positionChanged) {
|
|
||||||
marker.setLatLng([obj.latitude, obj.longitude]);
|
|
||||||
}
|
|
||||||
|
|
||||||
marker.setIcon(getMarkerIcon(obj));
|
|
||||||
if (marker.dragging) {
|
|
||||||
if (draggable) {
|
|
||||||
marker.dragging.enable();
|
|
||||||
} else {
|
|
||||||
marker.dragging.disable();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
bindMarker(marker, obj);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function handleMapContextMenu(event: MouseEvent) {
|
|
||||||
if (!map || !mapContainer.value) return;
|
|
||||||
|
|
||||||
event.preventDefault();
|
|
||||||
|
|
||||||
const rect = mapContainer.value.getBoundingClientRect();
|
|
||||||
const point = map.containerPointToLatLng(
|
|
||||||
L.point(event.clientX - rect.left, event.clientY - rect.top),
|
|
||||||
);
|
|
||||||
|
|
||||||
emit("contextmenu", {
|
|
||||||
x: event.clientX,
|
|
||||||
y: event.clientY,
|
|
||||||
latitude: point.lat,
|
|
||||||
longitude: point.lng,
|
|
||||||
object: null,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
onMounted(() => {
|
|
||||||
if (!mapContainer.value) return;
|
|
||||||
|
|
||||||
map = L.map(mapContainer.value).setView([55.7558, 37.6173], 11);
|
|
||||||
|
|
||||||
L.tileLayer("https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png", {
|
|
||||||
attribution: '© <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a>',
|
|
||||||
maxZoom: 19,
|
|
||||||
}).addTo(map);
|
|
||||||
|
|
||||||
markersLayer = L.layerGroup().addTo(map);
|
|
||||||
mapContainer.value.addEventListener("contextmenu", handleMapContextMenu);
|
|
||||||
syncMarkers();
|
|
||||||
});
|
|
||||||
|
|
||||||
onUnmounted(() => {
|
|
||||||
mapContainer.value?.removeEventListener("contextmenu", handleMapContextMenu);
|
|
||||||
map?.remove();
|
|
||||||
map = null;
|
|
||||||
markersLayer = null;
|
|
||||||
markerById.clear();
|
|
||||||
});
|
|
||||||
|
|
||||||
watch(
|
|
||||||
() => [props.objects, props.selectedId] as const,
|
|
||||||
() => syncMarkers(),
|
|
||||||
{ deep: true },
|
|
||||||
);
|
|
||||||
</script>
|
|
||||||
|
|
||||||
<template>
|
|
||||||
<div class="map-wrapper">
|
|
||||||
<div
|
|
||||||
ref="mapContainer"
|
|
||||||
class="map"
|
|
||||||
:class="{ 'has-selection': selectedId !== null }"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
</template>
|
|
||||||
|
|
||||||
<style scoped>
|
|
||||||
.map-wrapper {
|
|
||||||
position: relative;
|
|
||||||
z-index: 0;
|
|
||||||
flex: 1;
|
|
||||||
min-width: 0;
|
|
||||||
min-height: 0;
|
|
||||||
overflow: hidden;
|
|
||||||
isolation: isolate;
|
|
||||||
}
|
|
||||||
|
|
||||||
.map {
|
|
||||||
width: 100%;
|
|
||||||
height: 100%;
|
|
||||||
}
|
|
||||||
|
|
||||||
.map-wrapper :deep(.leaflet-container) {
|
|
||||||
z-index: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.map.has-selection :deep(.leaflet-marker-draggable) {
|
|
||||||
cursor: grab;
|
|
||||||
}
|
|
||||||
|
|
||||||
.map.has-selection :deep(.leaflet-marker-draggable:active) {
|
|
||||||
cursor: grabbing;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
@@ -1,343 +0,0 @@
|
|||||||
<script setup lang="ts">
|
|
||||||
import { computed, ref, watch } from "vue";
|
|
||||||
import {
|
|
||||||
deleteObjectMedia,
|
|
||||||
fetchObjectMedia,
|
|
||||||
formatFileSize,
|
|
||||||
uploadObjectMedia,
|
|
||||||
} from "../api/objects";
|
|
||||||
import { MEDIA_ACCEPT, OBJECT_TYPE_LABELS, type MapObject, type ObjectMedia } from "../types/object";
|
|
||||||
|
|
||||||
const props = defineProps<{
|
|
||||||
object: MapObject | null;
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const mediaItems = ref<ObjectMedia[]>([]);
|
|
||||||
const loadingMedia = ref(false);
|
|
||||||
const mediaError = ref("");
|
|
||||||
const uploading = ref(false);
|
|
||||||
|
|
||||||
const typeLabel = computed(() =>
|
|
||||||
props.object ? OBJECT_TYPE_LABELS[props.object.type] : "",
|
|
||||||
);
|
|
||||||
|
|
||||||
const formattedDate = computed(() => {
|
|
||||||
if (!props.object) return "";
|
|
||||||
return new Date(props.object.created_at).toLocaleString("ru-RU");
|
|
||||||
});
|
|
||||||
|
|
||||||
function isImage(media: ObjectMedia): boolean {
|
|
||||||
return media.content_type.startsWith("image/");
|
|
||||||
}
|
|
||||||
|
|
||||||
function isVideo(media: ObjectMedia): boolean {
|
|
||||||
return media.content_type.startsWith("video/");
|
|
||||||
}
|
|
||||||
|
|
||||||
async function loadMedia() {
|
|
||||||
if (!props.object) {
|
|
||||||
mediaItems.value = [];
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
loadingMedia.value = true;
|
|
||||||
mediaError.value = "";
|
|
||||||
|
|
||||||
try {
|
|
||||||
mediaItems.value = await fetchObjectMedia(props.object.id);
|
|
||||||
} catch (err) {
|
|
||||||
mediaError.value = err instanceof Error ? err.message : "Не удалось загрузить медиа";
|
|
||||||
mediaItems.value = [];
|
|
||||||
} finally {
|
|
||||||
loadingMedia.value = false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleUpload(event: Event) {
|
|
||||||
if (!props.object) return;
|
|
||||||
|
|
||||||
const input = event.target as HTMLInputElement;
|
|
||||||
const files = input.files ? Array.from(input.files) : [];
|
|
||||||
input.value = "";
|
|
||||||
|
|
||||||
if (files.length === 0) return;
|
|
||||||
|
|
||||||
uploading.value = true;
|
|
||||||
mediaError.value = "";
|
|
||||||
|
|
||||||
try {
|
|
||||||
for (const file of files) {
|
|
||||||
const uploaded = await uploadObjectMedia(props.object.id, file);
|
|
||||||
mediaItems.value = [...mediaItems.value, uploaded];
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
mediaError.value = err instanceof Error ? err.message : "Не удалось загрузить файл";
|
|
||||||
} finally {
|
|
||||||
uploading.value = false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function handleDelete(mediaId: number) {
|
|
||||||
mediaError.value = "";
|
|
||||||
|
|
||||||
try {
|
|
||||||
await deleteObjectMedia(mediaId);
|
|
||||||
mediaItems.value = mediaItems.value.filter((item) => item.id !== mediaId);
|
|
||||||
} catch (err) {
|
|
||||||
mediaError.value = err instanceof Error ? err.message : "Не удалось удалить файл";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
watch(
|
|
||||||
() => props.object?.id,
|
|
||||||
() => loadMedia(),
|
|
||||||
{ immediate: true },
|
|
||||||
);
|
|
||||||
</script>
|
|
||||||
|
|
||||||
<template>
|
|
||||||
<aside class="panel">
|
|
||||||
<h2>Объект</h2>
|
|
||||||
|
|
||||||
<div v-if="!object" class="empty">
|
|
||||||
<p>Выберите объект на карте, чтобы увидеть описание.</p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div v-else class="content">
|
|
||||||
<h3>{{ object.name }}</h3>
|
|
||||||
|
|
||||||
<dl>
|
|
||||||
<dt>Тип</dt>
|
|
||||||
<dd>{{ typeLabel }}</dd>
|
|
||||||
|
|
||||||
<dt>Описание</dt>
|
|
||||||
<dd>{{ object.description || "—" }}</dd>
|
|
||||||
|
|
||||||
<dt>Координаты</dt>
|
|
||||||
<dd>
|
|
||||||
{{ object.latitude.toFixed(6) }}, {{ object.longitude.toFixed(6) }}
|
|
||||||
<span class="drag-hint">Перетащите маркер на карте для перемещения</span>
|
|
||||||
</dd>
|
|
||||||
|
|
||||||
<dt>Создан</dt>
|
|
||||||
<dd>{{ formattedDate }}</dd>
|
|
||||||
</dl>
|
|
||||||
|
|
||||||
<section class="media-section">
|
|
||||||
<div class="media-header">
|
|
||||||
<h4>Медиа</h4>
|
|
||||||
<label class="upload-btn">
|
|
||||||
{{ uploading ? "Загрузка..." : "Добавить" }}
|
|
||||||
<input
|
|
||||||
type="file"
|
|
||||||
multiple
|
|
||||||
:accept="MEDIA_ACCEPT"
|
|
||||||
:disabled="uploading"
|
|
||||||
hidden
|
|
||||||
@change="handleUpload"
|
|
||||||
/>
|
|
||||||
</label>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<p v-if="loadingMedia" class="media-status">Загрузка медиа...</p>
|
|
||||||
<p v-else-if="mediaError" class="media-error">{{ mediaError }}</p>
|
|
||||||
<p v-else-if="mediaItems.length === 0" class="media-status">Медиафайлы не прикреплены</p>
|
|
||||||
|
|
||||||
<ul v-else class="media-list">
|
|
||||||
<li v-for="item in mediaItems" :key="item.id" class="media-item">
|
|
||||||
<img v-if="isImage(item)" :src="item.url" :alt="item.original_name" class="preview" />
|
|
||||||
<video
|
|
||||||
v-else-if="isVideo(item)"
|
|
||||||
:src="item.url"
|
|
||||||
class="preview"
|
|
||||||
controls
|
|
||||||
preload="metadata"
|
|
||||||
/>
|
|
||||||
<a v-else :href="item.url" class="file-link" target="_blank" rel="noopener">
|
|
||||||
{{ item.original_name }}
|
|
||||||
</a>
|
|
||||||
|
|
||||||
<div class="media-meta">
|
|
||||||
<span class="name" :title="item.original_name">{{ item.original_name }}</span>
|
|
||||||
<span class="size">{{ formatFileSize(item.size) }}</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<button type="button" class="delete-btn" @click="handleDelete(item.id)">
|
|
||||||
Удалить
|
|
||||||
</button>
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</section>
|
|
||||||
</div>
|
|
||||||
</aside>
|
|
||||||
</template>
|
|
||||||
|
|
||||||
<style scoped>
|
|
||||||
.panel {
|
|
||||||
position: relative;
|
|
||||||
z-index: 2;
|
|
||||||
width: 320px;
|
|
||||||
flex-shrink: 0;
|
|
||||||
background: #fff;
|
|
||||||
border-left: 1px solid #e0e0e0;
|
|
||||||
padding: 1.25rem;
|
|
||||||
overflow-y: auto;
|
|
||||||
}
|
|
||||||
|
|
||||||
.panel h2 {
|
|
||||||
margin: 0 0 1rem;
|
|
||||||
font-size: 1rem;
|
|
||||||
font-weight: 600;
|
|
||||||
text-transform: uppercase;
|
|
||||||
letter-spacing: 0.04em;
|
|
||||||
color: #666;
|
|
||||||
}
|
|
||||||
|
|
||||||
.empty {
|
|
||||||
color: #888;
|
|
||||||
font-size: 0.9rem;
|
|
||||||
line-height: 1.5;
|
|
||||||
}
|
|
||||||
|
|
||||||
.content h3 {
|
|
||||||
margin: 0 0 1rem;
|
|
||||||
font-size: 1.25rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
dl {
|
|
||||||
margin: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
dt {
|
|
||||||
margin-top: 0.75rem;
|
|
||||||
font-size: 0.75rem;
|
|
||||||
font-weight: 600;
|
|
||||||
text-transform: uppercase;
|
|
||||||
letter-spacing: 0.04em;
|
|
||||||
color: #888;
|
|
||||||
}
|
|
||||||
|
|
||||||
dt:first-child {
|
|
||||||
margin-top: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
dd {
|
|
||||||
margin: 0.25rem 0 0;
|
|
||||||
line-height: 1.5;
|
|
||||||
color: #333;
|
|
||||||
}
|
|
||||||
|
|
||||||
.drag-hint {
|
|
||||||
display: block;
|
|
||||||
margin-top: 0.25rem;
|
|
||||||
font-size: 0.75rem;
|
|
||||||
color: #2563eb;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-section {
|
|
||||||
margin-top: 1.5rem;
|
|
||||||
padding-top: 1rem;
|
|
||||||
border-top: 1px solid #eee;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-header {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: space-between;
|
|
||||||
margin-bottom: 0.75rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-header h4 {
|
|
||||||
margin: 0;
|
|
||||||
font-size: 0.75rem;
|
|
||||||
font-weight: 600;
|
|
||||||
text-transform: uppercase;
|
|
||||||
letter-spacing: 0.04em;
|
|
||||||
color: #888;
|
|
||||||
}
|
|
||||||
|
|
||||||
.upload-btn {
|
|
||||||
font-size: 0.8125rem;
|
|
||||||
font-weight: 500;
|
|
||||||
color: #2563eb;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
.upload-btn input:disabled + span,
|
|
||||||
.upload-btn:has(input:disabled) {
|
|
||||||
opacity: 0.6;
|
|
||||||
cursor: not-allowed;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-status {
|
|
||||||
margin: 0;
|
|
||||||
font-size: 0.875rem;
|
|
||||||
color: #888;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-error {
|
|
||||||
margin: 0;
|
|
||||||
font-size: 0.875rem;
|
|
||||||
color: #c62828;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-list {
|
|
||||||
list-style: none;
|
|
||||||
margin: 0;
|
|
||||||
padding: 0;
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 0.75rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-item {
|
|
||||||
border: 1px solid #e8e8e8;
|
|
||||||
border-radius: 6px;
|
|
||||||
overflow: hidden;
|
|
||||||
background: #fafafa;
|
|
||||||
}
|
|
||||||
|
|
||||||
.preview {
|
|
||||||
display: block;
|
|
||||||
width: 100%;
|
|
||||||
max-height: 160px;
|
|
||||||
object-fit: cover;
|
|
||||||
background: #000;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-link {
|
|
||||||
display: block;
|
|
||||||
padding: 0.75rem;
|
|
||||||
color: #2563eb;
|
|
||||||
text-decoration: none;
|
|
||||||
word-break: break-all;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-meta {
|
|
||||||
display: flex;
|
|
||||||
justify-content: space-between;
|
|
||||||
gap: 0.5rem;
|
|
||||||
padding: 0.5rem 0.75rem 0;
|
|
||||||
font-size: 0.75rem;
|
|
||||||
color: #666;
|
|
||||||
}
|
|
||||||
|
|
||||||
.name {
|
|
||||||
overflow: hidden;
|
|
||||||
text-overflow: ellipsis;
|
|
||||||
white-space: nowrap;
|
|
||||||
}
|
|
||||||
|
|
||||||
.delete-btn {
|
|
||||||
display: block;
|
|
||||||
width: calc(100% - 1rem);
|
|
||||||
margin: 0.5rem auto 0.5rem;
|
|
||||||
padding: 0.375rem 0.5rem;
|
|
||||||
border: none;
|
|
||||||
border-radius: 4px;
|
|
||||||
background: #fee2e2;
|
|
||||||
color: #b91c1c;
|
|
||||||
font-size: 0.75rem;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
@@ -1,199 +0,0 @@
|
|||||||
<script setup lang="ts">
|
|
||||||
import { computed } from "vue";
|
|
||||||
import type { MapObject } from "../types/object";
|
|
||||||
|
|
||||||
const props = defineProps<{
|
|
||||||
modelValue: number;
|
|
||||||
min: number;
|
|
||||||
max: number;
|
|
||||||
objects: MapObject[];
|
|
||||||
visibleCount: number;
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const emit = defineEmits<{
|
|
||||||
"update:modelValue": [value: number];
|
|
||||||
}>();
|
|
||||||
|
|
||||||
const range = computed({
|
|
||||||
get: () => props.modelValue,
|
|
||||||
set: (value) => emit("update:modelValue", value),
|
|
||||||
});
|
|
||||||
|
|
||||||
const formattedTime = computed(() =>
|
|
||||||
new Date(props.modelValue).toLocaleString("ru-RU", {
|
|
||||||
day: "2-digit",
|
|
||||||
month: "short",
|
|
||||||
year: "numeric",
|
|
||||||
hour: "2-digit",
|
|
||||||
minute: "2-digit",
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
|
|
||||||
const formattedMin = computed(() =>
|
|
||||||
new Date(props.min).toLocaleDateString("ru-RU", {
|
|
||||||
day: "2-digit",
|
|
||||||
month: "short",
|
|
||||||
year: "numeric",
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
|
|
||||||
const formattedMax = computed(() =>
|
|
||||||
new Date(props.max).toLocaleDateString("ru-RU", {
|
|
||||||
day: "2-digit",
|
|
||||||
month: "short",
|
|
||||||
year: "numeric",
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
|
|
||||||
const markers = computed(() => {
|
|
||||||
if (props.max <= props.min) return [];
|
|
||||||
|
|
||||||
const span = props.max - props.min;
|
|
||||||
return props.objects.map((obj) => {
|
|
||||||
const time = new Date(obj.created_at).getTime();
|
|
||||||
const percent = ((time - props.min) / span) * 100;
|
|
||||||
return {
|
|
||||||
id: obj.id,
|
|
||||||
name: obj.name,
|
|
||||||
percent: Math.min(100, Math.max(0, percent)),
|
|
||||||
visible: time <= props.modelValue,
|
|
||||||
};
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
const hasRange = computed(() => props.max > props.min);
|
|
||||||
</script>
|
|
||||||
|
|
||||||
<template>
|
|
||||||
<footer class="timeline">
|
|
||||||
<div class="timeline-header">
|
|
||||||
<span class="label">Таймлайн появления</span>
|
|
||||||
<span class="current-time">{{ formattedTime }}</span>
|
|
||||||
<span class="count">{{ visibleCount }} / {{ objects.length }} объектов</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="slider-wrap">
|
|
||||||
<span class="edge-label">{{ formattedMin }}</span>
|
|
||||||
|
|
||||||
<div class="slider-track">
|
|
||||||
<div
|
|
||||||
v-for="marker in markers"
|
|
||||||
:key="marker.id"
|
|
||||||
class="object-marker"
|
|
||||||
:class="{ visible: marker.visible }"
|
|
||||||
:style="{ left: `${marker.percent}%` }"
|
|
||||||
:title="marker.name"
|
|
||||||
/>
|
|
||||||
|
|
||||||
<input
|
|
||||||
v-model.number="range"
|
|
||||||
class="slider"
|
|
||||||
type="range"
|
|
||||||
:min="min"
|
|
||||||
:max="max"
|
|
||||||
:step="hasRange ? Math.max(1, Math.floor((max - min) / 500)) : 1"
|
|
||||||
:disabled="!hasRange"
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<span class="edge-label">{{ formattedMax }}</span>
|
|
||||||
</div>
|
|
||||||
</footer>
|
|
||||||
</template>
|
|
||||||
|
|
||||||
<style scoped>
|
|
||||||
.timeline {
|
|
||||||
position: relative;
|
|
||||||
z-index: 3;
|
|
||||||
flex-shrink: 0;
|
|
||||||
padding: 0.75rem 1.25rem 1rem;
|
|
||||||
background: #fff;
|
|
||||||
border-top: 1px solid #e0e0e0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.timeline-header {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 1rem;
|
|
||||||
margin-bottom: 0.625rem;
|
|
||||||
font-size: 0.8125rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.label {
|
|
||||||
font-weight: 600;
|
|
||||||
color: #333;
|
|
||||||
}
|
|
||||||
|
|
||||||
.current-time {
|
|
||||||
color: #2563eb;
|
|
||||||
font-weight: 500;
|
|
||||||
}
|
|
||||||
|
|
||||||
.count {
|
|
||||||
margin-left: auto;
|
|
||||||
color: #666;
|
|
||||||
}
|
|
||||||
|
|
||||||
.slider-wrap {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 0.75rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.edge-label {
|
|
||||||
flex-shrink: 0;
|
|
||||||
width: 5.5rem;
|
|
||||||
font-size: 0.75rem;
|
|
||||||
color: #888;
|
|
||||||
text-align: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
.edge-label:first-child {
|
|
||||||
text-align: left;
|
|
||||||
}
|
|
||||||
|
|
||||||
.edge-label:last-child {
|
|
||||||
text-align: right;
|
|
||||||
}
|
|
||||||
|
|
||||||
.slider-track {
|
|
||||||
position: relative;
|
|
||||||
flex: 1;
|
|
||||||
height: 2rem;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
.object-marker {
|
|
||||||
position: absolute;
|
|
||||||
top: 50%;
|
|
||||||
width: 8px;
|
|
||||||
height: 8px;
|
|
||||||
margin-left: -4px;
|
|
||||||
margin-top: -4px;
|
|
||||||
border-radius: 50%;
|
|
||||||
background: #bbb;
|
|
||||||
pointer-events: none;
|
|
||||||
z-index: 1;
|
|
||||||
transition: background 0.15s, transform 0.15s;
|
|
||||||
}
|
|
||||||
|
|
||||||
.object-marker.visible {
|
|
||||||
background: #2563eb;
|
|
||||||
transform: scale(1.25);
|
|
||||||
}
|
|
||||||
|
|
||||||
.slider {
|
|
||||||
position: relative;
|
|
||||||
z-index: 2;
|
|
||||||
width: 100%;
|
|
||||||
margin: 0;
|
|
||||||
cursor: pointer;
|
|
||||||
accent-color: #2563eb;
|
|
||||||
}
|
|
||||||
|
|
||||||
.slider:disabled {
|
|
||||||
cursor: default;
|
|
||||||
opacity: 0.5;
|
|
||||||
}
|
|
||||||
</style>
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
import { createApp } from "vue";
|
|
||||||
import App from "./App.vue";
|
|
||||||
import "./style.css";
|
|
||||||
|
|
||||||
createApp(App).mount("#app");
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
*,
|
|
||||||
*::before,
|
|
||||||
*::after {
|
|
||||||
box-sizing: border-box;
|
|
||||||
}
|
|
||||||
|
|
||||||
html,
|
|
||||||
body,
|
|
||||||
#app {
|
|
||||||
margin: 0;
|
|
||||||
height: 100%;
|
|
||||||
font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
|
|
||||||
color: #1a1a1a;
|
|
||||||
background: #f5f5f5;
|
|
||||||
}
|
|
||||||
|
|
||||||
button,
|
|
||||||
input,
|
|
||||||
select,
|
|
||||||
textarea {
|
|
||||||
font: inherit;
|
|
||||||
}
|
|
||||||
@@ -1,50 +0,0 @@
|
|||||||
export type ObjectType = "point" | "marker" | "zone" | "other";
|
|
||||||
|
|
||||||
export interface MapObject {
|
|
||||||
id: number;
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
type: ObjectType;
|
|
||||||
latitude: number;
|
|
||||||
longitude: number;
|
|
||||||
created_at: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface MapObjectCreate {
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
type: ObjectType;
|
|
||||||
latitude: number;
|
|
||||||
longitude: number;
|
|
||||||
created_at?: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface MapObjectUpdate {
|
|
||||||
name?: string;
|
|
||||||
description?: string;
|
|
||||||
type?: ObjectType;
|
|
||||||
latitude?: number;
|
|
||||||
longitude?: number;
|
|
||||||
created_at?: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface ObjectMedia {
|
|
||||||
id: number;
|
|
||||||
object_id: number;
|
|
||||||
original_name: string;
|
|
||||||
content_type: string;
|
|
||||||
size: number;
|
|
||||||
created_at: string;
|
|
||||||
url: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export const MEDIA_ACCEPT = "image/jpeg,image/png,image/gif,image/webp,video/mp4,video/webm";
|
|
||||||
|
|
||||||
export const OBJECT_TYPE_LABELS: Record<ObjectType, string> = {
|
|
||||||
point: "Точка",
|
|
||||||
marker: "Метка",
|
|
||||||
zone: "Зона",
|
|
||||||
other: "Другое",
|
|
||||||
};
|
|
||||||
|
|
||||||
export const OBJECT_TYPES: ObjectType[] = ["point", "marker", "zone", "other"];
|
|
||||||
Vendored
-7
@@ -1,7 +0,0 @@
|
|||||||
/// <reference types="vite/client" />
|
|
||||||
|
|
||||||
declare module "*.vue" {
|
|
||||||
import type { DefineComponent } from "vue";
|
|
||||||
const component: DefineComponent<Record<string, unknown>, Record<string, unknown>, unknown>;
|
|
||||||
export default component;
|
|
||||||
}
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "./tsconfig.json",
|
|
||||||
"compilerOptions": {
|
|
||||||
"composite": true,
|
|
||||||
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo"
|
|
||||||
},
|
|
||||||
"include": ["src/**/*.ts", "src/**/*.vue"]
|
|
||||||
}
|
|
||||||
@@ -1,21 +0,0 @@
|
|||||||
{
|
|
||||||
"compilerOptions": {
|
|
||||||
"target": "ES2020",
|
|
||||||
"useDefineForClassFields": true,
|
|
||||||
"module": "ESNext",
|
|
||||||
"lib": ["ES2020", "DOM", "DOM.Iterable"],
|
|
||||||
"skipLibCheck": true,
|
|
||||||
"moduleResolution": "bundler",
|
|
||||||
"allowImportingTsExtensions": true,
|
|
||||||
"isolatedModules": true,
|
|
||||||
"moduleDetection": "force",
|
|
||||||
"noEmit": true,
|
|
||||||
"jsx": "preserve",
|
|
||||||
"strict": true,
|
|
||||||
"noUnusedLocals": true,
|
|
||||||
"noUnusedParameters": true,
|
|
||||||
"noFallthroughCasesInSwitch": true,
|
|
||||||
"noUncheckedSideEffectImports": true
|
|
||||||
},
|
|
||||||
"include": ["src/**/*.ts", "src/**/*.tsx", "src/**/*.vue"]
|
|
||||||
}
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
{
|
|
||||||
"compilerOptions": {
|
|
||||||
"composite": true,
|
|
||||||
"skipLibCheck": true,
|
|
||||||
"module": "ESNext",
|
|
||||||
"moduleResolution": "bundler",
|
|
||||||
"allowSyntheticDefaultImports": true
|
|
||||||
},
|
|
||||||
"include": ["vite.config.ts"]
|
|
||||||
}
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
import { defineConfig } from "vite";
|
|
||||||
import vue from "@vitejs/plugin-vue";
|
|
||||||
|
|
||||||
export default defineConfig({
|
|
||||||
plugins: [vue()],
|
|
||||||
server: {
|
|
||||||
proxy: {
|
|
||||||
"/api": "http://localhost:8000",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
});
|
|
||||||
Reference in New Issue
Block a user