gitrusprusandCursor
6dff3c1c3d
Add developer docs for CA/CP/PI flows and wire cp-workers to the host SOCKS proxy so local Telegram auth and parsing work reliably. Co-authored-by: Cursor <cursoragent@cursor.com>
101 lines
3.6 KiB
Markdown
101 lines
3.6 KiB
Markdown
# MapMil — guide for AI agents
|
|
|
|
Quick orientation for Cursor and other coding agents working in this repository.
|
|
|
|
## What this is
|
|
|
|
**MapMil** is a monorepo platform: parsing center (ЦП) collects events from external sources, analytics center (ЦА) stores them and serves a map + admin UI, external consumers (ПИ) read via distribution API.
|
|
|
|
```
|
|
ЦА (centers/analytics) ←→ Redis ←→ ЦП (centers/parsing)
|
|
↓ ingest ↑ adapters
|
|
PostgreSQL + map UI Telegram / Crawl4AI / VIINA
|
|
↓
|
|
GET /api/v1/events → ПИ
|
|
```
|
|
|
|
## Repository map
|
|
|
|
| Path | Stack | Purpose |
|
|
|------|-------|---------|
|
|
| `contracts/` | Pydantic | `ingest`, `jobs`, `sources`, `queues` — shared truth |
|
|
| `centers/analytics/api/` | FastAPI, SQLAlchemy | Admin API, ingest, scheduler, map queries |
|
|
| `centers/analytics/frontend/` | Vue 3, Leaflet | Map, parsers, events, analytics, consumers |
|
|
| `centers/parsing/workers/` | Python async | CP workers, adapter registry, LLM extract |
|
|
| `docker-compose.yml` | Docker | `ca-*`, `cp-workers*`, `redis` |
|
|
| `data/` | Telethon session | **Gitignored** — do not commit |
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
cp .env.example .env # then set TELEGRAM_API_ID/HASH, optional DEEPSEEK_API_KEY
|
|
docker compose up --build # full stack → http://localhost:8080
|
|
docker compose config # validate compose
|
|
docker compose up --build cp-workers-web # single worker rebuild
|
|
```
|
|
|
|
Health: `curl http://localhost:8080/api/health`
|
|
|
|
Create Telegram parser:
|
|
|
|
```bash
|
|
curl -X POST http://localhost:8080/admin/jobs \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"source_type":"telegram","source_config":{"channel":"example","limit":50}}'
|
|
```
|
|
|
|
## Conventions
|
|
|
|
### Contracts-first
|
|
|
|
Any change to event shape, `source_config`, or queue routing starts in `contracts/`, then CA/CP/UI.
|
|
|
|
### CP adapters
|
|
|
|
- Protocol: `workers/adapters/base.py` (`SourceAdapter`)
|
|
- Registry: `workers/adapters/registry.py` (`ENABLED_ADAPTERS`)
|
|
- Queue family: `contracts/queues.py` (`SOURCE_FAMILY`)
|
|
- Config schema: `contracts/sources.py` (`CONFIG_MODELS`)
|
|
|
|
### Ingest output
|
|
|
|
Adapters return dicts matching `IngestEventItem` (`contracts/ingest.py`). Required: `source_url`. Dedup in CA by `source_url`.
|
|
|
|
### LLM extract (runtime, not dev)
|
|
|
|
`extract_mode: llm` uses DeepSeek via `workers/llm_extract.py`. Key: `DEEPSEEK_API_KEY` in `.env`. Do not confuse with Cursor dev agents.
|
|
|
|
## Common tasks
|
|
|
|
| Task | Start here | Skill |
|
|
|------|------------|-------|
|
|
| New parser source | `contracts/sources.py` | `add-parser-adapter` |
|
|
| Admin API change | `centers/analytics/api/app/routers/` | — |
|
|
| Map / UI | `centers/analytics/frontend/src/` | — |
|
|
| New worker image | `docker-compose.yml` + `requirements-*.txt` | `add-parser-adapter` |
|
|
|
|
## Do not
|
|
|
|
- Commit `.env`, `data/`, or API keys
|
|
- Add direct DB access from CP workers
|
|
- Change `IngestEventItem` without checking ingest + map + ПИ impact
|
|
- Modify Telegram listener/session logic unless the task requires it
|
|
- Reintroduce legacy root `backend/` or `frontend/` (removed; use `centers/`)
|
|
|
|
## Cursor setup
|
|
|
|
- Rule: `.cursor/rules/platform.mdc` (always on)
|
|
- Skill: `.cursor/skills/add-parser-adapter/` for end-to-end new sources
|
|
- Deep docs: `docs/` (overview + data-flow), `centers/analytics/ARCHITECTURE.md`, `centers/parsing/ARCHITECTURE.md`
|
|
|
|
## Commit style
|
|
|
|
Follow existing history — imperative, concise:
|
|
|
|
```
|
|
Add CP source adapter registry with multi-worker queues and LLM extract.
|
|
Add Telegram listener, scheduler, and extended parsers admin UI.
|
|
```
|
|
|
|
One logical change per commit; do not bundle unrelated zones.
|