Add developer docs for CA/CP/PI flows and wire cp-workers to the host SOCKS proxy so local Telegram auth and parsing work reliably. Co-authored-by: Cursor <cursoragent@cursor.com>
3.6 KiB
MapMil — guide for AI agents
Quick orientation for Cursor and other coding agents working in this repository.
What this is
MapMil is a monorepo platform: parsing center (ЦП) collects events from external sources, analytics center (ЦА) stores them and serves a map + admin UI, external consumers (ПИ) read via distribution API.
ЦА (centers/analytics) ←→ Redis ←→ ЦП (centers/parsing)
↓ ingest ↑ adapters
PostgreSQL + map UI Telegram / Crawl4AI / VIINA
↓
GET /api/v1/events → ПИ
Repository map
| Path | Stack | Purpose |
|---|---|---|
contracts/ |
Pydantic | ingest, jobs, sources, queues — shared truth |
centers/analytics/api/ |
FastAPI, SQLAlchemy | Admin API, ingest, scheduler, map queries |
centers/analytics/frontend/ |
Vue 3, Leaflet | Map, parsers, events, analytics, consumers |
centers/parsing/workers/ |
Python async | CP workers, adapter registry, LLM extract |
docker-compose.yml |
Docker | ca-*, cp-workers*, redis |
data/ |
Telethon session | Gitignored — do not commit |
Commands
cp .env.example .env # then set TELEGRAM_API_ID/HASH, optional DEEPSEEK_API_KEY
docker compose up --build # full stack → http://localhost:8080
docker compose config # validate compose
docker compose up --build cp-workers-web # single worker rebuild
Health: curl http://localhost:8080/api/health
Create Telegram parser:
curl -X POST http://localhost:8080/admin/jobs \
-H 'Content-Type: application/json' \
-d '{"source_type":"telegram","source_config":{"channel":"example","limit":50}}'
Conventions
Contracts-first
Any change to event shape, source_config, or queue routing starts in contracts/, then CA/CP/UI.
CP adapters
- Protocol:
workers/adapters/base.py(SourceAdapter) - Registry:
workers/adapters/registry.py(ENABLED_ADAPTERS) - Queue family:
contracts/queues.py(SOURCE_FAMILY) - Config schema:
contracts/sources.py(CONFIG_MODELS)
Ingest output
Adapters return dicts matching IngestEventItem (contracts/ingest.py). Required: source_url. Dedup in CA by source_url.
LLM extract (runtime, not dev)
extract_mode: llm uses DeepSeek via workers/llm_extract.py. Key: DEEPSEEK_API_KEY in .env. Do not confuse with Cursor dev agents.
Common tasks
| Task | Start here | Skill |
|---|---|---|
| New parser source | contracts/sources.py |
add-parser-adapter |
| Admin API change | centers/analytics/api/app/routers/ |
— |
| Map / UI | centers/analytics/frontend/src/ |
— |
| New worker image | docker-compose.yml + requirements-*.txt |
add-parser-adapter |
Do not
- Commit
.env,data/, or API keys - Add direct DB access from CP workers
- Change
IngestEventItemwithout checking ingest + map + ПИ impact - Modify Telegram listener/session logic unless the task requires it
- Reintroduce legacy root
backend/orfrontend/(removed; usecenters/)
Cursor setup
- Rule:
.cursor/rules/platform.mdc(always on) - Skill:
.cursor/skills/add-parser-adapter/for end-to-end new sources - Deep docs:
docs/(overview + data-flow),centers/analytics/ARCHITECTURE.md,centers/parsing/ARCHITECTURE.md
Commit style
Follow existing history — imperative, concise:
Add CP source adapter registry with multi-worker queues and LLM extract.
Add Telegram listener, scheduler, and extended parsers admin UI.
One logical change per commit; do not bundle unrelated zones.