Add Telegram listener, scheduler, and extended parsers admin UI.

Introduce background scheduling and migrations for analytics ingest, expand parser management and map toolbar UX, and ignore local data/session files from version control.

Co-authored-by: Cursor <cursoragent@cursor.com>

.gitignore

@@ -1,6 +1,5 @@
 .env
-data/telegram.session
-data/*.session
+data/
 __pycache__/
 *.pyc
 node_modules/

README.md

@@ -18,7 +18,7 @@ flowchart LR
 | Центр | Контейнеры | Назначение |
 |-------|------------|------------|
 | **ЦА** | `ca-db`, `ca-api`, `ca-frontend` | PostgreSQL, ingest API, карта, distribution API |
-| **ЦП** | `cp-workers` | Парсинг Telegram (`centers/parsing/workers`) |
+| **ЦП** | `cp-workers` | Парсинг Telegram: real-time listener (Telethon) + batch-задания из Redis |
 | **Общее** | `redis` | Очередь заданий ЦА → ЦП |
 
 ## Структура monorepo
@@ -67,7 +67,7 @@ docker compose up --build
 | Раздел | Путь | Описание |
 |--------|------|----------|
 | **Карта** | `/` | Интерактивная карта событий: навигация по датам (flatpickr), пресеты периода, фильтры региона/темы/источника, подложки Яндекс/OSM/Topo/ESRI, линейка, полноэкранный режим, центрирование по координатам и городам, поиск населённых пунктов (Nominatim). CRUD для ручных объектов (ПКМ). Поддерживает `?eventId=` |
-| **Парсеры** | `/parsers` | Создание заданий Telegram-парсинга, таблица статусов с автообновлением (5 с), повтор failed-заданий |
+| **Парсеры** | `/parsers` | Telegram-парсеры с периодическим запуском, настройка интервала, редактирование и удаление; дубликаты по `source_url` не записываются |
 | **События** | `/events` | Фильтрация, пагинация, просмотр деталей, ссылка «На карте» для событий с координатами |
 | **Аналитика** | `/analytics` | KPI-карточки, график динамики ingest за 30 дней, топ населённых пунктов и регионов |
 | **ПИ** | `/consumers` | CRUD подписчиков distribution API, ротация ключей, тест среза через `/api/v1/events` |
@@ -108,8 +108,10 @@ docker compose up --build
 
 | Метод | Путь | Описание |
 |-------|------|----------|
-| POST | `/admin/jobs` | Создать задание парсинга (ставится в Redis) |
-| GET | `/admin/jobs` | Список заданий |
+| POST | `/admin/jobs` | Создать парсер (сразу в очередь + периодический запуск) |
+| GET | `/admin/jobs` | Список парсеров |
+| PATCH | `/admin/jobs/{id}` | Изменить канал/лимит, `interval_seconds`, `is_active` |
+| DELETE | `/admin/jobs/{id}` | Удалить парсер |
 | POST | `/admin/jobs/{id}/retry` | Повторить failed-задание |
 | GET | `/admin/events` | События с фильтрами (`{ items, total }`) |
 | GET | `/admin/analytics/summary` | KPI-сводка |
@@ -148,14 +150,35 @@ curl http://localhost:8080/api/v1/events \
   -H 'Authorization: Bearer test-pi-api-key-change-me'
 ```
 
+## Парсинг Telegram
+
+`cp-workers` объединяет два режима в **одном процессе** (общая сессия `telegram.session`):
+
+| Режим | Как работает |
+|-------|----------------|
+| **Listener (real-time)** | Telethon `NewMessage` / `Album` на активных парсерах (`is_active=true`); новый пост сразу уходит в ingest |
+| **Batch (по расписанию)** | Планировщик в `ca-api` ставит задание в Redis; воркер забирает последние N постов (`iter_messages`) |
+
+Дубликаты по `source_url` при ingest пропускаются. Listener не меняет `status` парсера (флаг `listener: true` в ingest).
+
+Переменные `cp-workers`:
+
+| Переменная | По умолчанию | Описание |
+|------------|--------------|----------|
+| `TELEGRAM_LISTENER_ENABLED` | `true` | Включить real-time listener |
+| `TELEGRAM_LISTENER_REFRESH_SECONDS` | `60` | Как часто обновлять список каналов из БД |
+
+Internal API: `GET /internal/listener/subscriptions` — список активных каналов для listener.
+
 ## Поток данных
 
-1. Аналитик создаёт задание: `POST /admin/jobs`
-2. `ca-api` ставит задание в Redis (`cp:jobs`)
-3. `cp-workers` забирает задание, парсит Telegram через существующую сессию
-4. Результаты отправляются в `POST /internal/ingest`
-5. События с координатами автоматически появляются на карте как `MapObject`
-6. Внешние ПИ получают отфильтрованный срез через `/api/v1/events`
+1. Аналитик создаёт парсер: `POST /admin/jobs` (канал, лимит, интервал в секундах)
+2. **Listener** сразу подписывается на канал и ingest-ит новые посты
+3. **Планировщик** `ca-api` периодически ставит batch-задание в Redis (`cp:jobs`)
+4. `cp-workers` забирает batch-задание, парсит последние N постов
+5. Результаты → `POST /internal/ingest` (дубликаты по `source_url` пропускаются)
+6. Новые события с координатами появляются на карте как `MapObject`
+7. Внешние ПИ получают срез через `/api/v1/events`
 
 ## Миграция EventRecord → Event
 

centers/analytics/api/app/main.py

@@ -6,6 +6,8 @@ from fastapi.middleware.cors import CORSMiddleware
 from .database import Base, engine, get_db
 from .routers import admin, internal, map, objects, v1
 from .seed import seed_objects, seed_test_consumer
+from .services.migrations import migrate_schema
+from .services.scheduler import start_scheduler
 from .storage import ensure_upload_dir
 
 
@@ -13,6 +15,8 @@ from .storage import ensure_upload_dir
 async def lifespan(_: FastAPI):
     ensure_upload_dir()
     Base.metadata.create_all(bind=engine)
+    migrate_schema(engine)
+    scheduler_stop = start_scheduler()
     db = next(get_db())
     try:
         seed_objects(db)
@@ -20,6 +24,7 @@ async def lifespan(_: FastAPI):
     finally:
         db.close()
     yield
+    scheduler_stop.set()
 
 
 app = FastAPI(title="CA API (Analytics Center)", lifespan=lifespan)

centers/analytics/api/app/models.py

@@ -100,6 +100,8 @@ class ParseJob(Base):
     source_type: Mapped[str] = mapped_column(String(50), nullable=False)
     source_config: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
     schedule: Mapped[str | None] = mapped_column(String(100), nullable=True)
+    interval_seconds: Mapped[int] = mapped_column(Integer, default=3600, nullable=False)
+    is_active: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
     status: Mapped[str] = mapped_column(String(50), default="pending", index=True)
     last_run_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
     last_error: Mapped[str | None] = mapped_column(Text, nullable=True)

centers/analytics/api/app/routers/admin.py

@@ -14,6 +14,7 @@ from ..schemas import (
     EventRead,
     ParseJobCreate,
     ParseJobRead,
+    ParseJobUpdate,
     TimelinePoint,
     TopItem,
 )
@@ -42,6 +43,8 @@ def create_parse_job(payload: ParseJobCreate, db: Session = Depends(get_db)):
         source_type=payload.source_type,
         source_config=payload.source_config,
         schedule=payload.schedule,
+        interval_seconds=payload.interval_seconds,
+        is_active=payload.is_active,
         status="queued",
     )
     db.add(job)
@@ -70,6 +73,8 @@ def retry_parse_job(job_id: int, db: Session = Depends(get_db)):
     job = db.query(ParseJob).filter(ParseJob.id == job_id).first()
     if not job:
         raise HTTPException(status_code=404, detail="Job not found")
+    if job.status in ("queued", "running"):
+        raise HTTPException(status_code=409, detail="Job is already running or queued")
 
     job.status = "queued"
     job.last_error = None
@@ -80,6 +85,40 @@ def retry_parse_job(job_id: int, db: Session = Depends(get_db)):
     return job
 
 
+@router.patch("/jobs/{job_id}", response_model=ParseJobRead)
+def update_parse_job(
+    job_id: int,
+    payload: ParseJobUpdate,
+    db: Session = Depends(get_db),
+):
+    job = db.query(ParseJob).filter(ParseJob.id == job_id).first()
+    if not job:
+        raise HTTPException(status_code=404, detail="Job not found")
+
+    if payload.source_config is not None:
+        job.source_config = payload.source_config
+    if payload.interval_seconds is not None:
+        job.interval_seconds = payload.interval_seconds
+    if payload.is_active is not None:
+        job.is_active = payload.is_active
+
+    db.commit()
+    db.refresh(job)
+    return job
+
+
+@router.delete("/jobs/{job_id}", status_code=204)
+def delete_parse_job(job_id: int, db: Session = Depends(get_db)):
+    job = db.query(ParseJob).filter(ParseJob.id == job_id).first()
+    if not job:
+        raise HTTPException(status_code=404, detail="Job not found")
+    if job.status == "running":
+        raise HTTPException(status_code=409, detail="Cannot delete a running job")
+
+    db.delete(job)
+    db.commit()
+
+
 @router.get("/events", response_model=EventListResponse)
 def list_events(
     limit: int = Query(default=50, ge=1, le=1000),

centers/analytics/api/app/routers/internal.py

@@ -6,7 +6,7 @@ from sqlalchemy.orm import Session
 from ..database import get_db
 from ..deps import verify_internal_token
 from ..models import ParseJob
-from ..schemas import IngestRequest, IngestResponse
+from ..schemas import IngestRequest, IngestResponse, ListenerSubscription
 from ..services.ingest import ingest_events
 
 router = APIRouter(prefix="/internal", tags=["internal"])
@@ -18,10 +18,16 @@ def internal_ingest(
     _: None = Depends(verify_internal_token),
     db: Session = Depends(get_db),
 ):
-    ingested, updated, map_synced = ingest_events(db, payload.events, payload.job_id)
+    ingested, updated, skipped, map_synced = ingest_events(
+        db,
+        payload.events,
+        payload.job_id,
+        touch_job=not payload.listener,
+    )
     return IngestResponse(
         ingested=ingested,
         updated=updated,
+        skipped=skipped,
         map_objects_synced=map_synced,
     )
 
@@ -39,7 +45,31 @@ def update_job_status(
         raise HTTPException(status_code=404, detail="Job not found")
 
     job.status = status
-    job.last_run_at = datetime.now(timezone.utc)
+    if status in ("completed", "failed"):
+        job.last_run_at = datetime.now(timezone.utc)
     job.last_error = error
     db.commit()
     return {"id": job.id, "status": job.status}
+
+
+@router.get("/listener/subscriptions", response_model=list[ListenerSubscription])
+def listener_subscriptions(
+    _: None = Depends(verify_internal_token),
+    db: Session = Depends(get_db),
+):
+    jobs = (
+        db.query(ParseJob)
+        .filter(
+            ParseJob.is_active.is_(True),
+            ParseJob.source_type == "telegram",
+        )
+        .order_by(ParseJob.id.asc())
+        .all()
+    )
+    result: list[ListenerSubscription] = []
+    for job in jobs:
+        channel = job.source_config.get("channel") if job.source_config else None
+        if not channel:
+            continue
+        result.append(ListenerSubscription(job_id=job.id, channel=str(channel)))
+    return result

centers/analytics/api/app/schemas.py

@@ -111,11 +111,18 @@ class IngestEventItem(BaseModel):
 class IngestRequest(BaseModel):
     job_id: int | None = None
     events: list[IngestEventItem] = Field(default_factory=list)
+    listener: bool = False
+
+
+class ListenerSubscription(BaseModel):
+    job_id: int
+    channel: str
 
 
 class IngestResponse(BaseModel):
     ingested: int
     updated: int
+    skipped: int = 0
     map_objects_synced: int
 
 
@@ -123,6 +130,14 @@ class ParseJobCreate(BaseModel):
     source_type: str = "telegram"
     source_config: dict[str, Any] = Field(default_factory=dict)
     schedule: str | None = None
+    interval_seconds: int = Field(default=3600, ge=60, le=604800)
+    is_active: bool = True
+
+
+class ParseJobUpdate(BaseModel):
+    source_config: dict[str, Any] | None = None
+    interval_seconds: int | None = Field(default=None, ge=60, le=604800)
+    is_active: bool | None = None
 
 
 class ParseJobRead(BaseModel):
@@ -132,6 +147,8 @@ class ParseJobRead(BaseModel):
     source_type: str
     source_config: dict[str, Any]
     schedule: str | None
+    interval_seconds: int
+    is_active: bool
     status: str
     last_run_at: datetime | None
     last_error: str | None

centers/analytics/api/app/services/ingest.py

@@ -9,9 +9,12 @@ def ingest_events(
     db: Session,
     items: list[IngestEventItem],
     job_id: int | None = None,
-) -> tuple[int, int, int]:
+    *,
+    touch_job: bool = True,
+) -> tuple[int, int, int, int]:
     ingested = 0
     updated = 0
+    skipped = 0
     map_synced = 0
 
     for item in items:
@@ -22,44 +25,32 @@ def ingest_events(
         )
 
         if existing:
-            existing.source_type = item.source_type
-            existing.raw_text = item.raw_text
-            existing.title = item.title
-            existing.description = item.description
-            existing.locality = item.locality
-            existing.latitude = item.latitude
-            existing.longitude = item.longitude
-            existing.event_date = item.event_date
-            existing.region = item.region
-            existing.topic = item.topic
-            existing.tags = item.tags
-            existing.metadata_ = item.metadata
-            event = existing
-            updated += 1
-        else:
-            event = Event(
-                source_type=item.source_type,
-                source_url=item.source_url,
-                raw_text=item.raw_text,
-                title=item.title,
-                description=item.description,
-                locality=item.locality,
-                latitude=item.latitude,
-                longitude=item.longitude,
-                event_date=item.event_date,
-                region=item.region,
-                topic=item.topic,
-                tags=item.tags,
-                metadata_=item.metadata,
-            )
-            db.add(event)
-            ingested += 1
+            skipped += 1
+            continue
+
+        event = Event(
+            source_type=item.source_type,
+            source_url=item.source_url,
+            raw_text=item.raw_text,
+            title=item.title,
+            description=item.description,
+            locality=item.locality,
+            latitude=item.latitude,
+            longitude=item.longitude,
+            event_date=item.event_date,
+            region=item.region,
+            topic=item.topic,
+            tags=item.tags,
+            metadata_=item.metadata,
+        )
+        db.add(event)
+        ingested += 1
 
         db.flush()
         if sync_event_to_map_object(db, event):
             map_synced += 1
 
-    if job_id is not None:
+    if touch_job and job_id is not None:
         job = db.query(ParseJob).filter(ParseJob.id == job_id).first()
         if job:
             from datetime import datetime, timezone
@@ -69,4 +60,4 @@ def ingest_events(
             job.last_error = None
 
     db.commit()
-    return ingested, updated, map_synced
+    return ingested, updated, skipped, map_synced

centers/analytics/api/app/services/migrations.py

@@ -0,0 +1,28 @@
+from sqlalchemy import inspect, text
+from sqlalchemy.engine import Engine
+
+
+def migrate_schema(engine: Engine) -> None:
+    """Apply lightweight schema updates for existing deployments."""
+    inspector = inspect(engine)
+    if "parse_jobs" not in inspector.get_table_names():
+        return
+
+    columns = {col["name"] for col in inspector.get_columns("parse_jobs")}
+    statements: list[str] = []
+
+    if "interval_seconds" not in columns:
+        statements.append(
+            "ALTER TABLE parse_jobs ADD COLUMN interval_seconds INTEGER NOT NULL DEFAULT 3600"
+        )
+    if "is_active" not in columns:
+        statements.append(
+            "ALTER TABLE parse_jobs ADD COLUMN is_active BOOLEAN NOT NULL DEFAULT TRUE"
+        )
+
+    if not statements:
+        return
+
+    with engine.begin() as conn:
+        for stmt in statements:
+            conn.execute(text(stmt))

centers/analytics/api/app/services/scheduler.py

@@ -0,0 +1,58 @@
+import logging
+import threading
+import time
+from datetime import datetime, timezone
+
+from ..database import SessionLocal
+from ..models import ParseJob
+from .jobs import enqueue_job
+
+logger = logging.getLogger(__name__)
+
+TICK_SECONDS = 30
+RECURRING_STATUSES = ("completed", "failed")
+
+
+def run_scheduler_tick() -> None:
+    db = SessionLocal()
+    try:
+        now = datetime.now(timezone.utc)
+        jobs = (
+            db.query(ParseJob)
+            .filter(
+                ParseJob.is_active.is_(True),
+                ParseJob.interval_seconds > 0,
+                ParseJob.status.in_(RECURRING_STATUSES),
+            )
+            .all()
+        )
+        for job in jobs:
+            if job.last_run_at is None:
+                continue
+            elapsed = (now - job.last_run_at).total_seconds()
+            if elapsed < job.interval_seconds:
+                continue
+
+            job.status = "queued"
+            job.last_error = None
+            db.commit()
+            enqueue_job(job.id, job.source_type, job.source_config)
+            logger.info("Re-queued recurring job %s (interval %ss)", job.id, job.interval_seconds)
+    except Exception:
+        logger.exception("Scheduler tick failed")
+        db.rollback()
+    finally:
+        db.close()
+
+
+def _scheduler_loop(stop_event: threading.Event) -> None:
+    while not stop_event.wait(TICK_SECONDS):
+        run_scheduler_tick()
+
+
+def start_scheduler() -> threading.Event:
+    stop_event = threading.Event()
+    thread = threading.Thread(target=_scheduler_loop, args=(stop_event,), daemon=True)
+    thread.start()
+    logger.info("Parse job scheduler started (tick every %ss)", TICK_SECONDS)
+    return stop_event

centers/analytics/frontend/src/api/admin.ts

@@ -9,6 +9,7 @@ import type {
   EventRecord,
   ParseJob,
   ParseJobCreate,
+  ParseJobUpdate,
   TimelinePoint,
   TopItem,
 } from "../types/admin";
@@ -44,6 +45,22 @@ export function retryJob(jobId: number): Promise<ParseJob> {
   );
 }
 
+export function updateJob(jobId: number, payload: ParseJobUpdate): Promise<ParseJob> {
+  return request<ParseJob>(
+    `/jobs/${jobId}`,
+    { method: "PATCH", body: JSON.stringify(payload) },
+    ADMIN_BASE,
+  );
+}
+
+export function deleteJob(jobId: number): Promise<void> {
+  return request<void>(
+    `/jobs/${jobId}`,
+    { method: "DELETE" },
+    ADMIN_BASE,
+  );
+}
+
 export function fetchEvents(filters: EventFilters = {}): Promise<EventListResponse> {
   return request<EventListResponse>(
     `/events${buildQuery(filters as Record<string, string | number | undefined>)}`,

centers/analytics/frontend/src/components/map/CoordsTools.vue

@@ -1,5 +1,5 @@
 <script setup lang="ts">
-import { ref } from "vue";
+import { ref, watch } from "vue";
 import { CITY_PRESETS } from "../../config/cities";
 import type { LeafletMapApi } from "../../composables/useLeafletMap";
 import { parseCoordsInput } from "../../composables/usePlaceSearch";
@@ -16,13 +16,13 @@ function formatCoords(lat: number, lng: number): string {
   return `${lat.toFixed(6)}, ${lng.toFixed(6)}`;
 }
 
-async function copyText(text: string) {
-  try {
-    await navigator.clipboard.writeText(text);
-  } catch {
-    // ignore
-  }
-}
+watch(
+  () => props.centerCoords,
+  (coords) => {
+    coordsInput.value = formatCoords(coords.lat, coords.lng);
+  },
+  { immediate: true, deep: true },
+);
 
 function centerOnInput() {
   const parsed = parseCoordsInput(coordsInput.value);
@@ -36,37 +36,37 @@ function centerOnCity() {
   props.mapApi.flyTo(city.latitude, city.longitude, city.zoom ?? 12);
 }
 
-function copyCenter() {
-  void copyText(formatCoords(props.centerCoords.lat, props.centerCoords.lng));
-}
-
-function copyInput() {
-  void copyText(coordsInput.value);
+function clearCoords() {
+  coordsInput.value = "";
 }
 </script>
 
 <template>
-  <div class="coords-tools">
-    <label class="coords-label">Центрировать на:</label>
-    <input
-      v-model="coordsInput"
-      type="text"
-      class="coord-input"
-      placeholder="48.65, 37.67"
-      @keydown.enter="centerOnInput"
-    />
-    <button type="button" class="icon-btn" title="Перейти" @click="centerOnInput">➜</button>
-    <button type="button" class="icon-btn" title="Копировать" @click="copyInput">⎘</button>
+  <div class="coords-tools toolbar-group">
+    <div class="coord-field">
+      <input
+        v-model="coordsInput"
+        type="text"
+        class="coord-input"
+        placeholder="lat, lng"
+        title="Координаты центра"
+        @keydown.enter="centerOnInput"
+      />
+      <button
+        v-if="coordsInput"
+        type="button"
+        class="coord-clear"
+        title="Очистить"
+        @click="clearCoords"
+      >×</button>
+    </div>
+    <button type="button" class="icon-btn" title="Перейти к координатам" @click="centerOnInput">➜</button>
 
-    <select v-model="selectedCity" class="city-select" @change="centerOnCity">
-      <option value="" disabled>🏘 Город</option>
+    <select v-model="selectedCity" class="city-select" title="Город" @change="centerOnCity">
+      <option value="" disabled>🏘</option>
       <option v-for="city in CITY_PRESETS" :key="city.name" :value="city.name">
         {{ city.name }}
       </option>
     </select>
-
-    <span class="current-center-label">Текущий центр:</span>
-    <span class="current-coords">{{ formatCoords(centerCoords.lat, centerCoords.lng) }}</span>
-    <button type="button" class="icon-btn" title="Копировать центр" @click="copyCenter">⎘</button>
   </div>
 </template>

centers/analytics/frontend/src/components/map/MapToolbar.vue

@@ -21,6 +21,8 @@ const props = defineProps<{
   topic: string;
   sourceType: string;
   loading?: boolean;
+  statusText?: string;
+  statusError?: boolean;
 }>();
 
 const emit = defineEmits<{
@@ -33,8 +35,10 @@ const emit = defineEmits<{
 }>();
 
 const dateInput = ref<HTMLInputElement | null>(null);
+const rangeBtnRef = ref<HTMLButtonElement | null>(null);
 const rangeOpen = ref(false);
 const filtersOpen = ref(false);
+const dropdownPos = ref({ top: 0, left: 0 });
 let picker: flatpickr.Instance | null = null;
 
 const availableDates = computed(() => props.filters?.available_dates ?? []);
@@ -74,6 +78,31 @@ function selectRange(preset: DateRangePreset) {
   emit("apply");
 }
 
+function toggleRangeOpen(event: MouseEvent) {
+  event.stopPropagation();
+  if (!rangeOpen.value && rangeBtnRef.value) {
+    const rect = rangeBtnRef.value.getBoundingClientRect();
+    dropdownPos.value = { top: rect.bottom + 4, left: rect.left };
+  }
+  rangeOpen.value = !rangeOpen.value;
+  if (rangeOpen.value) {
+    filtersOpen.value = false;
+  }
+}
+
+function toggleFiltersOpen(event: MouseEvent) {
+  event.stopPropagation();
+  filtersOpen.value = !filtersOpen.value;
+  if (filtersOpen.value) {
+    rangeOpen.value = false;
+  }
+}
+
+function closeDropdowns() {
+  rangeOpen.value = false;
+  filtersOpen.value = false;
+}
+
 function resetFilters() {
   emit("update:region", "");
   emit("update:topic", "");
@@ -82,6 +111,8 @@ function resetFilters() {
 }
 
 onMounted(() => {
+  document.addEventListener("click", closeDropdowns);
+
   if (!dateInput.value) return;
   picker = flatpickr(dateInput.value, {
     locale: Russian,
@@ -101,6 +132,7 @@ onMounted(() => {
 });
 
 onUnmounted(() => {
+  document.removeEventListener("click", closeDropdowns);
   picker?.destroy();
 });
 
@@ -115,7 +147,8 @@ watch(
 
 <template>
   <div class="map-toolbar">
-    <div class="date-navigator">
+    <div class="map-toolbar-inner">
+    <div class="toolbar-group date-navigator">
       <button type="button" class="nav-btn" title="Первая дата" @click="navigateDate('first')">❮❮</button>
       <button type="button" class="nav-btn" title="Предыдущий" @click="navigateDate('prev')">❮</button>
       <div class="date-selector">
@@ -126,34 +159,46 @@ watch(
 
       <div class="filter-btn-container">
         <button
+          ref="rangeBtnRef"
           type="button"
           class="filter-btn"
           title="Период"
           :class="{ active: rangePreset !== 'all' }"
-          @click="rangeOpen = !rangeOpen"
+          @click="toggleRangeOpen"
         >⏳</button>
-        <div v-if="rangeOpen" class="dropdown-content">
-          <button
-            v-for="opt in RANGE_OPTIONS"
-            :key="opt.value"
-            type="button"
-            class="range-option"
-            :class="{ active: rangePreset === opt.value }"
-            @click="selectRange(opt.value)"
-          >
-            {{ opt.label }}
-          </button>
-        </div>
       </div>
     </div>
 
-    <div class="filter-row">
-      <button type="button" class="filter-btn mobile-toggle" @click="filtersOpen = !filtersOpen">📁</button>
+    <Teleport to="body">
+      <div
+        v-if="rangeOpen"
+        class="dropdown-content range-dropdown dropdown-fixed"
+        :style="{ top: `${dropdownPos.top}px`, left: `${dropdownPos.left}px` }"
+        @click.stop
+      >
+        <button
+          v-for="opt in RANGE_OPTIONS"
+          :key="opt.value"
+          type="button"
+          class="range-option"
+          :class="{ active: rangePreset === opt.value }"
+          @click="selectRange(opt.value)"
+        >
+          {{ opt.label }}
+        </button>
+      </div>
+    </Teleport>
 
-      <div class="filter-controls" :class="{ open: filtersOpen }">
+    <div class="toolbar-sep" />
+
+    <div class="toolbar-group filter-row">
+      <button type="button" class="filter-btn mobile-toggle" title="Фильтры" @click="toggleFiltersOpen">📁</button>
+
+      <div class="filter-controls" :class="{ open: filtersOpen }" @click.stop>
         <select
           :value="region"
           class="filter-select"
+          title="Регион"
           @change="emit('update:region', ($event.target as HTMLSelectElement).value); emit('apply')"
         >
           <option value="">Все регионы</option>
@@ -163,6 +208,7 @@ watch(
         <select
           :value="topic"
           class="filter-select"
+          title="Тема"
           @change="emit('update:topic', ($event.target as HTMLSelectElement).value); emit('apply')"
         >
           <option value="">Все темы</option>
@@ -172,6 +218,7 @@ watch(
         <select
           :value="sourceType"
           class="filter-select"
+          title="Источник"
           @change="emit('update:sourceType', ($event.target as HTMLSelectElement).value); emit('apply')"
         >
           <option value="">Все источники</option>
@@ -180,8 +227,22 @@ watch(
 
         <button type="button" class="btn btn-sm" @click="resetFilters">Сбросить</button>
       </div>
+    </div>
 
-      <span v-if="loading" class="toolbar-status">Загрузка...</span>
+    <div class="toolbar-sep" />
+
+    <slot name="coords" />
+
+    <div class="toolbar-sep" />
+
+    <slot name="search" />
+
+    <span
+      v-if="statusText"
+      class="toolbar-status"
+      :class="{ 'toolbar-status-error': statusError }"
+    >{{ statusText }}</span>
+    <span v-else-if="loading" class="toolbar-status">Загрузка...</span>
     </div>
   </div>
 </template>

centers/analytics/frontend/src/components/map/PlaceSearch.vue

@@ -30,12 +30,13 @@ function selectPlace(place: PlaceSearchResult) {
 </script>
 
 <template>
-  <div class="place-search">
+  <div class="place-search toolbar-group">
     <input
       v-model="query"
       type="text"
       class="search-input"
-      placeholder="Поиск населённого пункта"
+      placeholder="Поиск НП"
+      title="Поиск населённого пункта"
       @keydown.enter="runSearch"
     />
     <button type="button" class="icon-btn" title="Найти" :disabled="searching" @click="runSearch">

centers/analytics/frontend/src/styles/map-toolbar.css

@@ -1,33 +1,62 @@
 .map-toolbar {
-  display: flex;
-  flex-direction: column;
-  gap: 0.5rem;
-  padding: 0.5rem 0.75rem;
-  background: #fff;
-  border-bottom: 1px solid #e0e0e0;
+  position: relative;
+  z-index: 50;
   flex-shrink: 0;
+  background: #d4d4d4;
+  border-bottom: 1px solid #b0b0b0;
+  overflow: visible;
+}
+
+.map-toolbar-inner {
+  display: flex;
+  flex-direction: row;
+  flex-wrap: nowrap;
+  align-items: center;
+  gap: 0.35rem;
+  padding: 0.35rem 0.6rem;
+  min-height: 38px;
+  overflow-x: auto;
+  overflow-y: visible;
+}
+
+.toolbar-group {
+  display: flex;
+  flex-wrap: nowrap;
+  align-items: center;
+  gap: 0.3rem;
+  flex-shrink: 0;
+}
+
+.toolbar-sep {
+  width: 1px;
+  height: 24px;
+  background: #aaa;
+  flex-shrink: 0;
+  margin: 0 0.15rem;
 }
 
 .date-navigator {
   display: flex;
-  flex-wrap: wrap;
+  flex-wrap: nowrap;
   align-items: center;
-  gap: 0.35rem;
+  gap: 0.25rem;
 }
 
 .nav-btn,
 .filter-btn {
-  width: 30px;
-  height: 30px;
-  border: 1px solid #ccc;
-  border-radius: 4px;
-  background: #f0f0f0;
+  width: 28px;
+  height: 28px;
+  min-width: 28px;
+  border: 1px solid #aaa;
+  border-radius: 3px;
+  background: #ececec;
   cursor: pointer;
   display: flex;
   align-items: center;
   justify-content: center;
-  font-size: 0.85rem;
+  font-size: 0.8rem;
   padding: 0;
+  flex-shrink: 0;
 }
 
 .nav-btn:hover,
@@ -36,22 +65,37 @@
 }
 
 .date-selector {
-  width: 90px;
+  width: 82px;
+  flex-shrink: 0;
 }
 
 .date-picker-input {
   width: 100%;
-  height: 30px;
-  padding: 0 6px;
-  border: 1px solid #ccc;
-  border-radius: 4px;
-  font-size: 0.85rem;
+  height: 28px;
+  padding: 0 4px;
+  border: 1px solid #aaa;
+  border-radius: 3px;
+  font-size: 0.8rem;
   text-align: center;
   box-sizing: border-box;
+  background: #fff;
 }
 
 .filter-btn-container {
   position: relative;
+  z-index: 60;
+}
+
+.dropdown-content {
+  position: absolute;
+  top: calc(100% + 4px);
+  left: 0;
+  z-index: 1100;
+  background: #fff;
+  border: 1px solid #ddd;
+  border-radius: 5px;
+  box-shadow: 0 4px 16px rgba(0, 0, 0, 0.15);
+  min-width: 140px;
 }
 
 .filter-btn.active {
@@ -59,17 +103,13 @@
   background: #e6f2ff;
 }
 
-.dropdown-content {
-  position: absolute;
-  top: 100%;
-  left: 0;
-  z-index: 1000;
-  background: #fff;
-  border: 1px solid #ddd;
-  border-radius: 5px;
-  box-shadow: 0 2px 10px rgba(0, 0, 0, 0.1);
-  margin-top: 4px;
-  min-width: 140px;
+.range-dropdown {
+  white-space: nowrap;
+}
+
+.dropdown-fixed {
+  position: fixed;
+  z-index: 10000;
 }
 
 .range-option {
@@ -94,88 +134,127 @@
 
 .filter-row {
   display: flex;
-  flex-wrap: wrap;
+  flex-wrap: nowrap;
   align-items: center;
-  gap: 0.5rem;
+  gap: 0.3rem;
 }
 
 .filter-controls {
   display: flex;
-  flex-wrap: wrap;
+  flex-wrap: nowrap;
   align-items: center;
-  gap: 0.5rem;
+  gap: 0.3rem;
 }
 
 .filter-select {
-  height: 30px;
-  padding: 0 8px;
-  border: 1px solid #ccc;
-  border-radius: 4px;
-  font-size: 0.85rem;
+  height: 28px;
+  padding: 0 6px;
+  border: 1px solid #aaa;
+  border-radius: 3px;
+  font-size: 0.78rem;
+  max-width: 130px;
+  background: #fff;
+}
+
+.filter-select option {
   max-width: 200px;
 }
 
 .toolbar-status {
   margin-left: auto;
-  font-size: 0.8rem;
-  color: #888;
+  padding-left: 0.5rem;
+  font-size: 0.75rem;
+  color: #555;
+  white-space: nowrap;
+  flex-shrink: 0;
+}
+
+.toolbar-status-error {
+  color: #c62828;
 }
 
 .coords-tools {
   display: flex;
-  flex-wrap: wrap;
+  flex-wrap: nowrap;
   align-items: center;
-  gap: 0.4rem;
-  padding: 0.4rem 0.75rem;
-  background: #fafafa;
-  border-bottom: 1px solid #eee;
-  font-size: 0.85rem;
+  gap: 0.25rem;
+  font-size: 0.8rem;
 }
 
-.coords-label,
-.current-center-label {
-  color: #555;
+.coord-field {
+  position: relative;
+  display: flex;
+  align-items: center;
 }
 
 .coord-input,
 .search-input {
   height: 28px;
-  padding: 0 8px;
-  border: 1px solid #ccc;
-  border-radius: 4px;
-  font-size: 0.85rem;
-  width: 150px;
+  padding: 0 22px 0 8px;
+  border: 1px solid #aaa;
+  border-radius: 3px;
+  font-size: 0.78rem;
+  background: #fff;
+  box-sizing: border-box;
+}
+
+.coord-input {
+  width: 200px;
+  font-family: ui-monospace, monospace;
+}
+
+.search-input {
+  width: 140px;
+}
+
+.coord-clear {
+  position: absolute;
+  right: 4px;
+  width: 18px;
+  height: 18px;
+  border: none;
+  background: transparent;
+  color: #888;
+  cursor: pointer;
+  font-size: 1rem;
+  line-height: 1;
+  padding: 0;
+}
+
+.coord-clear:hover {
+  color: #333;
 }
 
 .city-select {
   height: 28px;
-  padding: 0 6px;
-  border: 1px solid #ccc;
-  border-radius: 4px;
-  font-size: 0.85rem;
-}
-
-.current-coords {
-  font-family: ui-monospace, monospace;
-  background: #f5f5f5;
-  padding: 2px 6px;
+  max-width: 110px;
+  padding: 0 4px;
+  border: 1px solid #aaa;
   border-radius: 3px;
+  font-size: 0.78rem;
+  background: #fff;
 }
 
 .icon-btn {
   width: 28px;
   height: 28px;
-  border: 1px solid #ccc;
-  border-radius: 4px;
-  background: #f0f0f0;
+  min-width: 28px;
+  border: 1px solid #aaa;
+  border-radius: 3px;
+  background: #ececec;
   cursor: pointer;
   font-size: 0.85rem;
+  flex-shrink: 0;
+}
+
+.icon-btn:hover {
+  background: #e0e0e0;
 }
 
 .place-search {
   display: flex;
   align-items: center;
-  gap: 0.35rem;
+  gap: 0.25rem;
   position: relative;
 }
 
@@ -194,6 +273,7 @@
   max-height: 200px;
   overflow-y: auto;
   box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
+  min-width: 260px;
 }
 
 .result-btn {
@@ -222,16 +302,6 @@
   background: #fff;
 }
 
-.map-tools-row {
-  display: flex;
-  flex-wrap: wrap;
-  align-items: center;
-  gap: 0.5rem;
-  padding: 0.25rem 0.75rem;
-  background: #fafafa;
-  border-bottom: 1px solid #eee;
-}
-
 .map-content {
   position: relative;
   display: flex;
@@ -240,35 +310,67 @@
   overflow: hidden;
 }
 
-.map-status {
-  padding: 0.35rem 0.75rem;
-  font-size: 0.8rem;
-  color: #666;
-  background: #fff;
-  border-bottom: 1px solid #eee;
+.map-toolbar .btn-sm {
+  height: 28px;
+  padding: 0 8px;
+  font-size: 0.75rem;
 }
 
-.map-status.error {
-  color: #c62828;
-}
-
-@media (max-width: 768px) {
+@media (max-width: 1100px) {
   .filter-controls {
     display: none;
-    width: 100%;
+    position: absolute;
+    top: 100%;
+    left: 0;
+    z-index: 1000;
+    flex-direction: column;
+    align-items: stretch;
+    background: #fff;
+    border: 1px solid #ddd;
+    border-radius: 4px;
+    padding: 0.5rem;
+    box-shadow: 0 2px 10px rgba(0, 0, 0, 0.12);
+    min-width: 200px;
+  }
+
+  .filter-row {
+    position: relative;
   }
 
   .filter-controls.open {
     display: flex;
   }
 
+  .filter-select {
+    max-width: none;
+    width: 100%;
+  }
+
   .mobile-toggle {
     display: flex;
   }
 }
 
-@media (min-width: 769px) {
+@media (min-width: 1101px) {
   .mobile-toggle {
     display: none;
   }
 }
+
+@media (max-width: 768px) {
+  .coord-input {
+    width: 150px;
+  }
+
+  .search-input {
+    width: 110px;
+  }
+
+  .city-select {
+    max-width: 80px;
+  }
+
+  .toolbar-sep {
+    display: none;
+  }
+}

centers/analytics/frontend/src/types/admin.ts

@@ -3,6 +3,8 @@ export interface ParseJob {
   source_type: string;
   source_config: Record<string, unknown>;
   schedule: string | null;
+  interval_seconds: number;
+  is_active: boolean;
   status: string;
   last_run_at: string | null;
   last_error: string | null;
@@ -13,6 +15,14 @@ export interface ParseJobCreate {
   source_type: string;
   source_config: Record<string, unknown>;
   schedule?: string | null;
+  interval_seconds?: number;
+  is_active?: boolean;
+}
+
+export interface ParseJobUpdate {
+  source_config?: Record<string, unknown>;
+  interval_seconds?: number;
+  is_active?: boolean;
 }
 
 export interface EventRecord {

centers/analytics/frontend/src/views/MapViewPage.vue

@@ -309,16 +309,17 @@ onUnmounted(() => {
       v-model:source-type="sourceType"
       :filters="mapFilters"
       :loading="loading"
+      :status-text="error || `${objects.length} объектов`"
+      :status-error="Boolean(error)"
       @apply="loadObjects"
-    />
-
-    <div class="map-tools-row">
-      <CoordsTools :map-api="mapApi" :center-coords="centerCoords" />
-      <PlaceSearch :map-api="mapApi" />
-    </div>
-
-    <div v-if="error" class="map-status error">{{ error }}</div>
-    <div v-else class="map-status">{{ objects.length }} объектов на карте</div>
+    >
+      <template #coords>
+        <CoordsTools :map-api="mapApi" :center-coords="centerCoords" />
+      </template>
+      <template #search>
+        <PlaceSearch :map-api="mapApi" />
+      </template>
+    </MapToolbar>
 
     <div class="map-content">
       <MapView

centers/analytics/frontend/src/views/ParsersView.vue

@@ -1,21 +1,55 @@
 <script setup lang="ts">
 import { onMounted, onUnmounted, ref } from "vue";
-import { createJob, fetchJobs, retryJob } from "../api/admin";
+import { createJob, deleteJob, fetchJobs, retryJob, updateJob } from "../api/admin";
 import type { ParseJob } from "../types/admin";
 
+const INTERVAL_OPTIONS = [
+  { value: 900, label: "15 мин" },
+  { value: 1800, label: "30 мин" },
+  { value: 3600, label: "1 ч" },
+  { value: 10800, label: "3 ч" },
+  { value: 21600, label: "6 ч" },
+  { value: 86400, label: "24 ч" },
+];
+
 const jobs = ref<ParseJob[]>([]);
 const loading = ref(true);
 const error = ref("");
 const submitting = ref(false);
+const editingJob = ref<ParseJob | null>(null);
+const editForm = ref({
+  channel: "",
+  limit: 50,
+  interval_seconds: 3600,
+  is_active: true,
+});
 
 const form = ref({
   source_type: "telegram",
   channel: "",
   limit: 50,
+  interval_seconds: 3600,
 });
 
 let pollTimer: ReturnType<typeof setInterval> | null = null;
 
+function channelFromConfig(config: Record<string, unknown>): string {
+  return typeof config.channel === "string" ? config.channel : "";
+}
+
+function limitFromConfig(config: Record<string, unknown>): number {
+  const limit = config.limit;
+  return typeof limit === "number" ? limit : 50;
+}
+
+function formatInterval(seconds: number): string {
+  const opt = INTERVAL_OPTIONS.find((item) => item.value === seconds);
+  if (opt) return opt.label;
+  if (seconds < 3600) return `${Math.round(seconds / 60)} мин`;
+  if (seconds < 86400) return `${Math.round(seconds / 3600)} ч`;
+  return `${Math.round(seconds / 86400)} д`;
+}
+
 async function loadJobs() {
   try {
     jobs.value = await fetchJobs();
@@ -42,6 +76,8 @@ async function handleSubmit() {
         channel: form.value.channel.trim(),
         limit: form.value.limit,
       },
+      interval_seconds: form.value.interval_seconds,
+      is_active: true,
     });
     form.value.channel = "";
     await loadJobs();
@@ -52,6 +88,61 @@ async function handleSubmit() {
   }
 }
 
+function openEdit(job: ParseJob) {
+  editingJob.value = job;
+  editForm.value = {
+    channel: channelFromConfig(job.source_config),
+    limit: limitFromConfig(job.source_config),
+    interval_seconds: job.interval_seconds,
+    is_active: job.is_active,
+  };
+}
+
+function closeEdit() {
+  editingJob.value = null;
+}
+
+async function handleSaveEdit() {
+  if (!editingJob.value) return;
+  if (!editForm.value.channel.trim()) {
+    error.value = "Укажите канал Telegram";
+    return;
+  }
+
+  submitting.value = true;
+  error.value = "";
+  try {
+    await updateJob(editingJob.value.id, {
+      source_config: {
+        channel: editForm.value.channel.trim(),
+        limit: editForm.value.limit,
+      },
+      interval_seconds: editForm.value.interval_seconds,
+      is_active: editForm.value.is_active,
+    });
+    closeEdit();
+    await loadJobs();
+  } catch (err) {
+    error.value = err instanceof Error ? err.message : "Не удалось сохранить задание";
+  } finally {
+    submitting.value = false;
+  }
+}
+
+async function handleDelete(job: ParseJob) {
+  if (!window.confirm(`Удалить парсер #${job.id} (${channelFromConfig(job.source_config)})?`)) {
+    return;
+  }
+  error.value = "";
+  try {
+    await deleteJob(job.id);
+    if (editingJob.value?.id === job.id) closeEdit();
+    await loadJobs();
+  } catch (err) {
+    error.value = err instanceof Error ? err.message : "Не удалось удалить задание";
+  }
+}
+
 async function handleRetry(jobId: number) {
   try {
     await retryJob(jobId);
@@ -88,7 +179,12 @@ onUnmounted(() => {
     <h2 class="page-heading">Парсеры</h2>
 
     <section class="card">
-      <h3>Новое задание</h3>
+      <h3>Новый парсер</h3>
+      <p class="hint">
+        Активные парсеры подхватываются real-time listener (новые посты сразу в БД).
+        Дополнительно batch-прогон по интервалу забирает последние N постов.
+        Дубликаты по <code>source_url</code> не записываются.
+      </p>
       <form class="admin-form" @submit.prevent="handleSubmit">
         <div class="form-row">
           <label>
@@ -105,10 +201,18 @@ onUnmounted(() => {
             Лимит
             <input v-model.number="form.limit" type="number" min="1" max="1000" />
           </label>
+          <label>
+            Интервал
+            <select v-model.number="form.interval_seconds">
+              <option v-for="opt in INTERVAL_OPTIONS" :key="opt.value" :value="opt.value">
+                {{ opt.label }}
+              </option>
+            </select>
+          </label>
         </div>
         <div class="form-actions">
           <button type="submit" class="btn btn-primary" :disabled="submitting">
-            {{ submitting ? "Создание..." : "Создать задание" }}
+            {{ submitting ? "Создание..." : "Создать парсер" }}
           </button>
         </div>
       </form>
@@ -116,14 +220,16 @@ onUnmounted(() => {
     </section>
 
     <section class="card">
-      <h3>Задания <span v-if="loading" class="muted">(загрузка...)</span></h3>
+      <h3>Парсеры <span v-if="loading" class="muted">(загрузка...)</span></h3>
       <div class="table-wrap">
         <table class="admin-table">
           <thead>
             <tr>
               <th>ID</th>
-              <th>Источник</th>
-              <th>Конфиг</th>
+              <th>Канал</th>
+              <th>Лимит</th>
+              <th>Интервал</th>
+              <th>Активен</th>
               <th>Статус</th>
               <th>Последний запуск</th>
               <th>Ошибка</th>
@@ -132,30 +238,122 @@ onUnmounted(() => {
           </thead>
           <tbody>
             <tr v-if="jobs.length === 0 && !loading">
-              <td colspan="7" class="empty">Нет заданий</td>
+              <td colspan="9" class="empty">Нет парсеров</td>
             </tr>
             <tr v-for="job in jobs" :key="job.id">
               <td>{{ job.id }}</td>
-              <td>{{ job.source_type }}</td>
-              <td class="mono">{{ JSON.stringify(job.source_config) }}</td>
+              <td>{{ channelFromConfig(job.source_config) }}</td>
+              <td>{{ limitFromConfig(job.source_config) }}</td>
+              <td>{{ formatInterval(job.interval_seconds) }}</td>
+              <td>{{ job.is_active ? "да" : "нет" }}</td>
               <td>
                 <span class="badge" :class="statusClass(job.status)">{{ job.status }}</span>
               </td>
               <td>{{ formatDate(job.last_run_at) }}</td>
               <td class="error-cell">{{ job.last_error ?? "—" }}</td>
-              <td>
+              <td class="actions-cell">
+                <button class="btn btn-sm" type="button" @click="openEdit(job)">Изменить</button>
                 <button
                   v-if="job.status === 'failed' || job.status === 'error'"
                   class="btn btn-sm"
+                  type="button"
                   @click="handleRetry(job.id)"
                 >
                   Повторить
                 </button>
+                <button
+                  class="btn btn-sm btn-danger"
+                  type="button"
+                  :disabled="job.status === 'running'"
+                  @click="handleDelete(job)"
+                >
+                  Удалить
+                </button>
               </td>
             </tr>
           </tbody>
         </table>
       </div>
     </section>
+
+    <div v-if="editingJob" class="modal-backdrop" @click.self="closeEdit">
+      <div class="modal card">
+        <h3>Редактировать парсер #{{ editingJob.id }}</h3>
+        <form class="admin-form" @submit.prevent="handleSaveEdit">
+          <div class="form-row">
+            <label>
+              Канал
+              <input v-model="editForm.channel" type="text" required />
+            </label>
+            <label>
+              Лимит
+              <input v-model.number="editForm.limit" type="number" min="1" max="1000" />
+            </label>
+            <label>
+              Интервал
+              <select v-model.number="editForm.interval_seconds">
+                <option v-for="opt in INTERVAL_OPTIONS" :key="opt.value" :value="opt.value">
+                  {{ opt.label }}
+                </option>
+              </select>
+            </label>
+            <label class="checkbox-row">
+              <input v-model="editForm.is_active" type="checkbox" />
+              <span>Активен (периодический запуск)</span>
+            </label>
+          </div>
+          <div class="form-actions">
+            <button type="button" class="btn" @click="closeEdit">Отмена</button>
+            <button type="submit" class="btn btn-primary" :disabled="submitting">
+              {{ submitting ? "Сохранение..." : "Сохранить" }}
+            </button>
+          </div>
+        </form>
+      </div>
+    </div>
   </div>
 </template>
+
+<style scoped>
+.hint {
+  margin: 0 0 1rem;
+  font-size: 0.875rem;
+  color: #666;
+  line-height: 1.5;
+}
+
+.actions-cell {
+  white-space: nowrap;
+}
+
+.actions-cell .btn {
+  margin-right: 0.25rem;
+}
+
+.btn-danger {
+  color: #b91c1c;
+  border-color: #fecaca;
+}
+
+.modal-backdrop {
+  position: fixed;
+  inset: 0;
+  z-index: 100;
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  background: rgba(0, 0, 0, 0.4);
+}
+
+.modal {
+  width: min(520px, 92vw);
+  margin: 0;
+}
+
+.checkbox-row {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+  margin-top: 1.5rem;
+}
+</style>

centers/parsing/workers/worker.py

@@ -1,10 +1,9 @@
-"""CP worker: poll Redis jobs, parse Telegram, ingest to CA."""
+"""CP worker: Redis jobs + real-time Telethon listener."""
 
 import asyncio
 import json
 import logging
 import os
-import time
 
 import httpx
 import redis
@@ -17,6 +16,8 @@ from workers.sources.telegram_client import (
     fetch_channel_posts,
     normalize_channel,
 )
+from workers.sources.telegram_listener import TelegramListener
+from workers.sources.telegram_session import close_shared_client, get_shared_client
 
 logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
 logger = logging.getLogger("cp-worker")
@@ -26,19 +27,30 @@ JOB_QUEUE_KEY = "cp:jobs"
 CA_API_URL = os.getenv("CA_API_URL", "http://ca-api:8000")
 INTERNAL_TOKEN = os.getenv("INTERNAL_TOKEN", "dev-internal-token")
 POLL_TIMEOUT = int(os.getenv("WORKER_POLL_TIMEOUT", "5"))
+LISTENER_ENABLED = os.getenv("TELEGRAM_LISTENER_ENABLED", "true").lower() not in (
+    "0",
+    "false",
+    "no",
+    "off",
+)
 
 
 def get_redis() -> redis.Redis:
     return redis.from_url(REDIS_URL, decode_responses=True)
 
 
-async def process_telegram_job(job_id: int, source_config: dict) -> tuple[list[dict], str | None]:
+async def process_telegram_job(
+    job_id: int,
+    source_config: dict,
+    *,
+    client=None,
+) -> tuple[list[dict], str | None]:
     channel = source_config.get("channel", "creamy_caprice")
     limit = int(source_config.get("limit", 100))
 
     try:
         username = normalize_channel(channel)
-        posts = await fetch_channel_posts(username, limit=limit)
+        posts = await fetch_channel_posts(username, limit=limit, client=client)
     except (TelegramConfigError, TelegramAuthError, ValueError) as exc:
         return [], str(exc)
     except Exception as exc:
@@ -60,7 +72,14 @@ async def post_ingest(job_id: int, events: list[dict]) -> None:
             headers={"X-Internal-Token": INTERNAL_TOKEN},
         )
         response.raise_for_status()
-        logger.info("Ingested %s events for job %s: %s", len(events), job_id, response.json())
+        result = response.json()
+        logger.info(
+            "Ingested job %s: new=%s skipped=%s map=%s",
+            job_id,
+            result.get("ingested", 0),
+            result.get("skipped", 0),
+            result.get("map_objects_synced", 0),
+        )
 
 
 async def patch_job_status(job_id: int, status: str, error: str | None = None) -> None:
@@ -76,7 +95,7 @@ async def patch_job_status(job_id: int, status: str, error: str | None = None) -
         response.raise_for_status()
 
 
-async def handle_job(payload: dict) -> None:
+async def handle_job(payload: dict, *, tg_client=None) -> None:
     job_id = payload["job_id"]
     source_type = payload["source_type"]
     source_config = payload.get("source_config", {})
@@ -85,7 +104,7 @@ async def handle_job(payload: dict) -> None:
     await patch_job_status(job_id, "running")
 
     if source_type == "telegram":
-        events, error = await process_telegram_job(job_id, source_config)
+        events, error = await process_telegram_job(job_id, source_config, client=tg_client)
     else:
         events, error = [], f"Unsupported source_type: {source_type}"
 
@@ -104,28 +123,54 @@ async def handle_job(payload: dict) -> None:
         await patch_job_status(job_id, "failed", error=str(exc))
 
 
-async def worker_loop() -> None:
+async def worker_loop(*, tg_client=None) -> None:
     r = get_redis()
     logger.info("CP worker started, polling %s", JOB_QUEUE_KEY)
 
     while True:
         try:
-            item = r.blpop(JOB_QUEUE_KEY, timeout=POLL_TIMEOUT)
+            item = await asyncio.to_thread(r.blpop, JOB_QUEUE_KEY, POLL_TIMEOUT)
             if not item:
                 continue
             _, raw = item
             payload = json.loads(raw)
-            await handle_job(payload)
+            await handle_job(payload, tg_client=tg_client)
         except redis.RedisError as exc:
             logger.error("Redis error: %s", exc)
-            time.sleep(3)
+            await asyncio.sleep(3)
         except Exception:
             logger.exception("Unexpected worker error")
-            time.sleep(1)
+            await asyncio.sleep(1)
+
+
+async def run_with_listener() -> None:
+    client = await get_shared_client()
+    listener = TelegramListener(client)
+    worker_task = asyncio.create_task(worker_loop(tg_client=client))
+    listener_task = asyncio.create_task(listener.run())
+
+    logger.info("Telegram listener enabled (shared session with batch worker)")
+    try:
+        await client.run_until_disconnected()
+    finally:
+        worker_task.cancel()
+        listener_task.cancel()
+        await close_shared_client()
+
+
+async def run_batch_only() -> None:
+    logger.info("Telegram listener disabled")
+    await worker_loop(tg_client=None)
 
 
 def main() -> None:
-    asyncio.run(worker_loop())
+    try:
+        if LISTENER_ENABLED:
+            asyncio.run(run_with_listener())
+        else:
+            asyncio.run(run_batch_only())
+    except KeyboardInterrupt:
+        pass
 
 
 if __name__ == "__main__":

centers/parsing/workers/workers/sources/telegram_client.py

@@ -4,6 +4,7 @@ import os
 import re
 from pathlib import Path
 
+from telethon import TelegramClient
 from telethon.errors import AuthKeyUnregisteredError, SessionPasswordNeededError
 
 from workers.models import TelegramPost
@@ -46,54 +47,76 @@ def _build_post_url(channel: str, message_id: int) -> str:
     return f"https://t.me/{username}/{message_id}"
 
 
-async def fetch_channel_posts(channel: str, limit: int = 100) -> list[TelegramPost]:
-    limit = max(1, min(limit, MAX_POSTS))
-    try:
-        api_id, api_hash, session_path = _get_config()
-    except ValueError as exc:
-        raise TelegramConfigError(str(exc)) from exc
-
+def message_to_post(message, channel: str) -> TelegramPost | None:
+    text = (message.text or message.message or "").strip()
+    if not text:
+        return None
     username = normalize_channel(channel)
+    return TelegramPost(
+        id=message.id,
+        text=text,
+        date=message.date,
+        url=_build_post_url(username, message.id),
+        channel=username,
+    )
 
-    if not Path(session_path).exists():
-        raise TelegramAuthError(
-            f"Файл сессии не найден: {session_path}. "
-            "Выполните: python scripts/telegram_auth.py"
-        )
 
-    client = create_client(session_path, api_id, api_hash)
-    posts: list[TelegramPost] = []
+async def fetch_channel_posts(
+    channel: str,
+    limit: int = 100,
+    *,
+    client: TelegramClient | None = None,
+) -> list[TelegramPost]:
+    limit = max(1, min(limit, MAX_POSTS))
+    username = normalize_channel(channel)
+    own_client = client is None
 
-    try:
-        await client.connect()
-        if not await client.is_user_authorized():
+    if own_client:
+        try:
+            api_id, api_hash, session_path = _get_config()
+        except ValueError as exc:
+            raise TelegramConfigError(str(exc)) from exc
+
+        if not Path(session_path).exists():
             raise TelegramAuthError(
-                "Telegram-сессия не авторизована. Выполните: python scripts/telegram_auth.py"
+                f"Файл сессии не найден: {session_path}. "
+                "Выполните: python scripts/telegram_auth.py"
             )
 
-        entity = await client.get_entity(username)
-        async for message in client.iter_messages(entity, limit=limit):
-            if not message.text:
-                continue
-            posts.append(
-                TelegramPost(
-                    id=message.id,
-                    text=message.text.strip(),
-                    date=message.date,
-                    url=_build_post_url(username, message.id),
-                    channel=username,
+        client = create_client(session_path, api_id, api_hash)
+        posts: list[TelegramPost] = []
+
+        try:
+            await client.connect()
+            if not await client.is_user_authorized():
+                raise TelegramAuthError(
+                    "Telegram-сессия не авторизована. Выполните: python scripts/telegram_auth.py"
                 )
-            )
-    except AuthKeyUnregisteredError as exc:
-        raise TelegramAuthError(
-            "Сессия Telegram недействительна. Переавторизуйтесь: python scripts/telegram_auth.py"
-        ) from exc
-    except SessionPasswordNeededError as exc:
-        raise TelegramAuthError(
-            "Для аккаунта включена двухфакторная аутентификация. "
-            "Авторизуйтесь через scripts/telegram_auth.py с паролем 2FA."
-        ) from exc
-    finally:
-        await client.disconnect()
 
+            entity = await client.get_entity(username)
+            async for message in client.iter_messages(entity, limit=limit):
+                post = message_to_post(message, username)
+                if post:
+                    posts.append(post)
+        except AuthKeyUnregisteredError as exc:
+            raise TelegramAuthError(
+                "Сессия Telegram недействительна. Переавторизуйтесь: python scripts/telegram_auth.py"
+            ) from exc
+        except SessionPasswordNeededError as exc:
+            raise TelegramAuthError(
+                "Для аккаунта включена двухфакторная аутентификация. "
+                "Авторизуйтесь через scripts/telegram_auth.py с паролем 2FA."
+            ) from exc
+        finally:
+            await client.disconnect()
+
+        return posts
+
+    assert client is not None
+    posts = []
+    entity = await client.get_entity(username)
+    async for message in client.iter_messages(entity, limit=limit):
+        post = message_to_post(message, username)
+        if post:
+            posts.append(post)
     return posts

centers/parsing/workers/workers/sources/telegram_listener.py

@@ -0,0 +1,150 @@
+"""Real-time Telethon listener for active Telegram parse jobs."""
+
+import asyncio
+import logging
+import os
+from typing import Any
+
+import httpx
+from telethon import TelegramClient, events
+
+from workers.converter import event_record_to_ingest
+from workers.parsers.telegram_events import parse_event_post
+from workers.sources.telegram_client import message_to_post, normalize_channel
+
+logger = logging.getLogger("cp-listener")
+
+CA_API_URL = os.getenv("CA_API_URL", "http://ca-api:8000")
+INTERNAL_TOKEN = os.getenv("INTERNAL_TOKEN", "dev-internal-token")
+REFRESH_SECONDS = int(os.getenv("TELEGRAM_LISTENER_REFRESH_SECONDS", "60"))
+
+
+class TelegramListener:
+    def __init__(self, client: TelegramClient) -> None:
+        self.client = client
+        self._channels: dict[str, int] = {}
+        self._chat_ids: set[int] = set()
+        self._handlers_registered = False
+
+    async def fetch_subscriptions(self) -> dict[str, int]:
+        async with httpx.AsyncClient(timeout=30.0) as http:
+            response = await http.get(
+                f"{CA_API_URL}/internal/listener/subscriptions",
+                headers={"X-Internal-Token": INTERNAL_TOKEN},
+            )
+            response.raise_for_status()
+            data = response.json()
+
+        channels: dict[str, int] = {}
+        for item in data:
+            raw = item.get("channel")
+            job_id = item.get("job_id")
+            if not raw or job_id is None:
+                continue
+            try:
+                key = normalize_channel(str(raw))
+            except ValueError:
+                logger.warning("Skip invalid channel in subscription: %r", raw)
+                continue
+            channels.setdefault(key, int(job_id))
+        return channels
+
+    async def refresh_subscriptions(self) -> None:
+        try:
+            channels = await self.fetch_subscriptions()
+        except Exception:
+            logger.exception("Failed to load listener subscriptions")
+            return
+
+        if channels == self._channels:
+            return
+
+        self._channels = channels
+        self._chat_ids = set()
+        for username in channels:
+            try:
+                entity = await self.client.get_entity(username)
+                self._chat_ids.add(entity.id)
+            except Exception:
+                logger.exception("Cannot resolve channel entity: %s", username)
+
+        logger.info(
+            "Listener subscriptions updated: %s",
+            ", ".join(sorted(channels)) or "(none)",
+        )
+
+    async def _ingest_post(self, channel: str, post) -> None:
+        record = parse_event_post(post)
+        event = event_record_to_ingest(record)
+        job_id = self._channels.get(normalize_channel(channel))
+
+        async with httpx.AsyncClient(timeout=60.0) as http:
+            response = await http.post(
+                f"{CA_API_URL}/internal/ingest",
+                json={
+                    "job_id": job_id,
+                    "events": [event],
+                    "listener": True,
+                },
+                headers={"X-Internal-Token": INTERNAL_TOKEN},
+            )
+            response.raise_for_status()
+            result = response.json()
+            logger.info(
+                "Listener ingest %s: new=%s skipped=%s",
+                post.url,
+                result.get("ingested", 0),
+                result.get("skipped", 0),
+            )
+
+    def _channel_from_event(self, event: events.common.EventCommon) -> str | None:
+        chat = event.chat
+        if chat is None:
+            return None
+        username = getattr(chat, "username", None)
+        if username:
+            return normalize_channel(username)
+        return None
+
+    async def _process_messages(self, event: events.common.EventCommon, messages: list[Any]) -> None:
+        channel = self._channel_from_event(event)
+        if not channel or channel not in self._channels:
+            return
+        if event.chat_id not in self._chat_ids:
+            return
+
+        for message in messages:
+            post = message_to_post(message, channel)
+            if not post:
+                continue
+            try:
+                await self._ingest_post(channel, post)
+            except Exception:
+                logger.exception("Listener ingest failed for %s", post.url)
+
+    def register_handlers(self) -> None:
+        if self._handlers_registered:
+            return
+
+        @self.client.on(events.NewMessage(incoming=True))
+        async def on_new_message(event: events.NewMessage.Event) -> None:
+            if event.grouped_id:
+                return
+            await self._process_messages(event, [event.message])
+
+        @self.client.on(events.Album)
+        async def on_album(event: events.Album.Event) -> None:
+            await self._process_messages(event, list(event.messages))
+
+        self._handlers_registered = True
+        logger.info("Telegram NewMessage/Album handlers registered")
+
+    async def refresh_loop(self) -> None:
+        while True:
+            await self.refresh_subscriptions()
+            await asyncio.sleep(REFRESH_SECONDS)
+
+    async def run(self) -> None:
+        self.register_handlers()
+        await self.refresh_subscriptions()
+        await self.refresh_loop()

centers/parsing/workers/workers/sources/telegram_session.py

@@ -0,0 +1,55 @@
+"""Единое подключение Telethon для listener и batch-заданий."""
+
+from pathlib import Path
+
+from telethon import TelegramClient
+from telethon.errors import AuthKeyUnregisteredError, SessionPasswordNeededError
+
+from workers.sources.telegram_settings import create_client, get_api_credentials, get_session_path
+from workers.sources.telegram_client import TelegramAuthError, TelegramConfigError
+
+_shared_client: TelegramClient | None = None
+
+
+async def get_shared_client() -> TelegramClient:
+    global _shared_client
+    if _shared_client is not None and _shared_client.is_connected():
+        return _shared_client
+
+    try:
+        api_id, api_hash = get_api_credentials()
+    except ValueError as exc:
+        raise TelegramConfigError(str(exc)) from exc
+
+    session_path = get_session_path()
+    if not Path(session_path).exists():
+        raise TelegramAuthError(
+            f"Файл сессии не найден: {session_path}. "
+            "Выполните: python scripts/telegram_auth.py"
+        )
+
+    client = create_client(session_path, api_id, api_hash)
+    try:
+        await client.connect()
+        if not await client.is_user_authorized():
+            raise TelegramAuthError(
+                "Telegram-сессия не авторизована. Выполните: python scripts/telegram_auth.py"
+            )
+    except AuthKeyUnregisteredError as exc:
+        raise TelegramAuthError(
+            "Сессия Telegram недействительна. Переавторизуйтесь: python scripts/telegram_auth.py"
+        ) from exc
+    except SessionPasswordNeededError as exc:
+        raise TelegramAuthError(
+            "Для аккаунта включена 2FA. Авторизуйтесь через scripts/telegram_auth.py."
+        ) from exc
+
+    _shared_client = client
+    return client
+
+
+async def close_shared_client() -> None:
+    global _shared_client
+    if _shared_client is not None:
+        await _shared_client.disconnect()
+        _shared_client = None

docker-compose.yml

@@ -48,6 +48,8 @@ services:
       - .env
     environment:
       TELEGRAM_SESSION_PATH: /data/telegram.session
+      TELEGRAM_LISTENER_ENABLED: "true"
+      TELEGRAM_LISTENER_REFRESH_SECONDS: "60"
       CA_API_URL: http://ca-api:8000
       REDIS_URL: redis://redis:6379/0
       INTERNAL_TOKEN: dev-internal-token