feat(logging): add HTTP page fetch logging and enhance click-TT proxy functionality
- Introduced a new logging mechanism for HTTP requests to click-TT/HTTV pages, improving traceability and debugging capabilities. - Implemented a proxy endpoint for iframe embedding, allowing direct HTML retrieval with enhanced error handling and validation for input URLs. - Updated the frontend to include a new navigation link for the click-TT feature, accessible to admin users. - Added a new route for the click-TT view in the router configuration.
This commit is contained in:
Torsten Schulz (local)
113
backend/CLICKTT_HTTV_README.md
Normal file
113
backend/CLICKTT_HTTV_README.md
Normal file
@@ -0,0 +1,113 @@
|
||||
# HTTV / click-TT HTTP-Seiten – Integration & Logging
|
||||
|
||||
Dieses Modul ermöglicht das Testen und Logging von HTTP-Aufrufen an die click-TT-Seiten verschiedener Tischtennis-Verbände (HTTV, RTTV, WTTV etc.).
|
||||
|
||||
## Zweck
|
||||
|
||||
- **Logging**: Jeder Aufruf wird in `http_page_fetch_log` protokolliert (URL, HTTP-Status, Response-Snippet, Fehler).
|
||||
- **Strukturanalyse**: Die Logs helfen zu verstehen, wie die Seiten je nach Verband und Saison aufgebaut sind.
|
||||
- **URL-Varianten**: Links können je nach Verein, Saison und Verband unterschiedlich sein.
|
||||
|
||||
## Verband → Domain
|
||||
|
||||
| Verband | Domain |
|
||||
|---------|--------|
|
||||
| HeTTV / HTTV | httv.click-tt.de |
|
||||
| RTTV | rttv.click-tt.de |
|
||||
| WTTV | wttv.click-tt.de |
|
||||
| TTVNw | ttvnw.click-tt.de |
|
||||
| BTTV | battv.click-tt.de |
|
||||
|
||||
## URL-Struktur (httv.click-tt.de)
|
||||
|
||||
### leaguePage – Ligenübersicht
|
||||
|
||||
```
|
||||
https://httv.click-tt.de/cgi-bin/WebObjects/nuLigaTTDE.woa/wa/leaguePage?championship=HTTV+25%2F26
|
||||
```
|
||||
|
||||
- `championship`: Saison/Championship, z.B.:
|
||||
- `HTTV 25/26` – Haupt-HTTV-Saison
|
||||
- `K43 25/26` – Bezirk Frankfurt
|
||||
- `K16 25/26` – Bezirk Werra-Meißner
|
||||
- `RL-OL West 25/26` – Regional-/Oberligen West
|
||||
|
||||
### regionMeetingFilter – Regionsspielplan
|
||||
|
||||
```
|
||||
https://httv.click-tt.de/cgi-bin/WebObjects/nuLigaTTDE.woa/wa/regionMeetingFilter?championship=HTTV+25%2F26
|
||||
```
|
||||
|
||||
### clubInfoDisplay – Vereinsinfo
|
||||
|
||||
```
|
||||
https://httv.click-tt.de/cgi-bin/WebObjects/nuLigaTTDE.woa/wa/clubInfoDisplay?club=1060
|
||||
```
|
||||
|
||||
- `club`: Vereins-ID in der click-TT-Datenbank
|
||||
|
||||
## UI-Seite
|
||||
|
||||
Unter **/clicktt** (nur für Admins) gibt es eine Vue-Seite, mit der du:
|
||||
|
||||
- Seitentyp wählen (Ligenübersicht, Vereinsinfo, Regionsspielplan oder direkte URL)
|
||||
- Verband, Championship/Saison und ggf. Vereins-ID eingeben
|
||||
- Die Seite im iframe laden und direkt bedienen (klicken, navigieren)
|
||||
|
||||
Alle Aufrufe werden in `http_page_fetch_log` protokolliert.
|
||||
|
||||
## API-Endpunkte
|
||||
|
||||
Die meisten Endpunkte erfordern Authentifizierung (Token). Der **Proxy** (`/api/clicktt/proxy`) ist ohne Auth nutzbar (für iframe-Einbettung).
|
||||
|
||||
### Ligenübersicht abrufen
|
||||
|
||||
```
|
||||
GET /api/clicktt/league-page?association=HeTTV&championship=HTTV+25%2F26
|
||||
```
|
||||
|
||||
### Vereinsinfo abrufen
|
||||
|
||||
```
|
||||
GET /api/clicktt/club-info?association=HeTTV&clubId=1060
|
||||
```
|
||||
|
||||
### Regionsspielplan abrufen
|
||||
|
||||
```
|
||||
GET /api/clicktt/region-meetings?association=HeTTV&championship=HTTV+25%2F26
|
||||
```
|
||||
|
||||
### Beliebige URL abrufen (nur click-tt.de / httv.de)
|
||||
|
||||
```
|
||||
GET /api/clicktt/fetch?url=https%3A%2F%2Fhttv.click-tt.de%2Fcgi-bin%2F...
|
||||
```
|
||||
|
||||
### Logs abrufen
|
||||
|
||||
```
|
||||
GET /api/clicktt/logs?limit=50&fetchType=leaguePage&association=HeTTV
|
||||
```
|
||||
|
||||
### URL-Info (Beispiele, Verband→Domain)
|
||||
|
||||
```
|
||||
GET /api/clicktt/url-info
|
||||
```
|
||||
|
||||
## Datenbank-Migration
|
||||
|
||||
```bash
|
||||
mysql -u USER -p DATABASE < backend/migrations/create_http_page_fetch_log.sql
|
||||
```
|
||||
|
||||
## Hinweis: mytischtennis.de vs. click-TT
|
||||
|
||||
Die **leaguePage** auf httv.click-tt.de zeigt eine Übersicht mit Links. Die eigentlichen **Tabellen und Spielpläne** verweisen auf **mytischtennis.de**:
|
||||
|
||||
```
|
||||
https://www.mytischtennis.de/click-tt/HeTTV/25--26/ligen/Hessenliga_Gr._Süd-West/gruppe/496273/tabelle/gesamt
|
||||
```
|
||||
|
||||
Diese mytischtennis.de-URLs werden bereits über den bestehenden MyTischtennis-URL-Parser und Auto-Fetch unterstützt. Die httv.click-tt.de-Seiten dienen vor allem der Navigation und der Ermittlung von Gruppen-IDs für verschiedene Bezirke/Saisonen.
|
||||
@@ -72,6 +72,7 @@
|
||||
|
||||
## API & Logging
|
||||
51. `api_log` - API-Logs
|
||||
52. `http_page_fetch_log` - HTTP-Aufrufe an click-TT/HTTV-Seiten (Logging)
|
||||
|
||||
## Gesamt: 51 Tabellen
|
||||
## Gesamt: 52 Tabellen
|
||||
|
||||
|
||||
@@ -6,11 +6,82 @@
|
||||
import express from 'express';
|
||||
import clickTtHttpPageService from '../services/clickTtHttpPageService.js';
|
||||
import HttpPageFetchLog from '../models/HttpPageFetchLog.js';
|
||||
import { authenticate } from '../middleware/authenticate.js';
|
||||
import { Op } from 'sequelize';
|
||||
import { authenticate } from '../middleware/authMiddleware.js';
|
||||
|
||||
const router = express.Router();
|
||||
|
||||
/**
|
||||
* GET /api/clicktt/proxy
|
||||
* Proxy für iframe-Einbettung – liefert HTML direkt (ohne Auth, für iframe src).
|
||||
* Query: type (leaguePage|clubInfo|regionMeetings), association, championship, clubId
|
||||
* ODER: url (vollständige URL, nur click-tt.de/httv.de)
|
||||
*/
|
||||
router.get('/proxy', async (req, res, next) => {
|
||||
try {
|
||||
const { type, association, championship, clubId, url } = req.query;
|
||||
let targetUrl = null;
|
||||
let fetchType = 'proxy';
|
||||
|
||||
if (url) {
|
||||
if (!url.includes('click-tt.de') && !url.includes('httv.de')) {
|
||||
return res.status(400).send('<html><body><h1>Fehler</h1><p>Nur URLs von click-tt.de oder httv.de sind erlaubt.</p></body></html>');
|
||||
}
|
||||
targetUrl = url;
|
||||
fetchType = 'arbitrary';
|
||||
} else if (type === 'leaguePage') {
|
||||
targetUrl = clickTtHttpPageService.buildLeaguePageUrl({
|
||||
association: association || 'HeTTV',
|
||||
championship: championship || 'HTTV 25/26',
|
||||
});
|
||||
fetchType = 'leaguePage';
|
||||
} else if (type === 'clubInfo' && clubId) {
|
||||
targetUrl = clickTtHttpPageService.buildClubInfoDisplayUrl({
|
||||
association: association || 'HeTTV',
|
||||
clubId,
|
||||
});
|
||||
fetchType = 'clubInfoDisplay';
|
||||
} else if (type === 'regionMeetings') {
|
||||
targetUrl = clickTtHttpPageService.buildRegionMeetingFilterUrl({
|
||||
association: association || 'HeTTV',
|
||||
championship: championship || 'HTTV 25/26',
|
||||
});
|
||||
fetchType = 'regionMeetingFilter';
|
||||
} else {
|
||||
return res.status(400).send('<html><body><h1>Fehler</h1><p>Parameter type (leaguePage|clubInfo|regionMeetings) oder url erforderlich. Bei clubInfo ist clubId nötig.</p></body></html>');
|
||||
}
|
||||
|
||||
const result = await clickTtHttpPageService.fetchWithLogging({
|
||||
url: targetUrl,
|
||||
fetchType,
|
||||
association: association || null,
|
||||
championship: championship || null,
|
||||
clubIdParam: clubId || null,
|
||||
userId: null,
|
||||
});
|
||||
|
||||
let html = result.body;
|
||||
// CSP/X-Frame-Options entfernen für iframe-Einbettung
|
||||
html = (html || '')
|
||||
.replace(/<meta[^>]*http-equiv=["']content-security-policy["'][^>]*>/gi, '')
|
||||
.replace(/<meta[^>]*http-equiv=["']x-frame-options["'][^>]*>/gi, '')
|
||||
.replace(/<meta[^>]*http-equiv=["']x-content-type-options["'][^>]*>/gi, '');
|
||||
|
||||
res.set({
|
||||
'Content-Type': 'text/html; charset=utf-8',
|
||||
'Access-Control-Allow-Origin': '*',
|
||||
'X-Frame-Options': 'ALLOWALL',
|
||||
'Content-Security-Policy': 'frame-ancestors *;',
|
||||
'Cache-Control': 'no-cache, no-store, must-revalidate',
|
||||
});
|
||||
res.send(html);
|
||||
} catch (error) {
|
||||
console.error('ClickTT Proxy Fehler:', error);
|
||||
res.status(500).send(
|
||||
`<html><body><h1>Fehler beim Laden</h1><p>${String(error.message)}</p></body></html>`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* GET /api/clicktt/league-page
|
||||
* Ruft die Ligenübersicht ab (leaguePage)
|
||||
|
||||
Reference in New Issue
Block a user