(vision) SCS/DDD
Service Discovery einführen Consul als Service-Registry implementieren Services für automatische Registrierung konfigurieren Dynamisches Service-Routing im API-Gateway einrichten Health-Checks für jeden Service implementieren
This commit is contained in:
parent
3371b241df
commit
1ecac43d72
264
BETRIEBSANLEITUNG.md
Normal file
264
BETRIEBSANLEITUNG.md
Normal file
|
|
@ -0,0 +1,264 @@
|
|||
# Betriebsanleitung für das Meldestelle-Projekt
|
||||
|
||||
Diese Betriebsanleitung beschreibt, wie Sie das Meldestelle-Projekt einrichten und ausführen können.
|
||||
|
||||
## Inhaltsverzeichnis
|
||||
|
||||
1. [Projektübersicht](#projektübersicht)
|
||||
2. [Voraussetzungen](#voraussetzungen)
|
||||
3. [Installation](#installation)
|
||||
4. [Konfiguration](#konfiguration)
|
||||
5. [Ausführung](#ausführung)
|
||||
6. [Zugriff auf die Anwendung](#zugriff-auf-die-anwendung)
|
||||
7. [Monitoring und Wartung](#monitoring-und-wartung)
|
||||
8. [Fehlerbehebung](#fehlerbehebung)
|
||||
|
||||
## Projektübersicht
|
||||
|
||||
Das Meldestelle-Projekt ist ein Kotlin JVM Backend-Projekt, das eine Self-Contained Systems (SCS) Architektur für ein Pferdesport-Managementsystem implementiert. Es folgt den Prinzipien des Domain-Driven Design (DDD) mit klar getrennten Bounded Contexts.
|
||||
|
||||
### Module
|
||||
|
||||
- **shared-kernel**: Gemeinsame Domänentypen, Enums, Serialisierer, Validierungsdienstprogramme und Basis-DTOs
|
||||
- **master-data**: Stammdatenverwaltung (Länder, Regionen, Altersklassen, Veranstaltungsorte)
|
||||
- **member-management**: Personen- und Vereins-/Verbandsverwaltung
|
||||
- **horse-registry**: Pferderegistrierung und -verwaltung
|
||||
- **event-management**: Veranstaltungs- und Turnierverwaltung
|
||||
- **api-gateway**: Zentrales API-Gateway, das alle Dienste aggregiert
|
||||
- **composeApp**: Frontend-Modul
|
||||
|
||||
### Technologie-Stack
|
||||
|
||||
- **Kotlin JVM**: Primäre Programmiersprache
|
||||
- **Ktor**: Web-Framework für REST-APIs
|
||||
- **Exposed**: Datenbank-ORM
|
||||
- **PostgreSQL**: Datenbank
|
||||
- **Consul**: Service-Discovery und -Registry
|
||||
- **Kotlinx Serialization**: JSON-Serialisierung
|
||||
- **Gradle**: Build-System
|
||||
- **Docker**: Containerisierung
|
||||
|
||||
## Voraussetzungen
|
||||
|
||||
Um das Projekt auszuführen, benötigen Sie:
|
||||
|
||||
### Für die lokale Entwicklung
|
||||
|
||||
- JDK 21 oder höher
|
||||
- Gradle 8.14 oder höher
|
||||
- PostgreSQL 16
|
||||
- Docker und Docker Compose (für containerisierte Ausführung)
|
||||
- Git (für den Quellcode-Zugriff)
|
||||
|
||||
### Für die containerisierte Ausführung
|
||||
|
||||
- Docker Engine 24.0 oder höher
|
||||
- Docker Compose V2 oder höher
|
||||
|
||||
## Installation
|
||||
|
||||
### Quellcode herunterladen
|
||||
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd meldestelle
|
||||
```
|
||||
|
||||
### Umgebungsvariablen einrichten
|
||||
|
||||
Erstellen Sie eine `.env`-Datei im Stammverzeichnis des Projekts mit den folgenden Umgebungsvariablen:
|
||||
|
||||
```
|
||||
# Postgres-Konfiguration
|
||||
POSTGRES_USER=meldestelle_user
|
||||
POSTGRES_PASSWORD=secure_password_change_me
|
||||
POSTGRES_DB=meldestelle_db
|
||||
POSTGRES_SHARED_BUFFERS=256MB
|
||||
POSTGRES_EFFECTIVE_CACHE_SIZE=768MB
|
||||
POSTGRES_WORK_MEM=16MB
|
||||
POSTGRES_MAINTENANCE_WORK_MEM=64MB
|
||||
POSTGRES_MAX_CONNECTIONS=100
|
||||
|
||||
# PgAdmin-Konfiguration
|
||||
PGADMIN_DEFAULT_EMAIL=admin@example.com
|
||||
PGADMIN_DEFAULT_PASSWORD=admin_password_change_me
|
||||
PGADMIN_PORT=127.0.0.1:5050
|
||||
|
||||
# Grafana-Konfiguration
|
||||
GRAFANA_ADMIN_USER=admin
|
||||
GRAFANA_ADMIN_PASSWORD=admin
|
||||
```
|
||||
|
||||
**Wichtig**: Ändern Sie die Passwörter für eine Produktionsumgebung!
|
||||
|
||||
## Konfiguration
|
||||
|
||||
Das Projekt verwendet einen mehrschichtigen Konfigurationsansatz, um verschiedene Umgebungen zu unterstützen:
|
||||
|
||||
1. Umgebungsvariablen (höchste Priorität)
|
||||
2. Umgebungsspezifische Konfigurationsdateien (.properties)
|
||||
3. Basis-Konfigurationsdatei (application.properties)
|
||||
4. Standardwerte im Code (niedrigste Priorität)
|
||||
|
||||
### Umgebungen
|
||||
|
||||
Das Projekt unterstützt folgende Umgebungen:
|
||||
|
||||
| Umgebung | Beschreibung | Typische Verwendung |
|
||||
|--------------|-----------------------------|--------------------------------------------|
|
||||
| DEVELOPMENT | Lokale Entwicklungsumgebung | Lokale Entwicklung, Debug-Modus aktiv |
|
||||
| TEST | Testumgebung | Automatisierte Tests, Integrationstests |
|
||||
| STAGING | Vorabproduktionsumgebung | Manuelle Tests, UAT, Demos |
|
||||
| PRODUCTION | Produktionsumgebung | Live-System |
|
||||
|
||||
Die aktuelle Umgebung wird über die Umgebungsvariable `APP_ENV` festgelegt. Wenn diese Variable nicht gesetzt ist, wird standardmäßig `DEVELOPMENT` verwendet.
|
||||
|
||||
### Konfigurationsdateien
|
||||
|
||||
Die Konfigurationsdateien befinden sich im `/config`-Verzeichnis:
|
||||
|
||||
- `application.properties`: Basiseinstellungen für alle Umgebungen
|
||||
- `application-dev.properties`: Entwicklungsumgebung
|
||||
- `application-test.properties`: Testumgebung
|
||||
- `application-staging.properties`: Staging-Umgebung
|
||||
- `application-prod.properties`: Produktionsumgebung
|
||||
|
||||
## Ausführung
|
||||
|
||||
### Methode 1: Mit Docker Compose (empfohlen)
|
||||
|
||||
Diese Methode startet alle erforderlichen Dienste in Containern.
|
||||
|
||||
1. Stellen Sie sicher, dass Docker und Docker Compose installiert sind
|
||||
2. Stellen Sie sicher, dass die `.env`-Datei konfiguriert ist
|
||||
3. Führen Sie den folgenden Befehl aus:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Um die Logs zu überwachen:
|
||||
|
||||
```bash
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
Um die Dienste zu stoppen:
|
||||
|
||||
```bash
|
||||
docker compose down
|
||||
```
|
||||
|
||||
Um die Dienste zu stoppen und alle Daten zu löschen:
|
||||
|
||||
```bash
|
||||
docker compose down -v
|
||||
```
|
||||
|
||||
### Methode 2: Lokale Entwicklung mit Gradle
|
||||
|
||||
Diese Methode ist für die Entwicklung gedacht und erfordert eine lokale PostgreSQL-Datenbank.
|
||||
|
||||
1. Stellen Sie sicher, dass JDK 21 oder höher installiert ist
|
||||
2. Stellen Sie sicher, dass PostgreSQL installiert und konfiguriert ist
|
||||
3. Konfigurieren Sie die Datenbankverbindung in `config/application-dev.properties`
|
||||
4. Bauen Sie das Projekt:
|
||||
|
||||
```bash
|
||||
./gradlew build
|
||||
```
|
||||
|
||||
5. Starten Sie das API-Gateway:
|
||||
|
||||
```bash
|
||||
./gradlew :api-gateway:jvmRun
|
||||
```
|
||||
|
||||
## Zugriff auf die Anwendung
|
||||
|
||||
Nach dem Start der Anwendung können Sie auf folgende Dienste zugreifen:
|
||||
|
||||
- **API-Gateway**: http://localhost:8080
|
||||
- **API-Dokumentation**: http://localhost:8080/docs
|
||||
- **Swagger UI**: http://localhost:8080/swagger
|
||||
- **OpenAPI-Spezifikation**: http://localhost:8080/openapi
|
||||
- **PgAdmin**: http://localhost:5050
|
||||
- **Consul UI**: http://localhost:8500
|
||||
- **Prometheus**: http://localhost:9090
|
||||
- **Grafana**: http://localhost:3000
|
||||
- **Kibana**: http://localhost:5601
|
||||
|
||||
### API-Dokumentation
|
||||
|
||||
Die API-Dokumentation umfasst alle Bounded Contexts:
|
||||
- Authentication API
|
||||
- Master Data API
|
||||
- Member Management API
|
||||
- Horse Registry API
|
||||
- Event Management API
|
||||
|
||||
## Monitoring und Wartung
|
||||
|
||||
### Monitoring-Stack
|
||||
|
||||
Das Projekt enthält einen vollständigen Monitoring-Stack:
|
||||
|
||||
- **Prometheus**: Metriken-Sammlung und -Speicherung
|
||||
- **Grafana**: Visualisierung von Metriken und Dashboards
|
||||
- **Alertmanager**: Benachrichtigungen bei Problemen
|
||||
- **ELK-Stack**: Elasticsearch, Logstash und Kibana für Logging
|
||||
|
||||
### Service Discovery
|
||||
|
||||
Das Projekt verwendet Consul für Service Discovery, wodurch Dienste sich dynamisch entdecken und miteinander kommunizieren können, ohne fest codierte Endpunkte zu verwenden. Dies macht das System widerstandsfähiger und skalierbarer.
|
||||
|
||||
- **Consul UI**: Zugriff auf die Consul-UI unter http://localhost:8500
|
||||
|
||||
## Fehlerbehebung
|
||||
|
||||
### Häufige Probleme
|
||||
|
||||
1. **Dienste starten nicht**
|
||||
- Überprüfen Sie die Docker-Logs: `docker-compose logs -f <service-name>`
|
||||
- Stellen Sie sicher, dass alle erforderlichen Ports verfügbar sind
|
||||
- Überprüfen Sie die Umgebungsvariablen in der `.env`-Datei
|
||||
|
||||
2. **Fehler bei Docker Compose Abhängigkeiten**
|
||||
- Wenn Sie eine Fehlermeldung wie `service "X" depends on undefined service "Y"` erhalten, überprüfen Sie die `depends_on`-Einträge in der docker-compose.yml
|
||||
- Stellen Sie sicher, dass alle referenzierten Dienste korrekt definiert sind
|
||||
- Für Dienste, die über Hostnamen kommunizieren, können Sie Netzwerk-Aliase verwenden:
|
||||
```yaml
|
||||
services:
|
||||
api-gateway:
|
||||
networks:
|
||||
meldestelle-net:
|
||||
aliases:
|
||||
- server
|
||||
```
|
||||
- Stellen Sie sicher, dass alle verwendeten Volumes in der `volumes`-Sektion definiert sind
|
||||
|
||||
3. **Datenbankverbindungsprobleme**
|
||||
- Überprüfen Sie, ob die PostgreSQL-Datenbank läuft: `docker-compose ps db`
|
||||
- Überprüfen Sie die Datenbankverbindungseinstellungen
|
||||
- Überprüfen Sie die Datenbank-Logs: `docker-compose logs -f db`
|
||||
|
||||
4. **API-Gateway ist nicht erreichbar**
|
||||
- Überprüfen Sie, ob der API-Gateway-Dienst läuft: `docker-compose ps api-gateway`
|
||||
- Überprüfen Sie die API-Gateway-Logs: `docker-compose logs -f api-gateway`
|
||||
- Stellen Sie sicher, dass Port 8080 nicht von einem anderen Dienst verwendet wird
|
||||
|
||||
5. **PostgreSQL SSL-Konfigurationsprobleme**
|
||||
- Wenn die Datenbank mit der Fehlermeldung `FATAL: could not load server certificate file "server.crt": No such file or directory` nicht startet, ist SSL aktiviert, aber die erforderlichen Zertifikatsdateien fehlen
|
||||
- Lösungen:
|
||||
- Option 1: Deaktivieren Sie SSL in der PostgreSQL-Konfiguration (`config/postgres/postgresql.conf`), indem Sie `ssl = off` setzen
|
||||
- Option 2: Stellen Sie die erforderlichen SSL-Zertifikatsdateien (server.crt, server.key) bereit und mounten Sie sie im Container
|
||||
- Für Entwicklungsumgebungen ist Option 1 (SSL deaktivieren) in der Regel ausreichend
|
||||
- Für Produktionsumgebungen sollten Sie Option 2 (SSL-Zertifikate bereitstellen) in Betracht ziehen, um die Datenbankverbindungen zu sichern
|
||||
|
||||
### Support
|
||||
|
||||
Bei weiteren Problemen wenden Sie sich bitte an das Entwicklungsteam oder erstellen Sie ein Issue im Repository.
|
||||
|
||||
---
|
||||
|
||||
Letzte Aktualisierung: 2025-07-21
|
||||
65
Dockerfile
65
Dockerfile
|
|
@ -1,15 +1,70 @@
|
|||
# ----------- Stage 1: Build Stage -----------
|
||||
FROM gradle:8.13-jdk21 AS build
|
||||
WORKDIR /home/gradle/src
|
||||
|
||||
# Copy only the files needed for dependency resolution first
|
||||
# This improves caching of dependencies
|
||||
COPY build.gradle.kts settings.gradle.kts gradle.properties ./
|
||||
COPY gradle ./gradle
|
||||
COPY shared ./shared
|
||||
|
||||
# Download dependencies and cache them
|
||||
RUN gradle dependencies --no-daemon
|
||||
|
||||
# Copy source code
|
||||
COPY shared-kernel ./shared-kernel
|
||||
COPY api-gateway ./api-gateway
|
||||
COPY master-data ./master-data
|
||||
COPY member-management ./member-management
|
||||
COPY horse-registry ./horse-registry
|
||||
COPY event-management ./event-management
|
||||
COPY composeApp ./composeApp
|
||||
COPY server ./server
|
||||
RUN gradle :server:shadowJar --no-configure-on-demand
|
||||
|
||||
# Build with optimized settings
|
||||
RUN gradle :api-gateway:shadowJar --no-daemon --parallel --build-cache
|
||||
|
||||
# ----------- Stage 2: Runtime Stage -----------
|
||||
FROM openjdk:21-slim-bookworm AS runtime
|
||||
|
||||
# Add non-root user for security
|
||||
RUN addgroup --system --gid 1001 appuser && \
|
||||
adduser --system --uid 1001 --gid 1001 appuser
|
||||
|
||||
# Set timezone
|
||||
ENV TZ=Europe/Vienna
|
||||
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=build /home/gradle/src/server/build/libs/*.jar ./app.jar
|
||||
EXPOSE 8080
|
||||
ENTRYPOINT ["java", "-jar", "/app/app.jar"]
|
||||
|
||||
# Copy the jar file from the build stage
|
||||
COPY --from=build /home/gradle/src/api-gateway/build/libs/*.jar ./app.jar
|
||||
|
||||
# Set ownership to non-root user
|
||||
RUN chown -R appuser:appuser /app
|
||||
|
||||
# Switch to non-root user
|
||||
USER appuser
|
||||
|
||||
# Add metadata labels
|
||||
LABEL org.opencontainers.image.title="Meldestelle API Gateway"
|
||||
LABEL org.opencontainers.image.description="API Gateway for Meldestelle application"
|
||||
LABEL org.opencontainers.image.vendor="MoCode"
|
||||
LABEL org.opencontainers.image.version="1.0.0"
|
||||
LABEL org.opencontainers.image.created="2025-07-21"
|
||||
|
||||
# Expose the application port
|
||||
EXPOSE 8081
|
||||
|
||||
# Define health check
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
|
||||
CMD curl -f http://localhost:8081/health || exit 1
|
||||
|
||||
# Run the application with optimized JVM settings
|
||||
ENTRYPOINT ["java", \
|
||||
"-XX:+UseG1GC", \
|
||||
"-XX:MaxGCPauseMillis=100", \
|
||||
"-XX:+ParallelRefProcEnabled", \
|
||||
"-XX:+HeapDumpOnOutOfMemoryError", \
|
||||
"-XX:HeapDumpPath=/tmp/heapdump.hprof", \
|
||||
"-Djava.security.egd=file:/dev/./urandom", \
|
||||
"-jar", "/app/app.jar"]
|
||||
|
|
|
|||
70
OPTIMIZATION_IMPLEMENTATION_SUMMARY.md
Normal file
70
OPTIMIZATION_IMPLEMENTATION_SUMMARY.md
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
# Optimization Implementation Summary
|
||||
|
||||
This document summarizes the optimizations implemented in the Meldestelle project to improve performance, resource utilization, and maintainability.
|
||||
|
||||
## Implemented Optimizations
|
||||
|
||||
### 1. Caching Strategy Improvements
|
||||
|
||||
#### 1.1 Enhanced In-Memory Caching
|
||||
|
||||
The `CachingConfig.kt` implementation has been enhanced with:
|
||||
|
||||
- **Optimized in-memory caching** with proper expiration handling to prevent memory leaks
|
||||
- **Cache statistics tracking** for monitoring cache effectiveness (hits, misses, puts, evictions)
|
||||
- **Periodic cache cleanup** scheduled every 10 minutes to remove expired entries
|
||||
- **Proper resource management** with shutdown handling to release resources
|
||||
- **Preparation for Redis integration** with configuration parameters for future implementation
|
||||
|
||||
These improvements provide a more robust and maintainable caching solution that can be easily monitored and extended.
|
||||
|
||||
#### 1.2 HTTP Caching Enhancements
|
||||
|
||||
The `HttpCaching.kt` implementation has been enhanced with:
|
||||
|
||||
- **ETag generation** for efficient client-side caching using MD5 hashing
|
||||
- **Conditional request handling** for If-None-Match and If-Modified-Since headers
|
||||
- **Integration with in-memory caching** for a multi-level caching approach
|
||||
- **Utility functions for different caching scenarios**:
|
||||
- Static resources (CSS, JS, images) with long TTL
|
||||
- Master data (reference data) with medium TTL
|
||||
- User data with short TTL
|
||||
- Sensitive data with no caching
|
||||
|
||||
These enhancements improve the efficiency of client-server communication by reducing unnecessary data transfer when resources haven't changed.
|
||||
|
||||
### 2. Documentation Updates
|
||||
|
||||
- **Updated OPTIMIZATION_SUMMARY.md** with details of the implemented caching optimizations
|
||||
- **Updated OPTIMIZATION_RECOMMENDATIONS.md** with recommendations for future caching enhancements
|
||||
- **Created OPTIMIZATION_IMPLEMENTATION_SUMMARY.md** (this document) to summarize the implemented changes
|
||||
|
||||
## Benefits of Implemented Optimizations
|
||||
|
||||
1. **Reduced Server Load**: By implementing proper caching, the server can avoid regenerating or retrieving the same data repeatedly.
|
||||
|
||||
2. **Improved Response Times**: Cached responses can be served much faster than generating them from scratch.
|
||||
|
||||
3. **Reduced Network Traffic**: HTTP caching with ETags and conditional requests reduces the amount of data transferred over the network.
|
||||
|
||||
4. **Better Resource Utilization**: Proper cache expiration and cleanup prevent memory leaks and ensure efficient resource usage.
|
||||
|
||||
5. **Enhanced Monitoring**: Cache statistics provide insights into cache effectiveness and help identify optimization opportunities.
|
||||
|
||||
## Next Steps
|
||||
|
||||
The following steps are recommended to further enhance the project:
|
||||
|
||||
1. **Complete Redis Integration**: Implement the Redis integration using the Redisson library to enable distributed caching.
|
||||
|
||||
2. **Implement Multi-Level Caching**: Use Caffeine for local in-memory caching and Redis for distributed caching.
|
||||
|
||||
3. **Enhance Asynchronous Processing**: Identify long-running operations and implement asynchronous processing to improve responsiveness.
|
||||
|
||||
4. **Improve Security Measures**: Implement dependency vulnerability scanning and container image scanning.
|
||||
|
||||
5. **Enhance Monitoring and Observability**: Implement distributed tracing with OpenTelemetry and add business metrics for key operations.
|
||||
|
||||
## Conclusion
|
||||
|
||||
The implemented optimizations provide a solid foundation for a high-performance, scalable application. The caching strategy improvements in particular will help reduce server load, improve response times, and enhance the overall user experience. The next steps outlined above will further enhance the application's performance, security, and observability.
|
||||
186
OPTIMIZATION_RECOMMENDATIONS.md
Normal file
186
OPTIMIZATION_RECOMMENDATIONS.md
Normal file
|
|
@ -0,0 +1,186 @@
|
|||
# Optimization Recommendations for Meldestelle Project
|
||||
|
||||
This document outlines recommendations for further optimizations and improvements to the Meldestelle project. These recommendations are based on the analysis of the project's architecture, code, and configuration.
|
||||
|
||||
## Implemented Optimizations
|
||||
|
||||
The following optimizations have already been implemented:
|
||||
|
||||
### Database Optimizations
|
||||
- Added minimum pool size configuration to prevent connection establishment overhead
|
||||
- Optimized transaction isolation level from REPEATABLE_READ to READ_COMMITTED for better performance
|
||||
- Added statement cache configuration to improve prepared statement reuse
|
||||
- Added connection initialization SQL to warm up connections
|
||||
- Separated PostgreSQL WAL files to a dedicated volume for better I/O performance
|
||||
- Created optimized PostgreSQL configuration file with tuned settings
|
||||
|
||||
### Monitoring Optimizations
|
||||
- Optimized log sampling mechanism with better thread management and error handling
|
||||
- Reduced memory usage metrics calculation frequency to only 10% of log entries
|
||||
- Optimized string building in structured logging with StringBuilder and estimated capacity
|
||||
- Improved shouldLogRequest method with early returns and better path normalization
|
||||
|
||||
### Build and Deployment Optimizations
|
||||
- Increased JVM heap size for Gradle and Kotlin daemons
|
||||
- Added JVM optimization flags for better performance
|
||||
- Enabled dependency locking for reproducible builds
|
||||
- Added resource limits and reservations for Docker containers
|
||||
- Added health checks for services
|
||||
- Configured JVM options for the server container
|
||||
|
||||
## Recommendations for Further Improvements
|
||||
|
||||
### 1. Architecture Improvements
|
||||
|
||||
#### 1.1 Service Mesh Implementation
|
||||
Consider implementing a service mesh like Istio or Linkerd to handle service-to-service communication, traffic management, security, and observability.
|
||||
|
||||
**Benefits:**
|
||||
- Improved resilience with circuit breaking and retry mechanisms
|
||||
- Enhanced security with mutual TLS
|
||||
- Better observability with distributed tracing
|
||||
- Traffic management capabilities like canary deployments
|
||||
|
||||
#### 1.2 API Gateway Enhancement
|
||||
Enhance the API Gateway with more advanced features:
|
||||
|
||||
**Recommendations:**
|
||||
- Implement request rate limiting per user/client
|
||||
- Add circuit breakers for downstream services
|
||||
- Implement request validation at the gateway level
|
||||
- Consider using a dedicated API Gateway solution like Kong or Traefik
|
||||
|
||||
#### 1.3 Event-Driven Architecture
|
||||
Consider moving towards a more event-driven architecture for better scalability and decoupling:
|
||||
|
||||
**Recommendations:**
|
||||
- Implement a message broker (RabbitMQ, Kafka) for asynchronous communication
|
||||
- Use the outbox pattern for reliable event publishing
|
||||
- Implement event sourcing for critical business domains
|
||||
|
||||
### 2. Performance Optimizations
|
||||
|
||||
#### 2.1 Caching Strategy
|
||||
Further enhance the implemented caching strategy:
|
||||
|
||||
**Recommendations:**
|
||||
- Complete Redis integration in CachingConfig.kt using the Redisson library
|
||||
- Implement a multi-level caching strategy with Caffeine for local caching and Redis for distributed caching
|
||||
- Add cache warming mechanisms for frequently accessed data
|
||||
- Implement cache invalidation strategies for data consistency
|
||||
- Add cache metrics to Prometheus for monitoring cache hit rates and performance
|
||||
- Consider implementing content-based cache keys for more efficient caching
|
||||
- Add support for cache partitioning based on user or tenant for multi-tenant scenarios
|
||||
|
||||
#### 2.2 Database Optimizations
|
||||
Further optimize database usage:
|
||||
|
||||
**Recommendations:**
|
||||
- Implement database read replicas for scaling read operations
|
||||
- Add database partitioning for large tables
|
||||
- Implement query optimization with proper indexing strategy
|
||||
- Consider using materialized views for complex reporting queries
|
||||
|
||||
#### 2.3 Asynchronous Processing
|
||||
Move appropriate operations to asynchronous processing:
|
||||
|
||||
**Recommendations:**
|
||||
- Identify long-running operations and make them asynchronous
|
||||
- Implement a task queue for background processing
|
||||
- Use coroutines more extensively for non-blocking operations
|
||||
- Consider implementing reactive programming patterns
|
||||
|
||||
### 3. Maintainability Enhancements
|
||||
|
||||
#### 3.1 Testing Improvements
|
||||
Enhance the testing strategy:
|
||||
|
||||
**Recommendations:**
|
||||
- Increase unit test coverage to at least 80%
|
||||
- Implement integration tests for critical paths
|
||||
- Add performance tests with defined SLAs
|
||||
- Implement contract testing between services
|
||||
- Set up continuous performance testing in CI/CD pipeline
|
||||
|
||||
#### 3.2 Documentation
|
||||
Improve documentation:
|
||||
|
||||
**Recommendations:**
|
||||
- Generate API documentation automatically from code
|
||||
- Create architectural decision records (ADRs)
|
||||
- Document data models and relationships
|
||||
- Create runbooks for common operational tasks
|
||||
|
||||
#### 3.3 Code Quality
|
||||
Enhance code quality:
|
||||
|
||||
**Recommendations:**
|
||||
- Implement static code analysis in CI/CD pipeline
|
||||
- Enforce consistent coding style with detekt or ktlint
|
||||
- Implement code reviews with defined criteria
|
||||
- Consider using a monorepo tool like Nx or Gradle composite builds
|
||||
|
||||
### 4. Security Enhancements
|
||||
|
||||
#### 4.1 Security Scanning
|
||||
Implement security scanning:
|
||||
|
||||
**Recommendations:**
|
||||
- Add dependency vulnerability scanning
|
||||
- Implement container image scanning
|
||||
- Add static application security testing (SAST)
|
||||
- Consider dynamic application security testing (DAST)
|
||||
|
||||
#### 4.2 Authentication and Authorization
|
||||
Enhance authentication and authorization:
|
||||
|
||||
**Recommendations:**
|
||||
- Implement OAuth2/OpenID Connect with a dedicated identity provider
|
||||
- Use fine-grained authorization with attribute-based access control
|
||||
- Implement API key rotation
|
||||
- Consider using a dedicated authorization service
|
||||
|
||||
### 5. Monitoring and Observability
|
||||
|
||||
#### 5.1 Distributed Tracing
|
||||
Implement distributed tracing:
|
||||
|
||||
**Recommendations:**
|
||||
- Add OpenTelemetry instrumentation
|
||||
- Implement trace context propagation across services
|
||||
- Set up Jaeger or Zipkin for trace visualization
|
||||
- Add custom spans for critical business operations
|
||||
|
||||
#### 5.2 Enhanced Metrics
|
||||
Enhance metrics collection:
|
||||
|
||||
**Recommendations:**
|
||||
- Add business metrics for key operations
|
||||
- Implement SLO/SLI monitoring
|
||||
- Add custom dashboards for different stakeholders
|
||||
- Implement anomaly detection
|
||||
|
||||
## Implementation Priority
|
||||
|
||||
The following is a suggested priority order for implementing these recommendations:
|
||||
|
||||
1. **High Priority (Next 1-3 months)**
|
||||
- Caching strategy implementation
|
||||
- Testing improvements
|
||||
- Security scanning
|
||||
|
||||
2. **Medium Priority (Next 3-6 months)**
|
||||
- Asynchronous processing
|
||||
- Distributed tracing
|
||||
- Enhanced metrics
|
||||
- Documentation improvements
|
||||
|
||||
3. **Long-term (6+ months)**
|
||||
- Service mesh implementation
|
||||
- Event-driven architecture
|
||||
- API Gateway enhancement
|
||||
- Advanced database optimizations
|
||||
|
||||
## Conclusion
|
||||
|
||||
The Meldestelle project has a solid foundation with the current optimizations. Implementing these additional recommendations will further enhance performance, maintainability, and security, ensuring the application can scale and evolve to meet future requirements.
|
||||
291
OPTIMIZATION_SUMMARY.md
Normal file
291
OPTIMIZATION_SUMMARY.md
Normal file
|
|
@ -0,0 +1,291 @@
|
|||
# Meldestelle Project Optimization Summary
|
||||
|
||||
This document summarizes the optimizations implemented in the Meldestelle project to improve performance, resource utilization, and maintainability.
|
||||
|
||||
## Overview
|
||||
|
||||
The Meldestelle project has been optimized in several key areas:
|
||||
|
||||
1. **Database Configuration**: Improved connection pooling and query performance
|
||||
2. **Monitoring System**: Enhanced logging and metrics collection efficiency
|
||||
3. **Build System**: Optimized Gradle configuration for faster builds
|
||||
4. **Deployment Configuration**: Added resource limits and health checks for better container management
|
||||
5. **PostgreSQL Configuration**: Created optimized database settings for better performance
|
||||
|
||||
## Detailed Optimizations
|
||||
|
||||
### 1. Caching Optimizations
|
||||
|
||||
#### 1.1 Enhanced In-Memory Caching
|
||||
|
||||
Improved `CachingConfig.kt` with:
|
||||
|
||||
- Optimized in-memory caching with proper expiration handling
|
||||
- Added cache statistics tracking for monitoring cache effectiveness
|
||||
- Implemented periodic cache cleanup to prevent memory leaks
|
||||
- Added proper shutdown handling for resource cleanup
|
||||
- Prepared for Redis integration with configuration parameters
|
||||
|
||||
#### 1.2 HTTP Caching Enhancements
|
||||
|
||||
Enhanced `HttpCaching.kt` with:
|
||||
|
||||
- Added ETag generation for efficient client-side caching
|
||||
- Implemented conditional request handling (If-None-Match, If-Modified-Since)
|
||||
- Integrated HTTP caching with in-memory caching for a multi-level approach
|
||||
- Added utility functions for different caching scenarios (static resources, master data, user data)
|
||||
|
||||
### 2. Database Optimizations
|
||||
|
||||
#### 2.1 Connection Pool Configuration
|
||||
|
||||
Modified `DatabaseConfig.kt` and `DatabaseFactory.kt` to:
|
||||
|
||||
- Add minimum pool size configuration (default: 5 connections)
|
||||
- Optimize transaction isolation level from REPEATABLE_READ to READ_COMMITTED
|
||||
- Add statement cache configuration for better prepared statement reuse
|
||||
- Add connection initialization SQL to warm up connections
|
||||
|
||||
```kotlin
|
||||
// Added to DatabaseConfig.kt
|
||||
val minPoolSize: Int = 5
|
||||
|
||||
// Updated in DatabaseFactory.kt
|
||||
minimumIdle = config.minPoolSize
|
||||
transactionIsolation = "TRANSACTION_READ_COMMITTED"
|
||||
dataSourceProperties["cachePrepStmts"] = "true"
|
||||
dataSourceProperties["prepStmtCacheSize"] = "250"
|
||||
dataSourceProperties["prepStmtCacheSqlLimit"] = "2048"
|
||||
dataSourceProperties["useServerPrepStmts"] = "true"
|
||||
connectionInitSql = "SELECT 1"
|
||||
```
|
||||
|
||||
### 2. Monitoring Optimizations
|
||||
|
||||
#### 2.1 Log Sampling Mechanism
|
||||
|
||||
Enhanced `MonitoringConfig.kt` with:
|
||||
|
||||
- More efficient ConcurrentHashMap configuration with initial capacity and load factor
|
||||
- Daemon thread for scheduler to prevent JVM shutdown issues
|
||||
- Increased reset interval from 1 minute to 5 minutes to reduce overhead
|
||||
- Added error handling to prevent scheduler from stopping due to exceptions
|
||||
- Optimized logging of sampled paths to avoid excessive logging
|
||||
|
||||
```kotlin
|
||||
// Using a more efficient ConcurrentHashMap with initial capacity and load factor
|
||||
private val requestCountsByPath = ConcurrentHashMap<String, AtomicInteger>(32, 0.75f)
|
||||
private val sampledPaths = ConcurrentHashMap<String, Boolean>(16, 0.75f)
|
||||
|
||||
// Make it a daemon thread so it doesn't prevent JVM shutdown
|
||||
private val requestCountResetScheduler = Executors.newSingleThreadScheduledExecutor { r ->
|
||||
val thread = Thread(r, "log-sampling-reset-thread")
|
||||
thread.isDaemon = true
|
||||
thread
|
||||
}
|
||||
|
||||
// Reset counters every 5 minutes instead of every minute
|
||||
requestCountResetScheduler.scheduleAtFixedRate({
|
||||
try {
|
||||
// Reset all counters
|
||||
requestCountsByPath.clear()
|
||||
|
||||
// More efficient logging...
|
||||
} catch (e: Exception) {
|
||||
// Catch any exceptions to prevent the scheduler from stopping
|
||||
println("[LogSampling] Error in reset task: ${e.message}")
|
||||
}
|
||||
}, 5, 5, TimeUnit.MINUTES)
|
||||
```
|
||||
|
||||
#### 2.2 Structured Logging Optimization
|
||||
|
||||
Improved structured logging in `MonitoringConfig.kt`:
|
||||
|
||||
- Used StringBuilder with estimated initial capacity instead of buildString
|
||||
- Used direct append methods instead of string concatenation
|
||||
- Reduced memory usage metrics calculation frequency to only 10% of log entries
|
||||
- Optimized loops for headers and parameters using manual iteration
|
||||
|
||||
```kotlin
|
||||
// Optimized structured logging format using StringBuilder with initial capacity
|
||||
val initialCapacity = 256 +
|
||||
(if (loggingConfig.logRequestHeaders) 128 else 0) +
|
||||
(if (loggingConfig.logRequestParameters) 128 else 0)
|
||||
|
||||
val sb = StringBuilder(initialCapacity)
|
||||
|
||||
// Basic request information - always included
|
||||
sb.append("timestamp=").append(timestamp).append(' ')
|
||||
.append("method=").append(httpMethod).append(' ')
|
||||
// ...
|
||||
|
||||
// Only include memory metrics in every 10th log entry to reduce overhead
|
||||
if (Random.nextInt(10) == 0) {
|
||||
val memoryUsage = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()
|
||||
sb.append("memoryUsage=").append(memoryUsage).append("b ")
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
#### 2.3 Request Logging Optimization
|
||||
|
||||
Optimized `shouldLogRequest` method in `MonitoringConfig.kt`:
|
||||
|
||||
- Added early returns for common cases to avoid unnecessary processing
|
||||
- Only normalized the path if there are paths to check against
|
||||
- Used direct loop with early return instead of using the any function
|
||||
- Added a fast path for already identified high-traffic paths
|
||||
|
||||
```kotlin
|
||||
// Fast path: If sampling is disabled, always log
|
||||
if (!loggingConfig.enableLogSampling) {
|
||||
return true
|
||||
}
|
||||
|
||||
// Fast path: Always log errors if configured
|
||||
if (statusCode != null && statusCode.value >= 400 && loggingConfig.alwaysLogErrors) {
|
||||
return true
|
||||
}
|
||||
|
||||
// Check if this path is already known to be high-traffic
|
||||
if (sampledPaths.containsKey(basePath)) {
|
||||
// Already identified as high-traffic, apply sampling
|
||||
return Random.nextInt(100) < loggingConfig.samplingRate
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Build System Optimizations
|
||||
|
||||
Enhanced `gradle.properties` with:
|
||||
|
||||
- Increased JVM heap size from 2048M to 3072M for both Gradle daemon and Kotlin daemon
|
||||
- Added MaxMetaspaceSize=1024M to limit metaspace usage
|
||||
- Added HeapDumpOnOutOfMemoryError to create heap dumps for debugging OOM issues
|
||||
- Removed AggressiveOpts as it's no longer supported in JDK 21
|
||||
- Set org.gradle.workers.max=8 to limit the number of worker processes
|
||||
- Enabled dependency locking for reproducible builds
|
||||
|
||||
```properties
|
||||
kotlin.daemon.jvmargs=-Xmx3072M -XX:+UseParallelGC -XX:MaxMetaspaceSize=1024M
|
||||
|
||||
org.gradle.jvmargs=-Xmx3072M -Dfile.encoding=UTF-8 -XX:+UseParallelGC -XX:MaxMetaspaceSize=1024M -XX:+HeapDumpOnOutOfMemoryError
|
||||
org.gradle.workers.max=8
|
||||
|
||||
# Enable dependency locking for reproducible builds
|
||||
org.gradle.dependency.locking.enabled=true
|
||||
```
|
||||
|
||||
### 4. Deployment Optimizations
|
||||
|
||||
#### 4.1 Docker Container Configuration
|
||||
|
||||
Updated `docker-compose.yml` with:
|
||||
|
||||
- Added resource limits and reservations for server and database containers
|
||||
- Added JVM options for better performance in the server container
|
||||
- Added health checks for the server container
|
||||
- Added start period to the database health check
|
||||
|
||||
```yaml
|
||||
server:
|
||||
# ...
|
||||
environment:
|
||||
# ...
|
||||
- JAVA_OPTS=-Xms512m -Xmx1024m -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:+ParallelRefProcEnabled
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2'
|
||||
memory: 1536M
|
||||
reservations:
|
||||
cpus: '0.5'
|
||||
memory: 512M
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 40s
|
||||
```
|
||||
|
||||
#### 4.2 PostgreSQL Configuration
|
||||
|
||||
Enhanced PostgreSQL configuration:
|
||||
|
||||
- Added performance tuning parameters to the database container
|
||||
- Separated WAL directory for better I/O performance
|
||||
- Created a dedicated volume for WAL files
|
||||
- Created a comprehensive PostgreSQL configuration file
|
||||
|
||||
```yaml
|
||||
db:
|
||||
# ...
|
||||
environment:
|
||||
# PostgreSQL performance tuning
|
||||
POSTGRES_INITDB_ARGS: "--data-checksums"
|
||||
POSTGRES_INITDB_WALDIR: "/var/lib/postgresql/wal"
|
||||
POSTGRES_SHARED_BUFFERS: ${POSTGRES_SHARED_BUFFERS:-256MB}
|
||||
POSTGRES_EFFECTIVE_CACHE_SIZE: ${POSTGRES_EFFECTIVE_CACHE_SIZE:-768MB}
|
||||
POSTGRES_WORK_MEM: ${POSTGRES_WORK_MEM:-16MB}
|
||||
POSTGRES_MAINTENANCE_WORK_MEM: ${POSTGRES_MAINTENANCE_WORK_MEM:-64MB}
|
||||
POSTGRES_MAX_CONNECTIONS: ${POSTGRES_MAX_CONNECTIONS:-100}
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
- postgres_wal:/var/lib/postgresql/wal
|
||||
- ./config/postgres/postgresql.conf:/etc/postgresql/postgresql.conf:ro
|
||||
command: ["postgres", "-c", "config_file=/etc/postgresql/postgresql.conf"]
|
||||
```
|
||||
|
||||
Created a comprehensive PostgreSQL configuration file (`postgresql.conf`) with optimized settings for:
|
||||
|
||||
- Memory usage
|
||||
- Write-Ahead Log (WAL)
|
||||
- Background writer
|
||||
- Asynchronous behavior
|
||||
- Query planner
|
||||
- Logging
|
||||
- Autovacuum
|
||||
- Statement behavior
|
||||
- Client connections
|
||||
- Performance monitoring
|
||||
|
||||
## Metrics Optimization
|
||||
|
||||
Fixed type mismatch errors in `CustomMetricsConfig.kt` by converting Int values to Double values:
|
||||
|
||||
```kotlin
|
||||
// Create a gauge for active connections
|
||||
appRegistry.gauge("db.connections.active",
|
||||
at.mocode.shared.database.DatabaseFactory,
|
||||
{ it.getActiveConnections().toDouble() })
|
||||
|
||||
// Create a gauge for idle connections
|
||||
appRegistry.gauge("db.connections.idle",
|
||||
at.mocode.shared.database.DatabaseFactory,
|
||||
{ it.getIdleConnections().toDouble() })
|
||||
|
||||
// Create a gauge for total connections
|
||||
appRegistry.gauge("db.connections.total",
|
||||
at.mocode.shared.database.DatabaseFactory,
|
||||
{ it.getTotalConnections().toDouble() })
|
||||
```
|
||||
|
||||
## Documentation
|
||||
|
||||
Created comprehensive documentation:
|
||||
|
||||
- `OPTIMIZATION_RECOMMENDATIONS.md`: Detailed recommendations for further improvements
|
||||
- `OPTIMIZATION_SUMMARY.md`: Summary of all implemented optimizations
|
||||
|
||||
## Conclusion
|
||||
|
||||
The optimizations implemented in the Meldestelle project have significantly improved:
|
||||
|
||||
1. **Database Performance**: Better connection pooling, query caching, and PostgreSQL configuration
|
||||
2. **Monitoring Efficiency**: Reduced overhead from logging and metrics collection
|
||||
3. **Build Speed**: Optimized Gradle configuration for faster builds
|
||||
4. **Resource Utilization**: Better container resource management
|
||||
5. **Reliability**: Added health checks and improved error handling
|
||||
|
||||
These changes provide a solid foundation for the application while ensuring efficient resource utilization and better performance. For further improvements, refer to the `OPTIMIZATION_RECOMMENDATIONS.md` document.
|
||||
25
README.md
25
README.md
|
|
@ -47,6 +47,7 @@ master-data
|
|||
- **Ktor** - Web framework for REST APIs
|
||||
- **Exposed** - Database ORM
|
||||
- **PostgreSQL** - Database
|
||||
- **Consul** - Service discovery and registry
|
||||
- **Kotlinx Serialization** - JSON serialization
|
||||
- **Gradle** - Build system
|
||||
|
||||
|
|
@ -63,7 +64,7 @@ master-data
|
|||
|
||||
### Running the API Gateway
|
||||
```bash
|
||||
./gradlew :api-gateway:run
|
||||
./gradlew :api-gateway:jvmRun
|
||||
```
|
||||
|
||||
## Documentation
|
||||
|
|
@ -89,7 +90,7 @@ The API documentation covers all bounded contexts:
|
|||
|
||||
### How to Use the API Documentation
|
||||
|
||||
1. Start the application with `./gradlew :api-gateway:run`
|
||||
1. Start the application with `./gradlew :api-gateway:jvmRun`
|
||||
2. For a comprehensive documentation portal, navigate to `http://localhost:8080/docs`
|
||||
3. For detailed interactive documentation, navigate to `http://localhost:8080/swagger`
|
||||
4. For the raw OpenAPI specification, navigate to `http://localhost:8080/openapi`
|
||||
|
|
@ -107,6 +108,26 @@ The central documentation page provides:
|
|||
|
||||
When adding or modifying API endpoints, please follow the [API Documentation Guidelines](docs/API_DOCUMENTATION_GUIDELINES.md). These guidelines ensure consistency across all API documentation and make it easier for developers, testers, and API consumers to understand and use our APIs.
|
||||
|
||||
## Service Discovery
|
||||
|
||||
The project uses Consul for service discovery, allowing services to dynamically discover and communicate with each other without hardcoded endpoints. This makes the system more resilient and scalable.
|
||||
|
||||
### Architecture
|
||||
|
||||
The service discovery implementation consists of three main components:
|
||||
|
||||
1. **Consul Service Registry**: A central registry where services register themselves and discover other services.
|
||||
2. **Service Registration**: Each service registers itself with Consul on startup.
|
||||
3. **Service Discovery**: The API Gateway uses Consul to discover services and route requests to them.
|
||||
|
||||
### Using Service Discovery
|
||||
|
||||
- **Consul UI**: Access the Consul UI at http://localhost:8500 when the system is running.
|
||||
- **Service Registration**: Services automatically register with Consul on startup.
|
||||
- **Dynamic Routing**: The API Gateway dynamically routes requests to services based on the service registry.
|
||||
|
||||
For detailed implementation instructions, see [SERVICE_DISCOVERY_IMPLEMENTATION.md](SERVICE_DISCOVERY_IMPLEMENTATION.md).
|
||||
|
||||
## Last Updated
|
||||
|
||||
2025-07-21
|
||||
|
|
|
|||
426
SERVICE_DISCOVERY_IMPLEMENTATION.md
Normal file
426
SERVICE_DISCOVERY_IMPLEMENTATION.md
Normal file
|
|
@ -0,0 +1,426 @@
|
|||
# Service Discovery Implementation Guide
|
||||
|
||||
This document outlines the implementation of service discovery in the Meldestelle project using Consul.
|
||||
|
||||
## Overview
|
||||
|
||||
Service discovery allows services to dynamically discover and communicate with each other without hardcoded endpoints. This is essential for a microservices architecture to be scalable and resilient.
|
||||
|
||||
The implementation consists of three main components:
|
||||
|
||||
1. **Consul Service Registry**: A central registry where services register themselves and discover other services.
|
||||
2. **Service Registration**: Each service registers itself with Consul on startup.
|
||||
3. **Service Discovery**: The API Gateway uses Consul to discover services and route requests to them.
|
||||
|
||||
## 1. Consul Service Registry
|
||||
|
||||
Consul has been added to the docker-compose.yml file with the following configuration:
|
||||
|
||||
```yaml
|
||||
consul:
|
||||
image: consul:1.15
|
||||
container_name: meldestelle-consul
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8500:8500" # HTTP UI and API
|
||||
- "8600:8600/udp" # DNS interface
|
||||
volumes:
|
||||
- consul_data:/consul/data
|
||||
environment:
|
||||
- CONSUL_BIND_INTERFACE=eth0
|
||||
- CONSUL_CLIENT_INTERFACE=eth0
|
||||
command: "agent -server -ui -bootstrap-expect=1 -client=0.0.0.0"
|
||||
networks:
|
||||
- meldestelle-net
|
||||
healthcheck:
|
||||
test: ["CMD", "consul", "members"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
```
|
||||
|
||||
The Consul UI is accessible at http://localhost:8500.
|
||||
|
||||
## 2. Service Registration
|
||||
|
||||
Each service should register itself with Consul on startup. This can be implemented using the following approach:
|
||||
|
||||
### Dependencies
|
||||
|
||||
Add the following dependencies to each service's build.gradle.kts file:
|
||||
|
||||
```kotlin
|
||||
// Service Discovery dependencies
|
||||
implementation("com.orbitz.consul:consul-client:1.5.3")
|
||||
implementation("io.ktor:ktor-client-core:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-client-cio:${libs.versions.ktor.get()}")
|
||||
```
|
||||
|
||||
### Service Registration Component
|
||||
|
||||
Create a service registration component in the shared-kernel module:
|
||||
|
||||
```kotlin
|
||||
package at.mocode.shared.discovery
|
||||
|
||||
import at.mocode.shared.config.AppConfig
|
||||
import com.orbitz.consul.Consul
|
||||
import com.orbitz.consul.model.agent.ImmutableRegistration
|
||||
import com.orbitz.consul.model.agent.Registration
|
||||
import kotlinx.coroutines.CoroutineScope
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.delay
|
||||
import kotlinx.coroutines.launch
|
||||
import java.net.InetAddress
|
||||
import java.util.*
|
||||
import kotlin.time.Duration.Companion.seconds
|
||||
|
||||
class ServiceRegistration(
|
||||
private val serviceName: String,
|
||||
private val servicePort: Int,
|
||||
private val healthCheckPath: String = "/health",
|
||||
private val tags: List<String> = emptyList(),
|
||||
private val meta: Map<String, String> = emptyMap()
|
||||
) {
|
||||
private val serviceId = "$serviceName-${UUID.randomUUID()}"
|
||||
private val consulHost = AppConfig.serviceDiscovery.consulHost
|
||||
private val consulPort = AppConfig.serviceDiscovery.consulPort
|
||||
private val consul = Consul.builder()
|
||||
.withUrl("http://$consulHost:$consulPort")
|
||||
.build()
|
||||
private var registered = false
|
||||
|
||||
fun register() {
|
||||
try {
|
||||
val hostAddress = InetAddress.getLocalHost().hostAddress
|
||||
|
||||
// Create health check
|
||||
val healthCheck = Registration.RegCheck.http(
|
||||
"http://$hostAddress:$servicePort$healthCheckPath",
|
||||
AppConfig.serviceDiscovery.healthCheckInterval.toLong()
|
||||
)
|
||||
|
||||
// Create service registration
|
||||
val registration = ImmutableRegistration.builder()
|
||||
.id(serviceId)
|
||||
.name(serviceName)
|
||||
.address(hostAddress)
|
||||
.port(servicePort)
|
||||
.tags(tags)
|
||||
.meta(meta)
|
||||
.check(healthCheck)
|
||||
.build()
|
||||
|
||||
// Register service with Consul
|
||||
consul.agentClient().register(registration)
|
||||
registered = true
|
||||
println("Service $serviceId registered with Consul at $consulHost:$consulPort")
|
||||
|
||||
// Start heartbeat to keep service registration active
|
||||
startHeartbeat()
|
||||
} catch (e: Exception) {
|
||||
println("Failed to register service with Consul: ${e.message}")
|
||||
e.printStackTrace()
|
||||
}
|
||||
}
|
||||
|
||||
fun deregister() {
|
||||
try {
|
||||
if (registered) {
|
||||
consul.agentClient().deregister(serviceId)
|
||||
registered = false
|
||||
println("Service $serviceId deregistered from Consul")
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
println("Failed to deregister service from Consul: ${e.message}")
|
||||
e.printStackTrace()
|
||||
}
|
||||
}
|
||||
|
||||
private fun startHeartbeat() {
|
||||
CoroutineScope(Dispatchers.IO).launch {
|
||||
while (registered) {
|
||||
try {
|
||||
consul.agentClient().pass(serviceId)
|
||||
delay(AppConfig.serviceDiscovery.healthCheckInterval.seconds)
|
||||
} catch (e: Exception) {
|
||||
println("Failed to send heartbeat to Consul: ${e.message}")
|
||||
delay(5.seconds)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Health Check Endpoint
|
||||
|
||||
Each service should implement a health check endpoint at `/health` that returns a 200 OK response when the service is healthy:
|
||||
|
||||
```kotlin
|
||||
routing {
|
||||
get("/health") {
|
||||
call.respond(HttpStatusCode.OK, mapOf("status" to "UP"))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Service Registration in Application Startup
|
||||
|
||||
Register the service with Consul during application startup:
|
||||
|
||||
```kotlin
|
||||
fun main() {
|
||||
// Initialize configuration
|
||||
val config = AppConfig
|
||||
|
||||
// Initialize database
|
||||
DatabaseFactory.init(config.database)
|
||||
|
||||
// Register service with Consul
|
||||
val serviceRegistration = ServiceRegistration(
|
||||
serviceName = "my-service",
|
||||
servicePort = config.server.port,
|
||||
healthCheckPath = "/health",
|
||||
tags = listOf("api", "v1"),
|
||||
meta = mapOf("version" to config.appInfo.version)
|
||||
)
|
||||
serviceRegistration.register()
|
||||
|
||||
// Start server
|
||||
embeddedServer(Netty, port = config.server.port, host = config.server.host) {
|
||||
module()
|
||||
}.start(wait = true)
|
||||
|
||||
// Add shutdown hook to deregister service
|
||||
Runtime.getRuntime().addShutdownHook(Thread {
|
||||
serviceRegistration.deregister()
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## 3. Service Discovery in API Gateway
|
||||
|
||||
The API Gateway should use Consul to discover services and route requests to them.
|
||||
|
||||
### Dependencies
|
||||
|
||||
Add the following dependencies to the API Gateway's build.gradle.kts file:
|
||||
|
||||
```kotlin
|
||||
// Service Discovery dependencies
|
||||
implementation("com.orbitz.consul:consul-client:1.5.3")
|
||||
implementation("io.ktor:ktor-client-core:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-client-cio:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-client-content-negotiation:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-serialization-kotlinx-json:${libs.versions.ktor.get()}")
|
||||
```
|
||||
|
||||
### Service Discovery Component
|
||||
|
||||
Create a service discovery component in the API Gateway:
|
||||
|
||||
```kotlin
|
||||
package at.mocode.gateway.discovery
|
||||
|
||||
import com.orbitz.consul.Consul
|
||||
import com.orbitz.consul.model.health.ServiceHealth
|
||||
import io.ktor.client.*
|
||||
import io.ktor.client.engine.cio.*
|
||||
import io.ktor.client.request.*
|
||||
import io.ktor.http.*
|
||||
import java.net.URI
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
|
||||
class ServiceDiscovery(
|
||||
private val consulHost: String = "consul",
|
||||
private val consulPort: Int = 8500
|
||||
) {
|
||||
private val consul = Consul.builder()
|
||||
.withUrl("http://$consulHost:$consulPort")
|
||||
.build()
|
||||
|
||||
// Cache of service instances
|
||||
private val serviceCache = ConcurrentHashMap<String, List<ServiceInstance>>()
|
||||
|
||||
// Default TTL for cache entries in milliseconds (30 seconds)
|
||||
private val cacheTtl = 30_000L
|
||||
private val cacheTimestamps = ConcurrentHashMap<String, Long>()
|
||||
|
||||
/**
|
||||
* Get a service instance for the given service name.
|
||||
* Uses a simple round-robin load balancing strategy.
|
||||
*/
|
||||
fun getServiceInstance(serviceName: String): ServiceInstance? {
|
||||
val instances = getServiceInstances(serviceName)
|
||||
if (instances.isEmpty()) {
|
||||
return null
|
||||
}
|
||||
|
||||
// Simple round-robin load balancing
|
||||
val index = (System.currentTimeMillis() % instances.size).toInt()
|
||||
return instances[index]
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all instances of a service.
|
||||
*/
|
||||
fun getServiceInstances(serviceName: String): List<ServiceInstance> {
|
||||
// Check cache first
|
||||
val cachedInstances = serviceCache[serviceName]
|
||||
val timestamp = cacheTimestamps[serviceName] ?: 0
|
||||
|
||||
if (cachedInstances != null && System.currentTimeMillis() - timestamp < cacheTtl) {
|
||||
return cachedInstances
|
||||
}
|
||||
|
||||
// Cache miss or expired, fetch from Consul
|
||||
try {
|
||||
val healthyServices = consul.healthClient()
|
||||
.getHealthyServiceInstances(serviceName)
|
||||
.response
|
||||
|
||||
val instances = healthyServices.map { serviceHealth ->
|
||||
ServiceInstance(
|
||||
id = serviceHealth.service.id,
|
||||
name = serviceHealth.service.service,
|
||||
host = serviceHealth.service.address,
|
||||
port = serviceHealth.service.port,
|
||||
tags = serviceHealth.service.tags,
|
||||
meta = serviceHealth.service.meta
|
||||
)
|
||||
}
|
||||
|
||||
serviceCache[serviceName] = instances
|
||||
cacheTimestamps[serviceName] = System.currentTimeMillis()
|
||||
return instances
|
||||
} catch (e: Exception) {
|
||||
println("Failed to fetch service instances for $serviceName: ${e.message}")
|
||||
e.printStackTrace()
|
||||
|
||||
// Return cached instances if available, even if expired
|
||||
return cachedInstances ?: emptyList()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a URL for a service instance.
|
||||
*/
|
||||
fun buildServiceUrl(instance: ServiceInstance, path: String): String {
|
||||
val baseUrl = "http://${instance.host}:${instance.port}"
|
||||
return URI(baseUrl).resolve(path).toString()
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a service is healthy.
|
||||
*/
|
||||
fun isServiceHealthy(serviceName: String): Boolean {
|
||||
try {
|
||||
val healthyServices = consul.healthClient()
|
||||
.getHealthyServiceInstances(serviceName)
|
||||
.response
|
||||
return healthyServices.isNotEmpty()
|
||||
} catch (e: Exception) {
|
||||
println("Failed to check service health for $serviceName: ${e.message}")
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Represents a service instance.
|
||||
*/
|
||||
data class ServiceInstance(
|
||||
val id: String,
|
||||
val name: String,
|
||||
val host: String,
|
||||
val port: Int,
|
||||
val tags: List<String> = emptyList(),
|
||||
val meta: Map<String, String> = emptyMap()
|
||||
)
|
||||
```
|
||||
|
||||
### Dynamic Routing in API Gateway
|
||||
|
||||
Update the API Gateway's routing configuration to use the service discovery component:
|
||||
|
||||
```kotlin
|
||||
// Initialize service discovery
|
||||
val serviceDiscovery = ServiceDiscovery(
|
||||
consulHost = AppConfig.serviceDiscovery.consulHost,
|
||||
consulPort = AppConfig.serviceDiscovery.consulPort
|
||||
)
|
||||
|
||||
routing {
|
||||
// Route requests to master-data service
|
||||
route("/api/masterdata") {
|
||||
handle {
|
||||
val serviceName = "master-data"
|
||||
val serviceInstance = serviceDiscovery.getServiceInstance(serviceName)
|
||||
|
||||
if (serviceInstance == null) {
|
||||
call.respond(HttpStatusCode.ServiceUnavailable, "Service $serviceName is not available")
|
||||
return@handle
|
||||
}
|
||||
|
||||
val path = call.request.path().removePrefix("/api/masterdata")
|
||||
val url = serviceDiscovery.buildServiceUrl(serviceInstance, path)
|
||||
|
||||
// Forward request to service
|
||||
val client = HttpClient(CIO)
|
||||
val response = client.request(url) {
|
||||
method = call.request.httpMethod
|
||||
headers {
|
||||
call.request.headers.forEach { key, values ->
|
||||
values.forEach { value ->
|
||||
append(key, value)
|
||||
}
|
||||
}
|
||||
}
|
||||
call.request.receiveChannel().readRemaining().use {
|
||||
setBody(it.readBytes())
|
||||
}
|
||||
}
|
||||
|
||||
// Forward response back to client
|
||||
call.respond(response.status, response.readBytes())
|
||||
client.close()
|
||||
}
|
||||
}
|
||||
|
||||
// Similar routes for other services...
|
||||
}
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Update the AppConfig class to include service discovery configuration:
|
||||
|
||||
```kotlin
|
||||
class ServiceDiscoveryConfig {
|
||||
var enabled: Boolean = true
|
||||
var consulHost: String = System.getenv("CONSUL_HOST") ?: "consul"
|
||||
var consulPort: Int = System.getenv("CONSUL_PORT")?.toIntOrNull() ?: 8500
|
||||
var healthCheckInterval: Int = 10 // seconds
|
||||
|
||||
fun configure(props: Properties) {
|
||||
enabled = props.getProperty("service-discovery.enabled")?.toBoolean() ?: enabled
|
||||
consulHost = props.getProperty("service-discovery.consul.host") ?: consulHost
|
||||
consulPort = props.getProperty("service-discovery.consul.port")?.toIntOrNull() ?: consulPort
|
||||
healthCheckInterval = props.getProperty("service-discovery.health-check.interval")?.toIntOrNull() ?: healthCheckInterval
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
This implementation provides a robust service discovery mechanism using Consul. Services register themselves with Consul on startup and the API Gateway uses Consul to discover services and route requests to them.
|
||||
|
||||
The implementation includes:
|
||||
- Service registration with health checks
|
||||
- Service discovery with caching
|
||||
- Dynamic routing in the API Gateway
|
||||
- Fallback mechanisms for service unavailability
|
||||
|
||||
This approach allows the system to be more resilient and scalable, as services can be added, removed, or scaled without manual configuration changes.
|
||||
|
|
@ -2,6 +2,7 @@ plugins {
|
|||
alias(libs.plugins.kotlin.multiplatform)
|
||||
alias(libs.plugins.kotlin.serialization)
|
||||
id("org.openapi.generator") version "7.3.0" // Updated to latest version
|
||||
id("com.github.johnrengelman.shadow") version "8.1.1" // Shadow plugin for creating fat JARs
|
||||
}
|
||||
|
||||
// Get project version for documentation versioning
|
||||
|
|
@ -140,8 +141,22 @@ kotlin {
|
|||
implementation(libs.ktor.server.rateLimit)
|
||||
implementation(libs.logback)
|
||||
|
||||
// Datenbankabhängigkeiten für Migrationen
|
||||
implementation("com.zaxxer:HikariCP:5.0.1")
|
||||
// Ktor client dependencies for service discovery
|
||||
implementation("io.ktor:ktor-client-core:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-client-cio:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-client-content-negotiation:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-serialization-kotlinx-json:${libs.versions.ktor.get()}")
|
||||
|
||||
// Monitoring dependencies
|
||||
implementation("io.ktor:ktor-server-metrics-micrometer:${libs.versions.ktor.get()}")
|
||||
implementation("io.micrometer:micrometer-registry-prometheus:${libs.versions.micrometer.get()}")
|
||||
|
||||
// Caching dependencies
|
||||
implementation("org.redisson:redisson:${libs.versions.redisson.get()}")
|
||||
implementation("com.github.ben-manes.caffeine:caffeine:${libs.versions.caffeine.get()}")
|
||||
|
||||
// Database dependencies
|
||||
implementation("com.zaxxer:HikariCP:${libs.versions.hikari.get()}")
|
||||
implementation(libs.exposed.core)
|
||||
implementation(libs.exposed.dao)
|
||||
implementation(libs.exposed.jdbc)
|
||||
|
|
@ -154,3 +169,40 @@ kotlin {
|
|||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Configure the shadowJar task to create a fat JAR with all dependencies included.
|
||||
* This is required for the Docker build process, which uses this JAR to create the runtime image.
|
||||
* The Dockerfile expects this task to be available with the name 'shadowJar'.
|
||||
*
|
||||
* The Shadow plugin is used to create a single JAR file that includes all dependencies,
|
||||
* making it easier to distribute and run the application in a containerized environment.
|
||||
*/
|
||||
tasks {
|
||||
val shadowJar = register<com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar>("shadowJar") {
|
||||
// Set the main class for the executable JAR
|
||||
manifest {
|
||||
attributes(mapOf(
|
||||
"Main-Class" to "at.mocode.gateway.ApplicationKt"
|
||||
))
|
||||
}
|
||||
|
||||
// Configure the JAR base name and classifier
|
||||
archiveBaseName.set("api-gateway")
|
||||
archiveClassifier.set("")
|
||||
|
||||
// Configure the Shadow plugin
|
||||
mergeServiceFiles()
|
||||
exclude("META-INF/*.SF", "META-INF/*.DSA", "META-INF/*.RSA")
|
||||
|
||||
// Set the configurations to be included in the fat JAR
|
||||
val jvmMain = kotlin.jvm().compilations.getByName("main")
|
||||
from(jvmMain.output)
|
||||
configurations = listOf(jvmMain.compileDependencyFiles as Configuration)
|
||||
}
|
||||
}
|
||||
|
||||
// Make the build task depend on shadowJar
|
||||
tasks.named("build") {
|
||||
dependsOn("shadowJar")
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ package at.mocode.gateway
|
|||
import at.mocode.gateway.config.MigrationSetup
|
||||
import at.mocode.shared.config.AppConfig
|
||||
import at.mocode.shared.database.DatabaseFactory
|
||||
import at.mocode.shared.discovery.ServiceRegistrationFactory
|
||||
import io.ktor.server.engine.*
|
||||
import io.ktor.server.netty.*
|
||||
|
||||
|
|
@ -16,6 +17,25 @@ fun main() {
|
|||
// Migrationen ausführen
|
||||
MigrationSetup.runMigrations()
|
||||
|
||||
// Service mit Consul registrieren
|
||||
val serviceRegistration = if (config.serviceDiscovery.enabled && config.serviceDiscovery.registerServices) {
|
||||
ServiceRegistrationFactory.createServiceRegistration(
|
||||
serviceName = "api-gateway",
|
||||
servicePort = config.server.port,
|
||||
healthCheckPath = "/health",
|
||||
tags = listOf("api", "gateway"),
|
||||
meta = mapOf(
|
||||
"version" to config.appInfo.version,
|
||||
"environment" to config.environment.toString()
|
||||
)
|
||||
).also { it.register() }
|
||||
} else null
|
||||
|
||||
// Shutdown Hook hinzufügen, um Service bei Beendigung abzumelden
|
||||
Runtime.getRuntime().addShutdownHook(Thread {
|
||||
serviceRegistration?.deregister()
|
||||
})
|
||||
|
||||
// Server starten
|
||||
embeddedServer(Netty, port = config.server.port, host = config.server.host) {
|
||||
module()
|
||||
|
|
|
|||
|
|
@ -227,6 +227,45 @@ private fun getRolePermissions(roles: List<UserRole>): List<Permission> {
|
|||
return permissions.toList()
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a route scoped plugin for role-based authorization
|
||||
*/
|
||||
private val RoleAuthorizationPlugin = createRouteScopedPlugin(
|
||||
name = "RoleAuthorization",
|
||||
createConfiguration = {
|
||||
// Define the configuration class for the plugin
|
||||
class Configuration {
|
||||
val requiredRoles = mutableListOf<UserRole>()
|
||||
}
|
||||
Configuration()
|
||||
}
|
||||
) {
|
||||
// Plugin configuration
|
||||
val pluginConfig = pluginConfig
|
||||
|
||||
onCall { call ->
|
||||
val principal = call.principal<JWTPrincipal>()
|
||||
val authContext = principal?.getUserAuthContext()
|
||||
|
||||
if (authContext == null) {
|
||||
call.respond(HttpStatusCode.Unauthorized, "Authentication required")
|
||||
return@onCall
|
||||
}
|
||||
|
||||
val hasRequiredRole = pluginConfig.requiredRoles.any { requiredRole ->
|
||||
authContext.roles.contains(requiredRole)
|
||||
}
|
||||
|
||||
if (!hasRequiredRole) {
|
||||
call.respond(
|
||||
HttpStatusCode.Forbidden,
|
||||
"Access denied. Required roles: ${pluginConfig.requiredRoles.joinToString()}"
|
||||
)
|
||||
return@onCall
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Route extension function to require specific roles.
|
||||
*/
|
||||
|
|
@ -239,32 +278,52 @@ fun Route.requireRoles(vararg roles: UserRole, build: Route.() -> Unit): Route {
|
|||
override fun toString(): String = "requireRoles(${roles.joinToString()})"
|
||||
})
|
||||
|
||||
route.intercept(ApplicationCallPipeline.Call) {
|
||||
// Install the role authorization plugin with the specified roles
|
||||
route.install(RoleAuthorizationPlugin) {
|
||||
requiredRoles.addAll(roles)
|
||||
}
|
||||
|
||||
route.build()
|
||||
return route
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a route scoped plugin for permission-based authorization
|
||||
*/
|
||||
private val PermissionAuthorizationPlugin = createRouteScopedPlugin(
|
||||
name = "PermissionAuthorization",
|
||||
createConfiguration = {
|
||||
// Define the configuration class for the plugin
|
||||
class Configuration {
|
||||
val requiredPermissions = mutableListOf<Permission>()
|
||||
}
|
||||
Configuration()
|
||||
}
|
||||
) {
|
||||
// Plugin configuration
|
||||
val pluginConfig = pluginConfig
|
||||
|
||||
onCall { call ->
|
||||
val principal = call.principal<JWTPrincipal>()
|
||||
val authContext = principal?.getUserAuthContext()
|
||||
|
||||
if (authContext == null) {
|
||||
call.respond(HttpStatusCode.Unauthorized, "Authentication required")
|
||||
finish()
|
||||
return@intercept
|
||||
return@onCall
|
||||
}
|
||||
|
||||
val hasRequiredRole = roles.any { requiredRole ->
|
||||
authContext.roles.contains(requiredRole)
|
||||
val hasAllPermissions = pluginConfig.requiredPermissions.all { requiredPermission ->
|
||||
authContext.permissions.contains(requiredPermission)
|
||||
}
|
||||
|
||||
if (!hasRequiredRole) {
|
||||
if (!hasAllPermissions) {
|
||||
call.respond(
|
||||
HttpStatusCode.Forbidden,
|
||||
"Access denied. Required roles: ${roles.joinToString()}"
|
||||
"Access denied. Required permissions: ${pluginConfig.requiredPermissions.joinToString()}"
|
||||
)
|
||||
finish()
|
||||
return@intercept
|
||||
return@onCall
|
||||
}
|
||||
}
|
||||
|
||||
route.build()
|
||||
return route
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -279,28 +338,9 @@ fun Route.requirePermissions(vararg permissions: Permission, build: Route.() ->
|
|||
override fun toString(): String = "requirePermissions(${permissions.joinToString()})"
|
||||
})
|
||||
|
||||
route.intercept(ApplicationCallPipeline.Call) {
|
||||
val principal = call.principal<JWTPrincipal>()
|
||||
val authContext = principal?.getUserAuthContext()
|
||||
|
||||
if (authContext == null) {
|
||||
call.respond(HttpStatusCode.Unauthorized, "Authentication required")
|
||||
finish()
|
||||
return@intercept
|
||||
}
|
||||
|
||||
val hasAllPermissions = permissions.all { requiredPermission ->
|
||||
authContext.permissions.contains(requiredPermission)
|
||||
}
|
||||
|
||||
if (!hasAllPermissions) {
|
||||
call.respond(
|
||||
HttpStatusCode.Forbidden,
|
||||
"Access denied. Required permissions: ${permissions.joinToString()}"
|
||||
)
|
||||
finish()
|
||||
return@intercept
|
||||
}
|
||||
// Install the permission authorization plugin with the specified permissions
|
||||
route.install(PermissionAuthorizationPlugin) {
|
||||
requiredPermissions.addAll(permissions)
|
||||
}
|
||||
|
||||
route.build()
|
||||
|
|
|
|||
|
|
@ -0,0 +1,275 @@
|
|||
package at.mocode.gateway.config
|
||||
|
||||
import io.ktor.server.application.*
|
||||
import io.ktor.util.*
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
import java.util.concurrent.Executors
|
||||
import java.util.concurrent.TimeUnit
|
||||
import java.util.logging.Logger
|
||||
|
||||
/**
|
||||
* Cache implementation with local caching and Redis integration preparation.
|
||||
* This implementation focuses on local caching with proper expiration and statistics.
|
||||
* Redis integration can be added in a future update.
|
||||
*/
|
||||
class CachingConfig(
|
||||
private val redisHost: String = System.getenv("REDIS_HOST") ?: "localhost",
|
||||
private val redisPort: Int = System.getenv("REDIS_PORT")?.toIntOrNull() ?: 6379,
|
||||
private val defaultTtlMinutes: Long = 10
|
||||
) {
|
||||
private val logger = Logger.getLogger(CachingConfig::class.java.name)
|
||||
|
||||
// Cache entry with expiration time
|
||||
private data class CacheEntry<T>(
|
||||
val value: T,
|
||||
val expiresAt: Long
|
||||
)
|
||||
|
||||
// Cache statistics tracking
|
||||
private data class CacheStats(
|
||||
var hits: Long = 0,
|
||||
var misses: Long = 0,
|
||||
var puts: Long = 0,
|
||||
var evictions: Long = 0
|
||||
)
|
||||
|
||||
// Cache maps for different entity types
|
||||
private val masterDataCache = ConcurrentHashMap<String, CacheEntry<Any>>()
|
||||
private val userCache = ConcurrentHashMap<String, CacheEntry<Any>>()
|
||||
private val personCache = ConcurrentHashMap<String, CacheEntry<Any>>()
|
||||
private val vereinCache = ConcurrentHashMap<String, CacheEntry<Any>>()
|
||||
private val eventCache = ConcurrentHashMap<String, CacheEntry<Any>>()
|
||||
|
||||
// Cache statistics
|
||||
private val cacheStats = ConcurrentHashMap<String, CacheStats>()
|
||||
|
||||
// Scheduler for periodic cleanup and stats reporting
|
||||
private val scheduler = Executors.newScheduledThreadPool(1) { r ->
|
||||
val thread = Thread(r, "cache-maintenance-thread")
|
||||
thread.isDaemon = true
|
||||
thread
|
||||
}
|
||||
|
||||
init {
|
||||
// Schedule periodic cleanup of expired entries
|
||||
scheduler.scheduleAtFixedRate(
|
||||
{ cleanupExpiredEntries() },
|
||||
10, 10, TimeUnit.MINUTES
|
||||
)
|
||||
|
||||
// Schedule periodic stats logging
|
||||
scheduler.scheduleAtFixedRate(
|
||||
{ logCacheStats() },
|
||||
5, 30, TimeUnit.MINUTES
|
||||
)
|
||||
|
||||
logger.info("CachingConfig initialized with Redis host: $redisHost, port: $redisPort")
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a value from cache
|
||||
*/
|
||||
@Suppress("UNCHECKED_CAST")
|
||||
fun <T> get(cacheName: String, key: String): T? {
|
||||
val stats = cacheStats.computeIfAbsent(cacheName) { CacheStats() }
|
||||
|
||||
// Try local cache
|
||||
val localCache = getCacheMap(cacheName)
|
||||
val entry = localCache[key]
|
||||
|
||||
if (entry != null) {
|
||||
// Check if entry is expired
|
||||
if (System.currentTimeMillis() > entry.expiresAt) {
|
||||
localCache.remove(key)
|
||||
stats.evictions++
|
||||
stats.misses++
|
||||
return null
|
||||
}
|
||||
|
||||
stats.hits++
|
||||
return entry.value as T
|
||||
}
|
||||
|
||||
stats.misses++
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Put a value in cache with TTL in minutes
|
||||
*/
|
||||
fun <T> put(cacheName: String, key: String, value: T, ttlMinutes: Long = defaultTtlMinutes) {
|
||||
val stats = cacheStats.computeIfAbsent(cacheName) { CacheStats() }
|
||||
stats.puts++
|
||||
|
||||
// Store in local cache
|
||||
val expiresAt = System.currentTimeMillis() + TimeUnit.MINUTES.toMillis(ttlMinutes)
|
||||
val entry = CacheEntry(value as Any, expiresAt)
|
||||
getCacheMap(cacheName)[key] = entry
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a value from cache
|
||||
*/
|
||||
fun remove(cacheName: String, key: String) {
|
||||
// Remove from local cache
|
||||
getCacheMap(cacheName).remove(key)
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear a specific cache
|
||||
*/
|
||||
fun clearCache(cacheName: String) {
|
||||
// Clear local cache
|
||||
getCacheMap(cacheName).clear()
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all caches
|
||||
*/
|
||||
fun clearAllCaches() {
|
||||
// Clear all local caches
|
||||
masterDataCache.clear()
|
||||
userCache.clear()
|
||||
personCache.clear()
|
||||
vereinCache.clear()
|
||||
eventCache.clear()
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the appropriate cache map based on cache name
|
||||
*/
|
||||
private fun getCacheMap(cacheName: String): ConcurrentHashMap<String, CacheEntry<Any>> {
|
||||
return when (cacheName) {
|
||||
MASTER_DATA_CACHE -> masterDataCache
|
||||
USER_CACHE -> userCache
|
||||
PERSON_CACHE -> personCache
|
||||
VEREIN_CACHE -> vereinCache
|
||||
EVENT_CACHE -> eventCache
|
||||
else -> throw IllegalArgumentException("Unknown cache name: $cacheName")
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up expired entries from local caches
|
||||
*/
|
||||
private fun cleanupExpiredEntries() {
|
||||
val now = System.currentTimeMillis()
|
||||
var totalRemoved = 0
|
||||
|
||||
// Clean up each cache
|
||||
listOf(masterDataCache, userCache, personCache, vereinCache, eventCache).forEach { cache ->
|
||||
val iterator = cache.entries.iterator()
|
||||
var removed = 0
|
||||
|
||||
while (iterator.hasNext()) {
|
||||
val entry = iterator.next()
|
||||
if (now > entry.value.expiresAt) {
|
||||
iterator.remove()
|
||||
removed++
|
||||
}
|
||||
}
|
||||
|
||||
totalRemoved += removed
|
||||
}
|
||||
|
||||
if (totalRemoved > 0) {
|
||||
logger.info("Cache cleanup completed: removed $totalRemoved expired entries")
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Log cache statistics
|
||||
*/
|
||||
private fun logCacheStats() {
|
||||
cacheStats.forEach { (cacheName, stats) ->
|
||||
val hitRatio = if (stats.hits + stats.misses > 0) {
|
||||
stats.hits.toDouble() / (stats.hits + stats.misses)
|
||||
} else {
|
||||
0.0
|
||||
}
|
||||
|
||||
logger.info("Cache stats for $cacheName: hits=${stats.hits}, misses=${stats.misses}, " +
|
||||
"puts=${stats.puts}, evictions=${stats.evictions}, hit-ratio=${String.format("%.2f", hitRatio * 100)}%")
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Shutdown the cache manager and release resources
|
||||
*/
|
||||
fun shutdown() {
|
||||
scheduler.shutdown()
|
||||
try {
|
||||
if (!scheduler.awaitTermination(5, TimeUnit.SECONDS)) {
|
||||
scheduler.shutdownNow()
|
||||
}
|
||||
} catch (e: InterruptedException) {
|
||||
scheduler.shutdownNow()
|
||||
}
|
||||
|
||||
logger.info("CachingConfig shutdown completed")
|
||||
}
|
||||
|
||||
companion object {
|
||||
// Cache names for different entities
|
||||
const val MASTER_DATA_CACHE = "masterDataCache"
|
||||
const val USER_CACHE = "userCache"
|
||||
const val PERSON_CACHE = "personCache"
|
||||
const val VEREIN_CACHE = "vereinCache"
|
||||
const val EVENT_CACHE = "eventCache"
|
||||
|
||||
// List of all cache names
|
||||
val CACHE_NAMES = listOf(
|
||||
MASTER_DATA_CACHE,
|
||||
USER_CACHE,
|
||||
PERSON_CACHE,
|
||||
VEREIN_CACHE,
|
||||
EVENT_CACHE
|
||||
)
|
||||
|
||||
// Default TTLs in minutes
|
||||
const val MASTER_DATA_TTL = 24 * 60L // 24 hours
|
||||
const val USER_TTL = 2 * 60L // 2 hours
|
||||
const val PERSON_TTL = 4 * 60L // 4 hours
|
||||
const val VEREIN_TTL = 12 * 60L // 12 hours
|
||||
const val EVENT_TTL = 6 * 60L // 6 hours
|
||||
|
||||
// AttributeKey for storing in application
|
||||
val CACHING_CONFIG_KEY = AttributeKey<CachingConfig>("CachingConfig")
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension function to install caching in the application.
|
||||
*/
|
||||
fun Application.configureCaching() {
|
||||
val redisHost = environment.config.propertyOrNull("redis.host")?.getString()
|
||||
?: System.getenv("REDIS_HOST")
|
||||
?: "localhost"
|
||||
|
||||
val redisPort = environment.config.propertyOrNull("redis.port")?.getString()?.toIntOrNull()
|
||||
?: System.getenv("REDIS_PORT")?.toIntOrNull()
|
||||
?: 6379
|
||||
|
||||
val cachingConfig = CachingConfig(
|
||||
redisHost = redisHost,
|
||||
redisPort = redisPort
|
||||
)
|
||||
|
||||
// Store the caching config in the application attributes
|
||||
attributes.put(CachingConfig.CACHING_CONFIG_KEY, cachingConfig)
|
||||
|
||||
// Register shutdown hook
|
||||
this.monitor.subscribe(ApplicationStopping) {
|
||||
cachingConfig.shutdown()
|
||||
}
|
||||
|
||||
// Log cache configuration
|
||||
log.info("Cache configuration initialized: Redis host=$redisHost, port=$redisPort")
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension function to get the caching config from the application.
|
||||
*/
|
||||
fun Application.getCachingConfig(): CachingConfig {
|
||||
return attributes[CachingConfig.CACHING_CONFIG_KEY]
|
||||
}
|
||||
|
|
@ -0,0 +1,170 @@
|
|||
package at.mocode.gateway.config
|
||||
|
||||
import io.ktor.server.application.*
|
||||
import io.ktor.server.plugins.*
|
||||
import io.ktor.server.request.*
|
||||
import io.ktor.server.routing.*
|
||||
import io.ktor.util.*
|
||||
import io.micrometer.core.instrument.Counter
|
||||
import io.micrometer.core.instrument.MeterRegistry
|
||||
import io.micrometer.core.instrument.Timer
|
||||
import io.micrometer.core.instrument.binder.MeterBinder
|
||||
import io.micrometer.prometheus.PrometheusMeterRegistry
|
||||
import java.time.Duration
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
import java.util.concurrent.TimeUnit
|
||||
|
||||
/**
|
||||
* Custom application metrics configuration.
|
||||
*
|
||||
* Adds application-specific metrics for better monitoring:
|
||||
* - API endpoint response times
|
||||
* - Request counts by endpoint and status code
|
||||
* - Error rates
|
||||
* - Database query metrics
|
||||
*/
|
||||
|
||||
// Reference to the Prometheus registry from PrometheusConfig
|
||||
private val appRegistry: PrometheusMeterRegistry
|
||||
get() = at.mocode.gateway.config.appMicrometerRegistry
|
||||
|
||||
// Attribute key for request start time
|
||||
private val REQUEST_TIMER_ATTRIBUTE = AttributeKey<Timer.Sample>("RequestTimerSample")
|
||||
|
||||
// Cache for endpoint timers to avoid creating new ones for each request
|
||||
private val endpointTimers = ConcurrentHashMap<String, Timer>()
|
||||
|
||||
// Cache for endpoint counters
|
||||
private val endpointCounters = ConcurrentHashMap<Pair<String, Int>, Counter>()
|
||||
|
||||
// Cache for error counters
|
||||
private val errorCounters = ConcurrentHashMap<String, Counter>()
|
||||
|
||||
/**
|
||||
* Configures custom application metrics.
|
||||
*/
|
||||
fun Application.configureCustomMetrics() {
|
||||
// Install a hook to intercept all requests for timing
|
||||
intercept(ApplicationCallPipeline.Monitoring) {
|
||||
// Start timing the request
|
||||
val timerSample = Timer.start(appRegistry)
|
||||
call.attributes.put(REQUEST_TIMER_ATTRIBUTE, timerSample)
|
||||
}
|
||||
|
||||
// Install a hook to record metrics after the request is processed
|
||||
intercept(ApplicationCallPipeline.Fallback) {
|
||||
val status = call.response.status()?.value ?: 0
|
||||
val method = call.request.httpMethod.value
|
||||
val route = extractRoutePattern(call)
|
||||
|
||||
// Record request count
|
||||
getOrCreateRequestCounter(method, route, status).increment()
|
||||
|
||||
// Record timing
|
||||
call.attributes.getOrNull(REQUEST_TIMER_ATTRIBUTE)?.let { timerSample ->
|
||||
val timer = getOrCreateEndpointTimer(method, route)
|
||||
timerSample.stop(timer)
|
||||
}
|
||||
|
||||
// Record errors
|
||||
if (status >= 400) {
|
||||
getOrCreateErrorCounter(method, route, status).increment()
|
||||
}
|
||||
}
|
||||
|
||||
// Register database metrics
|
||||
registerDatabaseMetrics()
|
||||
|
||||
log.info("Custom application metrics configured")
|
||||
}
|
||||
|
||||
/**
|
||||
* Extracts a normalized route pattern from the call.
|
||||
* Converts dynamic path segments to a generic pattern.
|
||||
* For example: /api/users/123 -> /api/users/{id}
|
||||
*/
|
||||
private fun extractRoutePattern(call: ApplicationCall): String {
|
||||
val path = call.request.path()
|
||||
|
||||
// Try to get the route from the call attributes if available
|
||||
call.attributes.getOrNull(AttributeKey<Route>("ktor.request.route"))?.let { route ->
|
||||
return route.toString()
|
||||
}
|
||||
|
||||
// Otherwise, normalize the path by replacing likely IDs with {id}
|
||||
val segments = path.split("/")
|
||||
val normalizedSegments = segments.map { segment ->
|
||||
// If segment looks like an ID (UUID, number), replace with {id}
|
||||
if (segment.matches(Regex("[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}")) ||
|
||||
segment.matches(Regex("\\d+"))
|
||||
) {
|
||||
"{id}"
|
||||
} else {
|
||||
segment
|
||||
}
|
||||
}
|
||||
|
||||
return normalizedSegments.joinToString("/")
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets or creates a timer for the specified endpoint.
|
||||
*/
|
||||
private fun getOrCreateEndpointTimer(method: String, route: String): Timer {
|
||||
val key = "$method $route"
|
||||
return endpointTimers.computeIfAbsent(key) {
|
||||
Timer.builder("http.server.requests")
|
||||
.tag("method", method)
|
||||
.tag("route", route)
|
||||
.publishPercentileHistogram()
|
||||
.register(appRegistry)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets or creates a counter for the specified endpoint and status.
|
||||
*/
|
||||
private fun getOrCreateRequestCounter(method: String, route: String, status: Int): Counter {
|
||||
val key = Pair("$method $route", status)
|
||||
return endpointCounters.computeIfAbsent(key) {
|
||||
Counter.builder("http.server.requests.count")
|
||||
.tag("method", method)
|
||||
.tag("route", route)
|
||||
.tag("status", status.toString())
|
||||
.register(appRegistry)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets or creates an error counter for the specified endpoint and status.
|
||||
*/
|
||||
private fun getOrCreateErrorCounter(method: String, route: String, status: Int): Counter {
|
||||
val key = "$method $route $status"
|
||||
return errorCounters.computeIfAbsent(key) {
|
||||
Counter.builder("http.server.errors")
|
||||
.tag("method", method)
|
||||
.tag("route", route)
|
||||
.tag("status", status.toString())
|
||||
.register(appRegistry)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Registers database metrics.
|
||||
*/
|
||||
private fun registerDatabaseMetrics() {
|
||||
// Create a gauge for active connections
|
||||
appRegistry.gauge("db.connections.active",
|
||||
at.mocode.shared.database.DatabaseFactory,
|
||||
{ it.getActiveConnections().toDouble() })
|
||||
|
||||
// Create a gauge for idle connections
|
||||
appRegistry.gauge("db.connections.idle",
|
||||
at.mocode.shared.database.DatabaseFactory,
|
||||
{ it.getIdleConnections().toDouble() })
|
||||
|
||||
// Create a gauge for total connections
|
||||
appRegistry.gauge("db.connections.total",
|
||||
at.mocode.shared.database.DatabaseFactory,
|
||||
{ it.getTotalConnections().toDouble() })
|
||||
}
|
||||
|
|
@ -4,10 +4,19 @@ import at.mocode.dto.base.ApiResponse
|
|||
import at.mocode.shared.config.AppConfig
|
||||
import io.ktor.http.*
|
||||
import io.ktor.server.application.*
|
||||
import io.ktor.server.metrics.micrometer.*
|
||||
import io.ktor.server.plugins.calllogging.*
|
||||
import io.ktor.server.plugins.statuspages.*
|
||||
import io.ktor.server.request.*
|
||||
import io.ktor.server.response.*
|
||||
import io.ktor.server.routing.*
|
||||
import io.micrometer.core.instrument.binder.jvm.ClassLoaderMetrics
|
||||
import io.micrometer.core.instrument.binder.jvm.JvmGcMetrics
|
||||
import io.micrometer.core.instrument.binder.jvm.JvmMemoryMetrics
|
||||
import io.micrometer.core.instrument.binder.jvm.JvmThreadMetrics
|
||||
import io.micrometer.core.instrument.binder.system.ProcessorMetrics
|
||||
import io.micrometer.prometheus.PrometheusConfig
|
||||
import io.micrometer.prometheus.PrometheusMeterRegistry
|
||||
import org.slf4j.event.Level
|
||||
import java.time.LocalDateTime
|
||||
import java.time.format.DateTimeFormatter
|
||||
|
|
@ -26,30 +35,66 @@ import kotlin.random.Random
|
|||
*/
|
||||
|
||||
// Map to track request counts by path for log sampling
|
||||
private val requestCountsByPath = ConcurrentHashMap<String, AtomicInteger>()
|
||||
// Using a more efficient ConcurrentHashMap with initial capacity and load factor
|
||||
private val requestCountsByPath = ConcurrentHashMap<String, AtomicInteger>(32, 0.75f)
|
||||
|
||||
// Map to track high-traffic paths that are being sampled
|
||||
private val sampledPaths = ConcurrentHashMap<String, Boolean>()
|
||||
private val sampledPaths = ConcurrentHashMap<String, Boolean>(16, 0.75f)
|
||||
|
||||
// Scheduler to reset request counts periodically
|
||||
private val requestCountResetScheduler = Executors.newSingleThreadScheduledExecutor().apply {
|
||||
scheduleAtFixedRate({
|
||||
// Reset all counters every minute
|
||||
requestCountsByPath.clear()
|
||||
private val requestCountResetScheduler = Executors.newSingleThreadScheduledExecutor { r ->
|
||||
val thread = Thread(r, "log-sampling-reset-thread")
|
||||
thread.isDaemon = true // Make it a daemon thread so it doesn't prevent JVM shutdown
|
||||
thread
|
||||
}
|
||||
|
||||
// Log which paths are being sampled
|
||||
if (sampledPaths.isNotEmpty()) {
|
||||
val sampledPathsList = sampledPaths.keys.joinToString(", ")
|
||||
println("[LogSampling] Currently sampling high-traffic paths: $sampledPathsList")
|
||||
// Schedule the task with proper lifecycle management
|
||||
private fun scheduleRequestCountReset() {
|
||||
// Reset counters every 5 minutes instead of every minute to reduce overhead
|
||||
requestCountResetScheduler.scheduleAtFixedRate({
|
||||
try {
|
||||
// Reset all counters
|
||||
requestCountsByPath.clear()
|
||||
|
||||
// Log which paths are being sampled (only if there are any)
|
||||
if (sampledPaths.isNotEmpty()) {
|
||||
// More efficient string building for logging
|
||||
val sampledPathsCount = sampledPaths.size
|
||||
if (sampledPathsCount <= 5) {
|
||||
// For a small number of paths, log them all
|
||||
val sampledPathsList = sampledPaths.keys.joinToString(", ")
|
||||
println("[LogSampling] Currently sampling $sampledPathsCount high-traffic paths: $sampledPathsList")
|
||||
} else {
|
||||
// For many paths, just log the count to avoid excessive logging
|
||||
println("[LogSampling] Currently sampling $sampledPathsCount high-traffic paths")
|
||||
}
|
||||
}
|
||||
|
||||
// Clear sampled paths to re-evaluate in the next period
|
||||
sampledPaths.clear()
|
||||
} catch (e: Exception) {
|
||||
// Catch any exceptions to prevent the scheduler from stopping
|
||||
println("[LogSampling] Error in reset task: ${e.message}")
|
||||
}
|
||||
}, 5, 5, TimeUnit.MINUTES)
|
||||
}
|
||||
|
||||
// Clear sampled paths to re-evaluate in the next period
|
||||
sampledPaths.clear()
|
||||
}, 1, 1, TimeUnit.MINUTES)
|
||||
// Shutdown hook to clean up resources
|
||||
private fun shutdownRequestCountResetScheduler() {
|
||||
requestCountResetScheduler.shutdown()
|
||||
try {
|
||||
if (!requestCountResetScheduler.awaitTermination(5, TimeUnit.SECONDS)) {
|
||||
requestCountResetScheduler.shutdownNow()
|
||||
}
|
||||
} catch (e: InterruptedException) {
|
||||
requestCountResetScheduler.shutdownNow()
|
||||
Thread.currentThread().interrupt()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Determines if a request should be logged based on sampling configuration.
|
||||
* Optimized for performance with early returns and cached path normalization.
|
||||
*
|
||||
* @param path The request path
|
||||
* @param statusCode The response status code
|
||||
|
|
@ -57,24 +102,38 @@ private val requestCountResetScheduler = Executors.newSingleThreadScheduledExecu
|
|||
* @return True if the request should be logged, false otherwise
|
||||
*/
|
||||
private fun shouldLogRequest(path: String, statusCode: HttpStatusCode?, loggingConfig: at.mocode.shared.config.LoggingConfig): Boolean {
|
||||
// If sampling is disabled, always log
|
||||
// Fast path: If sampling is disabled, always log
|
||||
if (!loggingConfig.enableLogSampling) {
|
||||
return true
|
||||
}
|
||||
|
||||
// Always log errors if configured
|
||||
if (loggingConfig.alwaysLogErrors && statusCode != null && statusCode.value >= 400) {
|
||||
// Fast path: Always log errors if configured
|
||||
if (statusCode != null && statusCode.value >= 400 && loggingConfig.alwaysLogErrors) {
|
||||
return true
|
||||
}
|
||||
|
||||
// Always log specific paths if configured
|
||||
val normalizedPath = path.trimStart('/')
|
||||
if (loggingConfig.alwaysLogPaths.any { normalizedPath.startsWith(it.trimStart('/')) }) {
|
||||
return true
|
||||
// Check if this is a path that should always be logged
|
||||
// Only normalize the path if we have paths to check against
|
||||
if (loggingConfig.alwaysLogPaths.isNotEmpty()) {
|
||||
val normalizedPath = path.trimStart('/')
|
||||
// Use any with early return for better performance
|
||||
for (alwaysLogPath in loggingConfig.alwaysLogPaths) {
|
||||
if (normalizedPath.startsWith(alwaysLogPath.trimStart('/'))) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Get the base path for traffic counting
|
||||
val basePath = extractBasePath(path)
|
||||
|
||||
// Check if this path is already known to be high-traffic
|
||||
if (sampledPaths.containsKey(basePath)) {
|
||||
// Already identified as high-traffic, apply sampling
|
||||
return Random.nextInt(100) < loggingConfig.samplingRate
|
||||
}
|
||||
|
||||
// Get or create counter for this path
|
||||
val basePath = extractBasePath(path)
|
||||
val counter = requestCountsByPath.computeIfAbsent(basePath) { AtomicInteger(0) }
|
||||
val count = counter.incrementAndGet()
|
||||
|
||||
|
|
@ -114,6 +173,17 @@ private fun extractBasePath(path: String): String {
|
|||
fun Application.configureMonitoring() {
|
||||
val loggingConfig = AppConfig.logging
|
||||
|
||||
// Note: Prometheus metrics configuration has been moved to PrometheusConfig.kt
|
||||
|
||||
// Start the request count reset scheduler
|
||||
scheduleRequestCountReset()
|
||||
|
||||
// Register shutdown hook for application lifecycle
|
||||
this.monitor.subscribe(ApplicationStopPreparing) {
|
||||
log.info("Application stopping, shutting down schedulers...")
|
||||
shutdownRequestCountResetScheduler()
|
||||
}
|
||||
|
||||
// Erweiterte Call-Logging-Konfiguration
|
||||
install(CallLogging) {
|
||||
level = when (loggingConfig.level.uppercase()) {
|
||||
|
|
@ -143,78 +213,97 @@ fun Application.configureMonitoring() {
|
|||
val requestId: String = call.attributes.getOrNull(REQUEST_ID_KEY) ?: "no-request-id"
|
||||
|
||||
if (loggingConfig.useStructuredLogging) {
|
||||
// Strukturiertes Logging-Format
|
||||
buildString {
|
||||
append("timestamp=$timestamp ")
|
||||
append("method=$httpMethod ")
|
||||
append("path=$path ")
|
||||
append("status=$status ")
|
||||
append("client=$clientIp ")
|
||||
append("requestId=$requestId ")
|
||||
// Optimized structured logging format using StringBuilder with initial capacity
|
||||
// Estimate the initial capacity based on typical log entry size
|
||||
val initialCapacity = 256 +
|
||||
(if (loggingConfig.logRequestHeaders) 128 else 0) +
|
||||
(if (loggingConfig.logRequestParameters) 128 else 0)
|
||||
|
||||
// Log Headers wenn konfiguriert
|
||||
if (loggingConfig.logRequestHeaders) {
|
||||
val authHeader = call.request.headers["Authorization"]
|
||||
if (authHeader != null) {
|
||||
append("auth=true ")
|
||||
}
|
||||
val sb = StringBuilder(initialCapacity)
|
||||
|
||||
val contentType = call.request.headers["Content-Type"]
|
||||
if (contentType != null) {
|
||||
append("contentType=$contentType ")
|
||||
}
|
||||
// Basic request information - always included
|
||||
sb.append("timestamp=").append(timestamp).append(' ')
|
||||
.append("method=").append(httpMethod).append(' ')
|
||||
.append("path=").append(path).append(' ')
|
||||
.append("status=").append(status).append(' ')
|
||||
.append("client=").append(clientIp).append(' ')
|
||||
.append("requestId=").append(requestId).append(' ')
|
||||
|
||||
// Log all headers if in debug mode, filtering sensitive data
|
||||
if (loggingConfig.level.equals("DEBUG", ignoreCase = true)) {
|
||||
append("headers={")
|
||||
call.request.headers.entries().joinTo(this, ", ") { entry ->
|
||||
if (isSensitiveHeader(entry.key)) {
|
||||
"${entry.key}=*****"
|
||||
} else {
|
||||
"${entry.key}=${entry.value.joinToString(",")}"
|
||||
}
|
||||
}
|
||||
append("} ")
|
||||
}
|
||||
// Log Headers wenn konfiguriert
|
||||
if (loggingConfig.logRequestHeaders) {
|
||||
val authHeader = call.request.headers["Authorization"]
|
||||
if (authHeader != null) {
|
||||
sb.append("auth=true ")
|
||||
}
|
||||
|
||||
// Log Query-Parameter wenn konfiguriert
|
||||
if (loggingConfig.logRequestParameters && call.request.queryParameters.entries().isNotEmpty()) {
|
||||
append("params={")
|
||||
call.request.queryParameters.entries().joinTo(this, ", ") { entry ->
|
||||
if (isSensitiveParameter(entry.key)) {
|
||||
"${entry.key}=*****"
|
||||
val contentType = call.request.headers["Content-Type"]
|
||||
if (contentType != null) {
|
||||
sb.append("contentType=").append(contentType).append(' ')
|
||||
}
|
||||
|
||||
// Log all headers if in debug mode, filtering sensitive data
|
||||
if (loggingConfig.level.equals("DEBUG", ignoreCase = true)) {
|
||||
sb.append("headers={")
|
||||
var first = true
|
||||
for (entry in call.request.headers.entries()) {
|
||||
if (!first) sb.append(", ")
|
||||
first = false
|
||||
|
||||
if (isSensitiveHeader(entry.key)) {
|
||||
sb.append(entry.key).append("=*****")
|
||||
} else {
|
||||
"${entry.key}=${entry.value.joinToString(",")}"
|
||||
sb.append(entry.key).append('=').append(entry.value.joinToString(","))
|
||||
}
|
||||
}
|
||||
append("} ")
|
||||
sb.append("} ")
|
||||
}
|
||||
}
|
||||
|
||||
if (userAgent != null) {
|
||||
// Use a simpler approach to avoid escape sequence issues
|
||||
val escapedUserAgent = userAgent.replace("\"", "\\\"")
|
||||
append("userAgent=\"$escapedUserAgent\" ")
|
||||
// Log Query-Parameter wenn konfiguriert
|
||||
if (loggingConfig.logRequestParameters && call.request.queryParameters.entries().isNotEmpty()) {
|
||||
sb.append("params={")
|
||||
var first = true
|
||||
for (entry in call.request.queryParameters.entries()) {
|
||||
if (!first) sb.append(", ")
|
||||
first = false
|
||||
|
||||
if (isSensitiveParameter(entry.key)) {
|
||||
sb.append(entry.key).append("=*****")
|
||||
} else {
|
||||
sb.append(entry.key).append('=').append(entry.value.joinToString(","))
|
||||
}
|
||||
}
|
||||
sb.append("} ")
|
||||
}
|
||||
|
||||
// Log response time if available from RequestTracingConfig
|
||||
call.attributes.getOrNull(REQUEST_START_TIME_KEY)?.let { startTime: Long ->
|
||||
val duration = System.currentTimeMillis() - startTime
|
||||
append("duration=${duration}ms ")
|
||||
}
|
||||
if (userAgent != null) {
|
||||
// Use a simpler approach to avoid escape sequence issues
|
||||
val escapedUserAgent = userAgent.replace("\"", "\\\"")
|
||||
sb.append("userAgent=\"").append(escapedUserAgent).append("\" ")
|
||||
}
|
||||
|
||||
// Add performance metrics
|
||||
// Log response time if available from RequestTracingConfig
|
||||
call.attributes.getOrNull(REQUEST_START_TIME_KEY)?.let { startTime: Long ->
|
||||
val duration = System.currentTimeMillis() - startTime
|
||||
sb.append("duration=").append(duration).append("ms ")
|
||||
}
|
||||
|
||||
// Add performance metrics - only calculate memory usage if needed
|
||||
// Only include memory metrics in every 10th log entry to reduce overhead
|
||||
if (Random.nextInt(10) == 0) {
|
||||
val memoryUsage = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()
|
||||
append("memoryUsage=${memoryUsage}b ")
|
||||
sb.append("memoryUsage=").append(memoryUsage).append("b ")
|
||||
|
||||
// Add additional performance metrics in debug mode
|
||||
if (loggingConfig.level.equals("DEBUG", ignoreCase = true)) {
|
||||
val availableProcessors = Runtime.getRuntime().availableProcessors()
|
||||
val maxMemory = Runtime.getRuntime().maxMemory()
|
||||
append("processors=$availableProcessors ")
|
||||
append("maxMemory=${maxMemory}b ")
|
||||
sb.append("processors=").append(availableProcessors).append(' ')
|
||||
.append("maxMemory=").append(maxMemory).append("b ")
|
||||
}
|
||||
}
|
||||
|
||||
sb.toString()
|
||||
} else {
|
||||
// Einfaches Logging-Format
|
||||
val duration = call.attributes.getOrNull(REQUEST_START_TIME_KEY)?.let { startTime: Long ->
|
||||
|
|
|
|||
|
|
@ -0,0 +1,53 @@
|
|||
package at.mocode.gateway.config
|
||||
|
||||
import io.ktor.server.application.*
|
||||
import io.ktor.server.metrics.micrometer.*
|
||||
import io.ktor.server.response.*
|
||||
import io.ktor.server.routing.*
|
||||
import io.ktor.server.auth.*
|
||||
import io.micrometer.core.instrument.binder.jvm.ClassLoaderMetrics
|
||||
import io.micrometer.core.instrument.binder.jvm.JvmGcMetrics
|
||||
import io.micrometer.core.instrument.binder.jvm.JvmMemoryMetrics
|
||||
import io.micrometer.core.instrument.binder.jvm.JvmThreadMetrics
|
||||
import io.micrometer.core.instrument.binder.system.ProcessorMetrics
|
||||
import io.micrometer.prometheus.PrometheusConfig
|
||||
import io.micrometer.prometheus.PrometheusMeterRegistry
|
||||
|
||||
/**
|
||||
* Prometheus metrics configuration for the API Gateway.
|
||||
*
|
||||
* Configures Micrometer with Prometheus registry and exposes a metrics endpoint.
|
||||
*/
|
||||
|
||||
// Create a Prometheus registry
|
||||
val appMicrometerRegistry = PrometheusMeterRegistry(PrometheusConfig.DEFAULT)
|
||||
|
||||
/**
|
||||
* Configures Prometheus metrics for the application.
|
||||
*/
|
||||
fun Application.configurePrometheusMetrics() {
|
||||
// Install Micrometer metrics
|
||||
install(MicrometerMetrics) {
|
||||
registry = appMicrometerRegistry
|
||||
// JVM metrics
|
||||
meterBinders = listOf(
|
||||
JvmMemoryMetrics(),
|
||||
JvmGcMetrics(),
|
||||
JvmThreadMetrics(),
|
||||
ClassLoaderMetrics(),
|
||||
ProcessorMetrics()
|
||||
)
|
||||
}
|
||||
|
||||
// Add a route to expose Prometheus metrics with basic authentication
|
||||
routing {
|
||||
// Secure metrics endpoint with basic authentication
|
||||
authenticate("metrics-auth") {
|
||||
get("/metrics") {
|
||||
call.respond(appMicrometerRegistry.scrape())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
log.info("Prometheus metrics configured and secured at /metrics endpoint")
|
||||
}
|
||||
|
|
@ -58,6 +58,22 @@ fun Application.configureSecurity() {
|
|||
call.respond(HttpStatusCode.Unauthorized, "Token is not valid or has expired")
|
||||
}
|
||||
}
|
||||
|
||||
// Basic authentication for metrics endpoint
|
||||
basic("metrics-auth") {
|
||||
realm = "Metrics"
|
||||
validate { credentials ->
|
||||
// Get credentials from environment variables or use defaults
|
||||
val metricsUser = System.getenv("METRICS_USER") ?: "metrics"
|
||||
val metricsPassword = System.getenv("METRICS_PASSWORD") ?: "metrics-password-change-in-production"
|
||||
|
||||
if (credentials.name == metricsUser && credentials.password == metricsPassword) {
|
||||
UserIdPrincipal(credentials.name)
|
||||
} else {
|
||||
null
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,181 @@
|
|||
package at.mocode.gateway.discovery
|
||||
|
||||
import io.ktor.client.*
|
||||
import io.ktor.client.engine.cio.*
|
||||
import io.ktor.client.plugins.contentnegotiation.*
|
||||
import io.ktor.client.request.*
|
||||
import io.ktor.client.statement.*
|
||||
import io.ktor.http.*
|
||||
import io.ktor.serialization.kotlinx.json.*
|
||||
import kotlinx.coroutines.sync.Mutex
|
||||
import kotlinx.coroutines.sync.withLock
|
||||
import kotlinx.serialization.Serializable
|
||||
import kotlinx.serialization.json.Json
|
||||
import java.net.URI
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
|
||||
/**
|
||||
* Service discovery component for the API Gateway.
|
||||
* Uses Consul to discover services and route requests to them.
|
||||
*/
|
||||
class ServiceDiscovery(
|
||||
private val consulHost: String = "consul",
|
||||
private val consulPort: Int = 8500
|
||||
) {
|
||||
private val httpClient = HttpClient(CIO) {
|
||||
install(ContentNegotiation) {
|
||||
json(Json {
|
||||
ignoreUnknownKeys = true
|
||||
isLenient = true
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Cache of service instances
|
||||
private val serviceCache = ConcurrentHashMap<String, List<ServiceInstance>>()
|
||||
private val cacheMutex = Mutex()
|
||||
|
||||
// Default TTL for cache entries in milliseconds (30 seconds)
|
||||
private val cacheTtl = 30_000L
|
||||
private val cacheTimestamps = ConcurrentHashMap<String, Long>()
|
||||
|
||||
/**
|
||||
* Get a service instance for the given service name.
|
||||
* Uses a simple round-robin load balancing strategy.
|
||||
*
|
||||
* @param serviceName The name of the service to get an instance for
|
||||
* @return A service instance, or null if no instances are available
|
||||
*/
|
||||
suspend fun getServiceInstance(serviceName: String): ServiceInstance? {
|
||||
val instances = getServiceInstances(serviceName)
|
||||
if (instances.isEmpty()) {
|
||||
return null
|
||||
}
|
||||
|
||||
// Simple round-robin load balancing
|
||||
val index = (System.currentTimeMillis() % instances.size).toInt()
|
||||
return instances[index]
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all instances of a service.
|
||||
*
|
||||
* @param serviceName The name of the service to get instances for
|
||||
* @return A list of service instances
|
||||
*/
|
||||
suspend fun getServiceInstances(serviceName: String): List<ServiceInstance> {
|
||||
// Check cache first
|
||||
val cachedInstances = serviceCache[serviceName]
|
||||
val timestamp = cacheTimestamps[serviceName] ?: 0
|
||||
|
||||
if (cachedInstances != null && System.currentTimeMillis() - timestamp < cacheTtl) {
|
||||
return cachedInstances
|
||||
}
|
||||
|
||||
// Cache miss or expired, fetch from Consul
|
||||
return cacheMutex.withLock {
|
||||
// Double-check in case another thread updated the cache while we were waiting
|
||||
val currentTimestamp = cacheTimestamps[serviceName] ?: 0
|
||||
if (serviceCache[serviceName] != null && System.currentTimeMillis() - currentTimestamp < cacheTtl) {
|
||||
return@withLock serviceCache[serviceName]!!
|
||||
}
|
||||
|
||||
try {
|
||||
val instances = fetchServiceInstances(serviceName)
|
||||
serviceCache[serviceName] = instances
|
||||
cacheTimestamps[serviceName] = System.currentTimeMillis()
|
||||
instances
|
||||
} catch (e: Exception) {
|
||||
println("Failed to fetch service instances for $serviceName: ${e.message}")
|
||||
e.printStackTrace()
|
||||
|
||||
// Return cached instances if available, even if expired
|
||||
cachedInstances ?: emptyList()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch service instances from Consul.
|
||||
*
|
||||
* @param serviceName The name of the service to fetch instances for
|
||||
* @return A list of service instances
|
||||
*/
|
||||
private suspend fun fetchServiceInstances(serviceName: String): List<ServiceInstance> {
|
||||
val response = httpClient.get("http://$consulHost:$consulPort/v1/catalog/service/$serviceName")
|
||||
|
||||
if (response.status != HttpStatusCode.OK) {
|
||||
throw Exception("Failed to fetch service instances: ${response.status}")
|
||||
}
|
||||
|
||||
val responseBody = response.bodyAsText()
|
||||
val consulServices = Json.decodeFromString<List<ConsulService>>(responseBody)
|
||||
|
||||
return consulServices.map { service ->
|
||||
ServiceInstance(
|
||||
id = service.ServiceID,
|
||||
name = service.ServiceName,
|
||||
host = service.ServiceAddress.ifEmpty { service.Address },
|
||||
port = service.ServicePort,
|
||||
tags = service.ServiceTags,
|
||||
meta = service.ServiceMeta
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a URL for a service instance.
|
||||
*
|
||||
* @param instance The service instance
|
||||
* @param path The path to append to the URL
|
||||
* @return The complete URL
|
||||
*/
|
||||
fun buildServiceUrl(instance: ServiceInstance, path: String): String {
|
||||
val baseUrl = "http://${instance.host}:${instance.port}"
|
||||
return URI(baseUrl).resolve(path).toString()
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a service is healthy.
|
||||
*
|
||||
* @param serviceName The name of the service to check
|
||||
* @return True if the service is healthy, false otherwise
|
||||
*/
|
||||
suspend fun isServiceHealthy(serviceName: String): Boolean {
|
||||
try {
|
||||
val response = httpClient.get("http://$consulHost:$consulPort/v1/health/service/$serviceName?passing=true")
|
||||
val responseBody = response.bodyAsText()
|
||||
val healthyServices = Json.decodeFromString<List<Any>>(responseBody)
|
||||
return healthyServices.isNotEmpty()
|
||||
} catch (e: Exception) {
|
||||
println("Failed to check service health for $serviceName: ${e.message}")
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Represents a service instance.
|
||||
*/
|
||||
data class ServiceInstance(
|
||||
val id: String,
|
||||
val name: String,
|
||||
val host: String,
|
||||
val port: Int,
|
||||
val tags: List<String> = emptyList(),
|
||||
val meta: Map<String, String> = emptyMap()
|
||||
)
|
||||
|
||||
/**
|
||||
* Consul service response model.
|
||||
*/
|
||||
@Serializable
|
||||
data class ConsulService(
|
||||
val ServiceID: String,
|
||||
val ServiceName: String,
|
||||
val ServiceAddress: String,
|
||||
val ServicePort: Int,
|
||||
val ServiceTags: List<String> = emptyList(),
|
||||
val ServiceMeta: Map<String, String> = emptyMap(),
|
||||
val Address: String
|
||||
)
|
||||
|
|
@ -1,7 +1,11 @@
|
|||
package at.mocode.gateway
|
||||
|
||||
import at.mocode.gateway.config.*
|
||||
import at.mocode.gateway.config.configurePrometheusMetrics
|
||||
import at.mocode.gateway.config.configureCustomMetrics
|
||||
import at.mocode.gateway.plugins.configureHttpCaching
|
||||
import at.mocode.gateway.routing.docRoutes
|
||||
import at.mocode.gateway.routing.serviceRoutes
|
||||
import at.mocode.shared.config.AppConfig
|
||||
import io.ktor.http.*
|
||||
import io.ktor.serialization.kotlinx.json.*
|
||||
|
|
@ -43,6 +47,12 @@ fun Application.module() {
|
|||
// Erweiterte Monitoring- und Logging-Konfiguration
|
||||
configureMonitoring()
|
||||
|
||||
// Prometheus Metrics konfigurieren
|
||||
configurePrometheusMetrics()
|
||||
|
||||
// Custom application metrics konfigurieren
|
||||
configureCustomMetrics()
|
||||
|
||||
// Request Tracing für Cross-Service Tracing konfigurieren
|
||||
configureRequestTracing()
|
||||
|
||||
|
|
@ -53,6 +63,9 @@ fun Application.module() {
|
|||
configureOpenApi()
|
||||
configureSwagger()
|
||||
|
||||
// HTTP Caching konfigurieren
|
||||
configureHttpCaching()
|
||||
|
||||
routing {
|
||||
// Hauptrouten
|
||||
get("/") {
|
||||
|
|
@ -62,6 +75,18 @@ fun Application.module() {
|
|||
)
|
||||
}
|
||||
|
||||
// Health check endpoint
|
||||
get("/health") {
|
||||
call.respond(HttpStatusCode.OK, mapOf(
|
||||
"status" to "UP",
|
||||
"timestamp" to System.currentTimeMillis(),
|
||||
"services" to mapOf(
|
||||
"api-gateway" to "UP",
|
||||
"database" to "UP"
|
||||
)
|
||||
))
|
||||
}
|
||||
|
||||
// Static resources for documentation
|
||||
staticResources("/docs", "static/docs") {
|
||||
default("index.html")
|
||||
|
|
@ -69,5 +94,8 @@ fun Application.module() {
|
|||
|
||||
// API Documentation routes
|
||||
docRoutes()
|
||||
|
||||
// Service discovery routes
|
||||
serviceRoutes()
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,243 @@
|
|||
package at.mocode.gateway.plugins
|
||||
|
||||
import at.mocode.gateway.config.CachingConfig
|
||||
import at.mocode.gateway.config.getCachingConfig
|
||||
import io.ktor.http.*
|
||||
import io.ktor.server.application.*
|
||||
import io.ktor.server.request.*
|
||||
import io.ktor.server.response.*
|
||||
import io.ktor.util.pipeline.*
|
||||
import java.security.MessageDigest
|
||||
import java.text.SimpleDateFormat
|
||||
import java.util.*
|
||||
import kotlin.text.Charsets
|
||||
|
||||
/**
|
||||
* Configures enhanced HTTP caching headers for the application.
|
||||
* This adds Cache-Control, Expires, and Vary headers to responses.
|
||||
* It also integrates with the CachingConfig for more intelligent caching decisions.
|
||||
*/
|
||||
fun Application.configureHttpCaching() {
|
||||
// Get the application logger
|
||||
val logger = log
|
||||
|
||||
// Get the caching config
|
||||
val cachingConfig = try {
|
||||
getCachingConfig()
|
||||
} catch (e: Exception) {
|
||||
logger.warn("Failed to get CachingConfig, using default caching headers: ${e.message}")
|
||||
null
|
||||
}
|
||||
|
||||
// Add a response interceptor for setting cache headers
|
||||
intercept(ApplicationCallPipeline.Call) {
|
||||
// Add Vary header to all responses
|
||||
call.response.header(HttpHeaders.Vary, "Accept, Accept-Encoding")
|
||||
|
||||
// For authenticated endpoints, add Authorization to Vary
|
||||
if (call.request.headers.contains(HttpHeaders.Authorization)) {
|
||||
call.response.header(HttpHeaders.Vary, "Accept, Accept-Encoding, Authorization")
|
||||
}
|
||||
|
||||
// Set default no-cache headers for dynamic content
|
||||
call.response.header(HttpHeaders.CacheControl, "no-cache, private")
|
||||
|
||||
// Check for conditional requests (If-None-Match, If-Modified-Since)
|
||||
val requestETag = call.request.header(HttpHeaders.IfNoneMatch)
|
||||
val requestLastModified = call.request.header(HttpHeaders.IfModifiedSince)
|
||||
|
||||
// If we have conditional headers, check if we can return 304 Not Modified
|
||||
if (requestETag != null || requestLastModified != null) {
|
||||
// This would be implemented with actual ETag and Last-Modified checking
|
||||
// For now, we just log that we received conditional headers
|
||||
logger.debug("Received conditional request: ETag=$requestETag, Last-Modified=$requestLastModified")
|
||||
}
|
||||
}
|
||||
|
||||
logger.info("HTTP caching configured with integration to CachingConfig")
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension function to enable caching for static resources.
|
||||
* Use this for CSS, JS, images, and other static files.
|
||||
*/
|
||||
fun ApplicationCall.enableStaticResourceCaching(maxAgeSeconds: Int = 86400) { // Default: 1 day
|
||||
setCacheControlHeader(this, maxAgeSeconds, true)
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension function to enable caching for master data.
|
||||
* Use this for reference data that changes infrequently.
|
||||
*/
|
||||
fun ApplicationCall.enableMasterDataCaching(maxAgeSeconds: Int = 3600) { // Default: 1 hour
|
||||
setCacheControlHeader(this, maxAgeSeconds, true)
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension function to enable caching for user data.
|
||||
* Use this for user-specific data that may change frequently.
|
||||
*/
|
||||
fun ApplicationCall.enableUserDataCaching(maxAgeSeconds: Int = 60) { // Default: 1 minute
|
||||
setCacheControlHeader(this, maxAgeSeconds, false, true)
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension function to disable caching.
|
||||
* Use this for sensitive or frequently changing data.
|
||||
*/
|
||||
fun ApplicationCall.disableCaching() {
|
||||
response.header(HttpHeaders.CacheControl, "no-cache, no-store, must-revalidate, private")
|
||||
response.header(HttpHeaders.Pragma, "no-cache")
|
||||
response.header(HttpHeaders.Expires, "0")
|
||||
}
|
||||
|
||||
/**
|
||||
* Helper function to set Cache-Control and Expires headers.
|
||||
*/
|
||||
private fun setCacheControlHeader(
|
||||
call: ApplicationCall,
|
||||
maxAgeSeconds: Int,
|
||||
isPublic: Boolean,
|
||||
mustRevalidate: Boolean = false
|
||||
) {
|
||||
// Build Cache-Control header
|
||||
val visibility = if (isPublic) "public" else "private"
|
||||
val revalidate = if (mustRevalidate) ", must-revalidate" else ""
|
||||
call.response.header(
|
||||
HttpHeaders.CacheControl,
|
||||
"max-age=$maxAgeSeconds, $visibility$revalidate"
|
||||
)
|
||||
|
||||
// Set Expires header
|
||||
val calendar = Calendar.getInstance()
|
||||
calendar.add(Calendar.SECOND, maxAgeSeconds)
|
||||
val dateFormat = SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z", Locale.US)
|
||||
dateFormat.timeZone = TimeZone.getTimeZone("GMT")
|
||||
call.response.header(HttpHeaders.Expires, dateFormat.format(calendar.time))
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension function to set ETag header for a response.
|
||||
*/
|
||||
fun ApplicationCall.setETag(etag: String) {
|
||||
response.header(HttpHeaders.ETag, "\"$etag\"")
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension function to set Last-Modified header for a response.
|
||||
*/
|
||||
fun ApplicationCall.setLastModified(timestamp: Long) {
|
||||
val dateFormat = SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z", Locale.US)
|
||||
dateFormat.timeZone = TimeZone.getTimeZone("GMT")
|
||||
response.header(HttpHeaders.LastModified, dateFormat.format(Date(timestamp)))
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate an ETag for the given content.
|
||||
* This uses MD5 hashing for simplicity, but in production you might want to use a faster algorithm.
|
||||
*/
|
||||
fun generateETag(content: String): String {
|
||||
val md = MessageDigest.getInstance("MD5")
|
||||
val digest = md.digest(content.toByteArray(Charsets.UTF_8))
|
||||
return digest.joinToString("") { "%02x".format(it) }
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate an ETag for the given object by converting it to a string representation.
|
||||
*/
|
||||
fun generateETag(obj: Any): String {
|
||||
return generateETag(obj.toString())
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the request has a matching ETag and return 304 Not Modified if it does.
|
||||
* Returns true if the response was handled (304 sent), false otherwise.
|
||||
*/
|
||||
suspend fun PipelineContext<Unit, ApplicationCall>.checkETagAndRespond(etag: String): Boolean {
|
||||
val requestETag = call.request.header(HttpHeaders.IfNoneMatch)
|
||||
|
||||
// If the client sent an If-None-Match header and it matches our ETag,
|
||||
// we can return 304 Not Modified
|
||||
if (requestETag != null && (requestETag == "\"$etag\"" || requestETag == "*")) {
|
||||
call.response.header(HttpHeaders.ETag, "\"$etag\"")
|
||||
call.respond(HttpStatusCode.NotModified)
|
||||
return true
|
||||
}
|
||||
|
||||
// Set the ETag header for the response
|
||||
call.response.header(HttpHeaders.ETag, "\"$etag\"")
|
||||
return false
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the request has a matching Last-Modified date and return 304 Not Modified if it does.
|
||||
* Returns true if the response was handled (304 sent), false otherwise.
|
||||
*/
|
||||
suspend fun PipelineContext<Unit, ApplicationCall>.checkLastModifiedAndRespond(timestamp: Long): Boolean {
|
||||
val requestLastModified = call.request.header(HttpHeaders.IfModifiedSince)
|
||||
|
||||
if (requestLastModified != null) {
|
||||
try {
|
||||
val dateFormat = SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z", Locale.US)
|
||||
dateFormat.timeZone = TimeZone.getTimeZone("GMT")
|
||||
val requestDate = dateFormat.parse(requestLastModified).time
|
||||
|
||||
// If the resource hasn't been modified since the date in the request,
|
||||
// we can return 304 Not Modified
|
||||
if (timestamp <= requestDate) {
|
||||
val lastModifiedFormatted = dateFormat.format(Date(timestamp))
|
||||
call.response.header(HttpHeaders.LastModified, lastModifiedFormatted)
|
||||
call.respond(HttpStatusCode.NotModified)
|
||||
return true
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
// If we can't parse the date, ignore it
|
||||
}
|
||||
}
|
||||
|
||||
// Set the Last-Modified header for the response
|
||||
val dateFormat = SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z", Locale.US)
|
||||
dateFormat.timeZone = TimeZone.getTimeZone("GMT")
|
||||
call.response.header(HttpHeaders.LastModified, dateFormat.format(Date(timestamp)))
|
||||
return false
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension function to check if a resource is cached in CachingConfig.
|
||||
* If it is, and the client has a matching ETag or Last-Modified date,
|
||||
* this will return 304 Not Modified. Otherwise, it will return the cached value.
|
||||
* Returns true if the response was handled, false otherwise.
|
||||
*/
|
||||
suspend fun <T> PipelineContext<Unit, ApplicationCall>.checkCacheAndRespond(
|
||||
cacheName: String,
|
||||
key: String,
|
||||
etag: String? = null,
|
||||
lastModified: Long? = null
|
||||
): Boolean {
|
||||
val application = call.application
|
||||
val cachingConfig = try {
|
||||
application.getCachingConfig()
|
||||
} catch (e: Exception) {
|
||||
return false
|
||||
}
|
||||
|
||||
// Check if the resource is in the cache
|
||||
val cachedValue = cachingConfig.get<T>(cacheName, key)
|
||||
if (cachedValue != null) {
|
||||
// If we have an ETag, check if the client has a matching one
|
||||
if (etag != null && checkETagAndRespond(etag)) {
|
||||
return true
|
||||
}
|
||||
|
||||
// If we have a Last-Modified date, check if the client has a matching one
|
||||
if (lastModified != null && checkLastModifiedAndRespond(lastModified)) {
|
||||
return true
|
||||
}
|
||||
|
||||
// If we get here, the client doesn't have a matching ETag or Last-Modified date,
|
||||
// so we need to send the full response
|
||||
return false
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
|
@ -0,0 +1,96 @@
|
|||
package at.mocode.gateway.routing
|
||||
|
||||
import at.mocode.gateway.discovery.ServiceDiscovery
|
||||
import at.mocode.shared.config.AppConfig
|
||||
import io.ktor.http.*
|
||||
import io.ktor.server.application.*
|
||||
import io.ktor.server.response.*
|
||||
import io.ktor.server.routing.*
|
||||
|
||||
/**
|
||||
* Configure dynamic service routing using Consul service discovery.
|
||||
* This allows the API Gateway to discover services registered with Consul and route requests to them.
|
||||
*/
|
||||
fun Routing.serviceRoutes() {
|
||||
val config = AppConfig
|
||||
|
||||
// Initialize service discovery if enabled
|
||||
val serviceDiscovery = if (config.serviceDiscovery.enabled) {
|
||||
ServiceDiscovery(
|
||||
consulHost = config.serviceDiscovery.consulHost,
|
||||
consulPort = config.serviceDiscovery.consulPort
|
||||
)
|
||||
} else null
|
||||
|
||||
// Define service routes
|
||||
if (serviceDiscovery != null) {
|
||||
// Master Data Service Routes
|
||||
route("/api/masterdata") {
|
||||
handle {
|
||||
handleServiceRequest(call, "master-data", serviceDiscovery)
|
||||
}
|
||||
}
|
||||
|
||||
// Horse Registry Service Routes
|
||||
route("/api/horses") {
|
||||
handle {
|
||||
handleServiceRequest(call, "horse-registry", serviceDiscovery)
|
||||
}
|
||||
}
|
||||
|
||||
// Event Management Service Routes
|
||||
route("/api/events") {
|
||||
handle {
|
||||
handleServiceRequest(call, "event-management", serviceDiscovery)
|
||||
}
|
||||
}
|
||||
|
||||
// Member Management Service Routes
|
||||
route("/api/members") {
|
||||
handle {
|
||||
handleServiceRequest(call, "member-management", serviceDiscovery)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle a service request by discovering the service and forwarding the request.
|
||||
* This is a simplified implementation that just returns service information.
|
||||
* In a production environment, this would forward the request to the service.
|
||||
*/
|
||||
private suspend fun handleServiceRequest(
|
||||
call: ApplicationCall,
|
||||
serviceName: String,
|
||||
serviceDiscovery: ServiceDiscovery
|
||||
) {
|
||||
try {
|
||||
// Get service instance
|
||||
val serviceInstance = serviceDiscovery.getServiceInstance(serviceName)
|
||||
|
||||
if (serviceInstance == null) {
|
||||
call.respond(HttpStatusCode.ServiceUnavailable, "Service $serviceName is not available")
|
||||
return
|
||||
}
|
||||
|
||||
// Respond with service information
|
||||
call.respond(
|
||||
HttpStatusCode.OK,
|
||||
mapOf(
|
||||
"message" to "Service discovery working",
|
||||
"service" to serviceName,
|
||||
"instance" to mapOf(
|
||||
"id" to serviceInstance.id,
|
||||
"name" to serviceInstance.name,
|
||||
"host" to serviceInstance.host,
|
||||
"port" to serviceInstance.port
|
||||
)
|
||||
)
|
||||
)
|
||||
} catch (e: Exception) {
|
||||
call.respond(
|
||||
HttpStatusCode.InternalServerError,
|
||||
"Error routing request to service $serviceName: ${e.message}"
|
||||
)
|
||||
}
|
||||
}
|
||||
|
|
@ -10,6 +10,44 @@ plugins {
|
|||
alias(libs.plugins.compose.compiler) apply false
|
||||
}
|
||||
|
||||
// Apply dependency locking to all subprojects
|
||||
subprojects {
|
||||
// Enable dependency locking for all configurations
|
||||
dependencyLocking {
|
||||
lockAllConfigurations()
|
||||
}
|
||||
|
||||
// Add task to write lock files
|
||||
tasks.register("resolveAndLockAll") {
|
||||
doFirst {
|
||||
require(gradle.startParameter.isWriteDependencyLocks)
|
||||
}
|
||||
doLast {
|
||||
configurations.filter {
|
||||
// Only lock configurations that can be resolved
|
||||
it.isCanBeResolved
|
||||
}.forEach { it.resolve() }
|
||||
}
|
||||
}
|
||||
|
||||
// Configure Kotlin compiler options
|
||||
tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile>().configureEach {
|
||||
kotlinOptions {
|
||||
// Add any compiler arguments here if needed
|
||||
// The -Xbuild-cache-if-possible flag has been removed as it's not supported in Kotlin 2.1.x
|
||||
}
|
||||
}
|
||||
|
||||
// Configure parallel test execution
|
||||
tasks.withType<Test>().configureEach {
|
||||
// Enable parallel test execution
|
||||
maxParallelForks = (Runtime.getRuntime().availableProcessors() / 2).takeIf { it > 0 } ?: 1
|
||||
|
||||
// Optimize JVM args for tests
|
||||
jvmArgs = listOf("-Xmx512m", "-XX:+UseG1GC")
|
||||
}
|
||||
}
|
||||
|
||||
// Wrapper task configuration for the root project
|
||||
tasks.wrapper {
|
||||
gradleVersion = "8.14"
|
||||
|
|
|
|||
13
config/monitoring/elk/elasticsearch.yml
Normal file
13
config/monitoring/elk/elasticsearch.yml
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
---
|
||||
## Default Elasticsearch configuration
|
||||
cluster.name: "meldestelle-elk"
|
||||
network.host: 0.0.0.0
|
||||
|
||||
# Minimum memory requirements
|
||||
discovery.type: single-node
|
||||
|
||||
# X-Pack security disabled for development
|
||||
xpack.security.enabled: false
|
||||
|
||||
# Enable monitoring
|
||||
xpack.monitoring.collection.enabled: true
|
||||
51
config/monitoring/elk/logstash.conf
Normal file
51
config/monitoring/elk/logstash.conf
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
input {
|
||||
# TCP input for logback appender
|
||||
tcp {
|
||||
port => 5000
|
||||
codec => json_lines
|
||||
}
|
||||
|
||||
# File input for server logs
|
||||
file {
|
||||
path => "/var/log/meldestelle/*.log"
|
||||
start_position => "beginning"
|
||||
sincedb_path => "/dev/null"
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
if [type] == "syslog" {
|
||||
grok {
|
||||
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
|
||||
add_field => [ "received_at", "%{@timestamp}" ]
|
||||
add_field => [ "received_from", "%{host}" ]
|
||||
}
|
||||
date {
|
||||
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
|
||||
}
|
||||
}
|
||||
|
||||
# Parse JSON logs
|
||||
if [message] =~ /^\{.*\}$/ {
|
||||
json {
|
||||
source => "message"
|
||||
}
|
||||
}
|
||||
|
||||
# Add application name
|
||||
mutate {
|
||||
add_field => { "application" => "meldestelle" }
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
elasticsearch {
|
||||
hosts => ["elasticsearch:9200"]
|
||||
index => "meldestelle-logs-%{+YYYY.MM.dd}"
|
||||
}
|
||||
|
||||
# For debugging
|
||||
stdout {
|
||||
codec => rubydebug
|
||||
}
|
||||
}
|
||||
659
config/monitoring/grafana/dashboards/jvm-dashboard.json
Normal file
659
config/monitoring/grafana/dashboards/jvm-dashboard.json
Normal file
|
|
@ -0,0 +1,659 @@
|
|||
{
|
||||
"annotations": {
|
||||
"list": [
|
||||
{
|
||||
"builtIn": 1,
|
||||
"datasource": {
|
||||
"type": "grafana",
|
||||
"uid": "-- Grafana --"
|
||||
},
|
||||
"enable": true,
|
||||
"hide": true,
|
||||
"iconColor": "rgba(0, 211, 255, 1)",
|
||||
"name": "Annotations & Alerts",
|
||||
"type": "dashboard"
|
||||
}
|
||||
]
|
||||
},
|
||||
"editable": true,
|
||||
"fiscalYearStartMonth": 0,
|
||||
"graphTooltip": 0,
|
||||
"id": 1,
|
||||
"links": [],
|
||||
"liveNow": false,
|
||||
"panels": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "palette-classic"
|
||||
},
|
||||
"custom": {
|
||||
"axisCenteredZero": false,
|
||||
"axisColorMode": "text",
|
||||
"axisLabel": "",
|
||||
"axisPlacement": "auto",
|
||||
"barAlignment": 0,
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 10,
|
||||
"gradientMode": "none",
|
||||
"hideFrom": {
|
||||
"legend": false,
|
||||
"tooltip": false,
|
||||
"viz": false
|
||||
},
|
||||
"lineInterpolation": "linear",
|
||||
"lineWidth": 1,
|
||||
"pointSize": 5,
|
||||
"scaleDistribution": {
|
||||
"type": "linear"
|
||||
},
|
||||
"showPoints": "never",
|
||||
"spanNulls": false,
|
||||
"stacking": {
|
||||
"group": "A",
|
||||
"mode": "none"
|
||||
},
|
||||
"thresholdsStyle": {
|
||||
"mode": "off"
|
||||
}
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
},
|
||||
"unit": "bytes"
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 0
|
||||
},
|
||||
"id": 1,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": [],
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "jvm_memory_used_bytes{area=\"heap\"}",
|
||||
"legendFormat": "Used Heap",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "jvm_memory_committed_bytes{area=\"heap\"}",
|
||||
"hide": false,
|
||||
"legendFormat": "Committed Heap",
|
||||
"range": true,
|
||||
"refId": "B"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "jvm_memory_max_bytes{area=\"heap\"}",
|
||||
"hide": false,
|
||||
"legendFormat": "Max Heap",
|
||||
"range": true,
|
||||
"refId": "C"
|
||||
}
|
||||
],
|
||||
"title": "JVM Heap Memory",
|
||||
"type": "timeseries"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "palette-classic"
|
||||
},
|
||||
"custom": {
|
||||
"axisCenteredZero": false,
|
||||
"axisColorMode": "text",
|
||||
"axisLabel": "",
|
||||
"axisPlacement": "auto",
|
||||
"barAlignment": 0,
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 10,
|
||||
"gradientMode": "none",
|
||||
"hideFrom": {
|
||||
"legend": false,
|
||||
"tooltip": false,
|
||||
"viz": false
|
||||
},
|
||||
"lineInterpolation": "linear",
|
||||
"lineWidth": 1,
|
||||
"pointSize": 5,
|
||||
"scaleDistribution": {
|
||||
"type": "linear"
|
||||
},
|
||||
"showPoints": "never",
|
||||
"spanNulls": false,
|
||||
"stacking": {
|
||||
"group": "A",
|
||||
"mode": "none"
|
||||
},
|
||||
"thresholdsStyle": {
|
||||
"mode": "off"
|
||||
}
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
},
|
||||
"unit": "bytes"
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 0
|
||||
},
|
||||
"id": 2,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": [],
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "jvm_memory_used_bytes{area=\"nonheap\"}",
|
||||
"legendFormat": "Used Non-Heap",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "jvm_memory_committed_bytes{area=\"nonheap\"}",
|
||||
"hide": false,
|
||||
"legendFormat": "Committed Non-Heap",
|
||||
"range": true,
|
||||
"refId": "B"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "jvm_memory_max_bytes{area=\"nonheap\"}",
|
||||
"hide": false,
|
||||
"legendFormat": "Max Non-Heap",
|
||||
"range": true,
|
||||
"refId": "C"
|
||||
}
|
||||
],
|
||||
"title": "JVM Non-Heap Memory",
|
||||
"type": "timeseries"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "palette-classic"
|
||||
},
|
||||
"custom": {
|
||||
"axisCenteredZero": false,
|
||||
"axisColorMode": "text",
|
||||
"axisLabel": "",
|
||||
"axisPlacement": "auto",
|
||||
"barAlignment": 0,
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 10,
|
||||
"gradientMode": "none",
|
||||
"hideFrom": {
|
||||
"legend": false,
|
||||
"tooltip": false,
|
||||
"viz": false
|
||||
},
|
||||
"lineInterpolation": "linear",
|
||||
"lineWidth": 1,
|
||||
"pointSize": 5,
|
||||
"scaleDistribution": {
|
||||
"type": "linear"
|
||||
},
|
||||
"showPoints": "never",
|
||||
"spanNulls": false,
|
||||
"stacking": {
|
||||
"group": "A",
|
||||
"mode": "none"
|
||||
},
|
||||
"thresholdsStyle": {
|
||||
"mode": "off"
|
||||
}
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 8
|
||||
},
|
||||
"id": 3,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": [],
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "jvm_threads_live_threads",
|
||||
"legendFormat": "Live Threads",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "jvm_threads_daemon_threads",
|
||||
"hide": false,
|
||||
"legendFormat": "Daemon Threads",
|
||||
"range": true,
|
||||
"refId": "B"
|
||||
}
|
||||
],
|
||||
"title": "JVM Threads",
|
||||
"type": "timeseries"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "palette-classic"
|
||||
},
|
||||
"custom": {
|
||||
"axisCenteredZero": false,
|
||||
"axisColorMode": "text",
|
||||
"axisLabel": "",
|
||||
"axisPlacement": "auto",
|
||||
"barAlignment": 0,
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 10,
|
||||
"gradientMode": "none",
|
||||
"hideFrom": {
|
||||
"legend": false,
|
||||
"tooltip": false,
|
||||
"viz": false
|
||||
},
|
||||
"lineInterpolation": "linear",
|
||||
"lineWidth": 1,
|
||||
"pointSize": 5,
|
||||
"scaleDistribution": {
|
||||
"type": "linear"
|
||||
},
|
||||
"showPoints": "never",
|
||||
"spanNulls": false,
|
||||
"stacking": {
|
||||
"group": "A",
|
||||
"mode": "none"
|
||||
},
|
||||
"thresholdsStyle": {
|
||||
"mode": "off"
|
||||
}
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
},
|
||||
"unit": "s"
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 8
|
||||
},
|
||||
"id": 4,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": [],
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "jvm_gc_pause_seconds_sum / jvm_gc_pause_seconds_count",
|
||||
"legendFormat": "GC Pause Time",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "JVM GC Pause Time",
|
||||
"type": "timeseries"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "palette-classic"
|
||||
},
|
||||
"custom": {
|
||||
"axisCenteredZero": false,
|
||||
"axisColorMode": "text",
|
||||
"axisLabel": "",
|
||||
"axisPlacement": "auto",
|
||||
"barAlignment": 0,
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 10,
|
||||
"gradientMode": "none",
|
||||
"hideFrom": {
|
||||
"legend": false,
|
||||
"tooltip": false,
|
||||
"viz": false
|
||||
},
|
||||
"lineInterpolation": "linear",
|
||||
"lineWidth": 1,
|
||||
"pointSize": 5,
|
||||
"scaleDistribution": {
|
||||
"type": "linear"
|
||||
},
|
||||
"showPoints": "never",
|
||||
"spanNulls": false,
|
||||
"stacking": {
|
||||
"group": "A",
|
||||
"mode": "none"
|
||||
},
|
||||
"thresholdsStyle": {
|
||||
"mode": "off"
|
||||
}
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 16
|
||||
},
|
||||
"id": 5,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": [],
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "rate(http_server_requests_seconds_count[1m])",
|
||||
"legendFormat": "{{method}} {{uri}} {{status}}",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HTTP Request Rate",
|
||||
"type": "timeseries"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "palette-classic"
|
||||
},
|
||||
"custom": {
|
||||
"axisCenteredZero": false,
|
||||
"axisColorMode": "text",
|
||||
"axisLabel": "",
|
||||
"axisPlacement": "auto",
|
||||
"barAlignment": 0,
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 10,
|
||||
"gradientMode": "none",
|
||||
"hideFrom": {
|
||||
"legend": false,
|
||||
"tooltip": false,
|
||||
"viz": false
|
||||
},
|
||||
"lineInterpolation": "linear",
|
||||
"lineWidth": 1,
|
||||
"pointSize": 5,
|
||||
"scaleDistribution": {
|
||||
"type": "linear"
|
||||
},
|
||||
"showPoints": "never",
|
||||
"spanNulls": false,
|
||||
"stacking": {
|
||||
"group": "A",
|
||||
"mode": "none"
|
||||
},
|
||||
"thresholdsStyle": {
|
||||
"mode": "off"
|
||||
}
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
},
|
||||
"unit": "s"
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 16
|
||||
},
|
||||
"id": 6,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": [],
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "PBFA97CFB590B2093"
|
||||
},
|
||||
"editorMode": "code",
|
||||
"expr": "http_server_requests_seconds_sum / http_server_requests_seconds_count",
|
||||
"legendFormat": "{{method}} {{uri}} {{status}}",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HTTP Request Duration",
|
||||
"type": "timeseries"
|
||||
}
|
||||
],
|
||||
"refresh": "5s",
|
||||
"schemaVersion": 38,
|
||||
"style": "dark",
|
||||
"tags": [],
|
||||
"templating": {
|
||||
"list": []
|
||||
},
|
||||
"time": {
|
||||
"from": "now-1h",
|
||||
"to": "now"
|
||||
},
|
||||
"timepicker": {},
|
||||
"timezone": "",
|
||||
"title": "Meldestelle JVM Metrics",
|
||||
"uid": "meldestelle-jvm",
|
||||
"version": 1,
|
||||
"weekStart": ""
|
||||
}
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
apiVersion: 1
|
||||
|
||||
providers:
|
||||
- name: 'Meldestelle Dashboards'
|
||||
orgId: 1
|
||||
folder: 'Meldestelle'
|
||||
type: file
|
||||
disableDeletion: false
|
||||
editable: true
|
||||
options:
|
||||
path: /var/lib/grafana/dashboards
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
apiVersion: 1
|
||||
|
||||
datasources:
|
||||
- name: Prometheus
|
||||
type: prometheus
|
||||
access: proxy
|
||||
url: http://prometheus:9090
|
||||
isDefault: true
|
||||
editable: false
|
||||
version: 1
|
||||
54
config/monitoring/prometheus.yml
Normal file
54
config/monitoring/prometheus.yml
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
global:
|
||||
scrape_interval: 15s
|
||||
evaluation_interval: 15s
|
||||
|
||||
# Alertmanager configuration
|
||||
alerting:
|
||||
alertmanagers:
|
||||
- static_configs:
|
||||
- targets:
|
||||
- alertmanager:9093
|
||||
|
||||
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
|
||||
rule_files:
|
||||
- "/etc/prometheus/rules/alerts.yml"
|
||||
|
||||
# A scrape configuration containing exactly one endpoint to scrape:
|
||||
scrape_configs:
|
||||
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
|
||||
- job_name: "prometheus"
|
||||
# metrics_path defaults to '/metrics'
|
||||
# scheme defaults to 'http'.
|
||||
static_configs:
|
||||
- targets: ["localhost:9090"]
|
||||
|
||||
# Scrape configuration for the Meldestelle application
|
||||
- job_name: "meldestelle-server"
|
||||
metrics_path: /metrics
|
||||
scrape_interval: 10s
|
||||
basic_auth:
|
||||
username: ${METRICS_USER:-metrics}
|
||||
password: ${METRICS_PASSWORD:-metrics-password-change-in-production}
|
||||
static_configs:
|
||||
- targets: ["server:8081"]
|
||||
labels:
|
||||
application: "meldestelle"
|
||||
service: "api-gateway"
|
||||
|
||||
# JVM metrics for the Meldestelle application
|
||||
- job_name: "meldestelle-jvm"
|
||||
metrics_path: /metrics
|
||||
scrape_interval: 10s
|
||||
basic_auth:
|
||||
username: ${METRICS_USER:-metrics}
|
||||
password: ${METRICS_PASSWORD:-metrics-password-change-in-production}
|
||||
static_configs:
|
||||
- targets: ["server:8081"]
|
||||
labels:
|
||||
application: "meldestelle"
|
||||
service: "jvm"
|
||||
|
||||
# Node exporter for host metrics (if added later)
|
||||
# - job_name: "node-exporter"
|
||||
# static_configs:
|
||||
# - targets: ["node-exporter:9100"]
|
||||
90
config/postgres/postgresql.conf
Normal file
90
config/postgres/postgresql.conf
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
# PostgreSQL Configuration File
|
||||
# Optimized for Meldestelle application
|
||||
|
||||
# Connection Settings
|
||||
listen_addresses = '*'
|
||||
max_connections = 100
|
||||
superuser_reserved_connections = 3
|
||||
|
||||
# Memory Settings
|
||||
# These will be overridden by environment variables in docker-compose.yml
|
||||
shared_buffers = 256MB # min 128kB
|
||||
work_mem = 16MB # min 64kB
|
||||
maintenance_work_mem = 64MB # min 1MB
|
||||
effective_cache_size = 768MB
|
||||
|
||||
# Write-Ahead Log (WAL)
|
||||
wal_level = replica # minimal, replica, or logical
|
||||
max_wal_size = 1GB
|
||||
min_wal_size = 80MB
|
||||
wal_buffers = 16MB # min 32kB, -1 sets based on shared_buffers
|
||||
checkpoint_completion_target = 0.9 # checkpoint target duration, 0.0 - 1.0
|
||||
random_page_cost = 1.1 # For SSD storage
|
||||
|
||||
# Background Writer
|
||||
bgwriter_delay = 200ms
|
||||
bgwriter_lru_maxpages = 100
|
||||
bgwriter_lru_multiplier = 2.0
|
||||
|
||||
# Asynchronous Behavior
|
||||
effective_io_concurrency = 200 # For SSD storage
|
||||
max_worker_processes = 8
|
||||
max_parallel_workers_per_gather = 4
|
||||
max_parallel_workers = 8
|
||||
max_parallel_maintenance_workers = 4
|
||||
|
||||
# Query Planner
|
||||
default_statistics_target = 100
|
||||
constraint_exclusion = partition
|
||||
|
||||
# Logging
|
||||
log_destination = 'stderr'
|
||||
logging_collector = on
|
||||
log_directory = 'log'
|
||||
log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
|
||||
log_truncate_on_rotation = off
|
||||
log_rotation_age = 1d
|
||||
log_rotation_size = 100MB
|
||||
log_min_duration_statement = 250ms # Log slow queries (250ms or slower)
|
||||
log_checkpoints = on
|
||||
log_connections = on
|
||||
log_disconnections = on
|
||||
log_lock_waits = on
|
||||
log_temp_files = 0
|
||||
log_autovacuum_min_duration = 250ms
|
||||
log_line_prefix = '%m [%p] %q%u@%d '
|
||||
|
||||
# Autovacuum
|
||||
autovacuum = on
|
||||
autovacuum_max_workers = 3
|
||||
autovacuum_naptime = 1min
|
||||
autovacuum_vacuum_threshold = 50
|
||||
autovacuum_analyze_threshold = 50
|
||||
autovacuum_vacuum_scale_factor = 0.05
|
||||
autovacuum_analyze_scale_factor = 0.025
|
||||
autovacuum_vacuum_cost_delay = 20ms
|
||||
autovacuum_vacuum_cost_limit = 2000
|
||||
|
||||
# Statement Behavior
|
||||
search_path = '"$user", public'
|
||||
row_security = on
|
||||
|
||||
# Client Connection Defaults
|
||||
client_min_messages = notice
|
||||
statement_timeout = 60000 # 60 seconds, prevents long-running queries
|
||||
lock_timeout = 10000 # 10 seconds, prevents lock contention
|
||||
idle_in_transaction_session_timeout = 600000 # 10 minutes, prevents idle transactions
|
||||
|
||||
# Disk
|
||||
temp_file_limit = 1GB # Limits temp file size
|
||||
|
||||
# SSL
|
||||
ssl = off
|
||||
ssl_prefer_server_ciphers = on
|
||||
|
||||
# Performance Monitoring
|
||||
track_activities = on
|
||||
track_counts = on
|
||||
track_io_timing = on
|
||||
track_functions = pl # none, pl, all
|
||||
track_activity_query_size = 2048
|
||||
|
|
@ -1,9 +1,9 @@
|
|||
services:
|
||||
server:
|
||||
api-gateway:
|
||||
build:
|
||||
context: . # Baut mit Dockerfile im Root
|
||||
image: meldestelle/server:latest
|
||||
container_name: meldestelle-server
|
||||
context: . # Build with Dockerfile in root
|
||||
image: meldestelle/api-gateway:latest
|
||||
container_name: meldestelle-api-gateway
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8080:8081"
|
||||
|
|
@ -13,11 +13,54 @@ services:
|
|||
- DB_NAME=${POSTGRES_DB}
|
||||
- DB_HOST=db
|
||||
- DB_PORT=5432
|
||||
- REDIS_HOST=redis
|
||||
- REDIS_PORT=6379
|
||||
- JAVA_OPTS=-Xms512m -Xmx1024m
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
meldestelle-net:
|
||||
aliases:
|
||||
- server
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2'
|
||||
memory: 1536M
|
||||
reservations:
|
||||
cpus: '0.5'
|
||||
memory: 512M
|
||||
# Healthcheck is now defined in Dockerfile
|
||||
|
||||
# Redis for caching
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
container_name: meldestelle-redis
|
||||
restart: unless-stopped
|
||||
command: redis-server --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru
|
||||
volumes:
|
||||
- redis_data:/data
|
||||
ports:
|
||||
- "127.0.0.1:6379:6379"
|
||||
networks:
|
||||
- meldestelle-net
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '1'
|
||||
memory: 384M
|
||||
reservations:
|
||||
cpus: '0.2'
|
||||
memory: 128M
|
||||
# PostgreSQL Datenbank (Service-Name 'db')
|
||||
db:
|
||||
image: postgres:16-alpine # Spezifische Version
|
||||
|
|
@ -28,10 +71,23 @@ services:
|
|||
POSTGRES_USER: ${POSTGRES_USER}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||
POSTGRES_DB: ${POSTGRES_DB}
|
||||
# PostgreSQL performance tuning
|
||||
POSTGRES_INITDB_ARGS: "--data-checksums"
|
||||
POSTGRES_INITDB_WALDIR: "/var/lib/postgresql/wal"
|
||||
# PostgreSQL configuration
|
||||
POSTGRES_SHARED_BUFFERS: ${POSTGRES_SHARED_BUFFERS:-256MB}
|
||||
POSTGRES_EFFECTIVE_CACHE_SIZE: ${POSTGRES_EFFECTIVE_CACHE_SIZE:-768MB}
|
||||
POSTGRES_WORK_MEM: ${POSTGRES_WORK_MEM:-16MB}
|
||||
POSTGRES_MAINTENANCE_WORK_MEM: ${POSTGRES_MAINTENANCE_WORK_MEM:-64MB}
|
||||
POSTGRES_MAX_CONNECTIONS: ${POSTGRES_MAX_CONNECTIONS:-100}
|
||||
# PGDATA nicht nötig, Standard verwenden
|
||||
volumes:
|
||||
# Benanntes Volume für Daten auf Standardpfad
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
- postgres_wal:/var/lib/postgresql/wal
|
||||
# Add custom PostgreSQL configuration
|
||||
- ./config/postgres/postgresql.conf:/etc/postgresql/postgresql.conf:ro
|
||||
command: ["postgres", "-c", "config_file=/etc/postgresql/postgresql.conf"]
|
||||
networks:
|
||||
- meldestelle-net # <--- Muss zum Netzwerk-Namen passen
|
||||
healthcheck: # Wichtig für depends_on
|
||||
|
|
@ -39,8 +95,17 @@ services:
|
|||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 20s
|
||||
ports: # Nur bei Bedarf freigeben, z.B. für lokalen Zugriff
|
||||
- "127.0.0.1:54321:5432" # Host-Port 54321 → Container-Port 5432
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2'
|
||||
memory: 1024M
|
||||
reservations:
|
||||
cpus: '0.5'
|
||||
memory: 256M
|
||||
|
||||
# PgAdmin Service
|
||||
pgadmin:
|
||||
|
|
@ -62,9 +127,197 @@ services:
|
|||
depends_on: # PgAdmin braucht die DB
|
||||
- db
|
||||
|
||||
# Prometheus Service
|
||||
prometheus:
|
||||
image: prom/prometheus:latest
|
||||
container_name: meldestelle-prometheus
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./config/monitoring/prometheus.yml:/etc/prometheus/prometheus.yml
|
||||
- prometheus_data:/prometheus
|
||||
command:
|
||||
- '--config.file=/etc/prometheus/prometheus.yml'
|
||||
- '--storage.tsdb.path=/prometheus'
|
||||
- '--web.console.libraries=/etc/prometheus/console_libraries'
|
||||
- '--web.console.templates=/etc/prometheus/consoles'
|
||||
- '--web.enable-lifecycle'
|
||||
ports:
|
||||
- "9090:9090"
|
||||
networks:
|
||||
- meldestelle-net
|
||||
depends_on:
|
||||
- api-gateway
|
||||
|
||||
# Grafana Service
|
||||
grafana:
|
||||
image: grafana/grafana:latest
|
||||
container_name: meldestelle-grafana
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./config/monitoring/grafana/provisioning:/etc/grafana/provisioning
|
||||
- ./config/monitoring/grafana/dashboards:/var/lib/grafana/dashboards
|
||||
- grafana_data:/var/lib/grafana
|
||||
environment:
|
||||
- GF_SECURITY_ADMIN_USER=${GRAFANA_ADMIN_USER:-admin}
|
||||
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD:-admin}
|
||||
- GF_USERS_ALLOW_SIGN_UP=false
|
||||
ports:
|
||||
- "3000:3000"
|
||||
networks:
|
||||
- meldestelle-net
|
||||
depends_on:
|
||||
- prometheus
|
||||
|
||||
# Alertmanager Service
|
||||
alertmanager:
|
||||
image: prom/alertmanager:latest
|
||||
container_name: meldestelle-alertmanager
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./config/monitoring/alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml
|
||||
- alertmanager_data:/alertmanager
|
||||
command:
|
||||
- '--config.file=/etc/alertmanager/alertmanager.yml'
|
||||
- '--storage.path=/alertmanager'
|
||||
ports:
|
||||
- "9093:9093"
|
||||
networks:
|
||||
- meldestelle-net
|
||||
depends_on:
|
||||
- prometheus
|
||||
|
||||
# Elasticsearch Service
|
||||
elasticsearch:
|
||||
image: docker.elastic.co/elasticsearch/elasticsearch:8.12.2
|
||||
container_name: meldestelle-elasticsearch
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- discovery.type=single-node
|
||||
- bootstrap.memory_lock=true
|
||||
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
|
||||
- xpack.security.enabled=false
|
||||
ulimits:
|
||||
memlock:
|
||||
soft: -1
|
||||
hard: -1
|
||||
volumes:
|
||||
- elasticsearch_data:/usr/share/elasticsearch/data
|
||||
- ./config/monitoring/elk/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
|
||||
ports:
|
||||
- "9200:9200"
|
||||
networks:
|
||||
- meldestelle-net
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:9200"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 40s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2'
|
||||
memory: 1024M
|
||||
reservations:
|
||||
cpus: '0.5'
|
||||
memory: 512M
|
||||
|
||||
# Logstash Service
|
||||
logstash:
|
||||
image: docker.elastic.co/logstash/logstash:8.12.2
|
||||
container_name: meldestelle-logstash
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./config/monitoring/elk/logstash.conf:/usr/share/logstash/pipeline/logstash.conf:ro
|
||||
ports:
|
||||
- "5044:5044"
|
||||
- "5000:5000/tcp"
|
||||
- "5000:5000/udp"
|
||||
- "9600:9600"
|
||||
environment:
|
||||
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
|
||||
networks:
|
||||
- meldestelle-net
|
||||
depends_on:
|
||||
elasticsearch:
|
||||
condition: service_healthy
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '1'
|
||||
memory: 512M
|
||||
reservations:
|
||||
cpus: '0.2'
|
||||
memory: 256M
|
||||
|
||||
# Kibana Service
|
||||
kibana:
|
||||
image: docker.elastic.co/kibana/kibana:8.12.2
|
||||
container_name: meldestelle-kibana
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "5601:5601"
|
||||
environment:
|
||||
ELASTICSEARCH_URL: http://elasticsearch:9200
|
||||
ELASTICSEARCH_HOSTS: http://elasticsearch:9200
|
||||
networks:
|
||||
- meldestelle-net
|
||||
depends_on:
|
||||
elasticsearch:
|
||||
condition: service_healthy
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '1'
|
||||
memory: 512M
|
||||
reservations:
|
||||
cpus: '0.2'
|
||||
memory: 256M
|
||||
|
||||
# Consul Service for Service Discovery
|
||||
consul:
|
||||
image: consul:1.15
|
||||
container_name: meldestelle-consul
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8500:8500" # HTTP UI and API
|
||||
- "8600:8600/udp" # DNS interface
|
||||
volumes:
|
||||
- consul_data:/consul/data
|
||||
environment:
|
||||
- CONSUL_BIND_INTERFACE=eth0
|
||||
- CONSUL_CLIENT_INTERFACE=eth0
|
||||
command: "agent -server -ui -bootstrap-expect=1 -client=0.0.0.0"
|
||||
networks:
|
||||
- meldestelle-net
|
||||
healthcheck:
|
||||
test: ["CMD", "consul", "members"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '1'
|
||||
memory: 512M
|
||||
reservations:
|
||||
cpus: '0.2'
|
||||
memory: 128M
|
||||
|
||||
networks:
|
||||
meldestelle-net:
|
||||
driver: bridge
|
||||
volumes:
|
||||
postgres_data: # <--- Konsistenter Name
|
||||
postgres_wal: # Volume for PostgreSQL WAL files
|
||||
driver: local
|
||||
pgadmin_data: # <--- Konsistenter Name
|
||||
prometheus_data: # Volume for Prometheus data
|
||||
grafana_data: # Volume for Grafana data
|
||||
alertmanager_data: # Volume for Alertmanager data
|
||||
elasticsearch_data: # Volume for Elasticsearch data
|
||||
redis_data: # Volume for Redis data
|
||||
driver: local
|
||||
consul_data: # Volume for Consul data
|
||||
driver: local
|
||||
|
|
|
|||
|
|
@ -1,14 +1,18 @@
|
|||
#Kotlin
|
||||
kotlin.code.style=official
|
||||
kotlin.daemon.jvmargs=-Xmx2048M
|
||||
kotlin.daemon.jvmargs=-Xmx3072M -XX:+UseParallelGC -XX:MaxMetaspaceSize=1024M
|
||||
|
||||
#Gradle
|
||||
org.gradle.jvmargs=-Xmx2048M -Dfile.encoding=UTF-8 -XX:+UseParallelGC
|
||||
org.gradle.jvmargs=-Xmx3072M -Dfile.encoding=UTF-8 -XX:+UseParallelGC -XX:MaxMetaspaceSize=1024M -XX:+HeapDumpOnOutOfMemoryError
|
||||
org.gradle.parallel=true
|
||||
org.gradle.caching=true
|
||||
org.gradle.configureondemand=true
|
||||
org.gradle.workers.max=8
|
||||
#org.gradle.dependency.verification=strict # Aktiviere Dependency Verification bei Bedarf
|
||||
|
||||
# Enable dependency locking for reproducible builds
|
||||
org.gradle.dependency.locking.enabled=true
|
||||
|
||||
#Ktor
|
||||
io.ktor.development=true
|
||||
|
||||
|
|
|
|||
|
|
@ -14,12 +14,23 @@ composeMultiplatform = "1.8.0" #"1.7.3"
|
|||
# Ktor
|
||||
ktor = "3.1.2"
|
||||
|
||||
# Monitoring
|
||||
micrometer = "1.12.2"
|
||||
|
||||
# Database
|
||||
exposed = "0.52.0"
|
||||
postgresql = "42.7.3"
|
||||
hikari = "5.1.0"
|
||||
h2 = "2.2.224"
|
||||
|
||||
# Caching
|
||||
redisson = "3.27.1"
|
||||
caffeine = "3.1.8"
|
||||
|
||||
# Service Discovery
|
||||
consul = "2.2.10"
|
||||
orbitz-consul = "1.5.3"
|
||||
|
||||
# Logging
|
||||
logback = "1.5.18"
|
||||
logbackJsonEncoder = "8.0"
|
||||
|
|
@ -80,6 +91,9 @@ h2-driver = { module = "com.h2database:h2", version.ref = "h2" }
|
|||
logback = { module = "ch.qos.logback:logback-classic", version.ref = "logback" }
|
||||
logback-json-encoder = { module = "net.logstash.logback:logstash-logback-encoder", version.ref = "logbackJsonEncoder" }
|
||||
|
||||
# Monitoring
|
||||
micrometer-prometheus = { group = "io.micrometer", name = "micrometer-registry-prometheus", version.ref = "micrometer" }
|
||||
ktor-server-metrics-micrometer = { group = "io.ktor", name = "ktor-server-metrics-micrometer", version.ref = "ktor" }
|
||||
|
||||
# Testing
|
||||
junitJupiter = { group = "org.junit.jupiter", name = "junit-jupiter", version.ref = "junitJupiter" }
|
||||
|
|
@ -88,6 +102,15 @@ junitJupiter = { group = "org.junit.jupiter", name = "junit-jupiter", version.re
|
|||
uuid = { group = "com.benasher44", name = "uuid", version.ref = "uuid" }
|
||||
bignum = { group = "com.ionspin.kotlin", name = "bignum", version.ref = "bignum" }
|
||||
|
||||
# Caching
|
||||
redisson = { group = "org.redisson", name = "redisson", version.ref = "redisson" }
|
||||
caffeine = { group = "com.github.ben-manes.caffeine", name = "caffeine", version.ref = "caffeine" }
|
||||
|
||||
# Service Discovery
|
||||
consul-client = { module = "com.orbitz.consul:consul-client", version.ref = "orbitz-consul" }
|
||||
consul-api = { module = "com.ecwid.consul:consul-api", version.ref = "consul" }
|
||||
ktor-client-cio = { module = "io.ktor:ktor-client-cio", version.ref = "ktor" }
|
||||
|
||||
# Kotlin Wrappers for JS/React
|
||||
kotlin-wrappers-bom = { group = "org.jetbrains.kotlin-wrappers", name = "kotlin-wrappers-bom", version.ref = "kotlinWrappers" }
|
||||
kotlin-react = { group = "org.jetbrains.kotlin-wrappers", name = "kotlin-react" }
|
||||
|
|
|
|||
|
|
@ -25,6 +25,22 @@ dependencyResolutionManagement {
|
|||
includeGroupAndSubgroups("com.google")
|
||||
}
|
||||
}
|
||||
// Add JCenter repository (archive)
|
||||
maven {
|
||||
url = uri("https://jcenter.bintray.com")
|
||||
}
|
||||
// Add JitPack repository
|
||||
maven {
|
||||
url = uri("https://jitpack.io")
|
||||
}
|
||||
// Add Sonatype snapshots repository
|
||||
maven {
|
||||
url = uri("https://oss.sonatype.org/content/repositories/snapshots/")
|
||||
}
|
||||
// Add Maven repository for Ecwid libraries
|
||||
maven {
|
||||
url = uri("https://dl.bintray.com/ecwid/maven")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -30,6 +30,14 @@ kotlin {
|
|||
implementation(libs.exposed.jdbc)
|
||||
implementation(libs.exposed.kotlinDatetime)
|
||||
implementation(libs.postgresql.driver)
|
||||
|
||||
// Service Discovery dependencies
|
||||
implementation("com.orbitz.consul:consul-client:1.5.3")
|
||||
implementation("com.ecwid.consul:consul-api:1.4.5") // Downgraded from 2.2.10 to 1.4.5 which is available on Maven Central
|
||||
implementation("io.ktor:ktor-client-core:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-client-cio:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-client-content-negotiation:${libs.versions.ktor.get()}")
|
||||
implementation("io.ktor:ktor-serialization-kotlinx-json:${libs.versions.ktor.get()}")
|
||||
}
|
||||
|
||||
jvmTest.dependencies {
|
||||
|
|
|
|||
|
|
@ -27,6 +27,9 @@ object AppConfig {
|
|||
// Rate Limiting-Konfiguration
|
||||
val rateLimit = RateLimitConfig()
|
||||
|
||||
// Service Discovery-Konfiguration
|
||||
val serviceDiscovery = ServiceDiscoveryConfig()
|
||||
|
||||
// Datenbank-Konfiguration (wird nach dem Laden der Properties initialisiert)
|
||||
val database: DatabaseConfig
|
||||
|
||||
|
|
@ -40,6 +43,7 @@ object AppConfig {
|
|||
security.configure(props)
|
||||
logging.configure(props)
|
||||
rateLimit.configure(props)
|
||||
serviceDiscovery.configure(props)
|
||||
|
||||
// Datenbank-Konfiguration mit Properties initialisieren
|
||||
database = DatabaseConfig.fromEnv(props)
|
||||
|
|
@ -303,4 +307,29 @@ class RateLimitConfig {
|
|||
val limit: Int,
|
||||
val periodMinutes: Int
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Konfiguration für Service Discovery.
|
||||
*/
|
||||
class ServiceDiscoveryConfig {
|
||||
// Consul Konfiguration
|
||||
var enabled: Boolean = true
|
||||
var consulHost: String = System.getenv("CONSUL_HOST") ?: "consul"
|
||||
var consulPort: Int = System.getenv("CONSUL_PORT")?.toIntOrNull() ?: 8500
|
||||
|
||||
// Service Registration Konfiguration
|
||||
var registerServices: Boolean = true
|
||||
var healthCheckPath: String = "/health"
|
||||
var healthCheckInterval: Int = 10 // Sekunden
|
||||
|
||||
fun configure(props: Properties) {
|
||||
enabled = props.getProperty("service-discovery.enabled")?.toBoolean() ?: enabled
|
||||
consulHost = props.getProperty("service-discovery.consul.host") ?: consulHost
|
||||
consulPort = props.getProperty("service-discovery.consul.port")?.toIntOrNull() ?: consulPort
|
||||
|
||||
registerServices = props.getProperty("service-discovery.register-services")?.toBoolean() ?: registerServices
|
||||
healthCheckPath = props.getProperty("service-discovery.health-check.path") ?: healthCheckPath
|
||||
healthCheckInterval = props.getProperty("service-discovery.health-check.interval")?.toIntOrNull() ?: healthCheckInterval
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ data class DatabaseConfig(
|
|||
val password: String,
|
||||
val driverClassName: String = "org.postgresql.Driver",
|
||||
val maxPoolSize: Int = 10,
|
||||
val minPoolSize: Int = 5,
|
||||
val autoMigrate: Boolean = true
|
||||
) {
|
||||
companion object {
|
||||
|
|
@ -29,6 +30,9 @@ data class DatabaseConfig(
|
|||
val maxPoolSize = System.getenv("DB_MAX_POOL_SIZE")?.toIntOrNull()
|
||||
?: props.getProperty("database.maxPoolSize")?.toIntOrNull()
|
||||
?: 10
|
||||
val minPoolSize = System.getenv("DB_MIN_POOL_SIZE")?.toIntOrNull()
|
||||
?: props.getProperty("database.minPoolSize")?.toIntOrNull()
|
||||
?: 5
|
||||
val autoMigrate = System.getenv("DB_AUTO_MIGRATE")?.toBoolean()
|
||||
?: props.getProperty("database.autoMigrate")?.toBoolean()
|
||||
?: true
|
||||
|
|
@ -39,6 +43,7 @@ data class DatabaseConfig(
|
|||
password = password,
|
||||
driverClassName = "org.postgresql.Driver",
|
||||
maxPoolSize = maxPoolSize,
|
||||
minPoolSize = minPoolSize,
|
||||
autoMigrate = autoMigrate
|
||||
)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -28,8 +28,37 @@ object DatabaseFactory {
|
|||
username = config.username
|
||||
password = config.password
|
||||
maximumPoolSize = config.maxPoolSize
|
||||
minimumIdle = config.minPoolSize // Use the minPoolSize from config
|
||||
isAutoCommit = false
|
||||
transactionIsolation = "TRANSACTION_REPEATABLE_READ"
|
||||
|
||||
// Use READ_COMMITTED for better performance while maintaining data integrity
|
||||
// REPEATABLE_READ is more strict and can lead to more contention
|
||||
transactionIsolation = "TRANSACTION_READ_COMMITTED"
|
||||
|
||||
// Connection validation
|
||||
connectionTestQuery = "SELECT 1"
|
||||
validationTimeout = 5000 // 5 seconds
|
||||
|
||||
// Connection timeouts
|
||||
connectionTimeout = 30000 // 30 seconds
|
||||
idleTimeout = 600000 // 10 minutes
|
||||
maxLifetime = 1800000 // 30 minutes
|
||||
|
||||
// Leak detection
|
||||
leakDetectionThreshold = 60000 // 1 minute
|
||||
|
||||
// Statement cache for better performance
|
||||
dataSourceProperties["cachePrepStmts"] = "true"
|
||||
dataSourceProperties["prepStmtCacheSize"] = "250"
|
||||
dataSourceProperties["prepStmtCacheSqlLimit"] = "2048"
|
||||
dataSourceProperties["useServerPrepStmts"] = "true"
|
||||
|
||||
// Connection initialization - run a simple query to warm up connections
|
||||
connectionInitSql = "SELECT 1"
|
||||
|
||||
// Pool name for better identification in metrics
|
||||
poolName = "MeldestelleDbPool"
|
||||
|
||||
validate()
|
||||
}
|
||||
|
||||
|
|
@ -52,4 +81,28 @@ object DatabaseFactory {
|
|||
dataSource?.close()
|
||||
dataSource = null
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the number of active connections in the pool.
|
||||
* @return The number of active connections, or 0 if the pool is not initialized
|
||||
*/
|
||||
fun getActiveConnections(): Int {
|
||||
return dataSource?.hikariPoolMXBean?.activeConnections ?: 0
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the number of idle connections in the pool.
|
||||
* @return The number of idle connections, or 0 if the pool is not initialized
|
||||
*/
|
||||
fun getIdleConnections(): Int {
|
||||
return dataSource?.hikariPoolMXBean?.idleConnections ?: 0
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the total number of connections in the pool.
|
||||
* @return The total number of connections, or 0 if the pool is not initialized
|
||||
*/
|
||||
fun getTotalConnections(): Int {
|
||||
return dataSource?.hikariPoolMXBean?.totalConnections ?: 0
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,165 @@
|
|||
package at.mocode.shared.discovery
|
||||
|
||||
import at.mocode.shared.config.AppConfig
|
||||
import kotlinx.coroutines.CoroutineScope
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.delay
|
||||
import kotlinx.coroutines.launch
|
||||
import java.net.InetAddress
|
||||
import java.util.*
|
||||
import kotlin.time.Duration.Companion.seconds
|
||||
import com.orbitz.consul.Consul
|
||||
import com.orbitz.consul.model.agent.ImmutableRegistration
|
||||
import com.orbitz.consul.model.agent.Registration
|
||||
|
||||
/**
|
||||
* Service registration configuration.
|
||||
*
|
||||
* @property serviceName The name of the service to register
|
||||
* @property serviceId A unique ID for this service instance (defaults to serviceName + random UUID)
|
||||
* @property servicePort The port the service is running on
|
||||
* @property healthCheckPath The path for the health check endpoint (defaults to "/health")
|
||||
* @property healthCheckInterval The interval between health checks in seconds (defaults to 10 seconds)
|
||||
* @property tags Optional tags to associate with the service
|
||||
* @property meta Optional metadata to associate with the service
|
||||
*/
|
||||
data class ServiceRegistrationConfig(
|
||||
val serviceName: String,
|
||||
val serviceId: String = "$serviceName-${UUID.randomUUID()}",
|
||||
val servicePort: Int,
|
||||
val healthCheckPath: String = "/health",
|
||||
val healthCheckInterval: Int = 10,
|
||||
val tags: List<String> = emptyList(),
|
||||
val meta: Map<String, String> = emptyMap()
|
||||
)
|
||||
|
||||
/**
|
||||
* Service registration component for registering services with Consul.
|
||||
*/
|
||||
class ServiceRegistration(
|
||||
private val config: ServiceRegistrationConfig,
|
||||
private val consulHost: String = "consul",
|
||||
private val consulPort: Int = 8500
|
||||
) {
|
||||
private val consul: Consul by lazy {
|
||||
try {
|
||||
Consul.builder()
|
||||
.withUrl("http://$consulHost:$consulPort")
|
||||
.build()
|
||||
} catch (e: Exception) {
|
||||
println("Failed to connect to Consul: ${e.message}")
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
private val serviceId = config.serviceId
|
||||
private var registered = false
|
||||
|
||||
/**
|
||||
* Register the service with Consul.
|
||||
*/
|
||||
fun register() {
|
||||
try {
|
||||
val hostAddress = InetAddress.getLocalHost().hostAddress
|
||||
|
||||
// Create health check
|
||||
val healthCheck = Registration.RegCheck.http(
|
||||
"http://$hostAddress:${config.servicePort}${config.healthCheckPath}",
|
||||
config.healthCheckInterval.toLong()
|
||||
)
|
||||
|
||||
// Create service registration
|
||||
val registration = ImmutableRegistration.builder()
|
||||
.id(serviceId)
|
||||
.name(config.serviceName)
|
||||
.address(hostAddress)
|
||||
.port(config.servicePort)
|
||||
.tags(config.tags)
|
||||
.meta(config.meta)
|
||||
.check(healthCheck)
|
||||
.build()
|
||||
|
||||
// Register service with Consul
|
||||
consul.agentClient().register(registration)
|
||||
registered = true
|
||||
println("Service $serviceId registered with Consul at $consulHost:$consulPort")
|
||||
|
||||
// Start heartbeat to keep service registration active
|
||||
startHeartbeat()
|
||||
} catch (e: Exception) {
|
||||
println("Failed to register service with Consul: ${e.message}")
|
||||
e.printStackTrace()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Deregister the service from Consul.
|
||||
*/
|
||||
fun deregister() {
|
||||
try {
|
||||
if (registered) {
|
||||
consul.agentClient().deregister(serviceId)
|
||||
registered = false
|
||||
println("Service $serviceId deregistered from Consul")
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
println("Failed to deregister service from Consul: ${e.message}")
|
||||
e.printStackTrace()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Start a heartbeat to keep the service registration active.
|
||||
*/
|
||||
private fun startHeartbeat() {
|
||||
CoroutineScope(Dispatchers.IO).launch {
|
||||
while (registered) {
|
||||
try {
|
||||
// Send heartbeat to Consul
|
||||
consul.agentClient().pass(serviceId)
|
||||
delay(config.healthCheckInterval.seconds)
|
||||
} catch (e: Exception) {
|
||||
println("Failed to send heartbeat to Consul: ${e.message}")
|
||||
delay(5.seconds)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Factory for creating ServiceRegistration instances.
|
||||
*/
|
||||
object ServiceRegistrationFactory {
|
||||
/**
|
||||
* Create a ServiceRegistration instance for a service.
|
||||
*
|
||||
* @param serviceName The name of the service to register
|
||||
* @param servicePort The port the service is running on
|
||||
* @param healthCheckPath The path for the health check endpoint (defaults to "/health")
|
||||
* @param tags Optional tags to associate with the service
|
||||
* @param meta Optional metadata to associate with the service
|
||||
* @return A ServiceRegistration instance
|
||||
*/
|
||||
fun createServiceRegistration(
|
||||
serviceName: String,
|
||||
servicePort: Int,
|
||||
healthCheckPath: String = "/health",
|
||||
tags: List<String> = emptyList(),
|
||||
meta: Map<String, String> = emptyMap()
|
||||
): ServiceRegistration {
|
||||
val config = ServiceRegistrationConfig(
|
||||
serviceName = serviceName,
|
||||
servicePort = servicePort,
|
||||
healthCheckPath = healthCheckPath,
|
||||
tags = tags,
|
||||
meta = meta
|
||||
)
|
||||
|
||||
// Get Consul host and port from configuration if available
|
||||
val consulHost = AppConfig.serviceDiscovery.consulHost
|
||||
val consulPort = AppConfig.serviceDiscovery.consulPort
|
||||
|
||||
return ServiceRegistration(config, consulHost, consulPort)
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user