feature Keycloak Auth
This commit is contained in:
@@ -0,0 +1,427 @@
|
||||
# Docker Configuration Optimization & Security Analysis
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document outlines the comprehensive analysis, corrections, and optimizations made to all Docker and docker-compose configurations in the Meldestelle project. The optimizations focus on **security hardening**, **performance improvements**, and **production readiness**.
|
||||
|
||||
### Key Achievements
|
||||
- ✅ **Critical Security Vulnerabilities Fixed**: Eliminated hardcoded credentials and exposed secrets
|
||||
- ✅ **Resource Management**: Added comprehensive CPU and memory limits for all services
|
||||
- ✅ **Security Hardening**: Implemented Docker secrets, non-root users, and security constraints
|
||||
- ✅ **Performance Optimization**: Enhanced health checks, startup dependencies, and resource allocation
|
||||
- ✅ **Production Readiness**: Added proper volume management, networking, and monitoring
|
||||
|
||||
---
|
||||
|
||||
## Security Improvements
|
||||
|
||||
### 🔐 Critical Security Issues Resolved
|
||||
|
||||
#### 1. **Secrets Management**
|
||||
**Problem**: Hardcoded credentials in environment variables
|
||||
```yaml
|
||||
# BEFORE (INSECURE)
|
||||
environment:
|
||||
POSTGRES_PASSWORD: meldestelle
|
||||
KEYCLOAK_CLIENT_SECRET: K5RqonwVOaxPKaXVH4mbthSRbjRh5tOK
|
||||
GF_SECURITY_ADMIN_PASSWORD: admin
|
||||
```
|
||||
|
||||
**Solution**: Docker secrets with secure file-based management
|
||||
```yaml
|
||||
# AFTER (SECURE)
|
||||
environment:
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
|
||||
KEYCLOAK_CLIENT_SECRET_FILE: /run/secrets/keycloak_client_secret
|
||||
GF_SECURITY_ADMIN_PASSWORD__FILE: /run/secrets/grafana_admin_password
|
||||
secrets:
|
||||
- postgres_password
|
||||
- keycloak_client_secret
|
||||
- grafana_admin_password
|
||||
```
|
||||
|
||||
#### 2. **Container Security Hardening**
|
||||
**Added Security Measures**:
|
||||
- `no-new-privileges:true` for all containers
|
||||
- Non-root user execution where possible
|
||||
- Read-only volume mounts for configuration files
|
||||
- Secure file permissions (600) for all secrets
|
||||
|
||||
#### 3. **Network Security**
|
||||
**Improvements**:
|
||||
- Custom isolated network with dedicated subnet (172.20.0.0/16)
|
||||
- Proper inter-container communication controls
|
||||
- Enhanced CORS and security headers for web applications
|
||||
|
||||
### 🛡️ Security Features Added
|
||||
|
||||
| Security Feature | Implementation | Benefit |
|
||||
|-----------------|----------------|---------|
|
||||
| Docker Secrets | File-based secrets management | Eliminates hardcoded credentials |
|
||||
| Non-root Users | Custom user/group for applications | Reduces attack surface |
|
||||
| Security Options | `no-new-privileges` flag | Prevents privilege escalation |
|
||||
| Read-only Mounts | Config files mounted read-only | Prevents runtime tampering |
|
||||
| Network Isolation | Custom bridge network | Isolates container communication |
|
||||
| Resource Limits | CPU/Memory constraints | Prevents resource exhaustion attacks |
|
||||
|
||||
---
|
||||
|
||||
## Performance Optimizations
|
||||
|
||||
### 🚀 Resource Management
|
||||
|
||||
#### Comprehensive Resource Limits
|
||||
All services now have properly configured resource limits and reservations:
|
||||
|
||||
**Infrastructure Services**:
|
||||
```yaml
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2.0'
|
||||
memory: 2G
|
||||
reservations:
|
||||
cpus: '0.5'
|
||||
memory: 1G
|
||||
```
|
||||
|
||||
**Resource Allocation Summary**:
|
||||
| Service | CPU Limit | Memory Limit | CPU Reserved | Memory Reserved |
|
||||
|---------|-----------|--------------|--------------|-----------------|
|
||||
| PostgreSQL | 2.0 | 2GB | 0.5 | 512MB |
|
||||
| Redis | 1.0 | 1GB | 0.25 | 256MB |
|
||||
| Keycloak | 2.0 | 2GB | 0.5 | 1GB |
|
||||
| API Gateway | 2.0 | 2GB | 0.5 | 1GB |
|
||||
| Kafka | 2.0 | 2GB | 0.5 | 512MB |
|
||||
| Grafana | 1.0 | 1GB | 0.25 | 256MB |
|
||||
| Prometheus | 1.0 | 2GB | 0.25 | 512MB |
|
||||
|
||||
### 🔧 Performance Enhancements
|
||||
|
||||
#### 1. **Optimized Health Checks**
|
||||
```yaml
|
||||
# Enhanced health check configuration
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "--fail", "--max-time", "5", "http://localhost:8080/health/ready"]
|
||||
interval: 15s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
```
|
||||
|
||||
#### 2. **JVM Optimization**
|
||||
**Kafka JVM Settings**:
|
||||
```yaml
|
||||
environment:
|
||||
KAFKA_HEAP_OPTS: "-Xmx1G -Xms512m"
|
||||
KAFKA_JVM_PERFORMANCE_OPTS: "-XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35"
|
||||
```
|
||||
|
||||
#### 3. **Database Performance**
|
||||
**PostgreSQL Enhancements**:
|
||||
- SCRAM-SHA-256 authentication for better security
|
||||
- Optimized connection settings
|
||||
- Proper data persistence with bind mounts
|
||||
|
||||
**Redis Optimizations**:
|
||||
- Memory management with `maxmemory` and `allkeys-lru` policy
|
||||
- Persistent storage with AOF (Append Only File)
|
||||
- Authentication enabled
|
||||
|
||||
---
|
||||
|
||||
## Configuration Structure
|
||||
|
||||
### 📁 File Organization
|
||||
|
||||
The optimized configuration consists of:
|
||||
|
||||
```
|
||||
├── docker-compose.yml.optimized # Infrastructure services
|
||||
├── docker-compose.services.yml.optimized # Microservices
|
||||
├── docker-compose.clients.yml.optimized # Client applications
|
||||
├── .env.template # Environment configuration template
|
||||
└── docker/
|
||||
└── secrets/
|
||||
├── setup-secrets.sh # Automated secrets generation
|
||||
├── postgres_user.txt # Database username
|
||||
├── postgres_password.txt # Database password (generated)
|
||||
├── redis_password.txt # Redis password (generated)
|
||||
├── keycloak_admin_password.txt # Keycloak admin password (generated)
|
||||
├── keycloak_client_secret.txt # API Gateway client secret (generated)
|
||||
├── grafana_admin_user.txt # Grafana admin username
|
||||
├── grafana_admin_password.txt # Grafana admin password (generated)
|
||||
├── jwt_secret.txt # JWT signing secret (generated)
|
||||
└── vnc_password.txt # VNC access password (generated)
|
||||
```
|
||||
|
||||
### 🔄 Profile-based Deployment
|
||||
|
||||
The optimized configuration supports selective service deployment:
|
||||
|
||||
```bash
|
||||
# Infrastructure only
|
||||
docker-compose -f docker-compose.yml.optimized up -d
|
||||
|
||||
# Infrastructure + Microservices
|
||||
docker-compose -f docker-compose.yml.optimized \
|
||||
-f docker-compose.services.yml.optimized up -d
|
||||
|
||||
# Full stack deployment
|
||||
docker-compose -f docker-compose.yml.optimized \
|
||||
-f docker-compose.services.yml.optimized \
|
||||
-f docker-compose.clients.yml.optimized up -d
|
||||
|
||||
# Selective services with profiles
|
||||
docker-compose -f docker-compose.services.yml.optimized \
|
||||
--profile members --profile horses up -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### 🚀 Quick Start
|
||||
|
||||
#### 1. **Generate Secrets**
|
||||
```bash
|
||||
# Generate all required secrets
|
||||
./docker/secrets/setup-secrets.sh --all
|
||||
|
||||
# Or generate individually
|
||||
./docker/secrets/setup-secrets.sh --generate
|
||||
./docker/secrets/setup-secrets.sh --validate
|
||||
```
|
||||
|
||||
#### 2. **Configure Environment**
|
||||
```bash
|
||||
# Copy template and customize
|
||||
cp .env.template .env
|
||||
|
||||
# Edit configuration values
|
||||
nano .env
|
||||
```
|
||||
|
||||
#### 3. **Create Data Directories**
|
||||
```bash
|
||||
# Create persistent data directories
|
||||
mkdir -p ./data/{postgres,redis,prometheus,grafana,keycloak,consul,monitoring,desktop-app}
|
||||
```
|
||||
|
||||
#### 4. **Deploy Services**
|
||||
```bash
|
||||
# Start infrastructure
|
||||
docker-compose -f docker-compose.yml.optimized up -d
|
||||
|
||||
# Verify all services are healthy
|
||||
docker-compose -f docker-compose.yml.optimized ps
|
||||
|
||||
# Add microservices
|
||||
docker-compose -f docker-compose.yml.optimized \
|
||||
-f docker-compose.services.yml.optimized up -d
|
||||
|
||||
# Add client applications
|
||||
docker-compose -f docker-compose.yml.optimized \
|
||||
-f docker-compose.services.yml.optimized \
|
||||
-f docker-compose.clients.yml.optimized up -d
|
||||
```
|
||||
|
||||
### 🔄 Migration from Original Configuration
|
||||
|
||||
#### Step 1: Backup Current Setup
|
||||
```bash
|
||||
# Stop existing services
|
||||
docker-compose down
|
||||
|
||||
# Backup current data (optional)
|
||||
cp -r data/ data.backup/
|
||||
```
|
||||
|
||||
#### Step 2: Update Configuration
|
||||
```bash
|
||||
# Generate secrets first
|
||||
./docker/secrets/setup-secrets.sh --all
|
||||
|
||||
# Update environment configuration
|
||||
cp .env.template .env
|
||||
# Edit .env as needed
|
||||
```
|
||||
|
||||
#### Step 3: Deploy Optimized Configuration
|
||||
```bash
|
||||
# Deploy with new configuration
|
||||
docker-compose -f docker-compose.yml.optimized up -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
### 🛡️ Production Security Checklist
|
||||
|
||||
- [ ] **Secrets Generated**: Run secrets setup script
|
||||
- [ ] **File Permissions**: Ensure secret files have 600 permissions
|
||||
- [ ] **Network Isolation**: Use custom Docker networks
|
||||
- [ ] **Resource Limits**: All services have CPU/memory limits
|
||||
- [ ] **Non-root Users**: Applications run as non-privileged users
|
||||
- [ ] **Read-only Mounts**: Configuration mounted read-only
|
||||
- [ ] **Security Options**: `no-new-privileges` enabled
|
||||
- [ ] **Health Checks**: All critical services have health checks
|
||||
- [ ] **Backup Strategy**: Regular data backups configured
|
||||
- [ ] **Monitoring**: Prometheus and Grafana configured
|
||||
- [ ] **Log Management**: Centralized logging configured
|
||||
|
||||
### 🔐 Security Monitoring
|
||||
|
||||
#### Access URLs (Default Configuration)
|
||||
- **Grafana Dashboard**: http://localhost:3000
|
||||
- **Prometheus Metrics**: http://localhost:9090
|
||||
- **Consul UI**: http://localhost:8500
|
||||
- **Keycloak Admin**: http://localhost:8180/admin
|
||||
|
||||
#### Security Metrics to Monitor
|
||||
- Failed authentication attempts
|
||||
- Resource usage patterns
|
||||
- Container restart frequency
|
||||
- Network connection anomalies
|
||||
- Secret access patterns
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### 🔍 Common Issues and Solutions
|
||||
|
||||
#### Issue 1: Secret File Permissions
|
||||
**Problem**: Containers cannot read secret files
|
||||
**Solution**:
|
||||
```bash
|
||||
# Fix permissions
|
||||
chmod 600 docker/secrets/*.txt
|
||||
|
||||
# Or regenerate with correct permissions
|
||||
./docker/secrets/setup-secrets.sh --force
|
||||
```
|
||||
|
||||
#### Issue 2: Resource Constraints
|
||||
**Problem**: Services failing due to resource limits
|
||||
**Solution**:
|
||||
```bash
|
||||
# Check resource usage
|
||||
docker stats
|
||||
|
||||
# Adjust limits in docker-compose files or increase system resources
|
||||
```
|
||||
|
||||
#### Issue 3: Network Connectivity
|
||||
**Problem**: Services cannot communicate
|
||||
**Solution**:
|
||||
```bash
|
||||
# Check network configuration
|
||||
docker network inspect meldestelle_meldestelle-network
|
||||
|
||||
# Verify service health
|
||||
docker-compose -f docker-compose.yml.optimized ps
|
||||
```
|
||||
|
||||
#### Issue 4: Volume Mount Issues
|
||||
**Problem**: Data not persisting or permission errors
|
||||
**Solution**:
|
||||
```bash
|
||||
# Create data directories with correct permissions
|
||||
mkdir -p ./data/{postgres,redis,prometheus,grafana,keycloak,consul}
|
||||
chown -R 999:999 ./data/postgres # PostgreSQL user
|
||||
chown -R 472:0 ./data/grafana # Grafana user
|
||||
```
|
||||
|
||||
### 📊 Health Check Commands
|
||||
|
||||
```bash
|
||||
# Check all service status
|
||||
docker-compose -f docker-compose.yml.optimized ps
|
||||
|
||||
# View service logs
|
||||
docker-compose -f docker-compose.yml.optimized logs [service-name]
|
||||
|
||||
# Check resource usage
|
||||
docker stats
|
||||
|
||||
# Validate secrets
|
||||
./docker/secrets/setup-secrets.sh --validate
|
||||
|
||||
# Test connectivity
|
||||
docker exec meldestelle-api-gateway curl -f http://postgres:5432
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### 🎯 Resource Optimization Guidelines
|
||||
|
||||
#### Memory Allocation Strategy
|
||||
1. **Infrastructure Services**: Higher memory allocation for databases and messaging
|
||||
2. **Application Services**: Balanced CPU/memory for microservices
|
||||
3. **Client Applications**: Lower resource requirements
|
||||
|
||||
#### CPU Allocation Strategy
|
||||
1. **I/O Bound Services** (Database, Redis): Moderate CPU, high memory
|
||||
2. **Compute Bound Services** (Application logic): Higher CPU allocation
|
||||
3. **Static Content Services** (Web apps): Lower overall resources
|
||||
|
||||
#### JVM Tuning for Java Services
|
||||
```yaml
|
||||
environment:
|
||||
JAVA_OPTS: |
|
||||
-XX:MaxRAMPercentage=75.0
|
||||
-XX:+UseG1GC
|
||||
-XX:+UseStringDeduplication
|
||||
-XX:+UseContainerSupport
|
||||
-Djava.security.egd=file:/dev/./urandom
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitoring and Observability
|
||||
|
||||
### 📈 Metrics Collection
|
||||
|
||||
#### Prometheus Metrics
|
||||
- Container resource usage
|
||||
- Application performance metrics
|
||||
- Health check status
|
||||
- Network traffic patterns
|
||||
|
||||
#### Grafana Dashboards
|
||||
- Infrastructure overview
|
||||
- Application performance
|
||||
- Security events
|
||||
- Resource utilization trends
|
||||
|
||||
#### Logging Strategy
|
||||
- Centralized logging via Docker logs
|
||||
- Structured JSON logging for applications
|
||||
- Log rotation and retention policies
|
||||
- Security event logging
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
The Docker configuration optimization provides:
|
||||
|
||||
1. **Enhanced Security**: Complete elimination of hardcoded credentials and implementation of Docker secrets
|
||||
2. **Production Readiness**: Comprehensive resource limits, health checks, and monitoring
|
||||
3. **Improved Performance**: Optimized resource allocation and container configurations
|
||||
4. **Operational Excellence**: Automated secret management, comprehensive documentation, and troubleshooting guides
|
||||
5. **Scalability**: Profile-based deployment and modular service architecture
|
||||
|
||||
### Next Steps
|
||||
|
||||
1. **Deploy optimized configuration** in development environment
|
||||
2. **Validate all security measures** are properly implemented
|
||||
3. **Monitor performance metrics** and adjust resource limits as needed
|
||||
4. **Implement backup and recovery procedures** for persistent data
|
||||
5. **Set up automated monitoring and alerting** for production deployment
|
||||
|
||||
For questions or issues with the optimized configuration, refer to the troubleshooting section or consult the detailed configuration comments in the docker-compose files.
|
||||
Reference in New Issue
Block a user