System Design¶

This page provides a detailed overview of Arctyk ITSM's high-level system architecture and component interactions.

System Components¶

Web Application Layer¶

Django 5.2.9 LTS - Handles HTTP requests/responses - Executes business logic - Renders templates - Serves API endpoints - Manages authentication and authorization

Data Layer¶

PostgreSQL 15+ - Primary data storage - Relational data integrity - Full-text search capabilities - JSONB field support for flexible data

Caching & Message Broker¶

Redis 7+ - Session storage - Query result caching - Celery message broker - Rate limiting storage

Background Task Processing¶

Celery 5.3+ - Asynchronous task execution - Scheduled jobs (email notifications, reports) - Retry mechanisms for failed tasks - Distributed task processing

Request/Response Flow¶

Standard Page Request¶

1. User → Browser sends GET request
2. Nginx → Routes to Gunicorn
3. Gunicorn → Passes to Django
4. Django → View function executes
5. View → Queries database via ORM
6. PostgreSQL → Returns data
7. View → Renders template with data
8. Template → Returns HTML
9. Nginx → Serves HTML to browser
10. Browser → Renders page

AJAX Request Flow¶

1. User action → JavaScript triggers AJAX request
2. Request → Django view with X-Requested-With header
3. View → Validates and processes data
4. View → Returns JSON response
5. JavaScript → Updates DOM without page reload

API Request Flow¶

1. Client → Sends API request with auth token
2. Django REST Framework → Validates token
3. View → Checks permissions
4. Serializer → Validates input data
5. Model → Saves to database
6. Serializer → Formats response
7. API → Returns JSON response

Authentication Flow¶

1. User submits login form
2. Django auth backend validates credentials
3. Session created and session ID stored in cookie
4. User object loaded and attached to request
5. User redirected to dashboard

Session Management¶

1. Each request includes session cookie
2. Django loads session from Redis
3. User object retrieved and cached
4. Permissions checked for requested view
5. Session updated with last activity

API Authentication¶

1. Client requests API token
2. Django creates token for user
3. Client includes token in Authorization header
4. Django REST Framework validates token
5. Request processed with user context

Background Job Processing¶

Email Notification Flow¶

1. Ticket created/updated
2. Django signal triggered
3. Celery task queued in Redis
4. Celery worker picks up task
5. Email sent via SMTP
6. Task result stored
7. Retry on failure (3 attempts)

Scheduled Report Generation¶

1. Celery beat scheduler triggers task
2. Task queries database for report data
3. PDF generated from template
4. PDF emailed to recipients
5. Task marked complete

Data Flow Diagrams¶

Ticket Creation Flow¶

┌─────────┐
│  User   │
└────┬────┘
     │ Submit form
     ▼
┌─────────────┐
│   View      │
│ (Validate)  │
└────┬────────┘
     │
     ▼
┌─────────────┐
│TicketForm   │
│ (Clean)     │
└────┬────────┘
     │ Valid
     ▼
┌─────────────┐
│   Model     │
│  (Save)     │
└────┬────────┘
     │
     ▼
┌─────────────┐
│  Database   │
└────┬────────┘
     │
     ▼
┌─────────────┐
│   Signal    │
│(post_save)  │
└────┬────────┘
     │
     ▼
┌─────────────┐
│Celery Task  │
│(Email send) │
└─────────────┘

Ticket List Query Flow¶

┌─────────┐
│  User   │
└────┬────┘
     │ Request page
     ▼
┌─────────────┐
│   View      │
│(QuerySet)   │
└────┬────────┘
     │
     ▼
┌─────────────┐
│  ORM Query  │
│ (SQL build) │
└────┬────────┘
     │
     ▼
┌─────────────┐
│  PostgreSQL │
└────┬────────┘
     │ Results
     ▼
┌─────────────┐
│  Template   │
│  (Render)   │
└────┬────────┘
     │ HTML
     ▼
┌─────────────┐
│  Browser    │
└─────────────┘

Deployment Architecture¶

Development Environment¶

┌───────────────────────────────────────┐
│         Docker Compose                │
│                                       │
│  ┌────────┐  ┌──────────┐  ┌──────┐ │
│  │  web   │  │    db    │  │redis │ │
│  │ Django │  │PostgreSQL│  │      │ │
│  └────────┘  └──────────┘  └──────┘ │
│                                       │
│  ┌────────┐                          │
│  │ celery │                          │
│  │ worker │                          │
│  └────────┘                          │
└───────────────────────────────────────┘

Production Environment¶

                    Internet
                       │
                       ▼
               ┌───────────────┐
               │     Nginx     │
               │ (Port 80/443) │
               └───────┬───────┘
                       │
        ┏━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┓
        ▼                              ▼
┌───────────────┐            ┌──────────────────┐
│   Gunicorn    │            │  Static Files    │
│ (8 workers)   │            │  (Nginx serve)   │
└───────┬───────┘            └──────────────────┘
        │
        ▼
┌───────────────┐
│    Django     │
│  Application  │
└───────┬───────┘
        │
┏━━━━━━━┻━━━━━━━┓
▼               ▼
┌─────────┐  ┌────────┐
│PostgreSQL│  │ Redis  │
└─────────┘  └───┬────┘
                 │
                 ▼
           ┌──────────┐
           │  Celery  │
           │  Worker  │
           └──────────┘

Scalability Considerations¶

Current Capacity¶

Users: 100-500 concurrent users
Tickets: Millions of tickets
Requests: 100-200 req/sec
Database: 100GB+ storage

Horizontal Scaling Strategy¶

Web Layer¶

Load Balancer
    ├── Gunicorn Instance 1
    ├── Gunicorn Instance 2
    └── Gunicorn Instance 3

Worker Layer¶

Celery
    ├── Worker 1 (email tasks)
    ├── Worker 2 (report tasks)
    └── Worker 3 (general tasks)

Database Layer¶

PostgreSQL Primary
    ├── Read Replica 1
    └── Read Replica 2

Performance Optimization¶

Database Indexing
Primary keys (automatic)
Foreign keys
Frequently queried fields (status, assignee, created_at)
Query Optimization
Use select_related() for foreign keys
Use prefetch_related() for reverse foreign keys
Pagination for large result sets
Caching Strategy
Template fragment caching
Query result caching (5-minute TTL)
Session storage in Redis
Static Assets
CDN for static files (future)
Asset compression (CSS/JS minification)
Browser caching headers

Security Architecture¶

Network Security¶

Internet → Cloudflare CDN → Nginx → Gunicorn → Django
           (DDoS protection)  (SSL)   (WSGI)    (App)

Application Security¶

HTTPS Only: Force SSL redirect
CSRF Protection: Token validation on forms
SQL Injection: Parameterized queries (ORM)
XSS Prevention: Template auto-escaping
Session Security: HTTPOnly, Secure, SameSite cookies

Data Security¶

Password Hashing: PBKDF2 with SHA256
Secret Management: Environment variables
Database Encryption: Encrypted connections (SSL)
Audit Logging: All ticket changes logged

Monitoring & Observability¶

Application Metrics¶

Request rate and latency
Error rates (4xx, 5xx)
Database query performance
Celery task queue length

Infrastructure Metrics¶

CPU and memory usage
Disk I/O and storage
Network throughput
Database connections

Logging¶

Application logs (Django)
Web server logs (Nginx)
Database logs (PostgreSQL)
Worker logs (Celery)

Technology Choices¶

Why Django?¶

✅ Mature framework - Battle-tested in production
✅ Batteries included - ORM, auth, admin out of the box
✅ Security defaults - CSRF, XSS, SQL injection protection
✅ Great documentation - Extensive official docs
✅ Large ecosystem - Many packages available

Why PostgreSQL?¶

✅ ACID compliance - Data integrity guarantees
✅ Advanced features - JSONB, full-text search, GIS
✅ Performance - Excellent for complex queries
✅ Django support - First-class ORM support
✅ Open source - No licensing costs

Why Redis?¶

✅ Fast - In-memory data structure store
✅ Versatile - Cache, message broker, session store
✅ Reliable - Persistence options available
✅ Simple - Easy to deploy and manage

Why Celery?¶

✅ Reliable - Proven task queue system
✅ Flexible - Supports many message brokers
✅ Monitoring - Flower for task monitoring
✅ Retries - Automatic retry with backoff

Future Architecture Considerations¶

Potential Enhancements¶

Microservices - Split into smaller services for large scale
GraphQL API - Alternative to REST for complex queries
Elasticsearch - Advanced search capabilities
WebSockets - Real-time notifications
Container Orchestration - Kubernetes for large deployments

When to Consider Changes¶

Users > 1000: Add load balancer and multiple app servers
Tickets > 10M: Consider database sharding
Global users: Add CDN and geographic distribution
Real-time needs: Implement WebSocket support
Search complexity: Add Elasticsearch