Pythmata Architecture Overview¶

Introduction¶

Pythmata is a Python-based BPMN workflow engine designed to provide robust process automation capabilities with a modern React frontend. This document outlines the high-level architecture, core components, and key design decisions.

System Architecture¶

Overview Diagram¶

+----------------+     +----------------+     +----------------+
|    Frontend    |     |    Backend    |     |   Database    |
|    (React)     |<--->|   (Python)    |<--->|  (PostgreSQL) |
+----------------+     +----------------+     +----------------+
                           ^      ^
                           |      |
                      +----+      +----+
                      |                |
              +----------+        +---------+
              |  Redis   |        | RabbitMQ|
              |(State)   |        |(Events) |
              +----------+        +---------+

Core Components¶

1. BPMN Engine¶

Parser & Validator: XML schema validation with custom extension support
Execution Engine: Token-based process execution
State Management: Redis-backed persistence with transaction support
Event System: Comprehensive BPMN event handling

2. Frontend Application¶

Process Designer: Visual BPMN modeling interface
Process Monitor: Real-time instance monitoring
Task Management: User task handling and forms
Admin Interface: Process and instance management

3. Backend Services¶

API Layer: RESTful endpoints with FastAPI
Process Service: Process definition management
Instance Service: Process instance execution
Task Service: Task management and assignment

4. Connection Management¶

Base Connection Manager: Common connection lifecycle handling
State tracking
Automatic reconnection
Error handling and recovery
Service-Specific Implementations:
Database connections with connection pooling
Redis state management connections
RabbitMQ event system connections

5. Storage Layer¶

PostgreSQL: Process definitions and instance data
Redis: State management and caching
RabbitMQ: Event handling and message queues

Key Features¶

Process Execution¶

Token-based execution model
Subprocess and call activity support
Multi-instance activities
Transaction management
Compensation handling

State Management¶

Distributed state tracking
Variable scoping
Token lifecycle management
Transaction boundaries
Event correlation

Integration Capabilities¶

External service tasks
Message correlation
Timer events
Signal handling
Error management

Design Decisions¶

1. Token-Based Execution¶

Enables precise process state tracking
Supports parallel execution paths
Facilitates transaction management
Enables state persistence and recovery

2. Connection Management Pattern¶

Unified Connection Interface: Common connection lifecycle across services
State Management: Accurate tracking of connection states
Automatic Recovery: Built-in reconnection for transient failures
Error Handling: Consistent error propagation and recovery
Resource Management: Proper cleanup and disposal of connections

3. Redis State Management¶

Fast in-memory state access
Built-in pub/sub for events
Atomic operations for consistency
Distributed locking support

3. Event-Driven Architecture¶

Loose coupling between components
Scalable message processing
Asynchronous execution support
Real-time updates

4. Modern Frontend Stack¶

React for component-based UI
TypeScript for type safety
Material-UI for consistent design
WebSocket for real-time updates

Security Considerations¶

Authentication & Authorization¶

JWT-based authentication
Role-based access control
Process-level permissions
API security measures

Data Protection¶

Secure variable handling
Encrypted connections
Audit logging
Access controls

Performance Considerations¶

Scalability¶

Horizontal scaling support
Load balancing ready
Distributed state management
Event-driven architecture

Optimization¶

Redis caching
Efficient token management
Query optimization
Resource pooling

Deployment Architecture¶

Container-Based Deployment¶

+-------------------+
|   Load Balancer   |
+-------------------+
         |
   +-----+-----+
   |     |     |
+-----+ +-----+ +-----+
| Web | | API | | Job |
+-----+ +-----+ +-----+
   |     |     |
+-------------------+
|     Databases     |
+-------------------+

Components¶

Frontend containers
Backend API containers
Job processing workers
Database clusters
Message brokers