Skip to content
flow8

Database Setup

Database Setup

MongoDB

Version

flow8 requires MongoDB 5.0+. MongoDB 6+ is recommended for production. The Docker Compose files pin to mongo:latest.

Connection

Set MONGODB_URI and MONGODB_DB in your .env:

MONGODB_URI=mongodb://localhost:27017
MONGODB_DB=ud

For authenticated MongoDB:

MONGODB_URI=mongodb://user:password@host:27017/ud?authSource=admin

For MongoDB Atlas:

MONGODB_URI=mongodb+srv://user:password@cluster.mongodb.net/?retryWrites=true&w=majority
MONGODB_DB=ud

Collection Initialization

All collections are created automatically on first run. No schema migration scripts are needed — MongoDB creates collections on first write. The full collection list:

flows              # Flow definitions
layers             # Flowlet (step) definitions
plays              # Flow execution instances
logs               # Execution log entries
groups             # Flow groups
companies          # Organizations (tenants)
company_users      # User-company associations
user_groups        # User group definitions
access             # Fine-grained permissions
schedules          # Cron trigger schedules
retention_policies # Data retention rules
queue_jobs         # Background job queue
channels           # HTTP/WS/MCP channel definitions
channel_routes     # Routes within channels
channel_requests   # Incoming channel requests
kvs                # Key-value store entries
links              # OAuth/API credential storage
component_configs  # Runtime component configurations
system_config      # Installation-wide settings
installations      # Deployment instance records
apps               # Module/app catalog
audit              # Audit log entries
ai_chat            # AI chat sessions
ai_chat_messages   # AI chat messages
ai_generation      # AI generation records
flow_test_suites   # Test suite definitions
flow_test_cases    # Individual test cases
test_fixtures      # Recorded component I/O
play_component_events # Component call logs for fixtures

flow8 does not auto-create indexes. Add these for production performance:

// flows
db.flows.createIndex({ companyId: 1, deleted: 1 })
db.flows.createIndex({ companyId: 1, permanentId: 1 })


// plays
db.plays.createIndex({ flowId: 1, state: 1, createdAt: -1 })
db.plays.createIndex({ companyId: 1, createdAt: -1 })


// layers
db.layers.createIndex({ flowId: 1 })


// kvs
db.kvs.createIndex({ companyId: 1, scopeType: 1, scopeKey: 1, key: 1 }, { unique: true })


// audit
db.audit.createIndex({ companyId: 1, createdAt: -1 })
db.audit.createIndex({ createdAt: 1 }, { expireAfterSeconds: 7776000 }) // optional TTL index


// logs
db.logs.createIndex({ playId: 1 })
db.logs.createIndex({ flowId: 1, createdAt: -1 })

MongoDB Express (Admin UI)

The dev and production compose profiles include MongoDB Express at port 8081:

EXPRESS_USER=admin
EXPRESS_PASS=securepassword

Access at http://localhost:8081 after running compose. Do not expose this port publicly.

Backups

Full Backup with mongodump

Terminal window
mongodump \
  --uri="mongodb://localhost:27017" \
  --db=ud \
  --out=/backups/$(date +%Y%m%d_%H%M%S)


# Compressed backup
mongodump --uri="mongodb://localhost:27017" --db=ud --gzip --archive=/backups/ud_$(date +%Y%m%d).gz

Restore

Terminal window
mongorestore \
  --uri="mongodb://localhost:27017" \
  --db=ud \
  /backups/20240101_120000/ud/


# From compressed archive
mongorestore --uri="mongodb://localhost:27017" --db=ud --gzip --archive=/backups/ud_20240101.gz

Docker-based Backup

Terminal window
docker exec mongo mongodump --db=ud --gzip --archive > /backups/ud_$(date +%Y%m%d).gz

Storage Directory Layout

The storage/ directory holds all runtime data and is gitignored. In Docker it is mounted as the app-data named volume.

storage/
├── db/          # JSON snapshots of certain model data (used by F8DB)
├── data/        # User-uploaded and flow-generated files
│   └── .files/  # Internal file storage, keyed by company/flow
├── logs/        # Application logs written by zerolog
└── tmp/         # Temporary files created during flow execution (auto-cleaned)

The paths are configurable in .config.yaml:

path:
  storage: "./storage"
  tmp: "./storage/tmp"
  db: "./storage/db"
  data:
    root: "./storage/data"
    files: "./storage/data/.files"
  logs: "./storage/logs"

F8DB (Built-in Database)

flow8 includes a lightweight built-in database (F8DB) backed by MongoDB collections: DBLocalDBDatabase, DBLocalDBTable, DBLocalDBRow. It provides SQL-like operations through flow modules without requiring an external database. Data is stored in MongoDB alongside all other platform data.

PostgreSQL (Optional)

PostgreSQL is supported as an optional component for flow-level database operations (not for platform data). It is included in docker-compose.local.yml on port 5744 for testing. Configure via a DBComponentConfig entry with kind: db and impl: postgresql, providing the connection URI in the component settings.