Adding New Modules

Adding New Modules

Modules are the individual units of work in flow8 flows. Each module is a Go struct implementing a defined interface, auto-discovered at startup via reflection.

Step-by-Step Guide

1. Create the Module File

Create a new file in pkg/plugins/layers/v2/:

pkg/plugins/layers/v2/module_mymodule.go

Naming convention: module_<lowercase_name>.go. Use underscores, no dashes.

package layers

import "strings"

// Step 2: Define the struct, embedding *PluginModule
type ModuleMyModule struct {
    *PluginModule
}

// Step 3: Factory method — THIS NAME IS CRITICAL
// Must be: NewModule<CamelCaseName> on receiver *PluginModule
// The reflection system scans for methods matching this pattern at startup.
func (p *PluginModule) NewModuleMyModule() *ModuleMyModule {
    return &ModuleMyModule{PluginModule: p}
}

2. Implement the Interface

All five methods are required:

// Type returns the unique module identifier used in flow definitions
func (m *ModuleMyModule) Type() string {
    return "my-module"  // kebab-case, globally unique
}

// RequiredComponents lists component kinds this module needs.
// Valid values: "storage", "ai", "db", "request", "console"
func (m *ModuleMyModule) RequiredComponents() []string {
    return []string{}
    // Example if you need HTTP and AI:
    // return []string{"request", "ai"}
}

// ArgsExample defines the expected input schema.
// Used for UI form generation and documentation.
func (m *ModuleMyModule) ArgsExample() map[string]any {
    return map[string]any{
        "input":   "text to process",
        "prefix":  "HELLO",
        "enabled": true,
    }
}

// OutExample defines the output schema.
// Used for expression autocomplete in the UI.
func (m *ModuleMyModule) OutExample() map[string]any {
    return map[string]any{
        "result": "processed text",
        "length": 42,
    }
}

3. Implement Run()

func (m *ModuleMyModule) Run(params Params) Response {
    // --- Access arguments ---
    input, ok := params.Args["input"].(string)
    if !ok || input == "" {
        return params.Response().Fail("input is required")
    }

    prefix, _ := params.Args["prefix"].(string)
    enabled, _ := params.Args["enabled"].(bool)

    // --- Log progress (visible in play layer output) ---
    params.Log("Processing input of length " + string(rune(len(input))))

    // --- Access a component ---
    // reqComp := params.GetComponent("request")  // HTTP client
    // aiComp := params.GetComponent("ai")        // AI provider

    // --- Access KV store ---
    // val, err := params.KV.Get("flow", params.FlowId, "myKey")

    // --- Business logic ---
    result := input
    if enabled {
        result = prefix + ": " + strings.ToUpper(input)
    }

    // --- Return DONE with output data ---
    return params.Response().
        Done(map[string]any{
            "result": result,
            "length": len(result),
        }).
        // Optionally update KV store
        WithKV([]KVUpdate{
            {Scope: "flow", Key: "lastResult", Value: result},
        })
}

4. Return States

Method	Meaning	Effect on Play
params.Response().Done(data)	Success	Next flowlets execute; data available in expressions
params.Response().Fail("reason")	Error	Play moves to FAIL state (unless retry configured)
params.Response().Skip()	Skipped	Flowlet skipped; play continues

5. Accessing Components

// HTTP requests (outbound)
reqComp, err := params.GetTypedComponent("request")
// use reqComp.Do(req) to make HTTP calls

// AI provider
aiComp, err := params.GetTypedComponent("ai")
// use aiComp.Chat(messages, options) for AI calls

// Storage (file I/O)
storComp, err := params.GetTypedComponent("storage")
// use storComp.Read(path) / storComp.Write(path, data)

// Database
dbComp, err := params.GetTypedComponent("db")
// use dbComp.Query(sql, args...)

6. Write Tests

Create module_mymodule_test.go:

package layers_test

import (
    "testing"
    "github.com/stretchr/testify/assert"
)

func TestModuleMyModule_BasicTransform(t *testing.T) {
    module := setupTestModule(t)  // helper that initializes test env

    params := Params{
        Args: map[string]any{
            "input":   "hello world",
            "prefix":  "GREET",
            "enabled": true,
        },
        // ... test fixture setup
    }

    response := module.Run(params)

    assert.Equal(t, StateDone, response.State)
    assert.Equal(t, "GREET: HELLO WORLD", response.Out["result"])
}

7. Verify Discovery

Set APPS_FORCEUPDATE=true, restart the server, then check:

# Module should appear in the apps collection
mongosh ud --eval "db.apps.find({ alias: 'my-module' }).pretty()"

Or via API:

GET /api/apps | jq '.[] | select(.alias == "my-module")'

8. Update Module Catalog CSV

go run cmd/modulescsv/main.go

Commit the updated modules.csv.

Common Mistakes

Mistake	Effect	Fix
Factory method on wrong receiver (*ModuleMyModule instead of *PluginModule)	Not discovered	Change receiver to *PluginModule
Method name doesn’t start with NewModule	Not discovered	Rename to NewModule<Name>
File in wrong package (package main instead of package layers)	Compile error	Fix package declaration
Returning nil from ArgsExample()	UI form broken	Return an empty map[string]any{}
Not handling type assertion failure	Runtime panic	Always use comma-ok: val, ok := args["key"].(string)