Roadmap

Fibril’s priorities are recovery safety, complete client semantics, compatibility, and cluster operations. This page tracks remaining work. Current capabilities and their limits are documented in implemented surface; project status summarizes their maturity.

Immediate rounds

The next rounds prioritize bounded operational work, recovery latency and adoption of the retained delivery experiment. Client and stream work can proceed as separate rounds when their dependencies permit.

1. Admin settings and recovery visibility. Unify defaults/validation metadata and extend requested-versus-applied reporting beyond node-local preallocation. Add explicit remote-node edit routing and classify additional live storage controls. Extend retained-generation and staging accounting to per-queue attribution before adding automatic reclamation. Extend worker timelines with cross-node detection and client-reconnect correlation.
2. Checkpoint policy and recovery overhead. Measure the opt-in agreed checkpoint path under higher sustained rates and many partitions. Tune the implemented event/byte/age triggers, verification budgets and retention limits; extend node-level disk accounting to per-queue attribution and stage-level resource costs. Evaluate remaining live-payload verification costs and silent-endpoint discovery across SDKs; explicit Rust EOF handling and per-replica seal/inspection overlap are implemented. See the fast-recovery plan.
3. Speculative delivery adoption. Reconcile the retained local-queue prototype with current recovery and ordered application. Close ACK-before-durability, crash/error, slow-consumer, memory-budget and expiry/fallback gaps, then repeat representative physical-storage measurements. Production configuration and confirmation/identity contracts are required before adoption. Replicated speculation remains a separate increment. Replication before owner fsync already ships as early replication, on by default.
4. Python and TypeScript performance. Profile scheduling, decoding, delivery and flushing against shared broker workloads. Apply useful transport changes across SDKs where the same mechanism applies, retaining consistent semantics.
5. Streams completeness. Resolve fail/retry and reconnect-ACK behavior across clients, then add stream and consumer fan-out workloads to the shared harness.
6. Many-queue resource use. Measure adaptive staging with idle, intermittently active and busy queues under realistic memory pressure. Track retained recovery disk separately from active logs and assess reclamation only after visibility.

A smaller optional round can add a coherent moving dashboard scenario: backlog builds, consumers catch up and a node recovers. Reuse production views and clearly label the simulated state. Allocator/THP repeats remain evidence-driven follow-ups.

Medium-term directions

These directions follow the immediate rounds. They are exploration and product targets, with implementation scope and acceptance refined individually.

Ordering across partitioned queues

Explore useful per-key ordering while distributing customers or jobs across partitions. Specify delivery versus processing order, retries, consumer ownership and repartitioning. Evaluate queue-wide ordering separately, including its sequencing and throughput costs.

Scaling across partitions, nodes and drives

Map aggregate and per-partition capacity with replica count as an independent axis. Spread owners and storage across nodes and physical drives, verify client headroom, and measure latency, CPU, memory and durable behavior at offered rates and saturation. Explore a million messages per second as a measurement target; report sustainable operating points and exact hardware/configuration conditions.

Low-latency service routing

Build a representative request → worker → response example with correlation, deadlines, retries and idempotency. Define worker completion, response durability and request acknowledgment ordering, including duplicate responses after failure. Measure both queue legs and full round-trip latency. Integrate speculative delivery only under its validated contract; pre-staging delivery and write avoidance remain a further design requiring explicit ordering and crash semantics.

Public/private endpoints and routing fallback

Private brokers and public brokers that forward to them are implemented for the full client surface, queues, streams and exclusive consumer groups, with upstream session resume and dashboard marking, see private brokers. Remaining are later options: tagged private endpoints for internal clients, a multiplexed upstream link if connection counts become a limit, and a client tunnel for continuity across a public broker failure. Overlapping topology groups and arbitrary multi-hop routing remain later options that require a concrete deployment use case. The design is in private nodes and internal routing.

MCP server and agent integration

Add a Model Context Protocol (MCP) server for structured, read-only inspection of queues, replication, recovery evidence and settings adoption. Return bounded results with observation times, explicit unknowns and links to relevant dashboard views. Reuse typed operations, validation and authorization across the dashboard, CLI and MCP interface, with shared diagnostic definitions. The proposed design is MCP server.

Extend the interface with scoped, audited administrative actions after the read-only surface is validated. Explore Python and TypeScript integrations for agent task submission, progress and results, with explicit confirmation, retry, idempotency and cancellation contracts. Use a small ingestion pipeline example to assess a client-side graph layer and integration with existing agent frameworks. Keep workflow state and external side-effect guarantees explicit. Validate the experience with worker interruption and recovery, alongside ordinary service work.

Path to 1.0

The 1.0 acceptance criteria cover four areas:

1. Cluster confidence: replication, recovery and failover withstand deterministic fault tests, sustained chaos/soak runs, and multi-node operation.
2. Compatibility: stable client APIs, versioned wire and storage formats, a written support policy, and automated compatibility checks.
3. Operational lifecycle: predictable drain, restart, reconnect and upgrade behavior, with typed client outcomes and failure-recovery runbooks.
4. Security: authenticated and encrypted client, admin and inter-broker connections, with documented trust configuration and credential lifecycle.

The remaining work is grouped below by dependency and operational impact.

Recovery safety

• Implement and validate Windows durable metadata replacement before enabling checkpoint installation and recovery seals on that platform.
• Extend accepted-history recovery coverage to packet-level asymmetric partitions, whole-process interruption during learner transfer and candidate replacement outside the fixed proposed replica set. Preserve confirmed history and fencing through every transition; see the failover plan.
• Extend local error/cancellation/process-kill coverage to power-loss persistence tests, larger records/snapshots and authoritative stream-state recovery.
• Design composite reconstruction for crossed payload/event histories and safe reclamation of retained recovery data. Keep unresolved authoritative divergence fenced and visible to operators.

Client lifecycle and compatibility

Complete lifecycle semantics before freezing the public APIs:

• Define stream fail, retry and delayed-retry behavior, including typed rejection of operations the stream cursor model cannot support.
• Ensure a resumed stream’s cursor ACK waits for re-subscription or is retained for retry until the subscription is ready.
• Add focused coverage for automatic recreation continuity and fuller multi-broker exclusive-consumer-group scenarios.

The compatibility work follows this order:

1. Finish the Offset/Epoch and Topic/Group type review, including the proposed Arc<str> representation, while preserving existing wire bytes.
2. Define wire and durable-format versioning, migration and support policies.
3. Review and freeze the Rust, TypeScript, Python, Go and C# client APIs.
4. Enforce the policies with vector regeneration checks, baseline-client/new-broker tests, mixed-version rolling upgrades and storage fixtures. Select explicit supported baselines and expand fixture coverage to coordination storage, users and runtime settings.

Cluster operations

• Design storage-health-driven relinquishment and cooperative owner handoff. Let an owner propose a successor with evidence for a final durable boundary. Preserve controller authority, writer fencing and the configured confirmation requirement. Add safe fallback for interrupted handoffs and verified replacement of persistently failing followers.

• Unify configuration defaults and validation metadata across startup seeds, the runtime API and the dashboard. Report requested versus locally applied settings, node/cluster scope and application boundaries; complete safe startup visibility for coordination and listener configuration.

• Extend node-local storage settings with explicit remote-node routing and classify fsync, batching and adaptive-buffer controls by their safe application boundary. Preserve in-flight durability requirements. Extend failure acceptance to full-disk persistence and process interruption during updates; report applied versions for settings beyond segment preallocation.

• Assess live updates for coordination heartbeat interval and liveness TTL, with coupled validation, safe adoption across nodes, controller propagation, admin visibility and rollback. Keep Raft election timing separate.

• Extend recovery acceptance to sustained high-rate and aged workloads, distinguishing retained history, live backlog and checkpoint age. Extend the internal matched RabbitMQ/JetStream history checks to more sustained traffic and failure modes. Add safe reclamation of retained recovery generations and measure shared-segment repair costs under storage pressure.

• Target common-case queue recovery within two seconds: measure release-build stage costs, remove avoidable waits, and tune agreed checkpoints with suffix-only comparison while preserving payload dependencies and quorum proof. See the staged fast-recovery plan.

• Extend eager failover acceptance to packet-level partitions, CPU/storage stalls, planned drains and durable streams. Measure detection, recovery time, false reassignments and healthy traffic disruption under the failover acceptance scenarios.

• Provide one-command node enrollment with short-lived invitations, trust verification and configuration exchange.

• Aggregate replication lag and in-sync status across brokers in the CLI and topology view.

• Add coordinated queue deletion, replicated purge, and node scale-up/scale-down workflows with explicit failure outcomes.

• Narrow exclusive-consumer subscriptions to their assigned partition subset.

• Define replicated policy for cluster-wide runtime locks while retaining local hardware settings such as storage paths and log tuning. The proposed design is configuration sources.

• Expand deployment, failover and rolling-upgrade runbooks.

• Slow and then pause publishers before disk or memory run out, add per-queue size limits and share spare segment files across a node. The proposed design is resource limits.

Large-history recovery availability

Extend large-history acceptance to sustained traffic, multiple simultaneous recoveries and storage pressure. Measure per-node state memory and unique physical temporary disk use across larger replica sets. Exercise combined source loss, process restart and repeated candidate replacement under those loads. Extend operation-level budget diagnostics with durable progress across continuation phases. Maintained boundary coverage is described in the recovery acceptance guide.

Add safe quarantine and refetch for corrupt tentative caches, with fresh authority and visible storage errors. Reclaim retained generations and installed stages only after proving that recovery and learner readers no longer need them. Expand membership recovery beyond a fixed proposed replica set.

The resumable phase design records the authority and retry requirements. Partial journals remain tentative, and completed local phases require fresh authority before installation or serving.

Assess pairwise history comparison scaling separately. Preserve equivalent evidence when reducing repeated reads or changing comparison scheduling.

Performance and observability

• Profile client scheduling, decoding and delivery costs in Python and TypeScript.
• Track publish and delivery capacity, latency and memory across payload sizes, storage devices, partition counts and replication policies.
• Test larger replica groups, starting with three and five replicas, then seven where resources allow. Measure steady-rate and saturation latency, throughput, CPU, memory and network use, plus owner-loss recovery with retained history. Record quorum policy and shared-host limits alongside each result.
• Improve memory use under large backlogs, high inflight load and many idle queues.
• Improve bulk DLQ replay, message inspection and sparse-queue diagnostics. The proposed replay design is dead-letter replay.
• Extend the shared benchmark harness with stream workloads, portable cluster/fault provisioning and repeated workload matrices.
• Let an existing queue change between durable and buffered with a fenced transition. Buffered queues themselves shipped as an experimental opt-in in October 2026, see buffered queues and the design record.
• Add OpenTelemetry export.
• Ship Grafana dashboards and alert rules, add latency, fsync, process and disk metrics, and keep telemetry history across restarts. The proposed design is monitoring presets and history.

Frontend maintainability

Audit the admin dashboard, demo, documentation components and landing page for ad hoc scripts, implicit types, globals and duplicated behavior. Define shared module boundaries and typed API/event contracts, and assess TypeScript or checked JavaScript with a reproducible build step. Keep canonical assets and production/demo behavior shared. Include accessibility, rendering safety, lifecycle cleanup and consistent styling. Plan incremental changes with behavior tests and clean Docker, CI and embedded-asset builds.

Dashboard demo and interactive documentation

Pin matching demo assets when archiving documentation versions. Extend inline examples to additional dashboard pages where they clarify operational behavior. Assess coherent moving scenarios for backlog growth, slow consumers and recovery.

Extend the shared stories with polling versus push/wakeup replication, richer controller planning and multi-partition failures. Expand local and replicated speculation scenarios as their correctness contracts evolve. Distinguish eager failure detection from early replication, and label implemented, experimental and proposed behavior. Show delivery, consumer ACK, local/replica persistence and publisher-confirmation boundaries explicitly. Pair overview diagrams with detailed recovery steps and failure branches. Maintain shared diagram sources and visual conventions, with readable static fallbacks and vector exports suitable for presentations. Version diagrams alongside the behavior documented on each page.

Longer-term options

These require a concrete use case or further design:

• Reclaim inflight ownership across broker restart and transfer sessions across nodes, with a defined startup grace window and redelivery policy.
• Per-topic authorization and tenancy controls. The proposed design for roles and permissions is access control.
• Additional clients, such as Java.
• Richer bounded stream filters and client-side wildcard publishing.
• Queue expiration based on coordinated inactivity.

Out of scope

Transactions and transactional publish/consume workflows are outside the broker’s scope. Content-routing scripts and SQL/stream processing belong in an external layer that uses the broker’s publish and subscribe APIs.