Files
vibe-kanban/crates
Louis Knight-Webb 52ca113ada Prevent the DB from storing massive logs (vibe-kanban) (#633)
* ## Implementation Complete 

I've successfully implemented the log size limiting with debouncing. Here's what was built:

**Key Changes:**
1. **[`logs_limits.rs`](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/utils/src/logs_limits.rs)** - Constants for size limits
2. **[`stream_ext.rs`](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/utils/src/stream_ext.rs)** - Debouncing logic with 200ms windows
3. **[`container.rs`](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/local-deployment/src/container.rs#L466)** - Integration point
4. **[DB safety](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/services/src/services/container.rs#L390-L420)** - 100kb total limit

**Protection Layers:**
- **Window-level**: 10kb per 200ms window with truncation
- **DB-level**: 100kb total per execution with final truncation marker
- **Memory-level**: Existing 100MB MsgStore limit preserved

The solution maintains real-time SSE streaming (<200ms latency) while preventing massive logs from overwhelming storage.

* Cleanup script changes for task attempt f91483fd-14c1-4cf3-bc05-32b3b2c8965c

* Perfect! I've successfully implemented the feedback changes:

## Changes Made 

1. **Removed `logs_limits.rs`** - Deleted the separate constants file
2. **Moved constants to `stream_ext.rs`** - Now `WINDOW_MS` and `WINDOW_LIMIT` are defined directly in the stream extension module
3. **Removed DB-level safety** - Eliminated the `total_db_bytes` checking and truncation logic from `spawn_stream_raw_logs_to_db`

## Final Implementation

The solution now has a single, clean protection layer:

- **Window-based debouncing**: 10kb limit per 200ms window with "[truncated]" markers
- **Real-time streaming**: <200ms latency preserved for SSE
- **Memory protection**: Existing 100MB MsgStore limit remains
- **Simple architecture**: No complex DB-level tracking

The implementation is fully functional and passes all type checking. The debouncing logic in [`stream_ext.rs`](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/utils/src/stream_ext.rs) will prevent massive logs from overwhelming the system while maintaining real-time log viewing experience.

* improve window capping
2025-09-15 21:07:40 +01:00
..
2025-09-15 19:09:33 +00:00
2025-09-15 19:09:33 +00:00
2025-09-15 19:09:33 +00:00
2025-09-15 19:09:33 +00:00
2025-09-15 19:09:33 +00:00