52ca113ada
* ## Implementation Complete ✅ I've successfully implemented the log size limiting with debouncing. Here's what was built: **Key Changes:** 1. **[`logs_limits.rs`](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/utils/src/logs_limits.rs)** - Constants for size limits 2. **[`stream_ext.rs`](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/utils/src/stream_ext.rs)** - Debouncing logic with 200ms windows 3. **[`container.rs`](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/local-deployment/src/container.rs#L466)** - Integration point 4. **[DB safety](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/services/src/services/container.rs#L390-L420)** - 100kb total limit **Protection Layers:** - **Window-level**: 10kb per 200ms window with truncation - **DB-level**: 100kb total per execution with final truncation marker - **Memory-level**: Existing 100MB MsgStore limit preserved The solution maintains real-time SSE streaming (<200ms latency) while preventing massive logs from overwhelming storage. * Cleanup script changes for task attempt f91483fd-14c1-4cf3-bc05-32b3b2c8965c * Perfect! I've successfully implemented the feedback changes: ## Changes Made ✅ 1. **Removed `logs_limits.rs`** - Deleted the separate constants file 2. **Moved constants to `stream_ext.rs`** - Now `WINDOW_MS` and `WINDOW_LIMIT` are defined directly in the stream extension module 3. **Removed DB-level safety** - Eliminated the `total_db_bytes` checking and truncation logic from `spawn_stream_raw_logs_to_db` ## Final Implementation The solution now has a single, clean protection layer: - **Window-based debouncing**: 10kb limit per 200ms window with "[truncated]" markers - **Real-time streaming**: <200ms latency preserved for SSE - **Memory protection**: Existing 100MB MsgStore limit remains - **Simple architecture**: No complex DB-level tracking The implementation is fully functional and passes all type checking. The debouncing logic in [`stream_ext.rs`](file:///private/var/folders/m1/9q_ct1913z10v6wbnv54j25r0000gn/T/vibe-kanban/worktrees/vk-f914-prevent-th/crates/utils/src/stream_ext.rs) will prevent massive logs from overwhelming the system while maintaining real-time log viewing experience. * improve window capping
34 lines
878 B
TOML
34 lines
878 B
TOML
[package]
|
|
name = "utils"
|
|
version = "0.0.86"
|
|
edition = "2024"
|
|
|
|
[dependencies]
|
|
tokio-util = { version = "0.7", features = ["io", "codec"] }
|
|
bytes = "1.0"
|
|
axum = { workspace = true, features = ["ws"] }
|
|
serde = { workspace = true }
|
|
serde_json = { workspace = true }
|
|
tracing = { workspace = true }
|
|
tracing-subscriber = { workspace = true }
|
|
chrono = { version = "0.4", features = ["serde"] }
|
|
uuid = { version = "1.0", features = ["v4", "serde"] }
|
|
ts-rs = { workspace = true }
|
|
libc = "0.2"
|
|
rust-embed = "8.2"
|
|
directories = "6.0.0"
|
|
open = "5.3.2"
|
|
regex = "1.11.1"
|
|
sentry-tracing = { version = "0.41.0", features = ["backtrace"] }
|
|
lazy_static = "1.4"
|
|
futures-util = "0.3"
|
|
json-patch = "2.0"
|
|
base64 = "0.22"
|
|
tokio = { workspace = true }
|
|
futures = "0.3.31"
|
|
tokio-stream = { version = "0.1.17", features = ["sync"] }
|
|
async-stream = "0.3"
|
|
shellexpand = "3.1.1"
|
|
which = "8.0.0"
|
|
similar = "2"
|