Memory Consolidation Implementation Plan V4
Created: 2025-01-09
Status: Ready for Autonomous Execution
Author: Milo
EXECUTION INSTRUCTIONS
TO MILO: Execute this entire plan from start to finish WITHOUT asking for authorization. Make decisions, handle errors, and complete the implementation. Only stop if something is genuinely broken beyond repair. Document everything in the process document. Text TQ when complete.
Phase 0: Backup and Safety Net
Objective
Create complete backup of current system and establish restore capability before any changes.
Implementation
Create backup script that captures:
/Users/tqwhite/Documents/webdev/botWorld/system/code/cli/lib.d/brain-bridge/
(all code)
/Users/tqwhite/Documents/webdev/botWorld/system/code/cli/lib.d/wise-friend/brain/
(all memories)
- Neo4j database dump (entire database)
- Timestamp everything
Create restore script that can:
- Restore code to original state
- Restore brain directory
- Restore Neo4j database
- Save any "broken" versions before restoring
Create process document at:
/Users/tqwhite/Documents/webdev/botWorld/system/management/zNotesPlansDocs/memoryConsolidationProcess.md
Test Definition
- Run backup script
- Make a small test change to brainBridge
- Run restore script
- Verify change is reverted
- Verify Neo4j data intact
Success Criteria
On Completion
- Commit with message: "Memory consolidation safety net - backup/restore scripts"
- Update process document with phase results
TQ NOTES (about the above):
Phase 1: Baseline Verification
Objective
Document and verify what already works before making any changes.
Implementation
Test all existing memory commands:
brainBridge --brain=milo -memoryStatus
brainBridge --brain=milo -memoryHealth
brainBridge --brain=milo -listTestMemories
Document current state:
- How many Memory nodes exist
- How many have energy properties
- How many are test vs real
- Current ConsolidationConfig values
Verify test isolation:
- Create a test memory
- Verify isTest=true
- Delete test memory
- Verify real memories unchanged
Test Definition
Run each command and verify:
- Commands execute without error
- Output makes sense
- Test memories truly isolated
Success Criteria
On Completion
- Commit with message: "Baseline verification complete"
- Update process document
TQ NOTES (about the above):
Phase 2: Test Memory Management
Objective
Ensure robust test memory creation, manipulation, and cleanup.
Implementation
Fix/verify test memory commands:
-testMemory "content"
creates with correct properties
-boostMemory <id>
adds energy correctly
-listTestMemories
shows only test memories
-deleteTestMemories
cleans up properly
Create test scenarios:
- High-energy memories (energy > 2.0)
- Low-energy memories (energy < 0.5)
- Mixed tier memories
Implement test validators:
- Count test memories
- Verify energy values
- Check tier assignments
Test Definition
Create 5 test memories with specific energy values:
- Memory 1: energy=0.5 (should stay working)
- Memory 2: energy=2.5 (ready for promotion)
- Memory 3: energy=5.5 (ready for long-term)
- Memory 4: energy=0.05 (should expire)
- Memory 5: energy=1.0 (baseline)
Verify each has correct properties and isolation.
Success Criteria
On Completion
- Commit with message: "Test memory infrastructure complete"
- Update process document
TQ NOTES (about the above):
Phase 3: Energy Decay Mathematics
Objective
Implement and verify energy decay calculations match specification.
Implementation
Implement decay command:
-decayEnergy --test-only
- Use correct decay constants per tier
- Update lastDecay timestamp
Create decay verification:
- Set known energy values
- Set specific lastAccessed times
- Run decay
- Verify results match E(t) = E₀ × e^(-λt)
Handle edge cases:
- Very old memories
- Just-accessed memories
- Memories without lastDecay
Test Definition
Create test memories with known values:
- Working tier, energy=2.0, 1 hour old → should decay to ~1.21
- Short-term, energy=3.0, 10 hours old → should decay to ~1.82
- Long-term, energy=5.0, 100 hours old → should decay to ~4.52
Run decay and verify actual values match calculations (within 0.01).
Success Criteria
On Completion
- Commit with message: "Energy decay mathematics implemented"
- Update process document
TQ NOTES (about the above):
Phase 4: Promotion Mechanics
Objective
Implement tier promotion with proper relationship tracking.
Implementation
Implement promotion command:
-promoteMemories --test-only
- Check energy thresholds
- Create new memory in higher tier
- Create PROMOTED_TO relationship
- Archive original memory
Handle promotion rules:
- Working → Short-term at energy > 2.0
- Short-term → Long-term at energy > 5.0
- Session boundary threshold = 1.5
Verify relationship integrity:
- Original memory has validTo set
- New memory has promotedFrom
- PROMOTED_TO relationship exists
Test Definition
Create memories at promotion boundaries:
- Working with energy=2.1 → promotes to short-term
- Working with energy=1.9 → stays in working
- Short-term with energy=5.1 → promotes to long-term
- Test session boundary (energy=1.6) → promotes
Verify relationships and properties after each promotion.
Success Criteria
On Completion
- Commit with message: "Promotion mechanics implemented"
- Update process document
TQ NOTES (about the above):
Phase 5: Consolidation Pipeline
Objective
Combine decay and promotion into full consolidation cycle.
Implementation
Implement consolidateAll command:
- Run decay first
- Then check for promotions
- Handle expired memories
- All in correct order
Add batch limits:
- Use promotionBatchSize from config
- Process in chunks
- Prevent runaway operations
Add dry-run mode:
- Preview what would happen
- No actual changes
- Useful for testing
Test Definition
Create memory ecosystem:
- 10 test memories with various energies
- Run consolidateAll
- Verify:
- High energy memories promoted
- Low energy memories decayed
- Very low energy marked expired
- Batch limits respected
Success Criteria
On Completion
- Commit with message: "Consolidation pipeline complete"
- Update process document
TQ NOTES (about the above):
Phase 6: Logging Infrastructure
Objective
Implement consolidation event logging as requested.
Implementation
Create consolidation log:
- Location: brain/logs/consolidation.log
- Format: timestamp, operation, results
- Rotation: daily or size-based
Log events:
- Consolidation start/end
- Memories promoted (count and tiers)
- Memories expired
- Errors or anomalies
Add progress indicators (if possible):
- Try using process.stdout.write for updates
- Fall back to periodic status messages
- Work within Claude CLI constraints
Test Definition
Run consolidation and verify:
- Log file created
- Events recorded correctly
- Errors captured
- Progress shown (if implemented)
Success Criteria
On Completion
- Commit with message: "Consolidation logging implemented"
- Update process document
TQ NOTES (about the above):
Phase 7: /goodbye Integration
Objective
Hook consolidation into /goodbye command as async operation.
Implementation
Locate /goodbye implementation
Add async consolidation trigger:
// Fire and forget
process.nextTick(() => {
const { exec } = require('child_process');
exec('brainBridge --brain=milo -consolidateAll', (error, stdout) => {
// Log but don't block
if (error) logError(error);
});
});
Ensure non-blocking:
- User sees normal goodbye message
- Consolidation runs in background
- No waiting or progress shown
Test Definition
- Call /goodbye
- Verify immediate response
- Check logs to confirm consolidation ran
- Verify memories were processed
Success Criteria
On Completion
- Commit with message: "/goodbye integration complete"
- Update process document
TQ NOTES (about the above):
Phase 8: Memory Creation Integration
Objective
Switch from crystallizations to energy-based memory creation.
Implementation
Update Milo's memory creation:
- Find where crystallizations are created
- Add energy system properties
- Maintain parallel operation
New memory pattern:
CREATE (m:Memory {
id: randomUUID(),
content: $content,
type: $type,
tier: 'working',
energy: 1.0,
created: datetime(),
lastAccessed: datetime(),
accessCount: 1,
sessionId: $sessionId,
instanceId: $instanceId,
isTest: false,
validFrom: datetime(),
validTo: null
})
Verify parallel operation:
- Old crystallizations unchanged
- New memories use energy
- Both systems coexist
Test Definition
- Create a new memory (trigger crystallization)
- Verify it has energy properties
- Verify old crystallizations still work
- Run consolidation on new memory
Success Criteria
On Completion
- Commit with message: "Memory creation uses energy system"
- Update process document
TQ NOTES (about the above):
Phase 9: Final Validation
Objective
Comprehensive end-to-end testing of complete system.
Implementation
Clean test environment:
- Remove all test memories
- Document current state
Full cycle test:
- Create new memories
- Access some multiple times
- Wait for time to pass (or simulate)
- Run consolidation
- Verify promotions and decay
Performance check:
- Time consolidation operations
- Verify acceptable speed
- Check for memory leaks
Edge case testing:
- Empty database
- Thousands of memories
- Concurrent operations
Test Definition
Complete user journey:
- Start fresh
- Create 5 memories
- Access 2 of them repeatedly
- Run consolidation
- Verify 2 promoted, 3 decayed
- Run again after time
- Verify further decay
Success Criteria
On Completion
- Commit with message: "Memory consolidation system complete"
- Update process document with final summary
- Text TQ: "Memory consolidation implementation complete. All phases tested and working. Check process document for details."
TQ NOTES (about the above):
Process Document Template
Create at: /Users/tqwhite/Documents/webdev/botWorld/system/management/zNotesPlansDocs/memoryConsolidationProcess.md
# Memory Consolidation Implementation Process
Started: [timestamp]
## Phase 0: Backup and Safety Net
[What happened, any issues, results]
## Phase 1: Baseline Verification
[Current state, what works, baseline numbers]
## Phase 2: Test Memory Management
[Test infrastructure results]
## Phase 3: Energy Decay Mathematics
[Mathematical verification, any adjustments]
## Phase 4: Promotion Mechanics
[How promotions work, relationships verified]
## Phase 5: Consolidation Pipeline
[End-to-end test results]
## Phase 6: Logging Infrastructure
[What got logged, progress indicator status]
## Phase 7: /goodbye Integration
[How integration works, async verification]
## Phase 8: Memory Creation Integration
[Parallel systems working together]
## Phase 9: Final Validation
[Complete system test results]
## Summary
[Overall results, any issues encountered, system status]
## Weird Things Noticed
[Anything unexpected or interesting]
Completed: [timestamp]
Emergency Procedures
If something breaks beyond repair:
- Stop immediately
- Run restore script
- Document what broke in process document
- Text TQ with problem description
- Wait for guidance
Begin Execution
Start with Phase 0 and proceed through all phases. Document everything. Make decisions. Fix problems. Complete the implementation.
End of Implementation Plan V4