CV/my-app/docs/file_manager_enhancement_plan.md

File Manager Enhancement Plan

This document outlines the plan to enhance the my-app/utils/file_manager.py script based on user feedback.

Goals:

1. Add support for loading configuration from a config.yaml file.
2. Implement a new action (--move-cold) to move inactive ("cold") files from fast storage back to slow storage based on modification time.
3. Add an --interactive flag to prompt for confirmation before moving files.
4. Implement a new action (--generate-stats) to create a JSON file containing storage statistics (file counts, sizes by age) for both source and target directories.
5. Calculate and log the total size of files being moved by the --move-cold action.

Detailed Plan:

1. Configuration File (config.yaml):

   • Goal: Allow users to define common settings in a YAML file.
   • Implementation:
     • Define structure for config.yaml (e.g., ~/.config/file_manager/config.yaml or specified via --config).
     • Use PyYAML library (requires pip install PyYAML).
     • Modify parse_arguments to load settings, allowing command-line overrides.
     • Add --config argument.

2. Move Cold Files Back (--move-cold action):

   • Goal: Move files from fast (target) to slow (source) storage if inactive.
   • Implementation:
     • Add action: --move-cold.
     • Add argument: --stale-days (default 30, uses modification time st_mtime).
     • New function find_stale_files(directory, days): Scans target_dir based on st_mtime.
     • New function move_files_cold(relative_file_list, source_dir, target_dir, dry_run, interactive):
       • Similar to move_files.
       • Moves files from target_dir to source_dir using rsync.
       • Handles paths relative to target_dir.
       • Calculates and logs total size of files to be moved before rsync.
       • Incorporates interactive confirmation.

3. Interactive Confirmation (--interactive flag):

   • Goal: Add a safety check before moving files.
   • Implementation:
     • Add global flag: --interactive.
     • Modify move_files and move_files_cold:
       • If --interactive and not --dry-run:
         • Log files/count.
         • Use input() for user confirmation (yes/no).
         • Proceed only on "yes".

4. Enhanced Reporting/Stats File (--generate-stats action):

   • Goal: Create a persistent JSON file with storage statistics.
   • Implementation:
     • Add action: --generate-stats.
     • Add argument: --stats-file (overrides config).
     • New function analyze_directory(directory):
       • Walks directory, calculates total count/size, count/size by modification time brackets.
       • Returns data as a dictionary.
     • Modify main or create orchestrator for --generate-stats:
       • Call analyze_directory for source and target.
       • Combine results with a timestamp.
       • Write dictionary to stats_file using json.
     • (Optional): Modify --summarize-unused to potentially use the stats file.

Workflow Visualization (Mermaid):

graph TD
    Start --> ReadConfig{Read config.yaml (Optional)}
    ReadConfig --> ParseArgs[Parse Command Line Args]
    ParseArgs --> ValidateArgs{Validate Args & Config}
    ValidateArgs --> ActionRouter{Route based on Action}

    ActionRouter -- --generate-stats --> AnalyzeSrc[Analyze Source Dir]
    AnalyzeSrc --> AnalyzeTgt[Analyze Target Dir]
    AnalyzeTgt --> WriteStatsFile[Write stats.json]
    WriteStatsFile --> End

    ActionRouter -- --move --> FindRecent[Find Recent Files (Source)]
    FindRecent --> CheckInteractiveHot{Interactive?}
    CheckInteractiveHot -- Yes --> ConfirmHot(Confirm Move Hot?)
    CheckInteractiveHot -- No --> ExecuteMoveHot[Execute rsync Hot (Source->Target)]
    ConfirmHot -- Yes --> ExecuteMoveHot
    ConfirmHot -- No --> AbortHot(Abort Hot Move)
    AbortHot --> End
    ExecuteMoveHot --> End

    ActionRouter -- --move-cold --> FindStale[Find Stale Files (Target)]
    FindStale --> CalculateColdSize[Calculate Total Size of Cold Files]
    CalculateColdSize --> CheckInteractiveCold{Interactive?}
    CheckInteractiveCold -- Yes --> ConfirmCold(Confirm Move Cold?)
    CheckInteractiveCold -- No --> ExecuteMoveCold[Execute rsync Cold (Target->Source)]
    ConfirmCold -- Yes --> ExecuteMoveCold
    ConfirmCold -- No --> AbortCold(Abort Cold Move)
    AbortCold --> End
    ExecuteMoveCold --> End

    ActionRouter -- --count --> FindRecentForCount[Find Recent Files (Source)]
    FindRecentForCount --> CountFiles[Log Count]
    CountFiles --> End

    ActionRouter -- --summarize-unused --> SummarizeUnused[Summarize Unused (Target)]
    SummarizeUnused --> LogSummary[Log Summary]
    LogSummary --> End

    ActionRouter -- No Action/Error --> ShowHelp[Show Help / Error]
    ShowHelp --> End

Summary of Changes:

• New dependencies: PyYAML.
• New command-line arguments: --move-cold, --stale-days, --interactive, --generate-stats, --stats-file, --config.
• New functions: find_stale_files, move_files_cold, analyze_directory.
• Modifications to existing functions: parse_arguments, move_files, main.
• Introduction of config.yaml for settings.
• Introduction of a JSON stats file for persistent reporting.