Advanced RSync

This project consists of creating an advanced continuous synchronization script that keeps two locations (FTP, ZIP, or local folders) fully synchronized in real-time, handling file creation, modification, and deletion while maintaining logs and resolving conflicts based on file timestamps.

Phase 1: Location Parsing and Connection Setup

In this phase, the script needs to understand the format of the two locations provided as input and establish access to them. The locations can be of three types:

  1. FTP – A remote folder accessed via FTP protocol. The format is: ftp:<username>:<password>@<host>/<path> Example: ftp:my_user:1234@127.0.0.1/folder

    • The script should parse the username, password, host, and path.
    • Establish an FTP connection to read/write files.

  2. ZIP – A local ZIP archive. The format is: zip:<full_path_to_zip> Example: zip:C:\abc\d.zip

    • The script should be able to read and write files inside the ZIP archive.
    • Any changes (creation, modification, deletion) should be mirrored inside the ZIP.

  3. Folder – A local folder in the file system. The format is: folder:<full_path_to_folder> Example: folder:C:\aaa

    • The script should monitor the folder for changes and copy files as needed.

During this phase, the script will:

  • Parse the input strings to determine the type of each location.
  • Extract relevant parameters (credentials for FTP, paths for ZIP or folder).
  • Test the connection or access to ensure the location is available.

Functional Output: Script can correctly interpret input locations and establish connections/access to both locations, ready for synchronization.

Phase 2: Initial Synchronization

Perform initial synchronization based on existing files in both locations.

  • If a file exists only in one location, copy it to the other location
  • If a file exists in both locations but differs, copy the newest version based on modification timestamp
  • Ensure conflicts are resolved deterministically

Functional Output: Both locations contain the same set of files with the newest versions present in both

Phase 3: Continuous Monitoring and File Creation Handling

Monitor both locations in real-time to detect newly created files.

  • Detect file creation in either location
  • Copy the new file to the other location
  • Support recursive directories if applicable

Functional Output: New files in one location appear automatically in the other location

Phase 4: File Deletion Handling

Detect file deletions in either location and propagate deletions to maintain sync.

  • Monitor for deleted files in each location
  • Delete corresponding files in the other location
  • Ensure that no extra files remain in either location

Functional Output: Files deleted in one location are removed from the other location in near real-time

Phase 5: File Modification Handling

Detect file modifications and ensure updated versions are propagated.

  • Monitor timestamps or checksums to detect changes
  • Copy the updated file to the other location, overwriting the old version
  • Handle concurrent modifications safely

Functional Output: Modified files in one location are updated in the other location automatically

Phase 6: Logging and Error Handling

Implement logging for all synchronization events and handle errors robustly.

  • Log file creations, deletions, and modifications with timestamps
  • Log connection errors, file access errors, and retry attempts
  • Provide a summary report of synchronization status

Functional Output: Clear logs showing all synchronization operations and errors for validation purposes