Downloading Detected part of recordings of pattern matching results from a Project Backup

This article goes through the steps to download detected part of recordings of pattern matching results from Arbimon using the project backup. You can download your detected part recordings using the recordings.csv , pattern_matching_rois.csv and species.csv files from your project backup.

The process will require you to install python. If you haven't installed it, you can follow the instructions from Downloading Recordings from a Project Backup.


The script requires ffmpeg and pydub for trimming recordings which also allow you to trim any audio formats using ffmpeg

Install 'pydub'

For Mac:

    pip3 install pydub

For Windows:

    pip install pydub

Install 'ffmpeg'

For Mac:

This required homebrew installed. You can follow the instruction from its site

    brew install ffmpeg

For Windows:

You can download the binary from this link and follow the instruction from Step 3: Add FFmpeg to PATH

Download Script

You can download the script from here

Put the script file into the folder containing the extracted files from your project backup (e.g. pattern_matchings_rois.csv, recordings.csv, etc.)

Note: If you want to download your recordings to an external hard drive, make sure the export folder (with script inside it) is moved to that hard drive before running the script.


Run Script

Open a terminal window/command prompt and run this command to navigate to the project backup files.

    cd <path-to-project-backup-folder>

Example: If the files from your backup are in folder named bci_export  which is in your Downloads folder, run cd Downloads/bci_export  .

Start the script by running the following command

For Mac:

    python3 download_only_detected_part.py

For Windows:

    python download_only_detected_part.py

The script will start downloading the files and output its progress in the terminal:

(Not recommended) If needed, you can stop the script by pressing ctrl + c. The script keeps track of which urls were processed, so if you run the script again, the download process will continue with the files that were still not downloaded.


The script reads the recording and pattern matching rois csv files in the current directory and downloads all detected parts from rois. The downloaded files are grouped by site (if site_id  is present in the csv file). The script should create a directory similar to this in your backup folder:

    ├── detected_recordings     # files from pattern_matching_rois.csv and recordings.csv
    │   ├── species_name_1              
    │   ├── species_name_2
    │   │   ├── present
    │   │   ├── absent
    │   │   │   ├── site_id
    │   │   │   ├── site_id     # recordings are grouped by species name   
        ...                       then present/absent then site id

You can look up which site name correspond to the site ID (in the above screenshot, site ID=3096) in the sites.csv folder in your project backup.

The recording name is in this format{YYYYMMDD_HHMMSS}-{recording_id}_{start_time}_{end_time}


Notes

  • start_time and end_time are the timestamp of original recording file with second format
  • If your project contains a large amount of files, the script might take several days to finish. Please keep your computer on while the script is running.
  • Do not manually edit the backup files until the download is complete, as the script relies on the original file structure. Feel free to create copies of the files in a separate folder for editing.
  • Do not delete the autogenerated *.downloaded.txt files (they keep track of the download process, so if the files are deleted, the download process will start from the beginning if the script is re-run).

If you run into difficulty with using the script, you can contact us (contact@arbimon.org) for further assistance.