LANCEDB_BUCKETS), ('retrievalblockhaidwus', 'retrievalblockhaisweu', LANCEDB_BUCKETS), ('retrievalblockhaidwus', 'retrievalblockhaisgwc', LANCEDB_BUCKETS), # Copy from the given staging location to the given prod location. ('retrievalblockhaiswus', 'retrievalblockhaipwus', LANCEDB_BUCKETS), ('retrievalblockhaiseus', 'retrievalblockhaipeus', LANCEDB_BUCKETS), ('retrievalblockhaisweu', 'retrievalblockhaipweu', LANCEDB_BUCKETS), ('retrievalblockhaisgwc', 'retrievalblockhaipgwc', LANCEDB_BUCKETS), ] ``` `SOURCE_DESTINATION_STORAGE_ACCOUNTS` defines tuples of: - Source storage account - Destination storage account - Which bucket list to copy (either LANCEDB_BUCKETS or RETRIEVAL_BUCKETS) 4. Main Logic: ```47:69:services/copy_retrieval_buckets/copy_retrieval_buckets.py def main(dry_run: bool): for source_storage_account, dest_storage_account, buckets in SOURCE_DESTINATION_STORAGE_ACCOUNTS: for bucket in buckets: if dry_run: print(f'Dry running copying {bucket} from {source_storage_account} to {dest_storage_account}') else: print(f'Copying {bucket} from {source_storage_account} to {dest_storage_account}') cmd = [ 'azcopy', 'sync', f'https://{source_storage_account}.blob.core.windows.net/{bucket}', f'https://{dest_storage_account}.blob.core.windows.net/{bucket}', '--s2s-preserve-access-tier=false', '--log-level=INFO', ] if dry_run: cmd.append('--dry-run') subprocess.run(cmd, check=True) ``` The script is designed to: - Copy data between Azure storage accounts in an organized way - Support both development->staging->production promotion paths - Allow dry runs to verify what would be copied - Use azcopy's sync functionality for efficient transfers