Mercurial > repos > iuc > data_manager_omamer

annotate data_manager/omamer.py @ 0:d28438704310 draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
author iuc
date Wed, 14 Aug 2024 15:42:38 +0000
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
1 #!/usr/bin/env python
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
2
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
3 import argparse
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
4 import json
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
5 import os
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
6 import sys
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
7
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
8 import requests
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
9
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
10 # URL for downloading OMAmer datasets
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
11 OMAMER_DATASETS_URL = "https://omabrowser.org/All/{dataset}"
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
12
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
13 # List of OMAmer data sets with versions
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
14 OMAMER_DATASETS = {
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
15 "Primates_v2.0.0": "Primates-v2.0.0.h5",
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
16 "Viridiplantae_v2.0.0": "Viridiplantae-v2.0.0.h5",
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
17 "Metazoa_v2.0.0": "Metazoa-v2.0.0.h5",
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
18 "LUCA_v0.2.5": "LUCA-v0.2.5.h5",
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
19 "LUCA_v2.0.0": "LUCA-v2.0.0.h5",
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
20 "Saccharomyceta": "Saccharomyceta.h5",
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
21 "Homininae": "Homininae.h5",
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
22 }
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
23
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
24 DEFAULT_OUTPUT_DIR = "database_omamer"
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
25
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
26
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
27 def download_file(url, dest):
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
28 try:
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
29 with requests.get(url, stream=True) as r:
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
30 r.raise_for_status()
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
31 with open(dest, 'wb') as f:
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
32 for chunk in r.iter_content(chunk_size=8192):
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
33 f.write(chunk)
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
34 print(f"Downloaded: {url} to {dest}")
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
35 except requests.exceptions.RequestException as e:
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
36 print(f"Error downloading {url}: {e}")
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
37 sys.exit(1)
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
38
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
39
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
40 def main(args):
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
41
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
42 with open(args.json) as fh:
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
43 params = json.load(fh)
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
44 target_directory = params["output_data"][0]["extra_files_path"]
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
45
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
46 # Create output directory if none exists
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
47 if not os.path.exists(target_directory):
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
48 os.makedirs(target_directory)
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
49
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
50 # Check if the selected dataset exists
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
51 if args.name not in OMAMER_DATASETS:
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
52 print(f"Error: Selected dataset '{args.name}' not found.")
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
53 sys.exit(1)
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
54
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
55 # Download the selected OMAmer dataset
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
56 dataset = OMAMER_DATASETS[args.name]
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
57 url = OMAMER_DATASETS_URL.format(dataset=dataset)
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
58 base_name = os.path.splitext(dataset)[0]
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
59 destination_path = os.path.join(target_directory, dataset)
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
60 download_file(url, destination_path)
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
61
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
62 data_manager_entry = {
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
63 "value": dataset,
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
64 "name": base_name,
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
65 "version": args.version,
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
66 "path": dataset,
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
67 }
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
68
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
69 # Creates a JSON dictionary representing the Data Manager configuration
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
70 data_manager_json = {"data_tables": {"omamer": [data_manager_entry]}}
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
71
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
72 # Writes this JSON dictionary to the specified output file
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
73 with open(args.json, "w") as fh:
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
74 json.dump(data_manager_json, fh, indent=2, sort_keys=True)
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
75
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
76
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
77 if __name__ == "__main__":
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
78 # Set up argparse to specify expected command line arguments
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
79 parser = argparse.ArgumentParser(description='Download data for OMAmer')
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
80 parser.add_argument('--name', default='Primates', choices=OMAMER_DATASETS.keys(), help='Select dataset to download')
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
81 parser.add_argument('--json', help='Path to JSON file')
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
82 parser.add_argument("--version", help="Omamer version")
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
83
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
84 args = parser.parse_args()
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
85
d28438704310 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_omamer commit b890cd87ef5cfaa81eda29cc935de224ecb05bb6
iuc
parents:
diff changeset
86 main(args)