Mercurial > repos > pimarin > data_manager_bakta

annotate data_manager/bakta_build_database.py @ 27:2879a0e702d5 draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
author pimarin
date Wed, 23 Aug 2023 14:38:16 +0000
parents 0408796bce2a
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
1 import argparse
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
2 import hashlib
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
3 import json
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
4 import os
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
5 import re
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
6 import sys
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
7 import tarfile
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
8 from datetime import datetime
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
9 from pathlib import Path
19
c90380f8bbbc planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 8eb28a93e6f2688bb2f3f85aea0389e1b1148816-dirty
pimarin
parents: 18
diff changeset
10
17
ecb7d3d41366 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit e7c35e529ae95a2c9f2ecd87a3716e6866feabfa-dirty
pimarin
parents: 16
diff changeset
11
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
12 import requests
16
718080c30a76 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit e7c35e529ae95a2c9f2ecd87a3716e6866feabfa
pimarin
parents: 15
diff changeset
13
718080c30a76 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit e7c35e529ae95a2c9f2ecd87a3716e6866feabfa
pimarin
parents: 15
diff changeset
14
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
15 class GetBaktaDatabaseInfo:
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
16 """
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
17 Extract bakta database information to make a json file for data_manager
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
18 """
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
19
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
20 def __init__(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
21 self,
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
22 data_table_name="bakta_database",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
23 db_version="latest",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
24 tarball_name="db.tar.gz",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
25 test_mode=False,
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
26 ):
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
27 self.bakta_table_list = None
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
28 self.db_url = None
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
29 self.db_name = "bakta-db"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
30 self.db_type = ""
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
31 self.data_table_entry = None
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
32 self.data_table_name = data_table_name
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
33 self.tar_name = tarball_name
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
34 self.db_version = db_version
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
35 self.DB_VERSIONS_URL = "https://raw.githubusercontent.com/oschwengers/bakta/master/db-versions.json"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
36 self.DB_TEST_URL = "https://zenodo.org/record/8021032/files/db-versions.json"
19
c90380f8bbbc planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 8eb28a93e6f2688bb2f3f85aea0389e1b1148816-dirty
pimarin
parents: 18
diff changeset
37 self.test_mode = test_mode
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
38
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
39 def get_database_type(self):
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
40 self.light_db = bool(re.search(pattern="light", string=self.db_version))
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
41 self.db_version = self.db_version.split(sep="_")[0]
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
42 if self.light_db:
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
43 self.db_type = "light"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
44 self.tar_name = "db-light.tar.gz"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
45 self.md5 = self.fetch_db_versions()["md5-light"]
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
46 else:
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
47 self.md5 = self.fetch_db_versions()["md5"]
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
48
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
49 def get_data_table_format(self):
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
50 """
18
04bee0f935a2 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 476cb50265f7dcb492fa2d0f03f595b1363faf7a
pimarin
parents: 17
diff changeset
51 Skeleton of a data_table format
04bee0f935a2 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 476cb50265f7dcb492fa2d0f03f595b1363faf7a
pimarin
parents: 17
diff changeset
52 return: a data table formated for json output
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
53 """
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
54 self.data_table_entry = {"data_tables": {self.data_table_name: {}}}
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
55 return self.data_table_entry
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
56
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
57 def fetch_db_versions(self):
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
58 """
19
c90380f8bbbc planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 8eb28a93e6f2688bb2f3f85aea0389e1b1148816-dirty
pimarin
parents: 18
diff changeset
59 List bakta database info related to the db_version selected
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
60 """
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
61
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
62 if self.test_mode:
19
c90380f8bbbc planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 8eb28a93e6f2688bb2f3f85aea0389e1b1148816-dirty
pimarin
parents: 18
diff changeset
63 self.DB_VERSIONS_URL = self.DB_TEST_URL
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
64 try:
11
c50e8b4f74d7 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 572bbceeba9a22cdb591d526abda362595f8f0c4
pimarin
parents: 10
diff changeset
65 with requests.get(self.DB_VERSIONS_URL) as resp:
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
66 versions = json.loads(resp.content)
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
67 except IOError as e:
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
68 print(e, file=sys.stderr)
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
69 raise e
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
70
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
71 if self.db_version == "latest":
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
72 db_date_list = []
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
73 for db_dic in versions:
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
74 db_date_list.append(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
75 datetime.strptime(db_dic["date"], "%Y-%m-%d").date()
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
76 )
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
77 filtered_version = max(versions, key=lambda x: x["date"])
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
78 else:
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
79 filtered_version = None
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
80 for item in versions:
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
81 if "{0}.{1}".format(item["major"], item["minor"]) == self.db_version:
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
82 filtered_version = item
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
83 break
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
84 if filtered_version is None:
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
85 print("No matching version detected in the list")
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
86 else:
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
87 self.db_url = f"https://zenodo.org/record/{filtered_version['record']}/files/{self.tar_name}"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
88 return filtered_version
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
89
19
c90380f8bbbc planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 8eb28a93e6f2688bb2f3f85aea0389e1b1148816-dirty
pimarin
parents: 18
diff changeset
90 def get_data_manager(self, bakta_database_info):
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
91 self.bakta_table_list = self.get_data_table_format()
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
92 bakta_name = (
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
93 f"V{bakta_database_info['major']}."
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
94 f"{bakta_database_info['minor']}{self.db_type}_"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
95 f"{bakta_database_info['date']}"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
96 )
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
97 tool_version = str(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
98 f"{bakta_database_info['software-min']['major']}."
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
99 f"{bakta_database_info['software-min']['minor']}"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
100 )
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
101 data_info = dict(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
102 value=bakta_name,
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
103 dbkey=bakta_database_info["record"],
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
104 bakta_version=tool_version,
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
105 path=self.db_name,
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
106 )
16
718080c30a76 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit e7c35e529ae95a2c9f2ecd87a3716e6866feabfa
pimarin
parents: 15
diff changeset
107 self.bakta_table_list["data_tables"][self.data_table_name] = [data_info]
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
108 return self.bakta_table_list
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
109
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
110
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
111 class InstallBaktaDatabase(GetBaktaDatabaseInfo):
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
112 """
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
113 Download the bakta database,
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
114 check md5 sum,
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
115 untar the download db and update for the amrfinderplus database
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
116 """
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
117
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
118 def __init__(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
119 self,
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
120 db_dir=Path.cwd(),
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
121 db_name="bakta-db",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
122 db_version="latest",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
123 test_mode=False
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
124 ):
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
125 super().__init__()
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
126 self.md5 = None
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
127 self.db_version = db_version
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
128 self.db_dir = db_dir
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
129 self.db_name = db_name
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
130 self.tarball_path = ""
19
c90380f8bbbc planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 8eb28a93e6f2688bb2f3f85aea0389e1b1148816-dirty
pimarin
parents: 18
diff changeset
131 self.test_mode = test_mode
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
132 self.get_database_type()
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
133
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
134 def download(self):
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
135 #self.db_name = f"{self.db_name}_{self.db_version}{self.db_type}"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
136 bakta_path = Path(self.db_dir).joinpath(self.tar_name)
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
137 try:
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
138 with bakta_path.open("wb") as fh_out, requests.get(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
139 self.db_url, stream=True) as resp:
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
140 total_length = resp.headers.get("content-length")
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
141 if total_length is None: # no content length header
15
ac4d5e1d1421 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit b082e89a0090f9c4b2a257bb06b7c6f141b42650-dirty
pimarin
parents: 14
diff changeset
142 for data in resp.iter_content(chunk_size=1024 * 1024):
ac4d5e1d1421 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit b082e89a0090f9c4b2a257bb06b7c6f141b42650-dirty
pimarin
parents: 14
diff changeset
143 fh_out.write(data)
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
144 else:
15
ac4d5e1d1421 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit b082e89a0090f9c4b2a257bb06b7c6f141b42650-dirty
pimarin
parents: 14
diff changeset
145 for data in resp.iter_content(chunk_size=1024 * 1024):
ac4d5e1d1421 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit b082e89a0090f9c4b2a257bb06b7c6f141b42650-dirty
pimarin
parents: 14
diff changeset
146 fh_out.write(data)
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
147 print(f"Download bakta database {self.db_version}")
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
148 self.tarball_path = bakta_path
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
149 except IOError:
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
150 print(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
151 f"ERROR: Could not download file from Zenodo!"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
152 f" url={self.db_url}, to={self.tarball_path}"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
153 )
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
154
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
155 def untar(self):
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
156 db_path = Path(self.db_dir).joinpath(self.db_name)
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
157 try:
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
158 with self.tarball_path.open("rb") as fh_in, tarfile.open(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
159 fileobj=fh_in, mode="r:gz"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
160 ) as tar_file:
16
718080c30a76 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit e7c35e529ae95a2c9f2ecd87a3716e6866feabfa
pimarin
parents: 15
diff changeset
161 tar_file.extractall(path=db_path)
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
162 print(f"Untar the database in {db_path}")
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
163 # return db_path
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
164 except OSError:
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
165 sys.exit(f"ERROR: Could not extract {self.tar_name} " f"to {db_path}")
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
166
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
167 def calc_md5_sum(self, buffer_size=1048576):
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
168 tarball_path = Path(self.db_dir).joinpath(self.tar_name)
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
169 md5 = hashlib.md5()
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
170 with tarball_path.open("rb") as fh:
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
171 data = fh.read(buffer_size)
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
172 while data:
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
173 md5.update(data)
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
174 data = fh.read(buffer_size)
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
175 if md5.hexdigest() == self.md5:
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
176 print("\t...md5 control database OK")
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
177 else:
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
178 print(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
179 f"Error: corrupt database file! "
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
180 f"calculated md5 = {md5.hexdigest()}"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
181 f" different from {self.md5} "
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
182 )
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
183
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
184
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
185 def parse_arguments():
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
186 # parse options and arguments
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
187 arg_parser = argparse.ArgumentParser()
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
188 arg_parser.add_argument("data_manager_json")
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
189 arg_parser.add_argument(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
190 "-d",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
191 "--database_version",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
192 help="Select the database version "
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
193 "(major and minor eg. 4.0),"
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
194 "default is the latest version",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
195 default="latest",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
196 required=True,
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
197 )
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
198 arg_parser.add_argument(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
199 "-t",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
200 "--test",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
201 action="store_true",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
202 help="option to test the script with an empty database",
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
203 )
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
204 return arg_parser.parse_args()
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
205
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
206
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
207 def main():
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
208 all_args = parse_arguments()
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
209 with open(all_args.data_manager_json) as fh:
16
718080c30a76 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit e7c35e529ae95a2c9f2ecd87a3716e6866feabfa
pimarin
parents: 15
diff changeset
210 params = json.load(fh)
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
211 target_dir = params["output_data"][0]["extra_files_path"]
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
212 os.makedirs(target_dir)
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
213 # init the class to download bakta db
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
214 bakta_upload = InstallBaktaDatabase(
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
215 test_mode=all_args.test, db_version=all_args.database_version
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
216 )
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
217 bakta_db = bakta_upload.fetch_db_versions()
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
218 # update the path for galaxy
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
219 bakta_upload.db_dir = target_dir
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
220 # download the database
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
221 bakta_upload.download()
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
222 # check md5 sum
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
223 bakta_upload.calc_md5_sum()
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
224 # untar db
16
718080c30a76 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit e7c35e529ae95a2c9f2ecd87a3716e6866feabfa
pimarin
parents: 15
diff changeset
225 bakta_upload.untar()
1
faae5d8ce0cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 4c11a77fb4c0bb74c0c4ae381c85971e01ed973e-dirty
pimarin
parents: 0
diff changeset
226 # make the data_manager metadata
19
c90380f8bbbc planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 8eb28a93e6f2688bb2f3f85aea0389e1b1148816-dirty
pimarin
parents: 18
diff changeset
227 bakta_data_manager = bakta_upload.get_data_manager(bakta_database_info=bakta_db)
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
228 with open(all_args.data_manager_json, "w") as fh:
11
c50e8b4f74d7 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 572bbceeba9a22cdb591d526abda362595f8f0c4
pimarin
parents: 10
diff changeset
229 json.dump(bakta_data_manager, fh, sort_keys=True)
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
230
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
231
27
2879a0e702d5 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 3bfd33ae9741216e50373ddd04914d82f9731883
pimarin
parents: 26
diff changeset
232 if __name__ == "__main__":
0
9d08486abf8e "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit 5f320f165a7f454193df36cc2ec77a87d50ec80f-dirty"
pimarin
parents:
diff changeset
233 main()