Mercurial > repos > bgruening > sklearn_stacking_ensemble_models

annotate keras_deep_learning.py @ 2:e18d9b17c322 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
author bgruening
date Fri, 09 Aug 2019 13:52:50 -0400
parents
children 963e449636d3
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
1 import argparse
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
2 import json
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
3 import keras
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
4 import pandas as pd
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
5 import pickle
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
6 import six
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
7 import warnings
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
8
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
9 from ast import literal_eval
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
10 from keras.models import Sequential, Model
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
11 from galaxy_ml.utils import try_get_attr, get_search_params
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
12
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
13
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
14 def _handle_shape(literal):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
15 """Eval integer or list/tuple of integers from string
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
16
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
17 Parameters:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
18 -----------
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
19 literal : str.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
20 """
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
21 literal = literal.strip()
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
22 if not literal:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
23 return None
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
24 try:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
25 return literal_eval(literal)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
26 except NameError as e:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
27 print(e)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
28 return literal
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
29
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
30
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
31 def _handle_regularizer(literal):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
32 """Construct regularizer from string literal
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
33
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
34 Parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
35 ----------
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
36 literal : str. E.g. '(0.1, 0)'
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
37 """
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
38 literal = literal.strip()
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
39 if not literal:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
40 return None
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
41
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
42 l1, l2 = literal_eval(literal)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
43
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
44 if not l1 and not l2:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
45 return None
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
46
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
47 if l1 is None:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
48 l1 = 0.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
49 if l2 is None:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
50 l2 = 0.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
51
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
52 return keras.regularizers.l1_l2(l1=l1, l2=l2)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
53
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
54
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
55 def _handle_constraint(config):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
56 """Construct constraint from galaxy tool parameters.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
57 Suppose correct dictionary format
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
58
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
59 Parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
60 ----------
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
61 config : dict. E.g.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
62 "bias_constraint":
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
63 {"constraint_options":
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
64 {"max_value":1.0,
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
65 "min_value":0.0,
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
66 "axis":"[0, 1, 2]"
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
67 },
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
68 "constraint_type":
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
69 "MinMaxNorm"
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
70 }
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
71 """
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
72 constraint_type = config['constraint_type']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
73 if constraint_type == 'None':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
74 return None
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
75
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
76 klass = getattr(keras.constraints, constraint_type)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
77 options = config.get('constraint_options', {})
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
78 if 'axis' in options:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
79 options['axis'] = literal_eval(options['axis'])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
80
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
81 return klass(**options)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
82
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
83
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
84 def _handle_lambda(literal):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
85 return None
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
86
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
87
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
88 def _handle_layer_parameters(params):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
89 """Access to handle all kinds of parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
90 """
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
91 for key, value in six.iteritems(params):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
92 if value == 'None':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
93 params[key] = None
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
94 continue
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
95
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
96 if type(value) in [int, float, bool]\
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
97 or (type(value) is str and value.isalpha()):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
98 continue
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
99
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
100 if key in ['input_shape', 'noise_shape', 'shape', 'batch_shape',
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
101 'target_shape', 'dims', 'kernel_size', 'strides',
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
102 'dilation_rate', 'output_padding', 'cropping', 'size',
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
103 'padding', 'pool_size', 'axis', 'shared_axes']:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
104 params[key] = _handle_shape(value)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
105
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
106 elif key.endswith('_regularizer'):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
107 params[key] = _handle_regularizer(value)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
108
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
109 elif key.endswith('_constraint'):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
110 params[key] = _handle_constraint(value)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
111
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
112 elif key == 'function': # No support for lambda/function eval
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
113 params.pop(key)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
114
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
115 return params
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
116
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
117
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
118 def get_sequential_model(config):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
119 """Construct keras Sequential model from Galaxy tool parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
120
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
121 Parameters:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
122 -----------
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
123 config : dictionary, galaxy tool parameters loaded by JSON
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
124 """
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
125 model = Sequential()
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
126 input_shape = _handle_shape(config['input_shape'])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
127 layers = config['layers']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
128 for layer in layers:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
129 options = layer['layer_selection']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
130 layer_type = options.pop('layer_type')
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
131 klass = getattr(keras.layers, layer_type)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
132 other_options = options.pop('layer_options', {})
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
133 options.update(other_options)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
134
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
135 # parameters needs special care
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
136 options = _handle_layer_parameters(options)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
137
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
138 # add input_shape to the first layer only
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
139 if not getattr(model, '_layers') and input_shape is not None:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
140 options['input_shape'] = input_shape
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
141
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
142 model.add(klass(**options))
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
143
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
144 return model
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
145
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
146
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
147 def get_functional_model(config):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
148 """Construct keras functional model from Galaxy tool parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
149
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
150 Parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
151 -----------
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
152 config : dictionary, galaxy tool parameters loaded by JSON
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
153 """
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
154 layers = config['layers']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
155 all_layers = []
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
156 for layer in layers:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
157 options = layer['layer_selection']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
158 layer_type = options.pop('layer_type')
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
159 klass = getattr(keras.layers, layer_type)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
160 inbound_nodes = options.pop('inbound_nodes', None)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
161 other_options = options.pop('layer_options', {})
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
162 options.update(other_options)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
163
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
164 # parameters needs special care
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
165 options = _handle_layer_parameters(options)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
166 # merge layers
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
167 if 'merging_layers' in options:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
168 idxs = literal_eval(options.pop('merging_layers'))
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
169 merging_layers = [all_layers[i-1] for i in idxs]
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
170 new_layer = klass(**options)(merging_layers)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
171 # non-input layers
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
172 elif inbound_nodes is not None:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
173 new_layer = klass(**options)(all_layers[inbound_nodes-1])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
174 # input layers
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
175 else:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
176 new_layer = klass(**options)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
177
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
178 all_layers.append(new_layer)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
179
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
180 input_indexes = _handle_shape(config['input_layers'])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
181 input_layers = [all_layers[i-1] for i in input_indexes]
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
182
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
183 output_indexes = _handle_shape(config['output_layers'])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
184 output_layers = [all_layers[i-1] for i in output_indexes]
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
185
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
186 return Model(inputs=input_layers, outputs=output_layers)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
187
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
188
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
189 def get_batch_generator(config):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
190 """Construct keras online data generator from Galaxy tool parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
191
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
192 Parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
193 -----------
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
194 config : dictionary, galaxy tool parameters loaded by JSON
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
195 """
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
196 generator_type = config.pop('generator_type')
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
197 klass = try_get_attr('galaxy_ml.preprocessors', generator_type)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
198
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
199 if generator_type == 'GenomicIntervalBatchGenerator':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
200 config['ref_genome_path'] = 'to_be_determined'
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
201 config['intervals_path'] = 'to_be_determined'
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
202 config['target_path'] = 'to_be_determined'
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
203 config['features'] = 'to_be_determined'
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
204 else:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
205 config['fasta_path'] = 'to_be_determined'
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
206
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
207 return klass(**config)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
208
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
209
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
210 def config_keras_model(inputs, outfile):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
211 """ config keras model layers and output JSON
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
212
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
213 Parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
214 ----------
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
215 inputs : dict
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
216 loaded galaxy tool parameters from `keras_model_config`
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
217 tool.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
218 outfile : str
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
219 Path to galaxy dataset containing keras model JSON.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
220 """
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
221 model_type = inputs['model_selection']['model_type']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
222 layers_config = inputs['model_selection']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
223
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
224 if model_type == 'sequential':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
225 model = get_sequential_model(layers_config)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
226 else:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
227 model = get_functional_model(layers_config)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
228
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
229 json_string = model.to_json()
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
230
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
231 with open(outfile, 'w') as f:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
232 f.write(json_string)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
233
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
234
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
235 def build_keras_model(inputs, outfile, model_json, infile_weights=None,
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
236 batch_mode=False, outfile_params=None):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
237 """ for `keras_model_builder` tool
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
238
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
239 Parameters
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
240 ----------
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
241 inputs : dict
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
242 loaded galaxy tool parameters from `keras_model_builder` tool.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
243 outfile : str
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
244 Path to galaxy dataset containing the keras_galaxy model output.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
245 model_json : str
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
246 Path to dataset containing keras model JSON.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
247 infile_weights : str or None
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
248 If string, path to dataset containing model weights.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
249 batch_mode : bool, default=False
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
250 Whether to build online batch classifier.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
251 outfile_params : str, default=None
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
252 File path to search parameters output.
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
253 """
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
254 with open(model_json, 'r') as f:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
255 json_model = json.load(f)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
256
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
257 config = json_model['config']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
258
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
259 options = {}
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
260
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
261 if json_model['class_name'] == 'Sequential':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
262 options['model_type'] = 'sequential'
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
263 klass = Sequential
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
264 elif json_model['class_name'] == 'Model':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
265 options['model_type'] = 'functional'
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
266 klass = Model
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
267 else:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
268 raise ValueError("Unknow Keras model class: %s"
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
269 % json_model['class_name'])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
270
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
271 # load prefitted model
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
272 if inputs['mode_selection']['mode_type'] == 'prefitted':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
273 estimator = klass.from_config(config)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
274 estimator.load_weights(infile_weights)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
275 # build train model
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
276 else:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
277 cls_name = inputs['mode_selection']['learning_type']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
278 klass = try_get_attr('galaxy_ml.keras_galaxy_models', cls_name)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
279
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
280 options['loss'] = (inputs['mode_selection']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
281 ['compile_params']['loss'])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
282 options['optimizer'] =\
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
283 (inputs['mode_selection']['compile_params']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
284 ['optimizer_selection']['optimizer_type']).lower()
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
285
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
286 options.update((inputs['mode_selection']['compile_params']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
287 ['optimizer_selection']['optimizer_options']))
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
288
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
289 train_metrics = (inputs['mode_selection']['compile_params']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
290 ['metrics']).split(',')
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
291 if train_metrics[-1] == 'none':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
292 train_metrics = train_metrics[:-1]
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
293 options['metrics'] = train_metrics
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
294
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
295 options.update(inputs['mode_selection']['fit_params'])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
296 options['seed'] = inputs['mode_selection']['random_seed']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
297
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
298 if batch_mode:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
299 generator = get_batch_generator(inputs['mode_selection']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
300 ['generator_selection'])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
301 options['data_batch_generator'] = generator
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
302 options['prediction_steps'] = \
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
303 inputs['mode_selection']['prediction_steps']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
304 options['class_positive_factor'] = \
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
305 inputs['mode_selection']['class_positive_factor']
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
306 estimator = klass(config, **options)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
307 if outfile_params:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
308 hyper_params = get_search_params(estimator)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
309 # TODO: remove this after making `verbose` tunable
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
310 for h_param in hyper_params:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
311 if h_param[1].endswith('verbose'):
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
312 h_param[0] = '@'
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
313 df = pd.DataFrame(hyper_params, columns=['', 'Parameter', 'Value'])
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
314 df.to_csv(outfile_params, sep='\t', index=False)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
315
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
316 print(repr(estimator))
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
317 # save model by pickle
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
318 with open(outfile, 'wb') as f:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
319 pickle.dump(estimator, f, pickle.HIGHEST_PROTOCOL)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
320
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
321
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
322 if __name__ == '__main__':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
323 warnings.simplefilter('ignore')
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
324
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
325 aparser = argparse.ArgumentParser()
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
326 aparser.add_argument("-i", "--inputs", dest="inputs", required=True)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
327 aparser.add_argument("-m", "--model_json", dest="model_json")
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
328 aparser.add_argument("-t", "--tool_id", dest="tool_id")
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
329 aparser.add_argument("-w", "--infile_weights", dest="infile_weights")
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
330 aparser.add_argument("-o", "--outfile", dest="outfile")
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
331 aparser.add_argument("-p", "--outfile_params", dest="outfile_params")
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
332 args = aparser.parse_args()
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
333
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
334 input_json_path = args.inputs
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
335 with open(input_json_path, 'r') as param_handler:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
336 inputs = json.load(param_handler)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
337
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
338 tool_id = args.tool_id
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
339 outfile = args.outfile
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
340 outfile_params = args.outfile_params
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
341 model_json = args.model_json
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
342 infile_weights = args.infile_weights
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
343
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
344 # for keras_model_config tool
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
345 if tool_id == 'keras_model_config':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
346 config_keras_model(inputs, outfile)
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
347
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
348 # for keras_model_builder tool
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
349 else:
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
350 batch_mode = False
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
351 if tool_id == 'keras_batch_models':
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
352 batch_mode = True
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
353
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
354 build_keras_model(inputs=inputs,
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
355 model_json=model_json,
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
356 infile_weights=infile_weights,
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
357 batch_mode=batch_mode,
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
358 outfile=outfile,
e18d9b17c322 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit c035d399196b3bef9982db4f8e47331411dbb20e
bgruening
parents:
diff changeset
359 outfile_params=outfile_params)