For now only strings are accepted as the measures parameter in GridSearchCV, RandomizedSearchCV, and cross_validate. It's thus impossible to use those with measures that take specific parameters as input (e.g. #156 ), or to use custom measures.
We should then accept callables in addition to strings.
Each callable should only take the predictions parameter. In order to handle measur
Currently, ILLIXR timesout with 60 seconds. We need a way to configure this time or turn the timeout off. This should probably be set in the config yaml file for runner.py.
cBench provides a unified CLI and API to reproduce results from ML&systems research papers on bare-metal platforms and participate in collaborative benchmarking and optimization using live scoreboards. See the real-world example for the MLPerf benchmark:
Register, vote, and volunteer to make an impact during U.S. 2020 elections.
For now only strings are accepted as the
measuresparameter inGridSearchCV,RandomizedSearchCV, andcross_validate. It's thus impossible to use those with measures that take specific parameters as input (e.g. #156 ), or to use custom measures.We should then accept callables in addition to strings.
Each callable should only take the
predictionsparameter. In order to handle measur