Skip to content

/ MLOpsPython Template

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Using AutoMLStep requires azureml.train.automl.runtime #196

Open
ross-p-smith opened this issue Feb 17, 2020 · 2 comments
Open

Using AutoMLStep requires azureml.train.automl.runtime #196

ross-p-smith opened this issue Feb 17, 2020 · 2 comments

Comments

@ross-p-smith
Copy link

@ross-p-smith ross-p-smith commented Feb 17, 2020

I am attempting to have an AutomMLStep in a new pipeline based off this sample. In order to add this step into my prepare_train.py script I need to use the following

from azureml.train.automl.runtime import AutoMLStep

This is not included in azureml.train.automl.client which is already installed in the mlops container.
This 'runtime' library cannot be pip installed using the mcr.microsoft.com/mlops/python:latest container as it has conflicts with numpy and pandas. So I have attempted to build the container myself following the docker-image-pipeline.yml example. However, when I build my own image I am having problems with psutil

creating build/temp.linux-x86_64-3.7/psutil
gcc -pthread -B /usr/local/envs/mlopspython_ci/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/usr/local/envs/mlopspython_ci/include/python3.7m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.7/psutil/_psutil_common.o
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1

My issue is how to AutomMLStep in the mlops pipeline?

Everything works fine in a Notebook!
@algattik
Copy link
Contributor

@algattik algattik commented Feb 18, 2020

Perhaps adding psutil and/or gcc conda package to your env should work.

name: mlopspython_ci
dependencies:
  - gcc
@ross-p-smith
Copy link
Author

@ross-p-smith ross-p-smith commented Feb 18, 2020

Thanks Alexandre, I did try that, but you cant install gcc in a container (apparently!). We have reverted back to our own ci container but instead of 1.0.85 we are 1.0.83 and we can now use AutoMLStep

  • azureml-sdk==1.0.83
  • azureml-train-automl==1.0.83
  • azureml-train-automl-runtime==1.0.83.1

Unsure whether this issue should be left open for 1.0.85 of azureml-train-automl
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
@ross-p-smith @algattik
You can’t perform that action at this time.