Publication

Nowadays, major cloud providers provide machine learning services (a.k.a machine learning clouds) to customers. Microsoft Azure Machine Learning Studio and Amazon Machine Learning are two of the most popular machine learning clouds, which raise the level of abstraction for specifying machine learning models and tasks to ease the deployment of real-world machine learning applications. However, real-world machine learning applications are not simple in general, and raising the level of abstraction for machine learning systems rarely comes for free. Thus, an important question comes out. What is the performance of machine learning clouds on real-world machine learning problems? In order to answer this question, we first construct a benchmark data set MLbench with the top winning solutions of different competitions on Kaggle. And then present the results obtained by running MLbench on top of these two machine learning clouds evaluated by Kaggle. Our studying reveals the strength and limitations of these machine learning clouds, and also point out possible improvement directions.

@mastersthesis{abc,
	abstract = {Nowadays, major cloud providers provide machine learning services (a.k.a machine
learning clouds) to customers. Microsoft Azure Machine Learning Studio
and Amazon Machine Learning are two of the most popular machine learning
clouds, which raise the level of abstraction for specifying machine learning models
and tasks to ease the deployment of real-world machine learning applications.
However, real-world machine learning applications are not simple in general, and
raising the level of abstraction for machine learning systems rarely comes for
free. Thus, an important question comes out. What is the performance of machine
learning clouds on real-world machine learning problems? In order to answer this
question, we first construct a benchmark data set MLbench with the top winning
solutions of different competitions on Kaggle. And then present the results
obtained by running MLbench on top of these two machine learning clouds evaluated
by Kaggle. Our studying reveals the strength and limitations of these machine
learning clouds, and also point out possible improvement directions.},
	author = {Luyuan Zeng},
	school = {162},
	title = {Evaluate Machine Learning Clouds with Kaggle},
	year = {2017}
}