Evaluate multiple runs

warning

The current cengine CLI version 0.0.16 is running into problems for this functionality. We are working hard to fix it and will be fixed the next release.

Comparison

Evaluation however should go beyond individual pipeline executions. As your pipelines are already grouped together in workspaces, why not compare pipelines between each other? Direct comparison will let you judge performance and results of configuraiton against each other.

Pipelines can be compared within a workspace. To compare multiple pipelines, set the relevant workspace, and run:

cengine pipeline compare

This will open up your browser to a local web app, that will help to compare the results of different pipeline runs.

Web App Tool

Summary tab

The summary tab simply shows the names of the pipelines in the workspace.

ce_compare_1

Analysis tab

The analysis tab is the meat of the compare tool. On the left hand side, you can see the pipeline runs. On the right hand side you can see all the hyper parameters used in these pipelines. You can use the widgets to toggle different configurations to compare the pipelines across metrics and slices of data and across hyper-parameters.

ce_compare_3