Workspaces

Making logical groupings of your pipelines

In the Core Engine, you are a part of an organization and it is possible for your organization to be working on numerous tasks or projects at the same time. Moreover, as a user, you might be participating in multiple projects simultaneously. The concept of workspaces is created to maintain an organized and efficient structure within the ML workflow of your organization.

What is it for?

Through a workspace, developers can:

  • Gather all their solutions about a problem under one roof

  • Collaborate on the same problem by sharing their experiments and solutions

Relation to pipelines

One workspace can have many pipelines. In addition, all pipelines within the same workspace:

  • Utilize a shared metadata store, which tracks all the executed steps within the workspace

  • Make use of the shared metadata store to skip previously executed processes for efficiency

    and to trace the results

  • Can be compared against each other with the pipeline compare tool

For most intents and purposes, pipelines within the same workspace should probably share the same datasource to be comparable across runs. However, you are free to use as many datasources as you want

Basics

In order to see the available workspaces within your organization:

Python SDK
CLI
Python SDK
workspaces = client.get_workspaces()
print(workspaces)
CLI
cengine workspace list

In order to create a new workspace:

Python SDK
CLI
Python SDK
workspace = client.create_workspace(name='WORKSPACE_NAME')
print(workspace)
CLI
cengine workspace create NAME

In order to make a workspace active (CLI only):

CLI
CLI
cengine workspace set WORKSPACE_ID

arguments

type

description

WORKSPACE_ID

int

the id of the selected workspace