Introduction To Pipeline Elements (Know-How)
- This guide explains the key interface components of the Rubiscape Pipeline and Workbook environment—Main Menu, Nodes, Properties Pane, Task Pane, Edit Mode, and View Mode.
- Understanding these elements helps users design, configure, and manage workflows efficiently.
1. Main Menu Bar
The Main Menu provides quick access to core actions required to build, validate, monitor, and manage pipelines/workbooks. Each option allows users to efficiently control workflow execution and configuration.
1. Edit
The Edit option allows you to modify the pipeline structure during design.
• Enable edit mode to add, delete, or rearrange nodes
• Modify connections between components
• Change configuration of existing nodes
• Add annotations and comments
2. Validate
The Validate option checks whether the pipeline is correctly configured.
• Identifies missing configurations
• Validates input/output mappings
• Checks for disconnected nodes
• Ensures pipeline is ready to run
3. Zoom Controls
Zoom options help you adjust the visual canvas for pipeline design.
• Zoom In (+)
• Zoom Out (–)
4. Minimap
The Minimap provides a miniature view of the entire pipeline layout.
• Helps navigate large pipelines
• Shows node positions visually
• Allows quick movement across sections
5. Save
The Save option stores the latest changes made to the pipeline.
• Saves node configuration, layout, and structure
• Ensures workflow is updated in workspace
• Disabled when no changes are pending
6. Run
The Run option executes the pipeline after validation.
• Starts workflow execution
• Moves data through nodes sequentially
• Displays execution logs and results
• Fails if pipeline is invalid
7. Variable
This option manages dynamic variables used across the pipeline.
• Add or edit global variables
• Pass dynamic values during run
• Useful for parameterized pipelines
• Enables environment-specific execution
8. ⋯ More Options (Three-Dot Menu)
The More menu includes additional management and operational tools.
8.1 Save As
• Create a copy of the current pipeline
• Useful for versioning or branching
• Save under a new name or category
8.2 Snapshot
• Captures the current visual state of the pipeline
• Used for documentation, reviews, and audit
• Saves an image-like representation of the workflow
8.3 Schedule
• Allows automated execution of pipelines at defined intervals
• Supports hourly, daily, weekly, etc. schedules
• Ideal for recurring ETL jobs or periodic data refresh
• Requires pipeline to be valid
8.4 Batch Processing
• Configure and run batch jobs
• Process large datasets in bulk
• Supports file-based batch, dataset batch, and scheduled batch runs
• Provides logs and output for each batch
Summary
• Edit → Modify pipeline
• Validate → Check correctness
• Zoom & Minimap → Navigation tools
• Save → Store changes
• Run → Execute pipeline
• Variable → Manage global parameters
• More Menu → Save As, Snapshot, Schedule, Batch Processing
2. Node
A Node represents an activity, transformation, reader, writer, or logic block in a Pipeline or Workbook. Nodes act as functional building blocks.
Node Characteristics
• Each node performs one specific task (Read, Transform, Join, Write, Predict, etc.)
• Nodes are connected to define data flow
• Node icons visually represent the operation (Reader, Filter, Join, ML Model, etc.)
• Clicking a node opens its details in the Properties Pane
Node Actions
• Add input/output connections
• Configure properties
• View logs or preview data
• Clone / Delete node
• Enable/Disable node in workflow
3. Properties Pane
The Properties Pane displays configuration details for the selected node or pipeline component.
What Appears in the Properties Pane?
• Node Name & Description
• Configuration form (varies by node type)
• Input / Output schema
• Advanced Settings
• Execution settings
• Run actions
4. Task Pane
5. Edit Mode
Edit Mode is used for designing, modifying, configuring, and building pipelines or workbooks.
6. View Mode
View Mode allows users to inspect the pipeline without making changes.
7. Logs (Trace Log & Execution Log)
Rubiscape provides two detailed logs to analyze performance, failures, and execution behavior.
8.1 Trace Log
The Trace Log provides a step-by-step trace of operations inside the pipeline.
• Displays node-by-node processing flow
• Shows execution order of components
• Details internal transitions and flow paths
• Helps diagnose issues like skipped nodes or incorrect branching
• Useful for debugging complex pipelines with multiple splits/joins
8.2 Execution Log
The Execution Log records execution-level details generated during pipeline run.
• Timestamp logs of pipeline start, end, and duration
• Success/failure indicators for each node
• Detailed error messages and stack traces
• Record counts processed in each node
• Download option for audit or offline analysis
Related Articles
External Pipeline Execution
1.1 Overview This document outlines the steps required to execute the Pipeline using the Rubiscape API. The process involves obtaining a login token, preparing the necessary parameters, defining the API endpoint and request body, making the API call ...Setting Variable Values from Scheduled Pipeline Execution
This functionality allows users to assign specific values to pipeline variables at the schedule level. When a pipeline runs through a schedule, the variable values defined in that schedule override the global variable values. This enables dynamic, ...Introduction to Mobile View
Refer Navigate Mobile View to access the Mobile View. To ensure your dashboard is displayed correctly in the RubiSight Mobile Application, you need to configure the Mobile View. Charts can be organized in any desired layout, independent of the layout ...MLOPS - Machine Learning Operations
Introduction: Why Rubiscape MLOps? Rubiscape MLOps provides an end-to-end environment for building, tracking, publishing, and serving machine learning models. It ensures experiment reproducibility, streamlined deployment, and centralized model ...Multiple Execution
Rubiscape provides a feature that enables the multiple execution of a complete pipeline. This allows the entire pipeline to be executed multiple times, creating a new instance for each run. Each instance can be configured independently. For multiple ...