Reader

Reader


  1. The Reader in Rubiscape Pipelines is a component used to ingest data from various sources into a pipeline.
  2. It acts as the entry point of the pipeline, enabling users to read structured, semi-structured from files, databases, APIs, cloud storage, or streaming sources.
  3. The Reader ensures smooth data flow into downstream pipeline steps for processing, transformation, analysis, or model execution.

1.  Overview

    The Reader node is responsible for:

    • Connecting to a data source
    • Fetching data in supported formats
    • Converting incoming data to a pipeline-ready dataset
    • Passing the data to the next pipeline component

    Readers are essential because every pipeline starts with a data ingestion step.


2.  Types of Readers in Rubiscape Pipelines

    Rubiscape supports multiple types of readers:

          • File Reader 

          • Database Reader 
          • API Reader 
          • Cloud Storage Reader 
          • File Server Reader
          • Streaming / Queue Reader 

3. Steps to Configure a Reader

  1. Open Rubiscape → Pipelines
  2. Create or edit a pipeline
  3. Drag the Reader node from the Task panel onto the canvas (TaskPane >> Connect >> Reader )
  4. The properties pane on right hand side show below details for reader configuration:

  1. The labels in above screenshot are labelled as below:
1 : Task Name - To update the task name/Reader Name
2 : Data Fields - Option to configure the columns of a Reader
3 : Metadata Refresh - Option to refresh metadata (Column Name Changes/Column Configuration changes)
4 : Filter Condition - Option to configure filtering condition based on logical expression

Note that,
The configured columns as from the Reader will get passed to the successor nodes with same column names and column data type/variable type.