RubiSpark

RubiSpark

RubiSpark is a feature within the code fusion on the Rubiscape platform for implementing Spark code. It operates as an independent node, meaning it cannot be connected to any predecessor, successor or reader nodes.

Using RubiSpark Code Editor

In the RubiSpark code editor, you can write your Spark code.
A sample code snippet is shown in the image below.

The table below explains the above code snippet.

Line of Code

Result

df= spark.createDataFrame([('2015-04-08',200)],['Date','Sales'])

It creates a DataFrame with column names Date and Sales with respective values.

df.show()

It will print the created DataFrame, which can be viewed in custom component log.


(info)Notes

  • RubiSpark supports large datasets.
  • RubiSpark supports following API's

Write Code Using RubiSpark

To write your custom code using RubiSpark, follow the steps given below.

  1. Create your algorithm flow. Refer to Building Algorithm Flow in a Workbook Canvas.

  2. Drag and drop RubiSpark on workbook canvas. 
  3. Select RubiSpark and click Configure


  4. The RubiSpark editor window is displayed. Enter your pyspark code and click Save


  5. Run the RubiSpark node.

    After successful execution, a confirmation message is displayed.
    You can view the output of the custom component under View Log > Custom Component Log.

    • Related Articles

    • RubiSpark

      RubiSpark is a feature within the code fusion on the Rubiscape platform for implementing Spark code. It operates as an independent node, meaning it cannot be connected to any predecessor, successor or reader nodes. Using RubiSpark Code Editor In the ...