IOT data packets are generally dependent on the sensors, devices ,gateways, systems that generate them. Also this data can be structured or unstructured.
For the sake of this post lets talk about structured data and challenges to build a mechanism of storing this data into database.
As we all know Java Script Object Notation or JSON (as it is popularly referred to). is the preferred mechanism of data exchange. It is simple and easy to understand and can easily represent complex data structures with ease.
JSON can be nested and multilayered depending on the data. This is where the challenge begins.
Consider a scenario where a gateway is registered and activated on RubiThings. This gateway can be monitoring diverse IOT devices , sensors or even PLCs
The data generated by each device or group of devices or sensors will be diverse. Similarly a typical PLC has 250 tags and any combination of those can be reporting at different frequency.
From a storage perspective RubiThings also needs to be able to handle all these data packets and store the data into a relational database for easy retrieval , grouping , aggregation and reporting.
Thus any payload JSON received from the gateway has to be normalized and made ready to be stored into the data base in such a fashion as to facilitate easy retrieval when invoked from internal / external system for consumption.
What this ensures is fast and easy access
Upside
Using this technique will ensure fast and easy retrieval , ease of sorting and ordering.
Downside
The more complex the JSON the more columns in the table.
In case there are nested lists then lot of repeated rows created with null values
Recent Topics
Rooted in Data: Visualizing India’s Agricultural Story
Indian agriculture has always been a story of complexity and uncertainty, shaped by shifting weather patterns, rising input costs, and vast regional differences in productivity. For decades, the data holding the key to these challenges has been fragmented
Make Sense of BI—Now at Your Fingertips with Rubiscape Mobile
In today’s fast-paced digital world, waiting for insights is no longer an option. Business decisions must happen in real time—whether you’re in a boardroom, on the shop floor, or catching a flight. The traditional desktop-bound BI model no longer meets
Showroom to Service Bay: How Rubiscape Redefines Automotive Intelligence
In today’s data-driven world, the automotive industry is experiencing rapid transformation. With evolving customer preferences, rising competition, and the growing importance of after-sales services, the need for deeper, data-backed decision-making has
What truly powers great dashboards, sharp analytics, and confident decision-making?
It’s not just the tools we use — it’s the foundation beneath them. Let’s shine a light on the unsung heroes of Business Intelligence — the essential elements that transform raw data into trusted insights: Data Modeling: Designing with Purpose Every strong
A Deep Dive into Christmas Gift Sales Analysis using Rubisight
Unwrapping the Festive Cheer: A Deep Dive into Christmas Gift Sales The holiday season is a time of joy, cheer, and, of course, gift-giving. As retailers gear up for the next festive season, understanding the nuances of consumer behaviour and sales trends
Unwrapping the Festive Cheer: A Deep Dive into Christmas Gift Sales Analysis
Unwrapping the Festive Cheer: A Deep Dive into Christmas Gift Sales The holiday season is a time of joy, cheer, and, of course, gift-giving. As retailers gear up for the next festive season, understanding the nuances of consumer behaviour and sales trends
I have a doubt
What are the differences between supervised and unsupervised learning?
Building structured data storage strategy for Things data
IOT data packets are generally dependent on the sensors, devices ,gateways, systems that generate them. Also this data can be structured or unstructured. For the sake of this post lets talk about structured data and challenges to build a mechanism of
About the RubiStudio category
This category consists of all topic discussions related to data preparation & modeling, machine learning, forecasting, textual analytics, and Pro Code possible with RubiStudio.
Quick Use of Data Dictionary and Workflow
Data Dictionary: With the use of pre-existing datasets, a new data dictionary can be produced. It enables you to create new datasets while reusing the existing ones and incorporating new features. By leveraging the preexisting datasets, the Data Dictionary
Flow of Information using Sankey Chart in RubiSight
A Sankey chart, also known as a Sankey diagram or flow diagram, is a type of data visualization that represents the flow of resources, energy, or other quantities between multiple entities or categories. It is often used to visualize the distribution
Securing Your Data at Every Level: Introducing Rubisight’s Data Level Security
In the digital age, data governance and security have emerged as paramount concerns for organizations across industries. With the proliferation of data breaches and regulatory requirements, safeguarding sensitive information and ensuring compliance has
Infographics and Specialized Widgets: When to Use, Best Practices, and Limitations
RubiSight widgets, also known as charts, are the building blocks of visual data storytelling dashboards within the Rubiscape platform. They are essentially visual representations of your data that help you explore, understand, and communicate insights
How to include drill-down to details option?
Question - For example, I have 10 records. 7 of which are ok, 3 are not ok. On dashboard I want to show count and aggregated values for 7 and 3 respectively. And details of 3 or 7 records should be available on user demand. There are two ways to drill
Use of HTML for Bookmarking in Rubisight
What is bookmarking? How do we achieve it using Power BI? Bookmarking is a common feature in various data visualization tools, and it generally refers to the ability to save and revisit a specific state or view of a report/dashboard. This feature allows
Rubisight Filters Explained: Finding Insights Faster
Rubisight filters help you focus on specific data within your dashboard, making it easier to spot trends and gain valuable insights. Here are the three main types: 1. Global Filters: Apply the same filter criteria to all pages on a dashboard. Useful for
Banking Analytics: Reimagining the Way Banks Do Business
Created a Comprehensive dashboard on Banking Analytics. With banking products becoming increasingly commoditized, Analytics can help banks differentiate themselves and gain a competitive edge. This dashboard showcase trends that help management in decision
Rubisight Overview - Key Functionalities
Find the dashboard used in Rubisight Overview Session. This dashboard demonstrated most of the functionalities in Rubisight platform. The funcationalities used in the dashboard are as follows - Data Dictionary used for Rubisight dashboard creation. Calculated
LeBron James vs. Michael Jordan: The Greatest Debate in Basketball History
My new Viz on Rubisight Shapes - This entire dashboard is prepared based on shapes and filters. The debate over who is the greatest basketball player of all time, LeBron James or Michael Jordan, has captivated fans and analysts for years. Both athletes
Outlier Detection
Outlier Detection Description Outlier Detection reveals the extreme values that deviate from the rest of the data in a real-world dataset. Why to use Numerical Analysis – Data Preparation When to use When there are certain values in the data which significantly
Hardik vs Rohit - An Inhouse Rivalry!
The Big Debate - Hardik vs Rohit ---- Mumbai Indians are already out of the IPL and whose fault !!! Is it Hardik ??? The news sources are debating on the two former Indian Player, is it a correct debate ? image723×712 181 KB Find the stats in Rubisight
How to convert the data type for a specific data point in Rubiflow?
In Rubiflow once the data node is processed, you can use an expression from the Data preparaion function (Data integration) and connect to the data source node. In that you will be able to see a functions on the left side - “Convert”, through this functionality
Welcome to Community
The Community hub inside Zoho Desk lets you build a powerful community around your business, while still serving customers via other channels. Your customers can post 4 types of content within your community. These are Ideas, Questions, Problems and Discussions.