Skip to main content
Version: Streamsheets 2.5

Introduction

The three main components of the Streamsheet platform are: Streamsheets, Apps and Streams. Streamsheets consume, process and produce data streams. In Excel, one or several sheets always form part of a Workbook. In a similar way, Streamsheets run in Apps and one App accommodates up to four sheets. Hence strictly speaking, the App consumes, processes, and produces data streams, and the Streamsheets contained in the App define how this happens exactly. Streams define the endpoints where messages are coming from and where they are sent to, hence data sources and data sinks. These endpoints can be e.g. MQTT and Kafka topics as well as OPC UA nodes, REST APIs, database collections or local directories.

StMFlow

Apps consume and produce streams. In the App`s Streamsheets the data flow logic is defined via functions.

Best Practices#

With Streamsheets, many ways lead to your desired outcome. This is great, because there is not only one right solution to get an application running and a great way to get started with Streamsheets and its possibilities.. As your abilities grow and your applications start to get more complex, there are a couple of best practices you should look into, to ensure your application is easy to maintain, bug free and resource-efficient. This guide provides some general recommendation and best practices for your app development.

Maintain#

  1. Divide tasks in multiple applications

Not filling one application with all your ideas, but managing these in a number of smaller apps has multiple perks.
One would be a good overview. Big applications have many cross-references, multiple sheets and a lot of times only a few comments and documentation. If there is a problem, it will take you way longer the bigger and more complex your app is to find out what the cause is. Create multiple different apps to keep track of what is going on. this is even possible, if the apps have to talk to each other. Use the integrated Mosquitto Broker to send your calculations form app one to app two. This also serves as a great quality check. If the arriving messages don´t show any sign of problematic content, usually you already know, that the problem occurs somewhere further down your line of tasks.

  1. Document what you are doing

Work with comments and document the steps you are taking. This is usually very annoying while building your app and does not feel necessary, because right now everything seems obvious. But let me assure you, your future self will be very thankful!

  1. Visually divide

Same as the division of tasks into multiple apps, divide parts of your calculations into visually divided components. You can either use multiple sheets or leave empty space between tasks. This helps you navigate visually through your calculations and supports you maintain your logic as it grows later on.

Bug free#

  1. Remember the core principle of Streamsheet calculation

Left to right and Top to bottom. This is the calculation principle of Streamsheets. The biggest difference between a normal spreadsheet and a Streamsheet is that a Streamsheets has an order of calculation. It starts by calculating A1, B1, C1, ..., then it jumps to the next row and calculates again A2, B2, C2, ... . This is an immensely powerful feature, but if forgotten can create false logic dependencies. Example: If you are using a READ() in e.g. B26 function, to get updated values for your sheet, watch out to not accidentally add functions with references to B26 above this cell. All functions referencing to B26 above B26 (e.g. A1) automatically are always one step behind, since the value in B26 is not yet updated when the function is calculated.

  1. Check your steps

Every step calculates the whole sheet one more time. Make sure the triggered steps do not affect functions in ways it does not make sense or allow for false calculations/outcomes.

Wrong calculation mode: If you are using STACKADD() or TIMESCALE.INSERT and your calculation mode is "continuously", these functions are executed every time the sheet calculates. This does not necessarily mean, that you have new information to work with. You would be adding the same info over and over again. Make sure, the functions are only triggered, when actually needed. Change your calculation mode to "on message" or try using IF-Cells or the IF() function.

Wrong message content: Even when your calculation mode is set to "on message", there still can be situations, where the calculation of a function would not make sense. For example, if there are multiple payload types arriving in your Inbox triggering your steps. Every payload triggers the whole sheet, but maybe only for a certain payload your function should be calculated. Get around this issue by using different sheets per payload, or use the If-Cells/IF() function.

Resource-efficient#

Streamsheets does depend on the hardware of your server/laptop. There are a couple of things you can do to minimize the load your application creates.

  1. Divide tasks in multiple applications

Every application runs on its own process within the server. Dividing tasks into multiple apps allows the server to prioritize its resources and allows for a better over all experience. For example: Create one application to store your data in a database. Based on that data you can now create multiple dashboards for multiple purposes. All in different apps.

  1. Keep Streamsheets clean

Make your app as lightweight as possible. There are a couple of performance dependencies you should look out for while creating your app.

  • Update rate: The faster your app is running, the more resources are needed. Do you really need to set the cycle speed to 10ms?
  • Amount of cells: Streamsheet is not a regular spreadsheet application. It is calculating EVERY cell on every new occurring step. Fewer cells and fewer functions lead to an overall better performance. Instead of creating huge tables, think about using databases and this way even persist your data. Creating big tables is usually nothing you need to do within Streamsheets itself. Save your calculated results and query them, if you need to.
  • Use of the third dimension: In Streamsheets you sometime use a third dimension to access data. Functions like TIMEAGGREGATE() or TIMESCALE.SELECT() hold data in their cell. The bigger these data sets get and the more you are using these functions, the bigger the load on your client. Usually you don´t need to query millions of data points. Aggregate your data in smart ways is key to a smoothly running app.
  1. Add more resources

Streamsheets is always limited by the amount of CPU, memory and disc space available. Make sure there is always enough. Since your apps are running within a docker environment first check the amount of data docker is allocating.

  • CPU and Memory: docker stats
  • Disc space: docker system df

Docker allows you to manage the allocated resources of your hardware. When using Docker Desktop, these resources are automatically limited. Go into your settings and adjust these in the Tab "Resources".

If your docker installation already allocates the maximum amount of resources, think about scaling up your setup.