## Streaming Data Framework for MATLAB Production Server Basics

Use Streaming Data Framework for MATLAB^{®} Production Server™ to read from and write to event streaming platforms, such as Kafka^{®}. Using this framework, you can:

Develop a streaming analytic function in MATLAB that filters, transforms, records, or processes event stream data.

Connect to a streaming source and test how the analytic function reads from and writes to event streams by using Streaming Data Framework for MATLAB Production Server functions.

Simulate the production environment for testing your streaming analytic algorithms (requires MATLAB Compiler SDK™).

Package the analytic function (requires MATLAB Compiler SDK) and deploy it to MATLAB Production Server.

### Install Streaming Data Framework for MATLAB Production Server

Install the Streaming Data Framework for MATLAB Production Server support package from the MATLAB Add-On Explorer. For information about installing add-ons, see Get and Manage Add-Ons (MATLAB).

After your installation is complete, find examples in

,
where
* support_package_root*\toolbox\mps\streaming\Examples

`support_package_root`

is the root folder of support packages on your system. To get the path to this folder, use
this command:fullfile(matlabshared.supportpkg.getSupportPackageRoot,'toolbox','mps','streaming','Examples')

### System Requirements

Streaming Data Framework for MATLAB Production Server has the same system requirements as MATLAB. For more information, see System Requirements for MATLAB.

### Write Streaming Analytic MATLAB Function

The event stream analytic function typically consumes a stream of input events and can produce a stream of output events. It can filter, transform, record, or process the stream of events by using any MATLAB functionality that is deployable to MATLAB Production Server.

Event stream analytic functions process windows or batches of events. An event consists of three parts:

Key — Identifies the event source

Timestamp — Indicates the time at which the event occurred

Body — Contains event data, specified as an unordered set of (name, value) pairs

Analytic functions read events into a timetable. Each row of the timetable represents a streaming event, typically in chronological order. If the analytic function produces results, they must also be timetables.

When processing a stream, you can call an analytic function several times, because the
window size is typically much smaller than the number of messages in the stream. The
stateless execution model of MATLAB
Production Server isolates the processing of each window, so the processing of one window does
not affect the processing of the next. *Stateful* functions that
require interaction between the processing of consecutive windows specify a MATLAB structure that is preserved between windows and passed to the next invocation
of the analytic function.

An analytic function can have one of three signatures:

Function Signature | Description |
---|---|

results = (data)`analyticFcn` | Stateless analytic function that emits a stream of results |

[ results, state ] = (data,
state)`analyticFcn` | Stateful analytic function that preserves state between batches and emits a stream of results |

(data)`analyticFcn` | Stateless analytic function that does not emit a stream of results |

#### Stateless Analytic Function

The following `plotSierpinski`

function is an example of a stateless
analytic function. `plotSierpinski`

plots the X and Y columns of the
input timetable. The source code for this function and a script to run it is located in
the `\Examples\ExportOptions`

folder.

function howMany = plotSierpinski(xyData) hold on arrayfun(@(x,y)plot(x,y,'ro-', 'MarkerSize', 2), [xyData.X], [xyData.Y]); hold off drawnow count = height(xyData); howMany = timetable(xyData.Properties.RowTimes(end), count); end

#### Stateful Analytic Function

The following `recamanSum`

function is an example of a stateful
analytic function. In stateful functions, the data state is shared between events and past
events can influence the way current events are processed. `recamanSum`

computes the cumulative sum of a numeric sequence. In returns two values:

`cSum`

— A table that contains the cumulative sum of the elements in the stream`state`

— A structure that contains the final value of the sequence

The source code for the `recamanSum`

function, it initialization
function `initRecamanSum`

, and the scripts used to run the analytic
function are located in the `\Examples\Numeric`

folder.

function [cSum, state] = recamanSum(data, state) timestamp = data.Properties.RowTimes; key = data.key; sum = cumsum(data.R) + state.cumsum; state.cumsum = sum(end); cSum = timetable(timestamp, key, sum); end

### Process Kafka Events Using MATLAB

To process events from a stream, you create an object to connect to the stream, read events from the stream, iterate the streaming analytic function to process the several windows of events, and, if the analytic function produces results, create a different stream object to write the results to stream.

The following code sample gives an
overview of processing one window of events using the framework. Assume that you have a
Kafka host running at the network address `kafka.host.com:9092`

that has a topic `recamanSum_data`

. Also, assume that the
`recamanSum_data`

topic contains the first 1000 elements of the Recamán
sequence..

Create a

`KafkaStream`

object for reading from and writing to the`recamanSum_data`

topic.inKS = kafkaStream("kafka.host.com",9092,"recamanSum_data");

Read events from the

`recamanSum_data`

topic into a timetable`tt`

.tt = readtimetable(inKS);

Call the

`recamanSum`

function and calculate the cumulative sum of a part of Recamán's sequence in`tt`

. Since`recamanSum`

is a stateful function, first call the`initRecamSum`

function, which initializes state.state = initRecamanSum(); [results, state] = recamanSum(tt,state);

For a detailed example of how to process several windows of events, see Process Kafka Events Using MATLAB.

### Simulate Production Using Development Version of MATLAB Production Server

Before deploying to MATLAB Production Server, you can test the streaming analytic function using the development version of MATLAB Production Server, which acts as a local test server. For a detailed example, see Test Streaming Analytic Function Using Local Test Server.

### Deploy Streaming Analytic to MATLAB Production Server

You can also package the analytic function and deploy it to MATLAB Production Server. For a detailed example, see Deploy Streaming Analytic Function to MATLAB Production Server.

## See Also

`readtimetable`

| `writetimetable`

| `kafkaStream`

| `inMemoryStream`

| `testStream`