# Simple Neural Networks

Generating Neural Networks can quickly become very complicated.

This workflow illustrates four extremely simple Neural Networks. The goal of each Neural Networks is the same – to predict the demand for a product based upon historical sales from the previous week.

The Neural Networks are all based upon the most simple 3-layer Neural Network:

• Input Layer
• Hidden Layer
• Output Layer

There are several ways to construct a Neural Network from within KNIME. This workflow illustrates two of the ways:

1. Keras Layer Nodes
2. DL Python Nodes

Two of the four models predict “Today’s Sales” (which are already known to the model). The other two models predict “Tomorrow’s Sales” which correlate to the sales from the same-day-last-week.

Viewing the node weights calculated by each of the models show how the Neural Networks identify the most useful inputs.

The data for the workflow comes from the Kaggle Store Item Demand Forecasting Challenge. The provided data included 5 years of store-item sales data for 50 different items from 10 different stores. The objective of the competition was to predict 3 months of item-level sales data at different store locations.

#### KNIME Installation Requirements:

This KNIME Node Use Case provides an example of a useful KNIME workflow. These workflows do not depend upon Market Simulation but can supplement a Market Simulation workflow. If you have not yet installed KNIME, go to Getting Started.

# Historic Sales Data

The input node data comprises of the following fields:

• Sales(-6)
• Sales(-5)
• Sales(-4)
• Sales(-3)
• Sales(-2)
• Sales(-1)
• Sales(0) = today

The single output node is tuned to predict either:

• Today Sales
• Tomorrow Sales

Note that Today’s sales data is included in the last seven days of input data, whereas tomorrows sales data is not.

Models that are trying to predict today’s sales should tune their weights so as to heavily depend upon today’s input data. Models that are trying to predict tomorrow’s sales should, instead, put greater weight on sales from the same-day-last-week. That is, last Monday’s sales are a good predictor of next Monday’s sales.

Last seven (7) days of historic sales + target data for today and tomorrow.

## Input Data

Sales for 50 products aggregated across 10 stores.

## Create Collection

Collects the last seven (7) days of historic sales data.

# Keras Layer Nodes

The KNIME Keras Layer Nodes allow users to set up deep learning architectures without writing a single line of code. KNIME provides over 60 layer nodes. This network utilizes only two of the layers:

• Keras Input Layer
• Keras Dense Layer

This network has 7 input nodes (for the 7 days of historic sales data) and 1 output node (coming out of the Dense Layer). The creation of the hidden layer is implied within the Keras Input Layer.

The two Keras Network Learner nodes are used to predict either today’s or tomorrow’s sales.

The DL Python Network Executor nodes are out of place in this branch as they contain lines of Python code. But they are useful here because they allow us to extract the node weights calculated by the learners.

Finally, the Keras Network Executor nodes are used to test the accuracy of the predictions.

## Input Layer

Defines the 7 input nodes and the hidden layer of the neural network.

## Dense Layer

Defines the 1 output node with a linear activation function.

## Network Learner

Inputs the collection of historic sales data from the last 7 days.

## Target Function

Target’s either “Today” sales or “Tomorrow” sales.

## Tuning

Set the number of Epochs (10), Batch Size (200), and Optimizer (Adam).

## Learning Monitor

The Learning Monitor can be viewed from the Keras Network Learner node. Unlike most KNIME views, this view updates while the Neural Network is learning.

## Loss

The calculated loss for the Training data and the Validation data can also be seen in the Learning Monitor. This chart was generated when tuning the model Target = Tomorrow.

## Get Weights

Python code extracts the weights from the model. These weights can be seen in the node’s Standard Output view and in the KNIME Console window:
[Preferences][KNIME GUI][INFO].

## Execute

Run the model against the test data.

## Results

The Mean Absolute Error is tiny when target = Today (as expected).

# DL Python Network

The KNIME DL Python Network Nodes require coding in Python but can be much more compact.

## Network Creator

Python Keras code defines the Neural Network model.

## Network Learner

More code defines the target and tuning parameters.

## Network Executor

Runs the test data against the model and outputs the model weights.

# Analysis

Today’s sales data is included in the last seven days of input data, whereas tomorrows sales data is not.

We expected that models trying to predict Today sales should tune their weights so as to heavily depend upon today’s input data. Models that are trying to predict Tomorrow sales should, instead, put greater weight on sales from the same-day-last-week. That is, last Monday’s sales are a good predictor of next Monday’s sales.

These results can, indeed, be seen in the calculated node weights.

## Today

Both the Keras Layer Nodes and the DL Python Nodes accurately tuned their models to heavily weigh the input from today’s Sales(0) input. The Mean Absolute Error from these models were both excellent.

## Tomorrow

When predicting Tomorrow sales, both models again did a good job of identifying the same-day-last-week Sales(-6) input as being the most useful.

But the models also identified the Sales(0) input as useful. This makes sense as there is seasonality and sales are increasing year-over-year. But it was surprising to see the model catch these trends given such a narrow view of historic sales.