Simple Neural Networks
Generating Neural Networks can quickly become very complicated.
This workflow illustrates four extremely simple Neural Networks. The goal of each Neural Networks is the same – to predict the demand for a product based upon historical sales from the previous week.
The Neural Networks are all based upon the most simple 3-layer Neural Network:
- Input Layer
- Hidden Layer
- Output Layer
There are several ways to construct a Neural Network from within KNIME. This workflow illustrates two of the ways:
- Keras Layer Nodes
- DL Python Nodes
Two of the four models predict “Today’s Sales” (which are already known to the model). The other two models predict “Tomorrow’s Sales” which correlate to the sales from the same-day-last-week.
Viewing the node weights calculated by each of the models show how the Neural Networks identify the most useful inputs.
The data for the workflow comes from the Kaggle Store Item Demand Forecasting Challenge. The provided data included 5 years of store-item sales data for 50 different items from 10 different stores. The objective of the competition was to predict 3 months of item-level sales data at different store locations.
KNIME Installation Requirements:
- KNIME Extension: KNIME Deep Learning – Keras Integration (Labs)
- Python with Keras: https://www.knime.com/deeplearning
Historic Sales Data
The input node data comprises of the following fields:
- Sales(0) = today
The single output node is tuned to predict either:
- Today Sales
- Tomorrow Sales
Note that Today’s sales data is included in the last seven days of input data, whereas tomorrows sales data is not.
Models that are trying to predict today’s sales should tune their weights so as to heavily depend upon today’s input data. Models that are trying to predict tomorrow’s sales should, instead, put greater weight on sales from the same-day-last-week. That is, last Monday’s sales are a good predictor of next Monday’s sales.
Keras Layer Nodes
The KNIME Keras Layer Nodes allow users to set up deep learning architectures without writing a single line of code. KNIME provides over 60 layer nodes. This network utilizes only two of the layers:
- Keras Input Layer
- Keras Dense Layer
This network has 7 input nodes (for the 7 days of historic sales data) and 1 output node (coming out of the Dense Layer). The creation of the hidden layer is implied within the Keras Input Layer.
The two Keras Network Learner nodes are used to predict either today’s or tomorrow’s sales.
The DL Python Network Executor nodes are out of place in this branch as they contain lines of Python code. But they are useful here because they allow us to extract the node weights calculated by the learners.
Finally, the Keras Network Executor nodes are used to test the accuracy of the predictions.
The Learning Monitor can be viewed from the Keras Network Learner node. Unlike most KNIME views, this view updates while the Neural Network is learning.
The calculated loss for the Training data and the Validation data can also be seen in the Learning Monitor. This chart was generated when tuning the model Target = Tomorrow.
Python code extracts the weights from the model. These weights can be seen in the node’s Standard Output view and in the KNIME Console window:
DL Python Network
The KNIME DL Python Network Nodes require coding in Python but can be much more compact.
Today’s sales data is included in the last seven days of input data, whereas tomorrows sales data is not.
We expected that models trying to predict Today sales should tune their weights so as to heavily depend upon today’s input data. Models that are trying to predict Tomorrow sales should, instead, put greater weight on sales from the same-day-last-week. That is, last Monday’s sales are a good predictor of next Monday’s sales.
These results can, indeed, be seen in the calculated node weights.
Both the Keras Layer Nodes and the DL Python Nodes accurately tuned their models to heavily weigh the input from today’s Sales(0) input. The Mean Absolute Error from these models were both excellent.
When predicting Tomorrow sales, both models again did a good job of identifying the same-day-last-week Sales(-6) input as being the most useful.
But the models also identified the Sales(0) input as useful. This makes sense as there is seasonality and sales are increasing year-over-year. But it was surprising to see the model catch these trends given such a narrow view of historic sales.