Find the best distribution using Tensorflow

First, my problem:

• I have several systems which take one input and produce one output (not linearly)
• I have to find how to distribute the input between the systems in order to obtain the highest output

Used libraries

import numpy as np
import tensorflow as tf
from tensorflow.keras import backend as K

Dummy input data

We create dummy data to test our code.

system_one_input, system_one_output = np.zeros(100), np.zeros(100)
system_two_input, system_two_output = np.zeros(100), np.zeros(100)

for i in range(0, 100):
    system_one_input[i] = i
    system_two_input[i] = i
    system_one_output[i] = i*(150 + i)
    system_two_output[i] = i*(10 + i * 3)

Dummy data

Creating models for each system

We create models according our dummy data. In a real application, models should be trained with some data. But, to avoid problems here we just create fix models (models with no learning ).

input = tf.keras.Input(shape=1)
minput = tf.keras.layers.Multiply()([tf.constant(np.asarray([1.0])), input])
output = tf.keras.layers.Add()([tf.constant(np.asarray([150.0])), minput])
output = tf.keras.layers.Multiply()([output, input])
model = tf.keras.Model(inputs=input, outputs=output)
model.summary()

tf.keras.utils.plot_model(model, to_file="test.png", show_shapes=True)

Model 1

input = tf.keras.Input(shape=1)
minput = tf.keras.layers.Multiply()([tf.constant(np.asarray([3.0])), input])
output = tf.keras.layers.Add()([tf.constant(np.asarray([10.0])), minput])
output = tf.keras.layers.Multiply()([output, input])
model2 = tf.keras.Model(inputs=input, outputs=output)
model2.summary()

tf.keras.utils.plot_model(model2, to_file="test.png", show_shapes=True)

Model 2

Create a model to find the best distribution

The model will be fit to predict the best output value. In order, to find the used distribution, we will check a Dense layer using a SoftMax activation function.

input = tf.keras.Input(shape=1, name="input_layer")
x = tf.keras.layers.Dense(2, activation="softmax", name="weight_layer", use_bias=False)(input)
split_tensor = tf.split(x, num_or_size_splits=2, axis=-1)
i1 = tf.keras.layers.Multiply()([split_tensor[0], input])
i2 = tf.keras.layers.Multiply()([split_tensor[1], input])
m1 = model(i1)
m2 = model2(i2)
output = tf.keras.layers.Add()([m1, m2])
full_model = tf.keras.Model(inputs=input, outputs=output)
full_model.summary()

tf.keras.utils.plot_model(full_model, to_file="test.png", show_shapes=True)

Final model

The loss function

The bigger output is, the better it is. Then we fit the model on the targeted input value. The model optimizer will find the best distribution for us.

def max_loss(y_true, y_pred):
    return 1/K.sum(y_pred)

system_test_input = np.asarray([0.5])
full_model.fit(system_test_input, system_two_input, , batch_size=1, epochs=5, verbose=0)
print(full_model.get_layer('weight_layer').call(tf.constant(np.asarray([[i/10]]))))
print(tf.keras.activations.softmax(tf.constant(full_model.get_layer('weight_layer').get_weights())))
prediction = full_model.predict(system_test_input)
prediction1 = model.predict(system_test_input)
prediction2 = model2.predict(system_test_input)
print(prediction, prediction1, prediction2)

Results

We use System 1 until value exceed 70, after we use System 2.

The combined system chose the best system in function of the input.