innsight
- Get the Insights of your Neural Networkinnsight
is an R package that interprets the behavior
and explains individual predictions of modern neural networks. Many
methods for explaining individual predictions already exist, but hardly
any of them are implemented or available in R. Most of these so-called
‘Feature Attribution’ methods are only implemented in Python
and thus difficult to access or use for the R community. In this sense,
the package innsight
provides a common interface for
various methods for the interpretability of neural networks and can
therefore be considered as an R analogue to iNNvestigate for
Python.
This package implements several model-specific interpretability (Feature Attribution) methods based on neural networks in R, e.g.,
Example results for these methods on ImageNet with pretrained network Vgg16:
The package innsight
aims to be as flexible as possible
and independent of a specific deep learning package in which the passed
network has been learned. Basically, a neural network of the libraries
torch
, keras
and neuralnet
can be passed, which is internally converted into a torch
model with
special insights needed for interpretation. Currently, the following
model and layer types are accepted:
torch::nn_sequential
with layers
nn_linear
, nn_conv1d
, nn_conv2d
,
nn_max_pool1d
, nn_max_pool2d
,
nn_avg_pool1d
, nn_avg_pool2d
,
nn_dropout
, nn_flatten
(see torch
issue #716 and use classname = "nn_flatten"
)keras::keras_model_sequential
or
keras::keras_model
with layers layer_dense
,
layer_conv_1d
, layer_conv_2d
,
layer_max_pooling_1d
, layer_max_pooling_2d
,
layer_average_pooling_1d
,
layer_average_pooling_2d
, layer_dropout
,
layer_flatten
But it is also possible to pass an arbitrary net in form of a named list (see vignette for details).
The package can be installed directly from CRAN and the development
version from GitHub with the following commands (successful installation
of devtools
is required)
# Stable version
install.packages("innsight")
# Development version
::install_github("bips-hb/innsight") devtools
Internally, any passed model is converted to a torch
model, thus the correct functionality of this package relies on a
complete and correct installation of torch
. For this
reason, the following command must be run manually to install the
missing libraries LibTorch and LibLantern:
::install_torch() torch
Note: Currently this can lead to problems under
Windows if the Visual Studio runtime is not pre-installed. See the issue
on GitHub here
or for more information and other problems with installing
torch
see the official installation vignette
of torch
.
You have a trained neural network model
on your model
input data data
. Now you want to interpret individual
datapoints or the overall behavior by using the methods from the package
innsight
, then stick to the following pseudo code:
# --------------- Train your model -----------------
# 'model' has to be an instance of either torch::nn_sequential,
# keras::keras_model_sequential, keras::keras_model or neuralnet::neuralnet
= ...
model
# -------------- Convert your model ----------------
# For keras and neuralnet
<- Converter$new(model)
converter # For a torch model the argument 'input_dim' is required
<- Converter$new(model, input_dim = model_input_dim)
converter
# ----------------- Apply method -------------------
# Apply global method
<- Method$new(converter) # no data argument is needed
result # Plot the result
plot(result)
# Apply local methods
<- Method$new(converter, data)
result # Plot individual results
plot(result)
# Plot a boxplot of all given data points in argument 'data'
boxplot(result)
library(innsight)
library(torch)
data(iris)
# Prepare Data
<- torch_tensor(as.matrix(iris[, -5]))
x <- torch_tensor(as.integer(iris[, 5]))
y
# Define Model
<- nn_sequential(
model nn_linear(4, 15),
nn_relu(),
nn_dropout(0.3),
nn_linear(15, 10),
nn_relu(),
nn_dropout(0.3),
nn_linear(10, 5),
nn_relu(),
nn_linear(5, 3),
nn_softmax(2)
)
# Train model
<- optim_adam(model$parameters, lr = 0.0002)
optimizer for (t in 1:2500) {
<- torch_log(model(x))
y_pred <- nnf_nll_loss(y_pred, y)
loss if (t %% 250 == 0) {
cat("Loss: ", as.numeric(loss), "\n")
}$zero_grad()
optimizer$backward()
loss$step()
optimizer
}#> Loss: 1.034311
#> Loss: 0.8588966
#> Loss: 0.6706114
#> Loss: 0.5762339
#> Loss: 0.5493336
#> Loss: 0.5144928
#> Loss: 0.4707141
#> Loss: 0.3532465
#> Loss: 0.2839047
#> Loss: 0.2687521
# create a Converter for this model
<- Converter$new(model, input_dim = c(4),
converter input_names = list(names(iris[,-5])),
output_names = list(levels(iris[,5])))
# Apply local method LRP with epsilon rule
<- LRP$new(converter, iris[,-5], rule_name = "epsilon")
lrp_eps
# Plot the individual result for two datapoints and all classes
<- plot(lrp_eps, data_idx = c(1,102), output_idx = 1:3) +
p1 ::labs(title = "LRP ('epsilon')")
ggplot2
# Plot the boxplot statistic for all datapoints and all classes
# without a preprocess function
<- boxplot(lrp_eps, output_idx = 1:3, ref_data_idx = 1, preprocess_FUN = identity)
p2
::grid.arrange(p1,p2, ncol = 1, layout_matrix = matrix(c(1,1,1,2,2), ncol = 1)) gridExtra
library(keras)
# Load image
<- image_load("man/images/imagenet_rooster.png", target_size = c(224,224))
image <- image_to_array(image)
image <- grid::rasterGrob(image / 255)
p_image
# Preprocess image
<- array_reshape(image, c(1, dim(image)))
x <- imagenet_preprocess_input(x)
x
# Get pretrained Vgg16
<- application_vgg16()
model
# Convert the model
<- Converter$new(model)
converter
# Apply LRP with rule 'alpha_beta' with alpha = 1
<- LRP$new(converter, x,
lrp_ab output_idx = c(8), # 8 is index for class 'cock'
rule_name = "alpha_beta",
rule_param = 1,
channels_first = FALSE)
# We have to flip the y axis before plotting
$result <- torch::torch_flip(lrp_ab$result, c(2))
lrp_ab<- plot(lrp_ab)
p_lrp_ab
::grid.arrange(p_image, p_lrp_ab, layout_matrix = matrix(c(1,2,2), nrow = 1)) gridExtra
If you would like to contribute, please open an issue or submit a pull request.
This package becomes even more alive and valuable if people are using it for their analyses. Therefore, don’t hesitate to write me (niklas.koenen@gmail.com) or create a feature request if you are missing something for your analyses or have great ideas for extending this package. Currently, we are working on the following:
keras
with multiple
in- and outputs, e.g. for mixed data of tabular and image dataThis work is funded by the German Research Foundation (DFG) in the context of the Emmy Noether Grant 437611051.