This execize aims at developing a named entity recognizer using an LSTM Network. All the information you need are provided below. The code to pre-process and format the data is already structured below. Your task is to build the neural network and optimize it to get the best performance you can.
You can fetch the dataset from here
#Importing the dataset
Map the input dataset tokens and tags to ids
#Apply the index mapping to our data
#Transform columns into sequential arrays
#Padd the data, and split it to training set and testing set
This is our input and output dimentions
#Building the model
The model is a many-to-many LSTM RNN model. You'll create 4 layers: embedding, bi-lstm and lstm layers, and TimeDistributed Dense layer.
Used to create representations of the onhot-encoded vectors that the LSTM would work with.
Checkout the keras embedding layer here
Bidirectional LSTM Layer
Serves to read the input both forwards and backwards. The idea is to split the state neurons of a regular RNN in a part that is responsible for the positive time direction (forward states) and a part for the negative time direction (backwards states).
Dealing with Many to Many RNN Architecture, where we expect output from every input sequence. Not using this layer will result in one final output.