Our current largest issue is the fact that our models tend to aim for tetris' more than surviving, leaving single columns on either side of the board for an 'I' piece that never comes. This issue can be seen in the below image where it ends the game rather than placing previous pieces and the line piece to continue.
Our AI does not have the ability to do skill shots such as a 'spin move' (rotating a piece into place or sliding a piece under another piece) which limits its ability to find the most optimal game state.
As the model improves, completing each episode takes longer and longer. This increases our training time. To improve this, we need to look into optimisation of the game environment itself.