Long Short-Term Memory (LSTM) Networks are powerful models to solve sequential problems in machine learning. Apart from their use on sequence classification, LSTMs are also used for sequence prediction. Predictive features of LSTMs have been used extensively to generate handwriting, music and several other types of sequences. Configuration and training of LSTM networks are relatively more arduous than non-sequential models, especially when input data is complex. In this research, the aim is to train LSTM networks and its different variations, use their generative features on a relatively obscure and complex type of sequences in machine learning; digital fonts. Controlled experiments have been performed to find the effects of different model parameters, input encodings or network architectures on learning font based sequences. All in all, in this document; the procedure of creating a dataset from digital fonts are provided, training strategies are demonstrated and the generative results are discussed.