Automatic Feature Engineering from Very High Dimensional Event Logs Using Deep Neural Networks

As communication networks have grown, event logs have increased in both size and complexity at a very fast rate. Thus, mining event logs has become very challenging due to their high variety and volume. The traditional solution to model raw event logs is to transform the raw logs into features with fewer dimensions through manual feature engineering. However, feature engineering is very time-consuming, and its quality is highly dependent on data scientists’ domain knowledge. Furthermore, repeatedly preprocessing event logs significantly delays the scoring process, which must scan all items in the logs.