Development of self-learning algorithm for autonomous system utilizing reinforcement learning and unsupervised weightless neural network / Yusman Yusof

One of the key issues of developing an autonomous system is that it requires pre-defined knowledge by an expert. This knowledge is then converted into computer program or by utilizing exhaustively trained and tested Artificial Intelligence (AI) algorithm. With these methods of development, prior to deployment the system must be prepared to handle all unanticipated circumstances that will occur during deployment. The testing and training of such system prior to deployment must be thorough. As an alternative, having a self-learning algorithm embedded in an autonomous system allows the system to instinctively acquire new knowledge and learn from experience. In this thesis the research of a self-learning algorithm will be presented, outlined and discussed in detailed manner. The development of the algorithm starts by reviewing the characteristic of an autonomous systems. From the reviews, it is evident that autonomous system is set to handle finite number of encountered states using finite sequences of actions. In order to learn the optimized states-action policy the self-learning algorithm is developed using hybrid AI algorithm by combining unsupervised weightless neural network, which employs AUTOWiSARD and reinforcement learning algorithm, which employs Q-learning. The AUTOWiSARD learns to classify states without supervision, while Q-learning will determine what best action to be taken for a state from reinforcement learning. By integrating both algorithms, a system will be able to acquire knowledge, learn, record and recall experience thus enables an autonomous system to self-learn. In the algorithm development a step-by-step example of the algorithm implementation is presented and then successfully implemented in Lego Mindstorm obstacle avoiding mobile robot as a proof of concept implementation of the hybrid AI algorithm. In order to further test the algorithm robustness, it is then implemented in mobile robot obstacle avoidance simulation in complex environment. In the simulation the robot is equipped with thirteen distance sensing sensors. From the simulation result, by using these sensors information the AUTOWiSARD algorithm can successfully differentiate and classify states without supervision, while the Q-learning algorithm is able to produce and optimized states-actions policy. These proves that without prior knowledge, the hybrid AI algorithm can self-learn. In the future the research on improving the algorithm learning will be studied and the implementation in other types of autonomous system other mobile robot obstacle avoidance will be considered.