Dynamic partitioning and data allocation method on heterogeneous architecture / Muhammad Helmi Rosli

In recent years, processing large data set to produce result in a timely manner poses a lot of challenges to ICT researchers. Currently most organization has an elaborate local network system whose computers are underutilized. These network form cluster of computing resources that simulates supercomputer. Processing images are computationally complex due to its data and task intensive nature. This can be solved by parallelizing the process in cluster environment. Most cluster environment have a variety of computer hardware specification namely heterogeneous environment.Optimizing the resources in heterogeneous environment during parallel processing is not a simple task. These involves partitioning and allocating task to each cluster node.The aim of these research is to investigate various method of partitioning and allocating task in cluster environment and produce a dynamic partitioning and allocating method. Initial stage of the research consist of exploring the heuristic performance of cluster and multi-threading involving five experiments; homogeneous architecture with node partitioning; heterogeneous architecture with node partitioning; heterogeneous architecture with node partitioning including multi-threading; heterogeneous architecture with node and core partitioning; heterogeneous architecture with node and core partitioning including multi-threading.The performances use sequential processing speed as a benchmark. Each experiment highlight the advantages and disadvantages of the experimental architecture.The disadvantages from each experiment prompts the design of dynamic parallel partitioning and allocating framework. The case study use for this experiment is Sobel edge detection algorithm. The test data set focuses on processing images of three different sizes; (IK x IK), (2K x 2K) and (3K x 3K). The performance evaluation is based on the processing speed in second, speedup, and efficiency. In conclusion, it is found that in idle situation heterogeneous architecture with node and core partitioning including multi-threading perform better from other experiment. However, in real working condition where some computer are serving users processes, the dynamic algorithm provides a potential alternative.