Rapidly growing data processing tasks require powerful and energy-efficient heterogeneous computing systems, and GPUs take on a significant mission for those systems in accelerating heavy workloads by executing multiple parallel tasks concurrently. Increasing architectural complexity and widening employment of GPUs bring error resiliency concerns for safety-critical applications. Furthermore, approaches that enhance performance and reduce energy dissipation handle error resiliency on GPUs through approximate computing solutions. Evaluating error resiliency in terms of either identifying error proneness of a system or investigating approximations without much disturbing the output necessities robust knowledge about the execution of a program on a device. In this thesis, we develop a runtime performance and power monitoring tool visualizing the execution with detailed micro-architectural metrics. By utilizing the tool, we acquire several fundamental understandings about runtime performance bottlenecks and how perturbations affect output quality. Afterward, we propose a framework predicting fault vulnerability for error-resilient GPU applications. The framework can accurately estimate error tolerance and saves from analyzing the fault occurrence probability requiring significant effort. Depending on the performance bottlenecks observed with the tool and the error propagation gained during prediction experiments, we introduce a hardware-based approximation computing approach targeting to improve the performance and power of GPU programs, especially memory-bound ones. The approximation method, which resolves memory utilization bottlenecks at runtime, enhances performance by 1.49× (up to 2.1×) and diminishes energy consumption by 28.4% (up to %52.6) while maintaining the accuracy on the output above 98%.