I started with running a MySQL benchmark test to collect a baseline. At the same time reading the on-line manual to get familiar features and area for optimization.
Then I run mysqltuner to see what need to be optimzed. After followed the recommendations, mysqld no longer hogs the CPU continuously. In fact, my web benchmark improve almost 40% (now load time ~0.260 sec). Yeah!

However, there are still work left to do. sqltuner founds that my application (which one?) is not closing connection properly, and join queries might be able to improve further. In addition, MySQL manual has some good advice on how to make better database design and performs faster query (here and here).
Furthermore, one of the tunings done is enable the build-in query cache. I think there is further improvement if I deploy a memory object cache like memcached, it minimizes the overhead of querying MySQL. However, coding change is needed. I will be looking at implement it into NucleusCMS in the future.